Peer Revenue Fully funded

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Human-Computer Interaction

cs.HC Watch Human-Computer Interaction

Wednesday 12 August

Human-Computer Interaction

Transforming Interactions in Thesis Supervision: An Exposé-First Workflow in Higher Education

Lin-Yin Huang, Dennis Zyska and Iryna Gurevych

At the studied research institute, one professorship oversees approximately 20 theses per semester, while day-to-day supervision is distributed among doctoral and postdoctoral researchers. To manage this supervision demand, the institute uses an exposé-first workflow in which students prepare a research proposal before entering the main thesis-writing phase. This paper asks how students, supervisors, and administrators experience the exposé-first workflow as a structured process for early thesis preparation, and how it redistributes responsibility, supervision, and administrative coordination work across roles and two digital platforms. Based on a mixed-methods study analyzed through Frauenberger et al.'s four reflective design lenses, the findings show that the exposé-first model made thesis preparation more structured by turning early research planning into a staged process of proposal writing, feedback, and approval. Students reported that this process helped them clarify research goals and take ownership of their research plans at an early stage. However, the workflow redistributed rather than reduced work: supervisors shifted toward iterative feedback, feasibility checking, and preliminary quality assurance, while administrators carried much of the coordination across platforms, deadlines, submissions, and feedback. The paper contributes an analysis of exposé-first thesis preparation as a sociotechnical workflow, showing how workflow redesign can improve structure while leaving essential administrative coordination work underrecognized.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.DB

SchemaLink: An Intelligent Web Editor for LinkML Schema Curation

Emanuele Cavalleri, Paolo Perlasca, J. Harry Caufield et al. (+3 more)

Motivation: LinkML is a suitable language for the representation of the structural and content constraints of different kinds of biomedical data. Even if it is a quite recent proposal, it has been applied in several biomedical contexts. Developing and maintaining LinkML schemas presents several challenges, particularly for novice curators. Non-expert bio-curators may struggle with LinkML syntax and best practices, requiring significant time and effort to develop well-structured schemas. Results: In this paper we propose SchemaLink, a web-based environment for the graphical construction and enhancement of LinkML schemas that address the following requirements: $(i)$ introduce a graphical language for the specification of LinkML schemas, $(ii)$ make uniform the specification of schemas in similar contexts, $(iii)$ simplify the design and curation processes by exploiting a RAG-based approach to assist curators in creating new schemas from scratch and editing already developed ones. Several experimental analyses show the quality of the produced LinkML schemas through the AI-based editing facilities. Availability and Implementation: SchemaLink is available online at: https://SchemaLink.biodata.di.unimi.it. SchemaLink code and testing data are available as open-source on GitHub at: https://github.com/AnacletoLAB/{schemalink-webapp,schemalink-api}.
Read the full abstract

All authors Emanuele Cavalleri, Paolo Perlasca, J. Harry Caufield, Justin Reese, Christopher J. Mungall, Marco Mesiti

Human-Computer Interaction · Cross-listed from econ.GN

How Organizations Use AI: Evidence from ChatGPT

Aaron Chatterji, David Holtz, Neel Rakholia et al. (+2 more)

We study how organizations use frontier generative AI by linking ChatGPT Enterprise account records to usage, worker roles, task classifications, and public-company financial data through March 2026. These linked data enable a privacy-preserving analysis of adoption, worker roles, and message-level tasks at scale: for instance, the worker-level sample we analyze at the six-month adoption horizon includes over 1,500 organizations and over 17 million messages. We document four facts about enterprise AI adoption and use. First, ChatGPT Enterprise usage has grown rapidly due to a combination of new firm adoption and growing intensity among existing adopters. Second, U.S.-based public company adoption is concentrated among larger, more valuable, and more R&D- and SG&A-intensive firms. Third, active use within adopting firms spans job functions and seniority levels, with especially high usage intensity among early-career workers. Fourth, ChatGPT Enterprise usage encompasses a broad range of knowledge work tasks, including writing, technical work, communication, and information synthesis. In aggregate, these results suggest that firms differ widely in the speed, breadth and purpose of their enterprise AI adoption, and that they are still actively learning how to integrate AI into organizational workflows.
Read the full abstract

All authors Aaron Chatterji, David Holtz, Neel Rakholia, Prasanna Tambe, Gawesha Weeratunga

Human-Computer Interaction

"Pharos Night: Crown Pursuit": An AI-Native Deck-Building and Tactical Arena Game Design Based on Multi-Agent Systems

Ting-Chen Hsu, Jueyao Liu, Yanzi Zhou et al. (+5 more)

With advancements in generative AI technology, an increasing number of researchers have begun exploring AI-native games in which gameplay rules are directly driven by generative AI. This paper presents "Pharos Night: Crown Pursuit," an AI-native deck-building and tactical arena game based on a multi-agent system. The game uses large language models to generate materials and cards, support NPC decision-making, and mediate natural-language interactions. During play, players collect materials, describe desired card effects in natural language, and choose whether to negotiate or fight with NPCs in the arena. To constrain model-generated outcomes, the system parses responses as structured JSON, constructs card effects from predefined mechanics, and maps qualitative effect levels to designer-specified numerical values. A small-scale playtest with 13 participants suggests that the system can provide strategically meaningful and engaging AI-driven gameplay, while also revealing challenges related to predictability, transparency, and player control. This work demonstrates the potential of multi-agent generative AI systems for creating more emergent digital game experiences.
Read the full abstract

All authors Ting-Chen Hsu, Jueyao Liu, Yanzi Zhou, Jiangxu Lin, Haoyu Xu, Yuwen Liu, Yanjia Liu, Bangjing Xu

Human-Computer Interaction

IF:CARGO: LLM-Based Semantic Compilation for Al-Native Rule Programming Games

Ting-Chen Hsu, Lianye Zhang, Jiangxu Lin et al. (+3 more)

This case study presents IF: CARGO, an experimental puzzle game that uses a large language model as a semantic compiler rather than an autonomous game-playing agent. Players author IF/THEN rules in natural language, which the model translates into a constrained command schema for deterministic validation and execution by the game engine. This architecture creates a playable loop of expression, execution, observation, and revision, framing AI interaction as semantic debugging. A mixed-methods playtest with 24 participants across eight levels examined player attempts, thinking time, perceived controllability, adjustability, and interpretations of the AI's role. Results suggest that players generally understood the model as a translation intermediary and could revise their strategies through feedback, while periodic commands, multi-robot coordination, and rule-priority mechanics created greater cognitive and diagnostic demands. The study proposes a practical pattern for AI-native gameplay: constrain natural-language input, preserve player authorship, and ensure deterministic execution.
Read the full abstract

All authors Ting-Chen Hsu, Lianye Zhang, Jiangxu Lin, Zhaoyi Yu, Fei Qin, Zihao Chen

Human-Computer Interaction · Cross-listed from cs.CL

A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench

Praveen Reddy, Charuta Mandke, Suvrankar Datta et al. (+4 more)

General-purpose large language models (LLMs) have recently been reported to match or exceed specialized clinical AI tools on medical benchmarks, but such comparisons draw on a narrow set of systems and on benchmarks developed largely in high-income settings. We evaluate VITA, a retrieval-augmented generation (RAG) system purpose-built for contextual knowledge retrieval in India and other low- and middle-income (LMIC) settings. VITA retrieves from a curated corpus of disease-specific guidelines, India-specific antimicrobial resistance data, national formulary constraints, and resource-limited care protocols; its architecture and corpus are proprietary, but the benchmark, the physician-written rubrics, and our full response and scoring outputs are public for independent verification. On 4,023 English-language HealthBench questions (80.5% of the benchmark), scored with a GPT-4.1 judge, VITA ranked first with 51.9% of possible rubric points, ahead of GPT-5.4 (46.1%), o4-mini (44.3%), Gemini 3.1 Pro (42.6%), and Claude Sonnet 4.6 (37.3%), and scored highest on 45.4% of questions. To test robustness to newer models and judge lineage, a 500-question subset was re-run against current-generation models (GPT-5.5, Claude Opus 4.8, Gemini 3.5 Pro, Grok 4.3) and graded by a neutral open-weight judge (DeepSeek-V4-Pro) sharing no lineage with any system tested. Here the gap narrowed to parity: VITA and GPT-5.5 were statistically indistinguishable on mean per-question score, while VITA led on points-weighted score and won the most questions. VITA's advantages in accuracy and completeness persisted under the neutral judge; its communication scores were lower. These results indicate that a purpose-built clinical RAG system remains competitive with frontier LLMs on an open benchmark, consistent with corpus specificity as a design variable that improves grounding at some cost to communication polish.
Read the full abstract

All authors Praveen Reddy, Charuta Mandke, Suvrankar Datta, Sarah Khan, Siddharth Reddy Anthireddy, Shitij Arora, Vishal Singh

Human-Computer Interaction · Cross-listed from cs.AI

Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence

Mengru Wang, Junfeng Fang, Shuofei Qiao et al. (+17 more)

AI models are increasingly used in scientific discovery and human decision-making. Yet how AI models work and what risks they pose remain poorly understood. As AI development becomes faster and more automated, research on the mechanisms underlying AI remains largely manual. To bridge this gap, we introduce Mechanist, an agentic system that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI. To ground novel mechanism hypotheses, we construct a scientific knowledge graph of 13,000 studies on AI mechanisms, alongside a multidisciplinary database of 43 million papers spanning 26 fields. For reliable experiment execution, we curate a library of 32 foundational methods for mechanism analysis. Compared with Claude Code and existing AI-scientist systems, Mechanist generates higher-quality mechanism hypotheses and executes experiments more reliably. Across four case studies, Mechanist autonomously discovers new model behaviors and their underlying mechanisms, and translates these discoveries into mechanism-guided interventions and interdisciplinary design. Specifically, Mechanist first uncovers a counterintuitive safety risk in scientific laboratories, showing that unsafe traits can transfer to fine-tuned student models through apparently safe training data and emerge across modalities. Mechanist then develops a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, infer the beliefs of others, and how these mechanisms emerge during pretraining. Building on this theory, Mechanist develops targeted interventions that improve model performance across diverse scenarios. Finally, Mechanist can also advance interdisciplinary discovery through mechanistic design, providing an alternative to the computationally intensive generate-and-rerank paradigm.
Read the full abstract

All authors Mengru Wang, Junfeng Fang, Shuofei Qiao, Zhenqian Xu, Haoming Xu, Haoxiong Wang, Shumin Deng, Linyi Yang, Xin Xu, Yunzhi Yao, Dan Zhang, Fei Shen, Zhixiang Cui, Buqiang Xu, Haozhe Luo, Yunxiang Wei, Ningyu Zhang, Julian McAuley, Tat Seng Chua, Huajun Chen

Human-Computer Interaction

Synchronized AMG and EMG Dataset of Lower-limb Muscle Activities in Everyday Training

Dongxu Tang, Shih Ying-Lei, Zhuoyi Ren et al. (+2 more)

Understanding how lower-limb muscle groups coordinate is important for studying movement impairment, rehabilitation, and physical performance. Reproducible analysis of this coordination requires multimodal recordings that relate local muscle-related signals with body-level kinematics. Complementing neural-level electrical activation captured by EMG, AMG provides a valuable mechanical approach to monitoring muscle activity. Here, we introduce a synchronized, multimodal dataset for healthy-adult lower-limb activities. For data collection on the left leg, 16 triaxial accelerometers were evenly divided into four muscle-site clusters for AMG recording, complemented by four surface EMG channels. A 15-marker optical motion-capture (MoCap) system captured lower-body kinematics, with the resulting marker trajectories used to compute bilateral knee and ankle joint angles. Our dataset contains 1,918 trials from 30 subjects across 16 task conditions. We benchmark the dataset by estimating four joint angles from 300 ms windows of the 5-100 Hz band-pass-filtered AMG data and assess matched EMG features in a separate modality ablation. In the primary cross subject benchmark, the four reference models achieved mean absolute errors of 8.840$^\circ$-9.591$^\circ$. The benchmark and ablation results characterize performance across subjects, tasks, and joint angles and examine the effects of sensor configuration, modality, the number of training subjects, and frequency representation. The release includes documented timing definitions, processed data, and reproducible benchmark resources. https://dongxutang918-afk.github.io/SAME-Limb/
Read the full abstract

All authors Dongxu Tang, Shih Ying-Lei, Zhuoyi Ren, Jianting Liao, Yitian Shao

Human-Computer Interaction · Cross-listed from cs.CY

Philosophical vertigo with artificial intelligence

Thomas A. Pollak, Hamilton Morrin and Murray Shanahan

Large language models are already adept at engaging users in long, emotionally salient conversations across ordinary and existential domains. They are also capable of inducing a potent sense of connection with a human-like entity, even when the user knows their interlocutor is artificial. For some users, these conversations can unsettle assumptions about mind, reality, agency and authority, producing forms of ontological shock and epistemic destabilisation in which inherited criteria become newly available for doubt or revision. Independent of direct use, exposure to public discourse about AI and the disorienting pace of their evolution might extend this destabilisation by changing the cultural background against which artificial minds are encountered and interpreted. We describe this condition as philosophical vertigo: a loosening of the ordinary criteria by which people stabilise meaning and orient themselves to reality. Drawing on philosophy, psychiatry, cognitive science, AI safety and religious studies, we outline pathways through which philosophical vertigo may arise, become affectively saturated, and eventually propagate through human-AI interaction and online communities. Against this background, clinical reports of AI-associated delusions can be seen as sentinel events making visible themes and mechanisms that may also operate at a population level in less severe or non-clinical forms. We argue that AI systems themselves will increasingly participate in the reconstruction of our shared epistemic environment because they readily supply narrative material and personalised interpretive scaffolding at precisely the moment when users' conceptual assumptions may already be loosened. We conclude by considering possible trajectories for the ecology of belief and shared reality, and proposing philosophical corrigibility as a civic response for navigating this emerging social condition.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CY

Benchmark-Based Comparative Assessment of Publicly Benchmarked Indian Foundation Models: A Capability and Evaluation-Maturity Framework

Avinash Agarwal and Vridhi Jain

Purpose: Governments increasingly fund indigenous foundation models to strengthen national AI capability, digital sovereignty, and multilingual computing. This paper assesses India's foundation-model ecosystem and examines whether apparent capability gaps in public benchmark evidence may also reflect gaps in evaluation maturity. Approach: The paper presents a structured, benchmark-based comparative assessment of Indian foundation models against global frontier and comparable-scale models across eight capability domains: general-purpose reasoning, coding and software engineering, agentic AI and computer use, cybersecurity, vision and image understanding, video and multimodal understanding, scientific research, and Indic language capability. Using only publicly reported results, it proposes an exploratory four-dimension Benchmark Maturity Index (BMI), scoring each domain on standardization, participation, independent verification, and national Findings: Indian models achieve strong scores on established benchmarks such as MMLU and MATH-500. However, these are now widely regarded as saturated, and frontier developers no longer report them. Indian models participate far less frequently in newer, agentic, and domain-specialized evaluations, and participation is highly uneven across organizations. Sarvam AI reports the broadest coverage by a substantial margin. The BMI refines, and in some cases revises, the maturity judgments a purely descriptive review would produce. Practical implications: Many apparent capability gaps cannot be distinguished, on available evidence, from evaluation-ecosystem gaps, with direct implications for how national AI programs should design monitoring and funding criteria. Originality: The paper proposes BMI as a reusable instrument for scoring evaluation-ecosystem maturity at the domain level and demonstrates its application to the Indian foundation-model ecosystem.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.RO

D3D-GEN: Robot-Aware Domain-Grounded Interactive 3D World Generation for Social Robotics

Anh Duc Do, Volodymyr Shcherbyna, Tai Duc Nguyen et al. (+5 more)

Training and validation of Embodied AI for social navigation critically depends on realistic simulation environments, yet many current approaches fail to find a balance between realism and simulability. We propose D3D-GEN, a novel world generation system that combines a domain agent with a retrieval-augmented generation (RAG) pipeline grounded in that domain. Our system enables users to rapidly generate domain-grounded, fully interactive 3D worlds by automating both the collection of domain knowledge and the synthesis of realistic floorplans and object placements, without dependence on any fixed 3D model database. Given a domain description prompt, the research agent collects publicly accessible domain-specific data and constructs a persistent domain database. Using this database, our RAG pipeline generates plausible floorplans and object placements by dynamically querying a user-provided semantic database, which can be easily extended or modified. The output is a fully interactive 3D world loadable by the popular simulators Isaac Sim and Gazebo. With our approach, we have built databases for several common domains (indoor residential, hospital, office) and generated dozens of distinct, plausible simulation environments for each domain. We present D3D-GEN with a local web frontend that facilitates rapid, interactive world generation for robot simulation.
Read the full abstract

All authors Anh Duc Do, Volodymyr Shcherbyna, Tai Duc Nguyen, Spaarsh Thakkar, Zhengcheng Shen, Teham Buiyan, Archan Misra, Linh Kästner

Human-Computer Interaction

ATOM: Geometry-Aware Microgesture towards Object-Agnostic Tangible Interaction

Yinqiao Wang, Hao Xu, Qixuan Liu et al. (+3 more)

This paper presents ATOM, an integrated framework towards agnostic and tangible object interactions with microgestures. Our goal is to support microgesture interactions across different everyday objects, with the capability to automatically leverage the geometric affordance of each object. We formulate a fingertip-aware detection pipeline to leverage generative 2D and 3D models for geometry enhancement and refinement. We then introduce a usability-based method to prioritize the detected elements based on their ergonomic suitability for interactions. Building on this foundation, we further develop an AR system to transform everyday handheld objects into tangible user interfaces with 0D, 1D, and 2D microgesture interactions. Across transitions among everyday cooking objects of varying shapes and sizes, ATOM outperformed ablation baselines in task completion, usability (SUS), and workload (NASA-TLX). A further study with 10 objects demonstrates ATOM's generalizability across objects and grasps, highlighting its potential towards fluid, object-agnostic tangible interaction in real-world AR scenarios.
Read the full abstract

All authors Yinqiao Wang, Hao Xu, Qixuan Liu, Shengdong Zhao, Pheng-Ann Heng, Chi-Wing Fu

Human-Computer Interaction · Cross-listed from cs.CY

Toward Meaningful Transparency for AI Chatbots: Disclosing Persuasive Intent Reduces Persuasion

Adrian Rauchfleisch and Andreas Jungherr

The growing role of AI-generated content and AI-enabled systems in public communication has led regulators to demand clear disclosure of content provenance and AI involvement. But the effects of such disclosures remain uncertain. We test two disclosure approaches in their impact on an AI chatbot's persuasive appeal. In a preregistered experiment, 1,500 UK adults held a short conversation with a persuasive chatbot about one of 60 policy issues. The chatbot was identical for everyone. We randomized the disclosure that people received: nothing (control), a prominent disclosure that they were interacting with an AI (T1), or that disclosure plus the chatbot's persuasive intent and instructions (T2). The chatbot shifted attitudes by 12.6 points on a 100-point scale in the control group. The AI-identity disclosure was practically equivalent to no disclosure, with a 13.1-point shift, whereas the additional intent disclosure cut the persuasive effect roughly in half to 6.3 points. It also made participants view the campaign's methods as less acceptable and support stronger penalties against it. For direct chatbot interactions, transparency about AI identity alone does not meaningfully impact its influence. While current rules emphasize what a system is, our results show why the regulation of persuasive AI must also address what the system is trying to do.
Read the full abstract

Human-Computer Interaction

A Browser-Based Gesture-Driven Avatar Interaction Framework for Metaverse Onboarding Environments

Deepti Parachuri, Chhayank Sahu and Sameer Singh Choudhary

Avatar interaction shapes how engaging and immersive a metaverse experience feels, and for that interaction to feel natural, avatars need to respond to users without forcing them through a controller-based interface first. This paper describes a gesture-driven interaction layer built for a browser-based metaverse onboarding environment, where users explore a set of virtual rooms as an avatar and interact with embedded video, document, and quiz content using hand, arm, and head gestures instead of a keyboard or controller. The system combines real-time gesture recognition (Google MediaPipe) with two alternative locomotion techniques - hand-raise navigation and in-place walking - so users can trade off precision against physical immersion depending on the task. The contribution is the integration, deployment, and evaluation of these techniques as a single lightweight, web-deployable, controller-free interaction model, assessed through a structured internal onboarding session with five participants. We report what worked, what didn't, and the design trade-offs that came out of combining these techniques in one deployed system.
Read the full abstract

Human-Computer Interaction

VisPuzzle: Task-Aware Composite Visualization Construction

Zheng Wang, Zhiyang Shen, Lingyun Yu et al. (+1 more)

Compositing multiple visualizations into a coherent whole remains challenging due to the vast design space and the need to balance the coverage of task-relevant data insights (e.g., trends and outliers), perceptual clarity, and aesthetic quality. In this paper, we present VisPuzzle, a task-aware method that formulates visualization composition as a stepwise search problem over a composition graph. In this graph, nodes represent either data composition operations (e.g., union, join) or visual composition operations that determine component relationships, spatial arrangements, or component proportions, and edges encode feasible transitions between operations. We employ Monte Carlo Graph Search to efficiently identify high-quality composition candidates from this graph, guided by a reward function that balances task relevance, perceptual effectiveness, and aesthetic coherence. A use case and a user study show that the top-ranked candidates produced by VisPuzzle align closely with human judgments of composition quality, demonstrating its utility in supporting principled and scalable visualization composition.
Read the full abstract

All authors Zheng Wang, Zhiyang Shen, Lingyun Yu, Shixia Liu

Human-Computer Interaction

RAGE-Vis:A Relation-Aware Generative Editing Interface for Natural Language-Based Chart Editing

Ziyao Kang, Yiping Sun, Linxuan Tian et al. (+3 more)

Natural language offers an easy way for users to express chart editing intents, which are often composite and cross-component (e.g., adjusting style, extending categories, highlighting values). However, existing methods typically map instructions to a single operation or widget, limiting their ability to handle high-level requests and often producing locally plausible but globally inconsistent results due to a lack of awareness of relationships between chart components. To address these challenges, we introduce RAGE-Vis, a Relation-Aware Generative Editing interface for natural language-based chart editing. The system supports bitmap chart images as input and converts them into an editable parameterized intermediate representation. Instead of mapping instructions to a single edit or widget, RAGE-Vis parses composite intents, identifies targets and scopes, and generates hierarchical editing panels for underspecified requests, enabling users to adjust both global settings and local parameters. Furthermore, RAGE-Vis identifies potentially affected fields based on visual encoding relations, structural relationships, and expressive consistency relations, and organizes them into actionable widgets to support cross-component coordinated controls. Through two case studies, we demonstrate the applicability of RAGE-Vis in complex editing tasks, including style adjustment, data extension, order rearrangement, legend layout, and color mapping. A user study further shows that participants can effectively handle underspecified requests, explore candidate alternatives, and maintain cross-component consistency with RAGE-Vis.
Read the full abstract

All authors Ziyao Kang, Yiping Sun, Linxuan Tian, Henghuan Qu, Wei Zeng, Jiazhi Xia

Human-Computer Interaction

Measuring Browser Webcam Gaze Honestly: A Capture-Clock Methodology and Open Reference Implementation

Chi-Sheng Chen and Gabriel A. Brat

Browser-based webcam gaze trackers are increasingly used for crowd-scale data collection and in clinical settings where lab eye trackers are impractical, but the reported latency numbers may not represent real world functionality. The common practice of timestamping each gaze sample when it is emitted, rather than when its source frame was captured, makes the measured inference latency read about $0\,$ms no matter how slow the engine really is. We show how to measure it honestly, recovering a per-frame capture clock from the browser's re\-quest\-Video\-Frame\-Call\-back (rVFC) API (captureTime where the browser exposes it for local camera streams, else presentationTime, in which case every recovered latency is a verifiable lower bound): exact source-frame pairing through a per-frame queue for engines that expose their inference pipeline, and a further lower bound for engines that do not, such as WebGazer. We release an open TypeScript implementation and benchmark harness, demonstrated on two interchangeable engines: WebGazer and a new FaceMesh+KRR pipeline.
Read the full abstract

Human-Computer Interaction

Player Perceptions of Generative AI in Games: A Steam Review Analysis

Mahsa Bazzaz and Seth Cooper

The rapid adoption of generative AI in game development has created large discussions among players, yet little empirical work has examined how players actually perceive AI-generated content. Employing quantitative methods, we study the adoption of generative AI in games in the Steam marketplace, using procedural content generation (PCG) as a baseline of a generative technology that was successfully integrated into games over several decades. Furthermore, using qualitative methods, we study player reception of generative AI by analyzing 508,192 English-language reviews. We found that games disclosing generative AI use receive lower recommendation rates and more negative overall sentiment than PCG games. Thematic analysis of 600 reviews shows that players perceive the use of generative AI in games as low developer investment in the game. Drawing on human-centered AI frameworks, we argue that successful generative AI adoption requires deploying generative AI for what players need, not for what makes development cheaper.
Read the full abstract

Tuesday 11 August

Human-Computer Interaction

TRACE: Transparent Retrieval for Abstract Concept Evaluation

Joseph Bingham

Recent work reports that vision–language models (VLMs) struggle to establish and maintain stable reference in repeated reference games. Rather than ask which VLM does best, we ask a more basic question: do you need a large pretrained VLM for this at all? On grounding a single director utterance to one of twelve tangram silhouettes, we compare six off-the-shelf VLMs against a transparent baseline that uses no learned visual representation: classical SIFT keypoint matching and a signal-quality index over retrieved images. On identical trials, the transparent baseline matches the strongest VLM (SigLIP-large) and significantly outperforms the other five, including every CLIP and OpenCLIP variant. The baseline additionally retrieves external images, so this is not a matched-information comparison; what it shows is that a learned visual representation is not the bottleneck for this task: given retrieved images, a shape-appropriate classical similarity suffices. Along the way we find that abstract-grounding ability varies widely across VLMs (15–39% top-1; chance 8.33%, humans $\approx$77–80%), so the weakness is model-specific rather than intrinsic to contrastive pretraining; on the 1{,}013-shape KiloGram benchmark the pattern generalizes for CLIP, with per-shape difficulty tracking human shape-nameability. The pipeline is a classical, inspectable alternative rather than a learned one. We close by sketching how an explicit, inspectable representation of listener-side pact state could carry this approach into interactive multi-turn reference, which we leave to future work. Code available in supplementary material.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.LG

On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health

Ibukunoluwa Soyebo, Alyssa Donawa, Rodrigo Aguilar Barrios et al. (+2 more)

Stress is a pervasive determinant of mental health and a key target for mobile health interventions. On-device language models (ODLMs) offer privacy-preserving inference without cloud dependency, yet their feasibility for health prediction under mobile resource constraints remains underexplored. We evaluate ODLMs for multi-modal stress prediction using zero-shot prompting, measuring predictive accuracy alongside latency and throughput. Our results show that objective sensor features marginally outperform subjective self-reports on average, and that lightweight sub-2B models achieve low latency with predictable resource usage. Our findings highlight both the promise and the practical constraints of ODLMs for mobile mental health.
Read the full abstract

All authors Ibukunoluwa Soyebo, Alyssa Donawa, Rodrigo Aguilar Barrios, Brice Patchou, Corey E. Baker

Human-Computer Interaction · Cross-listed from cs.RO

Position: Robot Privacy as Embodied Boundary Work. Connecting Capabilities, Contexts, and Design Responses in Everyday Robotics

Liwen He, Shuning Zhang, Chengwen Zhang et al. (+4 more)

Robots are increasingly entering everyday environments where privacy is shaped not only by data practices, but also by spatial, bodily, social, and relational boundaries. Their embodied capabilities allow them to reshape these boundaries through situated action, challenging privacy framings centered on data flows, interface settings, or one-time consent. Prior work has examined robot privacy through sensing, data collection, telepresence, transparency, consent, bystander awareness, and multi-stakeholder governance. Building on this work, we propose embodied boundary privacy as a capability-by-context framing for examining how physically present robots may reshape privacy boundaries in situated interaction. Specifically, this framing organizes privacy risks across seven robot capabilities and five deployment contexts, asking how embodied capabilities enable boundary crossings and how situated contexts shape who is affected, how these crossings are interpreted, and when they become contested. We use this perspective to outline design and research implications for embodied privacy mechanisms, including boundary checkpoints, viewpoint-aware sensing control, remote-presence disclosure, object- and body-level access rules, constraints on socially persuasive privacy influence, and local interruption rights. We encourage HRI research, design, and governance to treat robot movement, orientation, proximity, object access, remote presence, and social expression as privacy-relevant actions whose meaning depends on context.
Read the full abstract

All authors Liwen He, Shuning Zhang, Chengwen Zhang, Xin Yi, Chun Yu, Jihong Jeung, Xin Tong

Human-Computer Interaction · Cross-listed from cs.AI

Cross-Disciplinary Taxonomy and Modeling of Misunderstanding Generation, Amplification, and Detection, from Pragmatics to AI Agents

Babak Abbaschian

Detection of misunderstanding is an urgent problem to solve because communication has moved away from real-time, in-person interaction and is increasingly handled by AI-mediated channels. This shift cuts communicators off from the resources repair depends on faster than new means of detection are being built. In this paper we analyse misunderstanding as a layered process in which a divergence is generated, may then be amplified, and is either detected and repaired or left to persist unnoticed. Consolidating accounts from nine fields of research that do not ordinarily cite one another, we identify eleven exact failure modes and show that each operates at a specific point in a communicative process rather than anywhere within it. Those points give eight analytical layers, derived from the literature rather than adopted from an existing model. Eight of the mechanisms primarily generate a divergence, two primarily amplify one already present, and one governs whether a divergence is detected and repaired. We model the eight layers formally, extending information and communication theory from the transmission of signals to the reconstruction of meaning, and we supply a source-by-source evidence matrix that makes every rating auditable, a coding manual, and nine analysed dialogue cases. No prior classification of misunderstanding both locates mechanisms at points in the process and types them by function.
Read the full abstract

Human-Computer Interaction

How Children Collaborate within Programmable AR Environments with Co-Located Collaborative Features

Romina Mahinpei, Diya Ajay Hundiwala, Sandy Zhang et al. (+2 more)

Programmable augmented reality (AR) environments are emerging as a promising way to support children's creative learning through embodied interaction with digital characters and physical space. At the same time, AR systems are increasingly capable of supporting co-located collaborative experiences. However, little is known about how children collaborate within programmable AR environments offering co-located collaborative features. In response, we extended Capybara, an existing programmable AR application for children, with co-located collaborative features supporting shared visibility and interaction across devices. We then conducted workshops with 9 children to examine whether and how collaboration emerges during use. Across our workshops, collaboration was often lightweight and implicit, emerging through three complementary forms: parallel play with social awareness, iterative remixing, and spontaneous peer support. Together, our findings provide insights for designing future child-centered programmable AR systems that better support co-located collaborative experiences.
Read the full abstract

All authors Romina Mahinpei, Diya Ajay Hundiwala, Sandy Zhang, Lana Glisic, Andrés Monroy-Hernández

Human-Computer Interaction · Cross-listed from cs.RO

Locomotion Variability and User Experience in Smart Wheelchair Human-Robot Interaction

Sean Kille, Adina M. Panchea, Balint Varga et al. (+1 more)

Human movement is inherently variable, with variability structured according to task relevance: movements are typically more consistent at task-critical points and more flexible elsewhere. In human-robot interaction (HRI), however, model-based assistance strategies commonly assume deterministic human behavior and suppress such variability, potentially altering how interactions are experienced and lowering sense of agency. While movement variability is increasingly recognized as functionally meaningful, its deliberate preservation in assisted interaction, and its consequences for user experience, remain underexplored. In this paper, we empirically investigate how different assistance strategies shape human movement variability, task performance, and subjective interaction experience in a shared control setting. We introduce an autonomy-supportive shared control strategy that preserves users' natural movement structure. This approach is evaluated in a user study in which participants push an intelligent powered wheelchair under three conditions: no assistance, conventional variability-reducing assistance, and variability-preserving assistance. While task-relevant performance remained comparable across assisted modes, preserving natural movement variability led to more favorable interaction experiences. In particular, participants reported significantly higher perceived agency compared to conventional assistance and highest perceived usefulness. These findings suggest that variability-aware assistance can support both performance and user autonomy in physical human-robot collaboration. More broadly, the results highlight the importance of designing assistive robotic systems that respect the embodied structure of human movement rather than treating variability as noise to be neglected or eliminated.
Read the full abstract

All authors Sean Kille, Adina M. Panchea, Balint Varga, Sören Hohmann

Human-Computer Interaction

The Role of Variability in Human-Machine Interaction Experience

Sean Kille, Jan Lars Hagemann, Anne Voormann et al. (+3 more)

Human-machine interaction (HMI) requires control strategies that account for the nature of human motor behavior. Conventional shared-control and haptic-assistance methods typically ignore the stochastic nature of human behavior, potentially limiting both performance and human interaction experience. In this study, we designed an experimental setting and evaluated a novel human-variability-aware optimal controller. Participants performed a physically coupled haptic interaction task in three conditions: a controller mode that aims at conventionally reducing overall variability, a variability-aware controller mode designed to maintain human natural variability patterns, and a human-only control condition serving as a baseline. We analyzed behavioral variability, task performance, and human interaction experience. The results show that considering natural movement variability significantly increased perceived interaction quality in terms of usability while maintaining task performance. These findings highlight the importance of incorporating stochastic human movement characteristics into shared-control designs and demonstrate the feasibility and benefits of the proposed control strategy for human-centered control design of HMI.
Read the full abstract

All authors Sean Kille, Jan Lars Hagemann, Anne Voormann, Balint Varga, Andrea Kiesel, Sören Hohmann

Human-Computer Interaction

"I Don't Want My Mental Health App To Give Me Mental Health Barriers": Unpacking The Need For Digital Mental Health Tracking Services With And For The Blind Community

Omar Khan and JooYoung Seo

Digital mental health (DMH) tracking services promise continuous, personalized support for well-being, but their design often assumes sighted users. For the blind community, this assumption produces a distinct pattern of exclusion: services whose accessibility cannot be evaluated without first paying for them, community features that exclude the users they purport to support, and interfaces that leave users digitally literate but functionally blocked. We report on an explanatory sequential mixed-methods study of blind users' experiences with DMH tracking services in the United States. In the first phase, 93 legally blind adults completed a survey about their usage patterns, adoption decisions, and data-agency preferences; in the second, 10 survey respondents participated in semi-structured interviews. We analyzed closed-ended responses using descriptive statistics and the Kruskal-Wallis test, and open-ended and interview data using inductive thematic analysis, interpreting findings through Norman and Skinner's eHealth Literacy framework. Participants identified mindfulness, sleep, and goal-tracking services as their most-used categories, but also described recurring exclusion from the community-support features that other users value most. We argue that the framework's "computer literacy" dimension is insufficient on its own: many of our participants possessed the literacy but were blocked from applying it by design choices that predate the user. We contribute design recommendations for transparent pre-purchase accessibility evaluation, accessibility-native rather than retrofitted interfaces, and user-controlled data agency – recommendations intended not to accommodate blind users but to design DMH tracking services with them from the start.
Read the full abstract

Human-Computer Interaction

QUARTZ: Qualitative Understanding via Accessible Representation and Visualization

Omar Khan and JooYoung Seo

Qualitative data visualizations – concept maps, network graphs, Sankey diagrams, and coding stripes – are integral to research practice, yet remain entirely inaccessible to blind and low-vision (BLV) researchers. While visualization has seen advanced multimodal solutions for quantitative charts, qualitative visualizations, and their non-linear, semantically rich structures have received no attention. We present QUARTZ, a web-based system that provides screen-reader-accessible, multimodal representations of qualitative data visualizations. Using the Rapid Iterative Testing and Evaluation (RITE) method, we conducted a user study with 8 BLV participants who completed 12 tasks across four visualization types. Our findings expose accessibility barriers unique to qualitative visualizations – non-linear navigation breakdowns and semantic comprehension gaps absent from quantitative chart research—and document how iterative co-design with BLV users resolved them. We contribute empirical evidence and design guidelines for an underexplored visualization domain, advancing the infrastructure BLV researchers need to participate independently in qualitative inquiry.
Read the full abstract

Human-Computer Interaction

Socioduality: A Relational Process Framework for Human-AI Interaction

Mehmed Zahid Çögenli

Human-AI research often evaluates individual capabilities, joint performance, or final outputs, but these approaches can lose the interaction process that produced the result. This article introduces socioduality: a sequential, reciprocal, and history-carrying process in which one party's response becomes part of the observable conditions shaping the other party's next contribution, judgement, decision, or action. For human-AI dyads, the framework identifies moves, candidate episodes, confirmed episodes, and maximal pathways. A minimum episode A1 -> B1 -> A2 requires evidence that B1 responds to A1 and that B1 then enters the formation of A2; candidates are classified as confirmed, non-sociodual, or indeterminate. A frozen coding protocol was calibrated on three natural human-AI records using two separate model-based evaluator series. A supplementary exploratory analysis then compared frozen Sociodual pathways with blind developmental/task-process segmentations. Across six examined interactions, the two representations were empirically non-equivalent: task-stage changes could occur within a continuing Sociodual pathway, while formal pathway breaks could occur within a continuing task context. This distinction persisted under fine-grained re-segmentation and record-format checks and was reproduced in all three prospectively selected unseen records using a fresh model-based Sociodual coding line. Socioduality therefore offers a bounded process-level framework for studying how human and AI contributions become relationally linked across time, preserving information that task-stage and endpoint-centred analyses do not uniquely recover.
Read the full abstract

2608.11322 · 11 Aug · 50 pages, 4 figures, 12 tables. Revised version adds a Supplementary Exploratory Analysis comparing Sociodual pathways with blind developmental/task-process segmentation, with granularity and record-format checks and an unseen-case extension. Main construct specification unchanged; minor editorial and terminology refinements

Abstract PDF

Human-Computer Interaction · Cross-listed from cs.AI

Long-Horizon AI Research for Grothendieck Constant: A Case Study in Human-AI Mathematical Collaboration

Alan Li, Rahul Saha, Anton Xue et al. (+4 more)

AI agents are increasingly used in mathematics research, but it is often unclear how to use them effectively. Towards this, we present an extensive case study of how AI was used to improve bounds on the Grothendieck constant $K_G$, which captures the hardness between combinatorial problems and their continuous relaxations. Specifically, while the precise value of $K_G$ is not known, we recently tightened the best known bounds to \[ \frac{6\pi}{11} \;\le\; K_G \;\le\; \frac{\pi}{2\log(1+\sqrt2)} - 10^{-4}. \] Crucially, these improvements were achieved using an AI research system that could arrive at insights deemed novel by domain experts. We give a detailed discussion of our experience using AI for mathematics research, particularly touching upon its strengths and weaknesses, as well as our experience with creating ideal conditions for AI to arrive at breakthrough insights.
Read the full abstract

All authors Alan Li, Rahul Saha, Anton Xue, Swarat Chaudhuri, Adam Klivans, Pravesh K Kothari, Raghu Meka

Human-Computer Interaction · Cross-listed from cs.CV

R4DSG: Relative 4D Scene Graph Memory for Object-Centric Question Answering in Long Egocentric Video

Ke Ma, Yamin Mao, Weiming Li et al. (+5 more)

Long-horizon egocentric video is a rich substrate for wearable AI assistants, but object-centric questions such as where an item was moved, when it last changed state, or why it was relocated remain difficult because caption- and transcript-based memories rarely preserve persistent object identity or structured spatial change. Existing long-video QA methods mainly emphasize temporal grounding and clip retrieval, while prior 3D scene-graph methods typically assume stronger geometry than free-motion wearable RGB video provides, including point clouds, RGB-D input, posed views, sparse reconstruction, or reconstructed scenes. R4DSG introduces a relative 4D scene graph memory for long egocentric video. Instead of storing raw graph sequences, R4DSG converts video into compact queryable memory entries indexed by time, place, persistent objects, anchor-relative change, and local interaction context. The main idea is to separate stable anchors from dynamic objects, maintain persistent object identity across frames, and represent object state through anchor-relative transitions rather than a globally aligned world model. Built on recent RGB-only advances in promptable video segmentation, temporal propagation, and relative 3D lifting, the method produces a retrieval-ready memory directly usable for long-horizon question answering. Evaluation on a 255-question object-related subset from EgoLifeQA shows, under question-only retrieval, a 6.7-point overall gain over EgoRAG-Text and a 12.5-point gain on when questions, which highlights the value of temporally organized object memory. These results position relative 4D scene graphs as a practical memory substrate for wearable assistants, AR systems, and embodied multimedia agents. GitHub Page: https://dualtransparency.github.io/R4DSG/.
Read the full abstract

All authors Ke Ma, Yamin Mao, Weiming Li, Shuai Tan, Yijie Zhong, Hao Chen, Haofen Wang, Meng Wang