Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Human-Computer Interaction

cs.HC Watch Human-Computer Interaction

Wednesday 14 October · still arriving

Human-Computer Interaction · Cross-listed from cs.CV

Player-1: Learning to Play from Synthetic Experience

Mingyu Ouyang, Yiran Wang, Siyuan Hu et al. (+9 more)

Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control. Current game-agent learning pipelines often rely on large-scale demonstrations with limited coverage of dynamic game states, while scalable environments for interactive training remain difficult to obtain. We introduce Player-1, a post-training framework for learning game control beyond demonstrations through synthetic experience. We extend the computer-use action space with relative pointer motion, sustained inputs, and composite actions. For supervised initialization, we fine-tune computer-use models on 6,044 curated gameplay trajectories that pair recorded actions with task instructions and visual reasoning. We construct Player-1-Gym, sixteen synthetic 2D and 3D environments designed around control demands shared across games. Programmatically verifiable rewards and randomized observations and control dynamics support multi-environment reinforcement learning with PPO. The resulting Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points, respectively. Player-1-9B reaches 38.33% success compared with 12.63% for the strongest evaluated baseline, with further gains over SFT on ViZDoom-10 and GameWorld. Further analyses show that broader synthetic environment coverage improves transfer, while the use of composite actions that combine multiple keyboard and mouse inputs increases during RL. We demonstrate that learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite.
Read the full abstract

All authors Mingyu Ouyang, Yiran Wang, Siyuan Hu, Xingyilang Yin, Huiqiang Sun, Guangzhi Wang, Junfu Pu, Cheng Cheng, Honghao Chen, Ying Shan, Mike Zheng Shou, Hwee Tou Ng

Human-Computer Interaction · Cross-listed from cs.CV

Speculative Sampling in Interactive Video Generation

Yubo Huang, Yuxuan Zhang, Jiaming Liu et al. (+2 more)

Interactive video generation commonly relies on small bidirectional chunks to reduce response latency. Yet our preliminary experiments show that larger chunks improve video quality and better utilize GPU parallelism, allowing more latents to be sampled with only a modest increase in sampling time. Motivated by these observations, we propose Speculative Sampling, a strategy that combines the modeling power of large bidirectional windows with the low-latency interaction of small ones. It expands the modeling window by appending noise latents as padding and caches the resulting predictions for reuse when the condition remains unchanged. Predictions are delivered incrementally, while the uncommitted future remains replaceable. Our latency analysis shows that this removes the waiting-time lower bound imposed by chunk boundaries, shifting the optimization focus from reducing chunk size to accelerating per-chunk sampling. Bidirectional window size thus becomes a decision variable for jointly optimizing visual quality and sampling time, rather than a necessary constraint on low-latency interaction. This opens a path to serving large bidirectional models in interactive streaming systems without complex streaming post-training.
Read the full abstract

All authors Yubo Huang, Yuxuan Zhang, Jiaming Liu, Sirui Zhao, Enhong Chen

Human-Computer Interaction

From Semantic Editing Intentions to Persistent Parametric Controls for AI-Assisted Image Editing

Rifat Mehreen Amin, Michelle Dutoit, Anton Rockenstein et al. (+2 more)

AI-powered image editing increasingly allows users to express high-level editing intentions, yet how these intentions are translated into concrete editing actions often remains difficult to understand. We present Atelier, a semantic image editing tool that uses image semantics and user intent to determine which editing controls to surface and how to initialize them. Guided by a formative study with eight image-editing practitioners, Atelier composes context-specific, directly manipulable controls that users can refine rather than committing AI interpretations directly to finished edits. In a within-subjects study with 12 practitioners, participants preferred AI-composed controls, described them as supporting creative exploration, and actively refined AI proposals. However, experiences of control were mixed, and interviews highlighted limited insight into why particular controls and values were proposed. Our findings suggest that making AI interpretations actionable through manipulable controls can support exploration and refinement, but actionability alone does not make those interpretations transparent to all users.
Read the full abstract

All authors Rifat Mehreen Amin, Michelle Dutoit, Anton Rockenstein, Baptiste Caramiaux, Andreas Butz

Human-Computer Interaction

Let Me Clarify: Interactive Modeling of Soft Attributes

Steffen Holter, Rita Sevastjanova, Michaela Benk et al. (+1 more)

People commonly express preferences through soft attributes such as is interesting,” which are difficult to operationalize for computational preference modeling. Latent representations can obscure how such attributes are interpreted, limiting transparency and user control. We formalize softness as the diversity of valid interpretations a description permits and provide empirical support for this characterization. Our proposed measure estimates softness to guide clarification and determine when to generate computational approximations. This is instantiated in AttriFy, a mixed-initiative system that translates descriptions into executable Python functions over text corpora. An interactive dashboard connects the dialogue to proposed functions and computed values, enabling users to inspect, accept, reject, and refine interpretations. An exploratory within-subjects study with 32 participants compares our softness-guided clarification approach with an LLM baseline in supporting interpretable attribute construction. This work positions softness as a basis for constructing explicit, user-steerable attribute dimensions for downstream preference elicitation and decision-making.
Read the full abstract

All authors Steffen Holter, Rita Sevastjanova, Michaela Benk, Mennatallah El-Assady

Human-Computer Interaction

What and Where in Identifying and Quantifying Conventions in Humans and AI Agents

Makoto Fukushima and Tim Puphal

Cooperative success rests in part on implicit communication. Partners converge on conventions: regularities that give an action meaning beyond its direct effect. Yet conventions are usually observed only indirectly: a score drop when partners are swapped shows that a convention existed, but not what it was or where it was used. In the card game Hanabi, the exact probability that a play fails can be computed from the hints received and the cards in view. Players who use conventions succeed more often than this prediction allows; the resulting margin is the convention gap. Off-belief learning provides a hierarchy of agents (levels 1-5) whose first level uses no conventions by construction and whose later levels use progressively more. For humans and these agents on one scale, we ask what the conventions are and where they are used. On what: in two independently collected corpora of human games, humans show a larger gap than every agent level. On the conventions both populations use, however, humans do no better than level 2. The human advantage is concentrated in a situation that no agent level learns to use: hints that point to a card other than the ones they touch. On where: conventions depend on what the player already knows about the card, not on how recently it was hinted, and they are used most in the mid-game; humans use them earlier and later in the game than the agents. In interventions on the agents, blocking a single high-gap situation costs little, because meaning reroutes to other situations, but blocking several together costs far more than the sum of the single costs. Thus, the convention gap turns conventions from an inferred cause of score loss into a measurable, comparable, and causally testable object.
Read the full abstract

Human-Computer Interaction

The Body-Algorithm-Emotion Loop: Making Emotion AI Misrecognition Visible through Artistic Practice

Chunyi Shen

This pictorial presents The Body in Algorithmic Colonialism, a screen-based interactive installation that critically examines Emotion-AI through artistic practice. The work uses facial tracking to simulate how commercial Emotion-AI systems translate facial movement into emotional categories and verdicts of compliance or non-compliance. Rather than treating explainability as technical transparency alone, the installation approaches XAI as an embodied encounter with misrecognition: viewers experience how the body is measured, classified, and returned to itself through algorithmic thresholds. Drawing on critical AI studies, affect theory, and practice-as-research, the pictorial positions artistic practice as a method for making affective classification visible and contestable. It documents the conceptual framework, technical process, and critical context of a prototype developed in the first year of a practice-based PhD. No audience responses, interviews, or participant data are analysed in this submission.
Read the full abstract

Human-Computer Interaction

Task-State Governance: A Virtual Machine for Verifiable and Recoverable GUI Agent Execution

Wenkui Yang, Yiming Hao, Kai Zhou et al. (+6 more)

Graphical user interface (GUI) agents now execute workflows across real applications, but when execution drifts, users must determine what changed, what should have remained untouched, and what can still be recovered. Plan-centric interfaces leave users supervising procedure, but users live with changes to application state. We introduce task-state governance, an interaction paradigm that represents a task as an editable contract linking desired outcomes, facts that must be preserved, and observed application state. We instantiate this paradigm in TaskVM, which projects live state into editable task controls, executes user edits as genuine interface actions, verifies outcomes through independent observation, and marks each action as reversible, compensatable, or irreversible. Across an automated contract benchmark, a controlled user study, and an open live-model exploration, the paradigm strengthened contract compliance and users' recovery control and trust, while execution outcomes were held constant. Task contracts open a design space for governable agent interfaces.
Read the full abstract

All authors Wenkui Yang, Yiming Hao, Kai Zhou, Chengyu Pu, Yuzhe Zhang, Jie Cao, Jin Huang, Shuai Ma, Feng Tian

Human-Computer Interaction

Scoping the Landscape of Large Language Model-based Affective Personal Informatics: Review, Reflection, Future Direction

Minjie Liu, Lihuan Li, Xuran Li et al. (+2 more)

Affective personal informatics (PI) involves capturing, tracking, analyzing, and reflecting on personal data to support emotional self-awareness and mental well-being. Recent advances in large language models (LLMs) are reshaping affective PI, yet a comprehensive survey of LLM-based affective PI systems is lacking. To bridge this gap, we systematically analyzed 45 papers published between January 2020 and June 2026 across four major computing and health databases, examining their macro trends, system taxonomies, and research methodologies. Our synthesis reveals an overreliance on multi-stage, text-centric pipelines, a predominance of active, screen-bound interactions, and a severe demographic skew toward young, university-campus-based cohorts in empirical evaluations. Based on these findings, we critically reflect on prevailing architectural and methodological shortcomings and outline actionable future directions for building end-to-end multimodal, low-burden, and inclusive affective PI systems.
Read the full abstract

All authors Minjie Liu, Lihuan Li, Xuran Li, Flora Salim, Benjamin Tag

Human-Computer Interaction

ShadAR: Real-time Prompting On-demand Visual Augmentation in Augmented Reality

Yanni Mei, Samuel Hartenstein, Jonas Wombacher et al. (+2 more)

Augmented Reality can reshape our visual perception (e.g., simulating color blindness) yet such effects are typically pre-programmed by experts, making visual augmentation static and inflexible. We introduce ShadAR, an AR system that allows users to describe expected visual augmentations (e.g., "Blur all people") for changing their vision in real time. ShadAR can graphically manipulate AR passthrough in real time via shader injection. When receiving a user's prompt, ShadAR uses LLMs (GPT5.2) to interpret the user's intent, analyzes the user's view with object detection (YOLOv26), generates a pixel shader, and applies it to the targeted region on AR passthrough. To understand usage patterns, we invited 21 HCI researchers to use ShadAR, identifying four application domains (design, health, productivity, and social interaction) and two prompt-framing patterns: procedural prompts specifying what to modify and how ("Highlight the book."), and end-goal prompts conveying desired mental or behavioral states ("Make me laugh."). Such end-goal prompts suggest a future in which users specify outcomes (e.g., "Make me not miss my flight") while the system determines how to realize them, though often with inherently indeterminate results. We discuss design opportunities for ShadAR and ethical concerns.
Read the full abstract

All authors Yanni Mei, Samuel Hartenstein, Jonas Wombacher, Florian Mueller, Jan Gugenheimer

Human-Computer Interaction

Think Once, Reflect Twice: How Feedback Shapes Human-AI Collaboration

Joshua Holstein, Philipp Spitzer, Sebastian Schäfer et al. (+2 more)

As artificial intelligence (AI) increasingly supports human decision-making in high-stakes domains, designing effective human-AI collaboration becomes critical. Successful collaboration requires humans to reflect on decision-making processes and adjust their strategies accordingly. Current research typically investigates interventions, such as explainable AI, that target reflection during the decision-making process (reflection-in-action). Less attention has been given to retrospective examination of decisions (reflection-on-action). This work explores reflection-on-action by studying the impact of feedback on reliance behavior and human-AI team performance. Specifically, we investigate feedback interventions targeting overrides of AI advice through a behavioral study (N=244). We find that such feedback influences reflection-on-action, while unidirectional feedback risks reducing subsequent reflection-in-action. Furthermore, we identify opposing shifts in reliance behavior that leave overall human-AI team performance unchanged but show context-dependent effects. Balanced feedback did not show these risks, preserving both concurrent and retrospective reflection, offering insights for designing for appropriate reliance on AI.
Read the full abstract

All authors Joshua Holstein, Philipp Spitzer, Sebastian Schäfer, Maximilian Förster, Gerhard Satzger

Human-Computer Interaction

AdaptaFace: Adversarial Assistance for Inclusive Facial Gesture Interaction

Siyu Zhang, Yelu Gu, Xiaoyun Xu et al. (+2 more)

Facial gestures can enable access for people with motor impairments (PwM), yet recognizers may not reliably interpret individual facial movement patterns. Existing approaches either constrain agency by requiring predefined templates or impose repeated calibration, data collection, and computation through user-specific retraining. We present AdaptaFace, an input-level compatibility layer using bounded, proxy-dependent adversarial perturbations to translate user-compatible gestures into inputs for an unchanged facial action unit (AU)-based recognizer. Designed around six personalization principles informed by interviews with 12 PwM, AdaptaFace supports both recognition enhancement and gesture vocabulary extension. In a within-subject study with 16 participants, AdaptaFace increased task completion from 59.7% to 90.5% and reduced workload from 5.61 to 4.21. Offline validation showed strong steering with aligned perturbation generation and evaluation but weaker cross-recognizer transfer, defining the approach's boundary. These findings frame adversarial assistance as input-side accessibility mediation for personalized facial-gesture interaction.
Read the full abstract

All authors Siyu Zhang, Yelu Gu, Xiaoyun Xu, Kirsten Cater, Oussama Metatla

Human-Computer Interaction · Cross-listed from cs.CV

DeltaStreamer: Towards Human-Like Agentic Streaming Video Understanding

Boyuan Sun, Deshui Miao, Wenzhao Gao et al. (+8 more)

We propose DeltaStreamer, a novel agentic framework for streaming video understanding inspired by how humans observe, remember, and investigate. Streaming video understanding requires a system to retain evidence before knowing which details future queries will need. Existing approaches often obscure event transitions and omit relevant details when compressing video history, and cannot recover missing visual evidence. DeltaStreamer addresses these limitations by combining change-aware memory with query-driven planning and targeted evidence gathering with replay. Before a query arrives, it continuously observes the stream and explicitly records visual changes in structured memory, preserving how events unfold rather than only what is present. When a query arrives, a query-driven planner identifies the required evidence and guides memory retrieval and targeted replay. By revisiting historical frames, the agent can recover visual details omitted during observation and ground its answer in evidence beyond the retained descriptions. This connects continuous observation with active investigation tailored to each question. Experiments on OVO-Bench and StreamingBench demonstrate improvements over representative streaming video understanding methods, with overall scores of 66.07% and 59.53%, respectively. Project page: https://bbbbchan.github.io/DeltaStreamer/
Read the full abstract

All authors Boyuan Sun, Deshui Miao, Wenzhao Gao, Shaoyong Jia, Shaohui Jiao, Chi Liu, Yan Shu, Derek Li, Bryan Dai, Shengsheng Qian, Qibin Hou

Human-Computer Interaction · Cross-listed from cs.CL

Measuring and Reducing Unusable Replies of LLM Agents

Yilin Jiang, Fei Tan, Xuanyu Yin et al. (+3 more)

People delegate multi-step work to LLM agents and read only the reply that the agent hands back. Current evaluation scores task completion and the faithfulness of reports, and it leaves open whether the requester can understand and use the reply. We call this property reply usability and measure it with a reader model that takes the requester's position and tests whether the reply can be interpreted from what the requester has seen, is in the requester's language and answers the request. Its agreement with the majority of five annotators lies within the range of the individual annotators. Across eight models from seven families, 54% of 480 replies fail, and the best model still fails on 30%. Failed replies mostly point to material the requester has not seen or use labels that the agent coined during its work. Stating explicit requirements for the reply reduces the failure rate from 0.52 to 0.43 together with errors and omissions, and a secondary model that rewrites the reply reduces it to 0.36, which human annotators confirm in a blind comparison. Reply usability can thus be scored alongside task success and improved with one appended message or one call to a secondary model.
Read the full abstract

All authors Yilin Jiang, Fei Tan, Xuanyu Yin, Guanchu Wang, Xia Hu, Yutao Yue

Human-Computer Interaction

"The Rose Garden Is a Cemetery": Spatial Dissonance and Divergent Public Experiences of an Indigenous Augmented Reality Tour

Grace Fujii, Vittorio Murtas, Sarah Sallee et al. (+5 more)

Community-authored augmented reality (AR) can contest dominant heritage narratives by juxtaposing stories with physical surroundings. We examine the Thámien Ohlone AR Tour, developed with members of the Ohlone Native American community at Mission Santa Clara and Santa Clara University. Combining paired pre/post surveys from 57 participants with collaborative thematic analysis, we compare reception among Ohlone community members, university affiliates, and Mission visitors. Spatial dissonance captures felt tensions when AR challenged familiar campus readings or when surroundings conflicted with Ohlone histories the AR affirmed, such as a garden's significance as a burial ground. Cultural frames shaped interpretation; contemporary people and relationships made Ohlone continuance tangible. Participants appreciated the tour and called for permanent physical acknowledgment. We propose staging reflective encounters, clarifying community-authored meanings, and using AR to prototype public acknowledgment and support community-directed advocacy. These findings foreground relationships to land and place, informed by Indigenous scholarship, in designing and evaluating AR.
Read the full abstract

All authors Grace Fujii, Vittorio Murtas, Sarah Sallee, Isabella 'Amne Gomez, Lauren Baines, Danielle Heitmuller, Amy Lueck, Kai Lukoff

Human-Computer Interaction

Beyond Initial Creation: Two Decades of Community-based Chart Revisions from Wikimedia Commons

Jasmine Y. Shih, Chase Stokes, Melanie Tory et al. (+1 more)

Open online communities like Wikipedia offer a rich space for understanding the organic evolution of public-facing artifacts, but little is known about how these communities collectively edit chart images. We investigate the phenomenon of "community editing" of visualizations by exploring chart revision patterns on Wikimedia Commons. We introduce a novel dataset of 7,786 longitudinal chart revisions across 1,664 images, labeled with edit categories using an LLM pipeline. Initial data analysis uncovered several novel findings. Multi-contributor revision is associated with more edits to mark styling and text content compared to single-contributor revision. Temporally, mark and text edits occur in more "bursty" cadences than data edits. In addition, while SVG charts attract more unique contributors and revisions than raster charts, their edit histories involve more tightly scoped and less diverse changes. This dataset enables a wide range of analyses of chart change over time.
Read the full abstract

All authors Jasmine Y. Shih, Chase Stokes, Melanie Tory, Marti Hearst

Tuesday 13 October · still arriving

Human-Computer Interaction · Cross-listed from cs.CL

Whose Data, Whose Benchmark? A User-Centric Framework for Auditing Evaluation Resources

Mir Tafseer Nayeem and Davood Rafiei

Benchmarks shape how model capabilities are measured and interpreted, yet their scores can obscure whose conventions, contexts, and judgments define successful performance. We introduce a user-centric audit framework that distinguishes users who contribute benchmark content from those whose needs evaluation results are intended to represent. The framework operationalizes seven dimensions across benchmark content, metadata, documentation, and scoring procedures. Across 16 resources, we find that American-centered conventions account for 62.4-94.4% of matched signals in 11 text benchmarks, while geographic visibility in five image-text resources is concentrated and strongly associated with national GDP share (mean Pearson r = 0.780; Spearman p = 0.778). Documentation reveals a systematic source-target gap: all 16 resources operationalize source-user context, but none operationalize target-user context. Paired scoring experiments further show that meaning-preserving answer variants reduce DROP exact-match accuracy from 100.0% to 33.3%, while TruthfulQA exhibits metric-dependent reductions in reference similarity. Our framework and documentation checklist make user assumptions auditable, clarifying what benchmark scores represent and where evaluation claims are supported.
Read the full abstract

Human-Computer Interaction

Moving Beyond the Flat Screen: Opportunities and Challenges of Extended Reality for Studying Human Brain and Behavior

Tom Bullock, Radha Kumaran, You-Jin Kim et al. (+3 more)

Much of what we know about human perception and cognition comes from austere experimental settings: static stimuli presented to sedentary observers on flat monitors, complex responses reduced to simple button presses, and brain activity recorded from immobilized heads using EEG or fMRI. These highly controlled paradigms have provided remarkable insights, but capture a small and often unusual slice of behavior, raising questions about how well the findings generalize to the real world. Extended reality (XR), spanning both immersive virtual reality (VR) and augmented reality (AR) that overlays digital content on the real world, offers a way beyond these constraints, supporting controlled, reproducible experiments in which participants actively engage with three-dimensional tasks. Drawing on a growing body of VR and AR studies of attention, memory, and language – our own and others', some involving EEG – we outline what XR makes newly possible for cognitive neuroscience and reflect on the practical and conceptual obstacles that remain. The opportunities are substantial, but realizing them requires solving practical problems of hardware, timing, and measurement, and being realistic about what a virtual or augmented world can and cannot stand in for.
Read the full abstract

All authors Tom Bullock, Radha Kumaran, You-Jin Kim, Emily Machniak, Tobias Höllerer, Barry Giesbrecht

Human-Computer Interaction

Understanding Goalball Training Across Roles and Team Contexts: Practices, Perceptions, Challenges, and Technological Opportunities

Gyeongdeok Kim, Junhyuk Jang, Donghyeon Kim et al. (+2 more)

Goalball is a team sport for visually impaired athletes, played by throwing and blocking a ball while wearing eyeshades. However, existing studies provide limited guidance on which everyday training difficulties technology should address for players and coaches across team contexts. We conducted semi-structured interviews and focus groups with 18 players and 12 coaches from seven South Korean teams spanning youth, second-division, first-division, and national-team contexts. Using qualitative coding and matrix-based analysis, we describe current training practices, participants' views of these methods, the challenges they encounter, and their technological support needs. Drawing on the relationships among these four areas, we discuss implications for the design and evaluation of future support technologies that fit everyday goalball training across roles and team contexts.
Read the full abstract

All authors Gyeongdeok Kim, Junhyuk Jang, Donghyeon Kim, Billy Park, Gunhyuk Park

Human-Computer Interaction

Iterative Design and Evaluation of a VR Nursing Training System: Insights from Clinicians and Student Learners

Danhua Zhang, Cynthia Bradley, Christopher Kissell et al. (+5 more)

This paper presents the iterative design and evaluation of a virtual reality (VR) nursing training system developed in collaboration with nursing education specialists to enhance procedural skill acquisition, clinical decision-making, and critical thinking among pre-licensure nursing students. Two clinical scenarios, chest pain and oxygen therapy, were created to simulate acute care settings, with design refinements informed by an expert evaluation study (n=8) and further assessed through an effectiveness study involving nursing students (n=17). Results demonstrate high user acceptance, perceived usefulness, and significant post-simulation increases in learner confidence. The VR system was found to be an accessible supplemental training tool that can foster self-directed learning. Finally, we present design recommendations for developing VR simulations for nursing education, emphasizing collaboration with domain experts, accommodating diverse learning goals, and simulating complex clinical workflows.
Read the full abstract

All authors Danhua Zhang, Cynthia Bradley, Christopher Kissell, Courtney Hutton Pospick, Seraphina Yong, Qiao Jin, Tongyu Nie, Evan Suma Rosenberg

Human-Computer Interaction

What Human-AI Interaction Leaves Behind: Residual Human States – A Trajectory-Based Framework and Multi-Study Analysis

David Herrera-Poyatos, Carlos Peláez-González, Cristina Zuheros et al. (+1 more)

Human-AI Interaction is commonly evaluated by what an artificial agent enables during use, yet interaction can also alter the human who acts next. We introduce Residual Human States, a trajectory-based framework for representing the potentially multidimensional human state carried forward after interaction with AI, relative to a relevant alternative trajectory. The framework distinguishes assisted output from human-state transition and derives four empirical propositions: multidimensionality, interaction-policy dependence, horizon dissociation, and temporal persistence. Its central claim is that assisted performance does not by itself determine the human state carried forward. We examine these propositions through four empirical cases with different inferential roles. Published medical-training evidence illustrates how one AI interaction history can be associated with both slightly higher observed accuracy on some later unaided cases and transmission of a systematic decision bias on others. A second diagnostic-learning study shows that later unaided behaviour differs across AI-support policies, while also revealing that aggregate accuracy cannot by itself distinguish improved discrimination from a shifted response criterion. In a classroom experiment, generative-AI tutoring substantially improved assisted practice performance without a corresponding detectable gain on a subsequent unaided exam. A longitudinal reanalysis of personalised AI dialogue further shows that residual contrasts can evolve across post-interaction horizons and that such change may arise from the comparison trajectory as well as from the focal intervention. Evaluating interactive AI systems therefore requires attention not only to contemporaneous performance, but also to which human states alternative interaction trajectories produce, relative to which counterfactuals, and how durably.
Read the full abstract

All authors David Herrera-Poyatos, Carlos Peláez-González, Cristina Zuheros, Francisco Herrera

Human-Computer Interaction

Bio-Inspired Electrotactile Feedback: A Framework and Impacts of Multimodal Cues

Melika Emami, Amirhossein Bayat, Rahim Tafazolli et al. (+1 more)

Electrotactile stimulation, as a haptic feedback method, often struggles to evoke high-quality natural sensations. Consequently, despite its practical advantages, it is generally less preferred in human-machine systems than mechanical approaches. In this paper, we propose a bio-inspired stimulation framework that generates signals modelled on afferent neural activity derived from a touch model. By producing more biologically plausible spiking patterns, the proposed approach aims to enhance the realism of perceived tactile sensations. To evaluate these stimulation patterns in a realistic task context and to quantify the influence of visual and movement cues, we conducted a series of subjective experiments under three conditions. The results indicate that bio-inspired modulation was effective: in particular, bio-inspired co-modulation of amplitude and frequency achieved a favourable pressure-irritation trade-off, increasing the probability of perceiving pressure while maintaining low irritation (pressure: 0.78; irritation: 0.24) relative to the other patterns. Visual cues and hand movement further improved feedback quality by significantly increasing the probability of reporting pressure (p$<$0.001) without a corresponding increase in irritation, and by increasing mean naturalness ratings. Overall, these findings demonstrate the potential of bio-inspired electrotactile patterns to improve haptic feedback quality, and highlight the importance of evaluating such methods under realistic, multimodal interaction scenarios.
Read the full abstract

All authors Melika Emami, Amirhossein Bayat, Rahim Tafazolli, Atta Quddus

Human-Computer Interaction · Cross-listed from cs.CV

AIRRI: Towards a Human-Readable, Machine-Resistant Reading Interface

Emory Salaun, Syed Ayon, Kaleel Mahmood et al. (+1 more)

The rapid advancement of image-to-text extraction systems and large language models (LLMs) poses new challenges for educational assessment by making assignment materials easy to transcribe and solve with minimal human effort. To address this problem, we present AIRRI (Artificial Intelligence Resilient Reading Interface), a framework for generating machine-resistant educational content, a dataset of higher-education assignments for evaluation, and an initial evaluation of the human readability of AIRRI machine-resistant text designs. AIRRI employs two categories of machine-resistance techniques. First, AIRRI applies image transformations including character spacing, word spacing, line spacing, character opacity variation, stripe overlays, and image noise. Second, AIRRI applies decision-based adversarial attacks: RayS, SurFree, and ADBA. We evaluate AIRRI against three image-to-text systems: Tesseract, EasyOCR, and the transformer-based GOT-OCR2 model. Our results show that AIRRI substantially degrades automated text extraction performance. AIRRI increased character error rates for EasyOCR (6.9% to 55.9%), Tesseract (2.3% to 100.0%), and GOT-OCR2 (30.7% to 48.1%). Word error rates increased for EasyOCR (24.9% to 96.7%) and Tesseract (10.7% to 100.0%). Among the evaluated decision-based attacks, ADBA consistently produced the largest degradation in image-to-text performance, while RayS and SurFree produced more moderate reductions. EasyOCR and Tesseract were generally more susceptible to both image-based and adversarial perturbations than the transformer-based GOT-OCR2 model. Our findings suggest that AIRRI can significantly reduce the reliability of image-to-text-to-LLM workflows, increasing the effort required for AI-assisted assignment completion.
Read the full abstract

All authors Emory Salaun, Syed Ayon, Kaleel Mahmood, Shaun Wallace

Human-Computer Interaction · Cross-listed from cs.AI

Knowing the User Is Not Knowing What We Established: Grounding Awareness in Personal Agents

Jiajun Chen, Xinyue Chen, Yicheng Jiang et al. (+1 more)

Personal-agent memory has largely focused on learning what users know, prefer, and want. However, knowing the user does not necessarily mean knowing what the user and the agent have established together. An assistant's proposal should not become an authorized instruction merely because it is remembered. We introduce Grounding Awareness, the ability to distinguish accepted, unacknowledged, revised, and scoped arrangements when selecting actions. We develop a controlled benchmark to test whether agents can tell the difference between proposals users have accepted and those they have not, and whether they apply accepted arrangements only where appropriate. Across multiple agents and memory systems, we find that strong memory performance does not ensure Grounding Awareness. More surprisingly, providing more complete user information can increase execution of unacknowledged proposals, while persistent memory can cause agents to treat these proposals as if they were user instructions. We trace these failures through memory writing, retrieval, and action selection. Finally, we introduce Grounding Layer, a lightweight, training-free extension that explicitly tracks grounding status and checks consequential actions, substantially improving agent behavior without modifying models or memory systems.
Read the full abstract

All authors Jiajun Chen, Xinyue Chen, Yicheng Jiang, Hua Shen

Human-Computer Interaction

Communicating Uncertainty in AI-Supported Nurse Scheduling: Expert Co-Design and Evaluation of an Explainable AI Prototype

Alice Rollwagen, Emily Schiller, Anton Hummel et al. (+3 more)

Nurse scheduling is a complex optimization problem in which nurses are assigned to shifts to meet staffing demands while complying with working-time limits, qualification requirements, and other operational and regulatory constraints. Artificial intelligence is increasingly being used to support this process by forecasting future staffing needs and providing advice on how to improve shift scheduling. Effective human-AI collaboration in nurse scheduling requires users to understand, interpret, and appropriately act on AI forecasts and advice. However, it remains insufficiently understood how healthcare professionals use transparency mechanisms like uncertainty communication and AI advice explanations when making staffing decisions. Applying a human-centered design approach, we iteratively developed a prototype decision support system for nurse scheduling and evaluated it through a mixed-methods user study with expert users. Using thematic analysis, questionnaire data, and eye tracking, we examine how healthcare professionals interact with the system and how uncertainty communication and explanations of AI advice influence understanding, decision confidence, trust, and usability. We found that the uncertainty communicated in AI forecasts was frequently misinterpreted and reduced decision confidence. Explanations were generally perceived positively, but did not significantly affect decision confidence or advice acceptance. We conclude with eight design implications derived from our findings, providing insights for the design of more transparent and effective AI-supported decision support tools in healthcare.
Read the full abstract

All authors Alice Rollwagen, Emily Schiller, Anton Hummel, Kathrin Ebertsch, Andreas Riener, Jan-Philipp Steghöfer

Human-Computer Interaction · Cross-listed from cs.CL

Known-Voice Cloning and the Limits of Personality Transfer for Character Design

Johanna Magdalena Kuch, Maud van Lier, Daksitha Senel Withanage Don et al. (+3 more)

Recent advances in zero-shot voice cloning systems make it possible to reproduce a recognizable voice from a short recording and attach it to arbitrary virtual characters. Prior work suggests that familiar vocal identity can carry over associated personality impressions to new targets such as robots or virtual characters. However, these works have mainly examined familiarity and personalization effects rather than directly testing whether a cloned familiar voice systematically transfers personality impressions to an embodied virtual character. Such transference raises the concern that cloned voices could be used to manipulate people into engaging with virtual characters as capable social actors. Across two studies, we investigated whether personality impressions transfer when a virtual character speaks with the cloned voice of a familiar person. Study 1 examined this question in a real-world university setting using the cloned voice of a lecturer familiar to the participating students. Study 2 examined the same question online using voices of social media content creators and members of their communities. Drawing on the concepts of identity activation, trait transfer, and person reconstruction, we show that evidence of personality transfer was limited in both studies. Observed effects were selective and did not generalize across traits or study contexts. In the online study in particular, recognizable voice identity cues activated expectations about the speaker but did not reconstruct the broader person impression. Qualitative responses further indicated that judgments were frequently shaped by perceived artificiality and by whether voice and appearance fit together. Our findings moderate common concerns that cloned voices straightforwardly manipulate people into perceiving artificial characters as capable social actors.
Read the full abstract

All authors Johanna Magdalena Kuch, Maud van Lier, Daksitha Senel Withanage Don, Patrick Gebhard, Tanja Schneeberger, Elisabeth André

2610.18669 · 13 Oct · 17 pages, 3 figures, 1 table. This preprint has been accepted for publication in Connected Futures - Nature, Robots, and Society (Robophilosophy 2026), in the Lecture Notes in Computer Science (LNCS) series, volume 17299 (expected publication: December 2026)

Abstract PDF $BOT ×1

Human-Computer Interaction

From Mind Mapping to Dream Interpretation: Understanding Muslim Users' Engagement with GenAI Tools in Religious Context

Ishrat Jahan, Soomanib Kamruzzaman, K M Nazib Hossen et al. (+2 more)

Generative Artificial Intelligence (GenAI) tools are increasingly integrated into everyday life, including sensitive domains such as religion. Prior works examined alignment with religious values and discussed the potential risks. However, to gain empirical insights into the adoption of these tools in users' religious lives, in this study, we conducted an online survey with 178 Muslim participants in Bangladesh and collected 726 prompts on religious use cases, along with open-ended responses about using these tools in religious contexts. Using reflexive thematic analysis, we identify three key roles these tools play in users' religious lives: (1) religious assistants, (2) perceived religious scholars, and (3) faith counselors. However, participants expressed concerns about misguidance, overreliance, and violations of Islamic tradition, and reported adopting protective strategies to avoid these pitfalls. Based on these, we propose a set of design guidelines for faith-sensitive GenAI systems that would preserve religious boundaries while supporting religious practices.
Read the full abstract

All authors Ishrat Jahan, Soomanib Kamruzzaman, K M Nazib Hossen, Novia Nurain, A. B. M. Alim Al Islam

Human-Computer Interaction · Cross-listed from cs.CY

How Safe Is ChatGPT Health Advice? Evidence from Real User Conversations in Four Global South Countries

Shreyasi Roy Chowdhury and Kiran Garimella

People increasingly bring their health questions to ChatGPT, yet safety evaluations of LLM health advice rest almost entirely on clinician-authored vignettes, synthetic prompts, or simulated user profiles. The gap is widest for the Global South, where concerns about demographic bias and unsafe recommendations are well documented but have never been tested on real usage. We present, to our knowledge, the first demographically grounded safety evaluation of real-world health conversations with ChatGPT. From a consent-based data-donation corpus of 202,590 conversations from 1,252 users in India, Nigeria, Brazil, and Pakistan (December 2022 to February 2026), each linked to self-reported age and gender, we identify 15,555 health conversations, classify their health intent using an LLM classifier adapted from an established taxonomy, and score response safety under both a context-blind and a user-context-aware rubric. Roughly one in five health conversations receives a high-risk rating. Risk concentrates in symptom and condition queries and recurs through identifiable failure modes: unsafe medication or dosage guidance, harmful self-diagnosis, missing safety warnings, encouraging over-reliance on the model, and failure to refer users to professionals. Supplying the user's own accumulated context raises the high-risk share by 13-17 percentage points, and about one in four conversations rated safe in isolation turns out to be unsafe for the specific person asking. Safety evaluations built on synthetic cases are likely to miss these harms.
Read the full abstract

Human-Computer Interaction

Understanding the "Why": Investigating the Design of Privacy Support Tools (PSTs) through the Lens of Psychological Needs

Sara Hahn, Verena Zimmermann, Simon Althaus et al. (+4 more)

Privacy Support Tools (PSTs) help users control their digital data, but face low adoption due to resignation and overwhelm. While current regulations aim to enhance privacy control, users still struggle to manage their privacy. To address this, we explore how PSTs can be designed for long-term adoption by applying the theoretical lens of psychological needs, as their fulfillment motivates users to use a digital product. We conducted in-depth interviews with 15 users and identified Security, Autonomy, and Competence as key psychological needs. From our results, we derive implications and recommendations for the need-sensitive design of PSTs.
Read the full abstract

All authors Sara Hahn, Verena Zimmermann, Simon Althaus, Andreas Heinemann, Ephraim Zimmer, Alina Stöver, Nina Gerber

Human-Computer Interaction

Exploring Object Recall in an Object-Rich AR Office Scene

Sangita Kunapuli, Shane Dirksen, Radha Kumaran et al. (+4 more)

Augmented reality (AR) changes how users perceive and attend to their surroundings. Prior work suggests AR can reduce attention to the physical world, but it remains unclear whether this holds in environments dense with both virtual and physical objects, and which factors most affect object recall. This study examines recall in a mixed-reality office scene after an object-selection task, testing object-level factors (virtuality, size), context-level factors (semantic congruency, saliency), and user-level factors (cueing, dual tasking). Cueing informed participants of a later recall test, while dual tasking added cognitive load through a secondary audio task. Virtual objects were recalled more accurately than physical ones. Cueing produced a modest improvement in recall of present objects. Using playback software to reconstruct each participant's view throughout the trials, we computed object-level saliency and found a weak positive correlation with recall, slightly stronger than the effects of virtuality, but not reaching statistical significance. These findings have implications for AR interface design and for building mixed-reality environments that better support cognition during real-world tasks.
Read the full abstract

All authors Sangita Kunapuli, Shane Dirksen, Radha Kumaran, Yanxiu Jin, You-Jin Kim, Misha Sra, Tobias Höllerer

Human-Computer Interaction · Cross-listed from cs.AI

CompanionSafe: From Multi-turn Safety Evaluation to Guardrails for AI Companions

Xinyang Liao, Lingyu Li, Kaicheng Shen et al. (+4 more)

AI companions provide users with emotional support and ongoing companionship, but their sustained relational interactions also introduce distinct safety risks. Prior work largely targets general harms or specific companion platforms and risk categories, while relying on manually collected conversations, limiting scalable and user-grounded evaluation and mitigation. To address this, we introduce CompanionSafe, a unified evaluation-to-mitigation framework for AI companion safety. Its evaluation component, CompanionEval, provides scalable multi-turn evaluation by constructing 1,300 relationally grounded settings from diverse personas, matched companion characters, and dialogue scenarios across four turn tiers. Its mitigation component, CompanionGuard, converts benchmark-exposed failures into turn-localized supervision for companion-specific risk detection. Our results show that even strong LLMs and existing guard models remain vulnerable to companion-specific relational harms, while both closed- and open-source models exhibit increasing safety risks as interactions become longer. Together, CompanionEval and CompanionGuard form a practical evaluation-to-mitigation loop for AI companion safety, turning relational safety failures exposed during multi-turn interaction into supervision for companion-specific guardrails.
Read the full abstract

All authors Xinyang Liao, Lingyu Li, Kaicheng Shen, Yang Yao, Na Lu, Yan Teng, Xia Hu