Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Human-Computer Interaction

cs.HC Watch Human-Computer Interaction

Friday 25 September

Human-Computer Interaction

CDBG: Causally Motivated Dual-Invariance Learning against Topological and Predictive Shifts in EEG Workload Recognition

Yuzhe Zhang, Wenmin Zhou, Chengxi Xie et al. (+5 more)

Generalizing Electroencephalography (EEG)-based mental workload recognition to unseen subjects remains a formidable challenge due to severe inter-subject variability. While functional brain graphs effectively model distributed cognitive dynamics, their inherent subject-specificity induces two coupled distribution shifts: a class-conditional topological shift in the underlying functional connectivity, and a predictive mechanism shift in the learned representation-to-label mapping. Motivated by the subject-induced distribution shifts, we propose CDBG, a Causally motivated Dual-invariance learning framework for Brain Graphs. CDBG disentangles and mitigates these shifts via a two-stage rationale learning pipeline. First, it employs stochastic edge masking to extract sparse, workload-predictive graph rationales, regularized by workload-conditional Laplacian spectral alignment to enforce topological invariance across subjects. Second, it applies subject-wise Invariant Risk Minimization (IRM) to the graph representations, ensuring environment-wise risk stationarity. Extensive experiments on a self-built air traffic controller EEG cognitive workload dataset and multiple public datasets under a strict leave-one-subject-out protocol demonstrate that CDBG significantly outperforms state-of-the-art cross-subject and graph-based baselines, improving the Macro-F1 score by up to 4.23%, while simultaneously providing neurophysiologically interpretable functional rationales.
Read the full abstract

All authors Yuzhe Zhang, Wenmin Zhou, Chengxi Xie, Kai He, Jihong Wang, Huan Liu, Man Yao, Daoqiang Zhang

Human-Computer Interaction

Sampling Safe Futures: Multimodal Trajectory Planning for Personalized Safety in Anthropomorphic AI

Benedetta Picano and Dusit Niyato

Anthropomorphic artificial intelligence systems increasingly remember personal details, display empathy, and are engaged with as social counterparts, creating forms of risk that emerge from the evolution of the user-system relationship over time. Existing safeguards largely operate at the level of individual conversational turns and cannot determine whether a sequence of seemingly acceptable interactions is cumulatively moving a particular user toward harm. This paper introduces personalized trajectory-level safety, a framework that treats relational safety as a sequential decision problem over a latent escalation state inferred from the user's messages and influenced by the system's responses. At each turn, a screening step first discards any response strategy that does not preserve at least one safe continuation of the interaction under every plausible model of the user. Among the remaining strategies, we formulate action selection as multimodal trajectory sampling, and use a Generative Flow Network to generate diverse future evolutions in proportion to their plausibility, safety, and utility. The system then selects the strategy that preserves the largest fraction of safe and useful continuations. We evaluate the framework in simulation, calibrated on statistics reported for real human-chatbot interactions, and using response strategies derived from public benchmarks. Results show that trajectory-aware decision making substantially reduces the frequency of harmful states while keeping helpful interaction. This work reframes safety for anthropomorphic AI from response-level filtering to personalized control over the future evolution of human-AI relationships. The source code is available at https://github.com/benedettapicano/ANTHROPOMORPHIC_SAFETY_TRAJ.
Read the full abstract

Human-Computer Interaction

Skill Profiling with Attributable Reasoning (SPAR): A Wearable Analysis System for Boxing

Nibraas Khan, Hanchen David Wang, Enya Bullard et al. (+5 more)

A punch is a ballistic, full-body action driven by a kinetic chain running from the legs through the trunk to the arm, where a small sequencing error separates a scoring strike from a miss. Wearable sensors can capture this movement in the gym, but most deployable systems only classify which punch was thrown rather than assess how well it was thrown. We present Skill Profiling with Attributable Reasoning (SPAR), an eight-IMU garment and pressure-insole system that classifies each punch as expert or novice and treats an explanation of that prediction as feedback. Feedback is only useful if the person receiving it can act on it, so SPAR explains the prediction at three tiers, a per-joint attribution for the analyst, a counterfactual over kinetic-chain layers for the coach, and a plain-language narrative of the two for the athlete. Across 17 participants and 4,713 punches, SPAR reaches a leave-one-participant-out AUC of 0.842 (95% CI [0.769, 0.907] over participants). A frozen time-series foundation model encodes the joint-angle and plantar-force series, and a small transformer trained on the cohort classifies the encoding. We audit the two quantitative tiers and report six themes from a thematic analysis of interviews with six practicing boxing coaches.
Read the full abstract

All authors Nibraas Khan, Hanchen David Wang, Enya Bullard, Ritam Ghosh, Ruj Haan, Aarav Agrawal, Meiyi Ma, Nilanjan Sarkar

Human-Computer Interaction · Cross-listed from cs.CY

Beyond the Last Truffula Tree: SustainAI - A Water-Aware, Closed-Loop Framework for Environmentally Accountable AI

Farnaz Farid, Tashfia Towkee, Sania Nasreen et al. (+1 more)

As artificial intelligence (AI) becomes embedded in everyday life, its environmental footprint, particularly water consumption remains largely invisible. While energy and carbon impacts are widely recognized, the substantial freshwater demands of data center cooling and electricity generation receive little attention. To address this gap, we introduce SustainAI, a water-aware, closed-loop framework incorporating environmental accountability into AI deployment. SustainAI integrates real-time water metering, a hallucination-aware penalty model, and a water-aware routing algorithm that accounts for regional water stress. Evaluated via Small Language Models (SLMs) extracting health misinformation, results reveal an 11-fold variation in water footprint across geographically distributed data centers (0.0477 mL to 0.5360 mL per inference). Across 1,335 inference runs, the system consumed approximately 399 mL of water but produced only 240 correct outputs, demonstrating that substantial resources are spent on inaccurate responses. Crucially, SustainAI extends beyond technical optimization through a Care by Design lens, framing AI sustainability around relational ethics, regional equity, and ecological stewardship. By combining water monitoring, adaptive accountability, and Care by Design principles, SustainAI provides a practical foundation for integrating ethical care and environmental responsibility into AI infrastructure design and lifecycle management.
Read the full abstract

All authors Farnaz Farid, Tashfia Towkee, Sania Nasreen, Sami bin Azad

Human-Computer Interaction

"If You're Not Doing It, Somebody Else Is": Active Negotiation and the Invisible Labor of Sustained LLM Use

Matt Viana, Patrick Erickson, Shomir Wilson et al. (+1 more)

Large language models (LLMs) have become fixtures of academic work even as their users describe them as degrading their writing, thinking, and skills. Dominant adoption frameworks read continued use as evidence of satisfaction, and cannot explain continued use of a distrusted tool. We interviewed 36 graduate student workers, balanced between English-as-a-foreign-language (EFL) and non-EFL speakers, and introduce the Active Negotiation framework: a model of sustained LLM use as a recurring cycle of risk, mitigation, and justification. A failure surfaces a risk, mitigation labor addresses it, and a justification renders the residual risk tolerable until the next failure reopens the cycle. The cycle runs across three dimensions: practical, auditing output; internal, auditing one's own cognition and identity; and social, managing how peers and institutions perceive use. EFL participants invoke linguistic parity as a further justification. We reframe continued adoption as compliance sustained by invisible labor.
Read the full abstract

All authors Matt Viana, Patrick Erickson, Shomir Wilson, Dana Calacci

Human-Computer Interaction · Cross-listed from cs.RO

Multi-Objective Human-in-the-Loop Bayesian Optimization of a Lower-Limb Exoskeleton

Neil Janwani, Matthew T. Lerner, Aaron J. Young et al. (+1 more)

Human-in-the-loop optimization (HILO) is a common approach for optimizing the control of assistive devices to account for the wearer's unique biomechanics and subjective preferences. However, despite research suggesting that a person may have a different prioritization of objectives depending on time-varying factors such as the environment, their mood, or energy levels, existing HILO approaches only consider a single objective or enforce a fixed weighting on a set of objectives. Neither approach is capable of representing an individual's preferences over objectives. In this work, we propose Multi-Objective Human-in-the-loop Bayesian Optimization (MO-HILBO), which builds on explicit multi-objective Bayesian optimization to efficiently infer a personalized set of Pareto-optimal controllers. We compare our approach with an existing multi-objective HILO method and experimentally demonstrate MO-HILBO on a lower-limb exoskeleton across two objectives: metabolic cost (efficiency) and ordinal human feedback (comfort). We find that MO-HILBO (1) discovers Pareto-optimal controllers, and (2) that the pairwise ordering of points on the Pareto front itself is consistent with validation trials. Lastly, we open-source mohilo, a Python package for running both HILO and MO-HILBO on wearable devices: https://dynamicmobility.github.io/mohilo/.
Read the full abstract

All authors Neil Janwani, Matthew T. Lerner, Aaron J. Young, Maegan Tucker

Thursday 24 September

Human-Computer Interaction

CraftTrace: Unflattening Videos into Malleable, Creation-Inspired Structures for Generative Editing

Boyu Li, Yuqian Zhou, Duotun Wang et al. (+6 more)

Recent generative video editing models enable video content modification (e.g., changing a character) but target short clips. Extending them to full multi-shot videos requires tedious work to locate relevant content across shots, segment it into clips, craft context-aware editing prompts for each clip, and repeatedly articulate complex editing intent. To address this, we explore an interaction paradigm for editing through underlying video structures (e.g., scripts, scenes, characters, shots, and their relationships). We present CraftTrace, an interactive prototype that transforms a video into a malleable, multilevel structure for generative editing. Users work in task-centric workspaces to modify elements or reshape relationships, while an AI agent translates and propagates changes across the video. A user study and expert review show that this structure helps users understand videos, formulate and refine editing intent, and explore alternatives, supporting rapid prototyping during early-stage exploration and full video post-production.
Read the full abstract

All authors Boyu Li, Yuqian Zhou, Duotun Wang, Ding Li, Zhe Lin, Nanxuan Zhao, Zeyu Wang, Lin-Ping Yuan, Hongbo Fu

Human-Computer Interaction

Steering Versus Teleporting in Mobile Virtual Reality

Kristen Grinyer, Daniel Zielasko and Robert J. Teather

Mobile virtual reality (MVR) provides low-cost access to extended reality (XR), but its limited input restricts use of common locomotion techniques such as head-decoupled and velocity-controlled steering. Using a low-cost controller with an audio-based button, we compared gaze- and controller-directed steering and teleportation in MVR. We included a controller pitch-based speed-control technique enabling continuous head-decoupled steering. Performance and participant feedback indicate that gaze was better suited to teleportation, whereas controller pointing better supports steering in primed search. We compare with similar techniques in standard VR and derive design considerations for low-cost, narrow-FOV hardware, demonstrating the potential to support varied travel techniques and velocity control in low-fidelity XR.
Read the full abstract

Human-Computer Interaction

Epstein Files Engine: Agentic Search for Investigative Journalism

Duy K. Nguyen, Teresa Mondría Terol, Dylan Freedman et al. (+1 more)

On Jan. 30, 2026, the U.S. Department of Justice released a mixed-media collection concerning Jeffrey Epstein, including about three million pages of PDFs. We describe the Epstein Files Engine, an A.I. agent The New York Times deployed to investigate the files. The Engine translated reporter questions into Google BigQuery SQL queries across three corpora: Epstein-related releases, the Times's archive and external, Epstein-related news headlines. It used an LLM to plan queries and returned citation-rich answers a reporter could verify and trust. More than 100 journalists used the Engine, and it contributed to at least 20 published stories. We report how reporters queried it and describe Diff, our text-and-visual duplicate matching method that amplified novelty signals and allowed the Engine to surface genuinely new information. We argue that newsroom agents serve newsrooms best not as autonomous writers, but as interfaces to source material and institutional knowledge.
Read the full abstract

All authors Duy K. Nguyen, Teresa Mondría Terol, Dylan Freedman, Zach Seward

Human-Computer Interaction

Orchestrating GenAI for Interdisciplinary Research

Shirley Anugrah Hayati, Moyan Zhou, Patricia Anugrah Setiani et al. (+3 more)

As researchers tackle interdisciplinary problems, they face the need to deepen expertise in primary areas while rapidly acquiring knowledge in secondary domains. Generative AI (GenAI) is increasingly positioned to meet this need, from general-purpose chat assistants to Deep Research tools marketed as autonomous research agents. Prior work has examined how researchers use GenAI to support single-discipline or general research tasks. However, we know little about the goals and GenAI practices in interdisciplinary research. We conducted a longitudinal study and semi-structured interviews with 15 interdisciplinary researchers to examine how interdisciplinary researchers actually orchestrate GenAI. Findings show that researchers leaned on GenAI to fill knowledge gaps while maintaining epistemic agency for novelty discovery. We also uncovered an expertise paradox: GenAI outputs were hardest to verify when most needed. Our empirical insights motivate GenAI designs that calibrate verification to researchers' expertise, nudge toward cross-domain synthesis, and adapt prompting and outputs to disciplinary conventions.
Read the full abstract

All authors Shirley Anugrah Hayati, Moyan Zhou, Patricia Anugrah Setiani, Ruizi Wang, Joseph Chee Chang, Dongyeop Kang

Human-Computer Interaction · Cross-listed from cs.AI

Thinking Less to Simulate Better: Intuitive Prompting Improves LLM Agents Simulating Individual Social Media Reactions, Including Unfamiliar Content

Ljubisa Bojic, Tijana Stanic, Joerg Matthes et al. (+3 more)

Platform policies are increasingly tested on artificial users, making agent fidelity important. Yet convincing fake profiles could also manipulate perceived public opinion before elections. Validation has concentrated on agreement with human behaviour and has paid little attention to whether an agent behaves in line with the profile it was given. The present study profiled eight Serbian participants through a questionnaire, a deep interview, and a written self-presentation, recorded their reactions to sixty-eight social media posts, and asked four language models to predict those reactions under five prompt conditions varying profile content and instruction style. Attitudinal content improved prediction over demographic backstories by a wide margin. Agents matched their stated profiles more closely than participants matched their own survey answers, and consistency proved unrelated to fidelity once profile information was present. Instructing models to respond intuitively and immediately rather than analytically gave the highest fidelity of any condition and cut the compression of individual differences from seven times the human level to three. The advantage held on posts about topics the questionnaire never raised, where that condition reached the highest fidelity of any setup and beat a crowd baseline by a wide margin, which suggests that agents prompted this way could serve as general-purpose simulated users rather than specialists on the topics they were profiled for. Results may bear implications for the development of language models, because intuition-based setups appear better suited to some tasks than reasoning-based ones.
Read the full abstract

All authors Ljubisa Bojic, Tijana Stanic, Joerg Matthes, Agariadne Dwinggo Samala, Bojana Dinic, Jue Wang

Human-Computer Interaction

Redesigning Trust: Replacing Dark Patterns with Fair Choice Architecture in Financial Interfaces

Oluwadamilola Awakan, Tawan Aroonwechkul, Roshan Gunjoor et al. (+2 more)

Digital financial platforms make enrollment effortless and cancellation laborious. This asymmetry is a dark pattern that manipulates users who have already decided to leave. Existing work identifies such patterns after deployment, and regulators sanction them after harm, yet neither provides designers with a criterion for building interfaces that avoid manipulation. We model the provider as an adversary whose instrument is effort and express fairness as a constraint requiring that leaving never cost more than joining. Defining interaction cost over navigation steps, mandatory inputs, and confirmation prompts, we prove that this constraint holds for every assignment of effort weights if and only if no component of the exit flow exceeds its counterpart at entry. Fairness is therefore verifiable by counting rather than by estimating cognitive effort, and exact equivalence is unnecessary because exit legitimately requires fewer inputs than entry. We instantiate the model, together with invariants for visual parity and linguistic neutrality, in a mobile credit card prototype with parallel sign-up and cancellation workflows.
Read the full abstract

All authors Oluwadamilola Awakan, Tawan Aroonwechkul, Roshan Gunjoor, Supriya Khadka, Sanchari Das

Human-Computer Interaction

A Benchmarking Framework for Context-aware XR Interfaces

Hyunsung Cho, Sarah Yewon Yun, Nancy Ruonan Sun et al. (+7 more)

Everyday Extended Reality (XR) systems aim to provide context-aware access to the right functionalities at the right time and place, with minimal manual reconfiguration as users switch context. Yet these interfaces are hard to evaluate: current prototyping and user-study workflows offer no systematic, repeatable way to compare adaptation methods across users and scenarios. We present ContextXR, a novel benchmarking framework for context-aware XR interfaces. ContextXR represents an XR application as a connected graph of functional facets, each a semantically coherent group of related capabilities that together support a shared user intent. On this representation, we build MineXR++, a dataset augmenting prior XR interface data with facet-level annotations, and formulate three canonical tasks of context-aware suggestion: context factor analysis, initial facet suggestion, and next facet suggestion. Our evaluation protocol scores suggestion methods by a simulated interaction metric, the navigation and search cost of reaching the desired functionality. Through experiments benchmarking global popularity, relational retrieval, and LLM-based methods, we demonstrate that ContextXR enables the systematic, reproducible evaluation of context-aware XR interfaces.
Read the full abstract

All authors Hyunsung Cho, Sarah Yewon Yun, Nancy Ruonan Sun, Ben Lafreniere, Mark Parent, Kashyap Todi, Tanya R. Jonker, Hrvoje Benko, Sherry Tongshuang Wu, David Lindlbauer

Human-Computer Interaction

Who Acts When the User Is Gone? Digital Remains, Survivor Claims, and Post-Mortem Governance

Supriya Khadka, Dhiman Goswami and Sanchari Das

Digital systems continue to govern accounts, devices, data, and recovery channels after an account holder dies, leaving survivors to manage digital remains through mechanisms built around a living user. We examine post-mortem digital governance as a sociotechnical problem of cooperative and contested work, focusing on who acts when the user is gone, what claims they make, and what barriers shape recovery, preservation, closure, and protection. We conducted a content analysis of $800$ Reddit posts about post-mortem digital privacy and security, coding posts across assets, actors, actions, privacy tensions, access barriers, policy gaps, emotional contexts, and risks. Findings show that phones/devices often act as gateways to other digital remains, socially connected actors make most claims, and data loss emerges as a central harm. We synthesize these findings into a Post-Mortem Digital Governance Framework for designing mechanisms that support survivor coordination while limiting access by purpose, asset, actor, and context.
Read the full abstract

Human-Computer Interaction

The Interviewer's Perspective: Unpacking the Impact of Real-Time AI Interviewing Assistance on Social Dynamics

Zhe Liu, Jiamin Dai and Joanna McGrenere

Eliciting rich data in semi-structured interviews is cognitively demanding, prompting recent work to explore real-time AI assistance for interviewers. However, introducing AI into the interviewer-interviewee interaction creates a triadic context whose social dynamics remain underexplored. We investigated how interviewers experience AI assistance for probing during semi-structured interviews. To elicit rich participant reflections, we implemented two variants of AI assistance differing in initiation and granularity in a high-fidelity prototype, ProbeAssist. We conducted a qualitative-first comparative structured observation study where 18 participants each completed three simulated interviews: one without AI and two with different AI variants. Findings showed that participants leveraged AI as a supportive tool but resisted it as an assessor or competitor. As they navigated AI's benefits and interaction costs, tensions emerged around agency, ownership, creativity, and interpersonal communication. We propose three implications for AI-assisted human-to-human interaction: managing social pressure, balancing idea alignment with inspiration, and preserving interpersonal presence.
Read the full abstract

Human-Computer Interaction

Beyond Driving: Envisioning Activities in Future Autonomous Vehicles through Experience-Centered Design

Keqi Chen, Xiao Xue, Xinyi Liu et al. (+4 more)

Autonomous vehicles (AVs) are poised to fundamentally alter personal transportation, offering occupants the freedom to engage in various non-driving-related activities (NDRAs). However, our current understanding of how people might actually use this time in fully autonomous vehicles (FAVs) is limited. Traditional research methods often struggle to capture the influence of diverse travel circumstances and purposes when exploring future scenarios. This paper introduces experience-centered design (ECD) as an approach to investigate potential NDRAs within FAVs by examining the intricate connections between individuals' daily routines, specific travel contexts, and the activities they might undertake in transit. Through a multi-phase study employing participatory techniques that facilitated narrative construction and exploration, including diary studies, scenario scripting, and mixed reality (MR) enactments, we enabled participants to ground speculative future scenarios in their own lived experiences. This process yielded nuanced insights into preferences and behaviors concerning potential NDRAs, alongside the underlying subjective meanings and sociotechnical considerations. Our findings lead us to conceptualize NDRAs not as isolated instances of "travel time use," but as dynamic sequences of interrelated activities deeply shaped by pre- and post-journey contexts. The effectiveness of our ECD approach in bridging current lived experiences with future scenarios was crucial for uncovering these insights. Ultimately, this study reconceptualizes AVs as complex sociotechnical systems that actively mediate human activity and interaction, suggesting a fundamental shift in their role within the urban fabric.
Read the full abstract

All authors Keqi Chen, Xiao Xue, Xinyi Liu, Runjia Tan, Chris Speed, Lee Kwan Min, Chen Lv

Human-Computer Interaction

Guardrails or Roadblocks? Effects of Pedagogical Style and Context Awareness in AI Teaching Assistants for Programming

Madeleine Eastwood, Harshith Narne, Joseph Hilby et al. (+3 more)

AI teaching assistants (AI TAs) backed by large language models (LLMs) and pedagogical guardrails are increasingly being integrated into programming courses, providing students with scalable access to hints, conceptual explanations, and code-level feedback. However, guardrails may also create friction. If students feel that the support provided is overly restrictive or poorly contextualized to their current progress, they may bypass approved tools for general-purpose LLMs. To investigate how AI TA design affects students' learning experiences, we conducted a randomized controlled trial with 132 students in an introductory programming course. Students completed three tasks related to code-writing and debugging and were randomly assigned to one of four AI TAs varied across two dimensions: pedagogical guidance style (Socratic vs. Direct instruction) and context awareness (no context vs. full context of the problem and student solution). We examined students' perceptions, interaction behaviors, and evidence of post-task comprehension. Students rated the Socratic AI TA with full context least favorably, reporting significantly lower perceived support for task completion. Descriptively, this condition also showed the highest observed interaction stress, the highest rate of external LLM use, and the lowest proportion of post-task explanations demonstrating full comprehension, though these differences were not statistically significant. These findings suggest that guardrailed AI TAs are not automatically better for learning. Instead, their effectiveness depends on how pedagogical guidance and contextual awareness are balanced in ways that students experience as useful, supportive, and worth continuing to use.
Read the full abstract

All authors Madeleine Eastwood, Harshith Narne, Joseph Hilby, Paul Denny, Ashish Aggarwal, Amanpreet Kapoor

Human-Computer Interaction

Will It Teach as Intended? How Teachers Configure Educational AI Chatbots

Bahare Riahi, Deniz Ozturk, Alice Guth et al. (+7 more)

Teachers are increasingly using generative AI to support instruction, yet it remains unclear how pedagogical intentions are translated into chatbot configurations and reflected in chatbot behavior. We studied a teacher-facing chatbot authoring tool in professional development workshops with 27 middle school teachers, analyzing focus-group interviews alongside configuration and interaction logs. Teachers envisioned chatbots as instructional scaffolds that could provide differentiated support, extend access to assistance, and preserve student thinking within teacher-defined boundaries. Configuration analysis showed that Purpose primarily captured instructional goals and content focus, whereas Rules more often specified pedagogical behavior, guardrails, and learner-specific adaptations. Log-based evaluation showed stronger alignment for responsiveness (88.9%) and persona (81.5%) than for rules (70.4%) and purpose (59.3%). These findings show that configurable controls alone do not ensure pedagogical fidelity and highlight the need for authoring tools that help teachers express, test, and refine intended chatbot behavior.
Read the full abstract

All authors Bahare Riahi, Deniz Ozturk, Alice Guth, Jiayu Li, Daksh Pratap Singh, Xiaoyi Tian, Jennifer Chiu, Nicholas Lytle, Tiffany Barnes, Veronica Catete

Human-Computer Interaction · Cross-listed from cs.LG

Improving Calibration of Black-Box Radiology AI Using Test-Time Augmentation

Nathan Le, Magdalini Paschali, Arogya Koirala et al. (+5 more)

Radiology AI systems increasingly inform clinical decisions such as triage, follow-up imaging, and treatment planning. For these decisions to be made safely, model outputs must be well calibrated, meaning predicted probabilities accurately reflect true risk. Many standard techniques for improving calibration, such as MC Dropout and Deep Ensembles, require access to model parameters or retraining. However, proprietary clinical AI systems operate as black boxes, preventing access to the model's internals. To that end, we propose a model-agnostic framework for improving calibration of black-box models using clinically grounded test-time augmentation (TTA). Our framework applies geometric and physics-inspired 3D CT perturbations and learns probability-level aggregation strategies without access to model internals or the original training data. Across pulmonary embolism and intracranial hemorrhage detection tasks, DualTTA achieved the strongest overall calibration among TTA methods, reducing the Expected Calibration Error by 54% (0.239 -> 0.109) and 43% (0.051 -> 0.029), respectively, while requiring only input-output access. Additionally, DualTTA outperformed uncertainty estimation techniques that require access to model internals, such as Temperature Scaling, MC Dropout, and Deep Ensembles, in most calibration metrics. These results demonstrate that learned TTA aggregation can improve the calibration of clinical AI systems, providing a practical approach for improving the reliability of black-box medical AI.
Read the full abstract

All authors Nathan Le, Magdalini Paschali, Arogya Koirala, Andrew Johnston, Zhongnan Fang, David B. Larson, Akshay S. Chaudhari, Camila Gonzalez

Human-Computer Interaction

Working with Agentic `Teammates': When a New Organizational Actor Collides with the Human Ecosystem of Work

Rida Qadri, Remi Denton, Michael Madaio et al. (+13 more)

Enterprise AI is transitioning from single-user, reactive tools toward proactive, multi-user 'teammates,' but our empirical understanding of this transition is limited. In this paper, we present an in-situ qualitative study of a persistent, proactive AI agent 'teammate' deployed across multiple teams in a large technology company. Our findings reveal the boundaries of the human-agent workplace are actively in flux, triggering breakdowns and negotiations across: 1) tacit rules of collaborative human workflows, 2) the relational boundaries of this new non-human actor, and 3) the redistribution of trust and human agency. We use these early micro-negotiations as signals to chart a new research, design, and organizational agenda that intentionally preserves human agency in a workplace shared with non-human organizational actors.
Read the full abstract

All authors Rida Qadri, Remi Denton, Michael Madaio, Mahima Pushkarna, Leslie Lai, Sherry Moore, Michelle Chen Huebscher, Andrew Butcher, Ritom Sen, Hsiao-Yu Tung, Shaan Mathur, Yimeng Liu, Shibl Mourad, Noah Fiedel, Edward Grefenstette, Michael Terry

Human-Computer Interaction · Cross-listed from cs.AI

PUBG Ally: A Conversational Embodied Agent as an AI Teammate

PUBG Ally Team: Irene Chen, Youngin Cho, Seungjun Chung et al. (+21 more)

We introduce PUBG Ally, an embodied agent for PUBG: BATTLEGROUNDS that can reason, act autonomously, and play alongside players as a voice-enabled teammate. Building such a teammate requires combining two difficult capabilities: it must perceive and respond to a constantly changing game world under strict latency constraints while interacting naturally with players, keeping its speech synchronized with its actions. Ally therefore combines agentic tool use with real-time game control. A language-model agent uses a controlled interface to inspect game information, interpret player speech, maintain context, decide what to say, and issue high-level action choices that steer a faster control layer for movement, combat, and recovery. Because the player's and Ally's speech and actions continually shape each other and the course of the match, training requires data from actual gameplay. We therefore collect data across nearly 39k sessions in which real players play alongside Ally, recording gameplay, player speech, agent decisions, tool use, actions, and player feedback, and use these records for iterative training. To evaluate teammate quality, we use player feedback and preference comparisons to identify gaps between offline evaluations and player preferences, and iteratively refine the evaluation criteria. Deploying Ally in live service further requires low-latency on-device execution and safeguards for player-facing communication, which we address through model compression, context compaction, targeted safety training, runtime guardrails, and memory redaction. During the live service, we surveyed players in 141 countries. Among respondents whose play with Ally was confirmed in game records, positive responses exceeded negative responses by 25.1 percentage points when asked whether they would recommend Ally, with players describing Ally not only as a tool but also as a teammate or companion.
Read the full abstract

All authors PUBG Ally Team: Irene Chen, Youngin Cho, Seungjun Chung, Jimin Hong, Hyeonbin Hwang, Hyeojung Im, Insub Im, Jaeseung Jeon, Seohyeon Jung, Beomsoo Kim, Byeongju Kim, Dohyun Kim, Dongwon Kim, Eunchong Kim, Hongmin Kim, Hyeonghwan Kim, Hyunseung Kim, Sungwoo Kim, Kangwook Lee, Minkyoung Park, Sue Hyun Park, Hyoseok Seol, Yujeong Son, Kiyoon Yoo

Human-Computer Interaction · Cross-listed from eess.AS

Voice Agents under Acoustic Stress: From Signal Degradation to Interaction and Action

Amir Ivry, Kai-Wei Chang, Lin Zhang et al. (+2 more)

Voice agents must complete users' tasks despite noise, reverberation, and competing speech. Evaluating agents' robustness therefore requires following how acoustic conditions affect the conversation and the actions taken on the user's behalf. This overview examines what existing benchmarks reveal about agents' ability to complete tasks under acoustic stress and where further task-based evaluation is required. We then introduce TRACE, a practical workflow for designing, running, and interpreting evaluations of acoustic robustness in task-oriented human-agent interactions: the same agent attempts a specified task with an original recording and an acoustically stressed copy, and the resulting conversations are scored for task completion, wrong actions, recovery, and user effort. Finally, we explain how results from these evaluations can guide changes to an agent to prevent wrong actions and improve recovery.
Read the full abstract

All authors Amir Ivry, Kai-Wei Chang, Lin Zhang, Sharon Gannot, Carlos Busso

Human-Computer Interaction

Calibrating LLM Judges for Human and AI Conversations

Maike Züfle, Patrícia Schmidtová, Vilém Zouhar et al. (+8 more)

Measuring how successful a conversation is remains difficult, even for humans judging spoken dialogue. We evaluate state-of-the-art LLMs as pointwise and pairwise judges of conversational success on CANDOR, finding pointwise scoring correlates moderately with human ratings, while pairwise comparison suffers from long transcripts and positional bias. Since this leaves judge scores incomparable across models, we propose a small anchor set and a calibration function that calibrates any judge onto a shared, interpretable scale. We further release the Voice Arena Goal Dataset (VA), 200 task-oriented human-AI and human-agent conversations with pairwise annotations, revealing a substantial gap between current judges and human-level discrimination. Using VA, we test whether CANDOR-fitted calibration transfers to human-AI conversations, finding it brings judges onto a shared scale despite never observing VA during fitting.
Read the full abstract

All authors Maike Züfle, Patrícia Schmidtová, Vilém Zouhar, Shree Harsha Bokkahalli Satish, Erica Cooper, Shobhit Banga, Vaibhav Nalawade, Manmeet Kaur, Jan Niehues, Markus Müller, Ondřej Klejch

Human-Computer Interaction · Cross-listed from cs.CL

Controlling Backchannels in Streamable Full-duplex Models

Maike Züfle, Peter Polák, Sefik Emre Eskimez et al. (+3 more)

Backchannels, brief acknowledgements like "uh-huh" produced while the other party may still be talking, are central to natural conversation, but full-duplex spoken dialogue models rarely model them explicitly. We introduce a lightweight backchannel head that predicts, from a full-duplex model's own hidden states, when a backchannel should begin. Once this probability crosses a tunable threshold, a backchannel is force-decoded. Attached to both a 7B (PersonaPlex) and a 1B (F-Actor) model, it generalizes across scale. Probing confirms the hidden states anticipate real human timing, and generation evaluation shows more frequent, better-timed backchannels. Human raters judge the resulting backchannels on par with real ones.
Read the full abstract

All authors Maike Züfle, Peter Polák, Sefik Emre Eskimez, Jan Niehues, Peter Bell, Ondřej Klejch

Human-Computer Interaction · Cross-listed from cs.AI

When No One Owns the Judgment: Accountability Under Contribution Dissolution in Human-AI Collaboration

Hengzhi Ye

Communities often respond to potentially AI-assisted work by asking three questions: Was AI used? Was that use disclosed? Can hidden use be detected? These questions place AI use itself at the center of accountability while overlooking a deeper problem: unowned judgment. Evaluations, claims, decisions, and creative directions can be shaped by AI with no accountable human or institution prepared to stand behind them. We develop this argument through two illustrative cases: AI-assisted peer review and concealed AI use in creative work. The first shows how contribution dissolution can weaken responsibility while the second shows how the fear of losing credit can discourage honest disclosure. The cases expose the limits of disclosure rules and provenance records as responses to AI-mediated collaboration. We offer three directions for discussion: distinguishing the roles AI plays, identifying judgments that require clear human ownership, and creating conditions in which AI involvement can be disclosed without default penalty. The broader aim is to make AI-shaped contributions discussable, creditable, contestable, and repairable.
Read the full abstract

Human-Computer Interaction

DocuTeam: Mixed-Initiative Multi-Agent Discussions around Evolving Documents

Heechan Lee, Juhyeon Choi, Tae Soo Kim et al. (+2 more)

In open-ended problem solving, collaborators often rely on discussion to surface concerns, challenge perspectives, and refine shared work as it evolves. While AI agents are increasingly used as discussion partners, existing multi-agent systems place a heavy burden on users to initiate and carefully orchestrate the discussions. We present DocuTeam, a mixed-initiative multi-agent discussion system in which both users and agents can initiate and steer conversations. Agents monitor document changes to proactively start and redirect discussions as the work evolves, while users can flexibly shape the conversation or adopt agent ideas. In a within-subjects study (N=20), participants using DocuTeam produced outcomes rated significantly more novel, relevant, and specific than with a baseline without any increase in cognitive load. Rather than using agents for one-off idea sourcing, participants engaged in an iterative refinement loop in which document changes prompted agent reactions, which led users to revisit and further develop their work.
Read the full abstract

All authors Heechan Lee, Juhyeon Choi, Tae Soo Kim, Juho Kim, Joseph Seering

Human-Computer Interaction

A Human-Like Pedestrian Model for Automated Driving Simulations

Ruofeng Wang, Patrick Ebel, Philipp Wintersberger et al. (+1 more)

Automated vehicles must be able to interact with pedestrians safely and efficiently across diverse traffic situations. Although driving simulators offer a scalable testbed for learning such capabilities, existing theory-inspired pedestrian models are narrow in scope and limited to go/no-go crossing decisions in single-lane settings. While data-driven approaches can predict pedestrian behavior in complex situations, they lack sufficient observations in rare, safety-critical scenarios. Here, we propose an approach to training pedestrian models in simulators so that learned policies generate demonstrably human-like behavior in realistic, complex traffic scenarios, including multiple lanes, heavy traffic, and dangerous driving styles. Our technical contribution is a novel definition of pedestrian-vehicle interaction as a partially observable Markov decision process (POMDP) with theory-grounded perceptual, cognitive, and motor constraints. It accounts for the highly adaptive nature of human behavior in traffic and simulates how people adjust their responses according to perceived danger, time pressure, and the complexity of the situation. When trained via deep reinforcement learning (RL) with domain randomization in a simulator, the model reproduces the broadest range of empirical findings shown so far on human crossing behavior, including gap acceptance, yielding acceptance, hesitation, and evasive speed adjustment. We show that learned policies transfer to unseen traffic environments, and can be further adapted to local traffic norms with finetuning. Together, these results establish a blueprint for simulator-ready pedestrian models that can support the development and evaluation of automated driving systems.
Read the full abstract

All authors Ruofeng Wang, Patrick Ebel, Philipp Wintersberger, Antti Oulasvirta

Human-Computer Interaction

Through Human Eyes and Machine Eyes: Understanding View Mismatch in Video See-Through Extended Reality

Yanming Xiu

Video see-through extended reality (VST XR) systems commonly use headset screenshots or captured frames as proxies for the user's first-person visual context. However, the system-captured view and the user's effective visible field do not necessarily coincide: a screenshot records a rectangular machine-readable frame, whereas the user's effective visible region can be more constrained and non-rectangular. This paper studies this human-system view mismatch in VST XR. We formalize the relationship between the system-captured region and the human-visible region by defining their co-visible, system-only, and human-only regions. \rev{We then conduct a pilot-level boundary measurement on Meta Quest 3, revealing a clear mismatch between the rectangular screenshot frame and the approximate human-visible boundary. Building on this model, we analyze how view mismatch can affect screenshot-based XR sensing and downstream vision-language model tasks. Through four representative case studies, we illustrate potential risks and failure modes including prompt injection, privacy leakage, human-invisible information bias, and missing human-visible information. Our results show that view mismatch is not only a geometric artifact, but can also introduce security, privacy, and reliability concerns for AI-integrated VST XR systems.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CY

AI-Moderated Interviews for Market Research and Digital Twins Calibration

Yuting Deng, Jingxuan Liu, Olivier Toubia et al. (+1 more)

AI-moderated interviews are emerging as a scalable market-research method for generating consumer insights and building consumer "digital twins." Yet it remains unclear whether they match human-moderated interviews or improve on simpler, static data collection methods. In a pre-registered, between-subjects study (N = 317) with three industry partners, we compare AI-moderated (N = 139), human-moderated (N = 24), and static interviews (N = 154). AI moderation matches human moderation in depth, covers more themes, and, holding budget constant, recovers significantly more customer needs than human moderation or static interviews. However, participants sound more emotionally engaged when speaking to a live human. We then create digital twins using interview data and evaluate each twin against the participant's own held-out responses to six real-world marketing stimuli. We find that digital twins created from AI-moderated interviews predict consumer responses better than demographics-only personas. However, the additional richness from AI moderation does not translate into better quantitative predictions compared to static interviews. By analyzing open-ended thoughts generated from humans versus their twins, we find that prediction errors are connected both to differences in (self-reported) thinking styles between twins and humans, and to gaps between training and validation data (i.e., asking questions that are too far out of distribution).
Read the full abstract

All authors Yuting Deng, Jingxuan Liu, Olivier Toubia, Naman Jain

Human-Computer Interaction

A Procedure for Classifying Attachments and Affective Social Bonds in Human-Robot Dyads

Imran Khan and Elisabeth Blagrove

Human-robot interaction (HRI) claims that people form attachments and social bonds with artificial agents, yet the terms are often applied without the behavioural and physiological criteria that give them content in their source disciplines. Without this empirical grounding, studies deploy widely divergent methods, frequently producing expansive relational claims that far outstrip their underlying evidence. To address this, we propose a standardised four-question procedure, grounded in criteria established in the developmental, ethological, and neuroendocrine literatures, that classifies a given human-robot tie as an attachment, an affective social bond, or no relationship, with intermediate classifications when evidence is incomplete. We specify minimum evidential requirements for each question, and provide candidate HRI study designs, adapted from validated human-human, human-animal, and animal-animal paradigms. We then demonstrate the procedure by applying it to a representative set of published HRI studies, showing how often relational claims outstrip what the reported designs can establish. Finally, we discuss the ethical and regulatory burdens created when artificial agents engage human biobehavioural systems. By replacing the divergent operationalisations with a unified, criterion-based classification, this paper gives HRI practitioners a standardised basis for evaluating, classifying, and comparing human-robot relationships, and sets out the experimental rigour that each classification demands. We therefore call on researchers of human-robot relationships to adopt such rigour, or to consider alternative terminology in their descriptions of these ties.
Read the full abstract