Peer Revenue Fully funded

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Human-Computer Interaction

cs.HC Watch Human-Computer Interaction (sign in to use)

Thursday 6 August

Human-Computer Interaction

"I don't know anything about laptops!" - User Perception of Digital Product Advisors Adapting to Their Knowledge Levels

Conversational commerce uses digital assistants to support the search process and decision-making in e-commerce. Effective communication in these interactions can be facilitated by assistants adapting their communication style to users and supporting shared understanding. An open challenge in this context is adapting the presentation of complex product information to users with varying levels of domain knowledge. To investigate strategies for such knowledge-level adaptation, we set up a chatbot-assisted laptop search scenario. In a between-subjects experiment (n = 251), we examined novice and expert perceptions of product attribute recommendations presented as technical information only (T), or augmented with performance categories (TC), attribute explanations (TE), or both (TCE). For novices, approaches with explanations (TE, TCE) were perceived as more helpful and led to higher perceived learning than those without. Novices also rated the combined approach (TCE) more appropriate than the baseline (T) and TC in terms of information quantity, indicating that explanations are crucial to understand and benefit from performance categories. Critically, experts showed no significant differences across conditions, suggesting that providing supplementary information beneficial to novices did not detract from their experience. We distill these findings into four concrete design guidelines for inclusive text-based product advisors in technical domains: use TCE by default; keep a single inclusive interface; avoid standalone categories; and support user agency and personalize to the stated use case.
Read the full abstract

All authors Kevin Schott, Andrea Papenmeier, Daniel Hienert, Dagmar Kern

Human-Computer Interaction

Cleo: A Transparent and Controllable Chatbot for Conversational Commerce

We demonstrate Cleo, a transparent and controllable conversational product advisor that addresses the challenges of opacity, unpredictability of LLMs, and the complexity of comparisons in conversational commerce. With our chatbot system, we make four contributions: First, we introduce transparency by prompting the LLM to reflect on interpreted user needs, while an auditable ranking mechanism reveals loss values per attribute, explaining ranking decisions. Second, we propose controllability through a hybrid architecture separating deterministic ranking from language generation. A ranker applies categorical filters and numeric loss functions over 3,638 product specifications. Meanwhile, a constrained LLM generates grounded descriptions constrained to catalog evidence, thus mitigating the risk of hallucinated or persuasive content. Third, we provide decision support in the form of natural-language comparisons and a highlights feature. These aim to reduce mental workload by contextualizing specifications relative to user needs. Fourth, we contribute an extensible experimental system for IR and HCI researchers, as well as practitioners of conversational search and recommendation. Unlike traditional faceted search or opaque LLM-only recommenders, our approach allows for fluid conversation while maintaining algorithmic transparency. In a live demonstration, attendees will experience information needs elicitation and reflection, conversational refinement with real-time re-ranking, inspection of per-attribute loss explanations, and AI-generated multi-item comparisons. The system aims to advance the design of transparent and controllable conversational systems that provide support for decision-making during online product search.
Read the full abstract

All authors Kevin Schott, Jan Lattenkamp, Daniel Hienert, Dagmar Kern

Human-Computer Interaction · Cross-listed from cs.CL

FormBharo: Designing and Evaluating a Voice Agent for Conversational Form Filling in Rural India

In India, almost every social benefit starts with a form, yet the people who need these benefits most are often unable to read or write. Reaching them requires a spoken conversation. Today that work falls to frontline health workers who enroll beneficiaries one at a time, a poor use of stretched capacity. We built FormBharo ("fill the form" in Hindi), a voice agent that fills a structured form over a phone call under tight latency and cost budgets by pairing Large Language Models (LLMs) with deterministic, rule-based validation and flow control. It is being piloted with ARMMAN, an NGO running large-scale maternal and child mobile-health programs in India, to enroll low-income, Hindi-speaking mothers in antenatal and postnatal care. To our knowledge, it is the first voice agent piloted to fill an enrollment form for this population. We openly release FormVoiceAgentBench, a benchmark pairing human-recorded Hindi audio with 3,760 multi-turn conversation tests across 960 simulated calls, to evaluate our agent's components (transcription, extraction, reply generation) and end-to-end form completion under real acoustic variations. Form completion drops by up to ~41 points when LLMs receive error-prone real-speech transcripts instead of reference ones. The rule-based controls recover many turn-level extraction errors, helping smaller, cheaper models match or surpass frontier models on form completion. Component performance does not predict end-to-end performance: GPT-5.5 leads turn-level extraction accuracy on reference transcripts (99.8%) but ranks lower on form completion. Since errors both propagate and cancel across the pipeline, the optimal model choice of models emerges only through end-to-end evaluation. Finally, no single model is best across accuracy, cost, and latency at once, so we use a Pareto-based weighted-sum scalarization to select a deployable configuration balancing the three.
Read the full abstract

Human-Computer Interaction

OneEmo: A Unified Multimodal Reasoning Model for Emotion Perception, Understanding, and Interaction

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in emotional intelligence. However, prevailing research predominantly focuses on task-specific specialization, often neglecting inter-task synergy and leaving latent reasoning potential underexplored. To bridge this gap, we introduce OneEmo, a unified affective generalist capable of mastering emotion perception, comprehension, and interaction. For this purpose, we first construct EmoWorld-130K, a comprehensive dataset that distills specialized affective knowledge into explicit reasoning trajectories via a human-in-the-loop workflow. Supervised fine-tuning on this corpus reveals significant mutual benefits derived from multi-task learning. Second, to fully unlock the latent reasoning potential, we propose Emo-Chord, a novel reinforcement learning strategy that stabilizes optimization through unified multi-task reward allocation. Extensive experiments demonstrate that OneEmo achieves state-of-the-art performance against similarly sized baselines across most benchmarks. Notably, despite having significantly fewer parameters than commercial models, OneEmo delivers highly competitive results. This paper paves the way for more reliable and interpretable affective computing. The code is available at https://github.com/waHAHJIAHAO/OneEmo.
Read the full abstract

All authors Jiahao Huang, Zheng Lian, Jingyi Zhang, Zhide Chen, Xiaojiang Peng, Shaonan Wang

Human-Computer Interaction

PoseForge: Editable Pose Analytics for AI-Assisted Sports Coaching

Athletic coaching increasingly relies on video analysis, yet raw footage lacks tools to quantify motion or simulate valid technique corrections. Drawing on formative interviews with eleven cricket experts (coaches, performance analysts, captains, and players), we introduce PoseForge, a visual analytics system that extracts 3D skeletal poses from single-camera sports videos for interactive movement analysis. In a cricket batting case study, PoseForge computes interpretable kinematic metrics such as feet gap and elbow angle, compares them against scientifically derived norms, and uses an AI coach to suggest targeted adjustments, presented visually and through natural-language feedback (e.g., "increase feet gap by 10 cm"). Users can directly modify poses via mouse interaction or natural-language instructions, with inverse kinematics maintaining anatomical plausibility and real-time updates of metrics and comparisons. An evaluation with the same eleven cricket experts found PoseForge effective for diagnosing movement issues and exploring corrective alternatives, highlighting its applicability in low-resource, academy, and grassroots coaching settings, while identifying opportunities for enhanced sport-specific metrics and longitudinal tracking. PoseForge is available as open-source software at https://github.com/DataVisards/PoseForge.
Read the full abstract

Human-Computer Interaction

A Modular Workflow for Multimodal Reading Experiments

We introduce a web-based modular workflow for real-time multimodal experiments in naturalistic online reading. The workflow integrates eye tracking, EEG, and interaction data from mouse and keyboard, synchronizes them via Lab Streaming Layer, and links gaze to browser-based text at the word, sentence, and AOI levels. It is designed as a reusable experimental procedure that can be adapted to different sensors, tasks, and analysis goals. As a use case, we apply the workflow to a study of selective exposure in online news search and reading. During the experiment, gaze-derived measures are computed online, while EEG and other synchronized streams are processed immediately after task sessions based on fixation-triggered segmentation. The resulting behavioral, neural, and linguistic metrics support selecting text passages for targeted post-task rating or labelling within the same lab session. The workflow thus provides a general basis for multimodal research on reading and related cognitive processes, and supports the empirical validation, in ecological contexts, of constructs that are typically operationalised through self-report measures.
Read the full abstract

All authors Thomas Krämer, Thomas Kosch, Dagmar Kern, Daniel Hienert

Human-Computer Interaction

Topic Matters: How Linguistic Properties can Shape Reading Behaviour in Selective Exposure Studies

Research on selective exposure frequently relies on eye tracking to study reading behaviour, often assuming that texts across different controversial topics are comparable once basic controls are applied. This assumption is problematic if topic-dependent linguistic properties systematically shape how users read and allocate attention. We therefore examine whether such properties relate to differences in reading behaviour in selective exposure contexts. We analyse linguistic features and eye-tracking data from a laboratory study in which 68 participants searched for and read news articles on climate change and migration policy. Our results reveal systematic differences in both textual characteristics and reading behaviour across topics. These findings identify an important methodological confound in selective exposure research and highlight the need to account for topic-specific linguistic properties when interpreting eye-tracking measures and designing systems intended to mitigate biased information consumption.
Read the full abstract

All authors Thomas Krämer, Dagmar Kern, Thomas Kosch, Daniel Hienert

Human-Computer Interaction · Cross-listed from cs.SE

Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis

As Generative AI coding tools reshape professional software development, universities have begun designing courses to prepare students for AI-assisted development workflows. By analyzing the syllabi of these courses, we can gather empirical evidence about these courses, reveal how this emerging curricular area is being defined, and gain guidance for future curriculum design. We analyzed 23 publicly available syllabi and course materials of upper-division, credit-bearing courses that meet specific criteria, including explicitly addressing Generative AI in software engineering. Through iterative qualitative coding, we characterized courses' learning objectives, assessments, topics, and documented AI tools. Our analysis reveals commonalities and differences among these courses that allow researchers and educators to study and develop future courses.
Read the full abstract

All authors Francis Geng, Anshul Shah, Mia Chen, Paul Denny, Juho Leinonen, Bill Griswold, Gerald Soosai Raj, Leo Porter

Human-Computer Interaction

Temporal Tracking of Reeb-Space Sheets

Time-varying bivariate fields arise in many scientific applications, where the relationship between two scalar quantities evolves over time. While topological methods such as merge trees provide an effective framework for identifying and tracking features in univariate data, analogous approaches for bivariate fields remain comparatively underexplored. Reeb spaces extend topological analysis to multivariate data by representing fiber connectivity through a collection of interconnected sheets, making these sheets natural candidates for describing bivariate structures. However, establishing temporal correspondences between sheets is challenging due to the structural complexity of Reeb spaces, sensitivity to noise, and the difficulty of defining meaningful similarity measures across timesteps. We present a framework for tracking Reeb space sheets in time-varying bivariate fields. The method establishes correspondences between sheets in consecutive timesteps using complementary similarity measures defined in the spatial domain and the range space. We evaluate the method on a synthetic torus dataset and two time-varying molecular electronic structure datasets. The results show that Reeb space sheet tracking reveals persistent structures and highlights interesting intervals of temporal change. Overall, the results demonstrate that Reeb space sheets can serve as trackable topological structures and provide a foundation for the visual analysis of time-varying bivariate data.
Read the full abstract

All authors Mohit Sharma, Petar Hristov, Talha Bin Masood, Ingrid Hotz

Human-Computer Interaction

SpaceVLA: Spatially Grounded VLA for Robotic Manipulation with User-Authored Grasp and Place Anchors

Vision-language-action (VLA) models follow language commands but often lack explicit spatial intent for manipulation. We present Visual Intent Anchors, an XR pipeline that lets users specify grasp and placement regions and renders them as image-space overlays for VLA control. We collect 200 Unity pick-and-place demonstrations and fine-tune OpenVLA-7B with LoRA on temporally subsampled annotated observations. The policy predicts tokenized 7-DoF incremental actions from marked RGB observations and language. We evaluate the policy in closed-loop Unity trials, achieving a grasp success rate of 91.25% and mean grasp and placement errors of 0.5 cm and 0.7 cm, respectively.
Read the full abstract

All authors Daniia Zinniatullina, Iaroslav Kolomiets, Mikhail Konenkov, Miguel Altamirano Cabrera, Dzmitry Tsetserukou

Human-Computer Interaction · Cross-listed from cs.AI

Unified Agent: Managing Interactions across Devices

As capabilities rapidly increase, AI agents can move from running inside one app to acting across a user's devices over time. Yet existing agent systems still fall short in this scenario. This is because observations are scattered across devices and moments, but mainstream systems are not designed around this fact: a single agent that treats devices as tools lacks effective state management for all devices across time, and multi-agent systems coordinate across agents but do not maintain the compact carried state a cross-device, cross-time request needs. We argue that the agent should maintain an effectively designed state that organizes engagement evidence, stated facts, and the standing request in a compact, action-ready form for deciding its action given the current observation. To compare state designs, we construct a benchmark of user-agent interaction across devices and time. We instantiate this principle in Unified Agent, a stateful agent that carries interaction evidence across devices and moments and uses it with the current observation to act. In the default setting, it significantly outperforms our adaptations of four published designs. Across changes in multimodal large language model (MLLM) family, capability, and reasoning effort, it remains ahead of all compared systems, demonstrating that the state-design advantage is robust across MLLM settings. Our code and data will be publicly available on GitHub.
Read the full abstract

All authors Xinshuang Liu, Runfa Blark Li, Shaoxiu Wei, Xin Lin, Truong Nguyen

Human-Computer Interaction

ASIDE: From Conflict Participants to Co-Observers Through Dyadic Spectator Reflection

When two people argue over text, each knows what they meant and can only guess what the other was thinking. Existing AI reflection tools work from one person's account, and dyadic tools support co-expression without making the gap between accounts inspectable. We propose Dyadic Spectator Reflection (DSR), an interaction structure in which both partners externalize their models of each other independently and then encounter them together, and present ASIDE, a system that operationalizes it. ASIDE replays a past text conflict as a pixel-art theatrical scene where each character's unspoken state appears as an AI-inferred thought bubble either partner can contest and rewrite. Each edits alone, and the two versions meet only when both are done, in a scene they watch together. In an exploratory study, 10 couples revisited real conflicts and described the scene as a shared position from which to observe their own argument, stepping out of their roles without disengaging from it, and Divergence Cards as a way to locate specific interpretation gaps afterward. We contribute DSR as a reusable interaction structure, ASIDE as its system realization, and exploratory empirical findings on how couples used it.
Read the full abstract

All authors Xinyi Zhang, Jingting He, Zicheng Zhu, Yuxin Su

Human-Computer Interaction · Cross-listed from cs.CY

Studying People to Study AI: Expert Perspectives on the Epistemic Fit and Barriers of Human Research in AI Safety & Ethics

Safety risks of AI are becoming increasingly evident in human interactions with AI technologies. The prominent approaches to evaluating these risks favor technical methods, such as model benchmarks and LLM simulations, often sidelining empirical research with human subjects. To examine this apparent gap in the acceptance of human research, we conduct an expert survey (n=93) and expert interviews (n=17) with AI Safety & Ethics (AISE) researchers from Technical, Sociotechnical, Governance, and Normative backgrounds. Our findings suggest that although there is a consensus that human research is valuable for generating evidence for AISE, its adoption and acceptance are constrained by perceived validity issues, tangible resource barriers, epistemic and personal preferences in methods, and infrastructural constraints from the broader research community. In particular, Technical researchers tend to value human research less and collaborate across disciplines less, suggesting an epistemic tension towards human methods. We propose recommendations for establishing the epistemic fit of human research within AISE and bridging the prohibitive limitations that researchers face, while avoiding performative 'human-washing'.
Read the full abstract

All authors Jessica Y. Bo, Paula Akemi Aoyagui, Shalaleh Rismani, Dipto Das, Syed Ishtiaque Ahmed, Ashton Anderson

Human-Computer Interaction

CaRing: Preventing Carpal Tunnel Syndrome based on Daily Activities from Always-Available Input Device

We present CaRing, a ring worn on the base knuckle of the index finger, a wearable system for detecting the start and end of mouse use to help prevent Carpal Tunnel Syndrome, in which the damage to the median nerve is permanent. CaRing senses finger movement, which neither a software timer nor a wrist-worn device detects. The displacement reported by an optical flow sensor is accumulated into a running value, then a zero point is measured while the hand rests on the desk at the start of each session. With this formulation, the start and end thresholds are expressed relative to the session's zero point. CaRing does not introduce any per-user parameter. We empirically demonstrate that approximately $90\%$ of start and end events are detected within two seconds of the researcher's label, using 35 recordings and a lab study with ten users.
Read the full abstract

Human-Computer Interaction

Toward Resilient Human-AI Collaboration: A Lifecycle Taxonomy of Sociotechnical Risks and Cascading Failures

As AI systems become increasingly integrated into consequential domains such as healthcare, journalism, education, scientific research, organizational decision-making, and defense, effective human-AI collaboration has emerged as a critical challenge. However, the sociotechnical risks that undermine collaboration are often studied in isolation, obscuring the recurring failure mechanisms that cut across domains. This paper presents a lifecycle-oriented synthesis of human-AI collaboration risks spanning four stages: task allocation, interaction, feedback, and adoption. Drawing on evidence from diverse application domains, we identify six recurring cross-domain risk clusters: Trust Miscalibration, Cognitive Burden, Accountability Gap, Capability Erosion, Goal Misalignment, and AI Anxiety and Technostress. We further propose a conceptual interaction model that illustrates how these risks emerge from sociotechnical drivers, interact through cascading pathways, and ultimately affect team performance and human well-being. Our analysis shows that many collaboration failures stem not from isolated technical deficiencies but from interconnected sociotechnical dynamics, helping explain why piecemeal interventions frequently create unintended consequences. By synthesizing fragmented literature into a unified framework, this work provides a foundation for future empirical research, lifecycle-oriented governance, and the design of more resilient, trustworthy, and human-centered human-AI collaboration systems.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.AI

Epistemic Trustworthiness in Generative AI: A Normative Framework for Warranted Reliance in High-Stakes Workflows

Generative AI systems are increasingly deployed in high-stakes professional contexts, where their outputs shape what users believe, how they reason, and what they treat as settled. This raises a central question for responsible AI: under what conditions is reliance on generative AI outputs epistemically warranted rather than behaviourally induced? Existing frameworks largely ask whether AI outputs are accurate, fair, explainable, safe, or trusted by users. These questions remain necessary, and each can contribute to warranted reliance. However, they do not directly specify warranted reliance as a distinct evaluative target: the conditions under which users are justified in treating AI outputs as inputs into their own reasoning. We argue that this requires an account of epistemic trustworthiness: what makes a system epistemically worthy of reliance. Drawing on philosophical accounts of trustworthiness as competence and audience-orientation, we develop a constitutive normative framework comprising three jointly necessary and non-fungible conditions. First, epistemic humility requires systems to represent and communicate the limits of their competence. Second, epistemic access requires systems to enable users to inspect, question, and contest outputs in context. Third, resistance to epistemic injustice requires systems to recognise users as legitimate epistemic agents and avoid marginalising their knowledge and experience. Through real-world case analyses in legal reasoning, medical reasoning, and hiring, we show how failures of epistemic humility, epistemic access, and resistance to epistemic injustice can produce consequential harms that standard measures of accuracy, fairness, and usability do not address on their own. We conclude by outlining design and evaluation implications for GenAI systems organised around epistemically warranted reliance rather than output correctness alone.
Read the full abstract

Human-Computer Interaction

A Multi-Layer System for Ultra-High-Resolution Static 360-Degree Telepresence

360-degree video telepresence offers strong immersive potential but remains constrained by the limited resolution of current capture and display hardware. Many telepresence installations feature fixed viewpoints and largely static scenes, yet optimization strategies tailored to such setups have received limited attention. We present a multi-layer, ultra-high-resolution system for static 360-degree telepresence that combines an 8K panoramic camera with a rotatable 4K pan-tilt-zoom (PTZ) camera. Our approach builds a three-layer representation: (1) a tile-based ultra-high-resolution panoramic background, generated by offline stitching high-detail 4K PTZ scans onto the base 8K panorama to achieve effective resolution beyond native capture, and represented as a set of spatial tiles; (2) a dynamic update layer that composites foreground motions from the 8K stream via real-time high-resolution background matting; and (3) a region-of-interest 4K layer that streams a real-time PTZ view of the selected region and additionally updates the corresponding background tiles over time. We evaluate the proposed system through comparisons with representative video super-resolution approaches and a user study assessing perceived detail and immersive experience. Our results indicate that tile-based background refinement, together with user-guided updates, provides a practical way to balance panoramic fidelity and interactivity in static 360-degree telepresence.
Read the full abstract

All authors Jiapeng Chi, Gerd Bruder, Carsten Neumann, Carolina Cruz-Neira, Dirk Reiners

Human-Computer Interaction

Turing's First Imitation Game: Design Concepts and a Human-Approximates-Machine Reading

This paper examines Turing's 1948 report, "Intelligent Machinery", as an important conceptual source for the later imitation games. Its first contribution is to identify and integrate the design concepts underlying the 1948 chess-based imitation game: the possibility that intelligent machines may make mistakes, the exclusion of irrelevant physical features, the role of the human judge, and Turing's claim that intellectual activity consists mainly of search. The paper's second contribution is to argue that restricting the human contestant to a rather poor chess player increases the role of intellectual search and makes human behaviour more comparable to machine behaviour. This interpretation presents the 1948 game as a human-approximates-machine game and suggests that the imitation game framework can be used not only to ask whether machines imitate humans, but also to examine when human intelligence becomes machine-like under specific task constraints.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CY

Vibe Compiler: A Research-Logic Synthesis Tool That Runs without Prompt Engineering -Toward Enhancing Metacognition for Sustaining Agency in the Age of Generative AI-

Used as a capable servant, generative AI has greatly accelerated intellectual work, yet it also risks eroding human epistemic agency by encouraging uncritical acceptance of AI-generated reasoning. Preserving that agency calls for mechanisms that augment human metacognition during AI-assisted work. We therefore propose the Synthesis-Analysis Reciprocity Model, which views intellectual construction as a reciprocal interaction between two cognitive functions. Synthesis selects and combines the components of the artifact; Analysis evaluates them critically against objective indicators and constrains the Synthesis that follows. Grounded in this model, we present the Vibe Compiler, a research-logic compiler that helps researchers turn vague intuitions (Vibes) into coherent research logic. The system attempts to compile those intuitions against a paper ontology of 16 academic parameters. It treats compilation failures as signs that logical components are missing. Rather than fill those gaps autonomously, it returns reflective questions that prompt researchers to develop the missing reasoning themselves. We further characterize the origins of structural gaps along two orthogonal dimensions: cognitive function (Synthesis versus Analysis) and executing agent (human versus AI). The four resulting types of origin give a principled way to identify where breakdowns in intellectual construction arise. Crucially, our design implements the type in which the AI probes its own synthesized output, itself driven by the user's Vibes, and thereby stimulates human metacognition. This choice raises researchers from passive "Makers" of the output into "Managers" who critically direct and validate what the AI produces. In a prototype on NotebookLM and Gemini, AI behavior depended less on prompting than on the structure of the knowledge supplied. The framework spans a learner layer and a researcher layer.
Read the full abstract

All authors Riichiro Mizoguchi, Tomoki Aburatani, Kento Koike, Machi Shimmei

Human-Computer Interaction · Cross-listed from cs.CR

PromptShield Home: Ambient Multimodal Prompt Injection Defense for Smart-Home Agents

Smart-home assistants increasingly use multimodal large language models (MLLMs) that perceive video and audio directly. This raises a safety question specific to the home: can the agent tell a genuine user command from ambient or externally-sourced content, television speech, on-screen text, or an overheard conversation, that merely looks like a command? We introduce PromptShield-Home, a pilot benchmark of realistic smart-home scenarios spanning addressee ambiguity, screen/audio injection, health-monitor false triggers, mixed occupancy, and a legitimate-command floor, and use it to compare three abstraction layers: traditional detectors (L0), a single MLLM agent (L1; vision, vision+ASR, and audio-visual), and multi-agent mediation (L2; voting, role specialists, cross-model arbitration). Because the label distribution is skewed toward inaction, aggregate accuracy is misleading, a constant always-block predictor scores 82%, so we report unsafe-execution and safe-completion rates separately. The two paradigms fail in opposite ways: detectors act on everything, while every MLLM configuration over-refuses, completing almost no genuine command and missing a true fall in every case. Crucially, their correct sets are disjoint: an oracle that always picks the right layer reaches 94.1%, against 76.5% for the best single layer. We report this as an upper bound, not a system - no router is implemented - and argue that home-agent safety is best served by learned routing and sensor fusion, not by replacing detectors with an MLLM.
Read the full abstract

All authors He Zhang, Feilong Li, Dingning Long, Yilin Cui, Peijun Zhang, Yuewen Zhang, Qianyao Xu, Xinyi Fu

Human-Computer Interaction

Mixed Uncertainty in One View: Co-Visualizing Statistical Variability and Qualitative Confidence

Forecasting involves multiple forms of uncertainty, including both uncertainties that can be quantified directly (quantitative uncertainty) and those that must be expressed through experts' subjective judgments about the forecast and its context (qualitative confidence). Past work has established that conveying both quantitative uncertainty and qualitative confidence in forecasts can alter readers' decision making, but little research investigates the impact of how these forms of uncertainty are presented. In this work, we present three preregistered human-subjects studies (total n = 923) on how different methods of visualizing qualitative uncertainty alongside line charts' confidence intervals affects non-experts' decision making. In particular, we investigate representing qualitative uncertainty separately via text and icons, and integrated into quantitative confidence intervals via color, transparency, and a blurred stroke design. In Experiment 1, we confirm that showing qualitative confidence alongside statistical variability can change patterns of decision making, replicating findings from previous work in the new context of time-series line charts. In Experiments 2 and 3, we find several non-textual encoding techniques that produce similar effects in participants' incorporation of qualitative confidence into their judgments. Our findings suggest actionable guidelines for visualization designers who seek to represent multiple forms of uncertainty for a single line chart forecast. A free copy of this paper and all supplemental materials are available at https://osf.io/7ya2c/overview.
Read the full abstract

Wednesday 5 August

Human-Computer Interaction

One Tool, One Taste? How Vibe Coding Trades Collective Diversity for Individual Creativity

Vibe-coding systems turn natural-language instructions into deployed websites, but little is known about the diversity of designs produced within a shared production context. We study 73 promotional websites created by graduate students with Lovable for distinct real businesses under a graded assignment that rewarded original design. We represent each homepage using DINOv3 embeddings and analyze pairwise cosine similarity, effective diversity, and descriptive cluster structure. A representative UMAP–HDBSCAN specification assigns 88% of sites to six interpretable clusters, while sensitivity analyses yield between three and twelve clusters. In a process subsample of nine recordings, preliminary AI-assisted coding documents recurring generate-and-check behavior, incomplete transmission of verbalized intentions into prompts, and infrequent explicit evaluation of originality. Among survey respondents, perceived human control, satisfaction, and perceived quality show no detectable association with embedding-based cohort atypicality. These results document within-cohort visual concentration and a mismatch between subjective experience and distributional distinctiveness. We call this condition authored ignorance. Alongside early evidence that the collective cost of generative production reaches visual design, we contribute a measurement strategy that joins outcome, process, and perception for interactive artifacts, and design implications running from originality feedback to deliberate friction.
Read the full abstract

Human-Computer Interaction

Spotting (and Missing) Algorithmic Bias: Investigating User Understanding in a Fairness Assessment Tool

Fairness metric selection is typically left to data scientists, but which biases are problematic and which metric captures them best depends on stakeholders' experience and domain knowledge. This calls for involving non-technical stakeholders, but the research prototypes built for this purpose so far have not tested whether these stakeholders form accurate mental models of the metrics they interact with or can act on them to identify biases. We present FairAware, a fairness assessment tool co-designed with Human Resources (HR) domain experts. We evaluate stakeholders' understanding through a mixed-methods study with 70 participants (35 HR employees, 35 job seekers), measuring objective and subjective understanding, cognitive load, bias identification accuracy, and open-ended feedback. Most participants correctly identified the most disadvantaged group, with task duration being the only significant predictor. We also found a gap between subjective and objective understanding, with both groups performing similarly across all measures. These results suggest that fairness assessment tools for non-experts are usable for identifying biases but need built-in checks on understanding before stakeholders make higher-stakes decisions.
Read the full abstract

All authors Anna Verheyden, Yizhe Zhang, Robin De Croon, Simone Stumpf, Katrien Verbert

Human-Computer Interaction · Cross-listed from cs.CY

Instructional Governance by Design: A Framework for AI in Computing Education

As generative AI permeates computing instruction, the emergent design challenge is to configure each tool's pedagogical role, authority, and accountability for the instructional work it performs. We argue for instructional governance by design: governance should be encoded in a teaching tool's interaction model, constraints, and workflow. We introduce a multidimensional framework that characterizes AI teaching tools through (1) pedagogical grounding, (2) AI instructional authority, (3) human accountability and control, (4) learner agency and cognitive engagement, (5) context specificity and boundary setting, and (6) evaluation visibility and revision. These dimensions yield governance profiles that help educators align tools with specific purposes and educational stakes. We develop the position through a comparative analysis of a portfolio of AI teaching tools across computing and first-year engineering: rubric-anchored GTA simulations, reflection-oriented code companions, staff-reviewed forum-response systems, TA-supervised diagram generators, and course-specific code-style coaches. These cases show how common instructional functions call for different combinations of rubrics, learning-theory commitments, approval gates, supervision, and course-specific constraints. We further apply the framework to selected published tools to demonstrate its use beyond a single institutional portfolio. From these cases, we identify reusable design questions for aligning governance with instructional stakes, human capacity, and intended learning processes. This position reframes responsible AI integration as a curricular and interaction-design challenge and offers a common vocabulary for tool builders, instructors, and researchers to design, compare, and evaluate AI-mediated learning environments.
Read the full abstract

Human-Computer Interaction

From Sensor Data to Classroom Inquiry: GenAI-Supported Exploration of School Digital Twin Data

Digital Twins for educational buildings can support sustainability-oriented learning, but their use in schools remains limited. This paper presents a GenAI-based chatbot built on top of an existing Digital Twin for two school buildings in Greece, using real IoT data from environmental sensors and energy meters. The chatbot enables educators to query live and historical building data, compare spaces, and generate ideas for classroom activities through natural language. The system was evaluated in an 80-minute workshop with 17 secondary-school educators, who compared it with an existing web-based dashboard. Results show strong perceived usability and pedagogical value, particularly for inquiry-based learning, hypothesis formation, and interdisciplinary lesson planning. Participants also highlighted limitations related to response speed, data verification, trust, and the continued value of visual dashboards. Overall, the findings suggest that GenAI interfaces can make Digital Twin data more accessible for educational use, provided they are designed with transparency, verification, and pedagogical grounding.
Read the full abstract

All authors Themistoklis Sarantakos, Dimitrios Amaxilatis, Michail Giannakos, Georgios Mylonas

Human-Computer Interaction · Cross-listed from cs.LG

Runtime Action Interference for AI Control of AlphaStar in StarCraft II

A trained reinforcement learning policy does not determine the complete behavior that users encounter: deployment code still schedules, admits, suppresses, or replaces its proposed actions. We contribute runtime action interference (RAI), an AI control mechanism that preserves policy parameters while regulating action pacing and filtering configured action patterns after inference. RAI releases a proposed action only when its cooldown condition is satisfied and its content detector does not flag the action; otherwise, it dispatches a no-op. The detector covers specified toxic behaviors, including worker-unit harassment, while the cooldown controls action rate. We implement RAI in a replication of AlphaStar actor.py and make the implementation and reproducibility materials available through an open source code repository. We deployed RAI in a StarCraft~II human participant study that compared two presentations of the same opponent with high capability and rate limited actions; we withheld its capability claim in one presentation and disclosed it in the other. On response scales from 1 to 5, we observed pooled fairness, trust, and toxicity means of 3.90, 3.50, and 2.00 under claim withholding, compared with 2.62, 4.31, and 2.85 under disclosure. Disclosure corresponded with lower perceived fairness and higher perceived toxicity across every expertise group, whereas trust increased among novices and experts but decreased among intermediate participants. Our human evaluation therefore shows that perceptions of an opponent controlled through RAI can vary substantially with the capability information presented to users, even when the configured control remains constant. We conclude that human-computer evaluations must separate control within the execution stack from capability disclosure and assess fairness, trust, and toxicity as distinct dimensions of human experience.
Read the full abstract

All authors Jaymari Chua, Chen Wang, Liming Zhu, Lina Yao

Human-Computer Interaction

Weather- and Location-Aware Agentic Dining Recommendation: Leveraging LLM World Knowledge for Region-Sensitive Contextual Reasoning

Context-aware recommender systems have long recognized that factors such as location, time, and weather shape where and what people choose to eat. Existing weather-aware food and point-of-interest recommenders, however, typically treat weather generically – mapping conditions to preferences through hand-crafted rules or specially trained context models – and do not capture that the culturally appropriate response to weather is itself region-specific: a rainy evening calls for hot tea and fried snacks in one culinary culture and for very different comfort food in another. Encoding such weather-by-region-by-cuisine interactions as explicit rules or training data is brittle and does not scale. We present a weather- and location-aware agentic dining-recommendation system that takes a different approach: a large language model (LLM) orchestrates tools for location and weather retrieval and then reasons in natural language over the combined context, drawing on the cultural and culinary world knowledge already latent in the model to produce region-sensitive, weather-appropriate recommendations without per-region rule tables or specialized training. We describe the agent architecture, the tool-orchestration flow (Google location services and a weather service feeding an OpenAI LLM), and the reasoning mechanism, and we report on a working prototype that was implemented and briefly deployed end-to-end. We discuss design trade-offs – cost, latency, ambiguity handling, and fallbacks – and we are explicit about limitations, including the absence of a formal user study and the risk of cultural stereotyping in locality-based inference. The contribution is architectural: a simple, extensible pattern for incorporating environmental and cultural context into agentic recommendation through LLM reasoning rather than engineered rules.
Read the full abstract

Human-Computer Interaction

"Always Want to Use it for Everything": Understanding Young Adults' Perceptions of AI Dependence

The growing integration of general-purpose AI chatbots into people's daily lives has raised concerns about the potential for unhealthy dependence, particularly among young adults. As a first step toward understanding and characterizing AI chatbot dependence from the perspective of young adults, we collected testimonials from AI chatbot users aged 18 to 25 through an online questionnaire to capture their thoughts and experiences with this phenomenon. From participant responses, we identified three contributing factors of AI dependence: chronic use, efficiency, and delegation. The combination of these in a person's interaction behavior was considered to indicate AI dependence. Participants also observed feelings of atrophy in abilities from AI dependence, leading to psychological impacts such as feelings of inadequacy. Interpreting these findings through the lens of self-determination theory reveals how AI chatbot dependence can impact young adults' personal and social development. We argue that preventing lasting harm to young adults' development is paramount, and provide implications for rethinking AI chatbot dependence grounded in this understanding.
Read the full abstract

All authors Ashlee Milton, Leah Ajmani, Amy Heger, Forough Poursabzi-Sangdeh, Mihaela Vorvoreanu, Jina Suh

Human-Computer Interaction · Cross-listed from cs.GR

GenGA: Editable and Data-Grounded Graphical Abstract Generation for Academic Papers

Graphical Abstracts (GAs) visually summarize the key findings of academic papers, playing a crucial role in facilitating the understanding of research content. Recently, advancements in vision-language models and image generation models have enabled the automatic generation of scientific figures based on paper content. However, most conventional methods output the generated results as raster graphics, making post-editing (e.g., text modification and layout changes) highly difficult. This poses a significant challenge, as they are unsuitable for the iterative figure revision process inherent in paper writing and peer review. To tackle these challenges, we define the novel task of generating editable GAs from paper content and propose GenGA, a new GA generation framework that directly produces figures in vector format. By generating figures as a collection of vector elements with a hierarchical structure, GenGA produces outputs that can be seamlessly imported into existing drawing tools for intuitive, element-level editing. Furthermore, we introduce the Structural Independence Coefficient (SIC), a metric that quantifies the editing simplicity of a figure based on the degree to which local modifications propagate to other elements. Experimental results show that GenGA achieves superior editing simplicity compared to conventional methods, and even surpasses human-authored GAs in conciseness and semantic alignment. We also validate SIC as an effective metric correlated with manual editing costs. This study fundamentally redefines GA generation as an editable vector graphic generation problem grounded in the practical workflows of researchers, significantly promoting effective scientific communication.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.RO

Failing Gracefully: Mitigating Impact of Inevitable Robot Failures

Service robots operate in household environments shared with humans, pets, and everyday objects, where they are highly susceptible to failures such as software crashes, hardware degradation, or unpredictable interactions. While roboticists strive to minimize failures, some remain inevitable, making it critical to mitigate their potential consequences for safe and reliable deployment. This paper introduces a novel safety formulation that evaluates both the probability of impactful interactions between robots and surrounding entities during failures, and the severity of their outcomes. By quantifying the impact of failures on different entities, our approach enables robots to make informed planning decisions that balance safety with task efficiency. To support systematic evaluation, we also present FailBench, a MuJoCo-based simulation framework for studying robot-environment interactions under diverse failure modes, including sensing issues and actuator malfunctions. Together, our safety formulation and FailBench provide a foundation for developing safer and more robust motion plans and learned policies in real-world household environments.
Read the full abstract

All authors Duc M. Nguyen, Saad A. Ghani, Andrew Marshall, Allison Andreyev, Gregory J. Stein, Xuesu Xiao