Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Human-Computer Interaction

cs.HC Watch Human-Computer Interaction

Thursday 1 October

Human-Computer Interaction · Cross-listed from cs.CY

Designing for Interpretation Uncertainty: Architecture and Principles for Topological Learning Analytics Dashboards

Hitoshi Inoue and Koichi Yasutake

Topological Data Analysis (TDA) offers novel methods for understanding temporal dynamics in complex systems, yet its application in information systems design faces a fundamental challenge: how should systems present analytical outputs when interpretation frameworks are still developing? This paper reports on the development of TopoLA, a dashboard system applying Zigzag Persistent Homology to learning management system data, and proposes three early design principles for interpretation support in emerging analytics: (1) separation of objective measurement from contextual interpretation, (2) graduated disclosure from metrics through patterns to reflective prompts, and (3) explicit acknowledgment of methodological uncertainty. The system implements a modular three-stage pipeline–feature extraction, topological computation, and interpretation support–enabling extension to additional analytical methods. This work contributes to information systems research by articulating preliminary design knowledge for systems that must communicate analytical insights from methods lacking established interpretation norms–a challenge increasingly common as novel computational techniques enter applied domains.
Read the full abstract

Human-Computer Interaction

Who Thinks First? Designing Productive Friction with Engage-to-Unlock GenAI

Xiaotian Su, Laura Rimell, Jiazheng Li et al. (+6 more)

Generative AI can support writing, but frictionless access may cause cognitive offloading before users develop their own ideas. We introduce Engage-to-Unlock, a productive-friction mechanism that unlocks generative capabilities after users meaningfully engage with the task. In a controlled experiment (N = 398), participants completed a writing task under one of four conditions: Human-Only, Standard Chatbot, Engage-to-Unlock, or Time-Matched Unlock, which matched unlock timing to Engage-to-Unlock participants but independent of users' engagement, then evaluated passages for evidence and inferential errors. Results show that Engage-to-Unlock redistributed effort across tasks: participants spent more time writing and less time evaluating, without increasing overall task duration. They also submitted more prompts than in other AI-assisted conditions and showed the highest accuracy-per-time evaluation efficiency across conditions. These findings suggest that designing GenAI access to encourage early human engagement may provide a productive form of friction, while retaining active AI use and efficient downstream evaluation.
Read the full abstract

All authors Xiaotian Su, Laura Rimell, Jiazheng Li, Amal Rannen-Triki, Ulrich Paquet, Lisa Anne Hendricks, Rida Qadri, Daphne Ippolito, Piotr Mirowski

Human-Computer Interaction · Cross-listed from cs.LG

ibUMAP: Coherent and Scalable Field Evaluation for UMAP Optimization

Bin Chen, Yumeng Xue, Patrick Paetzold et al. (+2 more)

UMAP achieves scalable layout optimization through stochastic negative sampling. However, this stochasticity can lead to unstable embeddings across reruns and downstream reuse, as the estimated repulsive forces depend on the ordering of sampling events. We present ibUMAP, a coherent field-based alternative that evaluates attraction and repulsion from a shared embedding snapshot and applies them synchronously. Its degree-weighted repulsive field is motivated by the conditional expectation of negative sampling for a fixed embedding and represented by three scalar moments, which are evaluated efficiently on CPUs and GPUs using an interpolation-based FFT scheme. This formulation avoids explicit all-pairs computations while inducing optimization dynamics that differ from those of standard online UMAP. Controlled experiments show that synchrony and kernel capping alter the local-global fidelity trade-off, whereas FFT evaluation produces small average changes in final quality. End-to-end benchmarks show median speedups of 3.29x unseeded and 5.79x seeded over umap-learn on CPU, and 1.44x over cuML on million-scale datasets under unseeded GPU execution. These gains accompany greater run-to-run stability and measurable fidelity trade-offs.
Read the full abstract

All authors Bin Chen, Yumeng Xue, Patrick Paetzold, Yunhai Wang, Oliver Deussen

Human-Computer Interaction · Cross-listed from cs.RO

PROMO: Preference-conditioned Multi-Objective Reinforcement Learning for Quadrupedal Robots

Amr Mousa, Rifny Rachman, Neil Karavis et al. (+2 more)

Quadrupedal locomotion requires balancing conflicting objectives such as command tracking, stability, and energy efficiency, yet conventional reinforcement learning (RL) hardcodes these priorities into a fixed scalar reward at training time. We present PROMO (Preference-Conditioned Multi-Objective Reinforcement Learning), a semantic multi-objective approach that makes this trade-off an explicit runtime input to a single locomotion policy. PROMO conditions the policy on deployment facing preferences while keeping embodiment-specific locomotion priors fixed, thereby separating operator intent from reward shaping terms required for viable gait generation. Compared with fixed-objective controllers, multi-objective baselines, and independently trained specialists, PROMO achieves objective specialization and robustness from a single deployable policy. Across 100 sampled preferences in simulation, 67 behaviors are non-dominated under exact Pareto dominance, with a mean preference-objective correlation of 0.843, demonstrating broad Pareto coverage and predictable preference response. The same policy transfers zero-shot to a Unitree Go2, where preference changes alone reduce specific energy by up to 30.4%, position error by 38.7%, and peak body-attitude deviation by 59.0% relative to the balanced preference. These results establish preference-conditioned multi-objective RL as a practical runtime interface for adaptive legged locomotion, extending its role beyond offline Pareto-set construction. Open-source code and videos are available at https://amrmousa.com/promo/.
Read the full abstract

All authors Amr Mousa, Rifny Rachman, Neil Karavis, Michele Caprio, Richard Allmendinger

Human-Computer Interaction · Cross-listed from cs.CY

WIP: DBWorkout: A Gamified SQL Practice Platform to Support Formative Learning in Database Courses

Sehrish Basir Nizamani, Deepika Devaraj, Tien Nguyen et al. (+4 more)

This research WIP paper presents DBWorkout, a web-based platform that supports formative SQL learning through sandbox-based execution, automated result-based feedback, and session-based gamification. Learning Structured Query Language (SQL) remains challenging for undergraduate students due to limited opportunities for interactive practice and immediate feedback. Students iteratively practice SQL on live database instances while receiving multi-dimensional feedback on query correctness, including row values, column structure, and ordering. To reduce instructor workload, DBWorkout incorporates large language model (LLM)-assisted tools for schema and task generation within a human-in-the-loop workflow. A pilot study with teaching assistants and a classroom deployment involving 170 undergraduate students across two in-class sessions show strong perceived learning value (90% agreement) and engagement (87% enjoyment), alongside low reported pressure (22%). However, only 42% of students found the automated feedback sufficiently actionable, a finding independently corroborated by 40% of open-ended responses raising feedback quality concerns, providing cross-method triangulation of this gap. These findings demonstrate the technical feasibility and early pedagogical potential of DBWorkout while identifying directions for enhancing feedback quality and supporting sustained SQL learning.
Read the full abstract

All authors Sehrish Basir Nizamani, Deepika Devaraj, Tien Nguyen, Khyati Goyal, Saad Nizamani, Sally Hamouda, Jaren Goldberg

Human-Computer Interaction · Cross-listed from cs.CY

Understanding Student Use of Large Language Models Across Computer Science Subfields

Sehrish Basir Nizamani, Yoonje Lee, Nikitha Donekal Chandrashekar et al. (+2 more)

This research full paper examines how undergraduate students use large language models (LLMs) across computer science subfields. As LLMs become increasingly integrated into computing education, understanding how their use varies across technical and pedagogical contexts is essential for designing effective, subfield-aware instruction. This paper presents a cross-subfield analysis of LLM usage among 211 undergraduate students in a problem-solving course intentionally designed to support responsible and effective LLM use through structured instruction and reflection. Using post-assignment reflection data collected across seven instructional modules spanning multiple computer science subfields, we examine prompt counts, LLM role conceptualization, and verification behavior. Results show that LLM adoption varies substantially by assignment, with higher usage in algorithms and web development and lower usage in software engineering. Students predominantly treat LLMs as assistive tools rather than authoritative sources, and verification is common across all subfields, with most students using multiple strategies. Verification behavior also varies by assignment context, with testing more common in structured tasks and web search more common in open-ended tasks. These findings suggest that assignment characteristics play a central role in shaping how students interact with and evaluate LLM outputs, even under a single, consistently applied instructional design. This work contributes empirical evidence on how LLM adoption, role conceptualization, and verification behavior vary across computer science subfields, extending our prior work on structured, reflective LLM instruction to show how its effects differ by task rather than only in aggregate.
Read the full abstract

All authors Sehrish Basir Nizamani, Yoonje Lee, Nikitha Donekal Chandrashekar, Margaret Ellis, Naren Ramakrishnan

Human-Computer Interaction

Scaling Peer Assessments: An Integrity Report from a Large Engineering Internship

Jinal Gupta, Pavani Ayinampudi, Aditya B.M.V. et al. (+5 more)

Assessing learning in large classrooms presents a significant challenge for individual instructors, who may have limited capacity to evaluate the understanding, participation, and assessment behaviour of every student. Peer assessments have been a way of distributing this responsibility among learners, allowing them to evaluate and provide feedback to one another while reducing dependence on instructor-led assessments. Building on this approach, we implemented a peer validation model within a large, multi-institutional internship programme in which students who demonstrated sufficient understanding were authorised to assess and validate their peers through short oral discussions. The assessment process began with the instructor validating a small group of students, who were then authorised to validate their peers, allowing the process to gradually expand across the cohort and operate at scale. This study examines how participants experienced the model and the extent to which assessment integrity was maintained, using an end-of-programme survey of 238 consenting respondents. Most participants regarded the activity as worthwhile, with 79.8% reporting that they solved problems they could not previously solve. However, 29.0% acknowledged at least one instance of reduced effort, a lowered validation standard, or reciprocal validation, while 88.7% believed that at least a little validation had occurred without proper examination. When asked how the process could be strengthened, participants selected post-validation discussion of solutions approximately twice as often as closer auditing or mentor-led validation. These findings provide descriptive evidence of both the potential and the integrity challenges of using peer validation as a scalable assessment approach in large learning environments.
Read the full abstract

All authors Jinal Gupta, Pavani Ayinampudi, Aditya B.M.V., Prakash Hegade, Rohit Sharma, Sakshi Sharma, Meenakshi V, S.R.S. Iyengar

Human-Computer Interaction · Cross-listed from cs.AI

Cybernetic and Epistemic: A Missing Vocabulary for Trustworthy Agentic Delegation

Jérémie Lumbroso

As code generation is increasingly delegated to AI systems, the bottleneck is shifting from writing code to supervising the systems that write it — a shift CS-education researchers have begun to name. This shift exposes a vocabulary gap: the field asks for "human oversight" without a working distinction between the two things language does in a delegation channel — coordinate action (cybernetic: words succeed when the world comes to match them) and coordinate understanding (epistemic: they succeed when they answer to the world and a hearer can check that they do). The failure this names is not cybernetic language but epistemic-form language doing cybernetic work: explanation-shaped output calibrated for approval rather than truth. Oversight that checks only whether an output was approved is satisfiable by rubber-stamping; oversight that holds an agent accountable requires the reasoning behind its work be retrievable and checkable. We present three delegation episodes — illustrations, not controlled evidence — in which epistemic engagement proved practicable while remaining auditable, one public record where a recommendation was withdrawn on its own stated terms, and one failure case illustrating oversight that requires no reasons for its discretionary choices. We propose a criterion for agentic-system governance, alongside existing technical trust properties: every consequential choice should carry the condition under which it would have gone otherwise, in a form a third party can test. Without such a condition, a third party cannot distinguish a decision from a rubber stamp. We give the criterion an operational form — a two-part reconstruction test scoring a delegation record by whether a second reader can predict what the agent does under a perturbation — and a deliberation-recording convention, ORRCF, that makes the condition a required component of every recorded choice.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.AI

ABDA-NL: A Natural-Language Scenario Explorer for Argument-Based Reasoning

Shawn Bowers, Martin Caminada, Haoyang Liu et al. (+1 more)

ABDA-NL adds a natural-language interface to ABDA, a system for argument-based discussion using ASPIC- knowledge bases under grounded semantics. Users see which conclusions are accepted, rejected, or undecided, open an interactive rendering of the grounded discussion game to learn why, explore what-if alternatives by suspending assumptions and rules or changing preferences, ask questions that are answered from a scenario's reference documents, and author new facts, assumptions, and rules in plain English. A large language model provides the bridge between language and formalism: it answers questions from the documents and the current state of the scenario, and it translates plain-English edits into candidate formal statements. The deterministic ABDA engine remains the sole source of arguments, attacks, and acceptance labels, and every proposal of the model is validated and confirmed by the user before it takes effect.
Read the full abstract

All authors Shawn Bowers, Martin Caminada, Haoyang Liu, Bertram Ludäscher

Wednesday 30 September

Human-Computer Interaction · Cross-listed from eess.AS

Intonation Perception in Real and Synthetic Speech across Varying Familiarity Levels: A Pilot Study of Equivalence Assessment

Hanrui Zhou, Gaoyuan Zhang, Yixiang Chen et al. (+4 more)

Language training relies on a corpus constructed by a large number linguistic materials. AI-powered voice clones provide a way to construct the corpus with relatively low cost. Singing voice conversion (SVC) model is used to generate synthetic voices. This study compares participants' performances on natural and synthetic speech in two experiments, similarity perception and intonation recognition. In the accuracy of similarity perception task, a significant interaction between speech type and intonation is found, suggesting that question may serve as a cue for speaker identification but may be influenced by synthetic features. In the accuracy of intonation recognition task, a significant interaction between speech type and familiarity is observed, indicating that speech type affects how much familiarity contributes to voice processing.
Read the full abstract

All authors Hanrui Zhou, Gaoyuan Zhang, Yixiang Chen, Yujie Xing, Feng Xu, Xurong Xie, Hui Chen

Human-Computer Interaction

LeanSide: A Formally Verified Co-Reasoning System for Natural-language Proofs

Chenjun Guo, Manooshree Patel, Arnav Mehta et al. (+7 more)

Large language models are increasingly used as collaborators on deductive-reasoning tasks, but their outputs can hallucinate or pull users away from intended reasoning. Formal proof assistants provide machine-checked verification, but have a steep learning curve and require more granular reasoning than human written proofs. We explore an interface that combines these strengths, allowing users to write and revise free-form natural-language proofs while a verified backend checks their reasoning and returns feedback at the user's granularity. We study this interface in the context of undergraduate mathematics education by developing LeanSide, a formally verified co-reasoning system, which auto-formalizes student reasoning into Lean and informalizes verifier output into understandable feedback. We conducted user studies through classroom deployment and analyzed which system properties helped students make progress and which caused them to get stuck. We use these findings to derive design implications for using a formally verified backend in human-AI co-reasoning systems.
Read the full abstract

All authors Chenjun Guo, Manooshree Patel, Arnav Mehta, Krishiv Kothari, Thomas Lu, Niels Voss, Rayna Bhattacharyya, Peter Donovan, Bjoern Hartmann, Gireeja Ranade

Human-Computer Interaction · Cross-listed from cs.RO

Toward Humanoid Robots in Construction: A Teleoperation Feasibility Study

Parastoo Ali Pour, David R. Martin, Chang Min Hur et al. (+6 more)

We present a teleoperation system that enables a single operator to perform construction tasks on a Unitree G1 humanoid, combining extended reality (XR) based upper body control with pedal-based locomotion to enable simultaneous manipulation and locomotion. Motivated by persistent labor shortages, hazardous working conditions, and challenges in humanoid autonomy, we investigate teleoperation as a practical near-term approach for reducing physical strain on workers while generating high quality demonstration data. We evaluate the system on two representative construction tasks drawn from O*NET occupational database, and report task success and completion time relative to a manual baseline. The system achieved 100% success on tool transport and 80% success on surface painting, with teleoperation requiring substantially more time compared to manual execution.
Read the full abstract

All authors Parastoo Ali Pour, David R. Martin, Chang Min Hur, Bo Zhang, Tommy Zhou, Brandon Thomas Lichter, Shane Stanfield, Pramod Khargonekar, Mohammad Abdullah Al Faruque

Human-Computer Interaction

From Images to Tasks: Characterizing Multimodal LLM Interactions in the Wild

Jinyi Ye, Scott Counts, Gaurav Verma et al. (+2 more)

Multimodal large language models (LLMs) increasingly integrate vision and text, yet how people use them in natural settings remains underexplored. We seek to answer the question: when users upload images, what tasks are they trying to accomplish? Analyzing over 40,000 de-identified image-upload conversations from Microsoft Copilot, we characterize real-world multimodal use through a hierarchical framework of ten capabilities, spanning perception, cognition, and generation. First, we characterize the distribution and composition of these capabilities, finding that the majority of image-upload tasks involve multiple capabilities. Second, we find that multimodal use spans a broader and more diverse task space than text-only interactions, with asymmetric coverage and task classes that rely on cross-modal grounding. Third, mapping observed capability demand onto 253 existing benchmarks reveals uneven alignment between benchmark coverage and real-world use: benchmarks concentrate on perception and reasoning toward fixed answers, while common workflows involving text, code, and data generation remain comparatively undertested. We validate our taxonomy and findings on an independent ChatGPT dataset. Our results provide a large-scale empirical characterization of what users seek to accomplish with multimodal LLMs and highlight opportunities for benchmark design grounded in observed user demand.
Read the full abstract

All authors Jinyi Ye, Scott Counts, Gaurav Verma, Kate Lytvynets, Weiwei Yang

Human-Computer Interaction · Cross-listed from cs.CY

From Scroll to Sale: Exploring the Impact of Interaction Type and Device Price on TikTok Advertisements

Nazanin Sabri, Cat Mai, Haodi Zou et al. (+4 more)

Companies and brands increasingly use dynamic pricing, including targeting social media ads to users based on their income. In this work we audit TikTok's feed using 56 automated accounts, which collect data on over 80,000 videos, across two studies. We test the impact of device price on ad load and ad types, using 12 phones of low ($0-$250), medium ($400-$650), and high ($750-$1,000+) price as a proxy for income. We also test the impact of interaction type (i.e., like, comment, share), age, and gender on the frequency and content of ads. Overall, the ad load was 29.4%, but liking and sharing content increased ad load significantly. We also found that as accounts spend more time on TikTok, the ad load steadily increases. We found some evidence that device price impacts both ad load and content – more expensive devices were targeted with fewer ads, while the least expensive devices received more discounts.
Read the full abstract

All authors Nazanin Sabri, Cat Mai, Haodi Zou, Isha Varada, Damon McCoy, Deepak Kumar, Kristen Vaccaro

Human-Computer Interaction

Sensing Instability, Adapting the Scene: A Real-Time Movement-Smoothing Design Framework for Stable VR Locomotion

Ramisa Fariha Joyee and M. Rasel Mahmud

Users experience different balance challenges while standing, walking, and turning in virtual reality (VR), yet most locomotion techniques apply the same visual behavior regardless of movement state. We present a real-time movement-smoothing design framework in this paper that organizes visual adaptations according to movement-specific balance demands. The framework introduces three design strategies targeting standing, walking, and turning, illustrating how state-aware visual adaptations can support postural stability during locomotion. This paper focuses on the framework design and implementation, while a user study is planned as future work. Our work guides the development of future context-aware VR locomotion systems that better support safe and comfortable navigation.
Read the full abstract

Human-Computer Interaction

Preliminary Evaluation of Transition-Aware Controller Locomotion Adaptations for Supporting Postural Stability in VR

Ramisa Fariha Joyee and M. Rasel Mahmud

We present three locomotion adaptation approaches: Motion Acceleration, Turn Acceleration, and Motion Deceleration to improve postural stability during body-state transitions in virtual reality (VR). The system detects standing-to-walking, turning, and walking-to-stopping transitions and applies adaptive locomotion smoothing. Motion Acceleration gradually increases locomotion speed when users begin walking, Turn Acceleration smooths rotation while turning, and Motion Deceleration gradually reduces movement speed before stopping. We evaluated these techniques in a virtual navigation task using objective and subjective balance measures. Preliminary results show reduced center of pressure (COP) velocity and improved balance confidence. These findings suggest that locomotion adaptations can improve balance and navigation experience.
Read the full abstract

Human-Computer Interaction

Faithful Chart Generation for Multimodal Deep Research: Frame-Evidence Co-Adaptation

Yuxin Yue, Yingchen Zhang, Ruqing Zhang et al. (+3 more)

Analytical charts in multimodal deep research encode quantitative claims, requiring every visualized value to be faithfully grounded in supporting evidence. Unlike retrieved images that mainly provide contextual information, charts require numerical fidelity: visualized values should not only match retrieved evidence quantitatively but also preserve its original meaning and scope. However, achieving such fidelity remains challenging because current systems usually construct visualization plans before knowing what quantitative evidence can actually be retrieved from the web. As a result, predefined plans may require entities, temporal ranges, or comparison dimensions that the retrieved evidence only partially supports. Existing approaches mainly address this issue through post-hoc verification after chart plans are fixed, enabling unsupported values to be identified but leaving the underlying visual frames unchanged. To address this challenge, we propose Frame-Evidence Co-Adaptation (FECA), an evidence-adaptive visual planning framework for multimodal deep research. Inspired by the bidirectional sensemaking process in Data-Frame Theory, FECA models chart generation as an iterative interaction between visual frames and retrieved evidence. Each visual frame is adaptive: the frame guides evidence acquisition, while retrieved evidence determines whether the frame should be accepted, revised, or dropped before rendering. By coupling visualization planning with evidence availability, FECA shifts chart generation from fixed-plan verification to adaptive evidence-grounded visual reasoning. Experiments on 100 real-world research topics show that FECA substantially improves numerical fidelity while preserving report quality and chart utility.
Read the full abstract

All authors Yuxin Yue, Yingchen Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Xueqi Cheng

Human-Computer Interaction · Cross-listed from cs.RO

Rethinking Legibility in Social Robot Hallway Navigation: Impact of Intent Representation and Human Distraction

Pranav Goyal, Andrew Stratton and Christoforos Mavrogiannis

We focus on legible robot motion generation in social navigation settings. Legibility in human-robot interaction (HRI) is often described as the property of robot motion that enables an observer to confidently infer the robot's intent. While mature frameworks exist for generating legible motion in front of static observers, social robot navigation presents a new challenge: the robot must clearly convey its intent while ensuring human safety in dynamic pedestrian environments where human attention is often divided. With the goal of enabling robots to generate legible motion in dynamic and constrained spaces, we investigate how the choice of representation and the level of human attention shape navigation performance and human impressions. Focusing on the ubiquitous and demanding scenario of hallway navigation, we conduct two controlled user studies involving alternative legibility formulations implemented within a shared model predictive control framework. Study 1 (N = 45) investigates the role of intent representation, showing that passing-side legibility, particularly when adaptively updated, leads to smoother human motion and is perceived as more competent and less mentally and physically demanding than destination-based and non-legible baselines. Study 2 (N = 45) examines the effect of pedestrian attention, demonstrating that legible motion allows for smooth human motion even under distraction, even if this is not consistently reflected in subjective ratings. Together, these findings suggest that effective legible motion in social robot navigation benefits from interaction-level intent representations that support coordination, with some effects persisting even when human attention is divided. Code is available at https://github.com/fluentrobotics/Legible_MPPI.
Read the full abstract

Human-Computer Interaction

Who Asked for This? Inline Annotations as Authoring Transactions for Provenance in Agentic Authoring

Chang Xiao

Writing with AI agents turns a paragraph into the outcome of many requests, yet the finished document rarely explains which request produced which change. We introduce Reactant, an interaction paradigm in which authors place typed inline annotations in their original documents. A verified transaction protocol records each request, skill identity, and the correspondences that identify additions, deletions, and transformations. The kernel validates the witness against the recorded states to establish word-level longitudinal lineage. We demonstrate Reactant through this paper's revision history, and report four months of three colleagues' self-directed use. Their uses include conversational inquiry into history and deriving reusable skills from recurring requests, illustrating how the transaction record serves as an extensible substrate for agentic authoring.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CL

JuryFlow: Disagreement-Guided Human-in-the-Loop Multi-Agent Evaluation

Mufeng Yang, Junwei Yu and Yepeng Ding

Large language models (LLMs) are increasingly deployed as automated judges for AI-generated content, yet a single judge is unreliable and even a panel of judges leaves a hard residue: when judges disagree, majority voting discards the conflict instead of resolving it. We present JuryFlow, a disagreement-guided, human-in-the-loop multi-agent evaluation framework that treats inter-judge disagreement not as noise to be averaged away, but as a precise, claim-level signal indicating where an evaluation is uncertain. JuryFlow decomposes each candidate response into atomic claims, has a panel of heterogeneous judges assign per-claim verdicts, and builds a disagreement graph whose nodes are scored by verdict entropy and whose edges encode structural similarity between claims. A human acts as a structural guide, selecting which disagreement to resolve through a single, minimal intervention rather than re-labeling the response, after which the focal claim is re-evaluated, the correction propagates along graph edges and to historically similar cases, and is crystallized into reusable rubric entries that all judges inherit, making the evaluator progressively self-refining. To enable large-scale, reproducible benchmarking without human studies, we evaluate JuryFlow in an automatic configuration in which focal selection is made by entropy ranking. On MT-Bench and LLMBar, JuryFlow improves agreement with gold labels over single-judge and majority-vote panel baselines, and ablations isolate the contributions of disagreement-targeted re-evaluation, propagation, and rubric induction. We contribute (1) a human-in-the-loop paradigm that recasts the human from labeler to structural guide, (2) the JuryFlow framework operationalizing it through a disagreement graph, focal re-evaluation, and closed-loop rubric induction, and (3) an evaluation protocol with ablations that isolate where the gains originate.
Read the full abstract

Human-Computer Interaction

Conversational Capture: A Trajectory-Level Framework for Evaluating Generative Engine Optimization in Multi-turn Human-Agent Interaction

Junwei Yu, Jieyu Zhou, Mufeng Yang et al. (+2 more)

Generative Engine Optimization (GEO) shapes content to increase its likelihood of being cited by answer engines built on retrieval-augmented large language models. GEO is typically evaluated as a single-turn property: for a fixed query, an evaluator measures a source's visibility in one answer. We argue that the single answer is an inadequate unit of analysis. Human-agent information seeking forms a closed loop: the agent's answer changes the user's beliefs and therefore the next question, which in turn determines what the agent retrieves. We introduce conversational capture, a phenomenon in which a source cited early becomes substantially more likely to be cited again. Capture operates through a machine-side channel, history-conditioned retrieval, and a human-side channel, follow-up questions directed toward the captured source. We formalize the interaction as a two-layer closed-loop system and derive trajectory-level constructs: cumulative conversational visibility; a direct/feedback decomposition of trajectory gain; a nested split of the feedback term into machine-side and human-side channels; a capture coefficient; a compounding ratio; and a misranking diagnostic. Using reinforcement-process (Pólya-urn) theory, we prove that the feedback term is zero under single-turn evaluation and that GEO's cumulative payoff grows superlinearly with conversation length while capture develops. A model-derived illustration shows that the feedback term can exceed the direct term, the compounding ratio exceeds two within ten turns, and single-turn and trajectory rankings agree only weakly (Kendall's $\tau = 0.4$). We connect the human channel to information foraging, trust calibration, and Bayesian persuasion, and discuss design implications for answer engines.
Read the full abstract

All authors Junwei Yu, Jieyu Zhou, Mufeng Yang, Yepeng Ding, Hiroyuki Sato

Human-Computer Interaction

Richard: Voice-First Mobile Interaction for Persistent Tasks

Xinyang Chen

Mobile terminals need to provide application and network services while supporting users' control over their attention. We explore voice-first interaction organized around requests and delegated tasks, allowing users to leave a conversation and later inspect, revise, and retrieve the work. We present Richard, a system prototype that manages voice sessions, task execution, and result delivery separately, linking them through persistent request records. Conversation and task views provide visual feedback, while the backend coordinates immediate responses, dedicated service operations, and agent tasks. Request revisions, execution states, and notifications remain associated with the relevant task. We examine this design through Android functional records, controlled lifecycle verification, and execution records of a real programming request. Controlled verification reproduces revision, execution after confirmation, and result retention; deployed-service records show backend progress and failure feedback after client disconnection. These observations inform the design of task continuity, user control, and service integration in mobile voice interaction, providing an implementation basis for personal computing devices that accommodate intermittent user participation.
Read the full abstract

Human-Computer Interaction

Diptych: Scoped, AI-Interpreted Comparison for Reference Listening in Music Production

Chongjun Zhong, Abhinaba Roy, Archishman Ghosh et al. (+2 more)

Reference listening is a common strategy in music production, but current comparison tools often obscure a key human judgment: deciding what should be compared. We present Diptych, an AI-assisted system that lets users define comparison scope across whole tracks or independently selected segments, while inspecting structured audio features and scope-specific AI interpretations. We evaluated Diptych in a within-participants study with 12 musicians, complemented by source-blinded ratings from four expert listeners. Participants used the system to surface additional differences, nine of ten of which received at least partial expert support, and reported good usability and greater clarity about possible next steps. These findings suggest that AI support for creative comparison should prioritize user-defined scope, inspectable evidence, and actionable guidance, while avoiding authoritative judgments that exceed what the evidence can support.
Read the full abstract

All authors Chongjun Zhong, Abhinaba Roy, Archishman Ghosh, Kejun Zhang, Dorien Herremans

Human-Computer Interaction

Understanding Parents' Complex Views of AI for Children's Pretend Play

Sungho Oh, Mohammad Namvarpour, Maxi Heitmayer et al. (+2 more)

AI could support children's pretend play, but it could also direct the play on behalf of children. Whether AI should have roles in children's lives is controversial because its influence on children remains uncertain. We conducted semi-structured interviews with 10 U.S. parents, each with at least one child aged 4-15. During the interview, we described the concept of AI-supported pretend play and provided participants with two boundary-case storyboards. We analyzed the interview data through codebook thematic analysis, using inductive coding and affinity diagramming organized around the research questions, and then used qualitative systems mapping to examine relationships within and across themes. We found that the same characteristics of AI, e.g., ability to assume characters, responsiveness, and adaptability, were seen by parents as potentially useful but also concerning. Parents imagined that AI could make role-based play accessible to all children or help parents participate in family play. However, they opposed the idea of AI for children's play without a clear understanding of how it works and its long-term influence on their children. Parents worried about children's loss of imagination and creativity, emotional attachment to AI, reduced human interaction, inappropriate behavior by AI and/or children, and their inability to manage children's AI use. Parents viewed AI not only as a play tool but also as a social actor and a possible perturbation in the existing family dynamics. The appropriateness of AI and child–AI interactions therefore emerged as a requirement for AI in children's pretend play, in addition to technical safeguards and parental control. We contribute an integrated account of parents' interdependent judgments and emphasize the need for longitudinal research with children and their diverse families.
Read the full abstract

All authors Sungho Oh, Mohammad Namvarpour, Maxi Heitmayer, Minahil Khalid, Afsaneh Razi

Human-Computer Interaction · Cross-listed from cs.CL

When a Kindergartener Solves Calculus: Measuring Capability Leakage in Role-Prompted Reasoning Models

Pakhapoom Sarapat, Saksorn Ruangtanusak, Kunat Pipatanakul et al. (+1 more)

We investigate the problem of role-capability leakage (RCL), in which a role-prompted reasoning model generates convincing in-role text while continuing to exhibit capabilities on benchmarks that exceed those implied by the assigned role. For example, when a model is prompted to assume the role of a kindergarten student, one might expect its performance on a mathematics benchmark to reflect kindergarten-level ability rather than expert-level proficiency in solving calculus problems. We introduce RoleCapBench, a curriculum-grounded benchmark for evaluating RCL across six educational roles and four assessment levels spanning elementary school through A-level, and use it to evaluate three open-weight reasoning models. We find that although the models can generate stylistically convincing in-role responses, they consistently fail to align their underlying capabilities with their assigned roles. Naive role prompting yields strong role-voice scores of 1.218–1.389 while retaining above-role accuracy of 0.811–0.898. RCL persists across a range of prompting conditions, including prompts that explicitly instruct the model to match the role's capability level. To mitigate this problem, we propose Injection, an inference-time intervention that combines explicit, role-specific capability guidelines with a guiding prefilled response prefix. Injection improves role-capability alignment across models, reducing above-role accuracy by up to 0.562 while preserving in-role accuracy with a marginal drop of less than 0.058 across most models. All artifacts, including scripts and evaluation data, will be released upon acceptance.
Read the full abstract

All authors Pakhapoom Sarapat, Saksorn Ruangtanusak, Kunat Pipatanakul, Pittawat Taveekitworachai

Human-Computer Interaction

Making Sense of Animal-to-Human Drug Development Evidence: Stakeholder Practices, Challenges, and Requirements for AI Tools

Rosni Vasu, Simona E. Doneva and Benjamin V. Ineichen

Animal models are widely used to study human biology and health interventions, yet translating findings from animal studies to humans remains challenging. Evidence across preclinical and clinical research informs experimental and translational decisions. Artificial Intelligence (AI) tools are increasingly reshaping how this evidence is searched, synthesized, and used, but it remains unclear how they should support the diverse stakeholders involved in assessing animal-to-human evidence. We conducted semi-structured interviews with 13 stakeholders to examine their evidence practices, challenges, and expectations for AI support. We found that stakeholders approach the same incomplete evidence base with different goals, expertise, and heuristics. Participants valued AI particularly for locating, screening, and extracting evidence, but were more cautious about automated interpretation and quality judgments. They emphasized transparency, source traceability, uncertainty communication, and human oversight. Based on these findings, we derive design implications for role-sensitive AI tools that support more systematic and transparent reasoning about animal-to-human translation.
Read the full abstract

Human-Computer Interaction

Engagement-Led Segmentation of Gamified Participation Data in a Large-Scale Remote Internship: A Mixed-Methods Study

Sakshi Sharma, Pavani Ayinampudi, Aditya B.M.V. et al. (+5 more)

Gamified points are widely used to represent learner participation, but cumulative point totals provide limited information about how participation changes over time. This limitation matters in continuously enrolling programmes, where learners have different opportunities to accumulate points. This study examines how gamified participation data can be interpreted as indicators of behavioural engagement in a large-scale, continuously enrolling remote internship. Using an anonymised operational dataset, 3,607 distinct started learners were first classified by participation status, separating dormant learners from those with observable participation. The 1,871 active learners were then grouped using K-means clustering on normalised longitudinal and opportunity-aware participation features. Four participation patterns emerged: Thriving Core, Tapering, Fast Starters, and Occasional Participants. Their trajectories differed in both level and direction. A perception survey of 597 respondents provided complementary learner-reported evidence, integrated with the behavioural strand through a joint display by segment. Perceptions differed across segments, while individual-level correlations between perceptions and behaviour were small, and learners with low recorded participation could report positive perceptions of the points system alongside external barriers. Segments assigned from the first four weeks were then checked against eleven weeks of later platform records: 94% of the Thriving Core attended at least ten further sessions and 22% completed the internship, against under 7% and under 1% in the low-engagement segments. The findings suggest that gamified participation data are more informative when interpreted as longitudinal, opportunity-aware behavioural indicators rather than cumulative scores alone.
Read the full abstract

All authors Sakshi Sharma, Pavani Ayinampudi, Aditya B.M.V., Jinal Gupta, Prakash Hegade, Rohit Sharma, Meenakshi V, S.R.S. Iyengar

Human-Computer Interaction · Cross-listed from cs.AI

Referential Uncertainty in Human–AI Collaboration

Christian Poelitz, Finale Doshi-Velez and Siân Lindley

Effective human-AI collaboration requires partners to establish references through interaction, which becomes fragile when descriptions are ambiguous, similar referents compete, or partners see different things. We study referential uncertainty - uncertainty over which candidate object a description refers to - in a collaborative puzzle task where a human Helper instructs an AI Worker to place pieces. The Worker must identify and communicate its uncertainty, and the Helper must recognize and act on it. We show that a separately elicited belief distribution over candidate pieces is better calibrated (ECE 0.15) and better discriminates correct from incorrect placements (AUROC 0.65) than raw action-token probabilities, which are severely overconfident (0.97 mean confidence, ECE 0.44). Across three frontier vision-language models (GPT-4.1, GPT-5, GPT-5.5), this elicited uncertainty rises predictably with instruction vagueness, but not with competing referents in context, even when those increase errors. The models seldom externalize it, asking for clarification on only 3.5-16.7% of turns. In a controlled human study (N=210), participants given only the Worker's default message accept 78% of wrong placements and cannot tell right from wrong (AUC 0.50). Precise descriptions and, especially, well-targeted hedges cut wrong-move acceptance to 36% while largely preserving correct-move acceptance, compensating for missing shared awareness such as not seeing the Worker's action. But this benefit depends on targeting: a deployable hedge derived from the model's own belief entropy inherits that signal's weakness and can do more harm than good. Externalized uncertainty helps a human partner only when it is accurately targeted.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CL

DuplexAct-Bench: Broadening Full-Duplex Speech Evaluation toward Proactive Interaction across Diverse Behavioral Requirements

Keyue Xing, Wentao Ding, Mengmeng Wang et al. (+3 more)

Existing full-duplex speech benchmarks cover only subsets of real-time interaction behaviors, often under limited contextual conditions. We introduce DuplexAct-Bench, a bilingual benchmark that systematically covers six complementary behaviors, from interruption and yielding to proactive initiation, active silence, and backchanneling, across Pre-session, In-session, and No-explicit conditions. Across 1,290 English and Chinese streaming trials, we evaluate 12 full-duplex speech systems on both Timing and Content. Results reveal substantial variation across behaviors, conditions, and systems, as well as frequent mismatches between semantic quality and behavioral timing. These findings show that current systems remain far from robustly managing when, whether, and how to participate as real-time interaction unfolds. Project page: https://alitaxky.icu/DuplexAct-Bench/
Read the full abstract

All authors Keyue Xing, Wentao Ding, Mengmeng Wang, Wenming Tu, Zilong Zheng, Yipeng Kang

Human-Computer Interaction

IDEAL: A Multimodal Domain Adaptation Framework for EEG-Eye Emotion Recognition

Yang Wu and Jinpeng Li

Electroencephalography (EEG) emotion recognition serves as a pivotal interface for human-computer interaction, yet the physiological variability across individuals complicates the already challenging task of fusing heterogeneous physiological signals (e.g., EEG and eye movements). However, most prevalent domain adaptation paradigms are tailored for unimodal scenarios, failing to address the heterogeneity of multimodal signals. Furthermore, they predominantly rely on feature-level alignment, overlooking the fundamental data-level discrepancy, which risks compromising fine-grained discriminative information during aggressive adaptation. To bridge these coupled gaps, we propose Instance-based Domain Expansion and Adversarial Learning (IDEAL), a unified framework that synergizes instance-level curriculum expansion with feature-level hierarchical adversarial alignment. IDEAL first introduces a multi-model collaborative screening mechanism, which propagates high-confidence target samples to explicitly bridge the distributional gap at the data level via a quantity-quality equilibrium strategy. We provide a theoretical analysis that this instance expansion strategy strictly tightens the upper bound of the target risk. Subsequently, a hierarchical adversarial network, augmented with the angular-contrastive constraints, progressively aligns representations from low-level statistics to high-level semantics while preserving class separability. Extensive experiments on four benchmark datasets demonstrate that IDEAL significantly outperforms state-of-the-art methods. To facilitate reproducibility and future research, our source code is publicly available at https://github.com/WY-BCI-Club/IDEAL.
Read the full abstract