Human-Computer Interaction · Cross-listed from cs.LG
Do Reasoning Representations Help Humans Evaluate LLM Outputs?
Jaewoo Lim, Sungbok Shin and Sanghyun Hong
Reasoning representations are increasingly used as explanations for large
language model outputs. Yet they are typically evaluated with model-centric criteria, such as answer accuracy and faithfulness, leaving it unclear whether they help people evaluate model responses. In this work, we study
reasoning representations as human-facing interfaces rather than proxies for model
reasoning ability. We conduct a controlled human study of six
reasoning formats across tasks of varying complexity, supported by a web-based framework that randomizes task domains, problem instances, and representation order. The study collects fine-grained judgments of structural understanding, error detection and localization, and trust calibration. Our study shows a mismatch between perceived preference and support for human evaluation. Participants prefer planning- and decomposition-based representations, but simpler chain-of-thought traces better support verification, trust, and interpretability. Preferred representations also introduce calibration risks, with more false alarms on correct traces and high trust despite low willingness to verify.
Read the full abstract ▾
Human-Computer Interaction
Embedded Human-Centered Data Science in a Graduate Programming Course: A Framework and Case Study
Victoria Chui, Kelly McConvey, Daniel Chui et al. (+2 more)
As AI and data-driven systems pervade practice, there is an imperative for instructors to embed societal impact and ethics content into computing courses. In response, we present the Human-Centered Education for Learning in Information and eXplainable Computing (HELIX) framework for information science programs, organized around three iterative pillars - knowledge building, decision-making, and empowerment - with concrete actions for instructors and students. We applied the framework in a graduate, introductory programming course using readings, algorithmic design activities, and scenario-based reflections. We present a pilot implementation of this framework to examine changes in students' (n=22) knowledge acquisition, decision-making processes, and self-reflection regarding human-centered perspectives in data science. We release an anonymized materials kit (survey, assignments, analysis code) to support adoption. We discuss design tensions (workload, assessment, relevance to diverse information science learners) and provide guidelines for integrating human-centered content without overwhelming technical outcomes. Findings suggest that the HELIX Framework is feasible in information science contexts and future work should use comparative survey assessment to strengthen causal inferences.
Read the full abstract ▾
All authors Victoria Chui, Kelly McConvey, Daniel Chui, Malayna Bernstein, Shion Guha
Human-Computer Interaction
To Stop or Not to Stop: Exploring the Intention-Behavior Gaps in Smartphone Usage
Jian Zheng and Eun Kyoung Choe
As smartphones become integral to daily life, researchers have sought to identify when the use becomes problematic. Previous studies have operationalized problematic smartphone usage (PSU) from either an intention or a behavior perspective. Both risk delivering interventions not welcomed by users. We propose a novel approach to operationalizing PSU as the intention-behavior gap (IBG). We collected self-reported data on intentions to stop phone usage, alongside usage behavior data, from 37 participants over two weeks. We calculated IBG, examined effects of demographic and contextual variables, and developed machine learning models to predict IBG in real time. We found that IBG was explained by gender, time, app, and input interactions, among other factors. Intention was predicted most accurately with only personal data, whereas behavior and IBG were predicted most accurately with both personal and global data. Our findings can inform the design of future intervention tools optimized for timing and adaptive intensity.
Read the full abstract ▾
Human-Computer Interaction
Healthcare Utilization, Chronic Condition Management, and Workplace Functioning Among Users of a Purpose-Built Mental Health AI (Ash): Cross-Sectional Study
Kristen M. Van Swearingen, Thomas D. Hull, Jeffrey Swigert et al. (+1 more)
Mental health challenges can exacerbate physical symptoms and complicate management of chronic conditions. Purpose-built artificial intelligence (AI) tools may offer scalable support for co-occurring mental and physical health concerns. This cross-sectional study compared past-6-month healthcare utilization, chronic condition management, physical health behaviors, mental health change, and workplace functioning between active (n = 169) and non-users (n = 73) of a mental health AI (Ash). Participants had at least one chronic condition (e.g. hypertension, chronic pain). Binary outcomes were modeled as adjusted risk differences (RDs) using linear probability models and continuous outcomes were modeled with linear regression; all models were adjusted for hypertension. Relative to non-users, active users were more likely to report improved mental health (61.4% vs. 34.3%; RD = 0.27), higher medication adherence (91.7% vs. 76.4%, RD = 0.15), fewer skipped or delayed chronic-condition care activities (b = -0.44), and were less likely to report repeat urgent care visits (9.5% vs. 23.3%; RD = -0.15) and monthly-or-more absenteeism (24.2% vs. 45.2%; RD = -0.20, all ps < .05). Findings provide preliminary evidence that use of purpose-built AI may be associated with positive symptom-based and utilization outcomes for those managing co-occurring mental and physical concerns.
Read the full abstract ▾
All authors Kristen M. Van Swearingen, Thomas D. Hull, Jeffrey Swigert, Caitlin A. Stamatis
Human-Computer Interaction · Cross-listed from cs.CV
ArmPoser: Real-Time, Calibration-Free Arm Pose Estimation from Smartwatch IMU
Bishnu Dev, Vasco Xu, Xi-Aan Loh et al. (+3 more)
Arm pose estimation enables applications in fitness, extended reality input, rehabilitation, and life logging. Prior smartwatch-based approaches rely on calibration poses and preprocessing pipelines that transform raw IMU measurements into standardized training formats. These steps hinder deployment in everyday settings and introduce errors due to imperfect calibration and sensor drift. We present ArmPoser, a calibration-free arm pose estimation system using a single smartwatch IMU. Our central contribution is training models directly in the reference frame native to consumer smartwatches, aligning learning with how IMU data is produced by deployed devices. By operating on device-native axes, ArmPoser removes the need for coordinate transformations, explicit
alignment, and bone-offset calibration used in prior work. We further augment training with physically grounded variations in watch placement and arm morphology to account for user-specific variability. ArmPoser also includes a wear-configuration module that infers anterior or posterior forearm placement and crown orientation. We evaluate pose estimation on public
benchmarks and on a 10-participant, 30-activity study using watchOS and Android smartwatches, where ArmPoser matches or exceeds calibrated baselines without any user calibration.
Read the full abstract ▾
All authors Bishnu Dev, Vasco Xu, Xi-Aan Loh, Chenfeng Gao, Henry Hoffmann, Karan Ahuja
Human-Computer Interaction · Cross-listed from cs.RO
A Controlled Comparison of Manual and Teleoperated Intraocular Instrument Motion for an Input Device
Korab Hoxha, Mirza Imamovic, Angelo Henriques et al. (+1 more)
Input devices for robotic microsurgery are frequently described as preserving the surgeon's trained technique, but the claim is rarely measured. We compared manual and teleoperated intraocular instrument motion with the trocar constraint, the instrument, the eye model and the tracking source common to both conditions, so that the control interface was the only factor varied. Prior comparisons cannot hold the instrument fixed, because a robotic instrument is not the tool used manually. Sixteen participants performed a navigation task on a commercial ophthalmic simulator by hand and through a three-degree-of-freedom input device commanding a five-joint
robot. Task outcome was equal but at ceiling: every participant acquired all five targets under both interfaces with no retinal or lens injury. Execution differed on every measure. Teleoperated trials took three times as long at a quarter of the median speed, covered less than half the angular working range, and were broken into 3.5 times as many separate movements. Completion time and movement fragmentation improved substantially across four trials of practice and had not plateaued; the measures set by the configured rate ceiling and joint limit changed the least. Finger activity doubled and pinch variability tripled, so reducing instrument degrees of freedom redistributed manual effort rather than reducing it. The interface preserves the outcome and reshapes the execution.
Read the full abstract ▾
All authors Korab Hoxha, Mirza Imamovic, Angelo Henriques, M. Ali Nasseri
Human-Computer Interaction
Rescuing Performance from the Demo: Co-Designing Drum Gesture Mappings with a Percussionist
Jordie Shier, Teresa Pelinski, Charalampos Saitis et al. (+2 more)
Augmenting instruments with sensors and neural network mappings is a well-explored digital musical instrument design approach. While augmentations can create new expressive opportunities, they also exert aesthetic influence and can constrain musicians' gestural
language, which, if left unchecked, can lead to technological capture. To examine this, we conducted a study with a professional percussionist, co-developing a gesture mapping toolkit and recording a ten-track album. Drawing on the concept of productive dissonance, our study aimed to hold the musician's aesthetic in tension with technological constraints. This, along with a practice-based reflective approach, supported the development of a continuous gesture recognition method for percussive mapping and surfaced insights into the design process. We identify knowing-when as a form of tacit knowledge that supported productive dissonance, and raise an open question: absent a musician's broader social context, how do we know whether a technology's influence is genuinely supporting their practice?
Read the full abstract ▾
All authors Jordie Shier, Teresa Pelinski, Charalampos Saitis, Andrew Robertson, Andrew McPherson
Human-Computer Interaction
Visualizing Colonial Regimes: A Multi-View Approach to (Historical) Political Transformation
Nicole Husemann, Steffen Kailitz and Christofer Meinecke
This paper takes a critical approach to visualization of colonial regimes. Drawing on critical hermeneutics and postcolonial theory, we created interactive visu- alizations revealing the constructed nature of colonial categories and exposing the hierarchies between empires and territories. Using the Varieties of Political Regimes
dataset, our approach combines temporal flow visualization and geo- graphic distribution mapping. This allows us to contextualize colonial rule within broader patterns of political change. Our visualization challenges conventional, static, and isolated representations of colonial data by making the interpre- tive frameworks underlying colonial categorization visible through coordinated temporal, spatial, and relational views.
Read the full abstract ▾
Human-Computer Interaction
Enhancing Communication in Speech Therapy: Exploring the Cognitive Synergy Between Gesture and Speech
Paul-Peter Arslan and Xiao Xiao
This paper examines the adaptation of a rhythm-based interface, originally designed for manual dexterity rehabilitation, for use in
speech therapy. The interface allows users to control synthesized vocal phrases through finger tapping, leveraging the cognitive link between gesture and
speech. Through interviews with four therapists and pilot tests with one
speech therapist and eight children with
speech impairments (autism, Down syndrome, verbal apraxia, dyslexia), we found that the interface improves motivation and therapeutic outcomes by facilitating more interactions between verbally challenged patients and the therapist. Our findings suggest that rhythmic gestures can enhance verbal communication, offering potential for broader therapeutic and educational applications.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.SD
Noise Adaptive Streaming Audio-Visual Speech Token Enhancement for Robust Full-Duplex Spoken Dialogue Models
Bella Godiva, Yeonju Kim and Yong Man Ro
Full-duplex spoken dialogue systems enable simultaneous listening and speaking, but their audio-only perception often fails under background noise and overlapping
speech, leading to incoherent responses. Recent audio-visual dialogue approaches show that incorporating visual cues such as lip movements improve robustness under audio corruption. However, existing approaches often adapt the large
speech dialogue model itself to process visual input, requiring costly
multimodal training. We propose AV-STE, a modular streaming audio-visual front-end that restores corrupted semantic
speech tokens from noisy audio and lip
video before they reach the
speech LLM. The downstream dialogue model remains entirely frozen, preserving its pretrained conversational capabilities. When integrated with frozen Moshi, AV-STE improves average GPT-4o-judged response coherence from 1.42 to 1.91 under same-
dataset speaker interference while largely preserving turn-taking behavior. Gains also transfer to out-of-domain Seamless Interaction.
Read the full abstract ▾
Human-Computer Interaction
"Here Be Sharks!": Enhancing Scientific Communication and Analysis through Authoring Interactivity
Caroline Berger, Josh Pollock, Dylan Wooton et al. (+2 more)
We report on a case study for designing authoring environments for interactive visualizations to enhance scientific work. We conducted a workshop and prototype review with a group of marine biologists. When it came to visualizing their data, participants identified challenges in conveying their research accurately and completely as well as and analyzing it with ease. Based on our findings, authoring environments should consider the context of scientific work aspects such as domain expertise, collaboration culture, and publication traditions. We propose consideration of non-traditional programming
languages and environments for scientific work, discuss ways of facilitating interactive visualizations for scientists, and examine ways of meaningfully integrating AI. The critiques, artifacts, and reactions from the scientists along with our analysis and discussion inform how computational tools should be designed for scientific work.
Read the full abstract ▾
All authors Caroline Berger, Josh Pollock, Dylan Wooton, Arvind Satyanarayan, Clemens Nylandsted Klokmose
Human-Computer Interaction · Cross-listed from eess.SP
Understanding emotional and behavioural responses through physiological signal analysis recorded during theatrical performance
Aditi Site, Annariina Lohiranta and Tarmo Lipping
Among various cultural interventions, theatre offers a powerful medium for emotional expression, reflection, and social connection. Live performances can positively influence mental well-being. Therefore, studying audience responses during theatrical performances can provide valuable insights into the relationship between theatre, emotional experiences, and well-being. Physiological responses have emerged as an important modality for identifying patterns associated with different emotional states. However, in naturalistic environments such as live performances, analysing these signals is challenging due to the absence of explicit labels. In this study, we focus on understanding emotional and behavioural responses using physiological data collected during a live theatrical performance. Data were recorded from participants using multiple sensing modalities. In addition, a comprehensive analytical framework was employed, which included demographic analysis, visual exploration of physiological signals, participant-level observations, scene-wise physiological analysis, and intensity-based analysis using z-score normalization to summarize the findings. The findings revealed that dramatic, surprising, and intense scenes produced the strongest activation, often with concurrent increases in HR and SCL, whereas empathy-driven scenes showed lower and more stable responses. Humorous scenes elicited moderate-to-high activation, particularly in the presence of sudden reactions or loud sounds. Analyses also identified shared physiological response patterns across participants, with activation concentrated around key emotional events. These results indicate that combined HR and SCL measures can effectively capture variations in audience engagement during live theatrical performances.
Read the full abstract ▾
Human-Computer Interaction
StitchOver: Technical Embroidery on Seamed Fabrics
Zekun Chang, Tianhong Catherine Yu, Yixuan Gao et al. (+1 more)
Smart textiles embed interactivity into everyday garments, supporting use cases like always-available sensing for medical applications or sports. Machine embroidery allows integrating functionalities into existing textiles. However, embroidering onto real-world textile goods remains challenging. Textile goods are rarely made of a single homogeneous substrate of fabric, and embroidery with functional materials such as conductive threads requires machines to be more tightly calibrated than for decorative embroidery. In particular, seams, which bring together different substrates, along with machine variability, cause shifts in tension and friction between the functional thread and the textile substrate that frequently lead to defects (70% of samples in our evaluation). We present a technique to reliably embroider on seamed fabric even when using functional threads. Our software tool automatically digitizes user-defined stitch patterns by introducing what we call "JumpStitches" to bypass seam interference. We evaluated our approach under varying machine states (under-tensioned, well-calibrated, and over-tensioned), and across multiple seam and pattern configurations. Our results show that the JumpStitch mechanism eliminates defects, while maintaining conductivity compared to 70% defects without JumpStitches, and even in poorly calibrated machine states continues to work well.
Read the full abstract ▾
All authors Zekun Chang, Tianhong Catherine Yu, Yixuan Gao, Thijs Roumen
Human-Computer Interaction · Cross-listed from cs.CV
DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting
Yuhang Wang, Chuheng Wei, Jingxin Yang et al. (+2 more)
Driver motion can provide cues to ongoing behavior,
attention, and near-term driving intent. However, most existing driver-centric
datasets focus on recognizing predefined driver behaviors from short
video clips, while human motion forecasting
benchmarks largely target motion outside the vehicle. We introduce DriveMotion, a multi-source
benchmark for continuous driver motion forecasting. DriveMotion contains 393 hours of 133-keypoint motion sequences at 10 Hz from 360 drivers, integrating naturalistic driving data, curated public in-cabin
videos, and the AIDE
dataset into a unified representation with per-joint validity masks and synchronized driving context. Naturalistic driving contains long periods of limited body movement, making uniformly sampled evaluation dominated by persistence and less sensitive to brief but behaviorally meaningful motion. To address this, we use dynamics-anchored evaluation, placing forecasting windows around vehicle maneuvers identified offline from CAN signals without providing CAN to the model at inference. Arm motion in pre-maneuver windows is 3.4x greater than in route-matched stable-driving controls. On these anchored windows, learned models reduce forecasting error over persistence by up to 15%, while maneuver-enriched training improves forecast-derived Part-State F1 by 44% over the zero-motion reference. Training on the full multi-source corpus further reduces forecasting error on held-out web drivers by 38% compared with BATON-only training. DriveMotion provides identity-disjoint splits, fixed evaluation subsets, and reference implementations for reproducible evaluation of continuous driver motion forecasting. The
dataset and
benchmark are available at https://huggingface.co/
datasets/HenryYHW/DriveMotion
Read the full abstract ▾
All authors Yuhang Wang, Chuheng Wei, Jingxin Yang, Xishun Liao, Hao Zhou
Human-Computer Interaction · Cross-listed from cs.AI
Artificial Intelligence-Assisted Digital Inventory of Cultural Heritage & Traditional Knowledge: Case for Indonesian Open Digital Library of Culture
Hokky Situngkir
The Indonesian Digital Library of Culture (Perpustakaan Digital Budaya Indonesia, PDBI; budaya-indonesia.org) is a participatory platform that has collected tens of thousands of entries on Nusantara cultural heritage through public contribution since 2007. Manual contribution faces three structural barriers: coverage (knowledge is scattered across
languages and sites), integrity (open sources mix authentic documentation with noise), and completeness (subjects are recorded but their data remain shallow). This paper presents a methodological framework for autonomous, AI-based harvesting of cultural knowledge from the open web, designed to expand corpus coverage while intensifying per-entry data depth. The methodology is organised as a five-stage economic funnel: focused crawling, multilingual extraction and canonicalisation, vector encoding with blocking, agentic decision-making, and idempotent publication, under the principle of deterministic orchestration, agentic decisions. Each stage is formalised: funnel economics and optimal filter ordering; crawl-frontier dynamics as a subcritical branching process that explains the necessity of recurrent re-seeding; fact-level novelty via a containment measure;
Bayesian multi-source evidence fusion with elevated publication thresholds for sacred categories; exactly-once effects via idempotent upserts and the transactional outbox; sliding-window inference budgeting with a reservation protocol; statistical quality auditing; and seed selection as submodular coverage maximisation. The framework retains four high-value human roles: curator of direction, escalation approver, quality auditor, and guardian of meaning, while machine autonomy is raised in stages. Ethical, legal, and cultural-sensitivity implications are discussed, including the architectural guarantee that the machine never overwrites human contributions.
Read the full abstract ▾