Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Human-Computer Interaction

cs.HC Watch Human-Computer Interaction

Sunday 27 September

Human-Computer Interaction

Separating Memory and Workflow Effects in Predicting Individual Answers

Tianzhu Qin, Leo Yang Yang, Ramit Debnath et al. (+1 more)

Language agents choose what to remember about a person and how to use that memory. We separate these choices when predicting a person's unseen answer to an interview question. On 1,768 tasks from 188 people, a concrete memory from a verified interview prefix outscores a trait description by 0.0158 (95% interval [0.0044, 0.0271]). Crossing both memories with one-shot generation and three-answer fusion, fusion lowers concrete-memory scores by 0.0123 ([-0.0189, -0.0056]); prompted and trained selectors do not detectably beat a random candidate, and one call on the longer, unrewritten record outscores every memory condition. At matched context budgets, OwnWords, one call on the person's BM25-ranked sentences, outperforms the written memory on 500 people outside the benchmark (+0.0127, [+0.0037, +0.0217]; an earlier held-out test was inconclusive) and across four budgets on 300 people (mean +0.0218, [+0.0138, +0.0298]), but does not detectably outperform recency truncation; these comparisons do not isolate verbatim wording. On a survey benchmark it predicts ordinal answers more closely than the memory but is not more accurate on exact choices (less accurate in one of two screener samples). Interview scores use a model-based content rubric without human ratings, and all benchmark and some confirmation participants were seen during development.
Read the full abstract

All authors Tianzhu Qin, Leo Yang Yang, Ramit Debnath, Davin Youchao Dong

Human-Computer Interaction · Cross-listed from cs.RO

Beyond Tasks: A Vision for Reproducing an Animal-like Behavioral Substrate Using Modern Robot Learning Techniques

Samiyuru Menik and Hemadri Jayalath

Recent advances in robot learning have produced increasingly capable embodied agents. Yet comparatively less attention has been given to a more basic form of competence that animals exhibit continuously: the ability to remain situated, responsive, and behaviorally coherent as physical, environmental, and social demands change over time. We propose the ethological behavioral substrate as a conceptual lens for studying this form of competence in artificial agents. Rather than treating these behaviors that animals exhibit as a set of isolated skills, we argue that their continual coordination under competing demands constitutes an important and underexplored target for modern robot learning. We further propose robotic animal companions as a useful research setting for studying sustained interaction and adaptation in human-centered environments. Such systems provide an opportunity to investigate how social behavior, memory, and continual learning develop over long periods of interaction. This perspective motivates further investigation of how such persistent behavioral competence may complement higher-level capabilities in embodied agents.
Read the full abstract

Human-Computer Interaction

PDFa11yMut: Measuring Mutation-Specific Detection in PDF Accessibility Checkers

Gauri Jain

Automated PDF accessibility checkers provide useful conformance evidence, but a clean report is not a complete accessibility oracle. PDFa11yMut measures mutation-specific checker behavior by applying paired structure-level transformations to reference-suite baselines, verifying intended deltas and non-target invariants, and recording hash-linked checker evidence. Across 30 conformance-oriented mutants, PAC and veraPDF each produced 30 and 30 direct target findings, respectively, while Acrobat produced 23 direct findings plus 3 prespecified consequence-proxy findings. The 39 semantic/assistive-representation mutants produced no automated target finding in the tested configurations, while Acrobat issued manual-review prompts for a subset; Class B is interpreted descriptively rather than as a universal checker obligation. A separate convenience-selected exploratory AT sample observed representation differences in 8 of 9 pairs under one fixed NVDA/Acrobat/Windows procedure. The artifact contributes reusable operators, structural and purity oracles, paired baseline/mutant evidence, and reproducible analysis for a scoped mutation-testing study rather than a general checker-accuracy benchmark.
Read the full abstract

Human-Computer Interaction

ParallelPilot: Supporting Coordination and Monitoring in Parallel AI Coding

Tao Long, Weili Shi, Hussein Mozannar et al. (+2 more)

As coding assistants become increasingly autonomous, developers run multiple sessions in parallel, shifting the challenge from code generation alone to coordinating and monitoring concurrent agent work. Through a formative study (N=14), we identified PILOT: five supervisory practices for Planning, Isolating, Logging, Observing, and Triaging parallel sessions. We present ParallelPilot, a design probe that instantiates PILOT through a planning interface, a run-logger, and an ambient dashboard alongside existing coding tools. In a counterbalanced within-subjects study (N=16), participants using ParallelPilot increased ticket throughput by 63% in short coding tasks and supervised an average of one more concurrent agent at peak, while their tracking effort and context switching dropped. ParallelPilot also clarified execution plans, task dependencies, and intervention cues, and 14 of 16 participants preferred it over their current setup. These gains were not accompanied by significant improvements in perceived control or perceived success in redirecting the agents. Our findings demonstrate the value of explicit supervision support and position PILOT as a scaffold for designing tools that help people supervise concurrent work within and beyond coding. We suggest that future coding assistants should pair high-level awareness with low-cost paths back to the implementation evidence developers need to judge and steer agent work.
Read the full abstract

All authors Tao Long, Weili Shi, Hussein Mozannar, Maya Murad, Rafah Hosn

Human-Computer Interaction

Medical Knowledge Is Not All You Need: When Medical Q&A Becomes Situated Patient Assistance

Shreya Bali, Riku Arakawa, Jill Fain Lehman et al. (+8 more)

Reliability in medical Q&A is often pursued by grounding responses in authoritative medical information. We show that when Q&A is embedded within ongoing care, reliability depends on more than what the system knows medically. In a study with 73 skin cancer patients practicing postoperative wound care, 41.9% of response-requiring questions depended on information beyond the procedure, including visual or physical state, environmental context, or prior actions. These demands varied across patients, consistent with patients recruiting the assistant into different informational roles. We then replayed the questions to seven LLMs while adding procedural and postoperative guidance. Errors remained substantial, including treating unknown states as known, even under explicit guardrails; with full procedural context, six of seven models more often introduced later steps prematurely. Based on these findings, we propose a design space for situated medical assistance that connects what the assistant and patient can each reliably establish to the form of assistance provided.
Read the full abstract

All authors Shreya Bali, Riku Arakawa, Jill Fain Lehman, Alexander K. Maytin, Brian Chen, Emma Russell, Haarika Reddy, Annalise Vaccarello, Dustin P. DeMeo, Bryan T. Carroll, Mayank Goel

Saturday 26 September

Human-Computer Interaction

Smelling the Way: Olfactory Modulation of Spatial Estimation and Path Integration in Virtual Reality

Siyeon Bak, Junho Kim, Dongyun Han et al. (+2 more)

Spatial cognition enables individuals to perceive and interpret spatial relationships, estimate locations, and navigate within their surroundings. While vision plays a dominant role, other sensory modalities, particularly olfaction, can support spatial processing when visual cues are limited. This paper explores the effects of scent-delivery within virtual reality (VR) environments by using fan-mediated olfactory devices. We conducted two formal studies to evaluate the effects on spatial information acquisition. Study 1 (N=24) examined the participants' ability to localize a scent source while walking a linear path using two different olfactory devices. Study 2 (N=19) employed a triangle completion task, in which participants attempted to return to their starting point under varying rotation angles and olfactory conditions. Results indicate that scent-delivery feedback influenced spatial behavior in distinct ways across the two studies. In Study 1, visual-olfactory misalignment produced systematic directional biases without improving absolute localization accuracy. In Study 2, distance-modulated scent-delivery feedback reduced terminal homing error but also increased traveled distance and completion time, suggesting a more deliberate goal-confirmation strategy.
Read the full abstract

All authors Siyeon Bak, Junho Kim, Dongyun Han, Sun-Jeong Kim, Isaac Cho

Human-Computer Interaction

VocalEyes: Speaker-Aware Augmented Reality Captioning through In-Conversation Registration

Yuxiao Wang, Xulong Tang, Chen Chen et al. (+1 more)

Co-located augmented reality (AR) captions make speech readable, but they can separate an utterance from the person who produced it. In unfamiliar groups, losing that source complicates immediate responses and later review: users must recover not only what was said, but also who said it. Conventional diarization returns anonymous clusters, while speaker recognition typically assumes pre-meeting enrollment. We built VocalEyes, a speaker-aware AR captioning system that creates named voice profiles from natural self-introductions. The interface coordinates speaker-attributed captions, a fixed profile card, and a visual cue that marks the articulating face. In a controlled within-subjects study with 20 participants who reported typical hearing, VocalEyes identified speakers with 88.0% accuracy and increased participant speaker-tracking accuracy from 47.2% with caption-only AR to 87.3% with the complete interface. Participants also reported lower workload with the complete interface. These findings show how in-conversation registration can preserve speaker attribution across live captions and meeting records in scripted small-group meetings.
Read the full abstract

All authors Yuxiao Wang, Xulong Tang, Chen Chen, Rawan alghofaili

Human-Computer Interaction

Exploring the Effects of Olfactory Cues and Ventilation on Teleportation-based Navigation in VR

Dongyun Han, Heecheol Kim, Siyeon Bak et al. (+3 more)

Spatial cognition supports how people interpret spatial relationships and navigate their surroundings. Although vision plays a dominant role, other sensory modalities, including olfaction, may also contribute under limited visual conditions. This paper investigates the role of olfactory cues in location recognition during virtual reality (VR) navigation. We conducted two formal studies using a wearable olfactory prototype. Study 1 examined whether olfactory cues and ventilation influenced users' recognition of encountered locations during \HDY{teleportation-} and dash-based navigation. Study 2 extended this investigation to repeated navigation and examined whether ventilation duration influenced recognition performance and user experience across successive movements. Results show that olfactory cues can support recognition of encountered locations during navigation, while ventilation was associated with reduced residual interference and improved usability. However, longer ventilation did not clearly improve recognition accuracy in repeated navigation. These findings suggest the potential of olfactory cues as contextual signals during VR navigation, while also highlighting the importance of scent management for maintaining perceptual clarity and user comfort.
Read the full abstract

All authors Dongyun Han, Heecheol Kim, Siyeon Bak, Chang-Guen Song, Sun-Jeong Kim, Isaac Cho

Human-Computer Interaction · Cross-listed from cs.AI

CLAIRE: A Schema-Grounded Hybrid Workflow for Healthcare Administrative Form Completion

Garapati Keerthana and Manik Gupta

Healthcare administrative staff transfer structured information from electronic health records, referrals, claims systems, provider rosters, and work queues into dynamic forms. We developed and evaluated CLAIRE (Clinical Language and Agentic Intelligence for Reasoning and Entry), a hybrid workflow that separates field-state discovery, source-to-field mapping, deterministic validation, bounded correction, escalation, and audit tracing. We tested five synthetic healthcare administrative schemas, 1,000 source records, four interface variants, two data-quality suites, and six comparators, yielding 24,000 benchmark episodes. A separate strict-output audit evaluated direct mappings from Qwen2.5-1.5B and Qwen2.5-7B, and a trace-derived operational simulation covered 6,000 episodes. Under the evaluated synthetic benchmark conditions, full CLAIRE achieved 1.000 episode success, field accuracy, required-field completion, and dependency completion in both suites; removing validation reduced stress-suite success to 0.500. In the simulation, 100.0% of clean and validation-stress episodes reached a staff-reviewable draft, compared with 68.6% of escalation challenge episodes, unsupported cases were blocked. Scenario-based savings were 149.7-165.5 seconds per case, not observed staff times. The findings support schema-grounded, validation-first healthcare administrative automation in which language-model components assist mapping but do not authorize unsupported or consequential actions.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CR

You Can't Spot a Deepfake?And Neither Can Your Brain Nor Eyes: A Neurophysiological Framework for Deepfake Exploitation of Cognitive Engagement and Implicit Visual Evaluation

Cagri Arisoy, Md Imanul Huq, Amy W. Hays et al. (+1 more)

Deepfakes have rapidly emerged as a pressing threat to information integrity and security because they exploit human trust in visual and auditory perception. Yet, little is known about whether humans and their underlying (sub)conscious neuro-physiological processes can reliably distinguish deepfake from real videos. We introduce DECEIVE (Deepfake Exploitation of Cognitive Engagement and Implicit Visual Evaluation), a framework that models how deepfake videos are validated as adversarial payloads through behavioral and neuro-physiological screening of viewers, and how attacks can be refined by selecting payloads that evade detection. The framework is dataset agnostic and applies to synthetic or real media. It is inherently dual-use: an adversary with equivalent measurements could iterate on candidate manipulations and retain those that evade human detection. This motivates open, defensive evaluation. Measuring which deepfakes defeat human perception establishes a realistic bound on attacker capability against which detection tooling, provenance and watermarking mechanisms, and user-facing protections can be assessed. As an instantiation, we conducted an EEG and eye-tracking study in which participants viewed real, deepfake, and look-alike videos drawn from Celeb-DF and a curated celebrity set, while behavioral judgments and implicit responses were recorded. Contrary to expectations of subconscious differentiation suggested by prior work on paintings and phishing websites, no statistically significant neuro-physiological differences emerged between real and deepfake videos, although clear distinctions were observed for look-alike videos. Behaviorally, participants accepted 26.68% of manipulated clips as authentic, rising to 31.94% for familiar identities, confirming the studied deepfakes as effective adversarial payloads within DECEIVE.
Read the full abstract

All authors Cagri Arisoy, Md Imanul Huq, Amy W. Hays, Nitesh Saxena

Human-Computer Interaction

Chatbot Engagement Does Not Always Beget Metalearning: Evidence from Three Countries

Kokil Jaidka, Insyirah Binte Imam Mujtahid, Peng Qi et al. (+5 more)

Chatbots deliver real-time fact-checks, but whether a chatbot correction leaves anything behind once the chatbot is gone - metalearning, distinct from correcting misbeliefs - is untested. We report a preregistered, three-country randomized experiment (USA, India, Singapore; N ~ 2,200) on out-of-context image misinformation, manipulating a correction's channel affordances (synchronicity, bandwidth) across four conditions: Control, Links-only, Static explanation, and a Socratic Chatbot built on a validated out-of-context detector, with an unaided retest one week later. The Chatbot produced the largest immediate discernment gain (d = 0.097, p = .023). All three interventions reduced sharing of false claims (d ~ -0.12, p < .01). One week later, no advantage persisted: the Chatbot arm declined relative to Control, most sharply in India and Singapore, and in India on claims it never discussed. Decay tracked affordance level and did not vary by country. Engagement mechanisms, we argue, do not substitute for slow AI literacy.
Read the full abstract

All authors Kokil Jaidka, Insyirah Binte Imam Mujtahid, Peng Qi, Harshit Aneja, Subhayan Mukerjee, Wynne Hsu, Mong Li Lee, Tsuhan Chen

Human-Computer Interaction

When the Environment Becomes the Interface: Multisensory Environmental Interfaces for Human-AI Interaction in Autonomous Vehicles

Keqi Chen, Runjia Tan, Xinyi Fu et al. (+3 more)

As AI increasingly assumes operational control, human-computer interaction is shifting from operating systems through explicit interfaces to inhabiting intelligent environments. This raises a fundamental question: when users no longer directly manipulate a system, what mediates their relationship with intelligent technologies? We introduce environmental interfaces: designed environmental conditions that shape human-AI relationships through ambient, holistic, and evaluative pathways rather than explicit functional interaction. Using autonomous vehicle cabins as a revealing context, we conducted a within-subject experiment with 24 participants (216 observations), manipulating lighting and scent in a simulated autonomous driving environment. Three findings emerged. First, perceived atmosphere accounted for 66.5% of the variance in overall journey experience, showing that environmental conditions can function as an interface rather than merely a supporting design element. Second, multisensory processing followed a hierarchical architecture: individual sensory appraisals were initially independent, while cross-modal integration emerged during higher-order environmental evaluation. Third, olfactory stimuli influenced affective responses and experience evaluation more strongly than visual stimuli, challenging the visual dominance of automotive interaction design. Environmental quality and sensory congruency also predicted trust in the autonomous system, suggesting that passengers may use environmental cues as proxy signals when direct assessment of AI competence is difficult. These findings establish environmental interfaces as a distinct interaction modality and point to a broader transition from designing interfaces for operating intelligent systems to designing environments for inhabiting them.
Read the full abstract

All authors Keqi Chen, Runjia Tan, Xinyi Fu, Shanhe Lou, Kwan Min Lee, Chen Lv

Human-Computer Interaction · Cross-listed from cs.AI

Artificial intelligences and human scientists exhibit complementary strengths in theory building

Ke Li, Spyros I. Zoumpoulis, Phanish Puranam et al. (+80 more)

We investigate the effectiveness of artificial intelligences (AI)-specifically large language models (LLMs)-relative to human scientists at high-level cognitive tasks in social science such as theory formulation, predictions of novel empirical results, and theory revision in response to new evidence. The research domain was academic discourse regarding gender and race inequality. Our findings, comparing 25 LLMs with 13 senior researchers and 60 doctoral scholars, reveal that the AIs outperformed most humans individually on most of the present tasks, while human theories were more diverse and exhibited greater gains in predictive accuracy from aggregation. AI-generated theories were more extensively elaborated, involving additional theoretical paths and latent variables, and were rated as higher quality than human theories by independent raters blinded to source. However, this theoretical complexity was in part ornamental, in that it was not associated with more accurate predictions about empirical patterns in data; in contrast, human scientists achieved greater predictive efficiency with simpler theories. The AIs were significantly more likely than human scientists to revise their theories to incorporate new evidence; human scientists updated their beliefs in a selective way that is sensitive to prior prediction errors. We speculate that the superior processing capacity of artificial intelligences makes them especially well-suited to tasks requiring grappling with complexity, but that the greater diversity of human ideas is essential to wise crowds and collective creativity.
Read the full abstract

All authors Ke Li, Spyros I. Zoumpoulis, Phanish Puranam, Philip Parker, Matthew Eshbaugh-Soha, Izzy Gainsburg, Michael Gilead, Igor Grossmann, Britt Hadar, Yoel Inbar, Almog Simchon, Robb Willer, Rui Ai, Ruicheng Ao, Gavin J. Bala, Matthew Bidwell, Shuang Cai, Kai Chang, Skyler Y. Chen, Cory J. Clark, Irmak Dai, Abhinandan Dalal, Connor Douglas, Alexis Du, Zhehang Du, Leyun Feng, Isabel Fernandez-Mateo, Linnea Gandhi, Cyrille Grumbach, Anmol Gupta, Vansh Gupta, Maria Hademer, Jay H. Hardy III, Chen Kai Huang, Jacob Xiangyu Jin, Ufuk Keskin, Na Hyun Kim, Mert Kobaş, Byounghoon Koh, Gabrielle Lamont-Dobbin, Gregory Lanzalotto, Sun Young Lee, Dingzhe Leng, Chenjun Li, Weiyuan Li, Zeyuan Li, Zhongyuan Liang, Ning Liu, Peihong Liu, Yuhan Liu, Jiuyao Lu, Wanteng Ma, Nicolas Martinet, Natnael Mulat, Christina A. Nguyen, Khai Nguyen, Quang Minh Nguyen, Naja Pape, Chanwoo Park, Stefanos Poulidis, Jeffrey Sanchez-Burks, Michael Schaerer, Isabelle Solal, Yanbo Song, Junghyo Sun, Qingyao Sun, Rui Sun, Roderick Swaab, Kevin Tan, Dequn Teng, Michelle A. Vaccaro, Robin Vigerbaeck, Xiaomeng Wang, Randol H. Yao, Duygu Yilmaz, Shun Yiu, Ecem Yucesoy, Allen Zang, Ruijia Zhang, Xilan Zhang, Yichi Zhang, Zhanhao Zhang, Eric Luis Uhlmann

Human-Computer Interaction

DashAct: A Progressive Diagnostic Benchmark for GUI Agents in Interactive Dashboard Analysis

Chuhan Zhang, Qi Xie, Ziyue Wang et al. (+4 more)

Interactive dashboards require users to reveal and connect evidence across stateful interactions. Although graphical user interface (GUI) agents could automate this process, existing dashboard benchmarks primarily report final answers or task success. They provide limited insight into whether failures arise from maintaining the analytical process, selecting actions, or grounding visual targets. We introduce DashAct, to our knowledge the first benchmark to diagnose these failures at a fine-grained level within the same dashboard task. DashAct contains 357 human-verified interaction trajectories with milestone dependencies and hierarchical target annotations. Its progressive diagnostic cascade evaluates end-to-end execution, restores verified context for next-action prediction, and provides target semantics and a local view for visual grounding. By progressively restoring the conditions for success, DashAct measures the minimum support an agent needs to recover rather than scoring isolated skills. Experiments show that current models struggle even as support is added. The cascade outcomes reveal bottlenecks hidden by end-to-end scores and provide actionable guidance for improving GUI agents.
Read the full abstract

All authors Chuhan Zhang, Qi Xie, Ziyue Wang, Jianing Yin, Yunfan Zhou, Dazhen Deng, Yingcai Wu

Human-Computer Interaction · Cross-listed from cs.RO

WSM-Aware HRI: An IoT-Enhanced Framework for Early Detection and Norm-Guided Repair of Failures with LLM Guidance

Hanlin Zhang, Yuquan Wang, Tianwei Zhang et al. (+1 more)

Human-robot interaction (HRI) failures remain a major barrier to deploying robots in real-world environments. Prior work often treats failures as isolated technical faults or focuses on post-hoc recovery behaviors. In practice, many breakdowns arise because humans and robots operate under inconsistent assumptions about the current world state. We propose WSM-Aware HRI, an IoT-enhanced modular framework that unifies diverse HRI breakdowns as World-State Mismatches (WSMs) between a human's instruction-implied assumptions and a robot's grounded world model built from multimodal perception and digital augmentation. A Large Language Model (LLM) is used to make implicit assumptions explicit, map them to a small set of mismatch types, and specify the evidence needed for verification against the robot's world state. WSM-Aware HRI shifts failure handling from execution-time recovery to proactive mismatch detection during intention formation, enabling interventions guided by safety, norm compliance, and multi-user coordination with transparent explanations. We evaluate mismatch identification in ten everyday cases spanning both visual and latent-state mismatches. The system can accurately produce the expected output results, and ablations show that reliable identification depends on appropriate grounding representations and verification-oriented refinement. These results indicate that treating interaction breakdowns as explicit world-state mismatches enables earlier detection of impending failures and offers a principled mechanism for integrating external evidence and social constraints into human-robot interaction.
Read the full abstract

All authors Hanlin Zhang, Yuquan Wang, Tianwei Zhang, Zhenglong Sun

Human-Computer Interaction · Cross-listed from cs.LG

Handwritten Digit Leakage from Smartphone Motion Sensors Across Unseen Users and Phone Models

Constantino Álvarez Casado, Erkka Rantahalvari, Matteo Pedone et al. (+6 more)

Smartphone motion sensors support interactive applications, but their readings may also reveal touchscreen input beyond their intended use. Assuming known drawing intervals, we study whether handwritten digits remain predictable across users and devices, as a 10-class problem on 19,628 HuMIdb recordings from 481 participants. We compare handcrafted features with classical machine learning algorithms, MiniRocket kernels, and a compact sensor patch transformer on accelerometer, linear acceleration, gyroscope, and gravity signals. The transformer achieves 57.74% accuracy and 82.64% top-3 accuracy on 75 unseen participants, and 58.77$\pm$0.95% over 3 seeds for unseen participants on 9 unseen phone models. Low motion recordings remain informative, accuracy is not monotonic in motion level, and the tested contrastive pretraining, augmentation, and derived signals give no consistent gains. Digits are thus predictable beyond familiar users and phone models under assumed segmentation, while acquisition-order shortcuts limit conclusions about practical privacy exposure. Code available at: https://github.com/Arritmic/motion-digit-leakage.
Read the full abstract

All authors Constantino Álvarez Casado, Erkka Rantahalvari, Matteo Pedone, Matti Matilainen, Manuel Lage Cañellas, Le Nguyen, Simo Hosio, Olli Silvén, Miguel Bordallo López

Friday 25 September

Human-Computer Interaction · Cross-listed from cs.CY

TeamLens in Critsly: A Consent-Based Team-Composition Interface and Synthetic Readiness Evaluation for Design Collaboration

Nizam Kadir

Discussing working preferences may support reflection within a design team, but a personality label should not become a performance prediction or a condition of participation. This technical report presents TeamLens, an optional Critsly interface for voluntarily sharing a self-reported MBTI type with a particular board. It separates account activation from disclosure, displays descriptive composition counts, and provides distinct controls for disabling visibility, withdrawing one report and deleting all of one's reports. The evaluation combines released-source inspection, independently specified synthetic aggregation cases, access and lifecycle checks, browser component tests, a local database microbenchmark and deployment records. All 65,536 binary eligibility subsets of sixteen fixed, distinct type reports matched an independent oracle; 128 seeded multiplicity fixtures also matched, after 4,259 synthetic share calls. The isolated HTTP suite passed 183 assertions. In 180 sequential in-memory SQLite read trials, median service-call time increased from 0.064 ms with no profile rows to 200.932 ms with 1,024 rows; instrumented SQL operations followed 11+3n. These findings concern exercised software behaviour and a bounded workload. They do not establish human usability, psychometric validity, learning gains, team-performance effects or production capacity. The contribution is an implemented disclosure-to-aggregation workflow and an auditable technical account of its correctness boundaries, privacy limitations and scaling cost. OpenAI Codex assisted with implementation, evaluation and manuscript preparation; the paper discloses this use and its limits.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.SE

CG-Diff: Organizing Code Changes Around Call Graphs

Bimal Raj Gyawali, Devamardeep Hayatpur and Nadia Polikarpova

Tools for code review present code changes file-by-file. We argue that, oftentimes, changes can be better organized around a call graph of the changed code. From an empirical study of GitHub pull requests (PRs) in-the-wild, we find that (1) in around 40% of the PRs, more than half of the changed functions (and methods) are connected by call graphs, and (2) necessary callees are often located in different files from their callers. Based on these findings, we develop the notion of CG-Diffs, subgraphs derived by decomposing a call graph of the changed code into smaller and more navigable directed graphs. We then implement a web-based interface for viewing PRs that restructures the PR around these CG-Diffs. Through a within-subject study comparing our interface against GitHub's PR view, we find that CG-Diffs help orient participants and provide them with more meaningful structures to navigate the PR. We also found several limitations: function nodes repeated across multiple CG-Diffs can be disorienting, and changes not contained in a function (e.g. globals, imports) are not as immediately apparent. Our study shows promise in using call graphs to help contextualize and navigate unfamiliar codebases, which may benefit new contributors to open source, and reviewers of unfamiliar LLM-generated PRs.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.AI

A Benchmark for LLM's Understanding of Middle School and High School Science Topics

Noah L. Schroeder, Yessy Eka Ambarwati, Yuji Zhang et al. (+1 more)

Large language models (LLMs) are increasingly integrated into educational settings, yet educators lack robust, standards-aligned tools to evaluate their effectiveness in K-12 science contexts. Existing benchmarks predominantly assess general language or advanced scientific reasoning, leaving a critical gap in understanding LLMs' performance on content directly relevant to secondary science curricula. To address this gap, we developed a comprehensive NGSS-aligned benchmark for both middle and high school science using a rigorous synthetic data pipeline, multi-judge validation, and item-level psychometric analysis. Nine open-weight LLMs were systematically evaluated using this benchmark, indicating that several smaller, locally deployable models achieved high accuracy across diverse science domains and question types. Our findings indicate that model size did not consistently predict performance, emphasizing the importance of intentional model selection for educational deployment. We then incorporated a human reviewer into the loop, reviewing the items generated by the LLMs for alignment with NGSS standards. The human review indicated that synthetically generated items were not in perfect alignment with the NGSS standards, indicating the benefits of human-in-the-loop item development, the need to explore the intersection of content and pedagogical knowledge, and the need to extend benchmarks to evaluate LLMs' capacity for interactive, evidence-based feedback in educational scenarios.
Read the full abstract

All authors Noah L. Schroeder, Yessy Eka Ambarwati, Yuji Zhang, ChengXiang Zhai

Human-Computer Interaction

Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners

Shani C Spivak, Aditi Krishna, Mahsan Nourani et al. (+1 more)

Large language models (LLMs) are enticing in their promise to support data visualization (Vis) through faster and simpler workflows for data prep, analysis, and visualization creation. Yet LLMs are notoriously error-prone and not built for data visualization tasks. Few studies have explored LLM adoption among Vis practitioners. To fill this gap, we conducted semi-structured interviews with members of the Data Visualization Society, a global community of data visualization designers. Our findings show that Vis designers actively use LLMs for both creative and technical aspects of the visualization process. A new visualization workflow is emerging, a process we call vibe analysis, analogous to vibe coding. Some key challenges raised by participants parallel those of vibe coding, while others are Vis-specific, like gaps in Vis knowledge and chart verification. This work opens up opportunities for research combining LLM-mediated work with Vis tools that incorporate data visualization guidance, constraints, and best practices.
Read the full abstract

All authors Shani C Spivak, Aditi Krishna, Mahsan Nourani, Melanie Tory

Human-Computer Interaction · Cross-listed from eess.AS

MAESTRO: a Multimodal Auditory-attention Egocentric Speech-TRacking Open corpus

K M Naimul Hassan, Ali Alavi and Donald S. Williamson

Humans rely on gaze, head movements, and visual cues to attend to speakers in noisy environments, yet auditory attention decoding (AAD) has been studied primarily using electroencephalography (EEG). We introduce the Multimodal Auditory-attention Egocentric Speech-TRacking Open (MAESTRO) corpus, the first AAD dataset to simultaneously record EEG, eye gaze, pupillometry, egocentric video, and head inertial measurement unit (IMU) data. MAESTRO includes four competing speakers and background noise across multiple signal-to-noise ratio (SNR) conditions, enabling attention decoding under realistic listening scenarios. Through a four-speaker attention decoding benchmark, we show that combining behavioral and physiological signals improves decoding performance over EEG-only approaches, enabling future advances in multimodal auditory attention decoding. These findings open the door to new applications, analyses, and methodological advances in multimodal AAD. The complete dataset is publicly available at https://huggingface.co/datasets/aspire-osu/maestro-eeg-dataset . The official code repository is available at https://github.com/ASPIRE-OSU/MAESTRO .
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CY

Anatomy of a Spreadsheet Failure, Analysing the EuSpRIG Horror Story Corpus

Simon Thorne and Angela Collins

Spreadsheet disasters have been documented for more than thirty years, but they have not gone away. This paper examines the EuSpRIG horror stories archive as evidence of how ordinary spreadsheets fail and the consequences that follow. After cleaning and deduplication, 121 incidents from 1995 to 2026 were classified into twelve failure categories and analysed by mechanism, context and consequence. Formula errors, data-entry slips and data-handling mistakes account for 71 cases and recur across the whole period, from Fidelity's missing minus sign in 1995 to a mistyped date that cost Norway's sovereign wealth fund about $92 million in 2024. A second pattern concerns information present in a workbook but not visible to the recipient: hidden rows, hidden sheets, pivot-table caches, embedded objects and retained source layers can cause harm even when no calculation is wrong. The clearest rise in reported cases is in this hidden or embedded data category, whose most serious example is the 2022 Ministry of Defence spreadsheet that exposed details of roughly 18,700 Afghan applicants through hidden rows. The paper argues that spreadsheet risk is no longer only a problem of wrong numbers. It also involves visibility, disclosure, governance and trust. Existing taxonomies remain useful for formula, entry and handling errors, but do not fully capture the public harm caused by spreadsheets used as uncontrolled operational infrastructure. The same mistakes have persisted for three decades, what has changed is the authority organisations give to the spreadsheets that contain them.
Read the full abstract

Human-Computer Interaction

Adapting for AI: How elementary teachers adjust their practices for an AI-integrated curriculum

Fasika Melese, Ruiyang Wu, Xinyue Cui et al. (+4 more)

Conversational AI tools are entering children's everyday experiences, and schools are interested in adopting them. However, successful classroom integration depends not only on the technology but also on the work teachers do to make it usable and appropriate for their students and classroom context. There is little known about how elementary teachers work as they implement conversational AI tools in real classrooms. In this study, we examine three teachers' experiences implementing an AI literacy and English Language Arts (ELA) curriculum built around ToyTalk, a conversational AI toy development platform, over 13 instructional days, a three-week summer camp. Drawing on daily individual reflections, group reflections, and post-camp interviews, we find that teachers' adaptive practices of repair, differentiation, translation, and balancing sit at the intersection of three tensions (technology, learner, and instruction). Teachers' understanding of AI and their role evolved over the camp experiences. From these findings, we contribute design implications and considerations for deploying conversational AI within elementary classrooms.
Read the full abstract

All authors Fasika Melese, Ruiyang Wu, Xinyue Cui, Joanna Perkins, Xiaoyi Tian, Tiffany Barnes, Shiyan Jiang

Human-Computer Interaction

PANEL: An Open-Source, Self-Hosted Web Platform for Human Evaluation of Generative Models

Matteo Spanio, Andrea Poltronieri and Martín Rocamora

Human judgement is the reference measure for evaluating generative models, yet the software used to collect it lags behing the methodology. Researchers adapt listening-test frameworks designed for perceptual protocols such as MUSHRA, rely on closed commercial survey platforms, or implement single-use web applications. Live arenas such as Chatbot Arena and Music Arena rank publicly deployed systems at scale, but do not support controlled comparisons of a laboratory's own models with its own participants. We present PANEL, an open-source, self-hosted platform for such studies. A study is authored in the browser and distributed as a single link, with audio, video, image, and text stimuli, seven question types, and screening and skip logic. The platform reports per-question summaries, across-condition significance tests, pairwise win rates and Bradley–Terry scores, and supports power analysis from pilot data. Consent versioning, self-service withdrawal, retention enforcement, and audit logging support GDPR-compliant operation. Each study exports as a machine-readable specification. PANEL is available at https://github.com/matteospanio/panel.
Read the full abstract

Human-Computer Interaction

EEG-based Word Association Paradigm for Adult ADHD Screening: An Exploratory Pilot Study

Caroline Peng and Tony Russell-Rose

With the prevalence of Attention Deficit Hyperactivity Disorder (ADHD) over the past decades, healthcare systems across the globe face critical diagnostic challenges due to long diagnostic waiting times and a reliance on subjective behavioural assessments that cannot distinguish ADHD from comorbid psychiatric disorders, especially for adult patients. This exploratory study investigates whether EEG-based word association paradigms show promise as a complementary approach to screening for ADHD in adults. Using a mixed-method approach, the study examines neurological and cognitive differences between neurotypical individuals (NT), clinically diagnosed ADHD participants (ND), and self-reported ADHD cases awaiting formal diagnosis (SR) across three word association tasks. While established EEG biomarkers (ERP N400, Theta/Beta Ratio, and Alpha Suppression) show no significant group differences, semantic distance analysis reveals a statistically significant main effect ($p=0.003$), with the SR group showing the most divergent associations. These preliminary findings suggest that word association paradigms may capture cognitive differences not detected by standard EEG metrics and encourage further large-scale investigation as a potential complement to existing adult ADHD screening tools.
Read the full abstract

Human-Computer Interaction

Cognitive Skills in the Age of AI: Computing Students and Experts Perceptions

Neha Rani, Vu Minh Anh Le, Austin M. Spangler et al. (+1 more)

AI is becoming increasingly integrated into daily workflows, especially in computing. We are gradually shifting towards an AI-rich future, an impending yet unknown one. One important emerging concern is whether we are accordingly preparing our future computing workforce. Further, we need to know what the important cognitive skills are to remain relevant in the computing workforce and if there are changes in cognitive skill importance. To investigate this direction, we conducted a mixed-methods study, collecting perceptions from computing students and computing experts regarding the importance of cognitive skills in the past, present, and future. We report that the perceived importance of most cognitive skills will decrease in the future, with an AI-rich environment, but critical thinking skills remain important. Further, we report reasons collected through interviews on why the importance of cognitive skills will change and how future computing students can prepare for it.
Read the full abstract

All authors Neha Rani, Vu Minh Anh Le, Austin M. Spangler, Erta Cenko

Human-Computer Interaction · Cross-listed from cs.RO

Evaluating the Impact of Adaptive Extended Reality on Human-Robot Interaction Across the Reality-Virtuality Continuum

Carl Tornberg, Alicia Torck, Lotfi El Hafi et al. (+1 more)

As populations in developed countries age and labor shortages intensify, Cybernetic Avatars (CAs) are proposed to extend human capabilities through robotic embodiments, requiring effective Human-Robot Interaction (HRI) frameworks. Extended Reality (XR), an umbrella term for Augmented Reality (AR), Augmented Virtuality (AV), and Virtual Reality (VR), offers such interfaces, but prior research typically fixes the XR modality without evaluating its effect on task outcomes. This study examines whether the XR modality impacts HRI performance and whether an adaptive interface adjusting the level of virtuality along the Reality-Virtuality Continuum (RVC) at runtime improves it. A custom XR application interfaced with a mobile manipulator supports immersive control and runtime modality switching. In a within-participant multi-room pick-and-place experiment comparing fixed AR, AV, and VR with dynamic RVC through task metrics, the NASA-TLX, and the System Usability Scale (SUS), this study demonstrates that 1) the fixed reality modality affects HRI results, and 2) dynamically changing the modality along the RVC improves them. AR yielded significantly lower mental demand, effort, and frustration than AV and VR, while the dynamic RVC condition achieved the highest throughput and lowest workload, highlighting the value of adaptive XR interfaces for human-robot symbiosis. The implementation is available at https://github.com/CarlTornberg/XR-HRI.
Read the full abstract

All authors Carl Tornberg, Alicia Torck, Lotfi El Hafi, Tadahiro Taniguchi

Human-Computer Interaction · Cross-listed from cs.RO

Comparative Evaluation of an XR Pen-based Control Interface for Semi-Autonomous Mobile Robot Navigation in Service Environments

Alicia Torc, Carl Tornberg, Eric Piette et al. (+5 more)

Service robots remain difficult to deploy in domestic environments, partly because fully autonomous operation is not yet reliable in unpredictable surroundings, and partly because conventional control methods remain inaccessible to novice users. Extended Reality (XR) enables operators to visualize robot information overlaid onto the real world and to interact with augmented elements. Yet, common XR control methods, such as motion controllers and hand gestures, are still perceived as unintuitive. This paper presents a control interface that uses a commercial XR pen to command a semi-autonomous mobile robot in Augmented Reality (AR): the operator points at a position in the room, selects it, and drags an augmented arrow to set the desired orientation of the robot at this destination. Two additional interfaces, based on the XR motion controllers and hand gestures, were developed within the same framework. To assess the performance and users' perception of these interfaces, and of the XR pen in particular, a study with 10 participants compared four control methods, i.e., the XR pen, the XR motion controllers, hand gestures, and a computer-based baseline RViz, in navigation tasks performed in a home-like environment. Results show that the XR pen significantly outperforms the other methods in task selection time with the most consistent selections, and that the XR motion controllers obtain the best perceived workload and usability scores, ahead of the computer-based baseline, supporting XR-based control as an intuitive alternative for novice users. However, technical limitations in the integration of the recently released XR pen currently hold back its user experience.
Read the full abstract

All authors Alicia Torc, Carl Tornberg, Eric Piette, Renaud Ronsse, Benoit Macq, Gustavo Alfonso Garcia Ricardez, Lotfi El Hafi, Tadahiro Taniguchi

Human-Computer Interaction

Confident, Not Wiser: The Dunning-Kruger Effect in Human-AI Interaction

Daniela Fernandes, Michelle Rausch, Agnes Mercedes Kloft et al. (+2 more)

AI assistance can improve performance without improving self-assessment. We report a study (N=366) comparing Human alone and Human+AI performance on reasoning tasks, for which the AI model is benchmarked on the same items. Participants estimated global and block performance and rated confidence in their answers. Human+AI achieved higher scores, but self-estimates tracked performance weakly. Average overestimation was similar across groups, covering individual errors. Across tasks, confidence distinguished correct from incorrect answers less accurately in the Human+AI group, while within-task differences remained uncertain. The Dunning-Kruger pattern was found in both groups, with a larger observed contrast in Human+AI. Controls for score noise reduced but did not eliminate the pattern, with the controlled group difference remaining inconclusive. An extended computational account describes global and block estimates. Our findings distinguish performance augmentation from metacognitive augmentation and motivate interfaces that support verification, communicate task-specific AI model performance, and help users evaluate the quality of their joint work rather than produce answers.
Read the full abstract

All authors Daniela Fernandes, Michelle Rausch, Agnes Mercedes Kloft, Daniel Buschek, Robin Welsch

Human-Computer Interaction

From Segments to Trajectories: Evolving Affective Graphs with Evidence Retrieval for Continuous EEG Emotion Recognition

Chi Yang, Jihong Wang, Chengxi Xie et al. (+5 more)

Electroencephalography (EEG)-based emotion recognition is important for affective computing and human-computer interaction, yet most existing methods divide a long trial into short segments and assign each segment the label of its source trial. Although this strategy increases the number of training samples, it reduces an evolving emotional response to a segment-level, coarse-grained, and static prediction problem. In reality, emotion may continuously emerge, intensify, weaken, and fluctuate as a stimulus unfolds, motivating the prediction of a time-aligned affective trajectory from the complete EEG trial. This task requires coordinated modeling of how spatial neural organization evolves throughout the trial and how local emotional fluctuations interact with longer-term trends. In this work, we formally define and systematically investigate continuous EEG emotion recognition as whole-trial affective trajectory prediction. We propose EAGER, an Evolving Affective Graph framework with Evidence Retrieval for continuous EEG emotion recognition. EAGER comprises two complementary modules: Affective State-guided Topology Evolution models the evolving spatial organization of EEG activity, while Multi-scale Temporal Evidence Retrieval integrates short-term fluctuations with longer-range temporal trends for time-aligned prediction. Experiments on MAHNOB-HCI, SEED-VII, and REFED show consistent gains in trajectory-tracking metrics over representative methods, with competitive pointwise errors.
Read the full abstract

All authors Chi Yang, Jihong Wang, Chengxi Xie, Kai He, Huan Liu, Man Yao, Shile Qi, Yuzhe Zhang