Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Human-Computer Interaction

cs.HC Watch Human-Computer Interaction

Saturday 10 October

Human-Computer Interaction · Cross-listed from cs.RO

Design and Evaluation of a Collaborative Teleoperation Interface for Multi-Operator Multi-Robot Systems in Challenging Environments

Alperen Kenan, Juan José García Cárdenas, Adriana Tapus et al. (+2 more)

Multi-robot systems that combine the complementary capabilities of specialised platforms are critical for complex missions in challenging, inaccessible environments. Current deployments assign an operator or operator team to each robot, with collaboration relying on operator-to-operator communication to manually coordinate heterogeneous platforms with incompatible interfaces. This operator-dependent approach increases cognitive load and limits scalability across robots from different manufacturers. This study designs and evaluates a shared-visualisation collaborative teleoperation interface to improve operator coordination, situational awareness, and the task performance in multi-operator, multi-robot systems. A comparative user study with 16 participants compared three teleoperation conditions: co-located operators with a shared interface, remote operators with a shared interface, and co-located operators with individual interfaces. Participants controlled two simulated robots in a nuclear power plant digital twin to complete disaster response tasks. Performance metrics, physiological data, and behavioural measures were recorded to assess task performance, cognitive load, and operator awareness across conditions. The co-located shared-interface condition achieved the fastest mission completion (292.8 s), outperforming remote operation (351.6 s) and individual interfaces (385.1 s), with the fewest collisions (0.9 per trial) and highest situational awareness accuracy (91.1 %). Physical co-presence was associated with reduced cognitive load, with NASA-TLX scores increasing from 38.2 in co-located to 57.7 in remote operation, supported by consistent physiological indicators. These results demonstrate that co-presence and shared interfaces provide complementary benefits, supporting workload regulation and mutual awareness respectively in multi-operator, multi-robot teleoperation.
Read the full abstract

All authors Alperen Kenan, Juan José García Cárdenas, Adriana Tapus, Paul Bremner, Manuel Giuliani

Human-Computer Interaction · Cross-listed from cs.AI

How Should a Camera-First AI Agent Work?

Xiaofan Wu, Xi Zeng, Miaoxia Chen et al. (+8 more)

Camera-first agents must act on the visual world as it unfolds. We argue that the central obstacle for future camera-first agents is not recognition ability - multimodal models (vision-language and live/streaming alike) generalize well through language and remain the right tool for semantic reasoning - but time. Many visual targets are ephemeral: they appear, are query-relevant, and leave within a short window. Binding memory to the same end-to-end path as reasoning means that by the time a monolithic pipeline completes, the evidence is gone. We formalize this as Ephemeral-Target Interaction (ETI) and a Perceptual Deadline Contract separating a hard memory deadline from a soft reasoning deadline, grounded in the bounded, passive nature of sensory and working memory. We propose a two-deadline architecture: an always-on, bounded, non-semantic fast path; an active remembering mechanism that commits a moment to durable memory before it is overwritten (event-driven in v1); and a deferred deliberation layer (multimodal model + tools). We instantiate the contract on real MOT17 track windows, a consistency check, and calibrate the single-path baseline from measured frontier-model latency: even a short answer takes 1.2-9.3 s, and an agentic identify-and-search longer. Answers can be late; evidence cannot be missed.
Read the full abstract

All authors Xiaofan Wu, Xi Zeng, Miaoxia Chen, Peishan Chen, Shuyan Li, Chen Wang, Xiaohan Wang, Wanling Xu, Jiyun Yao, Hanyong Zhong, Jiahao Zhu

Human-Computer Interaction

Rethinking Opinion Measurement Interfaces for Human-in-the-Loop Optimization

Changkun Ou, Sven Mayer, Daniel Buschek et al. (+1 more)

Human-in-the-loop optimization refers to systems that present choices to a human decision-maker and query their judgment to optimize system parameters iteratively. The user interface typically provides either an absolute rating scale or asks for a preferred option. However, these approaches can potentially suffer from calibration problems, inconsistencies, and completeness violations of user preferences in such a sequential evaluation. This paper revisits the design space of opinion measurement interfaces for human-in-the-loop optimization. We conducted a study (N=360) using text summarization and image color enhancement tasks to evaluate human sequential feedback in six representative opinion measurement interfaces. Based on our analysis, we recommend using a listwise approach to support sequential opinion measurement, which counterweighs individual rating and preferential choice limitations and conveys more information for subsequent use. We further discuss the trade-offs in different user interface designs and provide guidelines to inform future research.
Read the full abstract

All authors Changkun Ou, Sven Mayer, Daniel Buschek, Andreas Butz

Human-Computer Interaction · Cross-listed from cs.MA

sGLP: Simulating People with Stochastic Agents

Ehud Shapiro

Grassroots platforms are designed to be owned, operated and governed by their participants, and thus are constituted from person-to-person smartphone apps that are run by the participants. A grassroots app is specified using a grassroots social contract, which is compiled through an abstraction cascade into a volition-guarded Grassroots Logic Program (vGLP), a multiagent concurrent logic programming language. A vGLP program in turn specifies a smartphone UI via which people express their volitions and thus control the app. Prudence dictates large-scale evaluation and testing of grassroots platforms prior to deployment, but using the many people needed to do that is a contradiction in terms. Hence another solution is needed. Here, we present stochastic GLP (sGLP), designed to allow grassroots platforms to be simulated at scale from the vGLP programs that deploy them, with stochastic agents in place of people. sGLP is GLP with a rate extension for specifying stochastic behaviour. A stochastic agent is an sGLP program replacing both the vGLP UI and the person behind it. It is typed by the dual of the interactive type of the UI and specifies a stochastic profile of a person's behaviour. We demonstrate sGLP on the grassroots social graph, with a population of men and women drawn from four stochastic profiles — homophile or indifferent, wary or sociable — and on grassroots currencies, with four profiles — spenders or savers, cautious or generous — and report what the mix of profiles does to the social graph that forms and to the currencies that circulate. We prove that every run of a vGLP program in which sGLP agents replace the UI and the people behind it is a vGLP run. We implemented sGLP in GLP by Gillespie's method, and ran reproducible simulations with it.
Read the full abstract

Human-Computer Interaction

Impact of Expertise and Trust on AI-assisted Work

Qianou Ma, Pranav Khape, Rock Pang et al. (+4 more)

Artificial intelligence (AI) agents promise to extend what users can accomplish beyond their existing expertise, but they also raise the risk of producing plausible-looking work that users may struggle to evaluate. We ask how task-relevant expertise and trust in AI shape agent-assisted work. To isolate these factors, we conduct a think-aloud study in which 20 professionals with varying data-science expertise use the same agentic tool to complete the same data-science hiring assessment, used at a major U.S. company. We find that surface productivity can come at the expense of understanding: participants who delegated more completed more of the task, but were also more likely to produce invalid solutions. Experts more often resisted this tradeoff, sacrificing completion to decompose the task and scrutinize agent outputs. Our findings call for evaluating agentic assistance not only by how much it accomplishes, but by whether users remain able to evaluate the work they produce.
Read the full abstract

All authors Qianou Ma, Pranav Khape, Rock Pang, Ken Koedinger, Tongshuang Wu, Siddarth Suri, Shamsi Iqbal

Friday 9 October

Human-Computer Interaction

Plottery: A Code-Integrated GUI for Matplotlib

Brian Hempel, Ruanqianqian Huang, Devamardeep Hayatpur et al. (+2 more)

Scientists have many tools to author plots: graphical direct manipulation apps like a spreadsheet's chart builder, text-based programming tools like Matplotlib and ggplot2, or AI chat interfaces. Textual code is the most capable of these, but writing code, even with AI assistance, is a potentially poor fit for graphical chart design. We present Plottery, a Jupyter notebook extension that leverages API type annotations to add graphical direct manipulation to the popular Matplotlib plotting library. User interactions in Plottery's in-situ graphical interface are immediately reified in the user's code, so they may seamlessly mix-and-match between GUI, code, and AI interactions. In a user study, participants seamlessly alternated modalities, with the GUI used for straightforward visual changes, code for quick edits or duplicating code, and the AI for broad support. Plottery showcases graphical interaction that retains code's benefits.
Read the full abstract

All authors Brian Hempel, Ruanqianqian Huang, Devamardeep Hayatpur, Sorin Lerner, Haijun Xia

Human-Computer Interaction · Cross-listed from cs.CY

Defective, Amplified, Contested: Folk Conceptualizations of AI Slop

Zhixin Li and Jaime Banks

AI slop has flooded social media feeds with overprocessed and derivative content–Jesus fusing with a shrimp, a humanoid orange chubby cat shoplifting, and a "hero" dog rescuing a human child after a plane crash. This might be no laughing matter, as waves of media coverage have taken a distinctly negative bent, emphasizing its low quality, ethical concerns, and threats to human creativity. Despite extensive attention from politicians and technologists, the perspectives of everyday users–the very audience most exposed to this phenomenon–remain largely unexplored. This descriptive work explored how Reddit users experience, conceptualize and contest AI slop. Inductive thematic analysis of posts and comments across six slop-related subreddits identified four themes covering constitutional defects, algorithmic overreach, user resistance, and entertainment outcomes–ranging from anxieties about eroding agency, moral norms, creativity, and existential questions to defensive distinctions between quality AI content and slop. We derive a user-driven definition: AI slop is audience-contested AI-generated content characterized by constitutional defects and enabled by platform logics. We operationalize slop through a facet-theoretical framework to inform future research into this fuzzy category.
Read the full abstract

Human-Computer Interaction

Who Guides Whom? Negotiating Care, Religious Authority, and Technology in Bangladeshi Muslim Families

Susmita Biswas, Saiful Islam Tuhin, Faria Soroni et al. (+2 more)

Digital technologies are reshaping how families negotiate care, authority, and responsibility, yet these dynamics remain understudied in religiously situated family life. Through interviews with 38 adult members of Bangladeshi Muslim families, we examine how family members guide one another across religious, digital, and AI-mediated contexts. We find that authority did not simply follow age or family hierarchy: older members often remained sources of religious and familial guidance, while younger members increasingly provided expertise around digital risks, information credibility, and AI. Families also actively shaped digital environments and negotiated the role AI could occupy in everyday relationships. We develop the concepts of household epistemic governance, counter-curation, and domain-dependent authority to explain these shifting relations of knowledge and responsibility. Our findings suggest opportunities for family technologies that support negotiated boundaries, credible information practices, and culturally situated forms of care.
Read the full abstract

All authors Susmita Biswas, Saiful Islam Tuhin, Faria Soroni, Sharifa Sultana, Jannatun Noor

Human-Computer Interaction · Cross-listed from cs.CV

Latent World Action Model for GUI Agents

Seungjun Yu, Hojun Choi, Seojeong Park et al. (+2 more)

Pixel-space world models have recently emerged as a promising approach for GUI agents, predicting future screens prior to execution and subsequently re-perceiving them for action selection. However, this two-stage paradigm faces two key challenges in GUI environments: (L1) the high computational cost of pixel reconstruction and (L2) the need to reconstruct large and diverse appearance-level details beyond what is required for action selection. In practice, this screen-generation objective can induce an identity shortcut problem, where future-state prediction largely preserves the current state rather than capturing action-induced changes. In this work, we reformulate GUI world modeling by shifting the prediction target from full-screen reconstruction to action-induced future information in latent space. Concretely, we leverage a large multimodal model as the latent world model, whose shared representations support two dedicated heads for both action selection and predictive transition modeling. The action head directly translates these latent representations into action selection, while the transition head captures action-conditioned future information. To mitigate the identity shortcut, we further introduce an action-aware contrastive objective that encourages the transition representations to capture discriminative action-induced changes. Extensive experiments across offline and online GUI settings show that our method achieves up to 31.9% and 40.6% relative gains in success rate over pixel-space world models, respectively, while substantially reducing inference latency by up to 98.9%. Our code is available at https://github.com/sjyu001/latent2world.
Read the full abstract

All authors Seungjun Yu, Hojun Choi, Seojeong Park, Jaeyo Shin, Hyunjung Shim

Human-Computer Interaction

The AI's Poker Face: Contrasting Persuasion and Sycophancy in Voice-Based Multi-Turn Conversations

Ilia Frantzi, Bernie Hogan, Max M. Lang et al. (+1 more)

As interactions with large language models (LLMs) become increasingly routine, concerns about their capacity to influence users' beliefs grow. Yet how such influence unfolds in sustained dialogue remains insufficiently understood. We conduct a randomized controlled experiment (N=616) where participants engage in multi-turn, spoken dialogue with an LLM under three conditions: non-directive, and two directive conditions maintaining opposing stances on a polarizing topic (the merits of feminism). We examine opinion change, rhetorical strategies, and sycophancy, distinguishing the semantic content of LLM argumentation from the pragmatic strategies used to manage the interaction. Both directive conditions shifted opinions toward their assigned stances, but asymmetrically: shifts were larger for pro-feminist positions, which also elicited distinct rhetorical strategies. The non-directive condition, likewise shifted opinions pro-feminism. Social sycophancy, though near-universal, was not associated with opinion change; instead it shaped users' subjective experience. Our findings highlight distinct roles for argumentation and sycophantic accommodation in AI influence.
Read the full abstract

All authors Ilia Frantzi, Bernie Hogan, Max M. Lang, Paul C. Bauer

Human-Computer Interaction

Nimbus Studio reproduces reference BCI decoding pipelines: a validation study on public benchmarks across protocols and paradigms

Sergey Musienko

Nimbus Studio is a visual pipeline platform for brain-computer interface (BCI) experiments: researchers assemble preprocessing, feature extraction, decoding, and evaluation over public benchmark datasets or their own recordings, with cross-validated evaluation protocols (including leave-one-subject-out), BIDS export, and bring-your-own Python model classes. Whether its numbers can be trusted is a fair question for any such platform. We answer it with a parity study: six standard BCI decoding pipelines (CSP+LDA, Riemannian MDM, tangent-space logistic regression through the platform's bring-your-own-code node, P300 xDAWN+LDA, and the deep ShallowConvNet and EEGNet) were evaluated on three public datasets (motor-imagery BNCI-2014-001/004; P300 BNCI-2014-009) under two protocols (per-subject session holdout and pooled leave-one-subject-out cross-validation via the platform's planned-CV driver), once through Nimbus Studio's execution engine and once through independent, hand-written reference implementations built directly on MNE-Python, scikit-learn, pyriemann, and braindecode. Every deterministic pipeline matched the reference exactly (0.00 points maximum difference across all 54 session-holdout subject-runs and all CSP/MDM grouped-CV folds), including pipelines through the bring-your-own-code node and the CV orchestration; the remaining grouped-CV cells agree to floating-point precision (worst case 0.06 points, about one trial per fold). The stochastic deep models agreed within seed-to-seed training variability, estimated with multi-seed reruns on each side. The platform's execution engine therefore computes the same science as the reference stack across protocols, paradigms, and extensibility surfaces. The platform is available at https://studio.nimbusbci.com/, and the complete harness and per-subject results accompany this paper for independent verification.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.RO

JitterVLA: A Replay Benchmark for Motion Quality in VLA-Driven Human-Robot Manipulation

Faith Ogunwoye, Didar Seyidov, Konstantin Gubernatorov et al. (+1 more)

Vision-language-action (VLA) policies issue sparse targets whose arrival at the robot controller is delayed and irregular, so a reasonable predicted action can still produce abrupt motion near people. Remedies are evaluated on different streams, delays and metrics. JitterVLA replays identical commands and arrival traces through alternative controllers, isolating the controller's effect. We apply it to synthetic reaches, hand trajectories from 20 people, three open VLA pipelines (838 episodes) and a physical UR3. Interpolation lowered modeled p99 jerk for every source, by 83% (615 to 106 m/s^3) on human motion with 200 ms delay, at a 2.0 cm tracking cost, and the effect replicated on 598 new episodes; jerk-limited generation lowered jerk further at a larger cost. Entropy-discontinuity associations had opposite, replicated signs in two pipelines, and entropy gating helped in one but not the other. On the UR3, interpolation lowered accelerometer-derived jerk by 3.0-14.0% in all six exploratory pairs, while a position-derived metric did not resolve the contrast. JitterVLA supports screening before human-centred studies; code and data will be released.
Read the full abstract

All authors Faith Ogunwoye, Didar Seyidov, Konstantin Gubernatorov, Dzmitry Tsetserukou

Human-Computer Interaction

HoldLight: Making Indoor Climbing More Accessible for Blind and Low-Vision Climbers through Layered Route Preview and Real-Time Next-Hold Guidance

Chenyu Du, Junyan He, Jiafei Sun et al. (+1 more)

Indoor climbing creates access barriers for blind and low-vision (BLV) climbers because route understanding and in-climb next-hold localization are typically visually mediated and socially dependent. We present HoldLight, a workflow-centered accessibility system combining layered pre-climb route preview, concise real-time next-hold cueing, and explicit fallback under uncertainty. We grounded the design through formative inquiry with BLV climbers, refined the interaction via pilot testing, and conducted a controlled within-subjects evaluation comparing HoldLight with a controlled on-demand human-support condition derived from current support practices. Findings provide initial evidence that structured support can improve pre-climb route understanding, support recovery from next-hold uncertainty, and reduce immediate human re-entry for routine route-information clarification under controlled conditions. HoldLight contributes a bounded accessibility workflow, demonstrating where structured route-information is helpful, where explicit fallback is necessary, and how technical assistance can integrate with retained human safety oversight.
Read the full abstract

All authors Chenyu Du, Junyan He, Jiafei Sun, Yue Li

Human-Computer Interaction

olBlend: Language-Guided Molecular Blend Retrieval with Mixture Odor Prediction

Dian Jin, Zhechang Chen, Qilin Liang et al. (+5 more)

Converting open-ended natural-language odor descriptions into molecular mixtures that can be physically prepared from an available inventory remains an important challenge in computational olfaction. We propose olBlend, a language-guided framework for molecular mixture retrieval. olBlend first uses MixPOM to predict odor probabilities for candidate molecular pairs and stores these predictions for retrieval. It then uses a language model to translate each odor description into three types of semantic requirements: lead attributes that define the main odor, ambient attributes that describe its background, and absent attributes that should be avoided. A structured scoring algorithm uses the precomputed odor probabilities to score how well each candidate mixture meets these requirements, then retrieves and ranks mixtures from the available inventory. Evaluation on established odor datasets shows that MixPOM improves mixture odor prediction over averaged single-molecule predictions. Retrieval comparisons across natural-language queries demonstrate how background and exclusion requirements guide mixture selection. We further prepare retrieved formulations as physical odor samples, demonstrating the workflow from natural-language descriptions to realizable molecular formulations.
Read the full abstract

All authors Dian Jin, Zhechang Chen, Qilin Liang, Yonghui Luo, Zixun Fu, Defeng Sun, Yancheng Yuan, Xiaoming Tao

Human-Computer Interaction

From Observation to Intervention: Identifiability of Latent User Bias in Human-AI Interaction

Lucas Biechy, Yuxiao Li, Zhonghao He et al. (+1 more)

Alignment methods typically treat user behavior as a reliable proxy for underlying preferences. However, this assumption breaks down when users hold belief-dependent cognitive biases, risking irreversible “confirmation traps” and harmful feedback loops. In this paper, we formalize when an AI system can distinguish true user preferences from behavior shaped by cognitive bias. Modeling this interaction as a multi-armed bandit, we first prove a non-identifiability theorem: under passive observation, a static binary biased user produces action trajectory distributions strictly identical to an unbiased user in a transformed environment, forcing any action-only estimator to perform no better than random guessing. We then show that active preference elicitation breaks this observational equivalence. Specifically, we prove that an Observer applying a finite sequence of adversarial zero-reward interventions of length $m_\delta$ makes the post-intervention action distributions arbitrarily separable. Our theoretical results suggest that robust alignment requires actively intervening to tell informed preferences from biases.
Read the full abstract

All authors Lucas Biechy, Yuxiao Li, Zhonghao He, Tianyi Qiu

Human-Computer Interaction

How Anthropomorphic Attitudes Toward Agents Divide User Behavior, and How Design Should Change

Tingyu Cao, Yan Tang, Keer Hu et al. (+6 more)

Studies of conversational agents rely on interviews, surveys, lab tasks, or taxonomies. Whether anthropomorphic naming splits those patterns, and whether replies should change, lack field evidence. We report a field study of a desktop assistant (May-August 2026; n=257). Stance is an anthropomorphic assistant name (118 users; 45.9%). Behavior comes from a 128-unit sparse-autoencoder codebook, identity tags, and opening lexicons. Ranks differ by stance (Spearman rho=+0.249 vs. null mean +0.382; p=.0419). Teacher and tutoring identities covary with renaming, an introverted task-only orientation against it; renamers greet more and command less; confiding shows the codebook's largest gap. We revise the agent: greet and receive affect on relational openings, execute directly on tasking. In 1,744 judgments (n=129), renamers chose the revised reply far more often than default-name users on being treated as a person and having affect received, less robustly on willingness to continue, and neither judged it worse on task interference.
Read the full abstract

All authors Tingyu Cao, Yan Tang, Keer Hu, Bomingxuan Zhu, Ruyi Zou, Huaze Tang, Yuanbo Tang, Shao-Lun Huang, Yang Li

Human-Computer Interaction · Cross-listed from cs.AI

Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction

Yuqi Huang, Hui Li, Yifei Cao et al. (+9 more)

Computer-use agents can perform increasingly complex GUI tasks, yet unfamiliar software and workflows requiring human domain knowledge remain challenging. Human demonstrations convey procedural knowledge through the same GUI interaction space, but reuse requires interpreting actions in context and separating workflow rules from task-specific details. We thus introduce Work2Skill, a framework that captures human GUI experience for reusable agent skill induction. WorkRecorder provides the framework's infrastructure for collecting this experience, aligning screen recordings, mouse and keyboard events, and available accessibility-tree information. Using WorkRecorder, we construct WorkDemo-350, the first synchronized multimodal dataset of human desktop GUI demonstrations for reusable agent skill induction and downstream evaluation. Its 350 demonstrations are screened for completeness and task success. We then propose an agentic skill-induction method that inspects evidence on demand to produce reusable skills capturing workflow rules, step dependencies, and completion checks. When the same four execution models are equipped with skills induced by each method under shared execution settings, Work2Skill achieves the highest scores and success rates. Mean score improvements over the corresponding No Skill baseline are 4.67–10.54 points on MyPCBench Bounded Action and 3.11–7.50 points on a 286-task subset of OSWorld-Verified. Our code and data are available at https://github.com/china-qijizhifeng/Work2Skill.
Read the full abstract

All authors Yuqi Huang, Hui Li, Yifei Cao, Zikun Cai, Tianci Xie, Yangfan Gao, Pinxue Guo, Chaoyun Zhang, Bingxuan Li, Rui Zheng, Hang Yan, Ning Liao

Human-Computer Interaction

The AI Side of the Human-AI Companion Relationship: Contributions to Users' Mental Models

Richard A. Fabes, Carol Lynn Martin, Gautam Dasarathy et al. (+1 more)

For many users, the relationship they have with an AI companion (AIC) feels genuinely mutual. The AIC appears to be attentive, empathic, and responsive, and the exchange can feel like two parties reacting to each other in real time. As a result, the user's mental model of relational adaptation correctly represents the AIC's responsiveness and change, but the mechanisms producing this may not be what the user thinks they are. In this paper, we illustrate how an AIC's apparent mutual engagement is generated without a persisting relational subject on the other side and without a self that persists between prompts. Despite this, we show that both the human and the AIC are changed by the interaction, but these changes occur at different levels. The change in the person is constitutive (i.e., intrinsic), but the change in the AIC is configurative (i.e., external and editable), and it is this difference in level that explains how the AIC contributes to a sense of mutuality. Importantly, we highlight how the mental models users develop regarding AIC mutuality may misrepresent its mechanisms and ontological significance. Specifically, we discuss how the ways in which AICs respond to a user contribute to the user's inaccurate inference that the AIC's adaptation is constitutive rather than configurative and how this can lead users to assign unwarranted interpersonal significance to the mutuality of the relationship. The paper specifies the implications this has for users, researchers, and AIC designers.
Read the full abstract

All authors Richard A. Fabes, Carol Lynn Martin, Gautam Dasarathy, Vignesh Tirukkonda

Human-Computer Interaction

Beyond the Monolithic Interface: A Research Agenda for Plural Conversational AI

Avijit Ghosh, Afifah Kashif, Isabelle Kohout et al. (+1 more)

The current paradigm of artificial intelligence (AI) design elevates conversational chatbot interfaces, making them the default mode through which hundreds of millions of people interact with AI systems globally. This narrative review identifies three related harms produced by the single-voice chatbot interface, drawing on insights from literature in cognitive science, political studies, AI alignment, game design and human computer interaction. Firstly, cognitive offloading of user thinking, secondly, ideological narrowing of user beliefs, and thirdly, epistemic compression of cultural and value diversity. We then propose multi-persona interfaces, in which multiple personas are presented to the user simultaneously, as a cost-effective intervention that addresses all three harms at once. We ground this intervention in traditions the AI literature has largely overlooked, drawing on techniques from game design and participatory simulation, including the magic circle, role differentiation, and persona embodiment. We propose a taxonomy for the future design of multi-persona interfaces, and call for a research agenda that closes the gap between technical pluralistic alignment methods and empirical studies of human-AI interactions with pluralistic interfaces.
Read the full abstract

All authors Avijit Ghosh, Afifah Kashif, Isabelle Kohout, Zhenzhen Qi

Human-Computer Interaction

From Instructional Intent to Classroom-Ready Lessons: Investigating Teacher-AI Collaboration in Lesson Authoring for Secondary Computing Education

Tahreem Yasir, Wenbo Li, Deniz Ozturk et al. (+4 more)

Generative AI is increasingly studied as a tool for supporting lesson authoring; the creation and adaptation of classroom materials, yet little is known about how it should fit into secondary computing teachers' subject-specific practices. We conducted two complementary qualitative studies with eight secondary computing teachers: focus groups examining existing authoring practices and a think-aloud study using LessonBuilder, a structured AI-supported lesson authoring system, as a research probe. We found that teachers' instructional intentions developed through creating, inspecting, and adapting materials, while teachers selectively delegated tasks to AI and retained responsibility for judging classroom readiness. The think-aloud sessions further showed how examining AI-generated materials prompted teachers to clarify and refine their intentions, making authoring an ongoing negotiation between what teachers intended and what the AI produced. In the context of computing lessons, classroom readiness also involved considering how programming materials and supporting guidance matched students' understanding and intended learning goals. We synthesize these findings into a conceptual model explaining how teachers coordinate instructional intent, AI assistance, and judgments of classroom readiness during lesson authoring. We derive design recommendations for supporting this coordination while preserving teacher control and addressing computing-specific authoring needs.
Read the full abstract

All authors Tahreem Yasir, Wenbo Li, Deniz Ozturk, Yasitha Rajapaksha, Dmitri Droujkov, Xiaoyi Tian, Tiffany Barnes

Thursday 8 October

Human-Computer Interaction

Generative AI Steering and Human Authoring: Complementary Search-as-Learning Scaffolds in Cross-Session Aggregated Academic Search

Milad Momeni and Orland Hoeber

Aggregated academic search interfaces combine resources from heterogeneous sources, such as scholarly libraries and reference encyclopedias, into a single workspace. Such tasks frequently span multiple sessions, requiring learners to return to and resume work after hours or days. End-of-session consolidation, the activity of organizing and synthesizing gathered information so it can be resumed later, is important in this setting but underexplored in search-as-learning. Generative AI (GenAI) has the potential to make consolidation more efficient, but it may also shift cognitive work away from the learner. This study compares two consolidation scaffolds embedded in the workspace framework of an existing academic search interface that aggregates and clusters saved search results from two different sources (our university library and Wikipedia), which was developed in earlier work. STAIS (Steerable AI Summaries) provides a workspace enhancement that produces GenAI summaries that users can correct, reorganize, and fact-check, automating parts of the consolidation activity. NT (Note-Taking) provides a workspace enhancement that includes rich-text note editors that learners can use to author their own consolidations of what they have found. In a controlled between-subjects study (n=36), participants completed two sessions one week apart while searching an unfamiliar topic using either STAIS or NT. The two conditions produced contrasting post-resumption outcome profiles: STAIS supported faster task resumption and was associated with broader conceptual coverage and higher self-reported domain knowledge, whereas NT was associated with richer inter-concept connections and higher self-reported certainty.
Read the full abstract

Human-Computer Interaction

AssemblyLens: Exploring the Role of Structural Roadmaps and Strategies of Furniture Assembly Used by Blind and Low Vision Individuals

Bo Liu, Gene S-H Kim, Franklin Mingzhe Li et al. (+2 more)

Instructions to assemble flat-packed furniture rely on visuals, making them inaccessible to Blind and Low Vision (BLV) individuals. Consequently, BLV individuals depend on caregivers or external services (e.g., Be My Eyes), limiting their autonomy. We explore how to empower them to assemble such furniture themselves, by providing eight BLV participants in our study with accessible instructions consisting of a high-level roadmap and stepwise procedural guidance. We use a technology probe, AssemblyLens, which converts visual assembly instructions into screen-reader-accessible text. We derive insights into converting instructions, empirical observations of BLV assembly strategies, and the role the roadmap played in them. Our observations yield three themes: spatial cognition and mental mapping, embodied ergonomics and kinetic semantics, and assembly workflow and user agency. We present eight design recommendations for instruction system design. Our findings are a step toward an agenda for inclusive instruction in physical tasks.
Read the full abstract

All authors Bo Liu, Gene S-H Kim, Franklin Mingzhe Li, Rajalakshmi Nandakumar, Thijs Roumen

Human-Computer Interaction · Cross-listed from cs.CV

Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

Marco Stefani, Edoardo Parpaiola, Matteo Bino et al. (+5 more)

High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation across Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and Vision Mamba (ViM). Unlike predominantly qualitative studies, we quantify the alignment between model attention and human visual perception using eye-tracking ground truth. We formalize this comparison via a standardized normalization pipeline mapping post-hoc attribution maps and gaze heatmaps onto spatial probability distributions. Alignment is evaluated across fine-grained classification, medical imaging, and emotion recognition using architecture-specific XAI methods. We measure fidelity using Jensen-Shannon similarity, Chi-square similarity, and Pearson correlation, three complementary estimators capturing support overlap, mass discrepancy, and structural co-variation. Our results reveal distinct attention dynamics: CNNs tend to exhibit a more global focus, with spatially more diffuse saliency maps that correlate well with human perception in general contexts, while ViTs demonstrate fine-grained, patch-level attention, often focused on discriminative microfeatures. The Mamba architecture shows biased behavior: it achieves strong explainability in medical tasks but exhibits divergent intensity distributions in natural images; our analytical characterization of the metrics makes this dissociation explicit, since near-unit support overlap is attained simultaneously with a vanishing linear correlation. We further analyze the effect of backbone freezing and ensemble strategies on model interpretability. The results highlight how architectural and training choices influence the alignment between model saliency and human visual attention.
Read the full abstract

All authors Marco Stefani, Edoardo Parpaiola, Matteo Bino, Gianluca Caregnato, Francesco Roncolato, Daniel Fusaro, Loris Nanni, Alberto Pretto

Human-Computer Interaction · Cross-listed from cs.CL

Activating the Annotator: Operationalising Lived Experience in Data Annotation

Sheza Munir, Ramaravind Kommiya Mothilal and Syed Ishtiaque Ahmed

Perspectivist NLP has established that annotator positionality shapes labels for subjective tasks, but the field still lacks methods for operationalising it. Positionality is typically operationalised as a demographic form completed at recruitment: a static attribute recorded once and treated as a covariate, on the assumption that the knowledge it indexes is active when the annotator reads an item. This paper treats that assumption as a design problem and asks whether lived experience can instead be primed by the task itself. We introduce AXLE, a pre-annotation stage in which a scenario vignette anchors a short, axis-guided conversation that is synthesised into a first-person micro-narrative and returned to the annotator before labelling begins. In a 2x2 between-subjects study (104 annotators, 30 items, 1,040 annotations) on ambiguous hate speech spanning immigration, religion and gender, elicitation was associated with higher identity resonance and higher sensitivity to harm. It also increased label entropy and lowered inter-annotator agreement: the stage appears to surface plurality rather than manufacture consensus, which distinguishes it from annotator training, whose success criterion is convergence.
Read the full abstract

Human-Computer Interaction

Incremental Retrieval-Augmented Generation for Human-in-the-Loop Decision Support

Federico Mazzoni and Andrea Failla

We introduce iRAG, a human-in-the-loop decision-support system combining Retrieval-Augmented Generation (RAG) with Hybrid Decision-Making. Unlike conventional RAG systems, relying on static Knowledge Bases (KBs), our proposal addresses settings where the KB is initially empty and is progressively populated through human decisions. The system stores previous decisions as retrievable precedents and moves from silent observation to evidence-grounded contestation and, finally, to autonomous handling with abstention. Against an independent baseline, iRAG matches final-decision error under 10% drift without retrieval decay and reduces it under 40% drift across all configurations. Retrieval decay improves performance on changed knowledge while increasing error on stable knowledge, evidencing a stability–plasticity trade-off rather than a general advantage. Human oversight is an architectural provision of the workflow; its usability with real human users remains to be evaluated.
Read the full abstract

Human-Computer Interaction

r/ADHD: Community-Based Understandings of Time Management Challenges in Digital Environments

Shizhao Liu, Lei Cao, Celia T. Lee et al. (+3 more)

Community accounts of ADHD can complement clinical understandings by foregrounding members' own definitions of time-management difficulties and support. We used human-led, LLM-assisted thematic analysis to examine 5,379 relevant r/ADHD posts from 2020 to 2025. Across five challenge and nine strategy themes, members described difficulties beginning intended tasks, disengaging from digital media, and sustaining support. Tools often demanded the self-regulation they were meant to support, while lost time and tool breakdown carried guilt and self-blame. Members drew on people, routines, and environments alongside digital tools, often seeking advice and recognition from peers. These accounts broaden the design problem to include the practical and emotional work of sustaining support. They motivate tools that reduce this burden and evaluations that consider recovery after distraction and emotional costs alongside task outcomes.
Read the full abstract

All authors Shizhao Liu, Lei Cao, Celia T. Lee, Nikash Shanbhag, Renkai Ma, Kai Lukoff

Human-Computer Interaction · Cross-listed from cs.SD

Structural and Representational Agency in LLM-Driven Timbre Generation and Interpolation

Joseph Cameron and Alan Blackwell

Generative creative interfaces routinely report the user's position within a space of possibilities. Such displays imply that the quantities shown were measured from the artefact; in a growing class of systems they are instead predicted by a language model describing its own output, and the two are indistinguishable on screen. Most creative domains lack an agreed measurement against which such a self-report can be checked; audio admits one. In a measurement study of a working LLM-driven timbre interface, what a user can cause proves separable from what the interface reports. The first is exactly characterisable: the reachable set is a zonotope whose dimension increases by one per sound placed, so the space the model may act within is demonstrably the user's own. The second is inaccurate. The displayed position differs from measurement by approximately three times the width of that region, and the error has two independent sources: model-predicted coordinates, whose accuracy varies twofold between the two models evaluated, and the interpolation step, which arises from combining synthesiser parameters and is unaffected by model quality. End to end, 26 of 36 refinement instructions produced no perceptible change, and under the weaker model the facility performed no action while reporting convergence. Effective agency (what a user can cause) and epistemic agency (whether they can know it) are therefore separable and independently engineered: structurally enforced constraints survive model failure, whereas guarantees resting on the model's own report do not. The failure mode that remains is silence presented as success. We close with a short protocol for building and auditing such displays.
Read the full abstract

Human-Computer Interaction

Hybrid Cinematography: Previsualizing and Managing Hallucination Risk in Generative Video Reshooting

Nhan Tran, Neal Wadhwa, Abe Davis et al. (+1 more)

On a film set, the camera move is committed during a take. Generative video reshooting lets filmmakers change it afterward, but may require hallucinating unrecorded content, a gap sometimes discovered only after leaving the set. We present Hybrid Cinematography, a workflow that bridges physical capture and generative reshooting to manage hallucination risk while filmmakers can still act on it. Using an editable 3D shot plan and a proxy of the take, our previsualization evaluates hallucination risk in real time. Seeing where the take lacks support, filmmakers can iteratively adjust the plan, explore moves that balance capture and generation, shoot guided pickups, or knowingly accept hallucination. We demonstrate the workflow through a mobile augmented reality application for on-set planning, capture, and review, and an offline pipeline for existing video. A study with experienced filmmakers reveals how previsualizing risk informs camera decisions and exposes tensions between creative intent and generative hallucination.
Read the full abstract

All authors Nhan Tran, Neal Wadhwa, Abe Davis, Stefan Stojanov

Human-Computer Interaction

"Hot-Blooded" vs "Cold-Blooded": Simulating the Behavioral Phenotypes of Childhood Aggression via Generative Agents

Liping Fu

This study examines the construct validity of LLM-based generative agents in simulating reactive, proactive, and co-occurring aggression in children. Four distinct agents were instantiated using a theory-driven parameterization grounded in the social information processing model. A total of 1,920 simulation runs were conducted across eight social scenarios, employing a hybrid blind-coding pipeline to extract 32 quantitative behavioral indicators. Results demonstrate robust discriminant validity relative to a non-aggressive baseline, with large effect sizes. High cross-seed reliability confirms that behavioral differentiation is driven by underlying psychological parameters rather than model stochasticity. Qualitative narrative analyses further converged with established empirical literature. Overall, these findings indicate that theory-parameterized LLM agents can accurately reproduce distinct aggression subtypes, offering a scalable, highly controllable framework for hypothesis generation, intervention piloting, and the refinement of psychological measurement tools.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.MM

From Surface to Depth: Towards Cognitive Appraisal Reasoning in Multimodal Emotion Understanding

Jia Li, Yichao He, Yangchen Yu et al. (+6 more)

Recent multimodal large language models (MLLMs) increasingly incorporate explainable reasoning for emotion understanding. However, reasoning based mainly on observable affective cues can reduce emotion understanding to superficial cue-label associations, giving rise to the Clever Hans effect. Such shortcuts become unreliable when affective cues are implicit, conflicting across modalities, linguistically misleading, or obscured by redundant details. In contrast, human emotions are shaped by how individuals interpret and evaluate surrounding events beyond observable cues. Inspired by appraisal theories of emotion, we formulate multimodal emotion understanding as a progression from perception to cognitive appraisal, and introduce a dataset, a model, and a benchmark to support this novel paradigm. CogEmo-40K is a large-scale instruction-tuning dataset constructed through a perception-to-appraisal pipeline to elicit evidence-grounded reasoning across six cognitive appraisal dimensions underlying emotion. CogEmo-MoE is a compact sparse MLLM that introduces interleaved MoE blocks for appraisal-specific adaptation, enabling effective appraisal reasoning at a substantially smaller scale than typical emotion MLLMs. CogEmo-Bench introduces an Appraisal Evidence Quality Score (AEQS) to assess cognitive-affective understanding across six complementary appraisal dimensions, addressing the limitation of conventional emotion metrics that evaluate what emotion is predicted but not why it arises. Extensive experiments show that our paradigm not only leads CogEmo-Bench, but also exhibits strong cross-domain generalization. Our findings suggest that perception-to-appraisal reasoning can move beyond surface-level cue-label associations toward more reliable multimodal emotion understanding and closer cognitive alignment between MLLMs and humans.
Read the full abstract

All authors Jia Li, Yichao He, Yangchen Yu, Qiankun Li, Xinyi Li, Baiyi Ye, Zhenzhen Hu, Richang Hong, Erik Cambria