Human-Computer Interaction · Cross-listed from cs.CY
The Fabricated Front: Generative AI and the Opacity of Workplace Performance
Tom van Nuenen, Pratik S. Sachdeva and Sahiba Chopra
Generative AI (GenAI) has become a fixture of workplace life. Current research asks chiefly what this implies for jobs and outputs, measured in productivity, displacement, or bias. What remains underexamined are the interactional reconfigurations that GenAI produces at work. The emerging concept of effort opacity has begun to fill this gap by highlighting the systematic decoupling of observable output from human engagement. When GenAI makes interactional cues less diagnostic, it weakens the reciprocal exchange that sustains collaborative trust. Extending this account of effort opacity, we examine the interactional mechanics that produce opacity in everyday workplace encounters. Drawing on Erving Goffman's dramaturgical framework and 1,250 interview transcripts from Anthropic's AI Interviewer
dataset, we identify five opacity mechanisms through which workplace fronts are reorganized: voice (whose stance the words index), provenance (who can stand behind the artifact), vulnerability (whether the worker is uncertain),
attention (whether the worker is engaged), and investment (how much labor the output reflects). We show that professionals defend the identity mechanisms while freely producing opacity around the labor mechanisms, and trace this asymmetry to the output-centered organization of contemporary work, where deliverables already stand in for the labor process that produced them. The governance task, accordingly, is one of involvement management: specifying which forms of human involvement (
attention, effort, judgment) must remain inspectable, and to whom. Workplace AI policies built on universal disclosure will systematically misrecognize a social field in which inspectability is already audience-relative.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.RO
A Task-Agnostic Control Strategy for Dynamic Assistance with Pneumatically Actuated Soft Exosuits
Anoush Sepehri, Zachary Huang, Raymond de Callafon et al. (+2 more)
Pneumatic artificial muscles have provided new opportunities to develop upper-extremity soft exosuits for reha- bilitation, augmentation, and assisted daily living. However, the complex dynamics and limited bandwidth of these actuators has made providing responsive assistance based on user intention a longstanding challenge. In this work, we present an inverse-plant control strategy for pneumatically actuated soft exosuits that only relies on kinematic sensing for task-agnostic and dynamic assistance during daily living. We model the human-
robot system using a Hammerstein dynamic model, consisting of a Preisach hysteresis model and a linear time-invariant filter, to capture the static and dynamic behavior of the system. We personalize our model to each user using 140 s of data and approximate an inverse to integrate into our control loop. When evaluated on a test rig that emulated a soft assistive exosuit for the wrist, our controller reduced the interaction torque by up to 73% and the activation of key flexor and extensor muscles by up to 47% relative to the condition with no assistance for speeds ranging from 8{\deg}/s to 120{\deg}/s. Overall, this work presents a control strategy that can provide task-agnostic, dynamic assistance with pneumatically actuated soft exosuits without the need for physiological or force sensors to interpret user intention.
Read the full abstract ▾
All authors Anoush Sepehri, Zachary Huang, Raymond de Callafon, Michael T. Tolley, Tania K. Morimoto
Human-Computer Interaction · Cross-listed from cs.IR
AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence
Stephanie T. Wang, Jeffrey Gleason, Yakov Bart et al. (+2 more)
The integration of generative AI into web search delivers synthesized answers to user queries, changing how people navigate and assess information, while raising concerns about the downstream impacts on publishers who supply the underlying content. We conduct a preregistered field experiment (N=1,100) on Google Search, the dominant online search platform, to estimate the causal effects of AI Overviews and AI Mode on user behavior, perceptions, and publisher traffic. We show that removing AI Overviews and AI Mode increases click-through rates to publishers, while an AI Mode-only experience reduces click-through rates and erodes user experience and trust in information found on Google. These findings show that integrating generative AI into web search reshapes online
attention, with economic consequences for the online publishers that sustain both search platforms and the overall information ecosystem.
Read the full abstract ▾
All authors Stephanie T. Wang, Jeffrey Gleason, Yakov Bart, Christo Wilson, Danaé Metaxa
Human-Computer Interaction · Cross-listed from cs.CL
Artifact-centered Claim-aware Observability for Autonomous Scientific Agents
Xiangyu Yin, Ming Du, Michael H. Prince et al. (+1 more)
Autonomous scientific
agents now increasingly propose ideas, write code, run experiments, analyze results, and even draft papers. Observe and audit those
agents are necessary but logging every model call is not enough, scientists also need to inspect the artifacts and claims that the systems produced and their relations. This is driven by the fact that failures in scientific
agent systems are often distributed across several objects. A manuscript claim may cite the wrong evidence, a search process may select a degenerate candidate, a laboratory novelty claim may depend on an unstated rule, or a multi-
agent plan may change without a visible trigger. Existing tracing, experiment tracking, and archival provenance tools are valuable, but their native objects do not make these scientific audit relations first-class. We argue that autonomous scientific systems should emit portable, claim-aware artifact lineage as a minimum audit layer. We propose a compact observability profile organized around individuals, operators, fitness records, lineage, archives, runs, streams, and steering commands. In this profile, scientific claims are ordinary individuals with explicit evidence bindings and verification records. The profile is intended as a semantic layer that complements current telemetry and provenance standards. Execution details can remain in OpenTelemetry. Final packages can export to PROV-O or RO-Crate standards.
Read the full abstract ▾
All authors Xiangyu Yin, Ming Du, Michael H. Prince, Mathew J. Cherukara
Human-Computer Interaction · Cross-listed from cs.AI
ComponentBench: Diagnosing Component-Level Failures in Computer-Use Agents
Tianchen Guan, Xinlei Lin, Royce Cheng-Yue et al. (+2 more)
Current evaluation of computer-use
agents is split between long-horizon workflow
benchmarks and atomic GUI-grounding tests. This leaves an under-instrumented middle layer: realistic component-centered interactions (e.g., toggle a button set) that are short enough to diagnose and rich enough to capture the burdens of modern interfaces. We present ComponentBench, a
benchmark and diagnostic pipeline for component-level evaluation of computer-use
agents on modern web UIs. ComponentBench is organized around a library-agnostic ontology of 97 canonical UI components instantiated as 2,910 programmatically verified tasks across widely used component libraries, paired with cleaned human reference trajectories that enable evaluation of both task success and interaction efficiency. Beyond task collection, we introduce a scalable pipeline for auditing realized structural difficulty after implementation and synthesizing structured failure analyses across tasks and component families. Evaluating seven models – GPT-5.4, Gemini 3 Flash, GPT-5.4 mini, GPT-5 mini, Gemini 3.1 Flash-Lite, Qwen3-VL-235B, and UI-TARS-1.5-7B – across four observation and action spaces, we show that these design choices critically impact performance. Within a single shared harness, changing only the observation and action space shifts task success by more than 30% for the same model: GPT-5 mini falls from 83.1% with accessibility-tree observations to 48.9% with coordinate-only Pixel control. Moreover, even the fastest configuration takes 3.7x as long as the matched human reference, and spatial manipulations that are trivial for humans continue to challenge current
agents.
Read the full abstract ▾
All authors Tianchen Guan, Xinlei Lin, Royce Cheng-Yue, Xiangjun Wang, Shuyan Zhou
Human-Computer Interaction
Colour Blinded by the Noise
Harriet Mason, Rachel Rogers, Alison Kleffner et al. (+1 more)
Uncertainty visualisation is important for data transparency, especially for map visualisations where data is often aggregated. Despite the importance of this area, studies evaluating uncertainty visualisation lack consensus and produce conflicting results. This work introduces a new evaluation approach for uncertainty visualisation that attempts to assess uncertainty as noise, rather than signal. We evaluate five methods of visualising uncertainty: standard choropleth maps, value/variance bivariate maps, value-suppressing uncertainty palettes, overlaid sampling, and pixelated sampling maps. Built on principles of implicit testing, we put an 'uncertainty visualisation'
spin on the classic Ishihara colourblind test to create a novel test that is able to evaluate uncertainty as noise. We compare signal visibility to conventional hypothesis tests at various levels of group separation. By building our experimental design on top of established graphics theory, we isolate the plot components that facilitate successful signal suppression and establish foundational theory for the perception of uncertainty visualisation.
Read the full abstract ▾
All authors Harriet Mason, Rachel Rogers, Alison Kleffner, Dianne Cook
Human-Computer Interaction
Analysis of Types of Inquiries in Student-AI Interaction: A case study of two CS2 tasks
Matin Amoozadeh and Amin Alipour
Background and Context: Question and inquiry are integral parts of knowledge seeking and learning. Despite their importance, students tend not to ask enough questions in the classroom. However, studies have shown that students interact extensively with generative AI systems for learning and problem solving. Objective: In this paper, we seek to better understand the types of questions that students ask AI systems, and how those questions evolve during problem solving and across tasks. Method: We use the Graesser et al. taxonomy to classify students' inquiries into 18 types. We develop a few-shot learning approach to automatically classify students' interactions with AI into these categories. We use this system to analyze 830 interactions of CS2 students across two programming tasks. Findings: Our results suggest that a small subset of question types accounts for the majority of student inquiries, and that the types of questions students ask change substantially as the task progresses.
Read the full abstract ▾
Human-Computer Interaction
AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysis
Yangtian Liu, Yan Miao, Shuhan Liu et al. (+4 more)
Large
language models are pushing data science toward increasingly autonomous and agentic workflows, with recent systems already supporting multi-step and long-running analyses. As these workflows become more autonomous, conventional interfaces no longer provide adequate support for two critical requirements: observability for understanding an
agent's evolving
reasoning and evidence, and steerability for redirecting low-value directions or deepening promising ones during execution. Existing interactive approaches improve process visibility and open intervention points, but they remain largely designed for discrete, turn-by-turn exchanges rather than the parallel branches and evolving decision structures of long-running agentic analysis. We study this need as interactive oversight in long-running agentic data analysis and present AdaLens, an interactive system for monitoring and steering ongoing runs. AdaLens combines a storyline-based representation that unifies analytical plans, execution progress, intermediate findings, and data-column involvement with steering interactions grounded in these analytical elements for directional guidance and execution control. We evaluate AdaLens through two case studies and a user study, examining how it supports analysts in monitoring and steering long-running agentic data analysis.
Read the full abstract ▾
All authors Yangtian Liu, Yan Miao, Shuhan Liu, Yunfan Zhou, Dae Hyun Kim, Di Weng, Yingcai Wu
Human-Computer Interaction · Cross-listed from cs.LG
MotoSafety: Edge-AI with Learned Temporal Importance for Two-Wheeler Collision Risk Assessment Under Time Pressure
Sumit S. Shevtekar, Chandresh K. Maurya, Gourab Sil et al. (+1 more)
Powered two-wheeler riders face critical safety challenges in low- and middle-income countries, yet limited studies exist on how cognitive stressors such as Time Pressure influence collision risk. We address this gap by introducing a comprehensive
dataset consisting of over 129,000 labeled multivariate time-series feature windows, gathered across 153 simulator rides from 51 participants under No, Low, and High TP scenarios. Across each sequence, we capture 64 distinct attributes covering vehicle motion, rider control actions, spatial proximity, and rule compliance indicators. Using this
dataset, we introduce MotoSafety, a new edge-AI framework built on the Learned Temporal Importance (LTI) concept. MotoSafety achieves 94.97% accuracy and 99.33% ROC AUC, outperforming ten baselines, including TimesNet and LLM4TS, and achieves 0.039 MSE and 0.094 MAE for forecasting (4.4x lower error than Time-LLM and iTransformer). With only 1.15M parameters and 0.135 ms latency, it is suitable for edge deployment on low-cost CPU hardware. Using ground truth TP as an inductive bias improves accuracy from 94.09% to 94.97%, while predicted TP achieves 94.82%. Using only 21 IMU+GPS features, it achieves 93.91% accuracy, indicating practical deployment. Beyond PTW safety, the architecture shows better transferability to human activity (97.66%) and clinical (99.65%) domains. This lightweight framework advances PTW collision risk assessment, supporting the Safe System Approach for Intelligent Transportation Systems.
Read the full abstract ▾
All authors Sumit S. Shevtekar, Chandresh K. Maurya, Gourab Sil, Subasish Das
Human-Computer Interaction · Cross-listed from cs.CL
Interpretable Humans, Alien LLMs: Expert Analysis of Latent Structures in Assessment Responses
Alona Strugatski, Licol Zeinfeld, Jason Cooper et al. (+3 more)
The evaluation of large
language models (LLMs) relies heavily on human-designed assessments, implicitly assuming that AI and humans employ similar underlying cognitive constructs. Challenging this assumption, we investigate whether the latent factors governing LLM performance carry the same substantive, human-interpretable meaning as the cognitive constructs governing human learners. Using responses from humans and six LLMs across quantitative
reasoning and chemistry assessments, we conducted Exploratory Factor Analysis (EFA) separately for both groups. Subject-Matter Experts (SMEs) then blindly evaluated the resulting factor graphs to ascribe pedagogical meaning to the emerged constructs. SMEs successfully interpreted most of the human-derived factors. Conversely, they could not ascribe meaning to any LLM-derived factors in quantitative
reasoning and interpreted only half of the LLM factors in chemistry. By combining data-driven EFA with blind expert interpretation, this framework shows that LLMs frequently operate on statistically opaque mechanisms distinct from human
reasoning.
Read the full abstract ▾
All authors Alona Strugatski, Licol Zeinfeld, Jason Cooper, Shelley Rap, Gil Schwarts, Giora Alexandron
Human-Computer Interaction
From Substitution to Scaffolding: Breaking the Self-Reinforcing Harm Cycle of AI in Education (and Beyond)
Lucile Favero, Juan Antonio Pérez-Ortiz, Tanja Käser et al. (+1 more)
Artificial intelligence is being adopted in educational settings faster than its consequences are understood. We argue that the central risk is misalignment: AI that eliminates human effort erodes the very capacities education is meant to build. We organize this risk into an integrative framework of four interrelated dimensions -cognition, agency, emotional well-being, and ethics- linked by a self-reinforcing cycle where cognitive offloading reduces effort, weakens agency, and compounds emotional and ethical harm. We ground the framework in the perspective of a small cohort of students: an exploratory analysis of 49 International Baccalaureate argumentative essays about the impact of AI reveals that learners perceive these risks, with $80\%$ of essays reporting that AI reliance reduces thinking. At the same time, the essays articulate a consistent
vision of the AI the students want: systems that support rather than replace learning by withholding immediate answers, prompting recall, and encouraging reflection through questions instead of solutions. These desiderata closely align with established principles from the learning sciences. Building on these insights, we propose a single design principle, scaffold, do not substitute. We argue that this principle extends beyond education. It represents a broader challenge for the AI ecosystem: any system that mediates human thinking can either weaken human capabilities through substitution or strengthen them through scaffolding. We conclude by outlining a research agenda for developing AI systems that foster enduring human capacity, an imperative not only for learners but, ultimately, for democratic societies.
Read the full abstract ▾
All authors Lucile Favero, Juan Antonio Pérez-Ortiz, Tanja Käser, Nuria Oliver
Human-Computer Interaction
The Oracle of Chemnitz: An interactive art installation to reanimate old things in a garage featuring a rotary phone
Karola Köpferl and Albrecht Kurze
Garages have a long tradition of tinkering, creativity and innovative change. School of Garage, a participatory artistic summer school project in Chemnitz, the European Capital of Culture 2025, took up this tradition and turned old Eastern Bloc garages into temporary ateliers for collaborative making and discussion. In our HackLab garage we conceptualized and created the Oracle of Chemnitz within one week. It gives a place filled with history back its stories. It is an interactive installation of artifacts from the past typically found in garages: an old typewriter,
radio, desk, tires, mixer and a rotary-dial telephone. Each got a name, personality and story to tell. The phone rings when a visitor approaches. Once answered, it asks for name and month of birth before a story about a device is told, along with hints to other places in the city. Around 2,700 visitors interacted with the system over three months.
Read the full abstract ▾
Human-Computer Interaction
Procedural Collapse: A Structural Account of Disengagement in LLM-Assisted Writing
JaeWon Kim and Katelyn X. Mei
When students use large
language models for writing, the dominant explanation for disengagement is dispositional: they are over-reliant, and the remedy is to scaffold self-regulation. We argue that a structural explanation is needed, offering an alternative basis for design interventions to support appropriate AI-assisted writing. Current LLM writing interfaces induce procedural collapse: the replacement of an iterative, self-paced writing process with a single output that shifts the writer's task from generation to comprehensive evaluation. Because that evaluation is costly, shallow engagement becomes the default, and the cognitive work writing was supposed to produce goes unperformed. The framework points toward design directions that reduce the burden on writers to self-regulate, including decomposed interaction, goal elicitation as a default first step, and single-level output. They complement metacognitive scaffolding by restructuring the interaction itself.
Read the full abstract ▾
Human-Computer Interaction
SleuthTalk: Supporting Historical Photo Identification with Private Workspaces for Collective Sensemaking and Deliberation
Liling Yuan, Vikram Mohanty and Kurt Luther
Identifying individuals in historical photographs is a critical task across fields such as history, journalism, genealogy, and archival research. While AI-based facial recognition can efficiently generate candidate matches, it often produces ambiguous results that require deeper analysis and contextual interpretation. Existing platforms lack robust support for collaborative deliberation, especially in uncertain or high-stakes cases. We present SleuthTalk, a private collaborative workspace integrated into Civil War Photo Sleuth, designed to scaffold structured comparison, discussion, and group decision-making. SleuthTalk enables users to curate custom shortlists, annotate facial features, and build consensus through structured feedback. In a mixed-methods evaluation with experienced historical photo researchers, SleuthTalk enhanced self-reported confidence, surfaced diverse perspectives, and supported transparent, reflective identifications.
Read the full abstract ▾