Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Human-Computer Interaction

cs.HC Watch Human-Computer Interaction (sign in to use)

Thursday 24 September

Human-Computer Interaction

How People Use ChatGPT in Australia: A WildChat Analysis

Generative AI chatbots are increasingly embedded in everyday life, yet most large-scale studies describe global patterns. This paper presents an Australia-focused analysis of WildChat, a public dataset of real-world ChatGPT interaction logs. Using descriptive analysis and a multi-layer classification scheme, we analysed 37,845 conversations identified as Australian, examining language diversity, work relevance, interaction intent, topic distribution, turn-taking, temporal change, work activities, and Australia-related domains. Our findings show that the Australian subset is strongly action-oriented and comparatively work-oriented, with most interactions classified as doing and a majority of conversations classified as work-related. The dataset also shows multilingual use and a growing presence of self-expression over time. Australia-related conversations frequently invoke local institutions, laws, regulators, education systems, companies, cultural references, and public services. Finally, we outline implications for future research, including local AI evaluation, multilingual participation, context-aware design, and safeguards for everyday high-stakes domains.
Read the full abstract

All authors Ying Ma, Katy Gero, Clément Canonne, Craig Jin, Kanchana Thilakarathna

Human-Computer Interaction

AnomaSense: Anomaly-based Sensor Activation for Fine-Grained Human Activity Recognition

Audio carries rich cues about human activities, and microphones are already built into most wearable devices. However, microphones also capture speech, and this privacy risk limits their use in Human Activity Recognition (HAR). We present AnomaSense, a sensor activation approach for wrist wearables that keeps the microphone off by default and turns it on for at most one second when an unsupervised anomaly detector flags an IMU segment that is likely to produce sound. The captured audio is further masked before it reaches the recognition model. We study 20 activities from 15 participants, organized into five groups in which activities share similar wrist motion but differ in the object or material involved. With IMU data alone, our recognition model reaches 78.98% accuracy in leave-one-participant-out validation. With the short, masked audio windows added, accuracy reaches 96.89% with no masking and stays above 86% when 90% of each one-second audio window is removed. On the same data, the anomaly detector triggers the microphone with 86.46% precision and 74.28% recall relative to sound events. We also report a small preliminary check of automatic speech recognition on masked speech, which shows that contiguous masking degrades recognition far more than point-wise masking at the same masking ratio. Our evaluation is a controlled, offline feasibility study. We describe the threat model, what the approach does and does not protect, and the steps needed before deployment.
Read the full abstract

Human-Computer Interaction

HelpCoach: Scaffolding Targeted AI Help-Seeking During Problem-Solving

Students increasingly turn to AI for help with problem-solving, yet too much AI support can undermine learning itself. To benefit from AI, students need to specify the necessary knowledge and scaffold type in their questions. However, they struggle to formulate such targeted questions because they lack metacognitive skills to recognize and select effective help options. We developed HelpCoach, an add-on for chat interfaces that helps students formulate knowledge- and scaffold-specific questions and receive targeted help during problem solving. HelpCoach continuously assesses students' help-seeking performance and prompts students to improve through an adaptive revision template. Whereas prior work has largely taught help-seeking skills apart from learning tasks, HelpCoach's in situ scaffold enables concrete practice on metacognitive skills and immediate revisions to help-seeking behavior. In a study with 40 college students learning web programming, HelpCoach led to more specific questions during chatbot interactions and greater knowledge retention than pre-task help-seeking training alone.
Read the full abstract

All authors Hyoungwook Jin, Weirui Peng, Jieun Han, Q. Vera Liao, Xu Wang

Human-Computer Interaction

Characterizing LLM-Based Family Education through the Lens of Activity Theory: A Scoping Review of the HCI Literature

Large language models (LLMs) are increasingly involved in family education, yet HCI has not systematically explained the educational interactions that emerge around them. This scoping review analyzes 53 HCI studies from 6,540 records across 19 venues. Using activity theory and AODM, it relates participants and educational objects to mediation, labour, and rules. We find that the literature centers on child–parent interaction and on language, AI literacy, and relational learning. The introduction of LLMs enabled conversational, embodied, and spatial systems to generate support from the context of an unfolding interaction. LLMs redistributed educational labour, while family and institutional rules left parents and professionals responsible for interpreting outputs and deciding how they entered practice. Evidence across families and educational purposes remains limited, especially on sustained personalization, repair labour, and how families negotiate authority and rules. The review offers a framework explaining how LLM capabilities become organized through family participation.
Read the full abstract

All authors Lan Luo, Yuqi Liang, Jie Cai, Anqi Wang, Dongyijie Pan, Muzhi Zhou, Chun Yu, Pan Hui

Human-Computer Interaction

"You Can't Just Automate It": Negotiating and Sustaining a "Good" Family Life Through Energy Practices

This study examines how Taiwanese parent-child families negotiate a "good" family life through everyday energy use and imagine future smart homes that support it. We conducted in-home interviews and co-design sessions with 21 families, including 46 parents and children. We found that families pursued a good life through energy practices shaped by thrift, comfort, care, safety, and enjoyment. These arrangements were continually adapted and responded to changing bodies, schedules, people, and infrastructures. This adaptive work was unevenly distributed, which in turn shaped different smart-home imaginaries. Drawing on the lens of Nearby and adversarial design, we conceptualize adaptation as situated sociotechnical work through which families continually rework energy arrangements. We further distinguish collective goods from plural and contestable goods to show why family IoT must support shared values while preserving opportunities to question and revise household arrangements. We offer theoretical and design directions for more adaptive, participatory, and contestable family IoT.
Read the full abstract

All authors Yang Hong, Ying-Yu Chen, Wei-Chien Chang, Yu-Hsin Chou, Sharifa Sultana

Wednesday 23 September

Human-Computer Interaction

Signals of AI Hallucination: Designing Hallucination-Aware Cues for Embodied Conversational Agents in VR

LLM-powered conversational agents (CAs) often present uncertainty and provenance cues alongside their responses to help users assess response reliability and identify potential hallucinations. In immersive environments such as Virtual Reality (VR), CAs often take the form of speech-based embodied conversational agents (ECAs), where uncertainty and provenance cues cannot rely on persistent inline text and may be missed or disrupt comprehension when delivered through speech. We conducted a within-subjects study (N = 24) to compare three designs for presenting the hallucination-awareness information (uncertainty and provenance) in ECAs in VR against a no-cue baseline: embodied cues using gestures and posture, icon cues using visual indicators, and text cues using color-coded text with inline citations. We evaluated how these designs affect users' ability to identify hallucination-related information, trust in the ECA, and interaction experience (immersion and task load). Our results show that all three designs support users in identifying hallucinations. Embodied cues were associated with higher trust and immersion, text cues offered clearer interpretability, and icon cues preserved relatively good interpretability while causing less disruption to immersion compared with embodied cues and text cues. This work contributes to the VR and AI research community by comparing different designs of hallucination cues in immersive ECA settings and examining how they affect users' ability and experiences to identify hallucinations. It also offers practical insights and design implications for developing future hallucination-awareness interfaces for ECA.
Read the full abstract

All authors Xiaoran Yang, Yang Zhan, Xie He, Yuxuan Huang, Yichen Yu, Zhuo Wang, Noboru Matsuda, Qiao Jin

Human-Computer Interaction

The Interface Is Downstream: Designing the Terms of Human-Agent Collaboration

Before an agent responds or acts, much of the experience has already been designed. Memory and retrieval shape what it notices. Evidence rules shape what it may claim. Permissions shape what it can do. Learning rules shape what it carries into the next encounter. The argument comes from Alicia, a personal agent I've built and used since January 2026. A fine-tuning pilot produced no defensible model-performance result. It exposed a provenance failure: Alicia repeated an interpretation from a retrieved synthesis, cited a source note credited by that synthesis, and left the synthesis out of the visible chain. In a model-blind review of thirty-three citations, one reviewer judged that the retrieved intermediary supplied the claim in sixteen relayed citations and part of it in four. Five of twelve citations to directly retrieved targets lacked support in the target excerpt supplied for review. These judgments remain unadjudicated, and the packet is not public. I call the shared setting a humorphic environment: a persistent computational setting that translates a human practice into software. The first Humorphism paper translated partnership. This paper translates the studio, the room where practice happens. The failure prompted an audit of attention, evidence, action, and learning, with a review artifact and available recourse for each. The test is whether the person can inspect and contest what shaped the teammate's behavior. The output is downstream. Correction, consent, and learning carry the collaborative interface back upstream.
Read the full abstract

Human-Computer Interaction

Data Patching

To provide data-driven quality services to their citizens, every institution determines the acceptability of the citizen data and datafied mechanisms through institutional protocols, including preset standards and policies. However, data meeting standards at one institution might fail to meet a different institution's standards in subsequent phases of the intended citizen services due to mismatched protocols, leading to data devaluation in the citizen service ecosystem. Terming this phenomenon cross-institutional data devaluation, we investigate its causes and workarounds through interviews with 41 Bangladeshi participants. We found that traditional data auditing mechanisms cannot solely address data devaluation; hence, we draw on our findings and theory of cross-institutional AI audits to propose the Citizen-centered Cross-institutional Data Audit (CCDA). We also discuss design and policy implications of CCDA in HCI and datafied citizen services.
Read the full abstract

Human-Computer Interaction

How Spatial Biologists Direct and Verify AI-Assisted Analyses

Spatial biologists use visualization to assess computational analyses of tissue data. We examine how they direct and verify analyses when an AI agent performs this work. We synthesized workflows from fourteen contextual inquiries and conducted a formative pilot followed by an observational study with ten spatial biologists using Claude Science on their own data. Participants valued help with plotting, locating cells of interest, and tasks they found laborious or could not otherwise perform. Assessing the agent's work involved obtaining suitable evidence, sometimes through additional work in external tools, and interpreting it using knowledge of the tissue and its markers. Scientists also sought information about ongoing computation to decide how analysis should proceed. We contribute a workflow synthesis, an empirical account of scientists directing and verifying agentic analyses, and four design directions addressing execution control, familiar interactive views, source and execution information, and accessible verification across computing setups and experience.
Read the full abstract

All authors Ella Hugie, Alexandra Irger, Chiara Schiller, Lukas Hatscher, Denis Schapiro, Hanspeter Pfister

Human-Computer Interaction

Assembling, Breaking, and Refusing the Mask: Agency in AI-Mediated Self-Presentation in Livestreaming

Our mixed-method study examines how Chinese women livestreamers use masking to construct idealized mediated personas while navigating gendered, commercial, organizational, and platform pressures alongside personal agendas. We built on the concept of masking, analyzed 627 recruitment posts, and conducted livestream observations and interviews with 26 Chinese women streamers. We found that streamers assembled masks across bodies, AI-mediated technologies, spaces, performances, and social relations to become recognizable while protecting personal boundaries. These masks were continually negotiated, and participants sometimes broke, resisted, or refused them when demands became misaligned or unsustainable. We conceptualize masking as a sociotechnical assemblage in which agency lies in preserving, disrupting, and reconfiguring relations rather than controlling a single interface. We further theorize breaking as a consequential part of masking that exposes hidden labor and unequal costs of visibility. We offer theoretical and design directions for more negotiable, contestable, and agency-supporting AI-mediated self- presentation.
Read the full abstract

Human-Computer Interaction

Understanding Creative Design Practices among Data Artists

Creative and artistic data visualizations communicate stories and invite engagement, yet how designers develop their expressive forms remains poorly understood. We investigate the design process of data artists through three complementary studies: an analysis of 40 public project accounts, artifact-anchored interviews with seven experienced data artists, and a design task-based study with eight data artists. We find that stories and visual forms develop together through data exploration, reference adaptation, sketching, and prototyping. Inspiration comes from data, existing work, everyday imagery, and personal experience. Sketches help develop mappings, while prototypes with real data can reshape representations and intended stories. Designers consider multiple possibilities but typically develop one direction at a time, partly because producing alternatives is costly. Their choices balance meaning, visual appeal, readability, and feasibility. These findings inform tools connecting stories, references, and real data, including AI assistance that supports testing and revising alternatives while preserving designers' creative judgment.
Read the full abstract

Human-Computer Interaction

"A Necessary Evil": Teenagers' Sensemaking of Privacy and Safety Settings on Social Media

Social media platforms are embedded in teenagers' daily lives, supporting friendship and identity while exposing teenagers to unwanted contact and privacy harms. Previous scholarship has documented how attention capture strategies and dark patterns shape social media use, and we extend this work to better understand platform settings that ostensibly provide privacy and safety protection. We report on think-aloud sessions with 11 teenagers aged 14 to 17 who completed six privacy and safety tasks on Instagram, TikTok, Snapchat, and YouTube. We show how participants worked out what a setting meant through their routines, boundaries, and prior experiences, how they accommodated protections softer and less predictable than expected, and how they treated the platform as the authority on what protection should look like. We argue that feature-by-feature evaluation cannot establish whether teenagers are protected, and that platforms should carry the obligation to show that a protective action took effect and is durable.
Read the full abstract

All authors Jingxin Dong, Lingyun Chen, Chen Ling, Colin M. Gray

Human-Computer Interaction

Available but Not Usable: Dark Patterns and Interaction Cost in Social Media Privacy and Safety Settings for Teens

Social media platforms are central to teenagers' lives, and their designs can expose users to privacy, safety, and wellbeing harms. Platforms increasingly offer protective settings, though the presence of a control reveals little about whether teenagers can find, use, and benefit from it over time. We paired an expert evaluation of six privacy and safety tasks across TikTok, Instagram, Snapchat, and YouTube with moderated think aloud sessions in which 11 teenagers aged 14 to 17 attempted the tasks. Interaction cost and dark patterns analysis allowed us to compare the complexity designed into each task with the effort participants incurred as they located, configured, and interpreted controls. Recurring dark patterns appeared across tasks, and most participant attempts exceeded the expert baseline. Protective settings therefore risk being insufficiently usable or durable in practice, and we propose a wayfinding audit that integrates expert evaluation, usability testing, interaction cost, and dark pattern analysis.
Read the full abstract

All authors Jingxin Dong, Lingyun Chen, Chen Ling, Colin M. Gray

Human-Computer Interaction · Cross-listed from cs.RO

Talk2Escape: Conversational Grounding for Vision-and-Language Navigation

While Vision-and-Language Navigation (VLN) has demonstrated remarkable success, the prevailing single-turn paradigm exposes a fundamental vulnerability: agents operate in a strictly open-loop manner. In practice, factors such as perceptual aliasing, sensor noise, and odometry drift can cause minor deviations to accumulate over time, often leading to catastrophic mission failures with no built-in mechanism for error recovery. To address this, we introduce Talk2Escape, a proactive and model-agnostic dialogue intervention framework that reframes navigation as a closed-loop interactive process. At its core, a lightweight vision-language module continuously monitors agent kinematics. Upon detecting localized looping or severe trajectory divergence, it translates raw egocentric observations into concise, grounded queries to solicit targeted corrective feedback from either an algorithmic oracle or a human-in-the-loop. Extensive evaluations in high-fidelity simulators, including R2R-CE, RxR-CE, and VLNVerse, demonstrate that Talk2Escape exhibits consistent improvements across diverse base agents. Empirically, Talk2Escape achieves a 66.0% Success Rate on R2R-CE, outperforming the current supervised and zero-shot state-of-the-art methods. We further validate its sim-to-real transfer on a Unitree Go2 quadruped, proving that proactive dialogue drastically improves navigation robustness in physical environments.
Read the full abstract

All authors Zerui Li, Sihao Lin, Yanyan Shao, Jiwen Zhang, Xiangyu Shi, Shijie Li, Qi Wu

Human-Computer Interaction · Cross-listed from cs.SE

LLM-Assisted Workflow for Structural Difference Visualization in Evolving Software Requirements

This paper presents an LLM-assisted workflow for visualizing structural differences in evolving software require- ments. Implemented in the OntologyWeb environment, the work- flow represents baseline and current requirements as triple-based semantic graphs and supports side-by-side comparison of curated graph snapshots. The comparison view aligns matched entities and uses visual encoding to highlight structural changes.
Read the full abstract

2609.28002 · 23 Sep · o̧pyright} 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Human-Computer Interaction

Open Questions Towards Skill-Sustaining Reliance in Reflective AI Engagement

As AI systems are increasingly integrated into professional work, reflection strategies such as cognitive forcing and prompts that foster critical engagement have shown promise in reducing overreliance and improving decision quality. However, these strategies have primarily been evaluated as short-term interventions within single sessions. The next challenge is to assess whether such mechanisms sustain human agency and expertise over time. Drawing on prior work in AI-assisted decision-making, metacognition, and reflective AI engagement, we examine the challenges of designing and evaluating reflective mechanisms for long-term skill sustainability, considering individual differences in how users engage with such support, the organisational conditions under which it is implemented, and the gap between short-term evidence and long-term claims. We introduce open questions for the research community about the conditions under which reflective AI engagement can be sustained in practice.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CL

Brain-to-Language Decoding: Tasks, Signals, Methods, Evaluation, Practical Use and Beyond

Brain-to-language decoding translates neural activity associated with language production, internal speech and perception into linguistic or expressive outputs. It offers a route to restoring communication after speech loss and a means of studying how the brain represents language. Advances in neural recording and representation learning have expanded the field from constrained recognition and acoustic reconstruction to text generation, streaming personalised speech and facial animation. This survey synthesises these developments across invasive and non-invasive measurements, drawing on a search without a lower year limit and source-led updates through September 2026. We connect Articulated, Inner and Perceived tasks to the neural populations they engage, the representations available to decoders and the outputs those representations can support. We examine model development, public resources and the evolution of evaluation, and compare published performance and communication costs within their reported protocols. The synthesis identifies complementary routes to progress: phonetic, acoustic and semantic targets preserve different aspects of a message; shared representations support reuse across recording conditions and tasks; and online communication increasingly depends on calibration, feedback and user control alongside decoding accuracy. Shared benchmarks enable algorithmic comparisons, while longitudinal studies reveal the demands of sustained use. We discuss these developments and their remaining limitations, then outline a prospective five-level trajectory from commands and language to meaning, scenarios and bidirectional cognitive exchange
Read the full abstract

All authors Yiqian Yang, Yiqun Duan, Chenyu Liu, Yiqi Wang, Xinliang Zhou, Chin-Teng Lin, Yu Zhang

Human-Computer Interaction

ViMoWear: Visual Motion-Guided sEMG-IMU Representation Learning for Subject-Independent Thumb Gesture Recognition

Wearable sensing enables intuitive hand gesture recognition for human–computer interaction, augmented reality, and prosthetic control, yet subject–independent recognition remains challenging because wearable signals provide only indirect and highly subject-specific observations of hand motion. Although visual information can improve wearable gesture recognition, requiring it during inference increases sensing complexity and limits practical deployment. We propose ViMoWear, a visual-motion-guided framework that leverages synchronized 3D hand motion as training-only supervision while requiring only wearable sensing for gesture classification at inference. Specifically, Motion-Guided Cross-Subject Contrastive Learning (MGCL) promotes subject-robust representations, and Thumb-Aware Masked Motion Reconstruction (TMMR) preserves fine-grained motion information. The leave-one-subject-out experiments on a synchronized sEMG–IMU–pose dataset demonstrate consistent improvements over supervised baselines across multiple sensing configurations, while the learned representations also support classifier-free retrieval. The proposed training-only visual motion supervision improves the generalization of wearable representations to unseen subjects.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.MM

When Visual Quality Misleads: Intent Recognition under Rendered Avatar Distortions

Avatar-streaming systems are commonly evaluated with image and video quality assessment (IQA/VQA) metrics, implicitly treating visual fidelity as a proxy for communicative success. We test this assumption through a controlled behavioral study of rendered 3D avatars across a pristine condition and fourteen geometric, photometric, temporal, and combined distortions. Fifty-nine participants contributed 2,688 judgments of perceived action, response confidence, and visual quality. We identify Misleading Quality in this dataset as distorted renderings that retain above-average perceived quality but yield below-average action-recognition accuracy. We also derive an Intent Quality Score (IQS) combining recognition correctness and confidence as the behavioral target for objective metrics. Among 126 distorted content–condition cells, 31 (24.6%) exhibited Misleading Quality; temporal and geometric distortions showed the highest rates, at 50.0% and 31.1%, respectively. The results reveal a quality–accuracy dissociation where distortion families affect appearance and communication differently. Across 24 direct-scoring IQA/VQA metrics and three supervised feature-regression baselines, alignment with IQS remained limited; at $\lambda=0.5$, the best leave-one-content-out baseline reached PLCC $=0.4435$. Under this controlled protocol, visual fidelity alone is insufficient for avatar communication, motivating intent-aware quality assessment and streaming objectives.
Read the full abstract

Human-Computer Interaction

Forced Yet Free: What Magicians' Forcing Reveals Beyond Intentional Binding

In magicians' forcing, spectators may experience a choice as self-determined even when that choice has been externally directed. Research on the sense of agency has developed largely around action-outcome relations, most notably intentional binding; however, the problem of choice authorship (why a particular choice is experienced as originating from oneself) must be treated as distinct. This paper integrates research on agency and forcing by distinguishing the locus of intervention within the choice-action-outcome chain, self-attribution at the Decision, Action, and Outcome levels, and the processes by which feeling of agency and judgment of agency are constructed. On this basis, a distinction is drawn between intentional binding, which concerns the temporal/causal relation between action and outcome, and the different binding problem exposed by forcing: the relation between a choice and its author. This theoretical relation is termed authorship binding. Authorship binding does not denote a new implicit measure; rather, it refers to the constructive relation through which a choice whose formation has been directed by external factors can nevertheless be experienced as originating from the self. Forcing can sustain this relation not by eliminating agency, but by selectively preserving, substituting, and redistributing agency cues. XR, in which body-centred spatial relations can be manipulated, is further conceptualised as a medium for extending the forced-yet-free structure into space.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.SD

Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models

End-to-end speech-to-speech dialogue models listen and speak simultaneously, so a continuously open acoustic channel is exposed to adversarial manipulation. We formalize imperceptible attacks on full-duplex agents as optimization over additive perturbations confined beneath the psychoacoustic masking threshold of the carrier speech, under three goals: targeted semantic hijacking, response suppression, and policy jailbreaking. Against an undefended Moshi-style agent, white-box attacks succeed in up to 91.7% of trials. We then introduce psychoacoustically aligned latent smoothing (PALS), which injects anisotropic Gaussian noise shaped by local codebook covariance at the residual-vector-quantized latent interface, with input noise shaped by the masking threshold constraining the attacker and trained by a Kullback–Leibler consistency objective. Deployed with no inference-time cost, PALS reduces hijack to 8.3%, mute to 11.2%, and jailbreak to 9.1% at clean quality within 2.3%. A Monte Carlo-smoothed variant certifies an ellipsoidal latent radius up to 0.616, a guaranteed floor that the empirical robustness far exceeds.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CL

Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Models

Benchmarks for full-duplex spoken dialogue models score turn-taking with binary fixed-window rules that reward immediate response or silence by completeness of the prior turn. We argue that the appropriateness of a response offset, whether delayed silence or anticipatory overlap, is conditional on the speaker's latent intent, identifiable only from that speaker's behavior. We introduce TACT, a benchmark of 9,728 episodes and 73.2 hours from five dyadic corpora; each episode carries dialogue history, a per-speaker memory profile, and an annotator-derived posterior over six intent classes. Scoring replaces binary windows with a strictly proper threshold-weighted continuous ranked probability score whose weights are intent-conditioned timing kernels fitted to human floor-transfer-offset distributions, proving boundedness, consistency, and binary reduction. Across eleven systems the best model reaches 0.47 against a human topline of 0.86, is nearly invariant to speaker profiles, and TACT agrees with human judgments at Spearman 0.81 versus 0.46 for binary metrics.
Read the full abstract

Human-Computer Interaction

ASAP: Visual Analytics for Identifying and Analyzing Image Patterns in AI-generated Images

Generative image models can produce highly realistic images, raising concerns about potential misuse in creating deceptive content. Current deepfake approaches face several challenges, including limited generalizability, lack of interpretability, and poor actionability. To help address these, we present ASAP, an interactive visualization system designed to empower users in the analysis and summarization of deceptive patterns in AI-generated images. ASAP introduces a novel CLIP-adapted image encoder that generates interpretable representations, enabling the extraction of influential pixel regions via calculated masks. This approach facilitates the identification of key deceptive features through influence measurement techniques. These backend techniques are integrated into a visual analytics dashboard that allows users to quantify and analyze authenticity-indicative patterns in image collections containing both authentic and AI-generated images. This approach also supports the comparative analysis of various generative models, including GANs and diffusion models. We demonstrate ASAP's efficacy through a user study and two application scenarios using established fake image detection benchmarks, showcasing its ability to effectively extract and quantify deceptive patterns.
Read the full abstract

All authors Jinbin Huang, Yuki Ueno, Chen Chen, Aditi Mishra, Bum Chul Kwon, Zhicheng Liu, Chris Bryan

Human-Computer Interaction · Cross-listed from cs.CV

Can Vision-Language Models Analyze Human-Centered Video? Mapping Model Capabilities and Human-AI Collaborative Workflows

Video provides a rich record of human behavior, interaction, and situated contexts, offering important evidence for understanding people and conducting human-centered research. As vision-language models (VLMs) become increasingly capable of analyzing video, they offer opportunities to automate this traditionally human-intensive process. Yet a central question remains: when can VLMs analyze human-centered video independently, and when does reliable analysis still require human involvement? To address this question, we first characterize video analysis practices in human-centered research. We systematically analyze all 1,702 CHI 2026 full papers and identify 125 that annotate videos. Through iterative coding, we derive a five-dimensional taxonomy spanning analytic purpose, viewpoint, phenomenon, reasoning requirement, and annotation authority. Grounded in recurring annotation tasks captured by this taxonomy, we construct a benchmark of 15 representative tasks from open datasets to map the capabilities and limitations of a general-purpose VLM. We examine the division of labor between humans and VLMs by comparing three annotation workflows: VLM alone, human alone, and human verification of VLM outputs. Across tasks, VLM-alone annotation approaches human accuracy on average (HNS = 97.0, where 100 denotes human-alone performance), demonstrating substantial potential to automate human-centered video analysis. Human verification achieves the highest accuracy (HNS = 121.5) while reducing human annotation time by 48.9% and monetary cost by 31.3%-44.5% relative to human-alone annotation. Our findings connect real-world human-centered video analysis tasks and current VLM capabilities, and clarify how human-AI collaboration can make VLM-assisted analysis reliable and efficient.
Read the full abstract

All authors Xiyuan Shen, Jiuyang Lyu, Seokhyun Hwang, Huanfen Yao, Shwetak Patel, Zhihan Zhang, Jacob O. Wobbrock

Human-Computer Interaction

Listening and Mirroring: The Effects of Verbal Attunement and Behavioral Mimicry on Social and Empathic Perceptions of Embodied AI Agents in VR

As embodied agents take on increasingly social and relational roles in VR, visual realism and embodiment alone may be insufficient; users must also perceive these agents as emotionally attuned, supportive, and humanlike. Prior work suggests that verbal attunement and nonverbal mimicry can each improve users' social evaluations of embodied agents. However, behavioral mimicry has largely been studied outside of real-time, conversational AI interactions, leaving limited understanding of how users respond when an agent simultaneously generates contextually responsive dialogue and adapts its nonverbal behavior during an immersive conversation. To address this gap, we developed an embodied AI counselor that combines conversational AI with real-time facial-expression and posture mimicry, while producing either verbally attuned or neutral responses. We evaluated the system in a 2 X 2 within-subjects study with 20 participants, manipulating verbal attunement and behavioral mimicry. Results showed that verbal attunement was the most reliable driver of perceived empathy. Behavioral mimicry showed a marginal relationship with perceived humanness, while greater mimicry exposure showed preliminary, exploratory positive associations with empathy, positivity, and humanness, particularly among female participants. Together, these findings show that multimodal synchrony is not a simple additive strategy for designing empathic conversational agents in VR and underscore the need to consider how verbal and nonverbal behaviors are combined during real-time interaction.
Read the full abstract

All authors Nathalia Gomez, Haig Shamlian, Omar Khan, Tiffany D. Do

Human-Computer Interaction

CoBranchMR: Supporting Parallel Design and Conflict Resolution in Mixed Reality

We present CoBranchMR, a mixed reality (MR) system that enables distributed collaborators to work in parallel from different locations on the same digital representation of a physical object. CoBranchMR lets users branch an object into editable virtual copies, customize them independently, and then merge their work back into a shared object. When merging copies, the system displays potential conflicts on the object's surface and provides several resolution options. By adopting branch-and-merge workflows for embodied spatial collaboration, CoBranchMR introduces a new collaborative interaction model that supports parallel design, conflict resolution, and negotiation in remote creative work.
Read the full abstract

All authors Niloofar Sayadi, Kaiyuan Tang, Yunhao Xing, Simret Gebreegziabher, Chaoli Wang, Diego Gomez-Zara

Tuesday 22 September

Human-Computer Interaction · Cross-listed from cs.CR

Privacy Leakage Through AI-mediated Analysis of Smartphone Data

Over the past thirty years, the online advertising industry built a large-scale data collection ecosystem, with the goal of tracking a user's online activity to infer their demographics and interests. Traditionally, the ecosystem relied upon the collation and analysis of highly-structured text data like user IP addresses, GPS coordinates, e-commerce purchase histories, and visited URLs. However, recent ML models can parse not only structured text, but also multimedia files and unstructured text inputs—meaning a user's photos, videos, inboxes, and calendars are now ripe for automated analysis. The privacy risks are particularly acute in the context of smartphone apps. A user's phone already acts as a natural collation point for sensitive user information, but users may not understand that permitting an app to, for example, access a user's photo does not just give the app access to the bytes in the photo: the app also receives access to inferences about the user that are enabled by the photo. To explore these privacy risks, we built Priva-See, an LLM-based inference system for app-collected user data; Priva-See reflects our best understanding of how real-life adtech companies would leverage machine learning to build user profiles. Through an IRB-approved user study, 465 participants deployed Priva-See on their phones; Priva-See made privacy-invasive inferences despite having access to only a subset of a user's data. We see the experience significantly impacted participant willingness to share permissions data moving forward. Based on the observed privacy violations, we suggest changes to how smartphone OSes should gather user consent for data access, to better inform users about downstream data usage capability.
Read the full abstract

All authors Sarah Radway, Zoe Robert, Matthew Soto, Julianna Cimillo, Sebastian Diaz, Meg Marco, James Mickens

Human-Computer Interaction

ChartRevive: Reconstructing Data Visualizations from Chart Images Using MLLM

Static chart images are widely used in scientific publications, business reports, and presentations, yet recovering both the underlying data and visual design from chart images remains a labor-intensive manual process, making them difficult to reuse. While prior work has primarily focused on data extraction, the extraction of visual design specifications, including colors, marker shapes, and axis configurations, remains underexplored. To identify a suitable model for chart reconstruction, we systematically benchmark five multimodal large language models (MLLMs) across five basic chart types on both data and design extraction tasks. Our evaluation shows that textual and categorical information can generally be extracted reliably, whereas numeric and spatial information remain challenging. Among the evaluated models, GPT-5.4 achieves the best overall performance and is adopted as the backbone of our system. Guided by these findings, we present ChartRevive, a mixed-initiative system that combines MLLM-based extraction with an interactive verification interface, supporting users to efficiently inspect, correct, and refine reconstructed charts through overlay-based verification and real-time rebuilding.
Read the full abstract

Human-Computer Interaction

When Direct Manipulation Becomes a Guess: Productive Friction in AI-Mediated Multisensory Visualization

Generative visualization increasingly embeds large language models within direct manipulation and multisensory interaction. Speech, gaze, touch, gesture, sound, and haptics can make probabilistic inference feel like familiar, deterministic tool use. I call this gap a deterministic-affordance mismatch: deterministic interaction cues persist while AI weakens predictability, locality, reversibility, or provenance. Reading the malleable interfaces of Iron Man 2 against systems from Blade Runner 2049, I propose productive friction, including cross-sensory renderings whose detail reflects model uncertainty. Four frictions expose the inference boundary, match action to effect, make stochastic branches tangible, and attribute sensory agreement.
Read the full abstract

Human-Computer Interaction

ContraVis: Evidence-Grounded Visual Analytics for Contradiction Review in Legal Contracts

Legal contracts are structurally complex documents in which contradictions may emerge across distant and interconnected provisions. Although large language models (LLMs) improve legal language understanding, contradiction analysis remains a human-centered and evidence-grounded review task. We present ContraVis, a visual analytics system for human-in-the-loop contradiction analysis in legal contracts. The system models contracts as typed paragraph graphs that combine explicit contractual references with semantic relationships between paragraphs. This graph plays a dual role: it conditions LLM reasoning and serves as the interactive representation the analyst explores, keeping model context and human inspection aligned across coordinated views. In a controlled comparison, graph-conditioned reasoning recovered more injected contradictions than standalone LLM analysis as contract length grew, while surfacing additional candidates for analyst validation. A formative study with contract-domain lawyers indicated that in-context evidence comparison supported contradiction validation, and we distill design implications for evidence-grounded, LLM-assisted document review.
Read the full abstract

All authors Luis Sante, Paula Lima, Mariana Rocha, Jorge Poco