Peer Revenue Fully funded

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Human-Computer Interaction

cs.HC Watch Human-Computer Interaction

Sunday 9 August

Human-Computer Interaction

SocialFiVis: A Visual Analytics Sandbox for LLM-Grounded Multi-Agent Simulation in Social Finance

Yi-Fan Cao, Qing Shi, Liangwei Wang et al. (+5 more)

The emergence of social finance (SocialFi) transforms online communities into complex socio-economic systems. Within these spaces, collective decisions shape a "digital commons" characterized by social capital (e.g., community trust) and financial health (e.g., market liquidity). Governing such hybrid ecosystems is challenging because real-world interventions are costly and irreversible. While counterfactual simulation is essential for exploring alternative governance strategies, existing approaches fail to capture the non-linear interplay between governance rules, individual behaviors, and emergent economic outcomes. To systematically unpack this complexity, we operationalize the Institutional Analysis and Development (IAD) framework as our theoretical foundation, synthesizing prior literature with insights from formative expert interviews. Built on this framework, we present SocialFiVis, an IAD-embedded visual analytics sandbox. It introduces a robust model to quantify the dual-track digital commons, coupled with a two-phase simulation engine. This engine combines LLM-derived personas with a mechanism-guided Perception-Reasoning-Action (PRA) runtime to simulate heterogeneous, context-aware agents empirically grounded in the retained messaging cohort. A hierarchical multi-view interface with interpretable reasoning pathways enables community operators to explore counterfactual policies and trace system-level outcomes back to individual behavioral rationales. We evaluate SocialFiVis through two case studies, a user study, and follow-up interviews. Results demonstrate that SocialFiVis supports fine-grained behavioral attribution and helps explain emergent phenomena such as the structural decoupling of social capital and the resilience of messaging members under localized governance shocks.
Read the full abstract

All authors Yi-Fan Cao, Qing Shi, Liangwei Wang, Leo Yu-Ho Lo, Lin Chen, Yuzi Han, Yang Wang, Kani Chen

Human-Computer Interaction

Private Etymology: Designing Relational Reuse of Shared Symbols in Long-Term Human-AI Interaction

Miki Ueno

Previous studies have shown that people can develop shared symbols, partner-specific expressions, personal idioms, inside jokes, and other parts of a relational microculture. Recent work has also examined how humans and conversational AI negotiate and revise symbolic meanings. However, long-term human-AI systems still lack a clear design model for recording how a dyad-specific expression gains meaning, checking whether both sides still accept that meaning, and safely reusing the expression in later sessions. This concept-and-prototype paper introduces Private Etymology, a machine-representable relational provenance that records how a dyad-specific symbolic expression is proposed, interpreted, negotiated, repaired, reused, revised, stabilized, contested, forgotten, or retired over time. I also propose relational reuse: reactivating a dyad-specific expression in a later session without fully explaining its meaning again. The contribution is not the invention of shared symbols or relational microcultures. Instead, this paper integrates prior ideas into persistent, revisable, and evidence-grounded symbolic units for human-AI relationships. I present a lifecycle model, an illustrative machine-readable schema, a working Apple Watch prototype, and a longitudinal research agenda. In the prototype, a language model classifies discrete conversational evidence, while deterministic local code decides whether a Shared Symbol can be updated. This prevents a free-form model confidence score or an AI proposal by itself from directly updating the persisted symbol. Private Etymology is proposed as infrastructure for conversational agents to participate in changing relational microcultures without inventing their origins or treating relational meaning as a fixed memory value.
Read the full abstract

Human-Computer Interaction

Human-Guided Causal Knowledge Injection for Virtual Cells

Pengcheng Wang, Changjian Chen, Zhuo Tang et al. (+4 more)

Virtual cells employ machine learning models to simulate and predict cellular behaviors, serving as a critical computational framework for investigating health and disease. Injecting causal graphs into virtual cells can improve the interpretability, but such graphs are usually not available in real-world applications. Recently, many methods have been proposed to construct causal graphs from data, which group genes based on their similarities to form concepts and extract their causal relationships. However, since this automatic process is unsupervised, the causal graphs usually contain errors. In this paper, we propose a human-guided causal knowledge injection method for virtual cells. We developed a gene-similarity-aware causal graph visualization supported by a hybrid optimization algorithm to help explore both the causal relationships between concepts and the similarities between genes. Based on the exploration, we further developed a counterfactual analysis strategy supported by a counterfactual visualization and a causal path visualization to help validate and refine causal graphs. The effectiveness of our method is demonstrated through two real-world case studies, the extraction of scientifically meaningful causal insights, and positive feedback from domain experts.
Read the full abstract

All authors Pengcheng Wang, Changjian Chen, Zhuo Tang, You Wu, Long Wang, Feng Yu, Kenli Li

Human-Computer Interaction

FZ-VIS: A Visual Analytics Framework for Quantities-of-Interest-Aware Scientific Lossy Compression

Guoxi Liu, Yuxiao Li, Congrong Ren et al. (+6 more)

Modern scientific simulations generate massive volumes of data, making lossy compression essential for efficient storage and transmission. However, preserving critical quantities of interest (QoIs) under lossy compression is inherently data- and task-dependent, requiring domain scientists to navigate complex trade-offs between compression ratio and data fidelity. Exploring these trade-offs often involves large design and evaluation spaces, motivating human-in-the-loop approaches that combine interactive exploration with quantitative analysis. To address this challenge, we present FZ-VIS, an interactive framework for human-in-the-loop feature-oriented lossy compression design and visual analytics. FZ-VIS provides a web-based interface for rapidly generating and comparing compression configurations, along with integrated visualization tools for assessing reconstruction fidelity and QoI preservation through both visual inspection and quantitative metrics. We demonstrate the utility of FZ-VIS through case studies involving three representative user groups: novice users selecting compression methods, compressor developers examining internal pipeline behavior, and domain scientists investigating feature preservation. The case studies show how FZ-VIS helps users efficiently navigate complex design spaces and make informed decisions that balance compression performance with application-specific QoI requirements.
Read the full abstract

All authors Guoxi Liu, Yuxiao Li, Congrong Ren, Robert Underwood, Xin Liang, Bei Wang, Sheng Di, Franck Cappello, Hanqi Guo

Saturday 8 August

Human-Computer Interaction

Why and How People Check Generative AI Output for Mistakes

Patrick Gage Kelley, Derrick Feldmann, Reena Jana et al. (+2 more)

Generative AI output can contain errors, such as hallucinations, non-responsive results, or otherwise inaccurate or potentially harmful content. To explore the public's emerging understanding, attitudes, and behavior regarding such mistakes, we ran an online survey in the United States with 1,503 respondents, with a representative sample of the population. We report high public awareness of generative AI mistakes. Further, many respondents report checking generative AI output, for example, by comparing results with other online resources. We conclude with guidance for explanations and in-product disclosures about generative AI mistakes.
Read the full abstract

All authors Patrick Gage Kelley, Derrick Feldmann, Reena Jana, Colleen Thompson-Kuhn, Allison Woodruff

Human-Computer Interaction

Baseline Exposure to Common Data Visualization Types Among the U.S. Adult Population

Kiegan Rice, Nola du Toit, Quentin Brummet et al. (+1 more)

Data visualizations are a primary means of communicating statistical information to the public. Their expanding use across news media, public health, education, and government reporting places greater importance on audiences' ability to recognize and interpret them. While a significant body of prior research has established frameworks for the measurement of graph and visualization literacy, far less is known about everyday exposure to different kinds of charts and graphs among the general adult population. This gap limits our ability to meaningfully interpret differences in graph literacy, focus efforts on improving that literacy, and design visualizations that align with audience ability. Establishing population level estimates of exposure to data visualizations is therefore essential for improving visual communication and reducing misinterpretation of quantitative information. To address this gap, we surveyed a nationally representative sample of 1,168 U.S. adults via NORC's AmeriSpeak panel about their exposure to sixteen different common data visualization types. The resulting survey responses demonstrate pronounced differences in baseline exposure to data visualizations across chart types and demographic groups. While widely used formats such as bar, pie, line, and grouped bar charts are highly recognizable to the vast majority of U.S. adults, many other practitioner-favored chart types remain unfamiliar to substantial portions of the adult population. Exposure also differs significantly by age and education, with higher educational attainment linked to greater exposure and older adults reporting lower exposure overall. Our findings provide essential population level context on visualization literacy and underscore the importance of aligning data visualization design with audience exposure and experience.
Read the full abstract

All authors Kiegan Rice, Nola du Toit, Quentin Brummet, Heike Hofmann

Human-Computer Interaction

Designing Task-Induced Arousal: A Multimodal Stress Induction Method for Interactive Experiments

Morten Roed Frederiksen

HCI and HRI studies often require short, repeatable arousal manipulations that can run while participants continue interacting with a device or robot. These experiments are often challenged by the need to induce arousal in settings that still resemble real interaction. Participants must continue using a device, touching a robot, or producing sensor data while the manipulation unfolds. We aimed to develop a compact and repeatable way to induce controlled task-related arousal during interactive experiments by combining a lightweight browser-based pacing task, escalating timing demands and urgency cues, and a concurrent physical hotwire-style challenge. In an A-B-A within-participant study, the induction condition significantly increased mental demand, temporal demand, effort, frustration, and SAM arousal (all p < .001), while perceived performance decreased (p < .001). GSR peak rate increased relative to both calm conditions (p = .030) and escalated over time (p < .001). Grip variability also increased (p = .036), as did release speed (both p < .001), while valence remained above the scale midpoint. These results provide initial evidence that the combined procedure induces controlled, relatively high-valence task-related arousal and may serve as a reusable experimental tool for future human-computer and human-robot interaction studies.
Read the full abstract

Human-Computer Interaction · Cross-listed from eess.AS

CtrlSpeech: Coarse-to-Fine Control for Expressive Speech Synthesis

Zhisheng Zheng, Xiaohang Sun, Zhu Liu et al. (+5 more)

Recent Text-To-Speech (TTS) systems have achieved strong naturalness and zero-shot voice cloning performance, but fine-grained control of expressive speech at the word or phoneme level remains challenging. We propose CtrlSpeech, a controllable, expressive TTS framework with coarse-to-fine control. Built on the DiTAR architecture, CtrlSpeech combines global speaker conditioning with phone-aligned pitch, loudness, and duration signals, enabling localized prosodic control while preserving the target speaker's timbre. This design allows users to adjust expressive attributes at a fine temporal granularity, making speech refinement more flexible and controllable. Experimental results show that CtrlSpeech achieves competitive zero-shot TTS performance and improves controllability over expressive attributes, demonstrating its effectiveness for flexible and practical expressive speech synthesis.
Read the full abstract

All authors Zhisheng Zheng, Xiaohang Sun, Zhu Liu, Caren Chen, Rohith Kumar, Manoj Aggarwal, Gerard Medioni, David Harwath

Human-Computer Interaction

Dramarrator: Object-Based Audio Editing for Audio Drama Production from Books

Karim Benharrak, Oriol Nieto, Bryan Wang et al. (+2 more)

Audio dramas weave dialogue, sound effects, and music into immersive stories. Creators often adapt books into audio dramas, but this process remains labor-intensive, requiring them to interpret source material, author scripts, generate audio assets, and assemble them on a timeline. Because story elements like characters and scenes manifest across many interdependent assets, a single change can ripple into manual updates across the entire project. We present Dramarrator, an audio drama authoring tool built around object-based audio editing, where these story elements are represented as editable objects. Dramarrator extracts these objects from a book, generates linked audio assets (speech, sound effects, and music), and composes a multi-track audio drama. Edits to any object (e.g., a character's voice) automatically propagate to all dependent assets. In a user study with professionals (N=8), Dramarrator significantly lowered task load when creating audio dramas. A listener study (N=300) shows that creator-refined output from Dramarrator approaches the quality of productions made with existing professional tools, and an exploratory study (N=3) suggests object-based editing lowers entry barriers and generalizes beyond audio dramas.
Read the full abstract

All authors Karim Benharrak, Oriol Nieto, Bryan Wang, Zeyu Jin, Amy Pavel

Human-Computer Interaction

A Decision Framework for Selecting Technology Acceptance and Use Models: TAM, TAM2, TAM3, UTAUT, and UTAUT2

Nathalino Pachêco Britto

Introduction: Models such as TAM, TAM2, TAM3, UTAUT, and UTAUT2 underpin a substantial share of research on technology acceptance and use in HCI and related fields. Although they differ in terms of constructs and conditions of application, their selection is often not conceptually justified, frequently driven by pragmatic considerations, with implications for theoretical consistency and cross-study comparability. Objective: To propose a decision framework that guides the selection among the five models of the TAM/UTAUT lineage based on conceptual criteria derived from their structural differences. Methods: A conceptual, artifact-oriented approach was adopted, comprising comparative analysis of the models and critical review studies, derivation of conceptual dimensions, and formulation of operational decision criteria. Applicability was demonstrated through two contrasting research scenarios. Results: The framework articulates five analytical dimensions, 16 guiding questions, and descriptive suitability profiles for the models. The demonstration illustrated, in the two scenarios examined, its ability to identify misalignments between the declared model and the actual operationalization, as well as to confirm theoretical choices consistent with the conditions of the study.
Read the full abstract

Human-Computer Interaction

Situatedness in Visualization Design: Making Unresolved Work Actionable

Paul C. Parsons, Prakash Shukla, Phuong Bui et al. (+2 more)

Visualization design often proceeds under unresolved conditions—goals shift, data remain provisional, stakeholder needs evolve, and several plausible directions may remain available at once. Existing visualization frameworks help organize design work and articulate major decisions, yet offer limited explanation of how practitioners proceed before a path forward has become clear. Drawing on an episode-level analysis of a previously collected three-phase qualitative corpus involving eleven expert visualization practitioners, we examine situations in which the problem, representational target, or viable direction remained unsettled. We find that practitioners make such situations actionable through provisional local moves. These moves reveal patterns, distinctions, and interpretive possibilities; clarify what is tractable, viable, or worth pursuing; and sometimes reorient the work itself. The analysis shows that situated action, professional judgment, and explicit design reasoning are intertwined in expert practice. It also identifies a practical limit on how fully design activity can be specified in advance. When the meaning of the next move depends on what a situation reveals in response to action, prescriptive decision structures cannot fully determine the course of design. The paper contributes an empirical account of situatedness in visualization practice and explains how local action makes unresolved work interpretable enough for consequential design decisions.
Read the full abstract

All authors Paul C. Parsons, Prakash Shukla, Phuong Bui, Srishti Agrawal, Ali Baigelenov

Human-Computer Interaction

Design Knowledge in Data Visualization: Mapping the Epistemic Landscape

Paul C. Parsons, Colin M. Gray and Ali Baigelenov

Data visualization research has developed many influential forms of design knowledge, including perceptual principles, design guidelines, process models, and formalized representations of design constraints. These contributions have been effective at articulating explicit, portable, and codified forms of knowledge. Yet the broader landscape on which visualization design depends remains less clearly articulated, especially with respect to intermediate-level knowledge, precedents, tacit repertoires, and situated forms of knowing. In this paper, we draw on design theory to map this broader landscape of design knowledge in data visualization. Through this lens, we show how visualization research has built substantial strengths in some regions while leaving others comparatively underarticulated. We further argue that visualization design depends not only on knowledge artifacts such as theories, guidelines, and patterns, but also on knowledge-in-use—the situated interpretation, adaptation, and coordination of multiple forms of knowing in concrete design situations. This broader account has implications for how the field conceptualizes design expertise, evaluates and develops scholarly contributions, and approaches AI-assisted design. Rather than treating visualization design as either fully formalizable or wholly resistant to computational support, we argue for a differentiated view in which computational systems can support some forms of design knowing, while others remain inseparable from human judgment, contextual interpretation, and the ongoing reorganization of design work in practice.
Read the full abstract

Human-Computer Interaction

Underwater MMG-Based Muscle State Monitoring with Integrated Emergency Buoyancy Assistance

Xiao Jin, Yixian Fan, Zefeng Yuan et al. (+1 more)

This paper presents an underwater MMG-driven wearable emergency assistance system for lower-leg muscle-state monitoring and automatic buoyancy deployment. A compact microphone-based MMG sensor was waterproofed using a flexible 5 mil PE membrane, preserving identifiable muscle-vibration responses under immersion, depth variation, and stirring disturbances. Two lower-leg sensors captured stroke-dependent MMG patterns across four swimming styles, and a MiniRocket classifier achieved 91.91% window-level and 97.56% file-level accuracy. For cramp-related monitoring, a pattern-based risk score was used to identify representative pre-cramp abnormal muscle-state transitions during rhythmic motion. A controlled underwater test demonstrated the closed sensing–decision–actuation chain, triggering CO_2 release, airbag inflation, and flotation in less than 5~s. These results support underwater MMG as a sensing basis for wearable robotic emergency assistance in aquatic environments.
Read the full abstract

All authors Xiao Jin, Yixian Fan, Zefeng Yuan, Zhenhua Yu

Human-Computer Interaction · Cross-listed from cs.CR

Privacy-Preserving Data Drift Detection and Recovery for Large-Scale LLM Applications via Proxy Representations

Michael Levit, Josh Ledgard, Haoyu Dong et al. (+5 more)

LLM applications deployed at scale face a fundamental challenge: privacy constraints prevent direct inspection of user interactions, making it difficult to obtain any representative evaluation dataset or to track the ongoing evolution of production traffic. We present ProxyDrift, a framework that (i) identifies and measures drift between production traffic and offline evaluation sets, and (ii) constructs and refreshes those evaluation sets accordingly; all without access to raw user data. Our approach operates entirely on non-PII proxy representations: structured, multi-dimensional descriptors derived from LLM-based classification of user interactions. We introduce (1) a chance-calibrated, redundancy-aware (RA) alignment score that aggregates per-dimension drift measurements via mutual information; (2) a conditional sampler that generates synthetic proxies respecting inter-dimensional dependencies; (3) a roundtrip consistency analysis that exposes generator/classifier disagreements and guides proxy taxonomy refinement; and (4) a feedback-linkage analysis that ties per-dimension and per-value proxy distributions to user satisfaction, surfacing actionable failure and success modes. Serving hundreds of millions of users, ProxyDrift enables continuous drift monitoring and targeted synthetic data generation without exposing sensitive user data. Experiments confirm strong roundtrip consistency, discriminator-level indistinguishability of synthetic queries from human queries, and tight end-to-end alignment (RA~0.9) with production.
Read the full abstract

All authors Michael Levit, Josh Ledgard, Haoyu Dong, Vishwas Suryanarayanan, Eyal Kolman, Sharon Tan, Qiang Gan, Vishal Chowdhary

Human-Computer Interaction · Cross-listed from cs.CL

Focus particles and scalar inferences across humans and language models

Catherine M. Brousse and Nelu D. Radpour

Focus particles such as "even" and "only" are central to formal semantic theories that posit structured representations over sets of alternatives. "Even" highlights unexpected or extreme alternatives, while "only" enforces exclusivity. If such scalar representations are robust and generalizable, they should give rise to consistent judgments across contexts and systems. In this work, we test whether humans and large language models (LLMs) construct stable scalar representations from sentences containing these particles. Using a dataset of approximately 100 items, participants and models were asked to make scalar judgments. Preliminary results suggest that similar outputs across humans and LLMs may arise from different underlying mechanisms.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.RO

Spatiotemporal Context-dependent Personalized Movement Compensation in Delayed Telemanipulation

Sai Jiang and Zonghe Chua

Communication delay remains a central challenge in telerobotics, where it disrupts visuomotor coordination and reduces task precision. Motion scaling is an effective countermeasure to delay-induced overshoot, yet typical deployments rely on uniform gains that neglect individual and contextual variability. We propose a human-centered method that fits personalized delay-, direction-, and distance-specific scaling parameters for each participant. We conducted experiments with twenty participants who performed delayed reaching tasks in a virtual simulator. Scaling gains were computed to minimize mean overshoot in simulation in each combination of experimental conditions. Evaluation was done in simulation and on a telesurgical robot to evaluate assistance benefits. Performance was assessed across multiple delays, distances, and movement directions using overshoot, endpoint error, trajectory smoothness, economy of motion, and a composite error-time metric. Motion scaling consistently improved performance relative to unassisted trials, yielding up to 20-25% performance gains in key metrics. Effects were most pronounced at longer delays. Personalization demonstrated additional accuracy benefits for inward reaching at a short distance under moderate delay. The results highlight the potential of personalized scaling as a foundation for more adaptive frameworks that integrate contextual information to improve the safety and precision of teleoperated procedures.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.DC

Evaluating and Improving Weak Scalability Analysis of Visualization Algorithms

Marvin Petersen, Jonas Lukasczyk and Christoph Garth

Research on visualizing large-scale datasets traditionally relies on empirical evaluation of scalability, determining the effectiveness of specific computation methods, algorithmic strategies, or implementations. Weak scalability, which assesses the algorithm's performance as problem size and computing resources increase, is a valuable indicator for a method's applicability at scale. However, sufficiently large data sets with increasing size are needed for weak scalability studies. To this end, it is customary to use simple scaling techniques to increase problem size by generating larger input data sets from a base data set. Nevertheless, many visualization algorithms' workload depends on factors beyond input size, such as input data complexity or output size, leading to inaccuracies in the attributed weak scalability. In this work, we highlight different common data scaling methods on multiple algorithms and data sets, recognizing that the suitability of scaling approaches varies across algorithms and data sets. We present a method that effectively mitigates the observed inconsistencies in workload increases for the different scaling methods in a shared-memory setting. With this work, we aim to further the discussion on how to evaluate and report scalable visualization research.
Read the full abstract

Human-Computer Interaction

Understanding Security and Privacy Perceptions of Content Creators Regarding AI Labels of AI-Generated Content

Shuning Zhang, Hui Wang, Rongjun Ma et al. (+4 more)

AI labels, typically implemented via underlying tracing mechanisms such as watermarks and metadata, are crucial for protecting Artificial Intelligence-Generated Content (AIGC) against security threats like disinformation and evasion. However, the perceived devaluation of AI-assisted work discourages creators from disclosing AI use, incentivizing efforts to bypass labeling and compromising downstream traceability. Yet, how AIGC creators perceive the security and privacy (S&P) implications of these labels, and how their behaviors impact technical resilience remain underexplored. To this end, we conducted semi-structured interviews with 21 AIGC creators and measured images across 6 image generation platforms against 16 self-reported manipulation settings. Our findings reveal that creators conflate binary AI labels with granular traceability, and express strong fears of de-anonymization via platform identifiers. Driven by fears of algorithmic traffic suppression and reputational risks, they defensively removed digital traces. Through empirical tests, we show that targeted modifications like coarse quantization significantly degrade detection. AI detection capabilities are also inconsistent across platforms, and suffer from false positives even for human-authored images. Based on these insights, we advocate for workflow-resilient implicit AI labels that align technical guarantees with creators' incentives.
Read the full abstract

All authors Shuning Zhang, Hui Wang, Rongjun Ma, Xin Yi, Kanye Ye Wang, Robert Xiao, Hewu Li

Human-Computer Interaction

The Missing Link of XR: Empathy-Driven Reality for XR and Beyond

Yun Suen Pai, Tamil Selvan Gunasekaran, Jiashuo Cao et al. (+23 more)

Extended reality (XR) for socialising is becoming increasingly popular. However, unlike conventional social platforms, XR prioritises embodiment and immersion, factors that strongly impact one's physical and mental states. We envision a future for XR where all users, regardless of abilities and backgrounds, can understand one another, participate, and find safe socialisation spaces. An Empathy-Driven Reality (EDR) is a space where understanding each other's emotional, physical, and cognitive states takes centre stage. It has the potential to enhance empathy beyond how we normally perceive it. To explore this concept, we conducted a hybrid-style workshop over two months with 27 industry and academic researchers in XR, emotion, physiology, assistive technology, and social science. This paper reports on the findings and aims to establish a structure and reference for the 1) design guidelines, 2) research challenges, and 3) potential applications for the future of XR as an EDR. We frame EDR as a conceptual design framework rather than a validated system: a structured design space that links requirements to design mechanisms, together with a research agenda for empathy-centred XR.
Read the full abstract

All authors Yun Suen Pai, Tamil Selvan Gunasekaran, Jiashuo Cao, Kunal Gupta, Andreia Valente, Ken Jen Lee, Giulia Barbareschi, Kai Lukoff, Lingyuan Li, Ruofei Du, Fannie Liu, Jennifer Day, Tanner Person, Johann Wentzel, Misha Sra, Yuhang Zhao, Theophilus Teo, Elisabeth Andre, Mark Armstrong, Mark Billinghurst, Danielle Lottridge, Kinga Skiers, Anish Kundu, Erica Principe Cruz, Takuji Narumi, Kouta Minamizawa

Human-Computer Interaction · Cross-listed from cs.CY

Beyond "I Can't Help With That": How Child Safety Experts Evaluate AI Chatbot Safety

Hannah Cha, Neha Shukla, Solon Barocas et al. (+3 more)

Youth increasingly turn to AI chatbots for social and emotional support, raising concerns about how these systems respond, especially in high-stakes situations. However, existing child safety evaluations of AI lack grounding in real-world harms that youth experience, rely on unvalidated assumptions about what counts as an appropriate output (e.g., refusal), and typically focus on detecting adversarial prompts or surface-level harms in outputs only. Thus, these evaluations can fail to detect responses that pose harm to youth in practice. To better understand the limitations of current evaluation practices, we conducted interviews with 19 practitioners working directly with youth in vulnerable situations, including social workers, therapists, and psychologists, asking them to reflect on chatbots' responses to risky situations commonly faced by youth, as established in prior empirical work. Practitioners identified chatbot behaviors likely to cause harm as well as those that could meaningfully support youth in difficult moments, discussed the role that chatbots should (and should not) play in these interactions, and offered concrete recommendations for improving chatbot responses. Based on these findings, we provide recommendations for AI child safety evaluation and infrastructure, and highlight the need for incorporating practitioners' perspectives into safety work.
Read the full abstract

All authors Hannah Cha, Neha Shukla, Solon Barocas, Alexandra Chouldechova, Eugenia Kim, Jennifer Wortman Vaughan

Human-Computer Interaction · Cross-listed from cs.CV

How Much Does It Cost to Answer My Question? Benchmarking Cloud VLM-based VQA Systems

Henri Vanhuynegem, Weitao Xu, Yiran Shen et al. (+1 more)

Vision-language models (VLMs) are becoming a practical backend for mobile visual question answering (VQA) systems, enabling smartphones and smart glasses to answer users' questions about the physical world. Since modern VLMs remain difficult to run on mobile and edge devices, VQA systems increasingly offload inference to cloud-based VLMs. This gives mobile devices access to stronger computation, but it also makes visual input preparation a key system variable: how the image is prepared before offloading affects not only answer quality but also payload size, token cost, and system latency. Proprietary APIs expose little control over model internals or serving behavior, leaving client-side preprocessing as the main practical optimization space for downstream developers. Many such techniques have been proposed for visual offloading, yet their cost-quality impact on commercial cloud VLMs has never been studied. To fill this gap, we present VQABench, the first systematic benchmark that treats client-side input preprocessing as a controlled variable for cloud-VLM-based VQA. We evaluate 12 preprocessing techniques across three VQA datasets and four commercial VLMs from three providers, totaling 95,168 API calls. Our results show that preprocessing is not universally beneficial: its effectiveness depends on the target model, API paradigm, provider token-accounting rule, and task formulation. A poorly selected preprocessing strategy can increase deployment cost or latency while degrading answer accuracy. Overall, our benchmark clarifies when preprocessing helps, when it fails, and why, providing insights to guide future research and real-world deployment of VQA systems.
Read the full abstract

All authors Henri Vanhuynegem, Weitao Xu, Yiran Shen, Guohao Lan

Human-Computer Interaction

Marks, Channels, and Dead Ends: Stop Running Graphical Perception Studies and Start Modeling Visualizations as Images

Khairi Reda, Shambhawi Sharma, Luc Renambot et al. (+2 more)

Graphical perception studies are the visualization community's preferred tool for evaluating visualizations. By measuring how accurately people interpret arrangements of visual marks and channels, they aim to establish best practices for visual encoding. We argue that this model is fundamentally flawed, and no amount of additional empirical studies will fix it. Visualization theory frames effectiveness at the level of the encoder: which data-to-visual mappings work best in a given context. Human perception, however, operates as a fundamentally different decoder at the level of retinal images. This encoder-decoder asymmetry means that experimental results and guidelines can be poor predictors of perceptual performance. Moreover, the image reaching the visual system emerges from interactions among encoding rules, input data, and micro-design parameters–factors largely invisible to encoding theory. Consequently, small changes in data distributions or design variations can substantially alter perception even when the nominal encoding remains unchanged. We argue that visualizations should instead be studied as images and evaluated using computational models of human vision that take pixels as input. Such models capture the perceptual representations the visual system actually constructs, shifting evaluation toward the decoder rather than abstract encoding specifications. This approach is scalable, human-grounded, and sensitive to emergent image properties that both encoding theory and graphical perception studies miss. We first describe weaknesses of the current paradigm and propose a theory of visualization perception grounded in summary-statistical accounts of vision. We then show how image-based vision models can predict visualization discriminability in scatterplots while reproducing established results. We close by outlining a research agenda for vision-based visualization evaluation.
Read the full abstract

All authors Khairi Reda, Shambhawi Sharma, Luc Renambot, Fabio Miranda, Saeed Boorboor

Friday 7 August

Human-Computer Interaction

Beyond the Good, the Bad, and the Ugly: Colormap Assessment through Data-Aware Perceptual Metric

Xi Duan, Yiwei Lin, Shiqing Xin et al. (+4 more)

Continuous colormaps are widely used to visualize scalar fields, and their quality is typically evaluated using measures such as discriminative power and uniformity. Existing measures primarily characterize the intrinsic perceptual properties of the colormap itself, largely independent of the underlying data distribution. In practice, however, user perception arises not from the colormap alone but from the visualization generated by mapping data values through the colormap. The perceptual differences that users actually experience depend jointly on the colormap and the underlying data. We propose a data-aware formulation that complements existing data-independent colormap assessment approaches. Rather than analyzing the colormap in isolation, we model the color-encoded visualization as a composite mapping from the spatial domain of the data to perceptual color space. This approach yields data-aware counterparts of established measures, including discriminative power, uniformity, and smoothness, as well as additional measures such as perceptual anisotropy and degeneracy. We validate the proposed measures against both the existing data-independent framework and empirical results from perceptual studies, extend the formulation to 2D colormaps, and demonstrate its integration into colormap optimization. An interactive system is provided for exploring colormap assessment under varying data distributions. Results demonstrate that our formulation offers a principled foundation for data-aware colormap assessment and design.
Read the full abstract

All authors Xi Duan, Yiwei Lin, Shiqing Xin, Aoying Wang, Yucheng Wang, Changhe Tu, Qiong Zeng

Human-Computer Interaction

RealGUINoise: An Interactive Cross-Platform Benchmark for GUI Agent Robustness under Real-World Interface Noise

Yongjiang Wu, Junyuan Zhang, Ada Chen et al. (+2 more)

Graphical User Interface (GUI) agents and Computer-Using Agents (CUAs) are rapidly becoming practical tools. However, real-world deployment increasingly exposes performance failures and safety risks, while a major yet underexplored source of these problems lies in the complex and noisy conditions of everyday interfaces. Existing benchmarks largely assume clean environments or focus narrowly on security-specific settings, and lack a unified framework for consistent, automated end-to-end evaluation across diverse agents and platforms. Hence, we introduce RealGUINoise, an interactive cross-platform, extensible benchmark for systematically evaluating GUI Agents under common realistic interface noise in fully interactive environments. Specifically, RealGUINoise comprises 42 noise types spanning web, desktop, and mobile tasks and integrates 7 representative agent frameworks. It evaluates these agents on real-world daily tasks through real-time interaction, comparing their performance against task-specific golden rubrics and clean-environment trajectories in terms of reliability, safety, and trajectory-level behavior. Our experiments show that these noises not only degrade task performance but also substantially redirect agents' action trajectories and increase their propensity for unsafe behavior. These findings expose a critical gap between capability in clean environments and dependable operation in real-world settings, establishing RealGUINoise as a testbed for developing more robust and trustworthy GUI agents.
Read the full abstract

All authors Yongjiang Wu, Junyuan Zhang, Ada Chen, Kuiyi Gao, Wenxuan Wang

Human-Computer Interaction · Cross-listed from cs.CY

Identifying Harm in Personalized, Generative AI Systems Requires User-Centered Auditing at the Interaction Level

Hannah Cha

Personalized, generative AI systems increasingly adapt their behavior to individual users over time, fundamentally changing model behavior. While existing auditing approaches have been effective at surfacing harms in non-personalized contexts, they often rely on static, simulated evaluations and definitions of harm that aggregate across broad, group categories. In this position paper, we argue that such approaches can fail to capture emergent harms in personalized generative AI systems, where harms surface through interpretations of ongoing interaction and evolve with user history. We identify three presuppositions underlying many harm auditing paradigms: that harms can be (1) specified outside real-world interaction, (2) defined non-pluralistically within groups, and (3) treated as static. One might argue that personalized systems could simply learn definitions of what constitutes harm to individual users through repeated interactions. However, we argue that attempts to surface user harms through deeper personalization risk imposing asymmetric burdens of labor and privacy on marginalized users. Consequently, we propose reframing understandings of harm as adaptive, user- and community-centered processes, and outline design directions that shift auditing from retrospective evaluation toward infrastructures that support ongoing articulation of harm in interaction. Our work highlights the need for auditing and design practices that better reflect the pluralistic and evolving nature of harm understanding in personalized generative AI systems.
Read the full abstract

Human-Computer Interaction · Cross-listed from cs.CY

AI as a Democratizing Force in Indie Game Development

Brian Madanamootoo and Jatin Alla

The video game industry of 2024-2026 shows the deepest AAA-level contraction in its modern history alongside the largest-ever expansion of independent output. We examine AI's role in that divergence across four research questions, using public marketplace data, a title-level Steam catalog dataset cross-referenced with Steam's generative-AI disclosure records, and a fourteen-month log from an agentic AI game-production platform. RQ1 (barriers to entry): production planning, historically a salaried producer role at roughly $59 per hour, is generated in a mean of 5.1 minutes for $0.27-0.58 per plan. We operationalize "democratization" across seven dimensions and claim it for one, coordination cost, repriced by roughly four orders of magnitude; the regional dimension is argued from cost arithmetic, not measured. RQ2 (market acceptance): indie volume and unit sales expanded, AI-disclosed releases rose eightfold in eighteen months, professional sentiment collapsed across three surveys, and disclosed releases received catalog-typical reception (median 85.9 percent positive) in a verified subsample. RQ3 (a new wave): the moment resembles a third democratization wave after distribution (2008-2012) and construction (2013-2020); four pre-registered convergence tests align, and as production cost nears zero while mature-market demand contracts, the market shows preconditions of structural oversupply, forcing a new distribution paradigm. RQ4 (quality): the platform log measures plan generation, not shipped-game quality, so we compare quality signals for AI-disclosed versus non-disclosed titles; disclosure records AI use in production, not authorship. Releases doubled from 9,654 (2020) to over 20,000 (2025) while only about 300 titles grossed above $1 million, and participation in major markets fell below pre-pandemic levels; we close with implications for developers, educators, and platform design.
Read the full abstract

Human-Computer Interaction

Exploring a Real-time Feedback Display of Non-Verbal Cues in Online Work Meetings to Support Self-Presentation

Kevin Chow, Roy Rutishauser, André Meyer et al. (+2 more)

Expressing oneself appropriately in online meetings through non-verbal cues can be challenging for knowledge workers. Automatic non-verbal cue detection technologies have the potential to support workers' self-presentation efforts through real-time feedback, but little is known about workers' reactions to and the implications of doing so. We designed and implemented Novecs as a technology probe of a real-time feedback display that automatically detects and signals users' own non-verbal cues – smiling, nodding, gaze, and posture. Novecs was deployed in an exploratory field study (n=18) to support knowledge workers' self-presentation in their everyday meetings. Post-study interviews reveal how Novecs' real-time feedback helped increase in-the-moment self-awareness, and how neutrally-framed feedback may help navigate tensions between authentic and in-authentic self-presentation. Participants also emphasized the need for natural timing when adjusting non-verbal cues in-meeting. We discuss design opportunities and challenges of real-time, non-verbal cue feedback systems, such as personalizing feedback based on different meeting types.
Read the full abstract

All authors Kevin Chow, Roy Rutishauser, André Meyer, Joanna McGrenere, Thomas Fritz

Human-Computer Interaction · Cross-listed from cs.AI

IntelliAudit: Using Large Language Models to Evaluate Audit Controls

Allison Wilson, Sina Moradi Sabet, Diar Shakimov et al. (+4 more)

IT audits require auditors to judge whether heterogeneous organizational evidence satisfies semantic security and compliance controls. This judgment is difficult to automate because relevant evidence is distributed across policies, records, spreadsheets, and operational artifacts, and because audit conclusions depend on evidentiary sufficiency rather than keyword matching. We present IntelliAudit, a retrieval-grounded multi-agent system for IT audit evidence evaluation. Given a control and an evidence corpus, IntelliAudit retrieves relevant artifacts, generates an evidence-grounded assessment, challenges adverse findings, adjudicates disagreements, and produces an auditor-facing recommendation with cited evidence, rationale, missing-evidence analysis, and remediation guidance. We instantiate IntelliAudit on ISO/IEC 27001 and evaluate it across multiple simulated organizations using expert auditor review and audit-readiness user feedback. The evaluation shows that IntelliAudit can support control interpretation, evidence-grounded reasoning, and audit-preparation workflows, while also revealing the importance of human oversight for calibrating sufficiency judgments and correcting overly permissive recommendations. These results suggest that retrieval-grounded multi-agent systems can assist audit evidence review, but should remain decision-support tools rather than autonomous certification systems.
Read the full abstract

All authors Allison Wilson, Sina Moradi Sabet, Diar Shakimov, Panteha Shahrivar, Mohammad Reza Bagheri, Dean Konenkamp, Mohammad A. Tayebi

Human-Computer Interaction · Cross-listed from cs.AI

PsychoAgent: An Affect-Sensitive Cognitive Architecture for Conflict-Aware Memory in LLM Agents

Mohammad Amanlou, Parham Abed Azad, Farbod Davoodi et al. (+3 more)

Human-like cognition does not select past experience by topical similarity alone: affective significance and unresolved conflict also shape what becomes accessible. We present PsychoAgent, a cognitive architecture for LLM agents that separates factual and affective memory and integrates both through a conflict-aware executive controller. Affective memories are first filtered by semantic relevance and then re-ranked by salience, preserving topical fit while allowing emotionally important traces to enter the prompt. Across three controlled conflict scenarios, the full architecture retrieved more conflict-critical memories than semantic-affective and single-memory RAG baselines (0.933 vs. 0.500 and 0.667), with a small semantic-similarity cost. Five blinded raters evaluated 27 outputs. After within-rater standardization, the full architecture had the highest overall mean (+0.22 SD), but corrected pairwise differences were not significant. A three-day illustrative trace further shows persistent affect, offline memory recombination, and selective memory reweighting. The findings support affect-sensitive retrieval as an inspectable mechanism for modeling human-like conflict effects in LLM agents.
Read the full abstract

All authors Mohammad Amanlou, Parham Abed Azad, Farbod Davoodi, Mostafa Masumi, Behnam Bahrak, Abdol-Hossein Vahabie

Human-Computer Interaction

Hands-Off or Hands-On? Variation in Area Chair Practices and Implications for AI Support

Ines Arous, Neha Nayak Kennard, Andrei Mircea et al. (+3 more)

Area chairs (ACs) play a critical role in the peer-review process, managing conflicts and ensuring fair outcomes. Although AI tools have been proposed to support ACs, little is known about the challenges they face and their perceptions of these technologies. In this paper, we conduct interviews including a design probe with 27 ACs in AI to explore their challenges, strategies, and perspectives on potential AI tools. Through thematic analysis, we identify key tensions arising from the growing volume of submissions, uneven reviewer expertise, and the complex task of managing the relationship between reviewers and authors. Most importantly, we find substantial variation in how ACs engage with submissions and influence outcomes: some adopt a largely hands-off approach, while others take a more hands-on role in guiding discussions and decisions. This variation challenges the notion of a single, universal AC practice and highlights the need to account for diverse approaches. When reflecting on the potential use of AI tools, ACs expressed a cautious stance, drawing on their domain knowledge and heightened awareness of AI limitations. From these findings, we derive three design implications: tailoring AI assistance to diverse AC practices, design assistance for discussion moderation, and embedding human-centered AI principles that preserve human agency in decision-making.
Read the full abstract

All authors Ines Arous, Neha Nayak Kennard, Andrei Mircea, Emily Kuang, Jackie Chi Kit Cheung, Andrew McCallum