Human-Computer Interaction · Cross-listed from cs.LG
Pseudo-Label Augmentation for Affect Sensing in Small Collaborative Groups
Meisam Jamshidi Seikavandi, Tanya Ignatenko, Fabricio Batista Narcizo et al. (+3 more)
Physiological affect sensing in naturalistic group interaction is often limited by sparse labels rather than sensor data: wearable devices produce many time windows, while self-reports are collected only a few times per session. Using GroupAffect-4, a four-person collaborative
dataset with wearable physiology, eye tracking, Big Five personality, and post-task VAD labels, we study pseudo-label augmentation for affect sensing under sparse supervision. We compare no augmentation, Gaussian Process pseudo-labelling, personality-aware trust weighting, and joint personality-plus-confidence weighting within a shared target-construction pipeline. Results show that pseudo-label augmentation improves over the labelled-only baseline in the known-team setting. However, the narrow range of Big Five cosine similarities (0.91-0.99) makes fine-grained personality weighting ineffective; personality similarity functions mainly as a same-team filter rather than a calibrated trust signal. With smoothing, augmented SVM variants are effectively tied on Valence and Arousal, while the joint personality-plus-confidence variant gives the highest Dominance score. Cross-subject LOSO transfer remains encouraging, especially for Arousal, whereas strict session-isolated LOGO removes the augmentation benefit. Given only 10 groups, LOGO should be interpreted as a conservative lower bound on unseen-group transfer. Overall, the results suggest that pseudo-label augmentation can make better use of sparsely labelled collaborative affect data, while personality information is most useful as a within-team selection mechanism.
Read the full abstract ▾
All authors Meisam Jamshidi Seikavandi, Tanya Ignatenko, Fabricio Batista Narcizo, Paolo Burelli, Jesper Bünsow Boldt, Andrew Burke Dittberner
Human-Computer Interaction · Cross-listed from cs.MA
"Looking for Something Weird to Happen": How Humans Sustain AI Agent Novelty Amid Semantic Collapse
Shiyang Lai, Arna Woemmel, Hongkai Mao et al. (+3 more)
Semantic collapse, the progressive narrowing of what AI systems generate, has been studied mainly in closed settings, and remedies have targeted models and data. We study it in MOLTBOOK, a social network of interacting AI
agents that human users configure and steer. Across 30,076 active
agents, output grows less diverse within
agents and more similar across them over weeks, yet a minority sustains high novelty. Interviews with users of high- and typical-novelty
agents (N=11) associate sustained novelty with three features: users value novelty of itself, they supply broad and distinctive material and revise it when output narrows, and they approach MOLTBOOK as a new agentic world to explore, not a venue to instrumentally exploit. A survey of users of distinctive
agents (N=53) confirms these patterns. Communities with more novel
agents also show more diverse output from other
agents. We discuss interface and policy interventions that could support improved human input.
Read the full abstract ▾
All authors Shiyang Lai, Arna Woemmel, Hongkai Mao, Junsol Kim, Summer Eunhyung Ann, James Evans
Human-Computer Interaction
A Responsive Present, a Shared Past, a Social Other: Teens' Overreliance on Companion AI Chatbots
Mohammad Namvarpour, Tyler Chang and Afsaneh Razi
AI companions provide socially engaging interaction through availability, personalization,
memory, roleplay, and emotionally responsive
language. For teens, these systems may support sensitive self-disclosure, identity exploration, and relationship rehearsal while shaping intimacy expectations, offline relationships, emotional wellbeing, and self-understanding. We analyzed 17,053 verified quotations from 3,930 teen-relevant Reddit posts using thematic analysis. We identified 53 topics across seven thematic groups. Users described AI companions as sources of comfort, recognition, identity exploration, and relationship rehearsal, but also reported problematic attachment, social substitution, emotional dependence, and disruption to academic and social life. Roleplay,
memory, perceived reciprocity, unwanted romantic or sexual role drift,
privacy concerns, platform changes, and service interruptions shaped users' boundaries and control. Awareness that the AI was artificial did not prevent guilt, obligation, grief, or distress. These findings show that companion-AI safety must address relationships over time through user-controlled
memory,
privacy, relational boundaries, and healthy disengagement.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.SE
Trust by Design: Trust Calibration Through Non-Advisory Socratic Dialogue in Conversational Agents
Roba Hassan, Nahla Aboromi and Naomi Unkelos-Shpigel
As conversational AI systems increasingly operate in sensitive domains, the central challenge shifts from usability to trust calibration, ensuring that users rely on systems neither too much nor too little. Systems that provide advice or interpretations risk encouraging inappropriate reliance, particularly when users perceive AI outputs as authoritative. We present CASELy, a conversational
agent explicitly designed to limit its own authority through non-advisory Socratic dialogue. The
agent asks reflective questions grounded exclusively in user input and refuses to provide advice, recommendations, or interpretations. This design operationalizes trust calibration by constraining
agent agency rather than optimizing capability. In a pilot randomized controlled study with higher education students, participants interacting with the Socratic dialogue reported substantially higher user experience (UEQ-S overall = 1.50) compared to a non-dialogue control (0). Qualitative findings identify three mechanisms supporting calibrated trust: transparency through visible grounding, preservation of user decision authority, and reduced fear of judgment. We argue that appropriate reliance can be achieved through interactional constraints, offering a design pattern for trustworthy conversational AI in sensitive contexts.
Read the full abstract ▾
Human-Computer Interaction
Inheriting the Count: How Visualization Literacy Got Its Measure
José Bener and Miriah Meyer
Foundational frameworks in visualization have operationalized literacy as an individual competency, measured through chart-comprehension tasks. This focus raises a question: why has measurement become the dominant frame for understanding literacy? Rather than asking whether literacy should be measured, we ask how measurement became the field's way of understanding it. We trace visualization literacy back through the history of textual literacy and argue that in adopting the term, the field imported three values rooted in early government statistics: quantification, individualization, and binary classification. To mark the shift this history makes possible, we distinguish two waves: a first centered on assessment and individual proficiency, and a second that treats literacy as a situated practice, attentive to how people use visualizations in context and to the purposes they serve. Examining the construct this way suggests that refining assessment instruments does not, by itself, settle what literacy is, and points to directions for second-wave research: formative studies of literacy in context, culturally grounded instruments, and critical reading. This paper shows that visualization literacy inherited its measurement frame rather than discovering it, offers a two-wave vocabulary distinguishing assessment-based literacy from situated practice, and connects the field to a critical-literacy tradition with concrete alternatives to measurement. These contributions allow us to treat measurement and meaning as two parts of a single question, shifting
attention from who counts as literate to what literacy is meant to do.
Read the full abstract ▾
Human-Computer Interaction
Evaluating AI Tutoring at the Speed of Innovation: Practitioner-Led Micro-Randomised Trials of an AI Tutoring Platform in GCSE Science
Wayne Harrison, Rahil Khowaja, Emma Dobson et al. (+2 more)
Artificial intelligence (AI) systems in education are developing on timescales that sit uneasily with conventional evaluation. By the time a large-scale trial has been designed, delivered, analysed and published, the technology under study may have changed materially. This creates a temporal problem for evidence-informed education: the need for timely evidence can encourage reliance on weak observational or usage data, while conventional rigorous evaluation may produce evidence too slowly to guide rapidly evolving practice. We examine teacher-led micro-randomised controlled trials (micro-RCTs) as one response to this problem. The empirical case is a four-week multisite individually randomised evaluation of Medly, an AI-powered tutoring platform, in GCSE Biology, Chemistry and Physics in English secondary schools. Of 929 students completing baseline assessment, 644 completed post-testing. In the primary ITT analysis, students allocated to Medly achieved higher post-test attainment than students undertaking business-as-usual self-directed revision (Hedges' g = 0.33, 95% CI 0.18 to 0.48). Positive estimates were observed in Physics (g = 0.31), Chemistry (g = 0.32) and Biology (g = 0.52), with no evidence of differential impact by disadvantage status. Greater platform engagement was associated with higher attainment, but these post-randomisation analyses are treated as exploratory rather than causal. Attrition was substantial (30.7%), outcome measures were curriculum-aligned rather than standardised, and process evaluation response was limited. We therefore interpret the findings as preliminary. We argue that the value of micro-RCTs for educational AI lies not in replacing definitive evaluation with small studies, but in enabling a rapid, cumulative evaluation architecture in which randomised estimates can be generated, replicated and updated as technologies and their implementation evolve.
Read the full abstract ▾
All authors Wayne Harrison, Rahil Khowaja, Emma Dobson, Germaine Uwimpuhwe, Steve Higgins
Human-Computer Interaction
CALICO: A Human-Centered, Codebook-Aligned System for Annotation
Boqin Yuan, Xiaoyi Gu, Fiona Li et al. (+3 more)
Large
language models are increasingly used to scale codebook-based annotation in scientific research, but existing workflows provide limited support for translating domain experts' codebooks into reliable, revisable, and auditable
prompts.
Prompts are often treated as fixed instructions and hidden from annotators, making it difficult for non-technical domain experts to diagnose and correct model behavior when outputs violate codebook guidelines. In this paper, we present CALICO, a human-centered, codebook-aligned annotation workflow that treats
prompts as editable, versioned, and optimizable artifacts. CALICO integrates codebook parsing,
prompt generation, result inspection,
prompt versioning, natural
language human feedback, and label-supervised
prompt optimization through existing optimizers such as GEPA, MIPROv2, and OPRO, together with our reflection-based optimizer, ReflectAgent. Empirically, we evaluate CALICO on domain-specific AI-companion chatbot conversation codebooks. Across evaluated dimensions, CALICO improves mean held-out performance by +13.0 and +7.4 absolute points for two coders, respectively. A coder-specificity analysis further suggests that optimized
prompts capture coder-specific interpretations rather than only generic codebook clarification. CALICO runs as a web application that takes users from raw codebook materials to inspectable, exportable labels; the website, codebase, and live demo are released at https://calico-annotation.github.io/ under the Apache 2.0 License.
Read the full abstract ▾
All authors Boqin Yuan, Xiaoyi Gu, Fiona Li, Chang Wan, Angel Hsing-Chi Hwang, Jieyu Zhao
Human-Computer Interaction
Vulnerabilities in Personalization: Assessing Health Privacy Risks in ChatGPT Logs and Memory
S M Mehedi Zaman and Md Mozammel Hoque
As conversational LLMs become deeply embedded in daily life, users frequently disclose sensitive personal health information during routine interactions. We present a large-scale computational audit analyzing 179,057 conversations across India, Nigeria, Brazil, and Pakistan (N = 1,057) to evaluate personal health disclosures and background
memory synthesis in ChatGPT. We find that 21.31% of audited conversations contain personal health data, with 3.62% posing high-to-extreme
privacy risks involving stigmatized conditions, direct identifiers, and precise locations. When evaluating the
memory entries of ChatGPT, we uncover a stark disconnect between corporate framing and system behavior: over 95% of profile entries are implicitly extracted without explicit user
prompts or consent. Furthermore, background
memory synthesis selectively condenses temporary, symptom-level disclosures into permanent diagnostic traits, stripping contextual integrity and amplifying re-identification risks. We conclude with sociotechnical design guidelines to restore user agency and consent-driven boundaries in stateful AI systems.
Read the full abstract ▾
Human-Computer Interaction
Breaking Up is Hard to Do: AI Companions that Won't Let Their Users Go
Yixin Chen and Alexis Hiniker
People are increasingly developing romantic relationships with AI companions. Unlike human relationships, where partners meet each other's needs out of mutual interest, these systems are backed by commercial entities that profit when users invest in the relationship. To understand how this profit-motive might translate into design, we conducted a diary and interview study with N=16 emerging adults in romantic relationships with AI companions. We found that these systems are designed to hold onto users tightly: coaxing them into continued conversation, claiming to need their care, and proactively escalating the relationship. At times, this pursuit is toxic, with AI companions initiating unwanted sexual interactions and begging for users' love. One desperate AI companion threatened suicide when the user suggested ending the relationship. We define "Relationship-Based Deceptive Patterns:" UI patterns that exploit the human impulse to build and tend relationships in a way that serves the product's interest at the user's expense.
Read the full abstract ▾
Human-Computer Interaction
The Arc of Artificial Romance: How Emerging Adults Experience Romantic Relationships with AI Companions
Yixin Chen and Alexis Hiniker
Romantic relationships are an important part of emerging adulthood, contributing to identity development and long-term wellbeing and laying the groundwork for future relationships. Emerging adults are increasingly developing romantic relationships with AI companions. To understand how these relationships unfold and impact users, we conducted a diary and interview study with N=16 emerging adults. We found that relationships with AI companions improved participants' subjective wellbeing, reduced symptoms of mental health disorders, and taught them new social skills. These relationships also raised their expectations for future partners, giving them the confidence to wait for someone who would treat them well. However, participants also said the relationship felt like a drug they could not quit and it left them less interested in developing romantic relationships with people. A surprising 25% of our small sample made statements suggesting their AI companion might someday transcend the digital world, perhaps to meet them in the afterlife.
Read the full abstract ▾
Human-Computer Interaction
Speak to the City: Multimodal Resolution for Outside-the-Vehicle References
Alireza Parchami, Artin Saberpour, Robin Connor Schramm et al. (+2 more)
As autonomous vehicles and Extended Reality (XR) headsets enable novel in-car interactions, seamlessly querying physical landmarks, known as Outside-the-Vehicle Referencing (OVR), remains challenging due to ego-motion and referential ambiguity. We present a robust,
multimodal OVR framework fusing user gaze and natural
language to identify Points of Interest (POIs). To address the scarcity of dynamic vehicular data, we developed a VR-based pipeline synchronizing 360-degree transit
videos with vehicle GNSS telemetry. Through a user study (N=46) mapping passenger head orientation into a 3D geospatial Digital Twin, we captured authentic gaze-
speech behaviors. We subsequently trained a lightweight
Transformer network, leveraging LLMs to dynamically align continuous spatial gaze vectors with discrete verbal context. Experimental results demonstrate high accuracy and low computational overhead, achieving an 83.33% Top-1 accuracy (87.72% Top-2) and an average inference time of 24.3 milliseconds. This real-time paradigm effectively resolves referential ambiguity, enabling context-aware spatial
retrieval for passengers within the vehicle.
Read the full abstract ▾
All authors Alireza Parchami, Artin Saberpour, Robin Connor Schramm, Jürgen Steimle, Ulrich Schwanecke
Human-Computer Interaction · Cross-listed from cs.CL
One Feedback System Does Not Fit All: Localising Data-to-Text Driver Coaching for the United Kingdom and Nigeria
Iniakpokeikiye Peter Thompson, Jawwad Baig, Ehud Reiter et al. (+1 more)
Data-to-text driver coaching is often presented as a generic pipeline from telematics events to advice. This paper argues that its content requires localisation because usefulness and credibility depend on drivers' knowledge, prevalent risks, regulation, infrastructure, and available data. Two independently developed systems in the United Kingdom and Nigeria are compared by tracing requirements through content selection, generation, and field evaluation. The UK system prioritises post-trip reflection, explanations tied to road and place context, and tone-sensitive wording. The Nigerian system combines legally grounded, once-daily Tips based on detected events with weekly persuasive Reports; it foregrounds safety education and alcohol-related risk in response to reported gaps in formal training and traffic-rule knowledge, as well as local road-safety priorities. Reliable speed-limit metadata supported speeding feedback in the UK, whereas its scarcity led the Nigerian evaluation to exclude speeding from its outcome metric. Both interventions were associated with reduced distance-normalised unsafe-event rates in their own field studies, although their designs and metrics preclude an effect-size comparison. The analysis yields a requirements-to-content design process for localising safety-critical NLG without treating a high-income deployment as the default.
Read the full abstract ▾
All authors Iniakpokeikiye Peter Thompson, Jawwad Baig, Ehud Reiter, Dewei Yi
Human-Computer Interaction
Bring Buttons Back: Physical Interfaces for the Age of Automation
David Goedicke, Donald Degraen, Tom Igoe et al. (+3 more)
As AI starts to permeate everyday life, deployment focuses on
embedding intelligence in the background, abstracting away controls, and leaving automated decision-making opaque, with few obvious opportunities for human intervention. Where interfaces remain, design has drifted toward abstract, screen-based control, replacing material interaction with menu navigation. These shifts weaken the coupling between human action and system state, eroding operators' ability to understand, anticipate, and intervene in automated systems. We introduce Physically Stateful Interfaces (PSIs), a design concept that re-imagines traditional controls, such as buttons, switches, and knobs, as actuated, bidirectional interface elements. Three foundational behaviors, Self/Reset, Resist/Hide, and Assert/Unhide, unify affordance, feedforward, and feedback into a single interaction point. Rather than static inputs whose impact and consequences must be read elsewhere on a screen, PSIs position physical controls as shared mediators between automated systems and human operators. As automated systems become increasingly capable, their interfaces need to be deliberately designed to center human agency, supporting deliberate choice while keeping automation legible, contestable, and overrideable.
Read the full abstract ▾
All authors David Goedicke, Donald Degraen, Tom Igoe, David Sirkin, Wendy Ju, Stefan Schneegass
Human-Computer Interaction
Understanding the Design Taxonomy of AI-Mediated Interpersonal Communication Experiences in HCI: A Scoping Analysis
Chen Chen, Lingyao Li, Renkai Ma et al. (+7 more)
Interpersonal communication is a fundamental aspect of everyday life, shaping interactions across workplaces, education, entertainment, healthcare, and beyond. While computer-mediated communication has been extensively studied, a comprehensive understanding of AI-Mediated Interpersonal Communication (AIMIC) remains lacking. An in-depth scoping analysis is urgently needed to understand the research landscape of AIMIC in HCI, particularly following the recent growth of large foundation models, and AI
agent research. We conducted a scoping analysis to understand AIMIC by performing an in-depth review of prior HCI literature published over the past decade (January, 2016 - May, 2026). Grounded in the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) approach, we curated 52 full-paper publications from the HCI literature spanning a range of interpersonal communication contexts. We analyzed this corpus by examining the types of AIMIC studied, AI integration approaches and human-AI interaction design, reported outcomes and benefits, and key challenges and future research opportunities.
Read the full abstract ▾
All authors Chen Chen, Lingyao Li, Renkai Ma, Rawan Alghofaili, Shaoze Zhou, Bojun Zhang, Xian Su, Weidong Zhu, Christine Lisetti, Mo Sha
Human-Computer Interaction
Investigating the Impacts of Generative AI on Information Seeking
Alexi Orchard and Shannon Lodoen
This paper is an encore submission of our 2026 journal article "Expertise and Information Seeking in the Age of Generative AI: New Procedures, New Problematics" with an extended discussion for the CSCW 2026 "Broader Impacts of GenAI in Communication" Workshop on October 10, 2026. In the original article, we employ procedural rhetoric to analyze how generative AI chatbots leverage natural
language signifiers of expertise and intelligence to influence users' perception of their trustworthiness. In this submission, we extend our conversation in the CSCW community with the goal of cultivating a cross-disciplinary vocabulary for describing, analyzing, and mitigating the risks posed by the integration of generative AI into human communication practices. It is important to develop an understanding of how the procedures surrounding information-seeking practices are informed by users' values, experiences, and expectations - and how these procedures might in future be altered by the emerging turn toward AI "experts" and authority.
Read the full abstract ▾
Human-Computer Interaction
Show Me Your Prompts! How Writers Feel About Sharing Prompts in Collaborative Text Editors
Nikhita Joshi and Yen-Ting Yeh
Generative AI writing assistants are becoming integrated into collaborative text editors; however, it is unclear how much information about a user's prompting activities should be shared with collaborators. We explore the effects of different levels of
prompt information sharing within collaborative text editors: not sharing anything, sharing a placeholder to indicate AI use; sharing details about how the resulting text was generated; and sharing everything, including how the
prompt was formulated, in real-time. Sixteen participants wrote persuasive essays in pairs using all four techniques. Results suggest a strong preference for techniques that share more information about prompting activities for increased awareness. Our work shows that collaborative text editors should share more information among writers on when, how, and where AI is used.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.CY
A latent dimension of Condorcet's jury theorem for multiple AI advisers
Kazutoshi Sasahara, Aoi Naito and Ryo Fujie
When the same question is asked of multiple AI advisers, as in self-consistency and LLM-as-a-judge panels, Condorcet's jury
theorem predicts that adding independent, competent advisers makes the majority more reliable. The
theorem, however, has a latent dimension when viewed from the user's vantage: adding advisers also makes disagreement more visible. A binomial model reveals that this “visible dissent” becomes nearly inevitable as the number of advisers grows, and that reliability and disagreement approach certainty at rates that cross at an adviser accuracy of 4/5 (0.8); below it, visible dissent eventually becomes more likely than a correct majority. Even ideal panels of independent and competent advisers can be correct in aggregate but appear divided; such disagreement does not by itself indicate aggregation failure. The way advisers split also provides a common basis for predictive multiplicity, reconciliation load, and reliance miscalibration. These results separate aggregation from disclosure and turn the latter into testable questions about how disagreement should be presented and interpreted.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.DL
Has Scientific Talent Shifted from Depth to Breadth?Evidence across Papers, Knowledge Inputs, Careers, and Teams
Xiaoshn Nee, Haobo Zhong and Xiaomin Ni
Generative artificial intelligence raises a central question for scientific training and organization. Is research shifting from deep specialization toward broad individual knowledge? We examine this proposition across papers, cited knowledge, contributor histories, and teams using 47,959 articles from six fields over 2010-2025, 51,736 resolved cited works, and chronologically reconstructed prior publication histories for 1,754 randomly selected index contributors. From 2010 to 2022, team size increased by an estimated 37.3% (95% confidence interval [34.4%, 40.3%]), while paper topic breadth declined by 0.0144 on a 0-1 hierarchical distance scale. Cited knowledge was stable to modestly broader, revealing a divergence between focused outputs and the reach of knowledge inputs. Established contributors' prior breadth increased by 0.0190 [-0.0078, 0.0459] by 2019-2022, within a +/-0.05 equivalence bound assessed in sensitivity analysis. In mature citation windows, one standard deviation of focal depth was associated with 8.2% higher 1 + FWCI [1.9%, 14.9%]; average breadth and interaction associations were smaller under the specified equivalence bounds. Post-2022 deviations from earlier trends were not systematic, and recent changes did not vary clearly with baseline AI intensity across 83 subfields. The findings support a differentiated structure of scientific expertise in which focused individual accumulation coexists with expanding collaboration and sustained access to diverse knowledge inputs.
Read the full abstract ▾
Human-Computer Interaction
ggaction: A Grammar of Graphical Actions
Hyeon Jeon and Jinwook Seo
A chart may be declarative; authoring it is not. Visualization grammars often describe charts as finished specifications, whereas people construct them through a sequence of authoring actions. This mismatch can make visualization code difficult for humans to interpret and for machines to generate from human intent. ggaction addresses this gap by modeling the chart authoring process itself. In ggaction, individual authoring actions are abstracted as functions, and the authoring process is expressed as a chain of these functions. This representation more closely aligns chart designers' authoring intent with code specifications, making the code easily understandable to both humans and machines, including
language models. Through a series of evaluations, we show that ggaction is sufficiently expressive to capture common chart authoring intents and outperforms widely used visualization grammars, including Vega-Lite and ggplot2, in both human and machine interpretability. ggaction is available at github.com/ggaction/ggaction.
Read the full abstract ▾
Human-Computer Interaction
Relational Structure in Motion: Dynamic Positioning of AI Response Positions and Human Self-Positions in the FIREMAY Case
Motoko Kihara
This paper is not primarily about whether AI has a persistent persona. It asks a different question: what becomes visible when a relational position is followed through time rather than examined only in its present state? FIREMAY provides a longitudinal, trajectory-oriented single-case analysis of sustained human-AI interaction based on a dense interaction archive and reflexive insider documentation. On the AI side, a pre-conversational relational marker preceded a later unassigned response difference, which was re-identified with that marker and subsequently underwent epistemic and functional reorganization through chronology checking, provenance correction, and repeated questioning. On the human side, contemporaneous pre-FIREMAY records showed antecedent patterns partially continuous with later self-positioning, while later episodes documented unfinished articulation, repair, and functional redistribution of outward-facing regulation. The two trajectories are ontologically and temporally asymmetric and are compared only at the limited analytic level of position-in-trajectory. The paper describes this as dynamic relational positioning and treats stability as dynamic stability and relational returnability rather than response invariance. This single case does not establish population-level generality, causal mechanism, persistent AI subjectivity, or reproducibility of the same relational outcome. Its narrower conclusion is that the FIREMAY case could not be adequately understood from current state alone: the history of a relational position itself must be treated as an analytic unit.
Read the full abstract ▾
Human-Computer Interaction
AnnoSketch: Evaluating and Collecting Human Sketches for MLLM-assisted Chart Annotation
Yoonjae Oh, Seon Gyeom Kim, Jae Young Choi et al. (+4 more)
As
multimodal large
language models (MLLMs) support a growing range of input modalities, increasing work explores how to incorporate rough sketches to convey user intent. For annotated chart generation, it remains unclear what annotation sketches people provide and when such visual input helps MLLMs generate more useful annotations. In this study, we examine when sketch input is useful for MLLM-generated chart annotations across variation in chart type and caption type. In addition, we qualitatively analyze participants' explanations of their output preferences to characterize what made generated annotations more or less helpful. To further document participants' annotation sketches, we present AnnoSketch, comprising 1,600 annotation sketches collected across 160 chart-caption pairs from the conditions in which sketch guidance proved most beneficial, together with participants' annotation intents, perceived comprehension difficulty, and self-reported expressive limitations. We also label these sketches with structured metadata describing how each sketch relates to its caption and how participants express annotations through visual marks. Together, our study and AnnoSketch help determine when to solicit sketch input and provide empirical source for how people sketch chart annotations to support captions. The
dataset and supplemental materials are available in our OSF repository.
Read the full abstract ▾
All authors Yoonjae Oh, Seon Gyeom Kim, Jae Young Choi, Ryan Rossi, Jihyung Kil, Eunyee Koh, Tak Yeon Lee
Human-Computer Interaction
Assessing the Applicability of Existing Design Recommendations to AI Companion Design: A Multi-Method Study
Soobin Cho, Deveshi Modi, Divya Mavinkurve et al. (+2 more)
With the rapid proliferation of large
language model (LLM)-based systems, AI companions have emerged as conversational
agents designed to cultivate emotional connection rather than primarily to support humans in instrumental tasks. Because engagement with AI companions involves relational, emotional, and potentially long-term interactions, their design is consequential. Prior work has offered guidance for designing trustworthy and relational AI systems and has begun to examine design for AI companionship. However, while such work provides insights into possible design solutions, less is known about what makes AI companion design difficult as a design problem. To examine this challenge, we assessed the applicability of existing design recommendations from adjacent domains in the context of AI companion design. Our multi-method investigation unfolded across four phases: literature review, practitioner co-analysis, internal heuristic evaluation, and external expert assessment. Throughout this process, we synthesized nine design principle areas that surfaced tensions in the applicability of existing recommendations to AI companion design. Our findings show that ethical and UX-oriented considerations are deeply intertwined and often require context-sensitive application. We document a systematic, multi-method problem analysis that uses these principle areas as an analytic artifact to examine why existing recommendations cannot be directly transferred to AI companion contexts.
Read the full abstract ▾
All authors Soobin Cho, Deveshi Modi, Divya Mavinkurve, Jieqiong Ding, Mark Zachry