Human-Computer Interaction · Cross-listed from cs.CV
Reliability-Gated Fusion of Consumer Head and Foot IMUs for Lower-Body 3D Pose
Zhilin Guo, Boqiao Zhang, Oszkár Urbán et al. (+8 more)
Sparse inertial pose estimation promises camera-free motion capture from consumer devices, but consumer sensors are unreliable: firmware-fused orientations are biased, mounting varies between sessions, and streams drift or drop out. On a new 35-take single-subject
benchmark pairing an earbud head inertial measurement unit (IMU) with two smart-insole foot IMUs (SAM-3D-Body pseudo-ground-truth labels), we show the reliability problem is channel-level: a channel ablation isolates foot acceleration as the most informative input (66.6 mm vs. 79.0 mm head-only) and the firmware-fused foot orientation as the liability that destroys the gain. We therefore let the model learn how much to trust each channel of each stream: one temporal gate per stream per channel block, trained with an auxiliary reliability objective on synthetically corrupted pretraining data. The channel-gated model is the most accurate of our learned fusion arms on clean data (69.4 mm vs. 83.7 static, 86.6 ungated) and under every simulated fault (bias in training; drift, dropout eval-only); its gates suppress the natively biased foot-orientation channels on clean real data without test-time supervision and flag dropout bursts at 0.92-0.999 AUROC. Two contrasts: dropping a channel known a priori to fail is flat across foot faults but collapses when an unanticipated stream fails (head dropout: 92.9 vs. 79.3 mm); and a fine-tuned HMD-Poser is more accurate on clean data (64.4 mm) and nominally under drift, with no significant paired difference under bias or dropout, but a larger worst-case degradation from clean (+16.1 vs. +3.5 mm, single seed). Learning to gate reliability instead of sensor count is the lever for deployable sparse inertial capture. Code is available at https://github.com/ZhilinGuo/reliability-gated-imu-fusion.
Read the full abstract ▾
All authors Zhilin Guo, Boqiao Zhang, Oszkár Urbán, Josef Bengtson, Hakan Aktas, Wenzhao Li, Siyu Hong, Kyle Fogarty, Chenliang Zhou, Ali Senguel, Cengiz Oztireli
Human-Computer Interaction · Cross-listed from cs.SI
Reclaiming the social in social media
Luca Benn, Moritz Merz and David Grüning
Concerns about polarization, antisocial behavior, and mental health have broadly led to two responses to social media: adapting platforms through moderation and prosocial design, and restricting access to these platforms through age limits and comparable measures. Both address consequences of social media while leaving the
attention-driven architecture intact. We argue that the harms commonly attributed to social media arise not from technologies supporting social connection but from their implementation within the
attention economy. In response, we ask what a digital social environment built for genuine human connection would look like. From four principles - Purpose,
Alignment, Transparency, and Access - we can derive four technical properties that fulfill these principles: Operator Blindness, Algorithmic Sovereignty, Operational Parity, and Verifiability. To the best of our knowledge, no deployed alternative satisfies all four. We propose an architecture that does, making the
attention economy not just discouraged but structurally impossible. We explain how the resulting user experience refocuses on each user's individual social environment and differs from that of current platforms.
Read the full abstract ▾
Human-Computer Interaction
Alignment Games: A Framework for Conceptual Repair in Human-AI Collaboration
Hari Subramonyam, Maneesh Agrawala and Sean Follmer
The meaning of a concept in use is shaped by the situation, task, goals, and prior knowledge. For example, a request to make a poster "visually appealing for a five-year-old" might evoke bright colors and cartoon imagery for one collaborator, but less text, bold shapes, and visual simplicity for another. We call such task-relevant differences conceptual misalignment. We introduce
Alignment Games, a framework for making these differences visible and repairable during human-AI interaction. Drawing on theories of situated conceptualization, we characterize task-specific conceptual frames in terms of relevant attributes, values, relations, constraints, and priorities. We then define
alignment moves that intervene on the situation, the
reasoning used to interpret it, or the resulting frame. Through examples from educational content generation, creative coding, and argumentative writing, we show how these moves can be composed into repair sequences and derive design principles for supporting task-sufficient conceptual
alignment at runtime.
Read the full abstract ▾
Human-Computer Interaction
The Future of Visualization Dashboards in the Age of Generative AI
Vaishali Dhanoa, Duosi Dai, Gabriela Molina Léon et al. (+2 more)
Generative AI promises easier dashboard creation, raising questions about the future of dashboards and the people who create and use them. We interviewed 16 experts based in 14 countries about their practices and expectations. Almost all expected dashboards to persist for recurring questions, monitoring, and reporting. They anticipated adaptive views and combinations of
language, graphical controls, and gestures, while emphasizing interaction as part of human exploration and understanding. Participants expected authors' responsibilities to shift toward specifying requirements, curating generated work, and evaluating outputs, with design knowledge and communication remaining important. Easier creation also raised concerns about validation effort, users' understanding, maintenance, and personalization weakening shared understanding. We discuss seven opportunities for research and practice concerning validation, end-user education, dashboard proliferation and rot, organizational guidance, adaptation, novel visualizations, and accountability for AI-generated content. Our findings connect dashboard
evolution with the human and organizational work needed to sustain their use.
Read the full abstract ▾
All authors Vaishali Dhanoa, Duosi Dai, Gabriela Molina Léon, Eduard Gröller, Niklas Elmqvist
Human-Computer Interaction
Toward a Culturally Adapted Chinese Language Agent: A Wizard-of-Oz Study of Nonverbal Behavior in Chinese-German Intercultural Interaction
Siddhant Jain, Anna Lea Reinwarth, Dimitra Tsovaltzi et al. (+2 more)
Successful intercultural communication requires more than grammatical competence. It demands sensitivity to culturally embedded social norms whose violation triggers subtle but meaningful nonverbal responses. For German learners of Mandarin Chinese, acquiring this sensitivity is critical yet poorly supported by existing
language-learning
agents. We present a Wizard-of-Oz (WoZ) study design and supporting real-time system for collecting
multimodal behavioral data from native Chinese speakers reacting to social norm violations by German learners. The system features a photorealistic MetaHuman avatar driven by Live Link face capture and MediaPipe upper-body tracking, a wizard console for real-time behavior selection, and synchronized
multimodal logging across
agent and learner streams. A layered annotation framework, based on psychological theory and covering non-observable socioemotional reactions, norm interpretation, verbal, and observable behavior thereof, and future supervision targets enables the corpus to support training of future automated cultural interpretation and behavior generation models. Four ecologically valid interaction scenarios, developed with cultural and pedagogical experts, provide the methodological and technical foundation for a culturally adapted conversational
agent for Chinese
language learning.
Read the full abstract ▾
All authors Siddhant Jain, Anna Lea Reinwarth, Dimitra Tsovaltzi, Rafael Math, Julia Renner
Human-Computer Interaction · Cross-listed from cs.SE
Making the Invisible Visible: A Framework for Reflective AI Use in Software Engineering Education
Ali Shakiba and Thomas Chaffey
Generative AI (GenAI) is increasingly embedded in software engineering education, supporting activities such as requirements development, design exploration, documentation, and prototyping. However, educators often have visibility only into final artefacts, with limited insight into how students evaluate, verify, and refine AI-generated outputs during the learning process. This creates challenges for assessing evaluative judgement and responsible AI-assisted practice. This paper introduces the AI Journal, a structured reflection framework designed to make student-GenAI interaction visible in first-year software engineering education. The framework combines execution tracking, which records
prompts, outputs, intent, and interaction context, with cognitive auditing, which captures verification strategies, intervention decisions, confidence judgements, critical learning moments, and reflections on AI-supported work. Deployed in a first-semester software engineering course, the AI Journal enabled visibility into aspects of student learning not observable through artefact-based assessments alone. Preliminary observations suggested variation in verification practices, intervention strategies, and perceptions of AI-supported work. Critical learning moments frequently occurred when students evaluated contextual suitability, feasibility, and requirements
alignment rather than identifying obvious errors. The AI Journal demonstrates a practical, lightweight, and model-agnostic approach for making AI-assisted learning processes visible. By foregrounding verification, intervention, and reflection, it shifts
attention from product-focused assessment toward evaluative judgement and responsible AI-assisted practice.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.CV
One Sensor, Whole Body - 3D Body Pose from a Single Consumer Earbud IMU
Zhilin Guo, Boqiao Zhang, Oszkár Urbán et al. (+8 more)
Consumer earbuds already stream inertial motion data from the head, one of the most widely worn sensor locations on the body. We ask how much of the 3D body pose a single such head IMU can recover, and whether adding more consumer sensors actually helps. We build a
multimodal capture pipeline that records four-view RGB-D
video together with an AirPods head IMU and two Striv insole IMUs, synchronize the streams post-hoc, and generate pseudo-ground-truth with SAM 3D Body, yielding a 35-take single-subject
benchmark spanning gait, turning, vertical, everyday, and clinically inspired motions. Adapting two recurrent model families (IMUPoser and MobilePoser), we show that one head IMU recovers lower-body pose at 79.0 mm rigid-MPJPE and per-foot ground contact at 0.809 macro-F1, and that a causal variant retains most of this accuracy at streaming latency. In paired per-take significance tests across both families, adding the consumer foot IMUs never significantly improves pose and significantly degrades it in two of four model-split combinations; a mounting-bias probe and feet-only ablation identify insole orientation quality, not foot placement, as the mechanism. Extending the output to a 20-joint full-body skeleton maps the boundary: gross distal-arm motion is partially recoverable from the head alone, proximal upper-body pose is not, and staged fine-tuning recovers the leg accuracy that naive joint training sacrifices to multi-task dilution. For learned pose from consumer wearables, sensor reliability, not sensor count, is the binding constraint here. For the devices tested, the earbud is its sweet spot. Code is available at https://github.com/ZhilinGuo/one-sensor-whole-body.
Read the full abstract ▾
All authors Zhilin Guo, Boqiao Zhang, Oszkár Urbán, Josef Bengtson, Hakan Aktas, Wenzhao Li, Siyu Hong, Kyle Fogarty, Chenliang Zhou, Ali Senguel, Cengiz Oztireli
Human-Computer Interaction
From Early Participation to Later Completion: Evidence from a Large-Scale Self-Paced Learning Programme
Sakshi Sharma, Pavani Ayinampudi, Aditya B.M.V. et al. (+5 more)
Large-scale learning programmes generate records that make learner participation observable across different activities. Participation points are commonly used to record and encourage such participation, but their value may extend beyond the activities for which points are awarded. Existing evaluations often examine gamification outcomes within the activities or learning environments in which the game elements are implemented, providing limited evidence about whether early participation points contain information about later participation outside the points system. This study examines whether early participation points can provide information about learners' later participation in a self-paced learning track that does not award participation points. Using anonymised records from 876 learners in a large-scale remote software-upskilling internship, we examined participation points generated from live-session attendance and poll responses against later self-paced course completion. The primary analysis used the 438 learners who earned at least one point during the first week, while the full cohort was retained for the no-point analysis. Week-one participation points distinguished learners who later completed a self-paced course with an AUC of 0.89, increasing to 0.95 by the fourth week. Similar AUCs were observed at both stages of the self-paced course sequence, while the absence of week-one points identified learners who did not start or did not complete a self-paced course with 95% precision. These findings indicate that early participation points can provide information about later participation outside the activities that generate the points. Such information can help large-scale learning programmes identify learners who may require timely
attention while learning is still in progress, without treating participation points as a measure of overall learner engagement.
Read the full abstract ▾
All authors Sakshi Sharma, Pavani Ayinampudi, Aditya B.M.V., Jinal Gupta, Prakash Hegade, Rohit Sharma, Meenakshi V, S.R.S. Iyengar
Human-Computer Interaction
"Black Mirror?": Public Sensemaking of AI-Powered Lifelogging
Ying Ma, Jarod Govers, Le Fang et al. (+4 more)
AI-powered lifelogging wearables are emerging as a new class of consumer devices that transform everyday experience into searchable, AI-curated
memory archives. We study early public sensemaking around these systems at the moment of their
market entry, using the Looki L1 as an empirical lens. Analysing large-scale Chinese-
language and English-
language social media discourse (N = 5,053 comments), we combine topic clustering with inductive thematic analysis to examine how users interpret the social, moral, and political implications of AI-mediated
memory. Across contexts, users reference dystopian surveillance imaginaries, express
privacy resignation and bystander concerns, and debate assistive value alongside consumer logics. English-
language comments more often framed these devices through interpersonal power, evidentiary use, and hacking anxieties, while Chinese-
language comments more often foregrounded labour exploitation, governance surveillance, and technological inevitability.
Read the full abstract ▾
All authors Ying Ma, Jarod Govers, Le Fang, Shuning Zhang, Yongquan 'Owen' Hu, Xin Yi, Jorge Goncalves
Human-Computer Interaction
AI Tools Adoption across the Double Diamond Workflow: Phase, Mode, and Barriers in Designer Practice
Sepideh Tajarmakan and Khashayar Hojjati Emami
Designers are adopting AI faster than the tools built for them can keep up. This survey of 443 designers across 43 countries, among the first phase-disaggregated accounts of its kind, examined reported AI use across the four phases of the Double Diamond workflow (Discover, Define, Develop, Deliver). 79.7 percent reported confirmed AI use in at least one phase, but engagement was typically partial, spanning a mean of 2.89 of 4 phases, with adopters retaining the earliest phases and dropping the latest. Tool choice tracked each phase's dominant activity: conversational tools drove Discover and Define, AI-native image generation took over Develop, and Deliver showed a hybrid profile. AI-native and embedded AI use peaked in different phases, pointing to two distinct modes of human-AI collaboration: generating from scratch versus refining within existing software. Use intensity declined through fewer designers engaging, not scaled-back use. Adopters and non-adopters differed on one dimension: perceived usefulness.
Read the full abstract ▾
Human-Computer Interaction
Show your work: An exploratory study of student experiences of Turnitin Clarity
J. Roe and M. Perkins
Process-capture platforms promise to make the writing behind an assessed artefact visible, and vendors increasingly
market them as a
fairness measure rather than a detection tool. Their acceptability to the students who must write within them, however, is largely untested. This exploratory qualitative study examines how students' expectations of one such platform, Turnitin Clarity, compared with their experience of using it. Seven students at a UK university, given no onboarding to the platform, took part in pre-use focus groups. Six then completed an unassessed 500-word writing task over five days, before all seven returned for a post-use focus group. Data was analysed using reflexive thematic analysis, with Expectation-Confirmation Theory as an orienting frame. The overall pattern was one of mixed confirmation. Participants' functional expectations were largely met, and the sense of being watched that they had anticipated persisted after use: for some it was offset by perceived
fairness, while for others it heightened self-monitoring. The bounded AI assistant divided them: its limits were welcomed for marking out acceptable use, but some felt it weakened their ownership of the work and worried it would flatten what they produced. Across both phases, participants weighed the costs of observation against perceived gains in
fairness and asked for transparency to run in both directions. Several described already writing defensively in anticipation of accusations of AI misuse. Acceptance of process technologies was reported as being conditional on practice time, two-way transparency, and explicit data governance. Participants also questioned whether a single linear document can represent a writing process they described as messy and
multimodal. Across these accounts, process capture did not sit outside the writing it recorded but reorganised the practice it set out to observe.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.RO
Bayesian Active Learning for Intent Disambiguation in Interactive Robot Planning
Huao Li, Carson Sobolewski, Augustinos Saravanos et al. (+3 more)
Interactive
robot planning requires
robots to infer and execute human intentions from natural
language instructions that are often ambiguous, incomplete, or underspecified. Although large
language models (LLMs) provide a powerful interface for clarification, relying on the generative model to drive an multi-turn conversation can introduce systematic failures. We propose a
Bayesian framework that treats clarification as an active learning problem over grounded Signal Temporal Logic (STL) task specifications. Our method uses LLMs to initialize candidate formal specifications and translate informative contrasts into natural-
language clarification questions, while
Bayesian optimization maintains uncertainty estimation over user intent and selects queries that maximize information gain. After convergence, the inferred STL specification is passed to a formal planner to synthesize a verifiable
robot trajectory. Across four simulated and real-world task domains, our approach generally achieves higher task satisfaction and requires fewer clarification rounds than LLM baselines, while helping smaller models close the performance gap against larger
reasoning models.
Read the full abstract ▾
All authors Huao Li, Carson Sobolewski, Augustinos Saravanos, William Tan, John Karigiannis, Chuchu Fan
Human-Computer Interaction
When Models Choose the Question: Pedagogical Constraints in Bottom-Up Multi-Agent Inquiry
Yeri Hong and Lauren Hyoseo Yoon
What shapes a model-generated inquiry when no discussion question is supplied? We introduce a bottom-up forum framework inspired by Philosophy for Children, in which
language-model
agents read a philosophical narrative, propose and select questions, and develop a shared conclusion without a privileged model facilitator or aggregator. Across 576 forums, contrasting Aristotelian value personas interacted with a blank-slate participant receiving no value-specific instruction. The blank slate remained neutral and was selected more often for conclusions than questions. Yet inquiry narrowed in both form and source: varied initial questions increasingly became either/or alternatives, while discussion concentrated on directions already explicit in the text. Our ECO framework traces this source focus by distinguishing explicit philosophical framing, characters' modeled inquiry, and open-ended narrative material. Across analyzed chapters, participants drew most often on explicit framing. We describe this as a pedagogical constraint: freedom to formulate questions did not necessarily produce freedom from source framing.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.AI
PainterBench: A Figural Divergent-Thinking Benchmark for Tool-Using Language Models
Shane K.A. Dalumura Hettige and Jonas Oppenlaender
Figural divergent thinking is the ability to develop a given shape fragment into an original drawing. In humans, this ability is assessed with incomplete-drawing tasks. We introduce PainterBench, a
benchmark that ports the incomplete-drawing task to the agentic setting. The
agent draws on a canvas through tool calls and observes the result after every turn. The canvas includes a starting shape which cannot be erased, and the
agent's goal is to incorporate this shape into the most original drawing it can produce. The task is open-ended, and the
agent itself decides when the drawing is finished. The
benchmark tests incremental visual planning over a short horizon and the transfer of creative ability from pretraining to multi-turn tool use. We evaluate 14
multimodal language models from small to frontier scale. Across the primary study and six sensitivity analyses, we collect 2,700 drawings and crowdsource creativity and recognizability ratings for every drawing and for 300 human reference drawings. We also present ViDrA-adapted, an automated scorer that predicts human creativity ratings of
agent drawings (r = 0.85 on random held-out test split). Figural divergent thinking varies widely across the 14 models, and GPT-6 Astra produces the most creative drawings. Relative to the human drawings, the
agent drawings score higher in creativity but lower in recognizability. We release the final drawings, per-round canvas snapshots, tool call traces, stimulus bank,
benchmark harness, crowdsourced ratings (N = 72,000), and ViDrA checkpoint.
Read the full abstract ▾
Human-Computer Interaction
Exploring the Affordances of Generative Image AI for Supporting Early-stage Architect-client Communication
Chengzhi Zhang and Weijie Wang
Text-to-image generative AI can produce renderings from natural-
language prompts in near real time, making it increasingly popular for rapidly visualizing concepts in early-stage architectural design. Meanwhile, exchanging ideas efficiently and building shared understanding have long been central challenges in architect-client communication. How might the speed of generative image AI change this communication? To explore this question, we conducted a study with 11 architect-client pairs, in which each pair used generative image AI over
video conference to collaboratively produce early-stage renderings of the client's "dream house." Our findings suggest that generative image AI helped pairs develop a solid shared understanding by providing concrete visual materials and supporting the exchange of ideas. It also shifted conversation dynamics, enabling clients to participate more actively in shaping design direction. However, challenges emerged, including a stylistic bias toward particular types of images and unpredictable shifts in design direction caused by variation across generations. We conclude with implications for the design of future generative image AI-based systems that support architect-client communication.
Read the full abstract ▾
Human-Computer Interaction
HiThink Turn: An Intent-Aware Turn-Taking Control Module for Full-Duplex Dialogue
Feiyang Chen, Wenhan Yang, Bohan Wang et al. (+4 more)
Full-duplex dialogue requires timely yet selective interruption handling, which end-of-turn prediction alone cannot achieve: complete utterances may need no response, while unfinished requests may warrant interruption. To address this challenge, we propose HiThink Turn, an intent-aware streaming turn-state predictor that separates response intent from semantic completeness and conditions decisions on system playback state. A key contribution is minimal intent-sufficient prefix supervision, constructed through LLM judgments and
speech alignment, while training on audio truncated at chunk boundaries improves robustness to partial
speech. These components support streaming inference with 240-ms audio chunks, enabling low-latency, accurate full-duplex turn control. Experiments show that HiThink Turn leads the compared methods in Easy Turn macro accuracy, Full-Duplex-Bench average interaction rate score (0.933), and non-target-
speech average playback resume rate (0.735). Additionally, intent-prefix triggering raises interruption success from 89% to 98% and reduces mean stop latency by 60.9%.
Read the full abstract ▾
All authors Feiyang Chen, Wenhan Yang, Bohan Wang, Xinjian Gao, Rongjunchen Zhang, Jun Wang, Xinhui Hu
Human-Computer Interaction
Scanvas: Discovering and Developing Synergistic Opportunities in Generative Design Spaces
Yaqing Yang, Aniket Kittur, Hongyu Howie Wang et al. (+4 more)
Good design is often synergistic, creating super-additive value by linking goals so that existing resources produce greater outcomes. However, finding these synergistic opportunities in sparse design spaces is difficult, and current LLM-supported ideation tools largely default to additive paradigms such as feature blending, variant generation, or local patching. We present Scanvas, an AI-supported system for systematically discovering and developing synergistic design opportunities. Scanvas operationalizes synergy through a two-step computational process: first, it decomposes seed ideas into explicit properties (components, behaviors, surpluses, and issues) to enrich the design space; second, it systematically searches across enriched ideas using three theory-grounded strategy operators: unlocking or strengthening goals, turning weaknesses into resources, and sharing components across functions. We instantiate Scanvas as an auto-generation pipeline and an interactive system. Pipeline ablations and a user study with 12 professional designers demonstrate that Scanvas enables users to surface and develop significantly higher-quality, synergistic concepts compared to LLM ideation baselines.
Read the full abstract ▾
All authors Yaqing Yang, Aniket Kittur, Hongyu Howie Wang, Nikolas Martelaro, Matt Klenk, Yan-Ying Chen, Matthew K. Hong
Human-Computer Interaction
StructSim: Measuring Idea Similarity at Scale Through Structural Representation
Yaqing Yang, Vikram Mohanty, Mei-Xi Chia et al. (+3 more)
Measuring idea similarity is fundamental to creativity evaluation, especially as LLMs enable idea generation at increasing scale. However, text
embeddings collapse an idea into a single vector, making it difficult to capture structural similarity, including partial overlap across core and supporting components and differences across levels of abstraction. We introduce a shared structural representation that decomposes ideas into purpose, mechanism, and implementation components and organizes related components in a multi-layer concept graph. From this representation, we define measures of pairwise similarity and set-level mechanism coverage for assessing idea diversity. We evaluate our approach using controlled idea triples and assessments from 12 experts, focusing on differences in core mechanisms, implementations, and supporting components. Our method improves
alignment with expert judgments of structural similarity by 31% over the
embedding baseline and better reflects expert assessments of idea set coverage, supporting scalable evaluation of idea similarity and diversity.
Read the full abstract ▾
All authors Yaqing Yang, Vikram Mohanty, Mei-Xi Chia, Nikolas Martelaro, Dafna Shahaf, Aniket Kittur