Human-Computer Interaction
How Constraints and Preferences Shape Travel Planning: Implications for AI Planning Support
Fuling Sun, Yining Cao, Peiling Jiang et al. (+2 more)
Planning is a common yet complex activity shaped by constraints to satisfy and preferences to balance. Travel planning, as both an everyday activity and a frequent
benchmark for evaluating intelligent systems, offers a rich context for examining how constraints and preferences emerge and evolve. While recent AI systems have achieved impressive results in generating personalized itineraries, they often assume that users can articulate stable goals upfront. To understand how real-world planning unfolds, we conducted a two-part interview study: one with eight travelers reflecting on their planning experiences, and one with nine travel
agents sharing professional practices. We trace the dynamics of constraints and preferences as they are surfaced, refined, and coordinated throughout the planning process, and identified 11 actions revolving around constraints and preferences, which shaped the planning process. We offer design heuristics for planning tools that better support human-AI collaborative actions to support the fluid, contingent nature of planning.
Read the full abstract ▾
All authors Fuling Sun, Yining Cao, Peiling Jiang, Mingyi Li, Haijun Xia
Human-Computer Interaction · Cross-listed from cs.AI
Building Socio-Affective Artificial Intelligence for Interactive Multi-Agent Simulations
David Berga
The objective of this article is to provide design principles and a software architecture for enabling interaction between humans and multiple
agents in simulated dynamic worlds. This connects the current era of general artificial intelligence (AI/AGI) with the proliferation of
transformer-based conversational
agents and the increased computational capabilities. Given an overview of current and previous multi-
agent theories of mind (socially and affectively-aware
agents), the existence of an integrative design of
agent interactions with themselves and with humans must be crucial for understanding how to create sustainable and governance in future human-
agent reasoning systems. In this work is presented a software "AGIMUD" that integrates: A. socially-aware
reasoning and emotion in
agent behavior and interaction, B. a design of human
multimodal scheme for human users, artificial
agents and simulated worlds, and C. distributing the AI processing through the network to enable multiple autonomous
agents. These integrations allow the dynamic world recreation as multi-user dungeons (MUDs) where both
agents and humans can interact simultaneously in real time. Find the code online in https://github.com/dberga/AGIMUD.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.CL
Experts Rise Where LLMs Disagree: Using Cross-Model Disagreement to Target Expert Effort in LLM Codebook Revision for Large-Scale Annotation
Zeyu He, Zhuqian Zhou, Kirk Vanacore et al. (+2 more)
Large-scale text annotation brings expert insight to millions of documents, often through a codebook that AI annotators follow. Developing a robust codebook, however, takes months. Large
language models (LLMs) could speed this process by applying an early codebook to the data, surfacing cases with strong LLM disagreement, and eliciting expert feedback to address them. We examined three ways experts can provide feedback for LLM codebook revision: (i) editing LLM-generated revisions driven by cross-LLM disagreement (Codebook Verifying), (ii) answering questions about LLM disagreements (Question Answering), and (iii) labeling disagreement cases with rationales (Rationale Labeling). Experiments on thousands of tutoring-session transcripts show that Rationale Labeling yielded the highest LLM-labeling accuracy (64.9%) against expert labels, outperforming the expert-revised codebook (57.8%). The best Question Answering setting also outperformed it (60.5%). Our work shows that LLMs can be used to strategically target expert
attention, shortening months of codebook revision to days without sacrificing labeling performance.
Read the full abstract ▾
All authors Zeyu He, Zhuqian Zhou, Kirk Vanacore, Rene F. Kizilcec, Ting-Hao 'Kenneth' Huang
Human-Computer Interaction
Safety Nudges: User-Facing Interventions for Real-Time AI Risk Awareness
Varshini Elangovan, James Wedgwood, Chhavi Yadav et al. (+3 more)
Conversational AI systems can pose safety risks to their users such as
hallucination, sycophancy, overconfidence, and anthropomorphism, but these risks are difficult for users to detect during everyday use. We introduce Safety Nudges, a browser-based tool that provides lightweight, in situ flags when concerning behavior is detected in chatbot conversations. We evaluated Safety Nudges in a two-week field study with 45 frequent chatbot users, collecting interaction logs, surveys, and feedback on individual nudges. Participants found the tool useful, clear, and minimally disruptive, with nearly all users reporting an increased awareness of potential AI harms, though we found that this improved awareness alone did not necessarily lead to discernible behavioral changes. Our results suggest that user facing safety nudges can complement model-level safeguards by helping people critically evaluate AI responses in context, while highlighting the importance of relevance, calibration, and user control in nudge design for conversational AI safety. The code for our Safety Nudges extension is publicly available at https://github.com/jtbwedgwood/safety-nudges.
Read the full abstract ▾
All authors Varshini Elangovan, James Wedgwood, Chhavi Yadav, William Agnew, Sauvik Das, Virginia Smith
Human-Computer Interaction
Does AI Save Time on Product Design? A Randomized Controlled Experiment of AI Prompt-to-Design Workflows
Remy Stewart, Olabode Anise, Andrew Hogan et al. (+1 more)
AI tools for digital product design now offer
prompt-to-design capabilities, allowing designers and their non-designer colleagues to create prototypes through conversational workflows with large
language models (LLMs). While these tools promise time savings, experimental evidence in product design remains limited compared with evidence from software engineering. We conducted a randomized controlled trial with 50 product designers and 50 product managers to evaluate prospective time savings from leveraging Figma Make in design work. Participants attempted three standardized design tasks with or without access to Figma Make. Among participants who completed the study tasks, access to Figma Make was associated with approximately 20% shorter completion times, with larger gains among product managers. Our findings suggest that
prompt-to-design tools may enable product managers to further contribute to design work, while the benefits for professional designers may be task dependent.
Read the full abstract ▾
All authors Remy Stewart, Olabode Anise, Andrew Hogan, Augustus Griffin
Human-Computer Interaction
Stepping into the Margins: How Readers Want AI to Generate Footnotes
Piper Vasicek, Courtni Byun and Kevin Seppi
Footnotes can be powerful tools to aid understanding, providing information that augments the reading experience. However, static footnotes cannot address every reader question. Current reading tools allow readers to view curated footnotes, allow personal and social annotation, and link dictionaries to reading material. Many other existing tools and natural
language processing (NLP) techniques–such as generative AI, summarization and translation–could be used to address any reader question. However, no one has yet explored which of these features readers actually want. To bridge this gap, we conducted thirteen semi-structured interviews with readers from various backgrounds, followed by a thematic analysis of their responses. We develop themes describing the types of footnotes readers prefer and how to determine the quality of footnotes–specifically focusing on what sources of information a system considers, what the footnotes contain, and how the footnotes are presented to the reader.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.CL
Receptiveness, Not Sycophancy: Distinguishing Engagement from Deference in Language Models
Calvin Isley, Johann Gaebler, Max Lamparth et al. (+2 more)
A central concern with
language models is sycophancy: their tendency to defer to users' views at the expense of independent substantive judgment. In parallel, work on social sycophancy has focused on behaviors such as validation and positivity that may signal inappropriate deference. Yet the markers of social sycophancy are also characteristic of conversational receptiveness, a construct from social psychology shown to improve interactions across disagreement. We argue that this overlap creates a construct-validity problem for social sycophancy evaluations. Using a popular moral-advice
dataset, we find that responses classified as more socially sycophantic are also more receptive. Further, increasing the receptiveness of human-written responses—while preserving their substantive conclusions—causes them to be classified as more socially sycophantic. This tight coupling raises the possibility that social sycophancy evaluations inadvertently penalize desirable behavior. In a preregistered experiment comparing substantively equivalent responses, participants prefer the more receptive responses, expect users to be more likely to listen to them, and are more willing to seek advice from their authors. The same overall pattern persists even among participants who believe the original question asker is in the wrong. Finally, we introduce a simple approach that substantially increases receptiveness without increasing substantive deference, demonstrating that conversational receptiveness and substantive independence can be achieved together.
Read the full abstract ▾
All authors Calvin Isley, Johann Gaebler, Max Lamparth, Julia Minson, Sharad Goel
Human-Computer Interaction
E3Sense: Head-Confined Multimodal Sensing of Learner Engagement
Sidharth Anupkrishnan, Itir Sayar, Jeongah Lee et al. (+5 more)
Engagement-aware learning systems could provide hints or adjust pacing when learners struggle. Prior engagement sensing work distributes sensors across or outside the body rather than consolidating them at one site, or reduces engagement to shared affect or a single dimension (from behavioral, emotional, and cognitive engagement). We introduce E3Sense, a head-worn platform that co-locates electroencephalography, eye tracking, and electrodermal activity to personalize engagement measurement. During a lab study we collected 450 ratings of engagement levels on a five-level ordinal scale while participants watched educational
videos. For fifteen held-out participants, E3Sense achieved a within-one-level prediction score of 75.0%, compared with 63.0% for always predicting the most common rating. In an exploratory analysis of 18 participants from the same study who defined engagement, conditioning on learners' definitions raised the same measure by 6.9 points, from 64.6% to 71.5%. Our work provides a proof-of-concept of a head-site, personalized
multimodal sensing of engagement for adaptive educational interfaces.
Read the full abstract ▾
All authors Sidharth Anupkrishnan, Itir Sayar, Jeongah Lee, Sri Harsha Musunuri, Guan-Ming Su, Madeline Endres, Ravi Karkar, Phuc Nguyen
Human-Computer Interaction
Bespoke: Generating MOOC-Quality Industry-Personalized Lecture Videos at Scale
Romain Puech, Dewang Kumar Agarwal, Antonio Santamaría Escobar et al. (+1 more)
Lifelong learners from different industries often watch the same recorded lecture, even when they will apply the material in different workplaces. We present Bespoke, a system that takes the transcript of an existing lecture and generates a new
video customized to a target industry and duration, with new slides, narration, and charts. From 31 graduate lectures in analytics, machine learning, and optimization, we generated 209
videos for these three industries, plus a generic-audience version of each lecture. Twenty-five experts in the corresponding domains rated 92
videos on a five-point rubric. They judged 87% at or above the rubric midpoint corresponding to “a standard MOOC lecture's quality” (mean overall score 3.42 out of 5; 48% scored 4 or above). Overall quality was similar across industries, durations, and a 21-lecture held-out set unused while developing the system, at about \$0.22 API cost per minute of
video.
Read the full abstract ▾
All authors Romain Puech, Dewang Kumar Agarwal, Antonio Santamaría Escobar, Dimitris Bertsimas
Human-Computer Interaction · Cross-listed from cs.CV
Do Vision Model See Like the Brain? A Comparison Across EEG Encoding Model
Shashank Baghel, Kshitij Dwivedi, Dinesh Singh et al. (+1 more)
Convolutional neural networks (CNNs) and
vision transformers are both used to model the human visual system, but whether the two architectures diverge at a specific point in network depth is unclear. We compared six CNNs and two
vision transformers by computing the Pearson correlation (r) between each model's predicted and measured
EEG response at every layer or block, in ten participants viewing 200 natural images. For the
transformer models, we also tested four
token representations, from the classification (CLS)
token alone to CLS combined with all patch
tokens. CNNs showed strongest correspondence at the earliest layers, weakening at deeper layers, particularly later in the post-stimulus response.
Transformers instead sustained strong correspondence at their deepest blocks, though not at their earliest ones. This advantage depended on
token representation: pooled representations gave weaker peak correlations (r approx 0.48-0.51) than representations retaining all patch
tokens (r=0.640 for CLIP-ViT-B/32, r=0.656 for DINOv2-ViT-B/14). Controlled comparisons showed architecture, not training objective, drove this effect: MoCo-v1 and ResNet-50 (matched architecture) performed nearly identically (r=0.673, 0.670), whereas CLIP-RN50 and CLIP-ViT-B/32 (matched objective) diverged until patch
tokens were preserved. We propose that CNN training's classification bottleneck compresses
brain-relevant information at depth, unlike
transformers' self-
attention and non-classification objectives. A spatial topography analysis showed a common occipital-dominant pattern across all models, indicating these differences reflect signal strength and persistence rather than distinct
brain regions. Patch-preserving
transformer representations sustain
brain-predictive correspondence where CNNs collapse.
Read the full abstract ▾
All authors Shashank Baghel, Kshitij Dwivedi, Dinesh Singh, Sanjeev Nara
Human-Computer Interaction · Cross-listed from cs.CY
The Ethics of Artificial Intelligence in Military Operations
Nicolas Drapier, Florian Mauberger, Aladine Chetouani et al. (+1 more)
Deep learning systems now mediate military decisions to use force, yet their internal logic resists inspection, their evaluation practices are gameable, and their deployment fractures accountability across dispersed stakeholders. The ethical challenge posed by these systems is fundamentally epistemic: not just whether autonomous weapons should be permitted to kill, but whether the conditions for responsible human judgment can survive when critical functions are delegated to opaque
algorithms. We show that this epistemic condition produces a concrete accountability gap: responsibility diffuses across designers, operators, and policymakers while International Humanitarian Law presupposes capacities for judgment that current AI systems lack. To address this gap, we propose a governance framework that proceduralizes ethical constraints through named accountability roles,
adversarial auditing with undisclosed
benchmarks, tiered deployment thresholds, and a proposed NATO evaluation standard. Counterfactual analysis of eight documented cases (1988-2025) shows that each governance mechanism addresses a documented class of failure, but no single safeguard suffices in isolation: effective governance of military AI requires not only technical constraints but the institutional infrastructure to keep human judgment meaningful.
Read the full abstract ▾
All authors Nicolas Drapier, Florian Mauberger, Aladine Chetouani, Aurelien Chateigner
Human-Computer Interaction · Cross-listed from cs.RO
Benchmarking Robots for Everyday Environments: From Lab Experiments to Real-World Operations
Raphael Memmesheimer, Martina Overbeck, Dominik Beyer et al. (+19 more)
This study introduces an interdisciplinary framework for benchmarking
robots deployed in public environments, addressing the gap between traditional laboratory metrics and real-world benchmarking requirements. We evaluate three distinct
robots across diverse use cases - outdoor park cleaning, pedestrian underpass cleaning, and interactive library assistance - each representing unique challenges in public daily life. Over a three-year benchmarking process (2023-2025) comprising seven benchmarking events, a consensus workshop and six on-site evaluations (two per use case), we utilized realistic indoor and outdoor test environments to assess not only technical performance but also the broader implications of deploying
robots in unstructured, human-centric settings. An expert panel, spanning robotics, human-
robot interaction, safety, and economics, systematically developed and refined an evaluation concept to analyze the transition from laboratory prototypes to operational systems. Our findings highlight critical factors for successful deployment, including task fulfillment, interaction quality, safety, and economic feasibility. This work provides actionable insights for researchers and practitioners aiming to bridge the gap between robotic innovation and real-world applicability.
Read the full abstract ▾
All authors Raphael Memmesheimer, Martina Overbeck, Dominik Beyer, Björn Kral, Sabine Bellmann, Sven Schneider, Jan Zimmermann, Anna-Maria Meer, Medina Klicic, Simone Roth, Carolin Straßmann, Alexander Arntz, Marlene Wessels, Johannes Kraus, Paul Schweidler, Tristan Schnell, Christoph Zimmermann, Benedikt Pulver, Wilhelm Stork, Martin Gersch, Sven Behnke, Arne Rönnau
Human-Computer Interaction
AI-Generated Email Drafts Shift Culturally Distinctive Communication Styles in Professional Email
Shintaro Sakai, Alice Gao, Yuichi Shoda et al. (+1 more)
AI assistants that support email composition may shift cultural communication norms, such as the directness typical of low-context cultures like the US versus the indirectness and contextual sensitivity central to high-context cultures like Japan. Yet it remains unknown to what extent people adopt and edit AI drafts inconsistent with their cultural communication norms. We address this through a preregistered within-subject experiment in which Japanese and American participants wrote workplace emails in their native
language without AI, with a low-context AI, and with a high-context AI. We found that Japanese participants wrote emails with significantly more high-context markers (politeness, apologies) than Americans. But AI drafts shifted participants' emails toward the draft's style, with larger shifts when the draft was culturally misaligned: Japanese drifted most under low-context drafts, Americans most under high-context drafts. These findings suggest AI drafts risk overwriting cultural communication norms unless they adapt to users' communication styles.
Read the full abstract ▾
All authors Shintaro Sakai, Alice Gao, Yuichi Shoda, Katharina Reinecke
Human-Computer Interaction · Cross-listed from cs.CV
Faithful Faithfulness Evaluations: Challenges & Pitfalls Learned from a Breast MRI Case Study
Peachapong Poolpol, Henrik H. J. Detjen and Eike Petersen
Saliency maps are widely used to explain deep learning predictions in medical imaging, yet visually plausible explanations do not necessarily reflect a model's true decision process and may therefore mislead clinicians. We investigate this problem using a
Vision Transformer-based breast MRI classifier trained on the ODELIA Breast MRI Challenge
dataset and evaluate multiple saliency methods, including Last-layer
Attention,
Attention Rollout, Grad-SAM, Gradient
Attention Rollout, GMAR, Grad-CAM, and HiResCAM. Our study highlights two often-overlooked challenges in perturbation-based faithfulness evaluation. First, method rankings depend strongly on the perturbation strategy, varying across intensity-based perturbations and
transformer-based
attention masking. Second, benchmarking saliency methods requires distinguishing between class-specific and class-agnostic explanations. To enable fair comparisons, we introduce non-class-specific variants of gradient-based methods and evaluate both settings separately. Across protocols, Grad-CAM and Gradient
Attention Rollout consistently emerged as the strongest class-specific methods, although their relative ranking depended on the evaluation design. These findings expose important limitations of current saliency-based explainability approaches and highlight the need for more robust and standardized evaluation frameworks for trustworthy clinical AI systems.
Read the full abstract ▾
Human-Computer Interaction · Cross-listed from cs.CR
How It's Made: Uncovering Detection Engineering Processes for Network Intrusion Detection Rules
Koen T. W. Teuwen, Emmanuele Zambon and Luca Allodi
Many Security Operations Centers rely on signature-based Network Intrusion Detection Systems like Suricata, yet detection rule engineering remains understudied. We investigate this process by introducing SuriCap, a platform for rule engineering exercises, and hosting CTF-style workshops where 60 participants, trained MSc students, and experienced SOC professionals, created rules for four scenarios. Participants produced 3146 valid rules, enabling analysis of their methods, performance, and iteration patterns. Surprisingly, prior experience had limited impact on rule quality, suggesting that less experienced engineers can produce rules comparable to experts. We also observed challenges in generalizing rules beyond available tests, underscoring the need for sufficient labeled data. From our study, we identify three phases and a common pattern in rule engineering, offering SOC managers insights to improve their processes and expectations of engineer expertise.
Read the full abstract ▾
Human-Computer Interaction
The Impact of Deep Care Isa on Reducing Musculoskeletal Disorders and Enhancing Productivity Among Office Employees: A Comprehensive Study
Quanmin Liang, Junjie Yang, Mohammad Ali Nasseri et al. (+1 more)
Prolonged sedentary behavior, a pervasive issue in modern workplaces, has been closely linked to musculoskeletal disorders (MSDs) and reduced productivity. This study evaluates the effectiveness of Deep Care Isa, an advanced digital health assistant, in addressing these challenges. Utilizing data from over 2,300 participants across 50 corporations, the study demonstrates significant improvements in ergonomic practices, physical activity, hydration habits, and overall productivity, with notable reductions in MSD-related sick leave. The findings highlight the role of innovative ergonomic interventions in enhancing employee well-being and organizational efficiency. Comprehensive statistical analysis underscores the reliability and practical significance of these outcomes.
Read the full abstract ▾
All authors Quanmin Liang, Junjie Yang, Mohammad Ali Nasseri, Daniel Zapp
Human-Computer Interaction
Harnessing LLMs Without Surrendering Control: Delegation Boundaries in Visual Data Storytelling Authoring
Zhuojun Jiang, Yuki Ueno and Chris Bryan
Despite the emergence of large
language models (LLMs) for visual data storytelling workflows, there are open questions about how authors decide what activities or tasks to entrust to them and what should be "protected" or maintained under human control. To investigate this, we interviewed a cohort of 12 expert visual data storytellers. Our analysis shows that participants rarely treated LLMs as autonomous storytellers. Instead, they tend to selectively delegate execution-oriented tasks to LLMs while retaining control over activities that shape narrative intent and story meaning. Our findings show that LLM assistance is most productive after human seeding and constraint-setting, and that it shifts labor from production to verification. We discuss design implications for boundary-aware authoring tools, data-grounded generation, low-fidelity ideation, and reporting practices for LLM-based visualization research. Supplemental materials for this paper are available at https://osf.io/hcnp6.
Read the full abstract ▾
Human-Computer Interaction
SurgGraph: Quantitative Laparoscopic Video Understanding via Geometry-Grounded Scene Graphs
Jingying Wang, Rosiana Natalie, Marquise D Singleterry et al. (+7 more)
Surgical
videos are a primary resource for teaching trainees anatomy, tool usage, and procedural skills. Yet learning from them at scale requires systems that understand surgical scenes. Existing approaches fall short:
vision-
language models lack fine-grained domain
reasoning, task-specific models do not generalize, and prior scene graphs omit clinically meaningful detail. We present SurgGraph, a training-free pipeline that generates quantitative scene graphs from surgical
videos. Operating on segmentation masks and depth maps, SurgGraph encodes each clinically meaningful relation (attachment, occlusion, separation, tool actions) as a <subject, verb, object, value> tuple whose numeric value quantifies the relation's extent over time. Technical evaluations show more precise scene understanding than state-of-the-art surgical VLM baselines. We then build SurgGraphQA, a proof-of-concept learning application that retrieves meaningful and boundary-case exemplars and generates visual explanations and feedback. A study with 17 medical students and 2 resident surgeons shows significant learning gains, demonstrating its educational value.
Read the full abstract ▾
All authors Jingying Wang, Rosiana Natalie, Marquise D Singleterry, Filippos Bellos, Brian George, Gurjit Sandhu, Jason J Corso, Anhong Guo, Vitaliy Popov, Xu Wang
Human-Computer Interaction
SurgGaze: Implicit Calibration for Accurate Gaze Analysis in Operating Rooms with Wearable Eyetrackers
Jingying Wang, Rosiana Natalie, Keyuan Hu et al. (+5 more)
Accurate gaze tracking is essential for understanding surgeons' visual
attention and cognitive processes during laparoscopic surgery, yet wearable eye trackers produce large errors systematically correlated with ground-truth gaze locations, as demonstrated in Study 1. We introduce SurgGaze, an implicit calibration method that corrects these errors using high-confidence surgical moments. Building on evidence that surgeons' gaze converges near the tool-tissue contact point (TTCP) during dissection, SurgGaze uses TTCP as a surrogate for true gaze to construct training pairs. We evaluate SurgGaze in a simulated operating room trial and an authentic operating room case study. In simulation, SurgGaze reduced gaze estimation error by 40.6%, significantly outperforming conventional 9-point explicit calibration. The case study showed that these moments provide reliable training data and that calibrated gaze improves interpretation of surgeons'
attention beyond numeric error reduction. These findings demonstrate that structured behavioral signals can enable implicit calibration for gaze tracking in complex real-world settings.
Read the full abstract ▾
All authors Jingying Wang, Rosiana Natalie, Keyuan Hu, Wenqian Xu, Brian George, Vitaliy Popov, Anhong Guo, Xu Wang
Human-Computer Interaction
Deflecting the Value Compass: Interacting with Large Language Models Temporarily Shifts Human Value Priorities Toward Personal Focus
Hasibur Rahman, Malak Sadek and Smit Desai
Large
language models increasingly support decisions where values are in tension, yet little is known about whether interacting with them changes which values users prioritize. In a preregistered study, 200 U.S. adults interacted with ChatGPT, Claude, or Gemini as a thinking partner or read fixed AI-generated considerations. The
prompt asked LLMs to support
reasoning without recommending a decision and named no values. Participants advised people facing real dilemmas and completed parallel PVQ-RR forms before, immediately after, and one task later. Each LLM condition temporarily shifted value priorities toward personal focus relative to the control (d=0.37-0.51), primarily through increased Self-Enhancement. Participants' advice retained words and meaning from their exchanges. Thus, a brief LLM interaction that neither targets values nor seeks to persuade can reorient values active during judgment without detectable convergence in value directions or advice.
Read the full abstract ▾
Human-Computer Interaction
When the Strike Zone Becomes Algorithmic: Umpire Judgment and Player Challenge Decisions under AI Review
Kichang Lee, Gyeongmin Han, Sungmin Lee et al. (+1 more)
The Automated Ball-Strike challenge system that Major League Baseball adopted in 2026 offers a distinctive setting for studying human AI interaction in which umpires make every ball and strike call, while players can selectively ask an automated system to publicly overturn those decisions. We analyze 4,114,256 called pitches from 2015 through 2026 and 8,447 challenges from the 2026 season to examine how algorithmic review reshapes umpire judgment and player behavior. We study where umpires placed the effective strike zone boundary, how consistently they applied that boundary, how they responded to overturned calls, and which calls players chose to challenge. In 2026, the effective called boundary shifted toward the automated strike zone beyond the trajectory observed in prior seasons, while the consistency of that boundary largely continued its existing trend. Following an overturned call, umpires temporarily adjusted subsequent decisions near the corrected boundary, although these effects did not consistently persist into the next game. Count dependent variation in calling remained, while differences associated with player status narrowed. Players, meanwhile, left many overturnable calls unchallenged and appeared to base challenge decisions more strongly on immediately observable evidence than on the precise geometry of the automated zone. Together, these findings show that selective AI review does more than correct individual errors. It reshapes human judgment, adaptation, and strategic behavior around an algorithmic authority.
Read the full abstract ▾
All authors Kichang Lee, Gyeongmin Han, Sungmin Lee, JeongGil Ko