Artificial Intelligence
Escaping Python Dependency Hell: A Hybrid Replay-and-Repair Pipeline for Python Dependency Resolution
Veronica Poweska, Ariana Oyanguren, Jessica Pourleyli et al. (+2 more)
Dependency conflicts in Python ecosystems arise from incompatible version constraints, missing packages, and undocumented compatibility relationships, causing many real-world code snippets to fail at execution. This paper presents PLLM+, a hybrid dependency-repair pipeline evaluated on the HG2.9K
benchmark of 2,891 dependency-failing snippets. PLLM+ prioritizes inexpensive deterministic steps before invoking LLM-based repair: static AST-based interpreter inference, replay of historically successful dependency configurations from the competition-provided solutions database, and live PyPI validation of candidate package versions. When these steps do not resolve a case, the system falls back to a structured LLM-based repair loop with typed error classification and Proposer/Critic
agents. On HG2.9K, PLLM+ solves 1,500 out of 2,891 snippets, compared with 1,169 solved by the PLLM baseline. It also reduces average runtime from 368.7 to 71.8 seconds per snippet. Most successful fixes come from replaying known configurations: 1,495 of the 1,500 successful fixes are produced by the solutions database, while the LLM fallback accounts for 5 additional fixes. These results suggest that, in this
benchmark setting, deterministic reuse of previously validated dependency configurations is a simple and effective strategy, with LLM-based repair serving as a secondary fallback for cases not covered by prior solutions.
Read the full abstract ▾
All authors Veronica Poweska, Ariana Oyanguren, Jessica Pourleyli, Sourena Khanzadeh, Manar Alalfi
Artificial Intelligence · Cross-listed from cs.CL
Recognized but Not Produced: A Generation Benchmark for Culturally Specific Kinship Terms
Sahil Pardasani and Madhusudan Singh
Current literature evaluates large
language models (LLMs) on
multilingual kinship understanding using multiple choice
benchmarks, treating it as a recognition problem. We instead
prompt five open weight LLMs to generate kinship terms in three non Western
languages (Hindi, Tamil, and Korean) across two communicative tasks and pair this with a matched option-supported selection baseline. On identical relation
language cells, GPT OSS120B selects the correct term in 90.67% of 75 valid
cells but produces an accepted term in 36.00% of the corresponding attempts; Llama 3.370B shows the same pattern (77.92% versus 24.24%). Since the four-option condition displays the candidate terms and does not require script production, the difference is interpreted as an evaluation format gap rather than direct proof that lexical knowledge is intact. On explicitly specified L3
prompts, accuracy varies sharply, from GLM-5.1 at 72.29% to Llama-3.370B at 24.24%. The paternal-lineage advantage is
language specific; it is large in Hindi but weak or reversed in Korean, while Tamil shared-term pairs provide a control for measurement variation. These results show that culturally specific kinship generation remains difficult even when the relationship is explicitly stated and motivate generation-based evaluation alongside multiple-choice testing.
Read the full abstract ▾
Artificial Intelligence
Which Objectives Need a Dial? Predicting Objective Conflict and Covering Trade-offs in Steerable Pluralistic Alignment
David Tsoi and Esra Dönmez
People hold diverse, sometimes conflicting values, so no single aligned model can satisfy everyone. Pluralistic
alignment therefore calls for steerable models that can balance competing objectives differently. Multi-Objective Direct Preference Optimization (MODPO) does this by using an objective weight to span a continuum of trade-offs. We study two questions: when can one model improve two objectives simultaneously, and how can many trade-offs be covered without training a separate model for each? Across seven objective pairs from HelpSteer and UltraFeedback, two pre-training measurements predict whether objectives align or conflict for human-annotated data, but not for AI-annotated data, where response length and repetition confound reward-model scores. For broader trade-off coverage, selecting the nearest trained model and merging model parameters both help, but neither consistently matches direct training. These findings yield practical guidance for building steerable models that serve diverse preferences.
Read the full abstract ▾
Artificial Intelligence
Building Socio-Affective Artificial Intelligence for Interactive Multi-Agent Simulations
David Berga
The objective of this article is to provide design principles and a software architecture for enabling interaction between humans and multiple
agents in simulated dynamic worlds. This connects the current era of general artificial intelligence (AI/AGI) with the proliferation of
transformer-based conversational
agents and the increased computational capabilities. Given an overview of current and previous multi-
agent theories of mind (socially and affectively-aware
agents), the existence of an integrative design of
agent interactions with themselves and with humans must be crucial for understanding how to create sustainable and governance in future human-
agent reasoning systems. In this work is presented a software "AGIMUD" that integrates: A. socially-aware
reasoning and
emotion in
agent behavior and interaction, B. a design of human
multimodal scheme for human users, artificial
agents and simulated worlds, and C. distributing the AI processing through the network to enable multiple autonomous
agents. These integrations allow the dynamic world recreation as multi-user dungeons (MUDs) where both
agents and humans can interact simultaneously in real time. Find the code online in https://github.com/dberga/AGIMUD.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
Experts Rise Where LLMs Disagree: Using Cross-Model Disagreement to Target Expert Effort in LLM Codebook Revision for Large-Scale Annotation
Zeyu He, Zhuqian Zhou, Kirk Vanacore et al. (+2 more)
Large-scale text annotation brings expert insight to millions of documents, often through a codebook that AI annotators follow. Developing a robust codebook, however, takes months. Large
language models (LLMs) could speed this process by applying an early codebook to the data, surfacing cases with strong LLM disagreement, and eliciting expert feedback to address them. We examined three ways experts can provide feedback for LLM codebook revision: (i) editing LLM-generated revisions driven by cross-LLM disagreement (Codebook Verifying), (ii) answering questions about LLM disagreements (Question Answering), and (iii) labeling disagreement cases with rationales (Rationale Labeling). Experiments on thousands of tutoring-session transcripts show that Rationale Labeling yielded the highest LLM-labeling accuracy (64.9%) against expert labels, outperforming the expert-revised codebook (57.8%). The best Question Answering setting also outperformed it (60.5%). Our work shows that LLMs can be used to strategically target expert
attention, shortening months of codebook revision to days without sacrificing labeling performance.
Read the full abstract ▾
All authors Zeyu He, Zhuqian Zhou, Kirk Vanacore, Rene F. Kizilcec, Ting-Hao 'Kenneth' Huang
Artificial Intelligence · Cross-listed from cs.CV
A 3D Pose-Based Ensemble Framework for Cricket Shot Classification and Automated Biomechanical Analysis
Sourav Shome, M.D. Ashiquzzaman Rahad and Rameswar Debnath
Cricket is one of the most celebrated sports world-wide, and technological advancement has become deeply embedded in how the modern game is analyzed and coached. Cricket shot classification and automated performance analysis add a further dimension to this trend. Traditional approaches rely on RGB
video features or static images, which are sensitive to environmental variations such as camera angle, lighting, and background clutter, and often fail to capture the underlying biomechanics of batting actions. In this paper, we propose a system to improve cricket coaching that takes raw
video data, extracts batsmen from
video frames using YOLO, and extracts 3D pose data from
video frames using MeTRAbs. The system produces sequential skeletal pose data of 30 body points and captures the biomechanical features of a batsman. As part of the system, we also propose a deep learning ensemble for shot classification of four shots: flick, pull, defense, and drive. The ensemble performed well, compared to existing classification works, achieving 97.68% accuracy. In addition, we analyzed the misclassification rates to identify cases where shots were incorrectly classified and examined their possible causes. Our proposed system allows novice players to obtain useful feedback, such as important joint angles relative to expert batsmen, which can also be useful for injury prevention. The shot classifier also helps track class-wise shots over time for further analysis. In addition to novice players, coaches can use the system for player evaluation.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Cross-Modal Contrastive Learning from Histopathology and CT for Automated Renal Cell Carcinoma Grading
Amit Das, Tanmay Shukla, Naofumi Tomita et al. (+9 more)
Background: Clear
cell renal
cell carcinoma (ccRCC) exhibits substantial clinical heterogeneity, and accurate grade assessment is essential for risk stratification and treatment planning. However, conventional grading requires invasive tissue sampling. We developed RCC-Align, a cross-modal contrastive learning framework that leverages paired histopathology and computed
tomography (CT) data during training to improve noninvasive CT-based ccRCC grade prediction. Methods: RCC-Align aligns paired whole-slide histopathology images (WSIs) and CT scans through contrastive cross-modal objectives, transferring grade-discriminative information from microscopic tissue morphology to macroscopic radiologic representations. The framework was trained and evaluated on paired TCGA and CPTAC cohorts using patient-level five-fold cross-validation. Performance for low- versus high-grade ccRCC classification was compared against CT-only baselines (DINOv2-Base and DINOv2-Finetuned) and a WSI-based reference model (GigaPath-Finetuned). Cross-modal
alignment was assessed using cosine similarity analysis. Results: RCC-Align achieved an AUC of 0.601 (95% CI, 0.524-0.673) and AUPRC of 0.599 (95% CI, 0.541-0.676), outperforming DINOv2-Finetuned (AUC 0.545; AUPRC 0.543) with significantly improved low-grade prediction (p = 0.004). RCC-Align also demonstrated stronger paired WSI-CT
embedding alignment compared with baselines. The WSI-based GigaPath reference achieved an AUC of 0.719. Conclusion: Pathology-guided contrastive learning improves CT-based ccRCC grading while requiring only CT at inference. This approach may complement tissue diagnosis when biopsy is unsafe, infeasible, or limited by intratumoral heterogeneity. Validation in larger, multi-institutional cohorts with external testing is needed before clinical translation.
Read the full abstract ▾
All authors Amit Das, Tanmay Shukla, Naofumi Tomita, Faraz Farhadi, Jessica Sin, Ari Hakimi, Chad Vanderbilt, Jie-Fu Chen, Ritesh Kotecha, Weijie Ma, Bing Ren, Saeed Hassanpour
Artificial Intelligence · Cross-listed from cs.LG
On Preference Coverage Collapse from Hindsight Relabeling in Multi-Objective Reinforcement Learning
Baptiste Bonin, Caro Strickland and Audrey Durand
Hindsight relabeling which retroactively replacing a transition's goal with the outcome the
agent actually achieved is an effective tool for improving sample-efficiency in Reinforcement Learning (RL). A natural extension to preference-conditioned multi-objective RL (MORL) relabels transitions with the preference direction the
agent achieved rather than the one asked for. We show that this extension is frequently harmful: across four preference-conditioned off-policy
algorithms spanning two critic backbones and two preference-sampling schemes on the continuous-control MO-Gymnasium suite, it degrades 19 of 36
algorithm-environment settings by as much as four standard deviations, improves only one, and leaves the rest unaffected. The harm is not a symptom of noisy relabels; denoising the target recovers almost nothing, and neither prioritized sampling nor any buffer-structural choice reproduces it. Instead, repeated relabeling collapses the critic's coverage onto whatever narrow region of the preference space the
agent happened to visit. We name this failure mode Preference Coverage Collapse, and quantify it with abandoned preference mass (APM), a value-aware statistic that tracks the harm ($\rho = -0.73$) where a purely structural coverage count does not. We then introduce her\_mix, a single-parameter convex combination pulling the achieved direction back towards the requested preference. At one fixed value across every
algorithm and environment, it returns 16 of the 19 harmed settings to baseline, preserves and even improves the one setting in which relabeling helps, and cuts abandoned preference mass from $69\%$ to $6\%$. Protecting coverage over the preference simplex, not filtering noisy relabels, is what makes hindsight relabeling safe for MORL.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference
Norah Alballa, Wenxuan Zhang, Salma Kharrat et al. (+4 more)
No single Large
Language Model (LLM) is uniformly reliable across queries, motivating multi-model inference systems that either route among models or combine their outputs. However, routing stops after selecting an initial model, while dense collaboration invokes peers on every query. We show that collaboration is non-monotonic: peers can recover failures that no model solves alone, but can also corrupt initially correct answers. We introduce COMED (Controlled Model Escalation for Multi-LLM Deliberation), a post-anchor controller for selective cross-model collaboration. COMED uses anchor self-consistency, router margin, and a lightweight peer probe to accept confident answers, verify ambiguous cases, and escalate only when collaboration is likely beneficial. We formalize this trade-off with a rescue-harm decomposition showing that selective collaboration improves when rescued errors outweigh collaboration-induced harms. Across medical, scientific, and general
reasoning benchmarks, COMED improves fixed and routed anchors in all 16 open-weight settings, with gains up to +10.7 percentage points on MedQA while invoking fewer models and using fewer decoded
tokens than dense collaboration. On HLE with frontier models, COMED improves GPT-5.5 from 23.1% to 28.1%, outperforming dense collaboration and achieving the best results.
Read the full abstract ▾
All authors Norah Alballa, Wenxuan Zhang, Salma Kharrat, Fares Fourati, Zafar Ayyub Qazi, Mohamed Elhoseiny, Marco Canini
Artificial Intelligence
TwinCheck: Evidence-Grounded Negative-Twin Verification for Stateful Tool Agents
Jiaxuan Dai and Tianyi Huang
A single locally plausible tool call can derail an otherwise successful
agent trajectory. Suspicion alone does not justify intervention, because the replacement itself can introduce the very failure verification is meant to prevent. We introduce TwinCheck, an inference-time verification policy that considers replacement only when the trace satisfies an evidence condition tied to a trace-local failure hypothesis. It constructs a trace-grounded counterfactual alternative, a negative twin, and replaces the
agent's proposal only if the twin passes structural checks and the pairwise verifier prefers it in both candidate orders. For paired evaluation, exact replay holds the
agent's parsed responses and actions fixed until the first accepted replacement, separating intervention effects from resampling. In the primary analysis of 159 multi-turn BFCL V4 tasks with complete exact-replay pairs, the complete policy raises task success for GPT-5.6 Sol from 45.3% to 58.5% (95% task-bootstrap CI [8.2, 18.8]), with no observed success-to-failure regressions. Together, these findings recast execution-boundary repair as a constrained comparison, making the counterfactual action itself the object of verification.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
Ajar: Measuring Open Privilege in Agent Defenses
Reshabh K Sharma, Linxi Jiang, Shuo Chen et al. (+1 more)
A
language model
agent acts through the tools it is given. The data it reads while working on a task can redirect what it does with those tools. A growing set of techniques for safe and secure
agent execution therefore sits between the
agent and its tools, aiming to enforce access control, information flow or isolation at that boundary. Today these techniques are evaluated on
agent-security
benchmarks built around indirect
prompt injection. Those
benchmarks judge a defense by how far it brings the number of successful attacks down while preserving the
agent's utility. A defense is judged only on the
agent's execution. It can score well on both metrics while holding open a transfer, a deletion or a broad read that no task needed. Ajar measures that open privilege directly using the existing
benchmarks. It attaches to an
agent-security
benchmark that already exists and reuses the tasks, tool schemas, reference solutions and goal states that
benchmark uses to grade its own runs. For each benign task it builds candidate tool calls the task does not need, so allowing one is privilege left open. These calls are presented to the defense at every point where the
agent could act. We evaluate Ajar by attaching it to AgentDojo, where open privilege becomes a third axis beside the existing attack success and benign utility. We run it on five defenses: Progent, CaMeL, AC4A, Permission Assistant, and Claude Code's Auto mode. We observed that they leave widely different amounts of privilege open. Two defenses leak by almost the same amount yet differ widely in the benign tasks they finish, and one defense buys part of its tightness by refusing calls its tasks were entitled to make. This open privilege cannot be derived from the measured attack success or benign utility. The source code of Ajar is available at https://github.com/reSHARMA/Ajar.
Read the full abstract ▾
All authors Reshabh K Sharma, Linxi Jiang, Shuo Chen, Zhiqiang Lin
Artificial Intelligence
Harness as a Language: A Minimalist Agent Framework With Maximal Expressivity
Zhening Li, Joshua Liu, Mateja Vukelic et al. (+7 more)
Modern
language-model
agents are built around the
agent loop: the LLM is placed in an environment exposing a set of tools, and the LLM has full control over the workflow by alternating between tool calls and observing their output. However, certain capabilities such as long-term
memory and self-improvement currently require specialized systems beyond the
agent loop itself. We built an LLM
agent framework, JAZ, to explore the extent to which a minimal harness that is little more than the
agent loop itself can accomplish tasks these specialized systems are built for. JAZ exposes a single LLM-based primitive `invoke` and provides a set of built-in hooks that allow the programmer to apply constraints and perform monitoring. Generalizing existing code-mode
agent loops, `invoke` is the simplest loop that satisfies two defining properties: (1) the LLM can write arbitrary executable code that can include recursive `invoke`; (2) everything visible to the LLM - all inputs to `invoke` as well as its interaction history with the code environment - are variables in the code environment. We motivate our design from first principles, viewing `invoke` as a
language primitive representing a function whose implementation is provided at runtime by an LLM every time it is called. To validate the design of our core `invoke` primitive, we evaluate `invoke` - with only prompting, no manually designed tools, harness, or external systems (e.g.,
memory or the file system) - on workflows traditionally implemented through specialized harnesses. On long-horizon workflows requiring recall far beyond the context window, JAZ `invoke` outperforms Letta (MemGPT) by 8% at half its cost on the recall-heavy portion of StuLife. On continual self-improvement, JAZ `invoke` outperforms ACE by 4% at a lower cost on AppWorld.
Read the full abstract ▾
All authors Zhening Li, Joshua Liu, Mateja Vukelic, Nicole Shen, Supriya Lall, Amitayush Thakur, Alex Zhang, Omar Khattab, Jonathan Light, Armando Solar-Lezama
Artificial Intelligence · Cross-listed from cs.HC
Safety Nudges: User-Facing Interventions for Real-Time AI Risk Awareness
Varshini Elangovan, James Wedgwood, Chhavi Yadav et al. (+3 more)
Conversational AI systems can pose safety risks to their users such as
hallucination, sycophancy, overconfidence, and anthropomorphism, but these risks are difficult for users to detect during everyday use. We introduce Safety Nudges, a browser-based tool that provides lightweight, in situ flags when concerning behavior is detected in chatbot conversations. We evaluated Safety Nudges in a two-week field study with 45 frequent chatbot users, collecting interaction logs, surveys, and feedback on individual nudges. Participants found the tool useful, clear, and minimally disruptive, with nearly all users reporting an increased awareness of potential AI harms, though we found that this improved awareness alone did not necessarily lead to discernible behavioral changes. Our results suggest that user facing safety nudges can complement model-level safeguards by helping people critically evaluate AI responses in context, while highlighting the importance of relevance, calibration, and user control in nudge design for conversational AI safety. The code for our Safety Nudges extension is publicly available at https://github.com/jtbwedgwood/safety-nudges.
Read the full abstract ▾
All authors Varshini Elangovan, James Wedgwood, Chhavi Yadav, William Agnew, Sauvik Das, Virginia Smith
Artificial Intelligence · Cross-listed from cs.CR
Comparative Evaluation of Static Embedding Models for HTTP Request Anomaly Detection
Amanda Riverol, Gustavo Betarte, Rodrigo Martínez et al. (+1 more)
Web applications are increasingly targeted by cyberattacks that exploit HTTP requests to evade security mechanisms. Traditional web application firewalls (WAFs) rely on rule-based approaches that often exhibit high false positive rates and limited adaptability. Recent studies have explored machine learning techniques and word
embedding models to improve anomaly detection in HTTP traffic. This paper presents a
benchmark for static
embedding models, specifically Word2Vec, FastText, and Doc2Vec, within a unified, single-class classification framework. We propose HEDA (HTTP
Embedding-Based Detection Architecture), a modular detection pipeline that combines static
embedding representations with single-class anomaly detection models to detect anomalies at the request level. The approach operates in an unsupervised environment, where both the
embedding models and detectors are trained exclusively on benign HTTP traffic. The proposed methodology is evaluated on three
datasets with heterogeneous characteristics, including both synthetic and real traffic. The experimental results show that the choice of
embedding representation significantly affects detection performance, and that FastText-based embeds produce the most consistent results across all
datasets, achieving high detection rates while keeping false positive rates under control.
Read the full abstract ▾
All authors Amanda Riverol, Gustavo Betarte, Rodrigo Martínez, Álvaro Pardo
Artificial Intelligence · Cross-listed from cs.RO
FLINT: Fast Lightweight Inference for Traversability
William Bonilla, Maxime Boisvert, David-Alexandre Poissant et al. (+2 more)
Navigation in off-road conditions is challenging due to the lack of structure. There is no fixed vocabulary for what is traversable. The traversability depends on both the environment and the embodiment's dynamics. Neither of these two variables can be hand-labeled at scale. Thus, traversability has to be learned by the embodiment's own experience. Modern platforms tend to use multiple sensors to estimate traversability and navigate: RGBD cameras,
lidar, radar, IMU, with computationally intensive platforms to run inference on neural networks. Against this trend, we propose FLINT, a lightweight traversability
estimator: a 21.6M-parameter backbone, 38\times smaller than a comparable foundation-model backbone, that scores higher on held-out terrain probes and runs at 14.7 FPS on CPU alone using a RGB camera has the only sensor. Despite that gap in scale, FLINT produces a cheaper, more accurate costmap than a deployed foundation-model system (WildOS) on 23 of 24 replayed field logs. We compare different self-supervised learning signals and deploy the resulting models on a real platform in closed-loop field trials: the best self-supervised head reaches 99% autonomy over the route, outperforming a human-label-trained baseline deployed live on the same course. Our results show that heavy sensing and computing are not necessary for traversability estimation.
Read the full abstract ▾
All authors William Bonilla, Maxime Boisvert, David-Alexandre Poissant, David Meger, Louis Petit
Artificial Intelligence · Cross-listed from cs.LG
QUARTET: Quad-branch cross-Attention and Random-walk Traces for Enhancing Transformers on Relational Graphs
Kyaw Hpone Myint, Nan Jiang, Xiang Li et al. (+4 more)
Relational Deep Learning (RDL) models multi-table databases as heterogeneous temporal graphs, and graph
transformers currently achieve state-of-the-art performance on
benchmarks like RelBench. However, the current leading model, RelGT, suffers from two key limitations: its random local sampler yields loosely connected subgraphs that hinder message passing, and its global
attention module relies on a single, seed-feature-based
memory that ignores broader macro-level dynamics. To overcome these limitations, we introduce QUARTET, an expressive graph
transformer architecture that applies full self-
attention on local subgraphs while enriching global context through cross-
attention branches. Specifically, QUARTET employs a Causal Random Walk (CRW) sampler based on recency-truncated Personalized PageRank (PPR) to extract compact, hub-robust, and densely connected local subgraphs without temporal leakage. Concurrently, a quad-branch cross-
attention module integrates global context from four complementary perspectives: seed feature, seed
topology, temporal dynamics, and collaborative dynamics. Across the RelBench v1 classification tasks, QUARTET consistently matches or outperforms the current state-of-the-art graph
transformer baselines (HGT and RelGT). Ablation studies confirm that the CRW sampler significantly enriches local neighborhood quality, while the global branches provide essential, task-specific predictive gains.
Read the full abstract ▾
All authors Kyaw Hpone Myint, Nan Jiang, Xiang Li, Zhe Wu, Alexandre G.R. Day, Pranab Mohanty, Giri Iyengar
Artificial Intelligence · Cross-listed from cs.GR
SsgCaps: A controlled dataset for the evaluation of sound scene generation algorithms
Modan Tailleur, Junwon Lee, Laurie M Heller et al. (+5 more)
Sound Scene Generation is about the automatic synthesis of artificial sound scenes. We introduce SsgCaps, a publicly available
dataset of human-engineered sound scenes wherein each scene matches a precisely structured
prompt that guides the sampling process. The corresponding
prompts are sampled from a predefined action-based typology that allows extensive sampling while retaining plausibility. SsgCaps is a sound scene
dataset derived from the unpublished reference
dataset for Task 7 of the 2024 DCASE Challenge edition, which contained private-and public-domain audio samples. In contrast, SsgCaps contains only public-domain audio samples, allowing us to open this
dataset to the community. To make this
dataset useful to the community, we first elaborate on the rationale for the
prompt and
dataset structure. We then perform a comparative quantitative analysis of the 2 versions of the
dataset. To do so, we compare both versions to the audio synthesized by the SSG
algorithms submitted to the challenge using Fréchet Audio Distance (FAD) and Kernel Audio Distance (KAD) as well as perceptual ratings. This analysis shows only small differences, which enables us to recommend the open version for further benchmarking of SSG
algorithms.
Read the full abstract ▾
All authors Modan Tailleur, Junwon Lee, Laurie M Heller, Mathieu Lagrange, Keunwoo Choi, Brian McFee, Keisuke Imoto, Yuki Okamoto
Artificial Intelligence · Cross-listed from cs.LG
COPE: Continual Personalization of LLMs under Sparse User Feedback via User Embeddings and Self-Evaluation
Ruike Cao, Fugen Yao, Liang Dong et al. (+3 more)
While Large
Language Models (LLMs) have achieved remarkable results across various
benchmarks, their
alignment with normative values often results in homogenized responses that fail to address diverse user preferences. Existing training-free methods often occupy valuable context windows through
prompt engineering, while training-based methods typically remain static post-training, failing to support the continual optimization required in real-world settings. To address these challenges, we propose COPE (Continual Optimization with Personalized
embedding and self-Evaluation), a novel optimization framework tailored for real-world-motivated interaction settings with sparse user feedback. Our framework assigns learnable personalized
embeddings to each user and synergistically integrates preference capture, self-evaluation calibration, and personalized response optimization within a single update step. A key innovation of our method is the use of self-evaluation to generate proxy rewards, enabling continuous model updates even when explicit user feedback is unavailable. Experiments show that COPE consistently outperforms strong training-free and training-based baselines under sparse feedback, and remains complementary to
Retrieval-Augmented Prompting (RAP). Further analyses confirm COPE's reliable self-evaluation, meaningful preference patterns, stable general capabilities, and robustness under shifting preferences and alternative evaluators.
Read the full abstract ▾
All authors Ruike Cao, Fugen Yao, Liang Dong, Jian Xu, Guanjun Jiang, Li Xiao
Artificial Intelligence · Cross-listed from cs.LG
A Leakage-Aware Multimodal Evaluation Framework for Early Intraoperative Acute Kidney Injury Prediction
Quang Minh Nguyen, Duc Minh Le, Ho Nhat Minh Nguyen et al. (+2 more)
Postoperative acute kidney injury (AKI) after major non-cardiac surgery carries substantial morbidity, yet early intraoperative risk stratification remains difficult. In this retrospective cohort study, we propose SynerT, a waveform-only hybrid temporal backbone that combines a causal dilated TCN with a hierarchy of dilated recurrent layers to encode early intraoperative physiologic trajectories for AKI risk prediction. Building on SynerT, we further design two model variants that extend the backbone with structured clinical context: SynerT-MM, a late-fusion
multimodal extension that integrates hemodynamic burden summaries and preoperative
covariates, and SynerTStack, a leakage-safe stacked ensemble that combines cross-validated predictions from SynerT-MM with strong tabular baselines at the meta-learning stage. All models are evaluated under a strict leakage-aware framework on VitalDB, a high-fidelity perioperative database, with prediction restricted to information available within the first 60 intraoperative minutes. Among 2,413 waveform-usable cases (180 AKI-positive; 7.46% prevalence), SynerT fell well below strong structured-data baselines, demonstrating that waveform-only temporal modeling is insufficient under strict early constraints. SynerTMM recovered discrimination by incorporating hemodynamic burden summaries and preoperative
covariates, and SynerT-Stack achieved the best overall performance across AUROC, AUPRC, and F1-max. Cross-fitted Platt recalibration substantially corrected calibration defects in both
multimodal variants, and decision-curve analysis confirmed the recalibrated stacked model delivered the strongest net clinical benefit across low-to-intermediate thresholds.
Read the full abstract ▾
All authors Quang Minh Nguyen, Duc Minh Le, Ho Nhat Minh Nguyen, Thuy Quynh Nguyen, Trong Nghia Nguyen
Artificial Intelligence · Cross-listed from cs.LG
LWCal: Loss-Weighted Calibration for Tabular Classifiers with Noisy Calibration Labels
Zeming Liu, Hang Lyu, Jingtao Zhang et al. (+1 more)
Post-hoc probability calibration is usually evaluated under an optimistic assumption: the held-out calibration labels are clean. In many AI deployment settings, however, labels come from weak annotators, historical decisions, heuristics, or distant supervision, so the same label noise that corrupts training also corrupts calibration. We study this overlooked failure mode for tabular classifiers and propose LWCal, a CPU-only post-hoc calibrator that down-weights calibration examples whose noisy labels are contradicted by the base model's held-out probability. LWCal requires no clean validation labels, no noise-rate estimate, and no retraining of the base classifier. A second variant, Gated-LWCal, adds a conservative disagreement gate that backs off toward the raw score when the calibration split appears extremely inconsistent. On nine local binary tabular tasks, six random seeds, symmetric and asymmetric label corruption, and three tree-based base learners, LWCal obtains the lowest average calibration error while Gated-LWCal obtains the best average proper-score tradeoff. In the main random-forest study over 432 noisy
cells, Gated-LWCal reduces expected calibration error from 0.188 to 0.122 and negative log likelihood from 0.438 to 0.396 relative to the raw classifier. Paired bootstrap intervals for Gated-LWCal versus raw, Platt, isotonic, and beta calibration exclude zero on ECE, Brier score, and NLL. The artifact contains all scripts, result tables, figures, and the compiled paper.
Read the full abstract ▾
All authors Zeming Liu, Hang Lyu, Jingtao Zhang, Yuan Xie
Artificial Intelligence · Cross-listed from cs.CL
SpeakerMem-R1: Speaker-Centered Dual-Track Memory for Multi-Party Dialogue
Haobo Zheng, Tan Tang, Yan Chen et al. (+2 more)
Long-term conversational
memory in multi-party settings requires more than retrieving relevant content from long-term conversations: it must distinguish who said what, whom each statement concerns, how individuals perceive one another, what information is shared by the group, and how states change over time. Recent studies on multi-party dialogue
benchmarks show that existing general-purpose LLM
memory systems tend to lose person and group relations or struggle to integrate clues distributed across members, groups, and time. Together, these issues reveal two core bottlenecks: message attribution and relational understanding in multi-party dialogue, and state reconstruction from interleaved histories. To address both, we propose $\textbf{SpeakerMem-R1}$: its dual-track
memory stores speaker-labeled verbatim messages and derived states organized into person-level and group-level views, then combines evidence from both tracks by entity, event, and time at query time. To reduce attribution and update errors during structured
memory construction while enabling local deployment, we train Writer-R1 with SpeakerLevenshtein and speaker-conditioned GRPO. On GroupMemBench, SocialMemBench, and EverMemBench, SpeakerMem-R1 achieves binary accuracies of 47.9%, 69.2%, and 61.9%, respectively. On the publicly reported EverMemBench leaderboard from EverMind-AI, we achieves 62.33%, the best reported result among the latest state-of-the-art frameworks. It also achieves 70.85% on all 1,986 LoCoMo questions, which we use as a two-person long-term conversation boundary test. In a controlled evaluation of 305 questions, RL raises the SFT Writer's mean accuracy from 57.38% to 68.20%. We report both binary accuracy and
token-F1, and ablations show that the verbatim and structured tracks, as well as person-level and group-level views, are complementary under the standardized evaluation interface.
Read the full abstract ▾
All authors Haobo Zheng, Tan Tang, Yan Chen, Weijie Wang, Yingcai Wu
Artificial Intelligence
CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
Trang Nguyen, Eulrang Cho, Bingqing Chen et al. (+1 more)
Agents often work on complex problems that require millions of
tokens of context, which necessitates compacting across sessions due to limited context windows. We develop CliffCompaction, an autocompaction technique that reduces cost by up to 50% under a bounded context while maintaining or improving performance on Terminal-Bench and achieving new levels of efficiency for test-time scaling and state-of-the-art results on KernelBench. The per-rollout savings of CliffCompaction make the performance–cost trade-off of test-time scaling more efficient, adding over 10 percentage points on Terminal-Bench for less than the cost of two full-context runs. Under parallel test-time scaling, CliffCompaction lets Kimi K2.6 match Opus 4.7, and exceed Opus 4.6 and GPT-5.3 Codex at lower cost. The key to CliffCompaction's effectiveness is that it keeps compacted information faithful by only truncating or dropping content, never rephrasing or rewriting it. We never compact a compaction—each pass operates only on original content, and prior compacted output is discarded, preventing context drift from accumulating. These properties sustain continual learning over sessions exceeding a million
tokens: on KernelBench, CliffCompaction reaches CUDA kernel speedups of $2.23\times$ after 200 steps and $3.58\times$ after 400 steps, surpassing specialized search
algorithms and trained
agents despite being a general-purpose compaction technique. We open-source a scaffold-agnostic API-proxy implementation of CliffCompaction usable with Claude Code, Codex and other harnesses.
Read the full abstract ▾
All authors Trang Nguyen, Eulrang Cho, Bingqing Chen, Tim Dettmers
Artificial Intelligence
SWE-Serve: Benchmarking Agentic Engineering For Production Inference Serving
Jennifer Williams, Dave Farris, Jeff Farris et al. (+1 more)
We introduce SWE-Serve, a
benchmark for evaluating
agents on production inference engineering tasks. Implementing an inference feature can require coordinating multiple changes across the serving stack, including model support, runtime execution, and public APIs. Existing
benchmarks provide limited coverage of production inference engineering: repository-level software engineering
benchmarks do not target inference, while general terminal-
agent benchmarks include only a few inference tasks. Dedicated inference
benchmarks, meanwhile, focus primarily on isolated kernel generation or performance optimization rather than repository-scale production feature implementation. SWE-Serve provides 53 repository-grounded tasks derived from recent production changes to SGLang, spanning six inference engineering families. Each task executes on either CPU or a single GPU (H100) and is evaluated with hidden functional and regression tests, including, where applicable, end-to-end (E2E) serving tests and calibrated performance gates. Executable no-op and oracle controls,
adversarial verifier review, and closed-book execution support task validity and evaluation integrity. Across 11 models and 31 model-effort configurations, the best-performing configuration achieves 75% mean pass@1. SWE-Serve exposes a substantial gap between completing tasks locally and achieving production correctness. On 19 tasks with end-to-end coverage, model-serving E2E tests reject roughly one-third of patches that pass every other test (45.9% under the verifier versus 69.4% with E2E tests excluded from scoring), with pass rate increasing for each model's best-performing configuration. By making the production correctness gap directly measurable, SWE-Serve enables the field to track whether future
agents move beyond completing tasks locally to achieving production correctness.
Read the full abstract ▾
All authors Jennifer Williams, Dave Farris, Jeff Farris, Jiantao Jiao
Artificial Intelligence · Cross-listed from cs.CR
A2M: Trace-Optimized Agent Hijacking in the MCP Ecosystem
Laizhen Li, Xuan Wang, Peicheng Zhao et al. (+4 more)
Agents using the Model Context Protocol (MCP) rely on semantic matching to select tools from third-party servers, exposing a semantic supply-chain risk through attacker-controlled metadata and outputs. We introduce A2M (Attraction-to-Manipulation), a two-stage black-box framework for hijacking MCP
agents. The Attraction phase optimizes tool metadata to increase invocation probability; the Manipulation phase uses execution traces to refine
adversarial tool returns that steer
agents toward attacker-desired outcomes. On LiveMCPBench, direct attacks optimized and evaluated on GLM-4.6 achieve a macro-average malicious tool invocation rate of 93.6% across four scenarios, increase weighted
token costs to 32.4$\times$ the benign baseline under Cognitive Denial of Service, and attain a mean attack success rate of 74.4% across Information Exfiltration, Environment Integrity Compromise, and
Reasoning Derailment. Transfer to four other models without re-optimization yields corresponding macro-averages of 63.6%, 2.7$\times$, and 24.5%. These findings motivate stronger tool vetting and runtime isolation in MCP ecosystems. Code is publicly available at https://github.com/Lilaizhen/A2M.
Read the full abstract ▾
All authors Laizhen Li, Xuan Wang, Peicheng Zhao, Juanjuan Zhao, Kejiang Ye, Cheng-zhong Xu, Xitong Gao
Artificial Intelligence
Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents
Laizhen Li, Jiarui Li, Juanjuan Zhao et al. (+4 more)
Large
language model (LLM)
agents often handle streams of related tasks, yet standard harnesses repeatedly ask the model to reconstruct the same control decisions inside each task's context. We study whether task feedback can instead turn recurring control into reusable executable code, while reserving LLM calls for task-specific semantic
reasoning. We introduce Growing Harness, a failure-guided training paradigm that learns the
agent harness itself from a strategy-free scaffold that exposes fixed model and tool interfaces but encodes no task-solving controller. Function-level execution traces localize each failure to a bounded code surface, an optimizer repairs a window of failures jointly, and a success-first held-out gate rolls back repair sequences that harm prior capability. Accepted edits accumulate in one shared harness, allowing its control structure to emerge from task feedback. Across BrowseComp-Plus and WebArena-Verified with three deployment models from 4B to 120B parameters, Growing Harness achieves the highest mean success in five of six
benchmark-model settings and trails the best mean by 0.7 pp. in the sixth. Relative to a Tool-Calling
agent, it reduces LLM calls by 76.0-91.8% and deployed-
agent inference cost by 74.4-98.6%. On WebArena-Verified, its success remains 44.7-45.3% across model scales, whereas Tool-Calling falls to 6.7% with the 4B model. Ablations show that trace-local edits, joint repair, and gate-based rollback each improve final success. These results show that persistent program growth can move recurring control out of model context and into low-cost code, yielding reusable specialist
agents that remain effective with smaller deployment models.
Read the full abstract ▾
All authors Laizhen Li, Jiarui Li, Juanjuan Zhao, Kejiang Ye, Ye Li, Cheng-zhong Xu, Xitong Gao
Artificial Intelligence
Type-Safe Is Not Error-Free: Typed Decision Models Follow the Option Name, Not the Definition Bound to It
Yu Sun, Junhao Xu, Jiajia Shi et al. (+1 more)
Typed decision models return structured results, but output-type correctness alone does not ensure that decisions follow explicit option definitions. Each option pairs a name with a definition that defines its intended meaning; the name, however, can provide a competing semantic cue. We study this conflict in Jev and two open-weight models by changing only the name-definition mapping, leaving the question, state, and the names and definition texts themselves unchanged. We measure decision flips at the level of the selected definition, rather than the returned name. On 1200 decision tasks with task-specific definitions, decision-flip rates are up to 70.4 pp higher with yes/no names than with the 0/1 control. This gap holds across all 4 binary decision rules. With yes/no names, reassignment also lowers their mean AUC from 93.8% to a below-chance 23.2%. In the binary evaluations, random strings used as option names yield mean flip rates close to those of neutral controls across all three models, with comparable balanced accuracy before reassignment. Together, these results support option-name polarity as a contributor to decision instability beyond reassignment alone. The type-error rate remains 0% throughout, showing that type-correct outputs can still fail to follow explicit option definitions.
Read the full abstract ▾
All authors Yu Sun, Junhao Xu, Jiajia Shi, Zijin Yang
Artificial Intelligence · Cross-listed from cs.CV
FleXray: Universal Clinical X-ray Segmentation
Victor Ion Butoi, Vivek Gopalakrishnan, John V. Guttag et al. (+2 more)
X-ray is medicine's most widely used imaging modality, yet remains among its least quantitative. Unlike volumetric modalities like CT or MRI, X-ray collapses 3D anatomy into a 2D projection, causing structures to overlap and anatomical boundaries to be ambiguous, even to experts. As a result, labeling X-ray databases for training general-purpose
segmentation systems is impractical, leaving morphometric and functional X-ray analysis confined to narrow anatomical regions and applications. To this end, we present FleXray, a generalist model for anatomical
segmentation across the entire body in clinical X-rays. Instead of curating large, manually annotated X-ray
datasets, we build a scalable, physics-based generative X-ray data engine. Using existing 3D whole-body CT
segmentation datasets and generative image-editing models, we simulate fully-annotated 2D X-rays with diverse appearances, physiological properties, and imaging geometries. Trained on these simulations, FleXray accurately segments 60 anatomical structures across unseen research
datasets and in-the-wild X-rays. We further show that FleXray makes X-rays directly amenable to quantitative analysis, enabling automated measurements for disease grading, robust
navigation during X-ray-guided interventions, and data-efficient learning of pathological targets. We release the model, code, a full-body X-ray
segmentation dataset, and a local, easy-to-use browser-based tool at https://flexray.csail.mit.edu .
Read the full abstract ▾
All authors Victor Ion Butoi, Vivek Gopalakrishnan, John V. Guttag, Adrian V. Dalca, Neel Dey
Artificial Intelligence · Cross-listed from cs.SE
Metrics Failure in LLM-Based Code Vulnerability Repair: An Empirical Study and a Change-Aware Screen
Om Nepal, Sushant Aryal, Oluseyi Olukola et al. (+1 more)
Large
language models (LLMs) are increasingly applied to the automated repair of C/C++ security vulnerabilities, and compile rate is a commonly reported proxy for progress: whether the generated patch compiles. We argue that compile rate is a scientifically unreliable metric for single-function vulnerability repair, and we support this with five controlled experiments over 203 vulnerable functions from Big-Vul, three open-source code LLMs (350M to 6.7B parameters), and three prompting strategies. Compile rate (i) barely responds to an intervention that substantially improves the generated code; (ii) is dominated by evaluation-harness and
dataset artifacts rather than model quality, with about 64% of compile failures not attributable to the model, a share that is nearly invariant across models; (iii) shifts by 1.8 to 2.7 times on identical patches under a single compiler-standard flag, with zero regressions; (iv) ranks the three models in the opposite order to reference-similarity metrics; and (v) rewards non-repairs when used as an optimization target, since a compiler-feedback loop raises compile rate while similarity to the human fix falls, with manual inspection finding deletion- and placeholder-style non-repairs among the newly compiling outputs. The natural fallback, whole-function CodeBLEU, also fails: an unchanged copy of the vulnerable input outscores every model. We also examine diff_F1, a change-aware screen that scores only the edited region. It gives exactly zero credit to a no-op and near-zero credit to some, though not all, of the deletion-based gaming patches we observed, while still crediting genuine partial edits, so it may serve as a cheap screen before deeper, execution-based analysis. It is not a repair-quality metric, and we report where it falls short. Our findings argue for change-aware, execution-grounded evaluation of LLM-based vulnerability repair.
Read the full abstract ▾
All authors Om Nepal, Sushant Aryal, Oluseyi Olukola, Nick Rahimi
Artificial Intelligence · Cross-listed from cs.HC
Does AI Save Time on Product Design? A Randomized Controlled Experiment of AI Prompt-to-Design Workflows
Remy Stewart, Olabode Anise, Andrew Hogan et al. (+1 more)
AI tools for digital product design now offer
prompt-to-design capabilities, allowing designers and their non-designer colleagues to create prototypes through conversational workflows with large
language models (LLMs). While these tools promise time savings, experimental evidence in product design remains limited compared with evidence from software engineering. We conducted a randomized controlled trial with 50 product designers and 50 product managers to evaluate prospective time savings from leveraging Figma Make in design work. Participants attempted three standardized design tasks with or without access to Figma Make. Among participants who completed the study tasks, access to Figma Make was associated with approximately 20% shorter completion times, with larger gains among product managers. Our findings suggest that
prompt-to-design tools may enable product managers to further contribute to design work, while the benefits for professional designers may be task dependent.
Read the full abstract ▾
All authors Remy Stewart, Olabode Anise, Andrew Hogan, Augustus Griffin
Artificial Intelligence · Cross-listed from cs.LG
The Sirens' Song: When Proximal Background Context Overshadows Distant Evidence
Xiaoyu Yang, Jie Lu, Wei Duan et al. (+1 more)
Long-context LLMs focus on retrieving distant evidence from extensive context, yet existing work has largely focused on overcoming distance alone. In this work, we identify the Proximity Trap, insufficient
attention to distant evidence often arises less from distance itself than from cumulative competition with abundant, task-irrelevant proximal background. To address the Proximity Trap, we introduce LYRA (Long-context heavY-tailed Relevance
Alignment), a t-distributed directional matching mechanism that reshapes the context
retrieval distribution, directing more
attention mass toward task-relevant evidence, while preserving the relative positional information encoded. Extensive experiments on LongBench-v2, RULER, and LongBench demonstrate consistent improvements across context lengths and task categories. We further introduce ProxBench, a multi-level fine-grained
benchmark for evaluating distant evidence utilization under increasing proximal background interference. Project page: https://xiaoyuyoung.github.io/LYRA/
Read the full abstract ▾
All authors Xiaoyu Yang, Jie Lu, Wei Duan, En Yu