Artificial Intelligence · Cross-listed from cs.HC
Do Assessment Instruments Measure the Same Thing for Humans and LLMs? A Latent Structure Analysis
Alona Strugatski, Licol Zeinfeld and Giora Alexandron
The rapid development and growing deployment of large
language models (LLMs) have made it increasingly important to understand their capabilities. A common approach is to evaluate LLMs using assessment instruments originally designed to measure skills and competencies in humans, such as standardized exams, and to use performance on these instruments as evidence for generalizable claims about LLMs' underlying abilities on the same skills the assessments are intended to measure in humans. However, from a validity perspective, such inferences require that the relationship between observed performance and underlying constructs established for humans also holds for LLMs. In particular, a necessary condition for transferring score interpretations is similarity in the latent structure of responses to the assessment. In this study, we examine whether this condition holds in two educational contexts: high-school chemistry and a quantitative
reasoning section of a university entrance exam. Using a case study design, we compare human response data with responses generated by six
multimodal LLMs. Our analytical approach combines exploratory factor analysis, factor congruence, and resampling to assess latent structure similarity across human learners and LLMs. Across both instruments, we find systematic differences between human and LLM factor structures, showing evidence that the analyzed assessments may not measure the same constructs for humans and LLMs. These findings call into question the validity of evaluation practices that use educational assessments to make claims about AI capabilities.
Read the full abstract ▾
Artificial Intelligence
Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts
Seungyeol Baek, Yoonbyung Chai, Yonghyeon Lee et al. (+2 more)
Human Activity Recognition (HAR) with self-administered wearables, such as at-home rehabilitation and exercise monitoring, often requires reattaching inertial measurement units (IMUs) across sessions. In multi-IMU settings, this can induce independent orientation offsets across body locations, a deployment shift that conventional scalar HAR models do not structurally handle. Existing remedies rely on rotation augmentation, whose robustness depends on sampled transformations, or calibration and orientationnormalization pipelines requiring additional reference-frame assumptions or explicit procedures. We present Truly Rotation-Invariant HAR (TRI-HAR), a rotation-invariant framework that makes robustness to independent per-location IMU orientation offsets a structural model property. TRI-HAR reshapes accelerometer and gyroscope streams into triaxial vectors, applies a shared SO(3)-equivariant backbone and invariant projection to each IMU location, and fuses the resulting invariant features for activity classification. Across four multi-IMU
benchmarks, TRI-HAR preserves macro-F1 under fixed independent per-location SO(3) rotations and outperforms rotation-augmented baselines under this target shift without requiring rotational augmentation.
Read the full abstract ▾
All authors Seungyeol Baek, Yoonbyung Chai, Yonghyeon Lee, Sungjoon Choi, Sungho Suh
Artificial Intelligence
Bias-Corrected Ceilings of Emotion Predictability from Human Label Variation Based on Instance-Level Fano Bounds
Keito Inoshita
Emotion recognition from text keeps improving on
benchmarks, yet whether an accuracy ceiling has been reached is seldom asked with discipline. Our aim is not to pin this ceiling to a single number, but to quantify how far it depends on finite annotation,
estimator choice, annotation noise, and the evaluation protocol, and thereby to discipline how confidently saturation can be claimed. We propose Bias-corrected Affective Ceiling Estimation (BACE), an analysis framework that estimates a bias-corrected ceiling, separates irreducible from reducible error, and disciplines the resulting claims. An anchored Dirichlet-mixture empirical Bayes
estimator, bracketed between plug-in and NSB, recovers the human-consensus distribution; an annotator split, a noise deconvolution, and a fixed claim gate then attribute error without circularity. Methodologically, unconstrained point estimates place reachability anywhere from 0.38 to 1.03, so saturation cannot be decided by any single
estimator. Substantively, the only assertion passing the claim gate is that at least about 33% of a representative classifier's error on GoEmotions is irreducible, with the same pattern recurring on offensiveness and irony.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
EgoGazeLite: On-Device Egocentric Gaze Prediction for Token-Efficient Multimodal LLM Video Input
Matteo Stoiber and Niels Buus Lassen
The use of
multimodal LLMs (MLLMs) for
egocentric video understanding with wearable devices is constrained by the
token budget.
Memory and compute cost scale with the number of visual
tokens, and high-resolution
video quickly becomes expensive to transmit and process at scale. Prior work (GazeLLM) addresses this by cropping the
video around the camera wearer's gaze. This reduces the number of visual
tokens by about tenfold while maintaining or improving the quality of full-resolution descriptions. However, this compression strategy depends on dedicated eye-tracking hardware, which is unavailable on consumer smart glasses. Building a software-only substitute poses a joint constraint: the predictor must be accurate enough to preserve downstream description quality, yet light enough to run on-device, within the power and compute budget of a smartphone. We address this with EgoGazeLite, a lightweight dual-process gaze predictor for
egocentric video. Across two MLLMs, three automated metrics, and two LLM judges, predicted-gaze crops show no significant difference from ground-truth-gaze crops. Equivalence is confirmed in all ten cases. EgoGazeLite achieves this at 15.7M parameters, 6.71 GFLOPs, and runs the full gaze-and-crop pipeline end-to-end in real time (21.6 ms/frame) on consumer accelerator hardware. Together, these results remove the need for eye-tracking hardware for
token-efficient, gaze-conditioned
egocentric video understanding with MLLMs.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy
Qingyao Yang, Runming Yang, He Xiao et al. (+7 more)
While binary quantization theoretically promises extreme compression and acceleration for Large
Language Models (LLMs), existing research often overlooks the necessity of specialized hardware kernels, thus failing to unleash the full acceleration potential due to persistent reliance on expensive floating-point arithmetic or runtime dequantization overheads. To bridge this gap, we propose FluxBin (Flexible LUT-based Ultra-low-bit eXecution with Binary bases), an
algorithm-kernel co-design that synergizes post-training quantization with a highly optimized CUDA kernel. Algorithmically, we introduce Decoupled Row-Column Binary Decomposition to enhance representational capacity while maintaining hardware efficiency, complemented by a Hessian-guided saliency-aware hybrid bases that preserve critical information. At the kernel level, we implement a Lookup Table Building Approach with Scale Fusion to reduce floating-point arithmetic, featuring a Virtual Columnar Mapping that transforms irregular, sparse, and salient
matrices into dense execution. Extensive evaluations demonstrate FluxBin achieves up to $5.92\times$ speedup and $10.19\times$ energy savings across diverse model architectures, delivering comparable accuracy to heavily fine-tuned methods. This effectively enables the deployment of 70B-scale models on one single A100 GPU with a $4\times$
memory reduction. Code is available at https://github.com/nicyyyy/FluxBin.
Read the full abstract ▾
All authors Qingyao Yang, Runming Yang, He Xiao, Wendong Xu, Junyu Chen, Haobo Liu, Chenchen Ding, Ruihan Hu, Yik-Chung Wu, Ngai Wong
Artificial Intelligence
VARM-Bench: Benchmarking Verifiable Structured Reasoning in Chinese Abusive Speech Moderation
Mingyu Yuan, Shengtao Wen, Lingbing Guo et al. (+2 more)
The widespread circulation of abusive online content has increased the need for reliable moderation of Chinese social-media text. Existing Chinese
benchmarks support label classification, fine-grained toxicity categorization, and target-aware extraction, but do not provide a unified representation for deterministically verifying the stated basis of a moderation decision. We introduce VARM-Bench, a
benchmark for field-anchored chain-of-thought rationales in Chinese abusive-
speech moderation. Each instance contains a concise natural-
language rationale with explicit anchors for six decisions: target, target type, target explicitness, author stance, harmfulness label, and fine-grained category. Our deterministic protocol evaluates field correctness, target
alignment, output validity, complete-record agreement, and hidden record errors conditioned on correct final decisions, without relying on an LLM judge. Under a common structured-output protocol, we evaluate
language models across multiple model families using zero-shot prompting, taxonomy guidance, and structured CoT supervision, and analyze lexical-cue sensitivity and field-level errors. Results show that strong label-level performance can conceal substantial errors in complete moderation records. VARM-Bench provides an auditable and reproducible
benchmark for evaluating verifiable moderation rationales in Chinese abusive-
speech moderation.
Read the full abstract ▾
All authors Mingyu Yuan, Shengtao Wen, Lingbing Guo, Zhen Bi, Xiang Chen
Artificial Intelligence
TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation
Md Messal Monem Miah, Adrita Anika, Zhiyuan Yu et al. (+1 more)
Multi-turn jailbreak attacks have emerged as a critical safety threat to LLMs, as harmful objectives are decomposed across a sequence of apparently benign turns to bypass guardrails. Existing defenses lack the
reasoning capacity to identify evolving manipulation patterns, often trading helpfulness for safety by over-refusing benign requests related to sensitive topics. We introduce Trace, a multi-turn defense with trajectory-aware structured
reasoning. Before generating each response, the model identifies manipulation cues from the trajectory, evaluates both the benign and
adversarial interpretations of user intent, assigns a jailbreak score, and commits to an action: Allow, Caution, or Decline. We curate 4k multi-turn
adversarial conversations from five attack frameworks, pair them with 2.4k benign dialogs, and 600 sensitive-but-benign conversations. We train Llama-3.1-8B-Instruct with SFT and GRPO under a multi-component reward that jointly optimizes helpfulness on benign
prompts and robustness against jailbreak attempts. Across seven multi-turn attack
benchmarks, Trace attains an average attack success rate (ASR) of 14.5% against 31.4% for the strongest baseline and 74.9% for the undefended target, while significantly raising the attacker effort required per successful jailbreak. Trace also balances usability and safety, achieving a 93.3% average compliance on over-refusal
benchmarks.
Read the full abstract ▾
All authors Md Messal Monem Miah, Adrita Anika, Zhiyuan Yu, Ruihong Huang
Artificial Intelligence
When Entropy Is Not Enough: Reclaiming Lost Semantics in LLM Output Length Prediction
Feiyang Ren, Shengtao Wen, Yu Tian et al. (+5 more)
Efficient LLM serving is often bottlenecked by the need to pad sequences to a fixed maximum length, and this wastes compute and degrades throughput. Output length prediction enables length-aware scheduling and reduces overhead. Existing methods, including
entropy-guided
token pooling, rely on
token-level
entropy but overlook semantic differences between
tokens, causing them to underweight important
tokens and overemphasize less informative
tokens. We introduce ESTP(
Entropy-and-Semantic
Token Pooling), a lightweight framework that combines
entropy representations with
attention-based semantic representations. By deriving both representations directly during prefill, ESTP captures uncertainty and semantic importance while incurring negligible additional inference latency and
memory overhead compared with methods using external auxiliary models. On the ForeLen
benchmark, ESTP outperforms baseline methods, achieves better prediction accuracy and lower prediction errors across most scenarios. When integrated with a length-aware scheduler in end-to-end system tests, it further improves overall throughput and reduces the padding ratio.
Read the full abstract ▾
All authors Feiyang Ren, Shengtao Wen, Yu Tian, Yuanning Cui, Xiaofeng Cao, Jiangchao Yao, Xu Chen, Xiang Chen
Artificial Intelligence
Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback
Pouya Ghiasnezhad Omran, Michael Zimmermann, Duncan Cambridge et al. (+2 more)
Large
Language Model (LLM)
agents deployed in production environments face a fundamental tension: the
agent's behavior is frozen at deployment time, while the business rules and edge cases it must handle continue to evolve. Existing approaches address
agent construction and one-time evaluation but provide no structured mechanism for continuous post-deployment behavioral correction without modifying the
agent's source code. Most of the approaches offered in the
market, require intense collection of logs and traces, and re-examining the
agent design by the engineering team, a process which is heavy, long and negates the economical value of agentic transformation. We introduce
Agent Gym, a modular, domain-agnostic framework that wraps any existing LLM-based
agent in a continuous evaluation-and-
evolution loop. The framework provides six composable capabilities — Act, Evaluate, Investigate, Correct, Learn, and Observe — organized across three architectural zones: a constitution layer that codifies domain knowledge in configuration artifacts, a runtime inference pipeline that chains acting, investigation, and adaptive correction, and a learning loop that enables subject matter experts to discover and validate new correction rules through natural
language interaction. The key technical contributions include a hybrid deterministic-LLM correction engine with 21 condition operators and three-tier actions, a three-layer investigation architecture for ground-truth-free compliance validation, and a programmatic safety loop that guarantees rule correctness before human approval. We further introduce the Spec-to-Note Gap, an autoencoder-inspired view of agentic system transparency. An open-source reference implementation for invoice processing demonstrates that the framework is fully operational and ready for adoption.
Read the full abstract ▾
All authors Pouya Ghiasnezhad Omran, Michael Zimmermann, Duncan Cambridge, Ashmita Kapoor, Tanya Dixit
Artificial Intelligence · Cross-listed from cs.LG
GraniKV: Asymmetric Granularity KV-Cache Paging for Multi-Agent Systems with Long Shared Prefix
Jinhyun Jeon and Sungjoo Yoo
Production paged-serving engines apply uniform paging granularity to the KV cache, even though the two regions of a multi-
agent workload have opposite storage requirements: a long shared prefix demands contiguity, while the per-request suffix demands fine-grained allocation. We present GraniKV, a KV-cache layer that allocates the shared prefix in a contiguous HOT pool and the suffix in a
token-level COLD pool, combined with a per-step dispatcher which selects the appropriate backend among dual backends for each regime (compute-,
memory-, or communication-bound). To the best of our knowledge, GraniKV is the first system to apply asymmetric paging granularity to the KV cache of a production paged-serving engine. At $L_p{=}16$\,K shared
tokens GraniKV reaches $\mathbf{2.16\times}$, $\mathbf{1.98\times}$, and $\mathbf{1.57\times}$ output-
token throughput over the production baseline on Llama-3.1-8B/TP=1, Qwen-2.5-14B/TP=2, and Qwen-2.5-32B/TP=4. The gain decomposes: cascade
attention integration contributes the majority at saturation; the asymmetric storage layer adds $1.05$–$1.15\times$ end-to-end while being what makes the batched-GEMM prefix backend possible at all. Under heterogeneous multi-
agent serving with distinct
prompts of different lengths, the attribution inverts: GraniKV sustains $\mathbf{1.95\times}$ while batch-global cascade collapses to parity — the storage layer alone carries the win in the regime that motivates the paper.
Read the full abstract ▾
Artificial Intelligence
From Generalist to Specialist: A Context-Fusion Framework for Endoscopic Polyp Reporting with a Frozen VLM
Ruijie Yang, Yan Zhu, Peiyao Fu et al. (+7 more)
Reliable endoscopic polyp reporting requires integrating quantitative lesion sizing, standardized Paris classification, and clinically meaningful morphological description within a single record. General-purpose
vision-
language models (VLMs) offer a unified interface for image understanding and report generation. Existing specialization strategies, however, typically rely on task-specific models or model-weight adaptation, leaving unresolved how to introduce reliable specialist knowledge while preserving both this unified interface and the VLM's pretrained capabilities. We introduce a context-fusion framework that specializes a frozen general-purpose VLM through both implicit instruction context and explicit transduction context without modifying its pretrained weights. Specifically, a self-supervised polyp encoder retrieves related image-report pairs as explicit, query-specific evidence, while learned continuous specialist
tokens provide implicit instruction context shared across cases. Experiments were conducted on 2,056 expert-annotated public endoscopic images. We compared the framework with general-purpose VLMs, task-specific predictors, and weight-adaptation methods to assess specialist performance, unified reporting, and adaptation efficiency. Across numerical, categorical, and report-generation metrics, the proposed framework substantially improved direct frozen-VLM inference and achieved the strongest overall performance among the evaluated methods. It added trainable parameters equal to only 0.006% of the frozen VLM's parameter count. When the top-1 retrieved case carried the correct target category, our framework corrected 70.5% of the errors made by a weight-adaptation baseline. These findings support the context-fusion framework as a lightweight and effective strategy for specialist adaptation of a frozen VLM.
Read the full abstract ▾
All authors Ruijie Yang, Yan Zhu, Peiyao Fu, Siyuan Li, Te Luo, Zhihua Wang, Quanlin Li, Pinghong Zhou, Xian Yang, Shuo Wang
Artificial Intelligence · Cross-listed from cs.SE
Kozuchi Agent: A Language-Agnostic Open-Weight Agent for Software Repair
Mehdi Bahrami, Kosaku Kimura, Satoshi Munakata et al. (+24 more)
Industrial software-engineering teams increasingly need LLM
agents that turn bug reports into correct patches, yet
benchmark-scale operation adds long horizons, tool-use discipline, context persistence, heterogeneous clusters, and evaluation reuse. We present Kozuchi
Agent, a
language-agnostic open-weight repair
agent and CI-operated evaluation pipeline. Explicit phases, persistent state, deterministic tools, a model-independent action interface, and cross-
agent test-time selection make runs auditable and repeatable. With locally hosted Qwen3.5-27B, no fine-tuning, and TTS@8, Kozuchi resolves 374/500 SWE-bench Verified instances on the official evaluator. Unchanged on Multi-SWE-bench Java, the same 27-billion-parameter
agent resolves 41/128 instances (32.03%), ranking first among strict open-weight submissions and fourth of 42 overall; on Python it ranks 12th of 135 and first among open-weight systems. Per-phase behavior remains within +/-5 percentage points across
languages. Remaining failures mainly reflect semantic correctness, Java-specific harness issues, and selection errors. Across both tracks, results compare favorably with open/local peers by parameter count. Analysis of candidate diversity, selector regret, and patch reliability shows that the remaining gap is primarily semantic correctness and selection rather than edit formatting or proprietary-model access. Operationally, reusable CI stages reduce operator touch-points from five to one across heterogeneous internal clusters.
Read the full abstract ▾
All authors Mehdi Bahrami, Kosaku Kimura, Satoshi Munakata, Satoshi Nakashima, Yu Ishikawa, Kosuke Maeda, Nao Soma, Kenichi Kobayashi, Keisuke Miyazaki, Keizo Kato, Shigeki Fukuta, Tatsuo Kumano, Nobutaka Imamura, Kevin Musgrave, Shahbaz Abdul Khader, Kwun Ho Ngan, Joe Townsend, Fayas Asharindavida, Matthieu Parizy, Akira Sakai, Yuma Ichikawa, Yang Zhao, Michiaki Takizawa, Taku Fukui, Hiroki Ohtsuji, Wei-Peng Chen, Hiromichi Kobashi
Artificial Intelligence · Cross-listed from cs.SD
ARENA: Automated Red-Teaming for Large Audio Language Models
Jiaming He, Zhicong Huang, Tian Jin et al. (+5 more)
Large audio-
language models (LALMs) make it possible to interact with
language models through
speech, music, and environmental sound, but they also introduce a safety surface that is difficult to expose with text-only red-teaming. We study automated audio-grounded red-teaming, where a text query must remain safe in isolation while the joint text-audio input induces harmful target behavior. We propose ARENA, a closed-loop framework that trains a controller on an independent 2,000case text-audio
dataset. MD-Judge supplies training rewards and adaptive search feedback, while a separate, non-adaptive Llama Guard 3 evaluator alone labels final outcomes. On 520 held-out AdvBench objectives, ARENA achieves FDR/PSR of 87.9/100.0%, 71.5/96.3%, 68.1/100.0%, and 75.4/98.5% on Audio Flamingo 3, Qwen2-Audio,
MiMo-Audio, and GPTAudio, respectively. Ablations show that feedback-based refinement and audio-variant search substantially improve attack discovery.
Read the full abstract ▾
All authors Jiaming He, Zhicong Huang, Tian Jin, Zhen Sun, Cheng Hong, Yi Yu, Wenbo Jiang, Xudong Jiang
Artificial Intelligence · Cross-listed from cs.CV
Catching Hallucinated Citations in Video-LLM Question Answering: A Self-Verification Pipeline and Verifier Ablation Study
Yogesh Kumar
Video question answering systems built on
vision-
language models often produce timestamped claims with high confidence even when unsupported by the cited frame. This deceptive
hallucination arises because timestamps imply grounding without ensuring correctness, increasing user trust but not accuracy. We introduce a pipeline that closes this loop. A
retrieval-augmented
language model drafts answers with per-claim timestamp citations, and each cited frame is independently re-examined before being shown to the user. We compare against a plain baseline and ablate three verification designs, evaluated on both Apple Silicon (MLX) and Google Colab (HF
Transformers, CUDA). Directly asking the
vision model whether a frame supports a claim fails completely (0% catch rate on 40 claims) due to sycophancy. Blind re-captioning plus a general LLM judge improves results but is unstable, oscillating between 0% and 100% flagged depending on
prompt phrasing. Replacing that judge with a small natural
language inference model yields a stable, interpretable verifier that catches 79% of fabricated claims on
adversarial false-premise questions while leaving true claims untouched. We release the full pipeline, evaluation harness, and implementations for both Apple Silicon and Colab. Code is available at https://github.com/yogesh-iitj/grounded-
video-qa.
Read the full abstract ▾
Artificial Intelligence
Admission Without Answers: Label-Free Certification and Experience Learning for LLM-Based Optimization Modeling
Junbo Jacob Lian, Huiling Chen, Hanzhang Qin et al. (+1 more)
Agents that learn from experience improve at optimization modeling by storing solved trajectories and reusing them as skills. A wrong trajectory that enters the library can be retrieved again and again, and on a stream of new problems there is no ground-truth answer to decide with. Existing learners admit trajectories by matching known optima or labels, and label-free substitutes such as execution success or agreement at one instance can admit wrong models. We introduce ADMITOR, a label-free admission gate. It generates models from three model families, runs each on the stated problem and on instances with resampled parameters, keeps the largest group of models whose optimal values agree on every instance across families, and applies a threshold fitted on solver-verified problems to accept, abstain, or escalate, with a finite-sample bound on the false-discovery rate among accepted values. Inside a state-of-the-art skill learner, ADMITOR raises candidate-level admission precision to 0.927, against 0.871 for majority vote over the host's own samples and 0.726 for execution success, and its library, the smallest of the four, reaches the highest macro accuracy over five public
benchmarks, 58.4 against 54.8 for majority vote. An ablation on the same records shows that the gain comes from the accepted value being external to the learner and unanimous across families; on this stream, resampling never changed an accepted value and only reduced coverage. The false-discovery bound holds on the calibration set but not on the
benchmark stream: an audit of every false certificate traces most of them to
benchmark texts that omit or round the numbers needed to reproduce the labeled answer, and a label-free check of the extracted numbers against the text flags most of these cases.
Read the full abstract ▾
All authors Junbo Jacob Lian, Huiling Chen, Hanzhang Qin, Chung-Piaw Teo
Artificial Intelligence · Cross-listed from cs.LG
Amortised Post-Hoc Explanation with Exact Preservation for Dynamic Graph Anomaly Detectors
Iyad Assaad Nekka, Hamida Seba, Walid Khaled Hidouci et al. (+1 more)
Anomaly detection in dynamic graphs underpins financial fraud analysis, intrusion detection, and platform integrity, where automated decisions require human-interpretable justifications. StrGNN, the strongest performer in recent
benchmarks, produces no explanation: when an edge is flagged, the analyst receives only a score. Explanation metrics are undefined for StrGNN because no attribution vector exists. This paper closes that gap. We present X-StrGNN, a post-hoc explanation layer that wraps a trained, frozen StrGNN and emits, for every flagged edge, dual attributions: a structural attribution identifying which contextual interactions in the enclosing subgraph drove the decision, and a temporal attribution identifying which historical snapshot carried the signal. Both attributions are multiplicative masks identically one in the unexplained pass, so the layer is an exact pass-through: detection is preserved to machine precision, verified rather than asserted (Delta AUC = 0.0000, Delta AP = 0.0000, Delta P@100 = 0.0000). Attribution costs 0.66 ms per edge, making explanation of an entire alarm list feasible. We conduct the first controlled design study of attribution strategies for this architecture, comparing gradient attribution, per-instance mask optimisation, and amortised parameterisation under one protocol, one budget, and three seeds. X-StrGNN attains the highest stability (0.913) at 268x lower cost than per-instance optimisation, and its temporal attribution (1.601 against a measured random floor of 0.973) is separably better than its ablated control, while per-instance optimisation - the most expensive strategy - falls below that floor. Code, protocol, and per-seed measurements are released.
Read the full abstract ▾
All authors Iyad Assaad Nekka, Hamida Seba, Walid Khaled Hidouci, Karima Amrouche
Artificial Intelligence · Cross-listed from cs.RO
MistyPilot: Enabling Social-Robot Control through Multi-Agent LLM Skill Orchestration
Xiao Wang, Lu Dong, Ifeoma Nwogu et al. (+2 more)
Programming small social
robots from natural-
language instructions requires more than invoking isolated APIs. Interactive tasks combine reactive physical behaviors with stateful social behaviors, while existing interfaces often require developers to manually compose APIs into skills, configure their parameters, bind sensor events to skills, and manage task states at runtime. We present MistyPilot, a multi-
agent LLM framework that interprets high-level natural-
language instructions and orchestrates the corresponding skills on the Misty social
robot. A Task Router dispatches each instruction to one of two specialized
agents: a Physically Interactive
Agent for sensor-triggered
robot control and direct skill invocation, and a Social Interaction
Agent for dialogue-oriented task-state management and context-dependent
multimodal response generation. To improve efficiency, the Social Interaction
Agent reuses previously generated results when applicable and invokes full generation otherwise. We evaluate MistyPilot on five component-level suites, with sensor bindings and skill invocations executed on the physical Misty
robot, and a preliminary user study with 12 participants. MistyPilot attains high accuracy on routing, sensor-skill binding, task-state parsing, result reuse, and skill extension up to 100 skills, and lower variance than an otherwise identical single-
agent baseline, while participants report positive perceptions of usability and interaction quality. The code will be made publicly available via the project page.
Read the full abstract ▾
All authors Xiao Wang, Lu Dong, Ifeoma Nwogu, Srirangaraj Setlur, Venu Govindaraju
Artificial Intelligence · Cross-listed from cs.LG
Spectral Saliency for Machine Unlearning
Cedar Site Bai, Amber Yijia Zheng, Raymond A. Yeh et al. (+1 more)
Machine unlearning (MU) aims to remove the influence of specific training data while preserving model utility. As the name suggests, MU can be viewed as the inverse of learning, using gradient-based updates to reduce the influence of a forget-set by counteracting the previously learned behavior. Recently,
Muon, a gradient descent variant, has been introduced.
Muon applies spectral magnitude normalization to encourage exploration of rare directions and demonstrates promising performance. Inspired by
Muon, we adopt the spectral view for unlearning and propose Spectral Saliency Unlearning (SSU). SSU thresholds weak singular components and updates only those directions supported by a confident unlearning signal. We further provide theoretical justification for this thresholding approach from the perspective of the forgetting-retention trade-off. Experiments across image classifiers,
diffusion models, and LLMs demonstrate SSU's effectiveness.
Read the full abstract ▾
All authors Cedar Site Bai, Amber Yijia Zheng, Raymond A. Yeh, Brian Bullins
Artificial Intelligence
ATLAS: Scaffold-Free Algorithm Synthesis by LLMs via Embedding-Guided Quality-Diversity Search
Danial Yazdani, Mohammad Nabi Omidvar, Yuan Sun et al. (+2 more)
Most LLM-based automated
algorithm design methods optimize a designated component within a human-specified scaffold, fixing overall organization and component interactions. We present ATLAS, an
embedding-guided quality-diversity framework for scaffold-free full-
algorithm synthesis in combinatorial optimization. The problem specification supplies objectives and constraints; a minimal I/O interface fixes only instance and solution formats; the LLM chooses and restructures components, interactions, and control flow. This freedom enlarges the search space, risking invalid candidates and premature convergence to one design region. ATLAS independently detects execution, interface, and feasibility failures, recomputes objectives, and applies error-conditioned repair; similarity-based archive management preserves
algorithms across
embedding-space regions to counter premature convergence. Its three-layer search refines the best design, gives other regions dedicated refinement opportunities, and performs cross-region synthesis to recombine components and their interactions. Across four NP-hard problems, ATLAS outperforms several state-of-the-art component-synthesis methods and a matched full-synthesis baseline while remaining competitive with strong human-designed
algorithms. One ATLAS run retains several
algorithms with comparable performance from distinct
embedding-space regions rather than a single design. Code inspection finds that these multi-component designs differ in their primary construction or global-search backbone. Our results suggest that
embedding-guided quality-diversity search can make the enlarged full-
algorithm design space practically searchable. Source code and exact executable
prompts are available at https://github.com/Danial-Yazdani/ATLAS .
Read the full abstract ▾
All authors Danial Yazdani, Mohammad Nabi Omidvar, Yuan Sun, Maksud Ibrahimov, Xiaodong Li
Artificial Intelligence · Cross-listed from cs.CV
EA-LiteUNet: An Edge-Adaptive and Resource-Efficient U-Net for Boundary-Sensitive Dermoscopic Image Segmentation
Wang Jiangtao, Nur Intan Raihana Ruhaiyem, Fu Panpan et al. (+2 more)
Accurate boundary delineation remains a persistent challenge in dermoscopic image
segmentation because of blurred lesion margins, heterogeneous textures, and complex background artifacts. From a signal-processing perspective, lesion boundaries represent high-frequency components that are highly susceptible to aliasing, noise amplification, and information loss. Consequently, repeated downsampling and feature transformations in conventional convolutional architectures often lead to severely degraded boundary representations. To address these limitations, we propose EA-LiteUNet, an edge-adaptive and computationally efficient U-Net variant specifically designed for boundary-sensitive medical image
segmentation. The architecture integrates three core mechanisms: (1) boundary-aware representation learning to suppress aliasing and preserve high-frequency structural details; (2)
attention-guided feature modulation to selectively enhance boundary-relevant responses across multi-scale features; and (3) a resource-adaptive inference strategy to dynamically balance
segmentation accuracy and computational efficiency. Extensive evaluations across three public dermoscopic
datasets demonstrate that EA-LiteUNet consistently achieves superior boundary precision. Specifically, on the ISIC 2018
dataset, the method significantly reduces the 95% Hausdorff Distance (HD95) to 12.89 pixels while maintaining a robust Dice score of 92.08%. Notably, this strong performance is achieved with an ultralightweight configuration of merely 0.29M parameters and 1.17 GFLOPs. Ablation studies further validate the complementary effects of these components, confirming their contribution to enhanced boundary fidelity and stable optimization.
Read the full abstract ▾
All authors Wang Jiangtao, Nur Intan Raihana Ruhaiyem, Fu Panpan, Yang Yu, Huang Yan
Artificial Intelligence
From Contexts to Values: Context-Dependent Defeat in Abstract Argumentation
Albert Sadowski and Jarosław A. Chudziak
In value-based argumentation, an audience's ordering of values decides which attacks succeed as defeats. In many settings the deciding factor is not the audience but the circumstances: the same attack may succeed at one procedural stage, or under one regulation, and fail at another. Context-dependent argumentation frameworks (CDAFs), a model we recently introduced, capture this directly: one attack relation and a defeat function that switches each attack on or off per context, so every context induces an ordinary Dung framework. This raises a reduction question: can one value assignment with per-context orderings reproduce the defeat function, collapsing the CDAF into a VAF? We present a
polynomial-time decision procedure for this question and map the harder neighbouring problems, with upper bounds from NP to $\Sigma^p_3$. We also present a validated reference implementation and a measurement: representability is rare and falls fast with the number of contexts.
Read the full abstract ▾
Artificial Intelligence
Who Leads Now? Token-Level Modality Arbitration for Chart-to-Code Generation
Qinghao Fu, Yarong Wang, Shunlei Ning et al. (+6 more)
Chart-to-code generation requires a model to read the fine-grained visual details of a chart and write executable code that reproduces it. Existing chart-to-code methods either train visual and coding abilities separately, or fine-tune on chart-to-code data with the two abilities entangled. Neither strategy accounts for the distinct nature of the two abilities or the interference that arises when they are optimized together. We propose MoCA (Mixture of Cross-modal Arbitration), which separates the two abilities rather than blending them. MoCA is built on Cross-modal Arbitration Block (CAB), which maintains a visual branch and a code branch as two distinct pathways, and a lightweight arbiter that arbitrates their relative contributions at every layer and generated
token. We train MoCA in two stages: a supervised warm-up on self-distilled
reasoning trajectories that decomposes visual understanding into explicit steps, followed by reinforcement learning with rewards on both the
reasoning process and the final code. Analysis shows that the arbiter learns structured rather than arbitrary allocations, with expert contributions varying systematically across
tokens, layers, and instances. Across three
benchmarks, MoCA delivers competitive performance against general-domain and chart-specialized models. Ablation results show that the gains cannot be attributed to a larger model size alone, but instead arise from the joint contributions of complementary visual and code branch initialization and input-conditioned arbitration through CAB.
Read the full abstract ▾
All authors Qinghao Fu, Yarong Wang, Shunlei Ning, Yilin Wang, Shunwen Bai, Xinda Wang, Jiaotuan Wang, Yinan Nie, Wei Zhou
Artificial Intelligence
EcoVLA: Energy-Efficient Device-Edge Co-Inference for Vision-Language-Action Models under Real-Time Constraints
Ao Zhou, Bo Dai, Le Yu et al. (+5 more)
Vision-
Language-Action (VLA) models have emerged as a promising foundation for
Embodied AI, but their high inference cost poses significant challenges for deployment in robotic systems. In practice, on-device inference is constrained by limited compute capacity and energy budgets, struggling to simultaneously satisfy real-time control and energy efficiency requirements. Alternatively, offloading the inference workload to an edge server is susceptible to fluctuations in system conditions, introducing unpredictable latency risks. Device-edge co-inference offers a promising solution, but systematic research tailored to VLA models remains scarce, particularly a unified co-inference framework that jointly addresses real-time constraints and system-level energy efficiency. Thus, we propose EcoVLA, an adaptive device-edge co-inference framework for VLA models that maximizes system energy efficiency under real-time constraints. EcoVLA first introduces a unified stage-level abstraction over different VLA paradigms, establishing an architecture-agnostic co-inference design space. It then formulates a joint device-edge-network latency and energy prediction model to enable rapid runtime evaluation of candidate co-inference schemes. Building on this, EcoVLA continuously selects the energy-optimal scheme satisfying real-time constraints with millisecond-level overhead, adapting to runtime variations in network and system states. Furthermore, EcoVLA incorporates a lightweight transmission mechanism for inter-stage intermediate tensors to reduce the communication overhead incurred by cross-device collaboration. Experimental results across VLA models show that EcoVLA improves system energy efficiency by up to 236% over existing co-inference approaches under a 20 Hz action output frequency constraint, while consistently maintaining SLO satisfaction under dynamic network and edge workload conditions.
Read the full abstract ▾
All authors Ao Zhou, Bo Dai, Le Yu, Xingyu Liu, Zeyu Hao, Lingkun Long, Chunming Hu, Jianlei Yang
Artificial Intelligence
A Network-driven Framework for Public Event Forecasting via Dynamic Interaction Network Evolution
Jie Wei, Yue Liu, Xiaochuan Tang et al. (+3 more)
Effective public event
forecasting is essential for intelligent service systems, enabling proactive risk management, adaptive resource allocation, and timely decision-making. In many real-world scenarios, the
evolution of public events is driven by dynamic interactions among participants. Motivated by this observation, this paper proposes auto-ibDLM, a network-driven deep learning framework that represents events as dynamic interaction networks and predicts public event
evolution through participant growth
forecasting. The proposed framework adopts a hybrid representation learning strategy that first represents network
evolution using network science-informed structural metrics and subsequently transforms the resulting structural feature vectors into compact and robust latent representations through an auto-learning layer. A GRU-based temporal
forecasting module is then employed to capture temporal dependencies and predict future participant growth. Extensive experiments on 13 real-world public event
datasets and two publicly available dynamic network
datasets demonstrate that auto-ibDLM consistently outperforms representative state-of-the-art methods in both
forecasting accuracy and generalization capability, achieving over 97% accuracy in public event
forecasting. Comprehensive experimental analyses further validate the effectiveness of the proposed hybrid representation learning strategy and demonstrate its representation-level interpretability. These results indicate that auto-ibDLM provides an effective and practical solution for intelligent public event
forecasting.
Read the full abstract ▾
All authors Jie Wei, Yue Liu, Xiaochuan Tang, Biao Cai, Xiangtao Li, Yanmei Hu
Artificial Intelligence · Cross-listed from cs.CR
Bit-Flip Attacks on Vision-Language-Action Models: Action-Decoding Architecture Shapes the Vulnerability
Yudong Gao, Linghan Chen, Wenhan Wu et al. (+5 more)
Quantized
Vision-
Language-Action (VLA) models expose a weight-fault surface: Rowhammer-style faults can corrupt deployed INT8 bits. We present the first bit-flip attack on a VLA: a few gradient-selected flips reduce closed-loop success to $0\%$, while hundreds of random flips are harmless. Across four model variants spanning three action-head families, damaging bits concentrate in a few action-generating layers, but the empirical budget depends sharply on the head: direct regression and
token policies fall in $1$–$5$ flips, whereas the evaluated flow-matching policies require ${\sim}100$–$300$. Our fixed-direction
manifold-escape loss cuts \pizero{}'s budget from ${\sim}1000$ to ${\sim}100$ flips, and a matched five-direction sweep shows that the attack is not specific to an all-positive direction. On a direct head, protecting $3.1\%$ of weights preserves $60\%$ success at $K{=}100$, and protecting $5.3\%$ moves the open-loop break threshold from 3 to 100 flips. Finally, task-calibrated emulated $K{=}100$ flips yield $0/20$ real-
robot successes, versus $14/20$ clean and $16/20$ global-random. Weight integrity is therefore a security boundary for
embodied foundation models. Code is included as ancillary material.
Read the full abstract ▾
All authors Yudong Gao, Linghan Chen, Wenhan Wu, Mia Zhou, Jiyao Wang, Kaiyan Ji, Mingyu Guo, Honglong Chen
Artificial Intelligence · Cross-listed from cs.LG
Optimal Lower Bounds for Networked Information Aggregation
Ambar Pal
The problem of networked information aggregation, studied in Kearns et al. (2026), involves a group of learners situated on the vertices of a directed acyclic graph $G$, each learning a linear predictor $\widehat Y$ for a fixed random variable $Y$ given access to a local feature, as well as the predictors learnt by its parents. Learning proceeds iteratively, with learners ordered according to a topological sort of $G$. The main quantity of interest is the error incurred by the current learner, constrained to this flow of information, with respect to the best linear predictor using all the features seen so far. When the studied error is the MSE, i.e., $\mathbb{E} (\widehat Y - Y)^2$, Kearns et al. (2026) show that the error is at most $O(1/\sqrt{D})$ along a path of length $D$. They also obtain a hard instance where the MSE is lower bounded by $\Omega(1/D)$, leaving the correct order open. In this work, we resolve this central open problem, and obtain a family of worst case problem instances with a MSE lower bound of $\Omega(1/\sqrt{D})$. By exploiting invariances in the structure of the learnt predictors, our analysis generalizes to all convex loss functions $\ell(\widehat Y, Y)$ satisfying regularity conditions which include strong convexity in a ball around the origin, and that the ideal predictor minimizing the population loss is positively correlated with the label. We show that networked information aggregation on a gaussian instance in our worst case family incurs an $\ell$-error lower bounded by $\Omega(1/\sqrt{D})$ with respect to this ideal predictor. We demonstrate that a variety of common losses satisfy these regularity conditions. In particular, the logistic loss satisfies them, and hence our analysis also closes the gap between the upper and lower bounds in Bateni et al. (2026).
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Not All Attention Is Equal: A Quantitative Survey of the EEI Trade-off
Aditya Singh
Attention mechanisms have driven machine learning for a decade, from neural machine translation to
language models that do general-purpose
reasoning. This survey covers four connected threads: their formulation for sequence-to-sequence tasks, adaptation to computer
vision, efficiency innovations that address the quadratic bottleneck, and advances in interpretability. We define three criteria: efficiency, expressiveness, and interpretability, and compare twenty-one methods using an EEI scoring framework. Scores come from a single rater with an assumed +/-1-point perturbation range. A deterministic Monte Carlo analysis with 200,000 samples shows that, under this perturbation model, rank changes of more than one position occur in 67-70% of samples on average. A rank-matched null model reproduces a similar stability profile, so the results support coarse tier-level comparisons rather than fine-grained rankings. The survey traces
attention from Bahdanau-Luong
alignment through the
Transformer and into
vision architectures. It reviews fixed and learned sparse
attention, linear
attention, IO-aware exact
algorithms including FlashAttention, and state-space alternatives including Mamba. It also covers induction heads, superposition, and the
attention-SSM duality. We further provide a structured narrative review, a
benchmark synthesis with cross-study caveats, a five-problem research gap analysis, and a 2015-2026
evolution timeline. We conclude by framing
attention research as an expansion of the efficiency-expressiveness-interpretability frontier and identifying future directions including unified efficiency
benchmarks, learned routing for hybrid architectures, length generalization, and scalable mechanistic interpretability.
Read the full abstract ▾
Artificial Intelligence
Dynamic Multi-Byte Prediction With Hierarchical Language Models
Abraham Toluwase Owodunni, Chibuzor Okocha, Christan Grant et al. (+2 more)
Byte-level hierarchical
language models (LMs) have recently emerged as a robust alternative to their popular counterparts that use subword tokenization. However, generating one byte at a time remains a bottleneck for inference speed. To address this, we introduce multi-byte prediction (MBP), which generates multiple bytes in parallel, speeding up inference with minimal performance impact and no additional parameters. MBP builds on the popular multi-
token prediction (MTP) paradigm with two crucial innovations. First, we introduce a variable-length prediction window that aligns with the latent
tokens, or segments, of a hierarchical LM. Second, we implement a novel
attention-masking scheme that enables parallel byte prediction without violating causality. We show that multi-byte prediction strikes a Pareto-optimal trade-off across multiple generative tasks, instruction following, question answering, summarization, and machine translation, achieving the best trade-off between performance and inference throughput.
Read the full abstract ▾
All authors Abraham Toluwase Owodunni, Chibuzor Okocha, Christan Grant, Tomasz Limisiewicz, Sachin Kumar
Artificial Intelligence
Mental Model Management: An Operator-Based Framework for LLM Memory
Oliver Kramer
Large
language models process large amounts of information but usually lack an explicit mechanism for maintaining compact and evolving conceptual representations. We introduce Mental Model Management (3M), a framework in which knowledge is represented as mental models consisting of compact chunks. Rather than accumulating text passages, 3M continuously integrates new information into an existing conceptual representation. A set of operators extracts knowledge, retrieves relevant models, adds and updates chunks, reorganizes representations, detects inconsistencies, and derives new knowledge. We describe the main 3M operators and illustrate each operation using
Evolution Strategies as a running example.
Read the full abstract ▾