Human-Computer Interaction
LipCoder: Voice-Enabled Coding Toolkit
Hayoon Kim, Sungho Lee, Juhwi Kim et al. (+2 more)
AI-assisted programming environments have accelerated software development, giving rise to new paradigms like vibe coding. However, their benefits remain largely inaccessible to visually impaired programmers, as existing screen readers and assistive tools offer limited support for these emerging workflows. We introduce LipCoder, a voice-centric programming toolkit designed to deliver editor-level functionality through auditory and
speech-based interfaces. LipCoder offers features comprising
speech feedback and earcon cues for comprehension and validation, as well as natural
language input for
navigation and modification. In an exploratory evaluation, 5 visually impaired programmers performed a series of coding tasks comparing LipCoder with a baseline of VSCode, Copilot, and VoiceOver. Quantitative trends and qualitative feedback point to directions for auditory-first design that may broaden accessibility in
speech-driven coding environments.
Read the full abstract ▾
All authors Hayoon Kim, Sungho Lee, Juhwi Kim, Bongwon Suh, Kyogu Lee
Neural and Evolutionary Computing
Conjoint Audio-to-Spikes Encoding and Processing for Efficient Neuromorphic Speech Recognition
Valentin M. Meunier, Amélie Gruel, Pierre Lewden et al. (+2 more)
Obtaining data from neuromorphic sensors and processing it with Spiking Neural Networks is a promising solution to lower the energy cost of artificial intelligence. The current rarity of natively neuromorphic
datasets promotes the development of software tools to translate input sensory data into spikes. However, highly bio-mimetic simulators can be challenging to implement on digital hardware. In this work, we evaluate the neuromorphic encoding and subsequent classification of audio into spikes using a non-learnable, high-level, programmable encoder targeting hardware implementation on FPGA. We quantify the pipeline's efficiency with hardware-agnostic metrics based on the quantitative spiking activity. Our study focuses on the simultaneous optimisation of encoder and classifier: the first provides efficient and informative data so that the latter achieves a better performance with an overall lower energy cost at learning and inference. This work introduces the first end-to-end neuromorphic spike-encoding and evaluation of the TIMIT
dataset. Our simple feedforward network reaches a classification accuracy of 99.77% on a spike-encoded Heidelberg Digits, overcoming the neuromorphic state of the art on this
benchmark dataset.
Read the full abstract ▾
All authors Valentin M. Meunier, Amélie Gruel, Pierre Lewden, Adrien F. Vincent, Sylvain Saïghi
Computer Vision
Camera trap classification with deep learning under ground truth uncertainty
Leonard Hockerts, Peter S. Stewart, Sarthak Arora et al. (+1 more)
Supervised deep learning methods enable the rapid processing of ecological image data, but depend on a costly annotation process. Consequently, training labels are commonly derived from volunteer citizen science projects. However, disagreement among volunteers introduces uncertainty in the "ground truth" data that are assumed to be correct for model training and validation. Using two
datasets containing camera trap images with associated volunteer and expert classifications, we investigated the effects of training under higher ground truth uncertainty. We observed improved overall test accuracy, particularly for images that were more difficult for volunteers. Species-level accuracy also generally improved, but generalisation to a different
dataset did not. The benefits of ground truth uncertainty were enhanced by pre-training on ImageNet. Pre-training also reduced the number of training epochs required; further reductions in computational cost, but not gains in accuracy, resulted from additional pre-training on other camera trap images. With unbalanced training data, we still observed a clear benefit of increased ground truth uncertainty for overall accuracy, especially on difficult images. Class imbalance improved accuracy for common species, reduced rare species accuracy, and changed patterns of misclassification to more closely resemble mistakes made by volunteers. Our findings have implications for applying deep learning across ecological image types with multiple labels. Practitioners can improve accuracy, especially on difficult examples, by including moderate levels of label disagreement during training and using models pre-trained on general image data. In addition to improving the use of citizen science-derived labels in model training, our study suggests avenues for more effectively integrating human and deep learning classifications in combined workflows. (abridged)
Read the full abstract ▾
All authors Leonard Hockerts, Peter S. Stewart, Sarthak Arora, Tiffany J. Vlaar
Numerical Analysis
Model reduction of port-Hamiltonian systems via neural networks
Silke Glas, Alexander Heinlein, Harald Monsuur et al. (+1 more)
In this paper, we consider structure-preserving model reduction of port-
Hamiltonian (pH) systems which extend classical
Hamiltonian systems with dissipation and an input-output port. These pH systems are often used in multi-physics systems, as the interconnection of one or more \pH systems results again in a pH system. If particularly the system
matrices associated with the interconnection and/or dissipation of a pH system are state-dependent, then the evaluation of standard reduced-order models (ROMs) may depend on the dimension of the original full-order model, resulting in high computational costs. To circumvent these high costs, we propose to use structure-preserving neural networks. In particular, we perform two steps: (1) we use the generalized
manifold Galerkin projection to project the pH system onto the reduced space; then (2) we train a neural network to learn the map from the reduced-order state to the reduced-order interconnection and dissipation system
matrices. To ensure that the resulting ROM is again a pH system, the architecture of the neural network is chosen such that the skew-
symmetry and positive semi-definiteness of the reduced-order systems
matrices are maintained. In a numerical example, we consider a nonlinear mass-spring-damper system with state-dependent system
matrices. The numerical results show that the proposed method achieves a significant computational speed-up compared to the original \ROM with comparable accuracy.
Read the full abstract ▾
All authors Silke Glas, Alexander Heinlein, Harald Monsuur, Hongliang Mu
Artificial Intelligence
SkillZip Pro: Execution-Aware Dynamic Compression of Progressively Loaded Skills for Self-Evolving Agents
Xiaofan Bai, Chao Liu, Hongqiang Lin et al. (+5 more)
Production
agent skills are directory bundles, not isolated
prompts. The root is loaded at activation; references, schemas, scripts, assets, and nested subskills are loaded only when an execution path needs them. Compressing only the root misses most deployment cost and may move branch-specific details into the always-loaded context. Flattening instead destroys progressive-loading boundaries. We introduce \method, an evaluation-free compressor for complete, progressively loaded skill bundles. It leaves the
agent harness unchanged and emits an ordinary directory. The method combines two safeguards. First, it compresses across files, removing content from a reference or subskill when the root or a declared environment contract already provides it. Second, it preserves routing, so every required file and directly callable entry remains reachable after rewriting. Users can configure \method along two independent axes. One-Shot mode rebuilds the full bundle; Continual mode reuses state and applies Zip-on-Write after each
evolution patch. Persistent compression rewrites the shipped bundle to reduce storage and runtime context. Transient compression keeps that bundle byte-identical and builds a task-specific view, reducing only per-run context after build cost. Entry contracts mark private, public, and conditional resources; a multi-entry audit preserves standalone public subskills. On a production content-moderation skill evaluated by our industrial multi-round harness, \method removes \hl{38%} of skill bundle
tokens and \hl{10.4%} of end-to-end per-run
tokens with no quality loss, while an unprotected 71% configuration loses up to 26 accuracy points to one-sided false positives. On a multi-entry bundle, \method effeciently reduces
token cost while near-perfectly preserving every route and public entry.
Read the full abstract ▾
All authors Xiaofan Bai, Chao Liu, Hongqiang Lin, Di Wu, Mingli Song, Xuan Jin, Xipeng Cao, Yuhong Li
Computer Vision
PixelIR: Fidelity-Perception Decoupling via Pixel-Space Image-Residual Flow Matching for Efficient One-Step Real-World Super-Resolution
Bingtian Qiao, Yue Shi, Yong Guo et al. (+2 more)
Real-world image super-resolution (Real-ISR) aims to preserve structures supported by the degraded observation while reconstructing perceptually realistic details. However, existing Real-ISR methods largely optimize fidelity and perceptual quality within a shared network, causing the two objectives to interfere throughout training and making their balance difficult to control. Recent one-step methods reduce sampling steps, yet often inherit both this coupled optimization behavior and the expensive high-resolution backbone of their multi-step predecessors. We argue that efficient Real-ISR requires not only a shorter sampling trajectory, but also specialized modeling of faithful reconstruction and perceptual detail synthesis. Based on this insight, we propose PixelIR, a fidelity-perception decoupling framework built upon pixel-space image-residual flow matching. PixelIR first learns an image flow that maps the degraded observation to a faithful reconstruction. Then, a residual flow synthesizes the missing perceptual details from noise without repeatedly relearning or overwriting the complete restoration solution. We further distill the teacher into a deployment-oriented one-step student within a coarse-to-fine pyramid architecture. Extensive experiments show that PixelIR achieves leading PSNR, SSIM, and LPIPS on both RealSR and DRealSR. The final model completes pixel-space restoration in a single evaluation with only 32.9M parameters, 89.7G MACs, and 8.5ms latency, demonstrating a strong practical fidelity-perception-efficiency balance.
Read the full abstract ▾
All authors Bingtian Qiao, Yue Shi, Yong Guo, Wenjun Zhang, Jiezhang Cao
Machine Learning
Reciprocity Separates Gradient Flow from Rotation in Conservative Physical Learning
Ruiwu Niu, Xiaowen Bi and Michaël Antonie van Wyk
Physical learning lets a trainable material or network use its own physical response to carry error signals, reducing the need for a separately programmed backward computation. We ask what determines whether such a system follows conventional gradient descent or evolves along a genuinely different learning trajectory. Our canonical model is a directed layered transport network in which every node redistributes a fixed amount of flow, so learning preserves positivity and total mass. In this model, conservation constrains only the allowable learning directions. Within the matched response class studied here, adjoint matching gives the physical output response a symmetric form. Non-negative mode-wise feedback then produces a reciprocal closed-loop response and a reweighted gradient flow. Adding an antisymmetric boundary component makes the closed-loop response rotational: the learning path can turn while the error driving that update still decreases at that moment. Turning is not automatically beneficial. Its finite-step effect is set by local
curvature, and its accumulated effect also depends on step selection and on the new states visited along the path. Numerical consistency checks reproduce the exact response structure, predict the sign of the local effect across new network families, and show how trajectory drift can negate a local advantage. These results separate the roles of conservation, reciprocity, and nonreciprocity in physical learning.
Read the full abstract ▾
Robotics
Learning to infer and manipulate through distributed whole-arm interaction in a soft robot
Chuhan Zhang, Ebrahim Shahabi, Kseniia Khomenko et al. (+2 more)
In animals such as elephants and octopuses, acquiring non-visual information about an object and physically engaging with it are inseparable processes mediated by rich, large-area interactions between compliant appendages and the environment. Soft
robots provide a natural platform for translating this principle into engineered systems. Yet current robotic intelligence makes limited use of physical interaction, treating it primarily as a disturbance to be rejected or, at best, as a means of compensating for object misalignment. Here, we introduce a physical intelligence framework in which distributed compliant interactions jointly reveal task-relevant information and organize manipulation behavior. This results in an intrinsically partially observable problem: key task-relevant information is never measured directly, but must instead be inferred from the history of physical interactions. We propose a reinforcement-learning architecture that addresses this challenge by learning a
memory-based control policy end-to-end. The key innovations making this possible are (i) a pretrained exploration policy that provides a reference for broad workspace exploration, (ii) joint optimization that integrates exploration and grasping objectives within a single recurrent policy, and (iii) a two-stage sim-to-real adaptation including observation mapping and policy fine-tuning. We demonstrate this principle through blind whole-arm grasping with a hybrid rigid-soft robotic arm that we equip with IMUs embedded directly within its compliant structure, providing its only source of proprioceptive sensing. The learned policy successfully identifies and grasps various objects by autonomously coordinating workspace exploration, object encounter and localization, inference of grasp-relevant properties, and stable whole-arm wrapping.
Read the full abstract ▾
All authors Chuhan Zhang, Ebrahim Shahabi, Kseniia Khomenko, Wei Pan, Cosimo Della Santina
Numerical Analysis
Integral Invariants of Vasudeva Murthy's Relaxation Systems: Analysis and Numerical Validation
Sudipta Sahu and Rathan Samala
Vasudeva Murthy's relaxation approach [A.S. Vasudeva Murthy, J. Comput. Appl. Math., 203(2), pp. 437-443, 2007], originally proposed for the Jin-Xin relaxation model, provides an alternative formulation with invariant properties that is consistent and retains the semilinear structure incomparison to the standard one. In this work, Vasudeva Murthy's relaxation approach for various relaxation systems are proposed such as the shallow water equations, the Broadwell model, the Euler equations with heat transfer and two-dimensional Jin-Xin model. For proposed relaxation models, the associated integral invariants are rigorously established at the theoretical level. The main advantage of the integral invariant is that it provides a conserved quantity for the relaxation system by incorporating the coupled contributions of the solution variables in vector form. To validate the analytical results, numerical simulations are carried out for each model using three second-order numerical schemes: CS-EBT2, a semi-implicit second-order central finite-volume scheme for
hyperbolic systems with relaxation source terms [S. Sahu, E. Macca, and R. Samala, J. Comput. Phys., 563, 115100, 2026]; UCS2, a finite-volume central relaxation-type scheme [S. F. Liotta, V. Romano, and G. Russo, SIAM J. Numer. Anal., 38(4), 1337-1356, 2000]; and IMEX-RK2, a second-order Implicit-Explicit Runge-Kutta scheme [Pareschi and Russo, J. Sci. Comput., 25, 129-155, 2005]. Numerical results, compared to exact or finely resolved reference solutions, confirm that the models preserve integral invariants, remain stable under CFL restrictions, and exhibit robust and accurate behavior across all
benchmark systems tested.
Read the full abstract ▾
Machine Learning
TrainSDC: Characterizing and Mitigating Silent Data Corruption in Large Language Model Training
Zhipeng Xia, Haotian Xu, Siyu Yun et al. (+4 more)
LLM training is increasingly vulnerable to silent data corruption (SDC), yet existing protection methods largely treat
Transformer computations uniformly because their vulnerability remains poorly understood. We present the first systematic characterization of SDC vulnerability across major computation interfaces in both the forward and backward passes of
Transformer training. Our analysis reveals two distinct error propagation mechanisms: forward-pass vulnerability is highly location dependent, with faults on the Q/K path producing persistent training deviations, whereas backward-pass vulnerability is largely governed by gradient exponent distributions rather than computation locations. Motivated by these observations, we propose TrainSDC, a characterization-guided protection framework consisting of Q/K-path recomputation, residual-gain monitoring, and exponent-aware gradient scaling. Experiments on Llama 3.2-1B and Qwen3-0.6B show that TrainSDC maintains training behavior close to fault-free execution under both sparse and dense fault injection while introducing only 1.65%-6.76% runtime overhead.
Read the full abstract ▾
All authors Zhipeng Xia, Haotian Xu, Siyu Yun, Liqi Lin, Hu Liu, Yu Li, Cheng Zhuo
Computer Vision
CORAL: A Benchmark for Structure-aware and Brain-wide Neuron Reconstruction in Light Microscopy
Zekang Yang, Jiamin Li, Zhenghua Li et al. (+3 more)
Automatic
neuron reconstruction from light microscopy images is a central problem in computational neuroanatomy. While recent methods have achieved encouraging results on local image blocks, it remains unclear whether such progress translates to reconstruction that is both structurally accurate and scalable to the whole-
brain scale. We present CORAL, the first
benchmark for structure-aware evaluation of automatic
neuron reconstruction from light microscopy images at both local and whole-
brain scales. Built on a high-quality whole-
brain fMOST
dataset with carefully curated annotations, CORAL establishes two progressive tasks: block-level reconstruction, which evaluates reconstruction methods under limited spatial context, and
brain-wide reconstruction, which assesses complete
neuron reconstruction at the whole-
brain scale. To account for topological correctness beyond
geometric distance similarity, we introduce a structure-aware metric based on fiber prediction. To further achieve complete
neuron reconstruction across the entire
brain, we develop a
brain-wide
neuron tracing framework that extends arbitrary local reconstruction methods to the whole-
brain scale through an iterative local-to-global process. Using this
benchmark, we provide the first structure-aware comparison of mainstream methods for local
neuron reconstruction and further evaluate their performance in
brain-wide reconstruction. Our results underscore the importance of structure-aware evaluation and the need for more robust methods for complete
neuron reconstruction.
Read the full abstract ▾
All authors Zekang Yang, Jiamin Li, Zhenghua Li, Jiaqi Fan, Zengcai Guo, Xiaolin Hu
Machine Learning
T3S: Improving Multi-Task Reinforcement Learning with Task-Specific Feature Selector and Scheduler
Yuanqiang Yu, Tianpei Yang, Yongliang Lv et al. (+2 more)
Multi-task reinforcement learning (MTRL) is a technique to train multiple tasks simultaneously, where previous works usually train a single model to solve different tasks by sharing parameters across various tasks. However, these methods are faced with inter-task interference since what parameters should be shared across tasks is not addressed, dramatically reducing learning efficiency. To solve these problems, we propose a novel MTRL framework called Task-Specific feature Selector and Scheduler (T3S), which consists of two components: a feature selector and a task scheduler. Specifically, the feature selectors employ hypernetworks to construct task-specific soft masks, which can be applied by globally shared representation to construct task-specific features. The task scheduler selects tasks for learning through two metrics, where the selection probability is inversely proportional to task progress (e.g., success rate) and task learning speed. Experimental results show that T3S consistently outperforms the state-of-the-art MTRL
algorithms on various robotics manipulation tasks.
Read the full abstract ▾
All authors Yuanqiang Yu, Tianpei Yang, Yongliang Lv, Yan Zheng, Jianye Hao
Computer Vision
ChessQueries: Toward Better Chess Board Recognition
Joël Seytre
Chess board recognition is the task of mapping the image of a chess board to the information of which piece is on which square. So far this task has two established
benchmarks: ChessCog is synthetic, and ChessReD comes from smartphone pictures of a single chess board setup. We introduce ChessQueries, a new method combining a ViT encoder with a DETR-style decoder, which outperforms existing methods. On the ChessReD
benchmark, we improve the state of the art from 15.3% to 99.2%, and demonstrate strong capabilities on out-of-distribution
datasets. Our method saturates the task on the two
datasets, with an average 0.01 wrong squares per board (vs. SotA: 3.4 / 0.15 respectively). We also share a new, harder public
dataset, parsed from broadcasted top-level chess tournaments. Code, model weights and the SLCC data will be released.
Read the full abstract ▾
Machine Learning
PRACTICE: From Experience to Expertise in Self-Evolving Embodied Agents
Ziyi Bai, Siqi Li, Tinglei Huang et al. (+1 more)
Recent studies have shown that
multimodal large
language models (MLLMs) can serve as
embodied agents, translating
language instructions and visual observations into executable plans. However, building
agents that can continually improve through interaction and rapidly adapt to their environments remains challenging. Summing up experience from past interaction trajectories provides a promising solution, but existing experience-based methods often rely on manually designed prompting workflows to extract and update skills. Such fixed procedures may struggle to learn updated skills from new and diverse experiences. We introduce PRACTICE, which trains a skill learner to discover and maintain a persistent skill library from past interaction trajectories while keeping the task executor frozen. Given the historical accumulated skills and incoming trajectories, the skill learner produces structured batch-edits that add, refine, merge, or remove skills, and then hierarchical consolidate all collected edits into a consistent updated skill library. We train the learner with a two-stage curriculum. First, it learns basic skill generation and library maintenance from oracle trajectories. Then, by contrasting successful and failed trajectories from heterogeneous executors on the same tasks, it learn to identify invalid action patterns and recovery strategies. Finally, we apply online skill-edit
distillation to align the skill learner with a stronger teacher on its current edit distribution to further improves the policy. Experiments demonstrate that a compact skill learner delivers consistent performance improvements across successive library-update rounds for multiple frozen executors. On EB-ALFRED and EB-Habitat, PRACTICE further outperforms the strongest experience-based baselines. Project resources are publicly available at: https://baai-
agents.github.io/PRACTICE
Read the full abstract ▾
All authors Ziyi Bai, Siqi Li, Tinglei Huang, Börje F. Karlsson
Computer Vision
CheXGround: Anatomical Region Tokens for Grounded Longitudinal Chest X-ray Interpretation
Adonay Demewez Gebremedhin, Wessam Shehieb, Sara Alansari et al. (+4 more)
Recent radiology multi-modal
language models have made substantial progress in chest X-ray report generation, visual question answering, and temporal
reasoning. While longitudinal chest X-ray interpretation compares sequential examinations to describe change, visual grounding aims to connect clinical
language with localized image evidence. Although longitudinal modeling and visual grounding have each advanced radiology
language models, how localized visual evidence can support longitudinal interpretation remains under-explored. We introduce CheXGround, a region-grounded longitudinal chest X-ray
language model that represents paired studies through corresponding anatomical regions. CheXGround extracts anatomical regions from current and prior radiographs, encodes them as temporally enhanced Region-of-Interest (ROI)
tokens, and combines them with global temporal image context during generation. To connect these region
tokens with clinical text, we propose Temporal Region–Phrase
Alignment, a pretraining objective that aligns temporal anatomical representations with localized report phrases. We evaluate CheXGround on single-study and longitudinal Visual Question Answering (VQA), longitudinal findings generation, temporal grounded VQA, and anatomical grounding. Across these tasks, CheXGround improves clinical
language quality, temporal
reasoning, and localization accuracy over recent baselines. Our results suggest that organizing longitudinal evidence at the anatomical level is a strong representation for grounded radiology
language modeling. Project page: https://adonaydem.github.io/chexground-website
Read the full abstract ▾
All authors Adonay Demewez Gebremedhin, Wessam Shehieb, Sara Alansari, Mohamad Alansari, Muzammal Naseer, Sajid Javed, Naoufel Werghi
Artificial Intelligence
Which Rules Matter Now? Policy-Centroid Routing Before an Intelligent System Acts
Thomson D. Nguy
Before an intelligent system can decide whether an action is allowed, it must first know which rules the action has approached. A single proposed action can implicate several policy regimes at once. Their requirements may stack, overlap, or qualify one another, yet many remain written in natural
language while the action itself arrives as an incomplete description of intent. The first problem is not judgment. It is
attention. Policy-centroid routing creates a layer before adjudication. It compresses expressions within each policy regime into one or more representative centroids, places the proposed action in the same semantic space, applies a declared measure, and routes every regime crossing a declared threshold to authoritative review. Several regimes may trigger at once. The output is a review agenda, not permission, prohibition, legality, breach, compliance, certification, or enforcement. The paper develops six falsifiable propositions and seven follow-on studies comparing the hypothesis with structured workflows, lexical and semantic
retrieval, hierarchical and direct classification, and selective prediction under matched review burden. The studies are designed to identify where policy geometry recovers applicable regimes, where compression loses rare or overlapping obligations, and where the mechanism should abstain. The paper includes a synthetic worked example and reports no empirical efficacy result.
Read the full abstract ▾
Software Engineering
On the Prospects of Dynamic LLM Conversations in Software Development
Annemarie Wittig, Alina Mailach, Janet Siegmund et al. (+1 more)
Large
language models (LLMs) have become an essential tool for assisting developers, yet we still lack knowledge on ways to effectively support their interactions during development activities. That is, the quality of interactions with a chat-based LLM still strongly depends on how developers phrase
prompts and which information they include. Our goal is to evaluate whether interventions into these interactions with LLMs have an effect on software developers—be it harmful or beneficial. To this end, we conducted a four-month longitudinal study with third-semester computer science students working on a full-stack Web development project using chat-based LLMs under three conditions: (1) a context-aware group received intent-based conversation augmentation, (2) a proactive group received follow-up suggestions and tailored advice, and (3) a control group without intervention. Our augmentations are minimal: (i) to reduce confounding factors and (ii) to isolate treatment effects. Analyzing interaction logs and user surveys revealed no major differences in interaction patterns, indicating no detectable harmful effects in the measured outcomes when intervening in interactions. Moreover, we observed trends of increased satisfaction with the proactive treatment. The results indicate that even with minimal interventions, dynamic guidance mechanisms for developer-LLM interactions show observable effects, such that more severe augmentations may have the potential to substantially improve developer satisfaction.
Read the full abstract ▾
All authors Annemarie Wittig, Alina Mailach, Janet Siegmund, Norbert Siegmund
Computation and Language
CLIN: an Objective Framework for Evaluating Creativity in Short Persian Literary Text
Mohammad Reza Modarres, Armin Tourajmehr, Yadollah Yaghoobzadeh et al. (+1 more)
Evaluating creativity in large
language model (LLM) outputs remains challenging because creativity is multidimensional and human-centered. We examine how reliably LLMs evaluate short literary text in Persian, a low-resource
language, across multiple evaluation strategies and
prompt formulations. We find that LLM-human agreement varies substantially across dimensions:
alignment is stronger for structured TTCT-derived properties such as Originality, Fluency, and Elaboration, but considerably weaker for more subjective dimensions, particularly
Emotion and Attractiveness. Judgments are also sensitive to
prompt formulation, while few-shot prompting, ensembling, and multi-
agent debate provide no consistent improvement. Motivated by this dimension-dependent behavior, we investigate whether structured creativity dimensions can instead be approximated using simple, interpretable proxy metrics. We introduce CLIN, which evaluates three TTCT-derived dimensions separately using topic-aware novelty for Originality, contextual lexical clustering for Fluency, and lexical diversity for Elaboration. These proxies achieve human
alignment comparable to or better than the strongest zero-shot LLM judge in our setting while requiring substantially lower evaluation cost.
Read the full abstract ▾
All authors Mohammad Reza Modarres, Armin Tourajmehr, Yadollah Yaghoobzadeh, Mohammad Taher Pilehvar
Information Retrieval
Learning from What You Retrieve: Online RL Fine-Tuning for Semantic Retrieval
Shaowei Wei, Chong Huang, Songtao Fang et al. (+3 more)
In large-scale e-commerce
retrieval, dual-encoder retrievers are op- timized for contrastive similarity, whereas downstream rerankers capture finer-grained relevance preferences; this objective mis- match limits end-to-end
retrieval quality. Reinforcement Learning offers a way to use reward-model feedback for retriever adaptation, but we observe that standard policy-gradient updates can degrade
embedding geometry, especially when the document index must remain frozen due to industrial constraints. To address this, we propose PAO (Positive-Advantage-Only), a selective RL optimization method. Our analysis reveals that in- discriminate penalization of negative samples (pushing away) in a frozen high-dimensional space disrupts pre-trained semantic man- ifolds. PAO selectively applies gradient updates only to retrieved items with positive advantages, effectively pulling query embed- dings toward high-reward regions while preserving global topo- logical stability. Experiments on both a massive industrial
dataset and public
benchmarks demonstrate that PAO significantly outper- forms standard RL and
distillation baselines.
Read the full abstract ▾
All authors Shaowei Wei, Chong Huang, Songtao Fang, Jin Zhang, Zhuojun Wang, Chengfu Huo
Artificial Intelligence
Autoregressive Mosaics: Probing 2D Spatial Reasoning in Text-Only Language Models
Ashwin Nedungadi, Stefan Oehmcke and Stefan Lüdtke
Large
language models (LLMs) trained only on text and code can sometimes generate programs that draw recognizable images. However, it is unclear whether this reflects an internal representation of 2D spatial layout or simply the ability to translate spatial descriptions into code. We introduce Autoregressive Mosaics (AM-Bench), a
benchmark that separates these factors: First, a translation task gives a model a fully specified geometry of a picture in words as a
prompt and asks for the code that produces it. Second, a layout task requires the model to compose an image from an underspecified
prompt. Across eight open-weight text-and-code-only models, all models reliably translate specified geometry into code, but their open-ended layout performance differs substantially, indicating that these differences are not explained by code-generation ability alone. An output-medium ablation further shows that the interface or medium of expression that the model uses matters: replacing procedural code with raw SVG improves layout scores across all models. Finally, probing model activations shows that a coarse layout plan is present before generation, but reflects only the layout implied by the
prompt. During generation, models track the evolving
geometric state instead of executing an initially fixed plan. Overall, these results show that 2D spatial performance in text-only LLMs depends on both the model and the output medium, and is not explained by code-generation ability alone.
Read the full abstract ▾
Machine Learning
Do VLMs Share Safety Neurons Across Modalities?
Jiaxuan Li, Jiahao Zhang, Duc Minh Vo et al. (+3 more)
Vision-
language models (VLMs) can comply with harmful requests delivered through images, even when their LLM backbones would refuse the same content in text. While prior work characterizes these jailbreaks empirically or at the representation level, how visual inputs perturb safety pathways at the
neuron level remains uncharted. We close this gap with a causal,
neuron-level analysis of safety mechanisms in 10 VLMs. We propose a two-stage detection pipeline with iterative ablation that accounts for self-repair, and introduce two modality-isolated
benchmarks, ViSafe-Detect and ViSafe-Eval, which decouple visual and textual safety signals. Our analysis reveals: (i) Text safety in VLMs is localizable: $\sim$88
neurons ($<$0.01%) whose targeted ablation substantially reduces refusal. (ii) Text safety
neurons constitute the dominant refusal pathway: ablating them is the only intervention that consistently and substantially reduces refusal across all models. (iii) Visual safety is high-dimensional and diffuse at the single-
neuron level: text safety concentrates in $\sim$5 subspace directions while visual safety requires $\geq$50. This gap holds across architectures, explaining why current
alignment has not closed the visual safety gap. Project page is at: https://jiaxuan-li.github.io/vlm-safety-
neuron/ Warning: this paper may include examples of harmful content.
Read the full abstract ▾
All authors Jiaxuan Li, Jiahao Zhang, Duc Minh Vo, Huy H. Nguyen, Pride Kavumba, Koki Wataoka
Cryptography and Security
The Fragility of Jailbreak Robustness Across Operational States
Yuna Park, Hwang Youn Kim, Yujin Kim et al. (+3 more)
Existing jailbreak evaluations typically characterize robustness using a single attack success rate (ASR) measured in a default configuration (the vanilla state). However, user-LLM interactions can induce diverse operational states beyond the vanilla state. In this work, we find that jailbreak robustness is highly fragile to operational-state variation: even when the attack remains fixed, changing only an ordinary system
prompt not designed to affect safety can dramatically alter attack success rates. We systematically investigate this phenomenon across seven aligned models and three representative jailbreak attacks, observing substantial differences in ASR between vanilla and non-vanilla operational states. In one case, ASR increases by up to 56 percentage points (2% to 58%) solely due to a change in operational state. Remarkably, these increases occur even for attacks originally designed and optimized under vanilla-state evaluation. We further show that state-dependent robustness variation is systematically associated with differences in hidden representations along a refusal-related axis, and that projections onto this axis strongly predict jailbreak outcomes. Our results show that a single vanilla-state evaluation may not fully characterize jailbreak robustness, motivating evaluations that also examine how robustness changes across non-vanilla operational states.
Read the full abstract ▾
All authors Yuna Park, Hwang Youn Kim, Yujin Kim, Won Woo Ro, Suhyun Kim, Jae-In Hwang
Machine Learning
TDDM-Melatt: A Decoupled Memory and Diffusion Framework for Generalizable Encrypted Traffic Classification
Ze Chen, Qiming Yu, Zijia Song et al. (+2 more)
The widespread adoption of encrypted traffic poses severe challenges to current security situational awareness systems based on network traffic monitoring. In existing
dataset-driven training and testing studies, limitations such as shortcut learning induced by spurious feature correlations and sample imbalance caused by the long-tail distribution of real-world traffic result in weak generalization of traffic identification performance to real-world network traffic. To address these limitations, we propose TDDM-Melatt, a disentangled
memory-based traffic classification framework with
diffusion-based data augmentation. First, we design Melatt, a
memory-decoupled traffic representation model, which employs Competitive Gating Long Short-Term
Memory (CG-LSTM) to construct the encoder and decoder. We design a spurious-correlation-free pre-training and inference paradigm, employing strict
topology anonymization and a frozen pre-trained encoder strategy to cut off the model's learning pathways for spurious features. During inference, classification is performed efficiently by a downstream classifier on the frozen representations. Second, we propose a Traffic Denoising
Diffusion Model (TDDM) tailored to the characteristics of traffic data. Extensive experiments are conducted on 4 representative public
benchmark datasets. Under strict flow-level splitting and anonymization, TDDM-Melatt outperforms 6 basic classification models and 6 SOTA representation learning models. The proposed method provides a new and effective technical pathway for encrypted traffic classification in real-world network environments.
Read the full abstract ▾
All authors Ze Chen, Qiming Yu, Zijia Song, Guozheng Yang, Wei Yan
Machine Learning
Functional Degeneracy in Neural Networks: Measurement and Pruning
Maria Matveev, Pascal Esser, Ayush Bharadwaj et al. (+2 more)
A central question in modern machine learning is how much a trained model can be compressed without changing its behavior, to reduce the
memory, compute and energy required to deploy it. To study this, we quantify functional degeneracy through the behavioral recovery rank, defined as the number of leading behavioral-Hessian eigendirections required to recover a trained model's performance. Using the behavioral recovery rank as a
geometric benchmark for compression, we find that structural and magnitude
pruning retain more degrees of freedom, even after the task is saturated. This gap suggests that functional redundancy is distributed across parameter directions and is not exposed by individual weights or
neurons.
Read the full abstract ▾
All authors Maria Matveev, Pascal Esser, Ayush Bharadwaj, Lucius Bushnaq, Gitta Kutyniok
Computation and Language
Not All Fallbacks Are Failures: Understanding and Recovering from Fallbacks in Mobile Voice Assistants
Phillip Schneider, Alexandre Mercier, Joshua Oehms et al. (+2 more)
Robust understanding of user input is a core requirement for voice assistants deployed in real-world environments. In practice, these systems encounter heterogeneous fallback situations caused by noisy audio input, transcription errors, ambiguous requests, incomplete utterances, or unintended activations. Existing systems typically respond with generic fallback messages, which do not resolve the underlying interaction failure and can degrade user experience. We study fallback handling in a deployed smartwatch-based voice assistant for general health support in everyday environments. Our analysis is based on six months of real-world usage data from more than 500 users, yielding a
dataset of 3,030 anonymized, naturally occurring fallback-triggering utterances. We contribute (1) an operational taxonomy and the annotated VoxFallbacks
dataset of these interactions, (2) a comparative evaluation of different models within a classification pipeline under practical deployment constraints, and (3) practical lessons for designing robust and cost-efficient fallback mechanisms. Results show that lightweight
embedding-based classifiers outperform larger generative models on most classification tasks while requiring substantially fewer computational resources.
Read the full abstract ▾
All authors Phillip Schneider, Alexandre Mercier, Joshua Oehms, Kristiina Jokinen, Florian Matthes
Computation and Language
Calibrating Small Language Models for Claim Check-Worthiness Detection
Pratuat Amatya, V Venktesh and Vinay Setty
Assessing claim check-worthiness is an essential first step in automated fact-checking pipelines. This work is motivated by a real deployment challenge at an early-stage startup: running large
language models (LLMs) over every incoming claim is cost- and latency-prohibitive, yet smaller models sacrifice accuracy. We propose NN-PPI, a pointwise extension of Prediction-Powered Inference (PPI) that calibrates model predictions at inference time as a lightweight post-hoc layer, without re-training the underlying model. NN-PPI achieves weighted F1 gains ranging from 12% to 33.80% depending on the size and performance of the baseline model, bringing SLMs on par with larger LLMs. Beyond few-shot SLMs, NN-PPI further improves a production-deployed fine-tuned model, demonstrating that residual calibration is complementary to supervised fine-tuning. By recovering LLM-level accuracy from models that are an order of magnitude cheaper to serve, it makes accurate check-worthiness detection substantially cheaper to operate at scale. Our code and data can be found at https://anonymous.4open.science/r/arr-claim-worthiness-F237.
Read the full abstract ▾
Machine Learning
E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation
Wei Fan, Xinjie Shen, Xudong Guo et al. (+8 more)
Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies require Large
Language Models (LLMs) to continually explore, learn from experience, and adapt their policies over thousands of steps. We introduce E-Commerce Bench, the first open-source
benchmark that integrates multi-round counterpart negotiation and dynamic events into a year-long business operation. Over a 365-day year, an LLM
agent concurrently runs multiple online stores, researching the
market, negotiating with suppliers to source inventory, optimizing sales strategies, fulfilling orders, handling returns, and managing cash flow to maximize its end-of-year total assets. To construct a realistic merchant-side operating environment, the product and supplier data are derived from a real e-commerce platform, while a year-long calendar of promotions, natural disasters, and supply-chain shocks continually reshapes demand. For reproducibility, both sides of the
market are deterministic: customer purchases and returns follow a fixed demand model, while a negotiation kernel determines supplier pricing, concessions, and decisions, with an LLM used only to verbalize them. We evaluate 18 frontier models across seven dimensions, including year-end assets, and find that no single model dominates. GPT-5.6 Sol earns the most, growing the 100,000 opening stake into 1,431,425, yet it ranks 16th of 18 on fraud avoidance and trails Fable5 in operational efficiency. Among open-weight models, Qwen3.8-Max-Preview leads with 416,252, 38% above GLM 5.2 (high), and achieves the strongest learning over the horizon, progressively bargaining down prices across repeated orders. Our code is available at https://github.com/QwenLM/E-CommerceBench.
Read the full abstract ▾
All authors Wei Fan, Xinjie Shen, Xudong Guo, Jianhong Tu, Yang Su, Yinger Zhang, Lianghao Deng, Fengyu Wang, Baohua Dong, Yangqiu Song, Dayiheng Liu
Data Structures and Algorithms
On the maximum weight convex problem for some geometric graph-convexities
Fariza Aklouche, Pierre Bergé and Michel Habib
For a given
geometric graph-convexity on a graph $G$ equipped with a weight function on the vertices with value in $\mathbb{Z}$, the Max Weight Convex Set problem consists in determining the convex set $S$ with maximum weight (sum of the weight of the vertices in $S$). Although the problem is NP-complete in general, it remains
polynomial for particular cases. After a survey of known results, our main contribution uses a generalisation of the maximum subsequence problem to laminar trees. Then we derive a linear
algorithm for proper interval graphs and a quadratic one for interval graphs. Both improve the state of the art.
Read the full abstract ▾
Computer Vision
RailGen: Improving Railway Intrusion Detection via Agent-Guided Small-Scale Foreign Object Generation
Quan Hao, Ziyang Tao, Chenxi Zhang et al. (+3 more)
Small-object detection under long-tailed data distributions is a fundamental yet challenging problem in multimedia. Railway Foreign Object Detection (RFOD) epitomizes this challenge with easily confused small intrusions and scarce samples. To address these issues, we propose a generative-augmented detection paradigm that leverages
multimodal image generation to enrich the feature space of rare and small objects. We first construct RailGen, a
multimodal image generation
agent based on large models. Under semantic constraints, RailGen automatically invokes tools to generate railway scenes, calibrate intrusion positions, extract foreign objects, and fuse them into realistic intrusion effects. This process produces high-quality synthetic samples that effectively densify the feature representations of tail classes and complete the small-object feature space. Within this paradigm, we further propose FocalDEIM, a detection framework designed to enhance training with generated data. FocalDEIM improves dense matching with Focal Modulation for better small-object discrimination and adopts Focal Loss to emphasize hard samples, thereby alleviating blurred inter-class boundaries in complex railway scenes. Experimental results demonstrate that RailGen can generate high-quality small-scale foreign objects, reducing the object pixel area by up to 58x and 13.85x on average. Equipped with these challenging samples, our paradigm surpasses the baseline DEIM by 5.6% and 7.5% in mAP@50 and mAP@(50-95), respectively, and outperforms existing state-of-the-art methods. Ablation studies verify RailGen's feature-space enrichment and FocalDEIM's boundary discrimination. The paradigm provides an effective
multimodal generative solution for long-tailed small-object detection in safety-critical applications.
Read the full abstract ▾
All authors Quan Hao, Ziyang Tao, Chenxi Zhang, Yudong Wang, Rui Shi, Liguo Zhang
Artificial Intelligence
Multimodal Adaptive Expert Selection with Text Routing and Ordinal Prototype Optimization for Sentiment Analysis
Xiaode Chen, Jiakang Yu, Hongtao Deng et al. (+3 more)
Multimodal Sentiment Analysis (MSA) is a fundamental component of affective computing that aims to decipher complex emotional states by integrating verbal content with non-verbal cues including vocal intonation and facial micro-expressions. While recent disentanglement-based approaches have advanced the field, their potential is hindered by two methodological challenges. First, static computation graphs process all samples indiscriminately regardless of semantic complexity, which leads to suboptimal representation for diverse emotional expressions and contextual scenarios. Second, generic contrastive objectives often neglect the intrinsic ordinal hierarchy of sentiment intensities. To systematically address these limitations, we introduce
Multimodal Adaptive Expert Selection with Text Routing and Ordinal prototype optimization (MAESTRO), a novel framework designed to dynamically orchestrate and refine
multimodal representations. Drawing inspiration from an orchestra conductor, we design a Text-Guided Hybrid Mixture-of-Experts (MoE) mechanism. Unlike static fusion, this module utilizes linguistic context as a routing signal to dynamically activate specific audio-visual experts, thereby resolving cross-modal ambiguity through adaptive feature enhancement. Furthermore, to capture fine-grained sentiment gradations, we propose an Ordinal-aware Prototype Contrastive Learning (O-PCL). By incorporating distance-based penalties into the prototype learning objective, O-PCL enforces a structured latent space that preserves the natural order of
emotion. Extensive experiments on the CMU-MOSI and CMU-MOSEI
benchmarks demonstrate that MAESTRO achieves state-of-the-art performance, and qualitative analysis further confirms the interpretability of our dynamic routing paradigm.
Read the full abstract ▾
All authors Xiaode Chen, Jiakang Yu, Hongtao Deng, Huina Qu, Xun Zhu, Yinxia Lou