Machine Learning
UNISON: A Unified Framework for Learning from Scored Bags
Ofek Kirshenboim, Or Shkuri, Shani Finkelshtein et al. (+1 more)
We introduce UNISON, a unified framework that bridges multiple instance learning and recommendation systems by treating both as instances of learning from scored bag collections of (item, score) pairs. Unlike traditional approaches that address bag-level classification and personalized recommendation as separate problems, UNISON decomposes learning into three interconnected stages: (1) learning universal item representations, (2) modeling bag-specific preference patterns through lightweight personalized parameters, and (3) inferring bag-level characteristics from learned preferences. We justify the architectural choice of linear bag-specific scoring through a first-order analysis: when item
embeddings are sufficiently smooth in the score, a linear function approximates the score up to a Taylor remainder controlled by the
curvature of the underlying preference function. For cold-start scenarios, we show that the cold-bag fitting problem reduces to ridge-less linear regression with a closed-form solution, and provide convergence guarantees for the iterative variant. Empirically, UNISON matches or outperforms state-of-the-art approaches across diverse domains: entertainment (books, movies), immunology (T
cell receptor (TCR) based classification), and microbiome, offering superior computational efficiency and interpretability. A controlled synthetic experiment further confirms that the bag-specific parameters ($w_j, b_j$) recover the latent factors that drive item scores even when the projection $\phi$ has no access to the original scoring mechanism (AUC > 0.998). UNISON is available at https://github.com/louzounlab/UNISON.
Read the full abstract ▾
All authors Ofek Kirshenboim, Or Shkuri, Shani Finkelshtein, Yoram Louzoun
Machine Learning · Cross-listed from cs.LO
Vision-Predictor-Action: Towards Efficient Embodied AI via Neuro-Symbolic JEPA Predictors
Yeonseok Lee
Vision-
Language-Action (VLA) models commonly generate
robot actions
token by
token through Large
Language Model (LLM) decoders, causing inference latency to grow with action sequence length and limiting closed-loop control frequency. We present Goal-Conditioned
Vision-Predictor-Action (VPA), which replaces autoregressive decoders with a lightweight latent control loop. A shared
vision encoder and a frozen text encoder embed observations, goal images, and instructions; a neuro-symbolic selector chooses a discrete skill; a Joint-
Embedding Predictive Architecture (JEPA) predictor forecasts the next latent state; and a flow-matching solver generates an entire action chunk in a few integration steps, so that sequential network evaluations per decision step do not depend on the action horizon. We analyze inference complexity, regularization against representation collapse, and conditions under which latent sub-goal thresholding limits error accumulation across task phases. As an initial proof of concept, we evaluated VPA on the LIBERO-10 suite. The trainable model has under 20M parameters (checkpoint under 75 MB, excluding a frozen 63.4M-parameter CLIP text encoder); a decision step of a single-camera reference configuration takes 10-14 ms on an Apple M4; and trained only on LIBERO-10 demonstrations in under three hours on a single GPU, a two-camera policy succeeded in 40.0% of 200 evaluation episodes. These preliminary results from a single training run suggest the approach is workable. Code and weights are available at https://github.com/ylee-sling/
Vision-Predictor-Action and https://huggingface.co/ylee-sling/vpa-libero10, and archived at https://doi.org/10.5281/zenodo.22960920.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
ELECT: Settling Identity Before Rendering for Scalable Story Visualization
Mehrajul Abadin Miraj, Md. Mahmudul Hasan, Meherun Farzana et al. (+9 more)
Training-free story visualization without reference images must preserve recurring entities across frames while scaling to long narratives. Existing methods typically maintain consistency by conditioning each frame on other frames of the same story, coupling generation time, VRAM, or both to story length. We introduce ELECT, which instead settles cross-frame identity before final rendering through a
language-guided scene-entity mapping and stationary anchors elected from the model's own drafts. This makes the per-frame cost of the final rendering pass independent of story length. Joint
attention exceeds 96 GB of VRAM at 32 scenes, whereas ELECT remains at 8.3 s per frame and under 20 GB VRAM from 10 to 1000 scenes. On the Story50 long-story
benchmark, ELECT reaches 0.833 average Character Consistency Similarity (aCCS), compared with 0.826 for ten-round refinement, at one seventh of its generation cost, with lower text
alignment (0.327 vs. 0.340). At 1000 scenes, to our knowledge, the first evaluation of training-free story visualization at this length, ELECT exceeds ten-round refinement by 0.017 aCCS and the nearest flat-cost baseline by 0.091, while also surpassing refinement in text
alignment at one twenty-fourth of its generation time. These results show that long-horizon consistency need not require increasingly expensive inter-frame conditioning as stories grow.
Read the full abstract ▾
All authors Mehrajul Abadin Miraj, Md. Mahmudul Hasan, Meherun Farzana, Aniket Joarder, Mahmudul Hasan, Abir Chakraborty Partha, Md. Ahasanul Alam, Redwan Ahmed Rizvee, Md. Fahim Arefin, Md Mahmudur Rahman, Md. Tanvir Alam, Md Mosaddek Khan
Machine Learning · Cross-listed from math.OC
Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
Yuriy Dorn, Dmitrii Vishovan, Kirill Novoselov et al. (+3 more)
We study
federated learning from continuously arriving data: each client samples from its own distribution, these distributions drift over time, and the server must track the minimizer of the clients' weighted average loss. Clients are not alike as they collect data at different rates, with different noise, and change at different speeds. The main question is how much history should each one keep? A single shared window must serve every client at once, and so makes a fast-changing client carry the sampling noise of a slow, nearly stationary one. We give each client a
memory of its own, sized to that client's noise, arrival rate, and drift, and refresh it by restarted accelerated method that starts from a common point. The tracking error is then bounded by \[ R\left(\sum_m w_m^{3/2}\,\sigma_m\sqrt{\gamma_m/\lambda_m}\right)^{2/3}, \] for clients with weight $w_m$, noise $\sigma_m$, arrival rate $\lambda_m$, and drift rate $\gamma_m$ on a domain of diameter $R$. This is never worse than what any common window can achieve, and strictly better whenever noise and drift are spread unevenly across clients. A lower bound that puts different groups of clients on different drift clocks shows this quantity is the right one. The two bounds agree up to a constant whenever the clients occupy boundedly many refresh scales, and in general differ only by the sixth root of the number of scales. Experiments show the gain comes from the
memory rather than from the solver: it persists across accelerated,
stochastic-gradient, and one-step methods, and against a separately tuned common-
memory baseline.
Read the full abstract ▾
All authors Yuriy Dorn, Dmitrii Vishovan, Kirill Novoselov, Ilya Kuruzov, Darina Dvinskikh, Alexander Gasnikov
Machine Learning
Sensitivity Fidelity Reveals Physical Dependencies Beyond Forecast Accuracy in Deep-Learning Earth-System Models
Qi Chen, Peng Zhan, Hongxing Cui et al. (+2 more)
Deep-learning models increasingly rival numerical simulation in Earth-system forecasting, yet forecast error alone cannot determine whether their learned input-output dependencies are physically consistent. Here we introduce sensitivity fidelity as an additional criterion: alongside prediction error, model responses to input perturbations should be examined against a trusted reference or established physical knowledge. In an idealized Rayleigh-Bénard convection
benchmark, four surrogate architectures achieved low forecast errors yet encoded markedly different sensitivities. Quantitative comparison with a matched adjoint distinguished the models far more sharply than forecast accuracy and showed that predictive skill and sensitivity fidelity need not coincide, while finite-perturbation experiments further linked these differences to the propagation of initial-condition uncertainty. We then applied this strategy to El Niño-Southern Oscillation prediction, where no adjoint reference exists and learned sensitivities showed event-wise robustness, validity under finite perturbations, and consistency with established precursors. These results identify gradient-based evaluation as a discriminating complement to forecast-error metrics, providing a practical test of whether data-driven Earth-system models learn physically credible dependencies.
Read the full abstract ▾
All authors Qi Chen, Peng Zhan, Hongxing Cui, Wenqing Cui, Ibrahim Hoteit
Machine Learning · Cross-listed from cs.CR
Poisoned into Action: Context Attacks on Typed Decision Workflows
Chenkai Wu, Xing Chang and Jun Yan
Typed decision interfaces restrict models to predefined actions, yet schema-valid choices can still trigger policy-violating state changes. Evaluating context poisoning thus requires connecting model decisions to execution outcomes while accounting for legitimate automation retained by defenses. We introduce an execution-aware protocol tracing poisoned evidence through
retrieval, model choice, automated action, and persisted state. We evaluate it with 1,659 fresh calls to Jev-1.13.0 across 20 development tasks in two synthetic workflows, using three repetitions per condition. Under natural
retrieval, a standalone payload reaches all 60 trials and produces five security-violating executions, while a carrier-payload pair exposes the payload in 57 trials and produces one violation. An initial source-exclusion gate blocks the pair's observed violation but also withholds ten otherwise correct attacked decisions and sixteen correct clean decisions. Correcting its empty-scope behavior restores twelve clean decisions without additional model calls. Factorial controls show that adding correct evidence reduces wrong executions from 119/120 to 55/120 in the semantic arm and from 60/120 to zero in the neutral arm, suggesting that much of the improvement reflects general evidence value rather than attack-specific protection. Three prespecified contrasts are not statistically supported after Holm correction, so the findings remain exploratory. The protocol provides an auditable framework for studying which poisoning-induced errors reach execution and how much correct automation defenses preserve. Code is available at https://github.com/WuXixiong/jev-poisoning-bench.
Read the full abstract ▾
Machine Learning
OceanFlow: Coherent Flow Matching for Probabilistic Ocean Modeling
Yuan Gao, Fan Xu, Hao Wu et al. (+1 more)
Probabilistic ocean modeling is essential for risk-aware marine operations,
climate analysis, and scientific decision-making, where downstream decisions depend on the conditional distribution of future ocean states rather than a single deterministic trajectory. Recent data-driven ocean models have made deterministic global ocean modeling increasingly accurate and efficient, but probabilistic ocean modeling remains underdeveloped. Specifically, perturbation-driven ensembles can create spread without explicitly optimizing the joint distribution of their members, often yielding poor calibration and physical inconsistency. To address these issues, we propose OceanFlow, a coherent flow matching framework for probabilistic global ocean modeling. OceanFlow first establishes a sharp, stable mean anchor for the conditional ensemble. Flow matching then learns a coherent state-conditioned pathwise law that jointly reshapes the residual distribution around this anchor at each transition, preserving the predictive center while refining state-dependent uncertainty. We further introduce a Factorized Dynamics Operator (FDO), which combines factorized transitions with a learned state-conditioned response to parameterize the mean anchor and residual-flow velocity. Extensive experiments show that OceanFlow improves ensemble-mean accuracy, probabilistic reliability, and physical-consistency diagnostics over competitive baselines, supporting coherent flow matching as an effective route toward data-driven probabilistic ocean modeling. Code is available at https://github.com/YuanGao-YG/OceanFlow.
Read the full abstract ▾
All authors Yuan Gao, Fan Xu, Hao Wu, Xiaomeng Huang
Machine Learning
SARATHI: Decoupled Data-Free Probing and Adaptive Reconstruction for Structured LLM Pruning
Sankar Behera, Dhruv Singh, Satyadev Ahlawat et al. (+1 more)
Structured
pruning reduces the inference cost of large
language models (LLMs) by permanently removing
neurons and weight rows, yielding dense, hardware-agnostic speedup. Existing post-training methods couple
neuron selection and weight reconstruction through a shared calibration
dataset, introducing domain bias and out-of-
memory failures at high sparsity. We propose SARATHI, a framework that decouples these two stages. A data-free NMF Residual Subspace Probe scores
neurons using intrinsic weight geometry, without accessing any calibration data, and carries a formal connection to Robust PCA. Adaptive OBS then reconstructs the pruned weights with a localized calibration batch, operating in $\mathcal{O}(d^2)$ peak
memory per layer and eliminating the global
memory bottleneck of coupled methods. A Bias Shift Compensation step corrects systematic activation drift in post-LayerNorm architectures that would otherwise cause perplexity collapse at high sparsity. Experiments across six models from two architecture families (gated and non-gated FFN) at 15–40% sparsity show that SARATHI achieves competitive perplexity and matches or exceeds existing structured
pruning baselines on zero-shot accuracy.
Read the full abstract ▾
All authors Sankar Behera, Dhruv Singh, Satyadev Ahlawat, Yamuna Prasad
Machine Learning
CacheLink: Functional KV Transport for Zero-Prefill Cross-Model Context Handoff
Haomiao Qiu, Yining Tang, Zijia Song et al. (+3 more)
KV caches make long-context serving efficient by reusing past computation, but model switching breaks this reuse and forces the receiver to prefill the context again. We study zero-prefill cross-model context handoff: translating a sender cache into a receiver-compatible state so that the receiver can continue directly without prefilling same context. Existing affine transport methods show that receiver KV states can be substantially reconstructed from sender caches. However, through extensive analyses, we find that reconstruction fidelity is a poor proxy for receiver utility. Similar K/V reconstruction errors can lead to substantially different
attention patterns, predictive distributions, and downstream accuracy, because conventional reconstruction objectives overlook the query-dependent functional importance of K and V in receiver
attention. We thus introduce CACHELINK, which learns a coupled K/V correction through the receiver behavior directly. Across four independently trained sender–receiver pairs and seven evaluation panels, CACHELINK improves the equal-weight mean over all methods for every pair. In matched diagnostics, CACHELINK changes K/V squared error by less than 0.4% while substantially changing receiver
attention and next-
token behavior. At 32K context, it reduces handoff latency by up to $7.27\times$ relative to receiver re-prefill. Our findings verify that reconstruction fidelity alone is insufficient and that preserving the receiver computation induced by the transported cache is important for effective cross-model KV transport.
Read the full abstract ▾
All authors Haomiao Qiu, Yining Tang, Zijia Song, Ruizi Han, Ziyue Qiao, Miao Zhang
Machine Learning
Revisited Support Vector Machines: A Unified Joint Optimization Framework with Nyström Approximation for Scalable Adaptive Learning
Nassou Gongon Benjamin, Noulapeu Ngaffo and Patrick Njionou Sadjang
The theoretical elegance of Support Vector Machines (SVMs), grounded in statistical learning theory, establishes them as a reference for robust classification. However, in the era of Big Data, classical SVMs face two major bottlenecks: the prohibitive cubic complexity $\mathcal{O}(N^3)$ of training and the rigidity of static kernels. Traditional methodologies treat kernel selection as a discrete, "brute-force" GridSearch problem, effectively decoupling the learning of data representation from the optimization of the decision boundary. This paper proposes a paradigm shift with a so-called "Revisited SVM", a unified framework that recasts the learning process as a continuous %bi-level joint optimization problem. We derive a novel objective function $J(\alpha, \theta)$ that allows for the simultaneous optimization of dual Lagrange multipliers and kernel hyper-parameters. To solve this challenging non-convex problem efficiently, we introduce a hybrid Block Coordinate Ascent (BCA)
algorithm, tightly integrated with Nyström-based low-rank approximation, which drastically reduces computational complexity to $\mathcal{O}(Nm)$. Extensive experiments conducted across four heterogeneous domains (Bioinformatics, Finance, Cybersecurity, and Computer
Vision) demonstrate that our approach not only matches but often exceeds exhaustive GridSearch accuracy, while delivering training speedups of up to 100x. Notably, on the complex NSL-KDD intrusion detection
dataset, our adaptive kernel approach achieves 96.25% accuracy compared to a mere 50% for standard baselines, proving its unique ability to model complex non-linear
manifolds in high-dimensional spaces. We validate our results using rigorous statistical tests and provide a theoretical convergence analysis of the proposed
algorithm.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SP
RF Fingerprinting of DSP Implementations using Machine Learning: An Over-the-Air Case Study
Stefan Scholl
Radio frequency fingerprinting aims to identify transmitters by exploiting small differences in the emitted signal waveform. Common approaches exploit hardware-dependent imperfections for that purpose. This work instead investigates whether differences between digital signal processing implementations of a transmitter can be identified from their radiated signals. As a practical case study, five different software-based DSP implementations for generating a 2-FSK RTTY communication waveform are considered. In a monitoring receiver, a classifier can identify these DSP implementations using convolutional neural networks that are trained on augmented software-generated signals represented as raw IQ, power spectrum or instantaneous frequency. The trained networks are evaluated on over-the-air recordings obtained using a shortwave transceiver and a software-defined
radio receiver. The best neural network combines IQ and the power spectrum representation and achieves an accuracy of up to 90%. The study further examines the influence of the signal SNR and the observation bandwidth on the classification accuracy. The results demonstrate that DSP implementations leave distinguishable characteristics in the transmitted signals and suggest that DSP-dependent fingerprinting can provide complementary information for specific emitter identification systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
AlphaPilot: Multi-Turn Tool-Using LLM Agents for Adaptive Alpha Mining with Reinforcement Learning
Peiqi Jiang, Fan Zhang, Xin Xu et al. (+2 more)
Formulaic alpha mining seeks interpretable expressions that rank assets by future returns. The task is challenging because it demands simultaneous processing of unstructured
market descriptions, historical data, and quantitative evaluation of candidate factors, yet most existing methods exploit only a subset of these signals. We introduce AlphaPilot, a reinforcement-learning framework that trains multi-turn tool-using LLM
agents for adaptive alpha mining. The
agent conditions on
market descriptions, proposes factor expressions, queries real-data evaluator feedback, and updates a retained factor pool across turns. This design jointly leverages
market text, LLM financial knowledge, real data, and evaluation feedback. At inference time, AlphaPilot performs day-by-day search and pool updates to adapt to shifting
market regimes. Across three independent runs in out-of-sample rolling backtests on CSI300 and CSI500, AlphaPilot achieves mean total returns of 42.93% and 54.19%, with mean Sharpe ratios of 2.46 and 2.86, respectively. It attains the highest mean total return, Sharpe, and RankIC among the evaluated methods on both universes.
Read the full abstract ▾
All authors Peiqi Jiang, Fan Zhang, Xin Xu, Tianhao Chen, Can Yang
Machine Learning
Split Conformal Interval Length under Huber Contamination
Xiao Zhang, Bowen Zhang and Lie He
Existing analyses of interval length in split conformal prediction largely focus on clean data. However, how contamination at the training and calibration stages alters interval length relative to the corresponding clean population oracle remains understudied. In this paper, we consider Huber contamination and derive a finite-sample upper bound that holds uniformly over contaminating distributions, with leading terms scaling as $L(\delta_1+n_1^{-1/2}) +(n_2^{-1/2}+\delta_2)/((1-\delta_2)r_0)$, in addition to a clean-sample estimation term. We further establish a minimax lower bound matching the upper-bound rate up to constant factors on a regular asymmetric location submodel. For calibration contamination, we obtain an exact worst-case characterization of the population interval-length deviation. We also extend the upper-bound results to conformalized
quantile regression and adaptive residual scores. Experiments on synthetic and real-world
datasets provide empirical support for our theoretical findings on training and calibration contamination.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.med-ph
PMB-NN: Physiological-Model-Based Neural Network for Glucose Forecasting in Type 1 Diabetes
Yaowen Zhang, Shuhao Que and Ying Wang
Accurate, physiologically consistent glucose forecasts could help people with type 1 diabetes respond to impending excursions. We applied our physiological-model-based neural network (PMB–NN) framework to glucose forecasting. A temporal convolutional network (TCN) used 180 min of history, constrained during training by a personalized four-parameter physiological model. The main PMB–NN combined forecast and neural–physiological agreement losses; a three-term variant additionally fitted physiological predictions to measured glucose. Data comprised 6–8 weeks of continuous glucose monitoring, pump, and meal records from 25 AZT1D participants. Person-specific models used chronological 70/10/20 train/validation/test splits and 30-, 60-, and 120-min horizons. Median mean absolute errors were 16.24, 25.00, and 30.20 mg/dL with basal insulin records and 15.38, 24.21, and 31.79 mg/dL before these records became available. In the latter setting, PMB–NN had the lowest median error at 30 and 60 min; it outperformed the physiological model alone in all six settings. In input-increase tests, glucose predictions changed in the expected direction for 85% of correction boluses and 100% of carbohydrate rescues, versus 57% for active insulin and 56% for active carbohydrates. Most standalone physiological fits showed locally distinguishable parameter effects, with glucose-related coefficients more stable under resampling than insulin- or carbohydrate-related coefficients. The three-term variant improved hyperglycaemia accuracy with basal records, but did not significantly change overall MAE and reduced directional plausibility in three tests. PMB–NN combines personalized forecasting with an explicit physiological model, allowing assessment of predictive accuracy, physiological response behavior, and participant-specific coefficients.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
BridgeCodec: Mamba Enhanced Neural Audio Codec with Schrödinger Bridge at Low Bitrate
Zijian Lin, Jing Yang, Jinghao Luo et al. (+3 more)
Neural audio codecs achieve remarkable performance but operate as closed ecosystems with rigidly co-trained encoderdecoder pairs, severely hindering interoperability. To resolve this, we propose BridgeCodec, a versatile framework that decouples these pairs by translating latent representations between mismatched, frozen endpoints. We formulate this cross-codec translation as a Schrödinger Bridge optimal transport problem, enabling robust mapping between disparate distributions without predefined noise priors. To efficiently capture longrange
speech temporal dependencies, we integrate a Mambaenhanced U-Net architecture. We validate BridgeCodec on an extreme deployment scenario: mapping a lightweight 8 kHz source encoder directly to a 48 kHz target decoder at an ultralow bitrate of 1 kbps. Experimental results demonstrate that BridgeCodec achieves superior wideband reconstruction and high perceptual quality, successfully bridging heavily bottlenecked, heterogeneous codecs.
Read the full abstract ▾
All authors Zijian Lin, Jing Yang, Jinghao Luo, Zhuo Wang, Fan Fan, Zhiyong Wu
Machine Learning · Cross-listed from cs.CV
Harnessing SPAD Noise: Test-Time Gated Attention for Indoor Depth Estimation
Yiquan Gao
Monocular depth estimation (MDE) has emerged as a cost-effective alternative to hardware-dependent 3D sensing modalities, yet predicting precise metric depth remains a persistent challenge due to intrinsic scale ambiguity, edge bleeding, and limited zero-shot generalization in unseen environments. Recent multi-modal approaches incorporate 1D/3D Single-
Photon Avalanche Diode (SPAD) data to resolve these ambiguities, but they mainly treat SPAD noise as a harmful artifact required for denoising or leave the noise ignored. In this paper, we present an essentially different insight: native SPAD noise is an informative prior that implicitly encodes deterministic environmental characteristics. Driven by this paradigm shift, we introduce a novel, lightweight Test-Time Gated
Attention framework by Harnessing SPAD Noise, therefore dubbed HSN. Tailored for indoor scenarios, HSN leverages the synergistic combination of 1D SPAD histograms and Gate Adapters (GAs). Through three complementary optimization phases, it generates channel-wise gated
attention vectors to dynamically adjust encoder feature maps at inference time, effectively resolving ambiguous scale and boundary challenges without the heavy computational overhead or dense 3D modalities that other TTA methods suffer from. Extensive experiments on challenging real-world indoor
benchmarks show that HSN achieves state-of-the-art metric depth accuracy in in-domain settings, while exhibiting even greater potential under zero-shot generalization to unfamiliar environments. Furthermore, interpretability analysis reveals that the proposed gated
attention mechanism adaptively shifts
attention toward deeper semantic layers under intensifying SPAD noise. Code will be made publicly available.
Read the full abstract ▾
Machine Learning
World Modeling through Spectral Alignment
Holger Molin, William Peng, Marco Bagatella et al. (+1 more)
How do we tell a world model what information to preserve? We study this question through the lens of spectral representation learning. We introduce SpecWM, a JEPA framework that explicitly specifies desired relationships between observations through a target similarity kernel and trains the encoder to reproduce those relationships in its
embedding geometry. Different kernels provide different supervision, ranging from physical state information to a fully self-supervised temporal kernel requiring only the ordering of observations within trajectories. We characterize the global optima of the objective, establish conditions for linear recoverability and invariance to nuisance information, and connect temporal-kernel similarity to the behavioral successor measure under an idealized setting. Empirically, the choice of kernel strongly controls which physical quantities are recoverable from the representation. More importantly, temporal spectral
alignment improves CEM planning success over LeWorldModel from 52% to 68% on OGBench Cube, 49% to 77% on OGBench Scene, and 29% to 51% on CALVIN. These results suggest that explicitly specifying representation geometry provides a simple way to steer what latent world models preserve for control.
Read the full abstract ▾
All authors Holger Molin, William Peng, Marco Bagatella, Randall Balestriero
Machine Learning
Pessimistic Latent Task-aware Optimization for Robust Offline Meta-Reinforcement Learning
Chang-Hoon Jeong, Kisung Shin, Hanwool Sul et al. (+1 more)
Context-based offline meta-reinforcement learning (COMRL) has advanced almost entirely through better task representations, which separate task identity from artifacts of the behavior policy so that the latent reflects the task alone. The implicit expectation has been that a clean representation will transfer to out-of-support (parameter-extrapolation) tasks. On such a task, however, the encoder produces a latent outside the training support, where the policy has never been trained. We address this gap with Pessimistic Latent Task-aware Optimization (PLATO), which exposes the policy to off-support latents during training. PLATO assumes a mild
manifold hypothesis: task latents lie on a low-dimensional
manifold, and out-of-support tasks lie farther from the training centroid. It exploits this geometry by perturbing inferred latents outward from the centroid to synthesize counterfactual tasks. A learned decoder ensemble then rolls out short trajectories at each perturbed latent, and the policy is updated on the imagined transitions under a pessimism weight that down-weights steps on which the ensemble disagrees. We prove that the perturbation leaves the training support by a controllable margin and that the out-of-support generalization gap is bounded by the same perturbation reach, and we give explicit conditions, verified empirically, under which the epistemic disagreement at the perturbed latent stays bounded away from zero. On eight MuJoCo continuous-control
benchmarks, PLATO consistently improves out-of-support returns over strong COMRL baselines while remaining competitive on in-distribution tasks. Our code is available at https://github.com/deephoony/plato.
Read the full abstract ▾
All authors Chang-Hoon Jeong, Kisung Shin, Hanwool Sul, Byoung-Tak Zhang
Machine Learning
Riemannian Graph Tokenizer for Structural Knowledge Transfer
Qimin Zhou, Haibo Liu, Yujie Wang et al. (+2 more)
Foundation models are at the forefront of artificial intelligence. A tokenizer, converting the raw input into discrete representations that the model can understand, plays an important role to the success of foundation models. Unlike the text tokenizer that is well studied in large
language models, graph tokenizer is still at its early stage, facing the challenges of tackling the non-Euclidean structures and capturing the structural semantics. How to design a graph tokenizer for structural knowledge transfer? To this end, we propose a Riemannian Graph Tokenizer (RGT) that bridges the structural knowledge and quantized representations to support cross-domain structural knowledge transfer. The connection is established by Riemannian geometry. Specifically, we first define the
geometric vocabulary (trees, cycles and sequences), which captures fundamental structural patterns and reflects the intrinsic geometry of graph. Second, we construct a Riemannian quantizer with Riemannian Straight-Through
Estimator to tokenise graph structures across multiple domains into discrete
tokens. To ensure consistency and transferability across diverse
geometric spaces, RGT further incorporates a geometry-aligned decoder that projects
manifold-specific
tokens into a unified tangent space. The theoretical analysis and
geometric interpretations are provided to support the effectiveness of our proposed method. Extensive experiments across diverse
datasets demonstrate that RGT significantly enhances structural knowledge transferability across graph domains.
Read the full abstract ▾
All authors Qimin Zhou, Haibo Liu, Yujie Wang, Li Sun, Chuan Shi
Machine Learning · Cross-listed from cs.CL
Sensitivity or Criterion? Measuring and Moving When Language Models Act
Sixun Dong, Yebowen Hu, Ming Yin et al. (+3 more)
Before an
agent acts, it must decide whether action is warranted. A single accuracy cannot distinguish failure to recognize a warranted action from reluctance to act. Signal-detection theory separates sensitivity from criterion, but criteria estimated from authored binary labels depend on each
benchmark's example mix. We introduce FlipAct, a diagnostic of tool use in physical states. A rule over annotated tool properties computes each answer and a graded evidence score, fixing the Yes/No boundary at zero across models. Questions form 2-by-2 quads that hold required knowledge fixed while the state flips. The slope of the fitted Yes-rate curve measures sensitivity; its crossing at one half measures criterion. Across 44 open models, sensitivity rises with size within a family, and its ranking agrees with three of four external when-to-act
benchmarks. Models respond to evidence sign rather than magnitude, accepting actions that leave the state no better about as often as helpful ones. When Yes means act, most models are conservative across rephrasings; in some, this partly reflects a preference for No. Criterion is not determined by size. Activation steering shifts criterion with weights fixed;
distillation from a model's own steered copy puts the shift into its weights without a significant sensitivity change. The distilled shift transfers to six external
benchmarks, including two without Yes/No answers, while criterion-free AUC is preserved on the five
benchmarks where it is available. Accuracy improves where models start conservative and declines or remains within error where they start permissive. Project page: https://sixundong.com/projects/flipact
Read the full abstract ▾
All authors Sixun Dong, Yebowen Hu, Ming Yin, Yiran Chen, Fei Liu, Chen Chen
Machine Learning
Harnessing VLM in Embodied RL: From Reward Regressor to Success Detector via Exploration-Augmented Key-Observation Gating
Xin Liu, Yutong Jiang, Jian Su et al. (+4 more)
VLMs provide promising solutions to embodied reward design. Most current methods that train VLMs as reward regressors not only require extensive annotations but are also primarily designed for finetuning pretrained policies, and perform less well in the more challenging from-scratch RL setting. This issue stems from a fundamental mismatch: regressing precise rewards is misaligned with what VLMs naturally excel at. We posit that it is more aligned with VLMs' inherent multi-modal VQA capabilities for the models to function as success detectors, and propose VLM success reward via Exploration-Augmented Key-Observation Gating (VLMeg) to well harness VLM for embodied reward design. As the core of VLMeg, the proposed exploration-augmented key-observation gating first determines whether the current RL episode is judgment-worthy based on its similarity to a generated success trajectory. Then, the key observation with the lowest confusion from the judgment-worthy episode will be selected for VLM evaluation as episodic reward. This gating mechanism not only avoids redundant calls but also enables higher detection accuracy of VLM. Considering the sparsity of judgment-worthy episodes, the advanced exploration incentives based on the generated trajectory are introduced, enabling more efficient exploration toward judgment-worthy trajectories for more VLM reward triggers and thus gaining higher RL efficiency. Extensive experiments in MetaWorld demonstrate VLMeg achieves state-of-the-art from-scratch RL, outperforming advanced VLM-based baselines on 12 challenging reward-free tasks. Furthermore, VLMeg also enables effective RL without any ground-truth reward in 3 challenging real-world experiments. Code will be open after acceptance at https://github.com/Raina915/VLMeg.
Read the full abstract ▾
All authors Xin Liu, Yutong Jiang, Jian Su, Ziheng Wang, Chuang Wang, Haoran Li, Dongbin Zhao
Machine Learning
GA-SVD: Generation-aware Singular Value Decomposition for LLM Compression
Shengkun Tang, Cong Zeng, Zongfang Liu et al. (+2 more)
Large
language models (LLMs) demonstrate strong generalization across a broad range of
reasoning and generation tasks, but their large parameter counts impose substantial
memory and computation costs, hindering deployment in practical resource-constrained settings. One-shot low-rank compression is a promising solution since it reduces model size and inference cost effectively. However, existing SVD-based methods primarily target multiple-choice
benchmarks and often fail to preserve performance on generation tasks. Moreover, their rank allocation strategies are typically heuristic. We propose GA-SVD, a generation-aware low-rank compression framework for LLMs. First, we introduce Generation-Aware Weight Whitening (GAWW), which leverages gradients from on-policy knowledge
distillation to construct whitening transformations aligned with generative behavior. Second, we develop an evolutionary global rank allocation method that searches for effective layer-wise rank allocation under a given global compression ratio. Across extensive experiments on Mistral-7B, Qwen3 series, and Llama-3.3 models, evaluated on both multiple-choice and generative
benchmarks, GA-SVD consistently outperforms existing SVD-based baselines and achieves state-of-the-art (SoTA) results. We further demonstrate scalability by applying our method to models with up to 70B parameters.
Read the full abstract ▾
All authors Shengkun Tang, Cong Zeng, Zongfang Liu, Dan Alistarh, Zhiqiang Shen
Machine Learning
WUSPruner: Second-Moment-Optimal Transforms for Structured Pruning
Shengkun Tang, Dan Alistarh and Zhiqiang Shen
Structured
pruning is typically formulated as identifying which model dimensions to remove, often within the model's original representation. However, the effect of coordinate deletion fundamentally depends on the basis in which
pruning is performed. This raises a more fundamental question: how should the basis be chosen specifically for coordinate deletion? We introduce WUSPruner, a transform-based framework for structured
pruning of large
language models. We formulate basis selection as a local linear reconstruction problem that directly minimizes the reconstruction loss induced by
pruning. Inspired by the second-moment transformation in WUSH, we consider a WUS transform constructed jointly from activation and weight statistics. We show that under positive-definite second moments, WUS transform minimizes the local reconstruction loss over all invertible changes of basis. In the resulting basis, the activation and weight second moments are simultaneously diagonalized and balanced, causing the deletion error to decompose into independent spectral contributions and yielding a principled criterion for coordinate selection. Guided by this result, we develop a scalable
pruning method based on blockwise WUS transformations and compact input projections. Experiments on Llama and Qwen models from 8B to 72B parameters show consistent improvements over structured
pruning baselines across
language modeling, knowledge, commonsense
reasoning, mathematics, and code generation
benchmarks, while also delivering practical reductions in inference latency and
memory usage.
Read the full abstract ▾
Machine Learning
DC-Flow: Target-Conditioned Cyclic Peptide Design via Delayed-Coupling Flow Matching
Yitian Wang, Minjie Cheng, Yaning Cui et al. (+3 more)
Target-conditioned cyclic peptide design requires coordinating interfacial side chains with macrocyclic closure. Existing end-to-end generators concurrently update
geometric and chemical variables from noise, assigning residues to disordered coordinates. Meanwhile, predictor-assisted workflows often get stuck in local traps because they lack feedback in the propose-and-filter stage and ignore 3D structural proposals during sequence search. To overcome the drawbacks of these two paradigms, we propose DC-FLOW, a staged generative framework built around Delayed-Coupling Flow Matching. In principle, DC-FLOW postpones sequence and side-chain activation until a target-conditioned backbone takes shape, resolving structural ambiguity before chemical assignment. Then, it feeds delayed activations back into the backbone, achieving co-
evolution of backbone geometry and emerging residue identities by coupling their flows. This mechanism enables more reliable macrocyclic closure and favorable interface packing. For predictor-assisted workflows, DC-FLOW supplies 3D proposals with in-trajectory guidance, enabling cascaded test-time scaling to prune redundant backbones and terminate unpromising refinement early. Extensive experiments show that DC-FLOW improves cyclization fidelity, diversity, and interface energetics, substantially increasing the yield of confident cyclic peptide binders under a fixed final-candidate budget. Our code is available at https://github.com/mianqiu00/DCFlow.
Read the full abstract ▾
All authors Yitian Wang, Minjie Cheng, Yaning Cui, Maohua Yang, Dongdong Wang, Hongteng Xu
Machine Learning · Cross-listed from cs.CR
From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents
Binhong Wang, Haidong Kang, Yanfeng Zhang et al. (+3 more)
Audio Large
Language Models (Audio LLMs) are embedded in
agents for smart homes, vehicles, wearables, and embodied systems, where acoustic inputs can propagate from perception into persistent state and consequential execution. This setting extends security analysis beyond model behavior to the system conditions that determine how acoustic inputs can influence
agent behavior and external actions. Existing surveys examine acoustic security, LLM-based
agents, and edge deployment in isolation, leaving their security interactions in deployed Audio LLM-based
agents insufficiently characterized. This survey introduces an Acoustic Sensing-to-Action (S2A) system model that traces these security implications along the path from acoustic sensing to external execution, while accounting for edge-cloud runtime and lifecycle transformation. We use the model to synthesize attacks, defenses, and evaluation methods across the acoustic,
agent, and edge-system literature. The synthesis shows that security gaps can emerge within components and at the transitions between stages, where protections developed for one stage do not necessarily provide end-to-end assurance in the deployed system. These findings motivate security evaluation that follows acoustic influence from sensing to action across persistent state, system boundaries, and changing deployment conditions in real-world
agents.
Read the full abstract ▾
All authors Binhong Wang, Haidong Kang, Yanfeng Zhang, Hong-Ning Dai, Guo Yu, Shangce Gao
Machine Learning
Approximation in Contrastive Representation Learning
Yuanfan Li, Zihan Zhang, Yiming Ying et al. (+1 more)
Contrastive representation learning (CRL) is a fundamental paradigm for learning transferable representations. Recent work identifies its population target as the pointwise mutual information (PMI) $\log\frac{p(x,y)}{p_X(x)p_Y(y)}$ between two modalities $X,Y$, where $p$ is the joint density function and $p_X,p_Y$ are the marginals. However, it remains unclear why standard inner-product scores $f(x)^\top g(y)$ can effectively approximate a generally nonseparable population target PMI. In this paper, we address this question from the aspect of approximation for CRL. By linking PMI to a compact operator, we show that $D$-dimensional inner-product models perform rank-$D$ spectral approximation, with error controlled by the spectral tail. We then study neural network realizations: in general settings, we show that deep neural network scoring functions are universally consistent, and the approximation error rates depend on the smoothness of the PMI function; under structured low-rank data models we obtain improved rates attained by moderate-width encoders. Finally, we establish a fast calibration rate from contrastive excess risk to downstream
retrieval error under a density-ratio Tsybakov noise condition.
Read the full abstract ▾
All authors Yuanfan Li, Zihan Zhang, Yiming Ying, Ding-Xuan Zhou
Machine Learning
E8 Lattice Key Quantization for Low-Bit KV-Cache Compression
Sasho Nedelkoski, Alexander Acker, Soeren Becker et al. (+2 more)
Compressing the key-value (KV) cache is essential for serving long-context large
language models (LLMs) under
memory constraints. While current scalar quantization methods achieve significant compression, they typically face a quality-fidelity trade-off, either suffering from performance instability at 2-bit precision or necessitating 3-bit keys to maintain model quality. In this work, we demonstrate that a uniform low-bit k2v2 configuration (2-bit keys and 2-bit values) can preserve near-baseline quality by using vector quantization (VQ) based on the $E_8$
lattice for KV keys. We show that structured VQ exploits correlations across adjacent dimensions to achieve signal-to-quantization-noise ratios (SQNR) higher than those of independent scalar quantization at the same key payload rate. To ensure practical utility, we adapt a table-based asymmetric scoring
algorithm that avoids explicit codebook dequantization during
attention computation and preserves decode-throughput parity with scalar baselines. Across five LLM families, our $E_8$ k2v2 approach yields \textasciitilde$6\times$ KV compression, roughly 20% smaller than scalar k3v2 (3-bit keys and 2-bit values), while retaining near-baseline long-context perplexity. In our full-suite perplexity evaluation, $E_8$ improves over scalar k3v2 in 80% of model–
dataset–context settings. These results indicate that $E_8$
lattice VQ improves quality per byte while keeping throughput at parity, suggesting that structured key quantization is a useful primitive for KV-compression stacks. Overall, our work provides a low-overhead primitive for expanding the quality-
memory frontier in long-context LLM inference.
Read the full abstract ▾
All authors Sasho Nedelkoski, Alexander Acker, Soeren Becker, Dominik Scheinert, Odej Kao
Machine Learning · Cross-listed from eess.AS
LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models
Yejin Kim, Kyungsu Kim, Seungu Han et al. (+1 more)
Speaker anonymization (SA) aims to conceal speaker identity while preserving linguistic content and emotion, but existing streaming systems often lose emotion information through discrete content bottlenecks or require emotion-specific modeling. We present LiveAnon, a voice-conversion-based streaming SA system that directly conditions an autoregressive codec
language model on hidden features from a pretrained streaming ASR encoder, preserving emotion without emotion-specific supervision. To reduce codec-generation latency, we introduce capped RVQ decoding, which preserves dependencies among coarse codebooks while predicting the remaining codebooks in parallel. Under VPC24, LiveAnon improves intelligibility and emotion preservation over fixed-target Stream-Voice-Anon (SVA), a strong prior streaming SA system, matches emotion-prompted SVA in emotion preservation, and achieves the strongest semi-informed
privacy among streaming systems. Under VPC26, it ranks second across all metrics among the evaluated anonymization systems, demonstrating a strong
privacy-utility balance.
Read the full abstract ▾
All authors Yejin Kim, Kyungsu Kim, Seungu Han, Kyogu Lee
Machine Learning
ANTE: Anticipate, Then Execute – Learning Internal World Models for Coding Agents from Verified Beliefs
Yi Ge, Kangrui Ruan, Jifan Chen et al. (+6 more)
Rapid advances in large
language models have made software engineering
agents capable of tackling complex coding tasks. Within a containerized environment, a coding
agent receives tool responses or execution results, and determines its later actions by those earlier results. However, most existing coding
agents never look ahead: their policies do not anticipate the consequences of actions, and the terminal reward barely attributes a failure to certain earlier steps, so an ill-judged edit can corrupt the repository state irreversibly before any signal arrives. Recent code world models aim to address these challenges, but they are either trained and then frozen before reinforcement learning (RL), or reduced to an auxiliary next-
token loss on the environment's outputs. We present ANTE (Anticipate, Then Execute), an agentic RL framework that verifies the policy's pre-action predictions by real execution and turns them into per-step credit for its actions and trust-or-abstain decisions. With an internal belief stated before each action, Belief-Anchored Policy Optimization (BAPO) turns the settled predictions into per-step advantages by anchoring credit on the policy's own forecasts. To turn the trained forecasts into better actions at inference, thereby facilitating effective test-time scaling, ANTE-TTO (test-time optimization) selects among candidate actions without an external scorer. ANTE raises Qwen3-32B from 25.0% to 57.2% pass@1 on SWE-bench Verified, 6.2% above a matched GRPO baseline trained without the belief state; its world model's accuracy climbs from 44% to 69% during RL, and the learned foresight capability transfers across harnesses and out-of-distribution domains. In summary, we aim to build a code world model that knows what has happened, what holds now, and what the next action will achieve.
Read the full abstract ▾
All authors Yi Ge, Kangrui Ruan, Jifan Chen, Manish Prajapat, Christian Bock, Xiaoyang Wang, Rein Houthooft, Aritra Sengupta, Varun Kumar
Machine Learning · Cross-listed from cs.AI
TRACE: Temporal Risk Assessment and Concept Editing for Safer Language Generation
Ankur Garg, Xuemin Yu, Hassan Sajjad et al. (+1 more)
Large
language models (LLMs) can produce harmful content while generating a response. A useful safeguard should not only detect this risk during generation but also intervene while leaving safe responses largely unchanged and provide an auditable record of the internal patterns supporting its decisions. We propose TRACE (Temporal Risk Assessment and Concept Editing), a framework that tracks harmful-response risk during generation, selectively edits
token activations, and leaves an auditable trace of its detection and steering decisions. TRACE learns a codebook of recurring activation patterns, with each codebook vector representing a latent concept. At inference time, it assigns each
token activation to the nearest codebook vector to obtain the corresponding concept. A gated recurrent unit (GRU) processes the activations and their assigned concept features to predict completed-response harmfulness and a monotone cumulative streaming risk. During steering, activations assigned to harmful concepts are steered along directions defined by nearby benign targets. An external LLM labels the learned concepts post hoc, making detection sequences and harmful-to-benign edit records interpretable. Experiments across three generators and the WildGuard and S-Eval
benchmarks show that TRACE improves streaming F1 over Kelp in five of six comparisons while remaining competitive on completed-response detection. TRACE's selective steering reduces Harmbench attack success with little change in refusal on originally safe responses. The post-hoc audit reveals recurring semantic structure in the learned concepts and distinct traces associated with harmful detection, safe generation, and activation edits.
Read the full abstract ▾
All authors Ankur Garg, Xuemin Yu, Hassan Sajjad, Samira Ebrahimi Kahou