Machine Learning
Learning Local Heterogeneity and Cross-Region Context for Large-Scale Traffic Forecasting
Qi Feng, Zidong Wang, Bo Li et al. (+4 more)
Traffic flow
forecasting is essential to intelligent transportation systems. Large-scale traffic
forecasting requires jointly modeling local spatial dependencies and cross-region context.Spatial dependencies between geographically neighboring nodes are heterogeneous due to differences in road identity and travel direction, while acquiring global information through allpairs node interactions incurs substantial computational costs. Therefore, capturing local heterogeneity while efficiently acquiring long-range context remains an important challenge in largescale traffic
forecasting. To address these challenges, we propose LoReST, a Local-Region Spatial Temporal network that models spatial dependencies at two complementary granularities: node neighborhoods and road network regions. Specifically, relation-aware local aggregation captures heterogeneous dependencies within geographic neighborhoods through road and direction specific feature transformations. Cross-region interaction constructs region representations through mean pooling, exchanges long range context via inter-region
attention, and broadcasts it back to nodes. By integrating local information aggregation with crossregion interaction, LoReST is able to effectively achieve spatial dependency learning in large-scale road networks. Experiments on four
datasets of the LargeST
benchmark show average relative reductions of 4.78%, 3.60%, and 5.75% in MAE, RMSE, and MAPE, respectively.
Read the full abstract ▾
All authors Qi Feng, Zidong Wang, Bo Li, Xiaoguang Gao, Jiayu Zhang, Chenfeng Wang, Kaifang Wan
Machine Learning
Pheno-GS: Phenoscape-scale Geodesic Sinkhorn
Alistair Wilkinson, Christopher J. Tape and Smita Krishnaswamy
High-throughput single-
cell data is now collected across large patient cohorts. Understanding patient-level heterogeneity from cellular-level data motivates phenoscaping:
embedding each single-
cell distribution as a "datapoint," with distances given by optimal transport (OT). Computing geometry-aware OT at this scale, between all pairs of patient
datasets, remains an open challenge, since existing methods either rely on Euclidean ground metrics that distort
manifold structure or fail under sparse, unevenly sampled, or large-scale data. We present Pheno-GS (Phenoscape-scale Geodesic Sinkhorn), which computes accurate, scalable geodesic transport distances under noisy, unbalanced, large-scale settings via three components: ($1$) graph connectivity regularization for well-defined geodesics on sparse/disconnected
manifolds; ($2$) an unbalanced OT formulation via KL marginal penalties; and ($3$) a batched
matrix algorithm computing all pairwise distances in one heat
diffusion (over $200 \times$ faster than Geodesic Sinkhorn for $500$ distributions). We validate Pheno-GS on synthetic
benchmarks and a CyTOF perturbation
dataset.
Read the full abstract ▾
Machine Learning
Efficient Linear Bandits via Cluster-Aware Sketching
Hantao Yang, Hong Xie and Defu Lian
We study the problem of computational efficiency for linear bandits in high-dimensional settings with a finite arm set. In linear bandits, the increase in the dimension $d$ of the feature vectors leads to growing computational costs of $O(d^2)$ at each round of update. Traditional sketching-based methods such as SOFUL reduce computation via fixed-size
matrix sketching, yet run the risk of incurring vacuous linear regret when the spectral tail of the data is heavy and the sketch size is inadequately selected. To guarantee regret convergence and effectively reduce computational costs, we introduce a clustering mechanism and propose the Cluster Sketch Linear Bandit (CS-LB)
algorithm. Our method preserves the full covariance information in each cluster to guarantee robust sublinear regret without spectral-tail vulnerabilities, performs cluster switching by assigning a sentinel for each cluster, and reduces per-round update computation to $O(l^2d)$ via a tunable sketch size $l<d$. Experiments on synthetic
datasets demonstrate that our method consistently maintains a favorable trade-off between efficiency and regret.
Read the full abstract ▾
Machine Learning
Hidden not Deleted: How Networks Suppress Entangled Features
Akash Samanta, Manish Pratap Singh and Debasis Chaudhuri
Concept erasure methods that operate via linear projection assume that features occupy separable subspaces. We show this assumption fails under dense superposition: when two features are forced into an antipodal pair sharing a single subspace, state-of-the-art linear erasure destroys both, not just the target. Networks trained with gradient descent instead solve this problem non-linearly, but not uniformly: they converge to one of two distinct
circuit-level solutions depending on initialization, which we call mirror and shadow solutions. We map this bifurcation as a function of feature
entanglement, show it reflects a stable attractor structure rather than an artifact of our setup, and use targeted causal interventions to demonstrate that both solutions leave a substantial, measurable trace of the erased feature's representation intact, recoverable through a single scalar patch rather than requiring any further training. This mirrors a failure mode recently observed empirically in LLM unlearning, where suppression rather than deletion allows forgotten knowledge to resurface; our results offer a mechanistic, causally-validated account of why that failure mode occurs.
Read the full abstract ▾
Machine Learning
The Capability Manifold and ML Scaling Laws
Syed Ali Raza Zaidi and Maryam Hafeez
Existing machine learning (ML) scaling laws relate predictive loss to compute, model parameters, and data. However, as models are increasingly deployed through agentic harnesses, loss alone is insufficient to characterize downstream performance: models with similar loss can exhibit different capabilities in
reasoning,
retrieval, planning, and adaptation. Yet, no unified framework connects such capabilities to the coupled resources available across the ML lifecycle. We bridge this gap by introducing a capability
manifold, a multidimensional framework mapping downstream capabilities to pre-training, post-training, and test-time resources through bounded scaling functions. Analytical Jacobians quantify capability sensitivity to resource changes and interactions. As an initial application, we embed Kaplan- and Chinchilla-type scaling laws and test-time compute within the framework, demonstrating how existing scaling relationships can be unified as trajectories on a common capability
manifold.
Read the full abstract ▾
Machine Learning
Does Step Law Transfer to Small-Scale Language Models? An Empirical Recalibration Below 59M Parameters
Egor Romanyukov, Timofey Novikov, Timur Shokarov et al. (+3 more)
Step Law gives power-law formulas for the optimal peak learning rate eta* and batch size B* when pre-training
language models. It was calibrated on models between 59M and 1B parameters; the small-model regime N < 59M was never tested empirically by its authors. This regime matters for single-GPU training, interpretability research, educational experiments, and settings where larger models are infeasible on
memory or cost grounds. We test whether Step Law transfers to small
language models. We consider three outcomes: H1, the original coefficients work directly; H2, the power-law form holds but with different coefficients; and H3, a power law does not describe the optima in this regime. All experiments use a single nanoGPT/TinyStories pipeline with a 2048-
token BPE vocabulary, AdamW, and a warmup-cosine schedule. The optimum for each (N, D)
cell is extracted from the loss surface L(eta, B) via a local quadratic approximation in log-log coordinates over the smoothed training loss. The final
dataset contains 29 unique (N, D)
cells and 935 analysis-ready runs. The main refit uses 25
cells (815 runs) in the working range 4 <= D/N <= 600. On the pooled data we accept H2: the functional form is preserved, but the coefficients differ from the original. We obtain eta*(N, D) = 0.0985 N^(-0.508) D^(0.238) (R^2 = 0.834) and B*(D) = 3.6 x 10^(-4) D^(0.931) (R^2 = 0.950). Step Law's structural claim that B* is independent of N is reproduced (p = 0.87), but the growth of B* with D is nearly twice as steep as in the original work. Direct transfer of Step Law systematically overestimates the optimal learning rate: the median ratio eta_SL / eta* is approximately 4.0x, with a range of 2.4x to 6.6x.
Read the full abstract ▾
All authors Egor Romanyukov, Timofey Novikov, Timur Shokarov, Elizaveta Zorkina, Anastasia Palienko, Stepan Dergachev
Machine Learning
VCMM: Variance-Calibrated Momentum for Multimodal Learning
Zhongjing Gu, Chenyang Huang, Yufa Feng et al. (+3 more)
Multimodal joint training often suffers from modality imbalance, where a dominant modality suppresses the optimization of others. Existing methods mainly balance modality learning by modulating gradient magnitudes or directions, modifying optimization objectives, or adjusting training strategies, with most interventions focusing on the current update. However, when combined with widely used momentum-based optimizers, the update also incorporates accumulated information from previous gradients, which is not explicitly addressed by current-step modulation alone. To address this issue, we propose Variance-Calibrated MomentuM (VCMM), which adapts gradient
memory to modality-specific gradient dynamics. Specifically, VCMM estimates minibatch noise and temporal drift online and uses their relative strength to determine modality-specific momentum through a Kalman-inspired controller. We further center the control signal across modalities and apply exact bias correction for the time-varying first moment, enabling adaptive gradient
memory without extra network passes or explicit learning-rate scaling. Experiments on four
multimodal benchmarks demonstrate consistent improvements with modest training overhead.
Read the full abstract ▾
All authors Zhongjing Gu, Chenyang Huang, Yufa Feng, Chong He, Qinxu Ding, Yiming Cui
Machine Learning
DCRL: Decoupling and Coupling Reinforcement Learning via Policy-Reward Manifold Alignment
Henan Sun, Zehua Li, Haitao Hu et al. (+4 more)
Reinforcement learning (RL) has emerged as a key paradigm for improving the
reasoning capabilities of large
language models (LLMs). However, existing reward systems, such as rule-based and reward-model-based, often exhibit issues such as unstable optimization and reward hacking. In this work, we revisit the general
reasoning of LLMs from a
geometric perspective, conceptualizing it as a coupled
manifold composed of three interdependent sub-
manifolds: logical deduction, evaluation, and representation. Based on this perspective, response generation in RL can be interpreted as a decoupling process from the evaluation
manifold, while reward estimation corresponds to a decoupling process from the logical deduction
manifold. The limitations of rule-based and reward-model RL systems can be geometrically interpreted as the mismatch of policy-reward
manifolds during RL process. To address the aforementioned misalignment, we propose Decoupling and Coupling Reinforcement Learning (DCRL) framework, which incorporates two key components: (1) a syllogistic logic-based
prompt evolution mechanism that dynamically refines reward rubrics to enhance the expressiveness of the reward
manifold; and (2) a policy-reward re-coupling mechanism that jointly updates the reward and policy models, ensuring consistent evaluation and mitigating
manifold mismatch during training. Theoretical analysis and extensive experiments across multiple
reasoning domains demonstrate that DCRL consistently outperforms both rule-based and reward-model baselines. Notably, a Qwen3-4B model trained under DCRL surpasses a Qwen3-32B baseline and approaches the performance of a Qwen3-235B model, highlighting superior effectiveness and generalization in RL.
Read the full abstract ▾
All authors Henan Sun, Zehua Li, Haitao Hu, Qifan Zhang, Jianfeng Zhang, Nuo Chen, Jia Li
Machine Learning
TNLearn: An Open Source Python Package for Task-based Neurons
Meng Wang, Tieyun Li, Juntong Fan et al. (+5 more)
The
brain does not rely on a single type of
neuron to perform all kinds of tasks; instead, it designs different
neurons for different tasks. The concept of task-based
neurons represents a paradigm shift compared to task-based architectures. It argues that solving a specific problem requires customized
neurons, as task-based
neurons capture useful prior knowledge from task-related data. To facilitate the use of task-based
neurons in scientific research and industrial applications, we introduce TNLearn, an open-source Python package that provides automated construction of task-based
neurons and networks, enabling smooth training of task-based networks. Comprehensive documentation, including technical exposition, API reference, and representative examples, is available online. TNLearn is open-sourced at https://github.com/NewT123-WM/tnlearn and has become a PyTorch ecosystem project.
Read the full abstract ▾
All authors Meng Wang, Tieyun Li, Juntong Fan, Hanyu Pei, Jing-Xiao Liao, Yaodong Yang, Jianwei Ma, Fenglei Fan
Machine Learning
PhyMo: A Physical-Field Modality for Multimodal AI4Physics
Henan Sun, Haitao Hu, Jin Liu et al. (+4 more)
Multimodal learning is emerging as a powerful paradigm for AI for Physics (AI4Physics), where predicting physical systems requires the joint interpretation of heterogeneous observations, measurements, and domain knowledge. However, existing approaches typically represent physical quantities and governing equations as generic numerical or textual
tokens, overlooking the physical constraints that determine their spatiotemporal interactions. To address this limitation, we introduce the physical-field modality and propose PhyMo, a physics-grounded
multimodal framework that organizes heterogeneous measurements through PDE-associated operators. PhyMo follows a three-stage learning procedure: the physical-field encoder is first pretrained through field reconstruction under PDE residual supervision, its representations are subsequently aligned with visual
embeddings in a shared latent space, and the fused
multimodal representations are finally processed by corresponding downstream prediction heads. Experiments on five
datasets spanning diverse physical environments show that PhyMo achieves state-of-the-art performance, compared to the strongest baseline on each
dataset, demonstrating the superiority of PhyMo on
multimodal representation learning in AI4Physics.
Read the full abstract ▾
All authors Henan Sun, Haitao Hu, Jin Liu, Jianfeng Zhang, Lujia Pan, Nuo Chen, Jia Li
Machine Learning
EBRL: Asynchronous Embodied RL by Multi-Grained Resource Management
Liang Mi, Weijun Wang, Bowen Gao et al. (+12 more)
Embodied reinforcement learning (RL) improves model capabilities with a pipeline of environment simulation, action generation, and model updates. These stages show heterogeneous CPU and GPU demands, making efficient resource utilization difficult. Recent systems overlap rollout (simulation and generation) with training for efficiency, but exclusive GPU allocation and synchronized barrier in rollout still leave substantial hardware resource waste. In this paper, we present EBRL, an asynchronous
embodied RL training system with two core techniques. The asynchronous pipelined scheduler overlaps rollout and training, pipelines simulation and generation across environment groups, and carries out each environment independently, eliminating synchronization stalls. The fine-grained resource manager pools CPU cores and GPU streaming multiprocessors, and uses stage profiles and runtime feedback to adjust resource quotas and batch sizes to meet the shifting demands among stages. We implement EBRL on RLinf and evaluate it with four
embodied policies and four simulation
benchmarks across heterogeneous GPU testbeds. Experiments show that EBRL achieves 1.30-3.47 times the end-to-end rollout throughput and 2.5 times of training convergency compared to the SOTA
embodied RL systems.
Read the full abstract ▾
All authors Liang Mi, Weijun Wang, Bowen Gao, Tianze Yu, Zixu Hao, Han Xiao, Xin Ding, Mingzhe Huang, Xin He, Lu Shi, Hao Wu, Haipeng Dai, Guihai Chen, Yunxin Liu, Ting Cao
Machine Learning · Cross-listed from math.ST
Robustness of Diffusion Models under Distribution Shift
Wei Luo, Neil K. Chada, Shijie Zhang et al. (+1 more)
Score-based
diffusion models are increasingly considered in settings where the underlying data distribution may differ from the training distribution, yet existing theoretical guarantees largely focus on the no-shift setting. In this work, we study robust score estimation under Wasserstein perturbations of a reference distribution. For the Ornstein–Uhlenbeck
diffusion, we show that robust estimation decomposes into two fundamental components: the statistical cost of learning the reference distribution and the intrinsic cost of distribution shift. The latter scales quadratically with the Wasserstein radius, and this dependence is minimax optimal. We construct an explicit finite-sample
estimator achieving the resulting robust minimax rate without knowing the shift radius. When the reference distribution lies on an unknown low-dimensional subspace, the statistical term adapts to the intrinsic dimension while the shift cost remains unchanged. Finally, we show that the same decomposition governs positive-time reverse sampling and obtain matching minimax guarantees in KL divergence. Together, these results characterize how finite data, intrinsic dimension, and distribution shift affect the robustness of score-based
diffusion models.
Read the full abstract ▾
All authors Wei Luo, Neil K. Chada, Shijie Zhang, Lu Yu
Machine Learning
ProCredit: From Outcome Rewards to Progress Credit in Agentic Reinforcement Learning
Ming Ma, Yi Zhu, Yiran Zhong et al. (+7 more)
Long-horizon agentic tasks require an
agent to modify an environment through a sequence of tool calls, with success determined by the final state. The standard recipe assigns a single outcome reward at the end and compares trajectories sampled for the same task. As a result, a group with no successful trajectory yields no training signal, failed attempts cannot be told apart by how close they came to completion, and turns that advance the task receive the same credit as turns that only query the environment. Prior work refines the unit of comparison from the trajectory to the step, or trains a reward model to supply intermediate signal: the former still derives its signal from final success alone, and the latter estimates it with a model. We observe that the acceptance checks that decide success can also be run on intermediate states, so progress is as verifiable as the outcome. We propose ProCredit, which turns this verified progress into credit: it reruns the acceptance checks after each turn, rewards the turn by its change in progress, and uses these rewards to assign credit both across attempts at the same task and across the turns within a trajectory. Starting from Qwen3.5 base models at three scales on AppWorld, ProCredit outperforms outcome-reward baselines and progress-based baselines in task completion rate at every scale on both test sets, exceeding the strongest outcome-reward baseline by 4.1 percentage points at 4B, and results in a second environment show the same direction of improvement. Ablations show that adding the final progress to the trajectory score alone does not improve performance: the gain comes from crediting progress to the turn where it occurs.
Read the full abstract ▾
All authors Ming Ma, Yi Zhu, Yiran Zhong, Feida Zhu, Chonghan Liu, Pengkun Jiao, Qichao Wang, Yanhao Jia, Tianming Yang, Steven Hoi
Machine Learning · Cross-listed from cs.CV
M3D-Net: Hierarchical Coordination of Spatial Context, Feature Reuse, and Differential Attention for Mammography Classification
Zheng Yu, Xinhang Li, Jiabao Gao et al. (+2 more)
Breast image classification requires local detail and global tissue context, yet these cues can weaken as representations deepen. We present M3D-Net, a mammography encoder that hierarchically coordinates multi-scale coordinate
attention, bounded dynamic feature reuse, and differential
attention through resolution-aware operator placement. Within-stage
retrieval preserves access to earlier features, coordinate-aware aggregation integrates local and global context, and differential
attention operates at coarse resolutions. We evaluate image-only classification on AISSLab mammography and an adapted image–clinical model on BrEaST ultrasound. Against EdgeNeXt, RepViT, and TransXNet, the proposed implementations achieve the highest recorded validation accuracy and late-training accuracy, with the lowest endpoint cross-
entropy loss. Validation accuracies reach 97.78% and 80.39%, respectively. These results support further evaluation of hierarchical coordination across breast imaging settings; repeated-seed, component-controlled, and independent evaluations remain necessary.
Read the full abstract ▾
All authors Zheng Yu, Xinhang Li, Jiabao Gao, Boyang Wang, Xiang Li
Machine Learning · Cross-listed from cs.AI
WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents
Jingjie Ning, Xueqi Li, Yibo Kong et al. (+1 more)
AI research
agents must predict the effects of computational changes after budgeted experiments. WhatWorkedBench evaluates this experimental understanding through a delivered response surface of configuration scores.
Agents inspect workflow code and buy measurements; exhaustive CPU references score conditional component effects, mean pair interactions, configuration choice, and delivery. The catalog spans 36 task conditions, 30 sources, 8 workflow families, and 1248 indexed configuration records, with 4,206 numerical controls. At eight purchased measurements plus two free anchors, pair-effect ridge selects an exact optimum on 15 of 22 sources; 13 of these cases have at least one conditional-effect error exceeding 10% of the task utility range. Shared-
estimator comparisons measure acquisition and reconstruction on common observations. In a prospective typed study on 12 four-factor sources, DeepSeek Flash submits 12/12 direct tables and gains 0.147 recovery over a Gaussian process (GP) fitted to the same observations. Pro delivers 11/12 artifacts, with an all-attempt GP difference of -0.001 and a delivered-only difference of +0.063. On six prespecified new
agent-evaluation sources, Flash and Pro gains are 0.149 and 0.074. In eight typed six-factor episodes, seven final tables satisfy verified code equivalences. Four fresh
agents pass all six registered rules through named
estimators and deliver consistent tables at 0.682 recovery versus 0.710 for separate direct-table runs. WhatWorkedBench links acquisition, inference, program structure, and delivery.
Read the full abstract ▾
All authors Jingjie Ning, Xueqi Li, Yibo Kong, Dongting Li
Machine Learning
Learning Where to Look: A Shared Relative-Alignment Module for Time-Series Forecasting and PPG-to-Vital-Sign Reconstruction
Ragamayi Puli and Shunya Nagashima
PPG-to-vital-sign reconstruction turns a wrist-worn photoplethysmogram into clinical waveforms such as the ECG. Long-horizon multivariate time-series
forecasting underpins planning in energy, weather, and traffic. Both generate a target sequence from a condition sequence, and current models hard-code where each target position reads it, as a same-position copy or seasonal recurrence, so neither transfers between tasks. We propose ROOSTER, one conditioning module that handles vital-sign reconstruction and time-series
forecasting alike by learning this correspondence. Its core is a periodic-comb bias over the target-condition offset whose center, period, and sharpness are learned per head, so one module settles on the identity
alignment or a seasonal lag and reports which it found. On vital-sign reconstruction from PPG, ROOSTER outperformed the published baselines on four heart-rate and respiratory-rate
benchmarks. On multivariate time-series
forecasting, it achieved the best horizon-averaged MSE on four
benchmarks and outperformed the
forecasting model it extends on 20 of 24
dataset-horizon settings under matched three-seed training. An ablation study indicated that the relative bias, not content matching, carried the
alignment.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Video
Taeyoun Kwon, Seungjin Kim, Hyeonyu Kim et al. (+1 more)
Recent
video-
language models increasingly adopt hybrid architectures that interleave linear and full
attention layers for efficient long-context processing. While the recurrent state of linear
attention remains fixed in size, the KV cache of full
attention continues to grow with the
video stream, making eviction necessary under a bounded
memory budget. The key challenge in streaming is that eviction must occur before the question arrives, so what to retain has to be decided without the question. Existing eviction methods derive
token scores from the KV cache itself, using position,
attention, or key-value representations, and
attention-based scores further require proxy queries or extra computation. Hybrid backbones offer another source of signal. In gated-delta linear
attention, the recurrent state is updated by the residual between each input and what can already be retrieved from the state, so its change over a chunk of frames reflects how much new information the chunk brings. We propose DeltaS, a query-agnostic, training-free method that retains
video chunks inducing larger normalized state change, or state drift. In a controlled comparison with the budget and retention policy held fixed, state drift outperforms position-,
attention-, and key-value-based signals. With a signal costing only 1.9% of the forward pass, DeltaS surpasses the strongest query-agnostic bounded-
memory baseline by 2.1 points on average across six long-
video benchmarks and by 5.6 points on the longest
benchmark. These results suggest that the two
memories of hybrid architectures can work cooperatively. Code is available at https://github.com/MaumAI-Company/DeltaS.
Read the full abstract ▾
All authors Taeyoun Kwon, Seungjin Kim, Hyeonyu Kim, Moon Hwan Kim
Machine Learning
Quantum Reinforcement Learning for Cost and Delay Tradeoffs in Quantum Cloud Orchestration
An N. H. Phan, Dang Van Huynh, Muhammad Usman et al. (+1 more)
Quantum cloud computing, delivered through the
quantum-as-a-service (QaaS) model, provides access to
quantum computing resources. However, applying uniform time-based pricing across fundamentally heterogeneous
quantum resources significantly complicates task orchestration, particularly when addressing the tradeoff between execution costs and system performance. While heuristic methods rely on predefined scheduling rules, classical deep reinforcement learning (DRL) models may require more trainable parameters in this setting. Motivated by the potential of parameterised
quantum circuits (PQCs) as compact function approximators, we propose QRLQ, a cost-delay-aware
quantum cloud scheduling framework integrating PQCs with a dueling double deep Q-network (D3QN) to dynamically account for both cost and delay. Our simulation results show that QRLQ achieves lower mean cost and delay than the heuristic baselines, achieving a 5-11% lower mean cost relative to availability-based and rotation-based heuristics and reducing mean delay by 17% and 82% relative to the strongest and weakest heuristic baselines, respectively, while retaining execution fidelity within 2% of a fidelity-greedy policy. Compared with the classical DRL baseline, QRLQ achieves comparable scheduling performance while using 72% fewer trainable parameters. This work explores the feasibility of using QRL for task orchestration in
quantum cloud environments and demonstrates its potential for cost-delay-aware
quantum resource management.
Read the full abstract ▾
All authors An N. H. Phan, Dang Van Huynh, Muhammad Usman, Hoa T. Nguyen
Machine Learning
Stable Neural Decoding Across Sessions via Task-Conditioned Latent Alignment for Brain-Machine Interfaces
Canyang Zhao, Bolin Peng, J. Patrick Mayo et al. (+2 more)
Achieving stable long-term neural decoding in invasive
brain-machine interfaces (BMIs) remains challenging due to variations in recorded neural populations across sessions. Current latent
alignment approaches may overlook task-dependent structure during cross-session adaptation. We propose Task-Conditioned Latent
Alignment (TCLA), a framework that stabilizes neural decoding by learning a shared latent space. TCLA learns a low-dimensional source representation using neural reconstruction and continuous behavioral supervision. During target-session adaptation, the shared representation is fixed, while target neural activity is mapped into the source latent space by aligning source and target distributions separately for each task condition. We evaluated TCLA on seven nonhuman primate
datasets spanning multiple tasks. In long-term cross-session evaluation, TCLA achieved a mean $R^2$ of $0.476\pm0.014$ with a negative $R^2$ failure rate of only 6.8%. Across 1,356 within-subject session pairs, TCLA achieved a mean $R^2$ of $0.371\pm0.009$ with a failure rate of 6.8%. Across 2,134 cross-subject session pairs, TCLA achieved a mean $R^2$ of $0.218\pm0.004$ with a failure rate of 12.9%, substantially better than those of the comparison methods. These results demonstrate that by preserving behaviorally relevant and task-dependent latent structure, TCLA improves the robustness of neural decoding across recording sessions and subjects. The source code is publicly available at \href{https://github.com/FAMD-CASIA/TCLA}{https://github.com/FAMD-CASIA/TCLA}.
Read the full abstract ▾
All authors Canyang Zhao, Bolin Peng, J. Patrick Mayo, Ce Ju, Bing Liu
Machine Learning
Counterfactual Constraint-Conditioned On-Policy Distillation for Multi-Constraint Instruction Following
Yanzhao Zheng, Yuanqiang Yu, Tianze Xu et al. (+6 more)
Multi-constraint instruction following requires a model to respond to a query under many simultaneously active constraints. Even strong instruction-tuned models still routinely violate some of them. Existing approaches either augment supervision with sequence- or
token-level RL rewards from external verifiers or learned graders, or use on-policy
distillation (OPD) against a single full-context teacher whose probability mass becomes diluted as more constraints become simultaneously active. We propose CC-OPD (Counterfactual Constraint-Conditioned On-Policy
Distillation), which inverts the standard supervision-generation direction in
distillation. Rather than enriching the teacher with information beyond what the student sees, CC-OPD ablates each constraint from the teacher's conditioning in turn, and constructs the per-constraint signal from the resulting per-
token probability differentials. The resulting per-
token leave-one-out log-likelihood shifts are summed, clipped, and added to the vanilla OPD reward as a
token-level shaping term. All shaping terms are obtained from the frozen teacher, without an external verifier during
distillation, and the reward equals vanilla OPD wherever the aggregate shift is zero. Across two Qwen model pairs and seven
benchmarks, CC-OPD achieves the highest average among all evaluated student-training methods. A 1.5B student trained with CC-OPD surpasses its own 7B RL-trained teacher on the MulDimIF
benchmark.
Read the full abstract ▾
All authors Yanzhao Zheng, Yuanqiang Yu, Tianze Xu, Chao Ma, Zhentao Zhang, Jihuai Zhu, Baohua Dong, Hangcheng Zhu, Ruohui Huang
Machine Learning
When Labels Are Scarce: An Oscillatory State Space Model for Vibration Diagnosis
Mainak Mallick and Seung-Kyum Choi
Machine fault diagnosis from vibration requires learning from scarce labelled fault recordings while meeting the computational constraints of edge devices for local inference. We introduce DualRes, a compact oscillatory state-space model that combines two complementary spectral views of vibration, capturing rapid changes and fine frequency structure. Time-aligned views are processed by selective oscillatory
memory, which learns how long to retain temporal patterns. The encoder contains 39,528 parameters. We evaluate supervised learning across six bearing
datasets and a gearbox
benchmark, with an additional gearbox pilot. Recording-level splits and explicit accounting of labelled duration distinguish data efficiency from repeated exposure to correlated samples. On the main gearbox
benchmark, DualRes achieves state-of-the-art performance among the nine evaluated methods at six of seven label budgets. With about six labelled seconds per class, it improves macro-F1 by 16.1 percentage points over the next strongest comparator. On the same
benchmark, DualRes achieves a 1.44-fold recording-level speedup and a 24.8-fold reduction in checkpoint storage relative to a selective state-space baseline under matched hardware and runtime conditions. Bearing results reveal task-dependent trade-offs. These findings support oscillatory
memory as a compact approach to vibration diagnosis under limited labelled exposure.
Read the full abstract ▾
Machine Learning
Active Learning for Biodiversity Monitoring: From Label Efficiency to Reliable Ecological Inference
Ben McEwen, Shiqi Zhang and Dan Stowell
Limited expert annotation capacity is a pervasive constraint in biodiversity monitoring. Passive acoustic recorders and camera traps generate data faster than experts can analyse them. Machine learning (ML) models can process these data at scale, but their reliability depends on the quality, quantity, and coverage of labelled samples, so expert time remains a constraint. Active learning (AL) eases this bottleneck by selecting, under a fixed annotation budget, the samples expected to improve a model most, and published evidence shows it can reduce the labels needed to reach a target performance. Monitoring programmes, however, face a broader question: how should a limited expert budget be divided so that model training, validation, and the ecological estimates built on model outputs all remain reliable? Because AL selects samples non-randomly, its labels are unsuitable for validation, calibration, or threshold selection, a tension rarely acknowledged. We synthesise AL research across acoustic and image modalities and identify gaps and opportunities. Most studies evaluate query strategies on pre-labelled
benchmarks with simulated annotators; deployments in real monitoring workflows are rare and concentrate on birds and cetaceans. Bats, insects, amphibians, and fish are underrepresented, and
multimodal applications remain largely unexplored. Evaluation centres on headline reductions in annotation effort, often without random-sampling baselines, per-class results, or calibration analysis, and rarely accounts for the labels required for validation. We provide a tutorial treatment of the AL loop that makes these budget decisions explicit, and a roadmap towards AL methods that support label-efficient training, validation, and trustworthy downstream ecological inference.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
EvoAudio: Recursive Self-Improvement for Audio Understanding
Yuxiang Wang, Shengbo Cai, Yingda Shen et al. (+6 more)
Audio
language models understand what is said far better than how it sounds. Closing this gap takes more than data. Detailed acoustic annotation is costly, labels from stronger models inherit their errors and limits, and fixed data cannot adapt as the learner improves. We therefore propose EvoAudio, a recursive self-improvement system for audio understanding. To our knowledge, it is the first to evolve the model, waveforms, questions, and difficulty in one closed loop. EvoAudio uses the current model's performance to set the focus and difficulty of the next training data. A library of audio tools then constructs questions whose answers follow from how the audio was made, providing verifiable supervision without new human annotation. Reinforcement learning updates the model, and validation decides whether it enters the next
evolution round. Across 13 rounds, EvoAudio improves five models with different audio encoders and
language backbones on MMSU, MMAU-Pro, and MMAR. It achieves the highest average for every backbone, raising overall performance by up to 6.3 points. The improvement unfolds over successive rounds, with each stronger model starting the next round.
Read the full abstract ▾
All authors Yuxiang Wang, Shengbo Cai, Yingda Shen, Ming-Hao Hsu, Qinke Ni, Liqiang Zhang, Teddy Sun, Steve Yevs, Zhizheng Wu
Machine Learning
Forecast Workflow Bench: Evaluating Language-Model Decisions with Budgeted Forecast Tools
Shunya Nagashima
Time-series foundation models (TSFMs) provide forecasts for operational decisions, but accuracy alone does not determine their value. Evaluating
agents that use these models requires measuring decision quality and forecast cost. FWBench evaluates this capability on 1,251 electricity and cycle-hire cases using fixed forecast tools and simulated capacity contracts.
Agents select models, histories and horizons, then submit capacities to minimize a stated loss-cost objective. We evaluated two hosted and eight local configurations, including small
language models, and tested local models with and without TSFMs. GPT-6 Astra bought inexpensive short-horizon forecasts selectively, using 2.5% of the budget, and outperformed fixed policies when the saved decisions were scored with three loss-cost weightings. FWBench enables reproducible evaluation of how
language models select and use time-series forecasts to make decisions under cost constraints.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Models
Ke Wan and Chen Chen
Recurrent-depth
language models, such as looped
Transformers, repeatedly apply shared network blocks to refine latent representations without generating explicit intermediate
reasoning tokens. However, each step recomputes full
attention over the entire context, repeating costly global routing. We study how
attention routing evolves across recurrent depth and find a consistent separation in convergence timescales:
attention support and distributions stabilize substantially earlier than hidden states and
attention outputs. This suggests two stages of recurrent inference: early discovery of a sparse working set, followed by representation refinement over largely stable routing support. Motivated by this finding, we introduce WISE (Working-set Inference with Support Exploitation), a training-free method that uses unrestricted
attention during early recurrent steps to discover a block-structured working set, then reuses its support in later steps while keeping
attention weights and recurrent refinement dynamic. Controlled interventions show that multi-step discovery yields more effective working sets than first-step selection, and that support reuse better preserves model behavior than more restrictive forms of
attention reuse. Across multi-hop QA
benchmarks, WISE largely preserves full-
attention performance. Matched context-scaling experiments reveal an increasingly favorable quality-efficiency tradeoff as routing support becomes sparser with longer contexts. A sparse-
attention implementation achieves up to a 1.76x late-step
attention speedup over native FlashAttention at 4K context. Code: https://github.com/tbn5pj/WISE_code.
Read the full abstract ▾
Machine Learning
Anomaly-Free Self-Optimization via AUC Bounds
Kevin Wilkinghoff and Zheng-Hua Tan
Anomalies are rare, and anomalous data are often unavailable during development, making it difficult to determine which anomaly detection models and configurations will generalize to unseen anomalies. Recent approaches address this challenge by generating pseudo-anomalies and using bounds on the achievable area under the ROC curve (AUC) to select the optimal configuration from a finite set of candidates. Instead, we use the AUC bound as a differentiable, anomaly-free objective for directly optimizing continuous parameters of anomaly detection systems. We demonstrate this framework by optimizing ensemble weights and introducing a learnable score-rescaling mechanism that adapts pseudo-anomaly scores, enabling optimization beyond a predefined candidate set. Experiments across multiple
datasets and
embedding models show that AUC-bound optimization achieves significant performance gains over conventional model selection and prior development-set-based parameter selection. The results further show that direct optimization is less sensitive to the choice of pseudo-anomaly construction.
Read the full abstract ▾
Machine Learning
Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction
Jialu Wang, Jianing Deng, Shuqing Luo et al. (+6 more)
Unlearning ensures LLM compliance by removing the influence of private or copyrighted training data. However, since LLM models typically undergo post-training compression, like quantization, in practical deployment, it has been observed that the unlearning effect can be substantially weakened, with the forgetting behavior degrading more severely than that of model utility. This paper proposes a quantization-robust unlearning framework that makes forgetting robust to quantization while maintaining overall model utility. We analyze this gap through the lens of loss landscape. Specifically, our analysis reveals a
curvature-based criteria that pinpoints sensitive weights in the unlearned model that leads to both non-robust forgetting and reduced utility. We therefore propose sensitivity-guided noisy regularization, which is applied on the sensitive parameters to steer the model convergence towards a smoother minima of uniformly low forget and retain losses. Balancing unlearning and utility, we further propose forget-critical optimization, which updates only forget-critical layers, preserving most of the network to retain useful knowledge. Extensive experiments on the MUSE and TOFU
benchmarks across multiple LLM unlearning
algorithms show that our approach achieves substantially more quantization-resilient forgetting while maintaining utility. Code is available at https://github.com/jialuw0830/QRUnlearn.
Read the full abstract ▾
All authors Jialu Wang, Jianing Deng, Shuqing Luo, Yuanzhe Li, Dongwei Wang, Jingtong Hu, Huanrui Yang, Song Wang, Tianlong Chen
Machine Learning · Cross-listed from cs.AI
MolDesignBench: Evaluating LLM-based Agent for Scenario-grounded Molecular Design
Yongjun Jeong, Hanbum Ko, Ye Rin Kim et al. (+6 more)
Real-world molecular design remains challenging for large
language model (LLM)-based
agents. It requires them to interpret design contexts, satisfy multiple constraints, identify infeasible specifications, and reason over multi-step tool outputs. Existing
benchmarks do not capture this complexity, focusing instead on explicit and narrow constraints, only feasible problems, and single-path solutions. To address this gap, we propose MolDesignBench, a scenario-grounded
benchmark that more closely reflects real-world molecular design for evaluating tool-augmented LLM
agents. MolDesignBench comprises 2K generation and optimization instances that combine implicit requirements embedded in design narratives with explicit property and functional-group constraints, including infeasible cases, and require the effective use of 17 specialized chemistry tools. Experiments across diverse frontier LLMs reveal low success rates–with the best achieving only $\sim43$%–and frequent failures in implicit-constraint
reasoning, infeasibility detection, and tool
reasoning. The corresponding fine-grained failure-mode analysis identifies implicit constraint interpretation and infeasibility detection as the primary bottlenecks, establishing MolDesignBench as a rigorous testbed to guide future research on chemical
agents. The
benchmark, tool interface, and evaluation code are publicly available.
Read the full abstract ▾
All authors Yongjun Jeong, Hanbum Ko, Ye Rin Kim, Chanhui Lee, Rodrigo Hormazabal, Jaewan Lee, Sehui Han, Sungbin Lim, Sungwoong Kim
Machine Learning · Cross-listed from eess.SY
Evolving Inspectable O-RAN Slicing xApps with LLMs
Faezeh Dehghan Tarzjani and Bhaskar Krishnamachari
Open RAN (O-RAN) slicing xApps must adapt resource allocations to changing channel conditions and traffic demands while meeting service-level agreements (SLAs). Deep reinforcement learning can produce adaptive policies, but their allocation rules remain encoded in neural-network parameters. Our goal is to retain this adaptability while making the controller's decision logic directly inspectable and editable by operators. We use a large
language model (LLM) to evolve slicing controllers as compact Python programs whose decision logic remains readable and editable after optimization. The LLM proposes and revises candidates offline, while a calibrated simulator scores them, and the selected decision module runs unchanged in the O-RAN control path. On the NSF POWDER 5G testbed, the evolved controller releases resources from a guaranteed slice whose throughput target becomes unattainable under a sustained channel fade, improving best-effort throughput from 158.2 to 228.6 Mbps, a 44.5% gain over the best static allocation. Since the controllers are readable source code, their behavior can be predicted from their equations, defects can be diagnosed by reading the code, and calibration errors can be corrected with one-line edits, reducing SLA misses from 79.9% to 2.2% in one case and more than doubling fitness in another. In a four-slice trace-driven simulation calibrated to the same testbed, evolutionary search achieves higher average evaluation scores than independent prompting at a matched proposal budget, with mean normalized gains on held-out traces of 16.3% for prompting alone, 32.1% for
evolution from scratch, and 51.0% for
evolution from a starting program.
Read the full abstract ▾
Machine Learning · Cross-listed from math.NA
A Hybrid Iterative Deep Ritz Method for Elliptic Interface Problems
Tianhao Hu, Bangti Jin, Fengru Wang et al. (+1 more)
In this work, we propose a hybrid iterative deep Ritz method (H-IDRM) for a class of interface problems for second-order
elliptic operators. It is based on a new mixed formulation of the problem and involves solving a sequence of convex minimization problems. We employ a level-set neural network architecture, featuring a level-set representation of the interface, to accommodate the piecewise smoothness of the solution and the flux. The approach involves only volumetric representations instead of duality pairing on the interface and avoids explicit interface sampling that is inconvenient for complex interface geometries. Further, we present an analysis of the method, including the errors arising from the neural network approximation, Monte Carlo approximation, iterative scheme, and penalty parameters. Numerical experiments indicate that the H-IDRM outperforms existing neural solvers on problems with high-dimensional domains, intricate interface geometries, and lower subdomain regularity.
Read the full abstract ▾
All authors Tianhao Hu, Bangti Jin, Fengru Wang, Yifeng Xu