Machine Learning
Finite Volume-Informed Neural Network Framework for 2D Shallow Water Equations: Rugged Loss Landscapes and the Importance of Data Guidance
Xiaofeng Liu
Physics-informed neural networks (PINNs) are a simple surrogate-modelling paradigm for partial differential equations, but their standard strong-form residual formulation is ill suited to the shallow water equations (SWE). It cannot enforce local conservation, handle discontinuities, or leverage the boundary-conforming unstructured meshes used in real-world applications. We introduce “Data-Guided FVM-PINN”, a framework that replaces the strong-form residual with a differentiable, well-balanced Roe Riemann-solver finite-volume (FVM) loss evaluated on unstructured meshes. The major finding is that physics-only FVM-PINN training often fails on realistic 2D problems: the network collapses to a trivial low-momentum state that nearly satisfies the FVM-PINN residual but bears no resemblance to the true flow. A loss-landscape diagnostic shows that the FVM-PINN loss at zero momentum is only about $7\times$ larger than at the trained solution, a shallow basin that an ordinary optimizer falls into; adding even sparse data turns this into a $310\times$ separation, breaking the degeneracy. On a 2D block-in-channel
benchmark, just $200$ random velocity measurements drop the velocity-field $L_2$ error by $22\times$ versus physics-only; $50$ measurements still deliver a $7\times$ reduction. A controlled ablation isolates the contribution of the FVM-PINN loss: it reduces velocity-field $L_2$ by $\sim$$23\%$ in the sparse-data regime and is essentially neutral when dense reference data is available. On a real-world Savannah River reach ($1306$
cells, $3600$~s simulation, five Manning zones), the framework constructs an accurate surrogate from SRH-2D anchor data, with time-window decomposition reducing error monotonically via progressive initial-condition handoff.
Read the full abstract ▾
Machine Learning
SkillGen: Verified Inference-Time Agent Skill Synthesis
Yuchen Ma, Yue Huang, Han Bao et al. (+5 more)
Skills are a promising way to improve LLM
agent capabilities without retraining, while keeping the added procedure reusable and controllable. However, high-quality skills are still largely written by hand. We introduce SkillGen, a multi-
agent framework that synthesizes a single auditable skill from trajectories generated by a base
agent. The output is a human-readable artifact that can be inspected before use. Rather than merely summarizing trajectories, SkillGen leverages contrastive induction over both successful and failed trajectories to identify reusable success patterns, recurring failure modes, and behaviors that appear in nearby successes but are missing from failures. SkillGen then generates candidate skills and iteratively refines the skill. A key novelty in SkillGen is that we model
agent skills as interventions to empirically verify the net effect of skills on the overall performance. Specifically, we compare outcomes on the same instances with and without the skill, so that we account for both repairs (cases where the skill fixes a baseline failure) and regressions (cases where the skill breaks a baseline success). Across a broad range of
agents and
datasets, SkillGen consistently improves held-out performance, outperforms existing skill-generation baselines, and produces skills that transfer across models.
Read the full abstract ▾
All authors Yuchen Ma, Yue Huang, Han Bao, Haomin Zhuang, Swadheen Shukla, Michel Galley, Xiangliang Zhang, Stefan Feuerriegel
Machine Learning
Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures
Linhai Zhang and Yulan He
Existing approaches to LLM personalization focus on constructing better personalized models or inputs, while treating inference as a single-shot process. In this work, we study Test-Time Personalization (TTP) along an unexplored axis: scaling inference-time computation by sampling N candidates from a personalized policy model and selecting the best with a personalized reward model. We prove that oracle selection yields expected utility growing logarithmically with the number of sampled candidates, establishing a theoretical ceiling for test-time scaling. However, standard reward models fail to realize this potential. To diagnose why, we derive a unified scaling law that decomposes any reward model's Best-of-N curve into four measurable quantities and reveals two failure modes, user-level collapse (near-constant prediction for some users) and query-level reward hacking (negative correlation with true quality for some queries). Guided by this law, we propose a probabilistic personalized reward model whose learned variance effectively mitigates both failure modes. Experiments confirm both elements of our framework: TTP delivers consistent scaling across multiple policy models and personalized text generation tasks, and our scaling law closely matches observed scaling curves across reward-model variants.
Read the full abstract ▾
Machine Learning
SURGE: Surrogate Gradient Adaptation in Binary Neural Networks
Haoyu Huang, Boyu Liu, Linlin Yang et al. (+6 more)
The training of Binary Neural Networks (BNNs) is fundamentally based on gradient approximation for non-differentiable binarization operations (e.g., sign function). However, prevailing methods including the Straight-Through
Estimator (STE) and its improved variants, rely on hand-crafted designs that suffer from gradient mismatch problem and information loss induced by fixed-range gradient clipping. To address this, we propose SURrogate GradiEnt Adaptation (SURGE), a novel learnable gradient compensation framework with theoretical grounding. SURGE mitigates gradient mismatch through auxiliary backpropagation. Specifically, we design a Dual-Path Gradient Compensator (DPGC) that constructs a parallel full-precision auxiliary branch for each binarized layer, decoupling gradient flow via output decomposition during backpropagation. DPGC enables bias-reduced gradient estimation by leveraging the full-precision branch to estimate components beyond STE's first-order approximation. To further enhance training stability, we introduce an Adaptive Gradient Scaler (AGS) based on an optimal scale factor to dynamically balance inter-branch gradient contributions via norm-based scaling. Experiments on image classification, object detection, and
language understanding tasks demonstrate that SURGE performs best over state-of-the-art methods.
Read the full abstract ▾
All authors Haoyu Huang, Boyu Liu, Linlin Yang, Yanjing Li, Yuguang Yang, Xuhui Liu, Canyu Chen, Zhongqian Fu, Baochang Zhang
Machine Learning
Seeing the Needle in the Haystack: Towards Weakly-Supervised Log Instance Anomaly Localization via Counterfactual Perturbation
Yutszyuk Wong, Wentai Wu, Yuen-Ying Yeung et al. (+1 more)
Log anomaly detection is a critical task for system operations and security assurance. However, in networked systems at scale, log data are generated at massive scale while instance-level annotations are prohibitively expensive, posing great difficulties to fine-grained anomaly localization. To address this challenge, we propose LogMILP (Log anomaly localization based on Multi-Instance Learning enhanced by prototypes and Perturbation), a weakly supervised framework that enables both bag-level anomaly detection and instance-level anomaly localization using only bag-level labels. Our method guides the model to pinpoint the critical log entries using prototype-guided structural modeling with counterfactual perturbation consistency regularization, thereby improving localization reliability and interpretability under coarse-grained supervision. Experimental results on three public
datasets demonstrate that LogMILP achieves competitive detection performance while yielding significantly more reliable instance-level localization. Our code is open-sourced at https://github.com/YUK1207/LogMILP.
Read the full abstract ▾
All authors Yutszyuk Wong, Wentai Wu, Yuen-Ying Yeung, Weiwei Lin
Machine Learning
AESOP: Adversarial Execution-path Selection to Overload Deep Learning Pipelines
Tingxi Li, Mingfang Ji, Ravishka Shemal Rathnasuriya et al. (+3 more)
Modern machine learning deployments increasingly compose specialized models into dynamic inference pipelines, where upstream components produce intermediate predictions that determine the workload and inputs of downstream components. The cost of processing an input is therefore not determined by any single model, but by two coupled factors: the per-inference cost of each invoked component and its workload volume. Because these pipelines run under hard real-time constraints, efficiency is a fundamental requirement for system availability. We show that this structure creates an efficiency-attack surface that existing methods targeting single models cannot exploit: on identical inputs and budgets, path-aware targeting inflates FLOPs by $2,407\times$ while the strongest single-model baseline achieves $117\times$ – a $20\times$ gap attributable entirely to where the attack is directed. We formalize this as the
adversarial path-selection problem and present AESOP, a framework combining vulnerability-guided path ranking with adaptive loss weighting. We evaluate AESOP on five pipelines plus a production-realistic deployment variant with batching, bounded buffering, and confidence-threshold defenses. AESOP achieves up to $2,407\times$ FLOPs and $419\times$ latency inflation in white-box setting and 58$\times$ FLOPs / 17$\times$ latency in gray-box settings. Under system-level defenses, the attack is not neutralized but redirected: pipelines are forced to choose between throughput collapse ($0.578 \to 0.006$ input/s) and $96.7\%$ data loss to sustain throughput.
Read the full abstract ▾
All authors Tingxi Li, Mingfang Ji, Ravishka Shemal Rathnasuriya, Simin Chen, Yitao Hu, Wei Yang
Machine Learning
Structural Interpretations of Protein Language Model Representations via Differentiable Graph Partitioning
Siddhant Dutta, Edward Tan Beng Wai, Soumick Sarker et al. (+2 more)
Protein language models such as ESM-2 learn rich residue representations that achieve strong performance on
protein function prediction, but their features remain difficult to interpret as structural $\&$ evolutionary signals are encoded in dense latent spaces. We propose a plug-$\&$-play framework that projects ESM-2 representations onto
protein contact graphs $\&$ applies $\textbf{SoftBlobGIN}$, a lightweight Graph Isomorphism Network with differentiable Gumbel-softmax substructure pooling, to perform structure-aware message passing $\&$ learn coarse functional substructures for downstream prediction tasks. Across enzyme classification, SoftBlobGIN achieves 92.8% accuracy $\&$ 0.898 macro-F1. Unlike post hoc analysis of
protein language models alone, our method produces directly auditable structural explanations: GNNExplainer recovers biologically meaningful active-site residues, spatially localized functional clusters, $\&$ catalytic contact patterns. On binding-site detection, SoftBlobGIN improves residue AUROC from $0.885$ using an ESM-2 linear probe to $0.983$, indicating that these structural explanations are not recoverable from
language-model features alone. Learned blob partitions provide an additional layer of interpretability by automatically grouping residues into functional substructures, with blobs containing annotated active-site residues showing $1.85\times$ higher importance than other blobs ($\rho{=}0.339$, $p{=}0.009$), without any active-site supervision. Our framework requires no retraining of the
language model, adds only $\sim$1.1M parameters, $\&$ generalises across ProteinShake tasks, achieving $F_{\max}$ of $0.733$ on Gene Ontology prediction $\&$ AUROC of $0.969$ on binding-site detection. We position this as an interpretable structural companion to
protein language models that makes their predictions more transparent $\&$ auditable.
Read the full abstract ▾
All authors Siddhant Dutta, Edward Tan Beng Wai, Soumick Sarker, Pasan Gunawardane, Jagath C. Rajapakse
Machine Learning
TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment
Jiaming Li, Chenyu Zhu, Nanxi Yi et al. (+9 more)
Reinforcement learning (RL) has shown extraordinary potential in aligning
diffusion models to downstream tasks, yet most of them still suffer from significant reward hacking, which degrades generative diversity and quality by inducing visual mode collapse and amplifying unreliable rewards. We identify the root cause as the mode-seeking nature of these methods, which maximize expected reward without effectively constraining probability distribution over acceptable trajectories, causing concentration on a few high-reward paths. In contrast, we propose Trajectory Matching Policy Optimization (TMPO), which replaces scalar reward maximization with trajectory-level reward distribution matching. Specifically, TMPO introduces a Softmax Trajectory Balance (Softmax-TB) objective to match the policy probabilities of K trajectories to a reward-induced Boltzmann distribution. We prove that this objective inherits the mode-covering property of forward KL divergence, preserving coverage over all acceptable trajectories while optimizing reward. To further reduce multi-trajectory training time on large-scale flow-matching models, TMPO incorporates Dynamic
Stochastic Tree Sampling, where trajectories share denoising prefixes and branch at dynamically scheduled steps, reducing redundant computation while improving training effectiveness. Extensive results across diverse
alignment tasks such as human preference, compositional generation and text rendering show that TMPO improves generative diversity over state-of-the-art methods by 9.1%, and achieves competitive performance in all downstream and efficiency metrics, attaining the optimal trade-off between reward and diversity.
Read the full abstract ▾
All authors Jiaming Li, Chenyu Zhu, Nanxi Yi, Youjun Bao, Li Sun, Quanying Lv, Xiang Fang, Daizong Liu, Jianjun Li, Kun He, Bowen Zhou, Zhiyuan Ma
Machine Learning
$\xi$-DPO: Direct Preference Optimization via Ratio Reward Margin
Zhengyuan Fan, Zhonghua Wu, Yuxuan Du et al. (+1 more)
Reference-free preference optimization has emerged as an efficient alternative to reinforcement learning from human feedback, with Simple Preference Optimization(SimPO) demonstrating strong performance by eliminating the explicit reference model through a simple objective. However, the joint tuning of the hyperparameters $\beta$ and $\gamma$ in SimPO remains a central challenge. We argue that this difficulty arises because the margin formulation in SimPO is not easily interpretable across
datasets with different reward gap structures. To better understand this issue, we conduct a comprehensive analysis of SimPO and find that $\beta$, rather than merely serving as a reward scaling factor, primarily functions to filter noisy samples, while the reward margin $\gamma$ defined by SimPO is not semantically well aligned with explicitly enlarging the reward gap between chosen and rejected responses. Motivated by these observations, we propose $\xi$-DPO: Direct preference optimization via ratio reward margin. We first reformulate the preference objective through an equivalent transformation, changing the optimization target from maximizing the likelihood of reward gaps to minimizing the distance between reward gaps and optimal margins. Then, we redefine the reward in a ratio form between the chosen and rejected, which effectively cancels the effect of $\beta$ and yields a bounded and interpretable margin. This margin is called the ratio reward margin and is denoted by $\xi$. Unlike the $\beta$ and $\gamma$ in SimPO, $\xi$ can be determined from the initial reward gap distribution, thus much easier to tune. Our extensive experimental results show that $\xi$-DPO substantially outperforms existing preference optimization methods across open
benchmarks on multiple evaluation metrics.
Read the full abstract ▾
All authors Zhengyuan Fan, Zhonghua Wu, Yuxuan Du, Qun Chen
Machine Learning
LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection
Haohui Zhang, Zhiye Wang, Xiaoying Gan et al. (+2 more)
Diffusion Language Models (dLLMs) have garnered significant
attention for their potential in highly parallel processing. The parallel capabilities of existing dLLMs stem from the assumption of conditional independence at high confidence levels, which ensures negligible discrepancy between the marginal and joint distributions. However, the stringent confidence thresholds required to preserve accuracy severely constrain the scalability of parallelism. Through systematic
token-level statistical analysis, we reveal that a substantial proportion of
tokens converge to their correct predictions early in the denoising process yet fail to reach standard confidence thresholds, confirming that current confidence-based criteria are overly conservative. In response, we introduce LEAP (Lookahead Early-Convergence
Token Detection for Accelerated Parallel Decoding). LEAP is a training-free, plug-and-play method that leverages future context filtering and multi-sequence superposition to detect early-converging
tokens. By validating the
alignment between early convergence and correctness, we enable reliable early decoding of these
tokens. Benchmarking across diverse domains demonstrates that LEAP significantly lowers inference latency and decoding steps. Compared to confidence-based decoding, the average number of denoising steps is reduced by about 30%. On the GSM8K
dataset, combining LEAP with dParallel accelerates decoding to 7.2
tokens per step while preserving model precision. LEAP effectively breaks the reliance on high-confidence priors, offering a novel paradigm for parallel decoding.
Read the full abstract ▾
All authors Haohui Zhang, Zhiye Wang, Xiaoying Gan, Xinbing Wang, Bo Jiang
Machine Learning
ProactBench: Beyond What The User Asked For
Sepehr Harfi, Ahmad Salimi, Dongming Shen et al. (+1 more)
Most LLM
benchmarks score how well a model responds to explicit requests. They leave unmeasured a different conversational ability: noticing and acting on needs the user has implied but not said. We call this conversational proactivity. ProactBench decomposes it into three phase-tied types: Emergent, inference from a single disclosed anchor; Critical, synthesis across multiple anchors; and Recovery, grounded forward-looking value after task completion. We operationalise the
benchmark with three
agents: a Planner, a User
Agent, and an Assistant Model. Their information asymmetries defend against style-confounded scoring, rubric leakage, external-context contamination, and information dumps. The released corpus contains 198 curated dialogues with 624 trigger points across 24 communication styles drawn from a psychometric inventory and audited by an independent LLM judge. Across 16 frontier and open-weight models, Recovery is both difficult and weakly predicted by six standard
benchmarks, making it a useful new evaluation signal.
Read the full abstract ▾
All authors Sepehr Harfi, Ahmad Salimi, Dongming Shen, Alex Smola
Machine Learning · Cross-listed from cs.CR
The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
Ismail Hossain, Tanzim Ahad, Md Jahangir Alam et al. (+3 more)
Jailbreak attacks –
adversarial prompts that bypass LLM
alignment through purely linguistic manipulation – pose a growing operational security threat, yet the field lacks large-scale, reproducible infrastructure for generating, categorizing, and evaluating them systematically. This paper addresses that gap with three contributions. (1) Large-scale compositional jailbreak
dataset. We construct 114,000
adversarial prompts by applying 912 composing strategies to 125 harmful seed
prompts from JailBreakV-28K. Every
prompt is assigned to one of 14 cybersecurity attack categories (e.g., malware, phishing, privilege escalation) via a six-model majority-vote pipeline, and each strategy is ranked by effectiveness per category, enabling principled strategy selection grounded in concrete
adversarial objectives. (2) Automated jailbreak generation. We instruction-fine-tune category-aware LLMs on Moderate and Optimal subsets, producing models that synthesize fluent jailbreak
prompts from a harmful seed at inference time – no templates, no gradient search. Our generators achieve perplexity 24-39 versus 40-140 for AutoDAN and AmpleGCG, with safety-filter evasion rates of 0.29-0.51 Mal (LlamaPromptGuard-2-86M), enabling controllable, scalable red-teaming under realistic
adversarial conditions. (3) OPTIMUS: a training-free jailbreak evaluator. OPTIMUS is a continuous metric J(S,H) that jointly captures semantic similarity between the harmful seed and the jailbreak (S) and harmfulness probability (H) via calibrated penalty functions. Unlike binary attack success rate (ASR), OPTIMUS requires no task-specific training, generalizes across evolving strategies, and exposes a stealth-optimal regime (S*=0.57, H*=0.43) that ASR misses. Experiments across 114,000
prompts confirm that OPTIMUS separates Weak, Moderate, and Optimal jailbreaks with category-level evidence binary evaluation cannot supply.
Read the full abstract ▾
All authors Ismail Hossain, Tanzim Ahad, Md Jahangir Alam, Sai Puppala, Syed Bahauddin Alam, Sajedul Talukder
Machine Learning
SMIXAE: Towards Unsupervised Manifold Discovery in Language Models
Collin Francel
Sparse autoencoders (SAEs) have been used widely to decompose and interpret neural network activations, especially those of
transformer language models. One key issue with SAEs is their inability to directly model multidimensional features. Instead, SAEs may tile such features by a set of independent directions that must be grouped together after the SAE training phase, impeding discoverability and interpretation of learned feature representations. We begin to address this issue by introducing the Sparse MIXture of Autoencoders (SMIXAE) architecture. Empirically, we provide evidence that SMIXAE models have success both in directly learning previously identified
manifold structures, as well as finding novel structures, within the open source Gemma 2 2B and 9B models. Finally, we discuss several limitations and point towards areas for future work.
Read the full abstract ▾
Machine Learning
The Pokémon Theorem: Fairness Criteria Can't Catch 'Em All
Daniel Matsui Smola and Alex Smola
Debates around algorithmic
fairness rarely end. Models considered "fair" by one criterion are soon shown to be unfair by another. We prove that this is unavoidable. Satisfying any finite checklist of
fairness criteria never certifies
fairness itself, since the model may always violate yet another criterion. We call this the Pokémon
theorem: you can't catch them all. In fact, trying to catch all criteria recovers the classical impossibility
theorems. Since these violations come from differences between the groups' data, a natural remedy would be to make the data fair by learning fair features. Unfortunately, this, too, is impossible: features that scrub out sensitive group information also scrub out all predictive signal. While exact
fairness is out of reach, approximate
fairness is not. We derive the Pareto frontier between
fairness and predictive signal for approximately fair features, and show how to attain it.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Flame3D: Zero-shot Compositional Reasoning of 3D Scenes with Agentic Language Models
Sagar Bharadwaj, Ziyong Ma, Anurag Ghosh et al. (+2 more)
3D scene understanding spans
reasoning about free space, object grounding, hypothetical object insertions, complex
geometric relationships, and integrating all of these with external tools and data sources. Existing 3D understanding methods typically rely on large-scale 3D-
language training or focus on object grounding and simple spatial relationships. We argue that the broad generalization that motivates 3D-
language training can be achieved at inference time, without 3D-specific training. We propose Flame3D, a training-free framework that represents scenes as editable visual-textual 3D
memories and exposes them to an off-the-shelf MLLM through composable spatial tools. Flame3D also lets the
agent synthesize custom spatial programs at inference time, enabling open-ended
reasoning over layouts, empty space, and objects not yet present in the scene. External data and corrections can be added to the
memory without retraining. In addition to showing competitive performance to finetuned 3D-LMM methods on ScanQA, we study multi-hop 3D
reasoning capabilities of Flame3D by evaluating it on a curated compositional spatial-
reasoning benchmark, Compose3D. We find that fixed tools fall short and that the
agent's ability to synthesize spatial operations at inference time is essential. These results invite the question: should future progress in 3D scene understanding focus on richer scene
memories and expressive compositional abstractions?
Read the full abstract ▾
All authors Sagar Bharadwaj, Ziyong Ma, Anurag Ghosh, Srinivasan Seshan, Anthony Rowe
Machine Learning · Cross-listed from cs.AI
Learning the Preferences of a Learning Agent
Karim Abdel Sadek, Mark Bedaywi, Rhys Gould et al. (+1 more)
For AI systems to be useful to humans, they must understand and act in accordance with our values and preferences. Since specifying preferences is a hard task, inverse reinforcement learning (IRL) aims to develop methods that allow for inferring preferences from observed behavior. However, IRL assumes the human to be approximately optimal. This is a big limitation in cases where the human themselves may be learning to act optimally in an environment. In this paper, we formalize the problem of learning the preferences of a learning
agent: a predictor observes a learner acting online and tries to infer the underlying reward function being (initially suboptimally) optimized by the learner. We model the learner as either being no-regret, or as converging to an optimal Boltzmann policy over time. In each of these settings, we establish theoretical guarantees for various preference learning
algorithms, or otherwise show that such guarantees are impossible.
Read the full abstract ▾
All authors Karim Abdel Sadek, Mark Bedaywi, Rhys Gould, Stuart Russell
Machine Learning
Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability
Qingyue Zhao, Kaixuan Ji, Heyang Zhao et al. (+1 more)
Kullback-Leibler (KL) regularization is ubiquitous in reinforcement learning
algorithms in the form of reverse or forward KL. Recent studies have demonstrated $\epsilon^{-1}$-type fast rates for decision making under reverse KL regularization, in contrast to the standard $\epsilon^{-2}$-type sample complexity. However, for forward-KL-regularized objectives, existing statistical analyses are either not applicable or result in $\tilde{O}(\epsilon^{-2})$ slow rates. We take the first step towards addressing this problem via a streamlined analysis of forward-KL-regularized offline CBs. We give the first $\tilde{O}(\epsilon^{-1})$ upper bounds in tabular and general function approximation settings, both under notions of single-policy concentrability. In particular, our convex-analytical pipeline unifies these settings by exploiting the pessimism principle in a novel way and completely bypasses the proof routines in previous works based on the mean value
theorem, which might be of independent interest. Moreover, we provide rate-optimal lower bounds, manifesting the tightness of our upper bounds in terms of statistical rates. Our lower bounds also demonstrate that the forward-KL-regularized sample complexity recovers the unregularized slow rate in the low-regularization regime, similarly to the reverse-KL regularization.
Read the full abstract ▾
All authors Qingyue Zhao, Kaixuan Ji, Heyang Zhao, Quanquan Gu
Machine Learning · Cross-listed from math.AP
Kinetic theory for Transformers and the lost-in-the-middle phenomenon
Mitia Duerinckx, Borjan Geshkovski and Stefano Rossi
We study causal self-
attention dynamics – a toy model for decoder
Transformers – which we interpret as a non-exchangeable interacting particle system. Adapting cumulant expansions to the triangular causal dependency structure of the model, and appealing to non-hierarchical methods to estimate correlations using Glauber calculus, we prove a quantitative mean-field limit result and a next-order characterization of correlations. For iid uniformly distributed
tokens, the limiting correlation equation can be solved in closed form and we obtain a rigorous explanation of the empirically observed lost-in-the-middle phenomenon: the
token retrieval profile, as a function of the source position in the
prompt, is $\mathsf{U}$-shaped, with primacy, recency, and a unique interior minimum under an explicit smallness condition.
Read the full abstract ▾
Machine Learning
Rethinking Ratio-Based Trust Regions for Policy Optimization in Multi-Agent Reinforcement Learning
Chulabhaya Wijesundara, Andrea Baisero, Zhongheng Li et al. (+3 more)
Centralized training with decentralized execution (CTDE) is a standard framework for cooperative multi-
agent policy-gradient reinforcement learning, allowing
agents to learn from joint information while acting from local observations. Ratio-based trust-region methods such as Multi-
Agent Proximal Policy Optimization (MAPPO) and Multi-
Agent Simple Policy Optimization (MASPO) update decentralized actors using per-
agent probability ratios weighted by joint advantage estimates. Teammate non-stationarity increases the variance of these advantages, which in turn increases the variance in the local ratio updates. This exposes two method-specific failure modes: MAPPO's additive clipping removes gradients for outlier samples and weakens recovery from policy drift, while MASPO's soft quadratic penalty can allow probability collapse. We introduce Multi-
Agent Ratio
Symmetry (MARS), a novel policy optimization objective that replaces these additive ratio-based trust-region mechanisms with a multiplicatively symmetric
geometric barrier. MARS preserves corrective gradients while assigning unbounded cost as probability ratios approach zero. Across 47 tasks spanning eight multi-
agent environments, including novel JAX
benchmarks PaxMen and AeroJAX, MARS matches or exceeds MAPPO and MASPO in aggregate environment-level performance. Ablations show that these gains arise from the geometry of the symmetric barrier rather than from flexible trust-region boundaries alone.
Read the full abstract ▾
All authors Chulabhaya Wijesundara, Andrea Baisero, Zhongheng Li, Gregory Castañón, Alan Carlin, Christopher Amato
Machine Learning
TSNN: A Non-parametric and Interpretable Framework for Traffic Time Series Forecasting
Bowen Liu, Haijian Lai, Chan-Tong Lam et al. (+4 more)
Although many complex models were proposed to analyze time series data, some studies have demonstrated remarkable performance with simpler structures. A recent study proposed a non-parametric framework for 3D point cloud classification, which has the potential to be adapted for time series
forecasting and enable interpretability. Inspired by the previous works, we present TSNN, a non-parametric and interpretable framework for traffic time series
forecasting. TSNN consists of multiple layers that decouple the time series by matching the entries in a
memory bank, where the
memory bank is constructed using a similar matching process within the training set. It leverages the periodicity in traffic data to enhance
forecasting accuracy while maintaining a simple model architecture. The proposed model operates without trainable parameters, preserving its inherent interpretability. In the experiments, TSNN achieves competitive performance compared to the typical deep learning models in four real-world traffic flow
datasets. We also visualize the decoupling process to show the effectiveness of the components. Finally, we demonstrate the interpretability of the model and illustrate the contribution of each time step within the
memory bank.
Read the full abstract ▾
All authors Bowen Liu, Haijian Lai, Chan-Tong Lam, Junhao Dong, Benjamin Ng, Wei Ke, Sio-Kei Im
Machine Learning
LBI: Parallel Scan Backpropagation via Latent Bounded Interfaces
Shaun Christopher Lee and Sangeetha Abdu Jyothi
Backpropagation is inherently sequential across depth, creating an $O(K)$-deep dependency chain that bottlenecks parallel training. While parallel-scan formulations theoretically reduce this depth to $O(\log K)$, they are computationally prohibitive for modern architectures due to the $O(d^3)$ cost of composing full-rank $d\times d$ Jacobians over the entire hidden state. We introduce Latent Bounded Interfaces (LBI), an algorithmic formulation that makes scan-based backpropagation tractable by restricting inter-region communication to a low-dimensional latent interface, $ m_k \in \mathbb{R}^{r}$, where $r \ll d$. This reduces the adjoint recursion to a suffix scan over $r \times r$ Jacobians, cutting per-combine cost from $O(d^3)$ to $O(r^3)$ while preserving exact gradients under the bounded-interface model. We demonstrate that LBI maintains model quality across four architectures (Mamba-2, Mamba-3,
Transformer, and a Mamba–
Transformer hybrid) at 47–61M block parameters. Interfaces of dimension $r=16$ suffice to preserve training quality within 0.16–0.35 cross
entropy of dense baselines. The resulting framework provides an algorithmic foundation for region-parallel training, reducing cross-device backward communication to a single scan over $K$ fixed-size
matrices, of approximately 56 KB for our experimental configurations.
Read the full abstract ▾
Machine Learning
A Complete Characterization of Learnability for Adversarial Noisy Bandits
Steve Hanneke and Kun Wang
We study
adversarial noisy bandits given a known function class $\mathcal{F}$. In each round, the adversary selects a function $f \in \mathcal{F}$, the learner chooses an arm, and then observes a noisy reward determined by the chosen arm and the function $f$. The goal is to minimize the cumulative regret $R(T)$, defined as the difference between the learner's performance and that of the best fixed arm in hindsight over $T$ rounds. We say that a function class $\mathcal{F}$ is learnable if there exists an
algorithm achieving sublinear regret. Our main result is a complete characterization of learnability for
adversarial noisy bandits. The characterization is given in terms of a convexified variant of the generalized maximin volume introduced by Hanneke and Wang (2025): namely, the generalized maximin volume evaluated on the convex hull $\operatorname{co}(\mathcal F)$. We prove that $\mathcal F$ is learnable if and only if this convexified generalized maximin volume is positive at every scale. This condition characterizes learnability against both oblivious and adaptive adversaries, showing in particular that these two notions of learnability are equivalent in the noisy bandit setting. Our analysis reveals that the key complexity measure is closely connected to two new combinatorial notions, hitting set and distribution covering number, which may be of independent interest. These results establish the first complete characterization of learnability for
adversarial noisy bandits.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
RigidFormer: Learning Rigid Dynamics using Transformers
Zhiyang Dou, Minghao Guo, Haixu Wu et al. (+3 more)
Learning-based simulation of multi-object rigid-body dynamics remains difficult because contact is discontinuous and errors compound over long horizons. Most existing methods remain tied to mesh connectivity and vertex-level message passing, which limits their applicability to mesh-free inputs such as point clouds and leads to high computational cost. Efficiently modeling high-fidelity rigid-body dynamics from mesh-free representations therefore remains challenging. We introduce RigidFormer, an object-centric
Transformer-based model that learns mesh-free rigid-body dynamics with controllable integration step sizes. RigidFormer reasons at the object level and advances each object through compact anchors; Anchor-Vertex Pooling enriches these anchors with local vertex features, retaining contact-relevant geometry without dense vertex-level interaction. We propose Anchor-based RoPE to make
attention aware of the objects' spatial layout while respecting the unordered nature of objects and anchors: object-
token processing is permutation-equivariant, and the mean-pooled anchor descriptor is invariant to anchor reindexing. RigidFormer further enforces
rigidity by projecting updates onto the rigid-body
manifold using differentiable Kabsch
alignment. On standard
benchmarks, RigidFormer outperforms or matches mesh-based baselines using point inputs, runs faster, generalizes to unseen point resolutions and across
datasets, and scales to 200+ objects; we also show a preliminary extension to command-conditioned articulated bodies by treating body parts as interacting object-level components. Project page: https://frank-zy-dou.github.io/projects/RigidFormer/
Read the full abstract ▾
All authors Zhiyang Dou, Minghao Guo, Haixu Wu, Doug Roble, Tuur Stuyck, Wojciech Matusik
Machine Learning
Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World
Christopher M. Bryant and Hao Liu
The scaling laws guiding modern model training were calibrated for a single regime: data-rich, single-epoch pretraining. The dominant such scaling law form, Chinchilla's $L = E + A/N^\alpha + B/D^\beta$, has three structural limitations outside that regime: it diverges as unique data shrinks instead of saturating at the uninformed baseline; it cannot represent overfitting when capacity exceeds the data; and it conflates total examples seen with unique examples available. We propose a closed-form extension, $L(N, D, T) = E + (L_0 - E)\,h/(1+h)$ with $h = a/N^\alpha + b/T^\beta + c\,N^\gamma/D^\delta$, that decomposes loss into undercapacity, undertraining, and overfitting terms. It saturates between the irreducible loss $E$ and an uninformed baseline $L_0$ fixed by the loss type, and reduces to Chinchilla in the data-rich, single-epoch limit. We validate it on four multi-epoch experiments spanning four architecture families (MLPs, ResNets,
Fourier neural operators, and
transformers) across
vision, scientific ML, and
language domains, and refit it to five published LLM scaling-law grids. Extrapolating to higher compute and larger unique data than seen at fit time, our form achieves state-of-the-art RMSE on every published LLM grid we evaluate and on most
cells of our constructed experiments. Once calibrated, the form admits a cost-aware allocation that recovers Chinchilla's optimum when data is free and shifts toward smaller corpora and more epochs as data grows expensive.
Read the full abstract ▾
Machine Learning
DARE: Difficulty-Adaptive Reinforcement Learning with Co-Evolved Difficulty Estimation
Yang Zhou, Can Jin, Zihan Dong et al. (+7 more)
Reinforcement learning improves the
reasoning ability of large
language models but remains costly and sample-inefficient, as many rollouts provide weak learning signals. Difficulty-aware data selection methods attempt to address this by prioritizing moderately difficult
prompts, yet our analysis reveals three limitations: difficulty estimates become inaccurate under policy drift, data selection alone yields limited final-performance gains, and inference efficiency remains largely unchanged. These findings suggest that efficient and effective RL requires more than filtering by difficulty: the policy should learn to solve hard tasks while producing concise responses for easy ones. To this end, we propose **Dare**, a unified framework that co-evolves difficulty estimation with the policy via self-normalized importance sampling, maintains diverse difficulty coverage through a symmetric Beta sampling distribution, and applies tailored training strategies across difficulty tiers with adaptive compute allocation. Extensive experiments across multiple models and domains demonstrate that **Dare** consistently outperforms existing methods in training efficiency, final effectiveness, and inference efficiency, producing more concise responses on easy tasks while improving correctness on hard ones. Code is available at https://github.com/EtaYang10th/DARE.
Read the full abstract ▾
All authors Yang Zhou, Can Jin, Zihan Dong, Zhepeng Wang, Yanting Yang, Shiyu Zhao, Lei Li, Runxue Bao, Yaochen Xie, Dimitris N. Metaxas
Machine Learning · Cross-listed from cs.AI
Emergent Semantic Role Understanding in Language Models
Carla Griffiths and Mirco Musolesi
Understanding how linguistic structure emerges in
language models is central to interpreting what these systems learn from data and how much supervision they truly require. In particular, semantic role understanding ("who did what to whom") is a core component of meaning representation, yet it remains unclear whether it arises from pre-training alone or depends on task-specific fine-tuning. We study whether semantic role understanding emerges during
language model pre-training or requires task-specific fine-tuning. We freeze decoder-only
transformers and train linear probes to extract semantic roles, using performance to infer whether role information is already encoded in pre-training or learned during adaptation. Across model scales, we find that frozen representations contain substantial semantic role information, with performance improving but not fully matching fine-tuned models. This indicates partial but incomplete emergence from pre-training alone. We show that semantic role structure emerges from
language modeling objectives, but its internal implementation shifts toward more distributed representations as model scale increases.
Read the full abstract ▾
Machine Learning
Learning When to Stop: Selective Imitation Learning Under Arbitrary Dynamics Shift
Surbhi Goel, Jonathan Pei and James Wang
Behavior cloning provides strong imitation learning guarantees when training and test environments share the same dynamics. However, in many deployment settings the test environment's transitions differ from training, and classical offline IL offers no recourse: the learner must commit to an action at every state, even when its demonstrations are uninformative and could lead to arbitrary degradation of performance. This motivates the study of selective imitation, where the learner may choose to stop when it cannot act reliably. We introduce a model for selective imitation under arbitrary dynamics shift: given labeled expert demonstrations from a training environment and unlabeled state trajectories from the same expert in a test environment, the learner outputs a selective policy that is complete (rarely stops in training) and sound (incurs low regret before stopping in test). Our
algorithm, SeqRejectron, constructs a stopping rule using a small set of validator policies whose size is independent of the horizon or policy class. For deterministic policies, this yields horizon-free $\tilde{O}(\log|\Pi|/\epsilon^2)$ sample complexity, assuming sparse costs. For
stochastic policies, we obtain analogous horizon-free guarantees using a cumulative Hellinger stopping time. We extend the framework to misspecified experts and different expert policies across train and test and obtain results that gracefully degrade with the amount of misspecification.
Read the full abstract ▾
Machine Learning
Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers
Aditya Ranganath
Training large
language models requires optimization
algorithms that are not only statistically effective, but also computationally and
memory efficient at extreme scale. Although Adam remains the dominant optimizer for large-scale
language-model pretraining and fine-tuning, recent work has revisited nearly every component of the optimization stack: adaptive moment estimation, decoupled weight decay,
memory footprint,
curvature approximation, sign-based updates, large-batch stability, low-rank gradient structure, and
matrix-wise orthogonalized updates. This survey reviews optimizer design for large
language models through a systems-and-optimization lens. We organize the literature into classical first-order optimizers, adaptive optimizers,
memory-efficient variants, second-order and
curvature-aware methods, sign-based and discovered optimizers, low-rank and projection-based methods, and
matrix-based optimizers such as
Muon. We also discuss benchmarking methodology, including hyperparameter
fairness, scale dependence, wall-clock efficiency,
token efficiency,
memory overhead, and downstream evaluation. We argue that optimizer research for LLMs is entering a new phase: moving from single-
algorithm speedup claims toward rigorous, scale-aware comparisons that jointly evaluate convergence, stability,
memory, and implementation complexity.
Read the full abstract ▾
Machine Learning
WavesFM: Hierarchical Representation Learning for Longitudinal Wearable Sensor Waveforms
Peng Cao, Zhijian Yang, Tennison Liu et al. (+17 more)
Wearable sensors enable the continuous acquisition of high-resolution physiological waveforms, such as photoplethysmography and accelerometry, under free-living conditions. However, inferring health-related phenotypes from these signals presents significant challenges due to high sampling frequencies,
multimodal dependencies, and extreme sequence lengths (e.g., weeks of recordings), compounded by a scarcity of ground-truth labels. To address these challenges, existing self-supervised learning (SSL) methodologies typically follow two paradigms: (1) learning rich morphological representations from short waveform segments while collapsing longitudinal dynamics through simple aggregation, or (2) modeling behavioral patterns from coarse, hand-crafted features (e.g. heart rate, step counts) spanning longer horizons but foregoing subtle, predictive signatures in raw waveforms. To bridge this gap, we propose WavesFM, a foundation model utilizing a two-stage SSL framework for longitudinal physiological data. Specifically, we decompose the learning problem into two stages: first, a segment-level encoder is pretrained to extract local
embeddings from short waveforms; subsequently, a temporal encoder is trained to model the sequence of these
embeddings across a multi-day horizon. This hierarchical approach overcomes the computational complexity of high-resolution, long-sequence data, allowing the overall model to capture both local signal semantics and the complex circadian and inter-day variations governing physiological dynamics. Pretrained on over 6.8M hours (N=324k individuals) of recordings for the first stage and 5.3M hours (N=10k) for the second stage, WavesFM demonstrates superior performance across 58 diverse tasks spanning demographics, lifestyle, health conditions, and medications.
Read the full abstract ▾
All authors Peng Cao, Zhijian Yang, Tennison Liu, Jonathan Wang, Jiang Wu, Magdalena Proszewska, Arvind Pillai, Mingwu Gao, Amir Farjadian, Lawrence Cai, Emily Blanchard, Daniel McDuff, Pramod Rudrapatna, Matthew Thompson, Anupam Pathak, Mark Malhotra, Shwetak Patel, Dina Katabi, Paolo Di Achille, Ming-Zher Poh
Machine Learning
Prediction Bottlenecks Don't Discover Causal Structure (But Here's What They Actually Do)
Ankit Hemant Lade, Sai Krishna Jasti, Indar Kumar et al. (+1 more)
A Mamba state-space model trained only for next-step prediction appears to recover Granger-causal structure through a simple readout $S = |W_{out} W_{in}|$, with early experiments suggesting the phenomenon generalized across architectures and benefited from interventional data at $p < 10^{-5}$. We package the protocol used to test that claim – standardized synthetic generators (VAR/Lorenz/CauseMe-style), three intervention semantics ($do(X=c)$, soft-noise, random-forcing), edge-provenance cards on three real
datasets, and size-matched control arms – as a reusable falsification
benchmark, and walk the claim through it in five stages. The method-level claim does not survive: (i) a plain linear bottleneck does as well or better; (ii) tuned Lasso beats the bottleneck on synthetic CauseMe-style
benchmarks, and on Lorenz-96 (the only real
benchmark with unambiguous ground truth) classical PCMCI and Granger lead a tight cluster in which the bottleneck trails; (iii) the headline intervention advantage is roughly 60% a sample-size confound, and the residual disappears under standard $do(X=c)$ interventions, surviving only under a non-standard random-forcing scheme; (iv) even that residual reproduces, with a larger effect, in classical bivariate Granger – the effect is method-agnostic. What survives is a narrow characterization result; the
benchmark is the lasting artifact, and each stage above is one of its control arms.
Read the full abstract ▾
All authors Ankit Hemant Lade, Sai Krishna Jasti, Indar Kumar, Aman Chadha