Machine Learning
In-Context Learning for Data-Driven Censored Inventory Control
Sohom Mukherjee, Anh-Duy Pham, Richard Pibernik et al. (+1 more)
We study inventory control with decision-dependent censoring, focusing on the censored or repeated newsvendor (R-NV), where each order quantity determines whether demand is fully observed or censored by sales. Existing approaches based on parametric Thompson sampling (TS) can be brittle under prior mismatch, while offline imputation methods need not transfer to online learning. Motivated by the predictive view of decision making, we combine these ideas by taking oracle actions on learned completions of latent demand. We propose in-context generative posterior sampling (ICGPS), which uses modern generative models that are meta-trained offline and deployed online by in-context autoregressive generation. Theoretically, we show that the
Bayesian regret of ICGPS with a learned completion kernel is bounded by the
Bayesian regret of a TS
benchmark with the ideal completion kernel plus a deployment penalty scaling as $\sqrt{T}$ times the square root of the completion mismatch. This yields a plug-in template for operational problems with known TS regret bounds. For R-NV, we derive sublinear
Bayesian regret by reducing censored feedback to bandit convex optimization feedback. We also show that, under reasonable coverage and stability assumptions, the online completion mismatch is controlled by the offline censored predictive mismatch, so offline predictive quality transfers to online performance. Practically, we instantiate ICGPS with ChronosFlow, which combines a frozen time-series
transformer backbone with a trainable conditional normalizing-flow head for fast censoring-consistent sampling. In
benchmark experiments, ChronosFlow-ICGPS matches correctly specified TS, outperforms myopic and UCB-style baselines, and is robust to prior mismatch and distribution shift. ChronosFlow-ICGPS also performs well for the real-world SuperStore
dataset, especially under heavy censoring.
Read the full abstract ▾
All authors Sohom Mukherjee, Anh-Duy Pham, Richard Pibernik, Yunbei Xu
Machine Learning
GenAI for Energy-Efficient and Interference-Aware Compressed Sensing of GNSS Signals on a Google Edge TPU
Thorben Wegner, Lucas Heublein, Tobias Feigl et al. (+3 more)
Traditional methods for classifying global
navigation satellite system (GNSS) jamming signals typically involve post-processing raw or spectral data streams, requiring complex and costly data transmission to cloud-based interference classification systems. In contrast, our proposed approach efficiently compresses GNSS data streams directly at the hardware receiver while simultaneously classifying jamming and spoofing attacks in real time. Given the growing prevalence of GNSS jamming, there is a critical need for real-time solutions suitable for power-constrained environments. This paper introduces a novel method for compressing and classifying GNSS jamming threats using generative artificial intelligence (GenAI), specifically variational autoencoders (VAEs), deployed on Google Edge tensor processing units (TPUs). The study evaluates various autoencoder (AE) architectures to compress and reconstruct GNSS signals, focusing on preserving interference characteristics while minimizing data size near the receiver hardware. The pipeline adapts large-scale AE models for Google Edge TPUs through 8-bit quantization to ensure energy-efficient deployment. Tests on raw in-phase and quadrature-phase (IQ) data, Fast
Fourier Transform (FFT) data, and handcrafted features show the system achieves significant compression (>42x) and accurate classification of approximately 72 interference types on reconstructed signals (F2-score 0.915), closely matching the original signals (F2-score 0.923). The hardware-centric GenAI approach also substantially reduces jammer signal transmission costs, offering a practical solution for interference mitigation. Ablation studies on conditional and factorized VAEs (i.e., FactorVAE) explore latent feature disentanglement for data generation, enhancing model interpretability and fostering trust in machine learning (ML) solutions for sensitive interference applications.
Read the full abstract ▾
All authors Thorben Wegner, Lucas Heublein, Tobias Feigl, Felix Ott, Christopher Mutschler, Alexander Rügamer
Machine Learning · Cross-listed from cs.AI
Interestingness as an Inductive Heuristic for Future Compression Progress
Vincent Herrmann and Jürgen Schmidhuber
One of the bottlenecks on the way towards recursively self-improving systems is the challenge of interestingness: the ability to prospectively identify which tasks or data hold the potential for future progress. We formalize interestingness as an inductive heuristic for future compression progress and investigate its predictability using tools from Kolmogorov Complexity and Algorithmic Statistics. By analyzing complexity-runtime profiles under Length, Algorithmic, and Speed priors, we demonstrate that the inductive property of interestingness – the capacity for past progress to signal future discovery – is theoretically viable and empirically supported. We prove that expected future progress depends exponentially on the recency of the last observed breakthrough. Furthermore, we show that the Algorithmic Prior is significantly more optimistic than the Length Prior, yielding a quadratic increase in expected discovery for the same observed profile. These findings are experimentally confirmed across three diverse universal computational paradigms.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
K-Models: a Flexible and Interpretable Method for Ordinal Clustering with Application to Antigen-Antibody Interaction Profiles
Giulia Patanè, Alessandra Menafoglio, Alexander Krauth et al. (+4 more)
Existing clustering methods for functional data often prioritize partitioning accuracy over interpretability, making it challenging to extract meaningful insights when the data-generating process follows a specific underlying structure and an ordinal relationship among clusters is suspected. This work introduces K-Models, a novel framework that integrates ordinal constraints and estimates key underlying elements of the random process generating the observed functional profiles, improving both interpretability and structure identification. The proposed method is evaluated through simulations and real-world applications. In particular, it is tested on Region of Interest (ROI) curves, which represent reaction profiles from a reflectometric sensor monitoring biomolecular interactions, such as antigen-antibody binding. These curves represent changes in reflected light intensity over time at multiple measurement spots with immobilized antigens during analyte exposure, capturing the binding dynamics of the system. The goal is to identify intrinsic signal patterns solely from the observed dynamics, making this
dataset an ideal
benchmark for assessing the added interpretability of the proposed approach. By incorporating structural assumptions into the clustering process, K-Models enhances interpretability while maintaining performance comparable to state-of-the-art techniques, providing a valuable tool for analyzing functional data with an underlying ordinal structure.
Read the full abstract ▾
All authors Giulia Patanè, Alessandra Menafoglio, Alexander Krauth, Peter Fechner, Luca Dede', Bianca Maria Colosimo, Federica Nicolussi
Machine Learning
ToMAToMP: Robust and Multi-Parameter Topological Clustering
Ludo Andrianirina and Mathieu Carrière
Topological clustering, and its main
algorithm ToMATo, is a clustering method from Topological Data Analysis (TDA) which has been applied successfully in several applications during the last few years. This is due to its high versatility, as clusters are detected from the persistent components in the sublevel sets of any user-defined function (gene expression, pixel values, etc), and efficiency, as topological clustering enjoys robustness guarantees. However, ToMATo is also limited in several ways. First, a graph on the data points needs to be provided as a hyper-parameter of the method (whose fine-tuning is left to the user). Second, ToMATo is known to be very sensitive to outlier values in the function range. Finally, and most importantly, ToMATo can only handle one function at a time, whereas it is critical to use several functions in various applications. In this article, we introduce ToMAToMP: the first topological clustering method able to handle several functions at the same time with theoretical guarantees. More specifically, we leverage a recent tool from multi-parameter persistent homology, called MMA decomposition, to design our clustering
algorithm, and prove that it enjoys robustness properties. As corollaries, we show that it can be used to make ToMATo independent of graph tuning, and robust to outliers. Finally, we provide a set of numerical experiments showcasing the efficiency and quality of the clusterings produced by ToMAToMP, by showing strong improvement over non-topological and topological baselines for various
datasets.
Read the full abstract ▾
Machine Learning
GFMate: Empowering Graph Foundation Models with Test-time Prompt Tuning
Yan Jiang, Ruihong Qiu and Zi Huang
Graph
prompt tuning has shown great potential in graph learning by introducing trainable
prompts to enhance the model performance in conventional single-domain scenarios. Recent research has extended graph
prompts to improve Graph Foundation Models (GFMs) by few-shot tuning auxiliary
prompts. Despite their progress, most existing methods embed source-domain information into
prompts, which serve either as input to GFMs or encoded during model pre-training. Such
prompt entanglement with specific source domains and GFM pre-training strategy restricts their generalisability to other domains and different GFMs. Furthermore, existing GFM
prompts merely rely on few-shot tuning for adaptation, neglecting the rich information in unlabelled target domain test data. Motivated by these insights, this paper aims to empower GFMs with pre-training-agnostic test-time graph
prompt tuning, named GFMate. GFMate introduces centroid and layer
prompts applied after pre-training on target domains, avoiding
entanglement with specific source domains and model pre-training. In addition, a test-time complementary learning objective is devised to exploit both labelled and unlabelled target domain data for effective test-time
prompt tuning. Extensive experiments on 12
benchmark datasets demonstrate the superior performance and efficiency of GFMate, achieving improvements of up to 30.63%. Code is available at https://github.com/YanJiangJerry/GFMate.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces
William Lugoloobi, Samuele Marro, Jabez Magomere et al. (+2 more)
As LLM-based
agents increasingly browse the web on users' behalf, a natural question arises: can websites passively identify which underlying model powers an
agent? Doing so would represent a significant security risk, enabling targeted attacks tailored to known model vulnerabilities. Across 14 frontier LLMs and four web environments spanning information
retrieval and shopping tasks, we show that an
agent's actions and interaction timings, captured via a passive JavaScript tracker, are sufficient to identify the underlying model with up to 96% F1 within a fixed harness. Although fingerprints are harness-specific, with cross-harness transfer falling to near chance, pooling traces across harnesses recovers model identification at 0.74 to 0.81 Macro-F1. We further show that strong classifiers can be trained from roughly one third of the available traces, and that identification improves steadily as more events are observed. Injecting randomised timing delays between actions substantially degrades classifier performance, but does not provide robust protection: a classifier retrained on delayed traces largely recovers performance. We release our code and a labelled corpus of
agent traces at https://github.com/KabakaWilliam/known_actions.
Read the full abstract ▾
All authors William Lugoloobi, Samuele Marro, Jabez Magomere, Joss Wright, Chris Russell
Machine Learning
Understanding Imbalanced Forgetting in Rehearsal-Based Class-Incremental Learning
Alberto Tamajo, Srinandan Dasmahapatra and Rahman Attar
Neural networks suffer from catastrophic forgetting in class-incremental learning (CIL) settings. Rehearsal$\unicode{x2013}$replaying a subset of past samples$\unicode{x2013}$is a well-established mitigation strategy. However, recent results suggest that, despite balanced rehearsal allocation, some classes are forgotten substantially more than others. Despite its relevance, this imbalanced forgetting phenomenon remains underexplored. This work shows that imbalanced forgetting arises systematically and severely in rehearsal-based CIL and investigates it extensively. Specifically, we construct, from a principled analysis, three last-layer coefficients that capture different gradient-level sources of interference affecting each past class during an incremental step. We then demonstrate that, together, they reliably predict how past classes will rank in terms of forgetting at the end of that step. While predictive performance alone does not establish causality, these results support the interpretation of the coefficients as a plausible mechanistic account linking last-layer gradient-level interactions during training to class-level forgetting outcomes. Notably, one coefficient$\unicode{x2013}$capturing self-induced interference$\unicode{x2013}$emerges as the strongest predictor, with controlled experiments providing evidence consistent with this coefficient being influenced by the new-class interference coefficient. Overall, our findings provide valuable insights and suggest promising directions for mitigating imbalanced forgetting by reducing class-wise disparities in the identified sources of interference.
Read the full abstract ▾
Machine Learning
Peng's Q($\lambda$) for Conservative Value Estimation in Offline Reinforcement Learning
Byeongchan Kim and Min-hwan Oh
We propose a model-free offline multi-step reinforcement learning (RL)
algorithm, Conservative Peng's Q($\lambda$) (CPQL). Our
algorithm adapts the Peng's Q($\lambda$) (PQL) operator for conservative value estimation as an alternative to the Bellman operator. To the best of our knowledge, this is the first work in offline RL to theoretically and empirically demonstrate the effectiveness of conservative value estimation with a multi-step operator by fully leveraging offline trajectories. The fixed point of the PQL operator in offline RL lies closer to the value function of the behavior policy, thereby naturally inducing implicit behavior regularization. CPQL simultaneously mitigates over-pessimistic value estimation, achieves performance greater than (or equal to) that of the behavior policy, and provides near-optimal performance guarantees – a milestone that previous conservative approaches could not achieve. Extensive numerical experiments on the D4RL
benchmark demonstrate that CPQL consistently and significantly outperforms existing offline single-step baselines. In addition to the contributions of CPQL in offline RL, our proposed method also contributes to the offline-to-online learning framework. Using the Q-function pre-trained by CPQL in offline settings enables the online PQL
agent to avoid the performance drop typically observed at the start of fine-tuning and to attain robust performance improvements. Our code is available at https://github.com/oh-lab/CPQL.
Read the full abstract ▾
Machine Learning
Beyond What to Select: A Plug-and-play Oscillatory Data-Volume Scheduling for Efficient Model Training
Suorong Yang, Hanqi Zhu, Hai Gan et al. (+4 more)
Data selection accelerates training by identifying representative training data while preserving model performance. However, existing methods mainly focus on designing sample-importance criteria, i.e., deciding what to select, while typically fixing the selected data volume as the target ratio throughout training. Thus, they are often dynamic in sample identity but static in data volume. In this work, we revisit data selection from an optimization perspective and show that selected-data training induces an implicit regularization effect modulated by the instantaneous selection ratio. This reveals a key trade-off: lower ratios amplify selection-induced regularization, whereas higher ratios preserve data coverage and optimization fidelity. Motivated by this insight, we propose PODS, a Plug-and-play Oscillatory Data-volume Scheduling framework. Rather than introducing another sample-scoring metric, PODS serves as a lightweight module that dynamically schedules how much data to select over training. Under the target selection ratio, PODS alternates between low-ratio regularization phases and high-ratio recovery phases to exploit selection-induced regularization without sacrificing optimization stability. With its lightweight, ratio-level, and task-agnostic design, PODS is compatible with existing static and dynamic selection methods and broadly applicable across training paradigms. Experiments across various
datasets, architectures, and tasks show that PODS consistently improves the efficiency-generalization trade-off, e.g., reducing ImageNet-1k training cost by 50% with improved accuracy and accelerating LLM instruction tuning by over 2x without performance degradation.
Read the full abstract ▾
All authors Suorong Yang, Hanqi Zhu, Hai Gan, Fangjian Su, Guang Li, Furao Shen, Soujanya Poria
Machine Learning · Cross-listed from cs.CV
BioHuman: Learning Biomechanical Human Representations from Video
Yujun Huo, He Zhang, Chentao Song et al. (+3 more)
Understanding human motion beyond surface kinematics is crucial for motion analysis, rehabilitation, and injury risk assessment. However, progress in this domain is limited by the lack of large-scale
datasets with biomechanical annotations, and by existing approaches that cannot directly infer internal biomechanical states from visual observations. In this paper, we introduce a simulation-based framework for estimating muscle activations from existing motion capture
datasets, resulting in BioHuman10M, a large-scale
dataset with synchronized
video, motion, and activations. Building on BioHuman10M, we propose BioHuman, an end-to-end model that takes
monocular video as input and jointly predicts human motion and muscle activations, effectively bridging visual observations and internal biomechanical states. Extensive experiments demonstrate that BioHuman enables accurate reconstruction of both kinematic motion and muscle activity, and generalizes across diverse subjects and motions. We believe our approach establishes a new
benchmark for
video-based biomechanical understanding and opens up new possibilities for physically grounded human modeling.
Read the full abstract ▾
All authors Yujun Huo, He Zhang, Chentao Song, Honglin Song, Zongyu Zuo, Tao Yu
Machine Learning
Composable Crystals: Controllable Materials Discovery via Concept Learning
Nian Liu, Yuwei Zeng, Ryoji Kubo et al. (+7 more)
De novo
crystal generation, a central task in materials discovery, aims to generate
crystals that are simultaneously valid, stable, unique, and novel. Existing methods mainly rely on black-box
stochastic sampling, providing limited control over how generated structures move beyond the observed distribution. In this paper, we introduce a concept-based compositional framework for
crystal generation. We train a vector-quantized variational autoencoder to automatically discover a shared set of reusable
crystal concepts, which serve as building blocks for guided generation. These learned concepts naturally exhibit interpretability from both local atomic environments and global
symmetry patterns, and generalize to
crystals from different distributions. By recombining such concepts, our framework enables controllable exploration of novel
crystals beyond the training distribution, rather than relying solely on unconstrained random sampling. To further improve composition efficiency, we introduce a composition generator and iteratively refine it using high-quality samples generated by the model itself. The resulting concept compositions are then used to condition downstream
crystal generation. Numerical experiments on MP-20 and Alex-MP-20 show that compositing concepts separately increase base model up to 53.2% and 51.7% on V.S.U.N metric, with particular gains in novelty.
Read the full abstract ▾
All authors Nian Liu, Yuwei Zeng, Ryoji Kubo, Nikita Kazeev, Stephen Gregory Dale, Artem Maevskiy, Pengru Huang, Thomas Laurent, Kostya S. Novoselov, Xavier Bresson
Machine Learning
Compositional Sparsity as an Inductive Bias for Neural Architecture Design
Hongyu Lin, Antonio Briola, Yuanrong Wang et al. (+1 more)
Identifying the structural priors that enable Deep Neural Networks (DNNs) to overcome the curse of dimensionality is a fundamental challenge in machine learning theory. Existing literature suggests that effective high-dimensional learning is driven by compositional sparsity, where target functions decompose into constituents supported on low-dimensional variable subsets. To investigate this hypothesis, we combine Information Filtering Networks (IFNs), which extract sparse dependency structures via constrained information maximisation, with Homological Neural Networks (HNNs), which map the inferred
topology into fixed-wiring sparse neural graphs. We formalise the design principles underlying this construction and present an interpretable pipeline in which abstraction emerges through hierarchical composition. HNNs are orders of magnitude sparser than standard DNNs and require only minimal hyperparameter tuning. On synthetic tasks with known sparse hierarchies, HNNs recover the underlying compositional structure and remain stable in regimes where dense alternatives degrade as dimensionality increases. Across a broad suite of real-world
datasets, HNNs consistently match or outperform dense baselines while using far fewer parameters, exhibiting lower variance and showing reduced sensitivity to hyperparameters.
Read the full abstract ▾
All authors Hongyu Lin, Antonio Briola, Yuanrong Wang, Tomaso Aste
Machine Learning
Crys-JEPA: Accelerating Crystal Discovery via Embedding Screening and Generative Refinement
Nian Liu, Nikita Kazeev, Stephen Gregory Dale et al. (+8 more)
De novo
crystal generation seeks to discover materials that are not merely realistic, but also stable and novel. However, most existing generative models are trained to maximize the likelihood of observed
crystals, which encourages samples to stay close to known materials yet not necessarily align with the criteria that matter in discovery. Our empirical analysis shows that current
crystal generative models exhibit a clear conflict between stability and novelty: samples near the observed distribution tend to retain stability but offer limited novelty, whereas samples farther from it often lose stability rapidly. This suggests that the useful region for discovering
crystals that are both stable and novel is extremely narrow. To move beyond this limitation, we introduce Crys-JEPA, a joint
embedding predictive architecture for
crystals that learns an energy-aware latent space preserving formation-energy differences. In this space, stability assessment can be reformulated as an
embedding-based comparison against accessible training
crystals, reducing the reliance on expensive energy evaluation and task-specific external references. Building on Crys-JEPA, we further develop a screening-and-refinement pipeline that identifies promising generated
crystals and reintroduces them to refine the generative model. On MP-20 and Alex-MP-20
datasets, we achieve improvements over baselines up to 53.8% and 72.7% on V.S.U.N. metric, respectively.
Read the full abstract ▾
All authors Nian Liu, Nikita Kazeev, Stephen Gregory Dale, Artem Maevskiy, Yuwei Zeng, Ryoji Kubo, Pengru Huang, Thomas Laurent, Yann LeCun, Kostya S. Novoselov, Xavier Bresson
Machine Learning
Cognitive-Uncertainty Guided Knowledge Distillation for Accurate Classification of Student Misconceptions
Qirui Liu, Hao Chen, Weijie Shi et al. (+2 more)
Accurately identifying student misconceptions is crucial for personalized education but faces three challenges: (1) data scarcity with long-tail distribution, where authentic student
reasoning is difficult to synthesize; (2) fuzzy boundaries between error categories with high annotation noise; (3) deployment parado-large models overlook unconventional approaches due to pretraining bias and cannot be deployed on edge, while small models overfit to noise. Unlike traditional methods that increase diversity through large-scale data synthesis, we propose a two-stage knowledge
distillation framework that mines high-value samples from existing data. The first stage performs standard
distillation to transfer task capabilities. The second stage introduces a dual-layer marginal selection mechanism based on cognitive uncertainty, identifying four types of critical samples based on teacher model uncertainty and confidence differences. For different data subsets, we design difficulty-adaptive mechanism to balance hard/soft label contributions, enabling student models to inherit inter-class relationships from teacher soft labels while distinguishing ambiguous error types. Experiments show that with augmented training on only 10.30% of filtered samples, we achieve MAP@3 of 0.9585 (+17.8%) on the MAP-Charting
dataset, and using only a 4B parameter model, we attain 84.38% accuracy on cross-topic tests of middle school
algebra misconception
benchmarks, significantly outperforming sota LLM (67.73%) and standard fine-tuned 72B models (81.25%). Our code is available at https://github.com/RoschildRui/acl2026_map.
Read the full abstract ▾
All authors Qirui Liu, Hao Chen, Weijie Shi, Jiajie Xu, Jia Zhu
Machine Learning · Cross-listed from cs.CL
Non-linear Interventions on Large Language Models
Sangwoo Kim
Intervention is one of the most representative and widely used methods for understanding the internal representations of large
language models (LLMs). However, existing intervention methods are confined to linear interventions grounded in the Linear Representation Hypothesis, leaving features encoded along non-linear
manifolds beyond their reach. In this work, we introduce a general formulation of intervention that extends naturally to non-linearly represented features, together with a learning procedure that further enables intervention on implicit features lacking a direct output signature. We validate our framework on refusal bypass steering, where it steers the model more precisely than linear baselines by intervening on a non-linear feature governing refusal.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Video2GUI: Synthesizing Large-Scale Interaction Trajectories for Generalized GUI Agent Pretraining
Weimin Xiong, Shuhao Gu, Bowen Ye et al. (+5 more)
Recent advances in
multimodal large
language models have driven growing interest in graphical user interface (GUI)
agents, yet their generalization remains constrained by the scarcity of large-scale training data spanning diverse real-world applications. Existing
datasets rely heavily on costly manual annotations and are typically confined to narrow domains. To address this challenge, we propose
Video2GUI, a fully automated framework that extracts grounded GUI interaction trajectories directly from unlabeled Internet
videos.
Video2GUI employs a coarse-to-fine filtering strategy to identify high-quality GUI tutorial
videos and convert them into structured
agent trajectories. Applying this pipeline to 500 million
video metadata entries, we construct WildGUI, a large-scale
dataset containing 12 million interaction trajectories spanning over 1,500 applications and websites. Pre-training Qwen2.5-VL and
Mimo-VL on WildGUI yields consistent improvements of 5-20% across multiple GUI grounding and action
benchmarks, matching or surpassing state-of-the-art performance. We will release both the WildGUI
dataset and the
Video2GUI pipeline to support future research of GUI
agents.
Read the full abstract ▾
All authors Weimin Xiong, Shuhao Gu, Bowen Ye, Zihao Yue, Lei Li, Feifan Song, Sujian Li, Hao Tian
Machine Learning
Selective Safety Steering via Value-Filtered Decoding
Bat-Sheva Einbinder, Hen Davidov, Yee Whye Teh et al. (+2 more)
While large
language models (LLMs) are trained to align with human values, their generations may still violate safety constraints. A growing line of work addresses this problem by modifying the model's sampling policy at decoding time using a safety score. However, existing decoding-time steering methods often intervene unnecessarily, modifying generations that would have been safe under the base model. Such unnecessary interventions are undesirable, as they can distort key properties of the base model such as helpfulness, fluency, style, and coherence. We propose a new test-time steering method designed to reduce such unnecessary interventions while improving the safety of responses. Our approach filters
tokens using a value-based safety criterion and provides an explicit bound on the probability of false interventions. A single threshold hyperparameter controls this bound, allowing practitioners to trade off higher rates of unnecessary intervention for better output safety. Across multiple
datasets and experiments, we show that our value-filtered decoding method outperforms existing baselines, achieving better trade-offs between safety, helpfulness, and similarity to the base model.
Read the full abstract ▾
All authors Bat-Sheva Einbinder, Hen Davidov, Yee Whye Teh, Yarin Gal, Yaniv Romano
Machine Learning
TAPIOCA: Why Task- Aware Pruning Improves OOD model Capability
Krish Sharma, Omar Naim, Soumadeep Saha et al. (+3 more)
Recent work has promoted task-aware layer
pruning as a way to improve model performance on particular tasks, as shown by TALE. In this paper, we investigate when such improvements occur and why. We show first that, across controlled
polynomial regression tasks and large
language models, such
pruning yields no benefit on in-distribution (ID) data but consistently improves out-of-distribution (OOD) accuracy. We further show empirically that OOD inputs induce layerwise norm and pairwise-distance profiles that deviate from the corresponding ID profiles. This leads to a
geometric explanation of task-aware
pruning: each task induces a task-adapted geometry, characterized empirically by the representation profiles observed on ID inputs. OOD inputs can introduce a distorted version of the task-adapted geometry. Task-aware
pruning identifies layers that create or amplify this distortion; by removing them, it shifts OOD representational norms and pairwise distances toward those observed on the adapted distribution. This realigns OOD inputs with the model's task-adapted geometry and improves performance. We provide causal evidence through controlled distribution shifts and residual-scaling interventions, and demonstrate consistent behavior across model scales.
Read the full abstract ▾
All authors Krish Sharma, Omar Naim, Soumadeep Saha, Vinija Jain, Aman Chadha, Nicholas Asher
Machine Learning · Cross-listed from cs.SD
IsoNet: Face-Guided Speech Extraction for Compact Microphone Arrays
Dinanath Padhya, Sajen Maharjan, Binita Adhikari et al. (+1 more)
Listening to one chosen talker in a crowded room is effortless for people but hard for small devices, whose microphone arrays are too compact for classical beamforming to separate nearby voices. We present IsoNet, a select-to-listen system: the user selects a face in the camera view, and a 4-microphone array with a 9.4 cm aperture returns that person's
speech. IsoNet fuses multi-channel complex spectra, GCC-PHAT spatial cues, and a face
embedding at the bottleneck of a U-Net mask
estimator, trained with auxiliary direction-of-arrival supervision and an SNR curriculum on simulated reverberant VoxCeleb mixtures. On a hard test set spanning -1 to 10 dB SNR, IsoNet reaches 9.31 dB SI-SDR, a 4.85 dB improvement over the mixture, and raises PESQ from 1.40 to 2.13 and STOI from 0.72 to 0.84. The gains are largest where the task is hardest (5.71 dB SI-SDR improvement at -1 to 1 dB SNR), and IsoNet outperforms oracle-steered delay-and-sum and MVDR beamformers by 9.7 and 10.9 dB SI-SDR. Ablations show that visual and spatial cues are complementary, and a 4.9M-parameter audio-spatial variant stays within 0.16 dB of the full model while running 26 times faster than real time.
Read the full abstract ▾
All authors Dinanath Padhya, Sajen Maharjan, Binita Adhikari, Ishwor Raj Pokharel
Machine Learning · Cross-listed from cs.AI
Agentifying Patient Dynamics within LLMs through Interacting with Clinical World Model
Minghao Wu, Yuting Yan, Zhenyang Cai et al. (+9 more)
Sepsis management in the ICU requires sequential treatment decisions under rapidly evolving patient physiology. Although large
language models (LLMs) encode broad clinical knowledge and can reason over guidelines, they are not inherently grounded in action-conditioned patient dynamics. We introduce SepsisAgent, a world model-augmented LLM
agent for sepsis treatment recommendation. SepsisAgent uses a learned Clinical World Model to simulate patient responses under candidate fluid–vasopressor interventions, and follows a propose–simulate–refine workflow before committing to a prescription. We first show that world-model access alone yields inconsistent LLM decision performance, motivating
agent-specific training. We then train SepsisAgent through a three-stage curriculum: patient-dynamics supervised fine-tuning, propose–simulate–refine behavior cloning, and world-model-based agentic reinforcement learning. On MIMIC-IV sepsis trajectories, SepsisAgent outperforms all traditional RL and LLM-based baselines in off-policy value while achieving the best safety profile under guideline adherence and unsafe-action metrics. Further analysis shows that repeated interaction with the Clinical World Model enables the
agent to learn regularities in patient
evolution, which remain useful even when simulator access is removed.
Read the full abstract ▾
All authors Minghao Wu, Yuting Yan, Zhenyang Cai, Ke Ji, Chuangsen Fang, Ziying Sheng, Xidong Wang, Rongsheng Wang, Hejia Zhang, Shuang Li, Benyou Wang, Hongyuan Zha
Machine Learning · Cross-listed from cs.GR
SCAMP: Sparse-anchor Control is One Small Projection
Pengcheng Fang, Tengjiao Sun, Xiaoyu Zhan et al. (+4 more)
Authoring with a text-to-motion generator needs sparse anchors: chosen joints, at chosen frames, at given positions. Meeting them currently costs a conditioning branch trained for the task, or hundreds of per-clip optimisation steps in the architecture's native variables. In any generator that decodes a continuous state through a frozen differentiable decoder, the anchors ask for little and leave most of the state free: a few hundred numbers against a state of tens of thousands. Every control method is a choice among the states that satisfy them, and the choices differ along the directions the anchors cannot see and the motion can. SCAMP makes the choice that moves none of them: damped Gauss-Newton in the space of the anchors, through the frozen decoder alone, training-free, with one dimensionless damping constant. Every increment is a combination of the rows of the anchors' Jacobian, so the correction is orthogonal to everything the anchors never see, and the system solved is the size of the request rather than of the state. Applied unchanged to seven published generators spanning
diffusion,
token and latent designs, it matches or exceeds in anchor error every released control method it is measured against, and closes the anchors on hosts that ship none. Confined to those rows, a correction can only take the shapes the decoder admits, so what it costs belongs to the decoder, and holding the solver fixed makes that cost measurable: it divides by decoder family, windowed decoders staying within a small multiple of the unconstrained generator's foot skating where analytic recoveries multiply it several times over. Built to that criterion, our own generator reaches 0.083 m anchor error at FID 0.102 in 0.50 s per clip. The decoder's temporal support is a design criterion for controllable motion generation.
Read the full abstract ▾
All authors Pengcheng Fang, Tengjiao Sun, Xiaoyu Zhan, Yanwen Guo, Hansung Kim, Xiaohao Cai, Dongjie Fu
Machine Learning
NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces
Konstantinos Kontras, Trui Osselaer, Stylianos G. Mouslech et al. (+12 more)
Foundation models (FMs) promise to extract unified representations that generalize across downstream tasks. They have emerged across fields, including electroencephalography (
EEG), but it is less clear how effective they are in this particular field. Published evaluations differ in
datasets, in the
EEG-specific preprocessing that might influence reported results, and in the reported metrics, frequently obscuring the clinical relevance in
EEG. We introduce NeuroAtlas, the largest
EEG benchmark to date: 42
datasets and 260k hours covering clinical
EEG (epilepsy, sleep medicine,
brain age estimation) and
brain-computer interfaces, and include multiple
datasets per task along with bespoke clinical evaluation metrics. Besides evaluating
EEG-FMs with respect to supervised baselines, we present results from generic time-series FMs. We report three findings. First,
EEG-specific FMs do not consistently outperform time-series FMs, which have neither
EEG-focused architectures nor been pretrained on
EEG. Second, standard machine learning metrics are insufficient to assess clinical utility: thus, we thoroughly evaluate more appropriate measures such as the quality of event-level decision-making, hypnogram-derived features, and the
brain-age gap in the domains of epilepsy, sleep, and
brain age, respectively. Third, model rankings and performance can vary substantially within domains. We conclude that pretrained models perform largely on par, with only narrow advantages for a few, and that current models do not yet deliver on the promise of an out-of-the-box unified
EEG model. NeuroAtlas exposes this gap and provides the
datasets and metrics for the next generation of unified
EEG FMs.
Read the full abstract ▾
All authors Konstantinos Kontras, Trui Osselaer, Stylianos G. Mouslech, Angeliki-Ilektra Karaiskou, Guido Gagliardi, Thomas Strypsteen, Mohammad Hossein Badiei, Anku Rani, Maarten Vanmarcke, Miguel Bhagubai, Chanakya Ekbote, Jaedong Hwang, Christos Chatzichristos, Paul Pu Liang, Maarten De Vos
Machine Learning
The Rate-Distortion-Polysemanticity Tradeoff in SAEs
Tommaso Mencattini, Francesco Montagna and Francesco Locatello
Sparse Autoencoders (SAEs) that can accurately reconstruct their input (minimizing distortion) by making efficient use of few features (minimizing the rate) often fail to learn monosemantic representations (highly interpretable), limiting their usefulness for mechanistic interpretability. In this paper, we characterise this tension in learning faithful, efficient, and interpretable explanations, introducing the Rate-Distortion-Polysemanticity tradeoff in SAEs. Under toy-modeling assumptions, we theoretically and empirically show that restricting the SAE to be monosemantic necessarily comes with an increase in rate and distortion. Assuming a generative model behind the input observations, we further demonstrate that the degree of polysemanticity of optimal SAEs is determined by the training data distribution, especially by the probability of features to co-occur. Finally, we extend the analysis to real-world settings by deriving necessary conditions that a polysemanticity measure should satisfy when the data-generating process is unknown, and we
benchmark existing proxy metrics on SAEs trained on Large
Language Models. Taken together, our findings show that polysemanticity is a data problem that should be accounted for when addressing it at the architectural and optimization level.
Read the full abstract ▾
Machine Learning
ReMIA: a Powerful and Efficient Alternative to Membership Inference Attacks against Synthetic Data Generators
Davide Scassola, Andrea Coser and Sebastiano Saccani
Tabular data sharing under
privacy constraints is increasingly important for research and collaboration. Synthetic data generators (SDGs) are a promising solution, but synthetic data remains vulnerable to attacks, such as membership inference attacks (MIAs), which aim to determine whether a specific record was part of the training data. State-of-the-art MIAs are powerful but impractical: they rely on shadow modeling, requiring hundreds of SDG training runs, and need auxiliary data several times larger than the original training set. Fast proxy metrics like distance to closest record (DCR) are efficient but have limited sensitivity to MIA risk. We introduce ReMIA (Relative Membership Inference Attack), a practical
privacy metric that requires only two SDG training runs and additional data no larger than the original training set. Rather than predicting whether a record was in the training set, ReMIA generates two synthetic
datasets from two source
datasets and measures whether a classifier can identify which source a record came from. Experiments across multiple tabular
datasets and SDGs show that ReMIA has a sensitivity comparable to state-of-the-art MIAs while being substantially more practical. We further observe that SDGs can achieve
privacy-utility trade-offs that traditional noise-based anonymization methods do not match. Code is available at https://github.com/aindo-com/remia.
Read the full abstract ▾
Machine Learning
Spontaneous symmetry breaking and Goldstone modes for deep information propagation
Nabil Iqbal, T. Anderson Keller, Yue Song et al. (+2 more)
In physical systems, whenever a continuous
symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation over long distances and times. In this work, we study deep neural networks whose internal layers are equivariant under a continuous
symmetry and may therefore support analogous Goldstone-like degrees of freedom. We demonstrate, both analytically and empirically, that these degrees of freedom enable coherent signal propagation across depth and recurrent iterations, providing a mechanism for stable information flow without relying on architectural stabilizers such as residual connections or normalization. In feedforward networks, this results in improved trainability and representational diversity across layers. In recurrent settings, we demonstrate the same mechanism is valuable for long-term
memory by propagating information over recurrent iterations, thereby improving performance of RNNs and GRUs on long-sequence modeling tasks.
Read the full abstract ▾
All authors Nabil Iqbal, T. Anderson Keller, Yue Song, Takeru Miyato, Max Welling
Machine Learning
AQKA: Active Quantum Kernel Acquisition Under a Shot Budget
Jian Xu, Chao Li, Delu Zeng et al. (+2 more)
Estimating an $N \times N$
quantum kernel from
circuit fidelities requires $\Theta(N^2 S)$ measurement shots, the dominant bottleneck for deployment on near-term hardware. Existing budget-saving methods (Nyström-QKE, ShoFaR, kernel-target
alignment) sub-sample which entries to measure but allocate shots uniformly within their chosen subset, ignoring how much each entry drives the downstream classifier. We close this gap with two contributions. First, a complete regime decomposition for shot-budgeted
quantum kernel learning: a principled menu of when each allocator wins. Our method, AQKA, dominates the budget-limited regime ($B \lesssim 16 n_{\mathrm{pairs}}$) on sparse-sensitivity KRR, with the gap growing from $+8$ to $+25$ pts over uniform as $N$ scales $225{\to}1000$ and reaching $+26$–$32$ pts on an ibm\_pittsburgh (156-
qubit Heron) hardware kernel; Nyström-QKE wins at saturating budgets on planted-sparse via low-rank reconstruction; ShoFaR is competitive only at extreme low budgets. Second, a closed-form pair-level acquisition theory: $s_{ij}^{\star} \propto |g_{ij}|\sqrt{K_{ij}(1-K_{ij})}$ with explicit gradient $g_{ij}$ for KRR (Lemma~1, $|\beta_i\alpha_j+\beta_j\alpha_i|\sqrt{K_{ij}(1-K_{ij})}$) and SVM via the envelope
theorem ($|\eta_i^*\eta_j^*|\sqrt{K_{ij}(1-K_{ij})}$); a corrected sparsity-aware Cauchy–Schwarz rate $\rho \le 2m/N$ matching empirics (vs.\ the naive $m^2/N^2$); an explicit-constant plug-in regret bound (
Theorem~2); and a tighter SVM ceiling $\rho^{\mathrm{SVM}} \le m_{\mathrm{sv}}^2/N^2$. We close with the first multi-seed live online adaptive shot allocation on
quantum hardware: $+17.0 \pm 4.8$ pts at $N{=}20$ on ibm\_aachen ($3.5\sigma$, 5 seeds), with the advantage holding at $N{=}30$ at higher budget on ibm\_berlin ($+14.0 \pm 8.5$ pts, 5 seeds).
Read the full abstract ▾
All authors Jian Xu, Chao Li, Delu Zeng, John Paisley, Qibin Zhao
Machine Learning · Cross-listed from math.OC
Scalable Solution of the Stochastic Multi-path Traveling Salesman Problem via Neural Networks
Xiaochen Chou, Ludovica Di Marco and Enza Messina
The multi-path Traveling Salesman Problem with
stochastic travel costs arises in hybrid vehicle routing applications designed for Smart City and City Logistics, where multiple paths exist between each pair of locations. Travel times along these paths are typically affected by real-time traffic conditions and therefore modeled as
stochastic. The objective of the problem is to determine a
Hamiltonian tour that minimizes the expected total travel cost under uncertainty. In this work, we adopt a two-stage
stochastic programming formulation. In the first stage, a predefined route specifying the sequence of locations to be visited is determined, while taking into consideration a second-stage recourse problem that selects the optimal path from the feasible set of alternative paths for each pair of locations, once real-time traffic conditions are realized. To reduce the computational burden imposed by the large number of scenarios required to capture travel time uncertainty, the innovation of this work is the integration of neural network-based surrogate models to approximate the expected value of the second-stage recourse problem. Different architectures and training strategies for the neural networks are proposed and analyzed, with performance evaluated in terms of computation time, solution quality, and generalization capability. Preliminary findings demonstrate the enhanced scalability and practical applicability of the approach for complex vehicle routing problems under uncertainty.
Read the full abstract ▾
Machine Learning
Slower Generalization, Faster Memorization: A Sweet Spot in Algorithmic Learning
Shin So, Kyelim Lee and Albert No
Once generalization becomes possible, does adding more data always make learning faster? In Needleman-Wunsch
matrix generation, we find that small
Transformers reach high exact-match validation accuracy in the fewest optimizer updates at an intermediate
dataset size, while larger
datasets converge more slowly. A random-suffix probe shows that, at larger
dataset sizes, models fit the structured
matrices before memorizing their random suffixes, suggesting that shared rule learning aids training-set fitting. To examine what makes rules easier to learn, we study elementary cellular automata (ECA), in which each binary
cell updates from its own state and those of its two neighbors. These experiments identify lower-order predictive signals-output biases or predictive relationships involving only a few input bits-as a key factor in learnability. In Rule 90, finite-sample fluctuations create output biases and single-input dependencies absent from the uniform population. These signals weaken as the
dataset grows. Biased training that introduces these signals accelerates generalization on an unchanged uniform validation distribution. These results support an explanation for faster convergence at intermediate
dataset sizes: sampling fluctuations can facilitate rule acquisition, and larger
datasets weaken this advantage.
Read the full abstract ▾
Machine Learning
Unbiased and Second-Order-Free Training for High-Dimensional PDEs
Jaemin Seo, Surin Lee and Jae Yong Lee
Deep learning methods based on backward
stochastic differential equations (BSDEs) have emerged as competitive alternatives to physics-informed neural networks (PINNs) for solving high-dimensional partial differential equations (PDEs). By leveraging probabilistic representations, BSDE approaches can avoid the curse of dimensionality and often admit second-order-free training objectives that do not require explicit Hessian evaluations. It has recently been established that the commonly used Euler-Maruyama (EM) time discretization induces an intrinsic bias in BSDE training losses. While high-order schemes such as Heun can fully eliminate this bias, such schemes re-introduce second-order spatial derivatives and incur substantial computational overhead. In this work, we provide a principled analysis of EM-induced loss bias and propose an unbiased, second-order-free training framework that preserves the computational advantages of BSDE methods. Our code is available at https://github.com/seojaemin22/Un-EM-BSDE.
Read the full abstract ▾