Machine Learning · Cross-listed from cs.CL
Zhihan Yang, Wei Guo, Shuibai Zhang et al. (+5 more)
While
diffusion has drawn considerable recent
attention from the
language modeling community, continuous
diffusion has appeared less scalable than discrete approaches. To challenge this belief we revisit Plaid, a likelihood-based continuous
diffusion language model (DLM), and construct RePlaid by aligning the architecture of Plaid with modern discrete DLMs. In this unified setting, we establish the first scaling law for continuous DLMs that rivals discrete DLMs: RePlaid exhibits a compute gap of only $20\times$ compared to autoregressive models, outperforms Duo while using fewer parameters, and outperforms MDLM in the over-trained regime. We
benchmark RePlaid against recent continuous DLMs: on OpenWebText, RePlaid achieves a new state-of-the-art PPL bound of $22.1$ among continuous DLMs and superior generation quality. These results suggest that continuous
diffusion, when trained via likelihood, is a highly competitive and scalable alternative to discrete DLMs. Moreover, we offer theoretical insights to understand the advantage of likelihood-based training. We show that optimizing the noise schedule to minimize the ELBO's variance naturally yields linear cross-
entropy (information loss) over time. This evenly distributes denoising difficulty without any case-specific time reparameterization. In addition, we find that optimizing
embeddings via likelihood creates structured geometries and drives the most significant likelihood gain.
Read the full abstract ▾
All authors Zhihan Yang, Wei Guo, Shuibai Zhang, Subham Sekhar Sahoo, Yongxin Chen, Arash Vahdat, Morteza Mardani, John Thickstun
Machine Learning · Cross-listed from math.OC
Tight Complexity of Muon under Heavy-Tailed Noise
Jiayu Zhang and Tianyi Lin
A growing lesson from neural network optimization is that optimizer design should respect how the model is parametrized. The layerwise input-output structure of neural networks motivates
Muon and Scion, whose updates also support hyperparameter transfer. At the same time,
stochastic gradient noise in deep learning is often far from sub-Gaussian and may exhibit heavy tails. These observations have shaped recent algorithmic principles for training neural networks, yet their joint theoretical consequences are underexplored. In particular, it is unclear what dimension dependence is unavoidable for gradient-based methods when the problem class is defined using an input-output norm and the
stochastic gradient noise is heavy-tailed, and if higher-order smoothness can accelerate optimization. We study these questions through nonconvex smooth
stochastic optimization over $\mathbb{R}^{m\times n}$ equipped with general norms and under $p^\mathrm{th}$-moment heavy-tailed noise, where the goal is to achieve an $\epsilon$-stationary point in the dual norm. Our first contribution is a dimension-dependent lower bound: when $\max\{m,n\}$ is large enough, any method using only the
stochastic gradients requires $\Omega(\min\{m, n\}\epsilon^{-\frac{3p-2}{p-1}})$ oracle calls for the problem class defined by the spectral norm, which is a common input-output
matrix norm. We prove that a batched
Muon method with the spectral norm can achieve the matching upper bound of $O(\min\{m, n\}\epsilon^{-\frac{3p-2}{p-1}})$. To exploit higher-order smoothness, we propose a transported
Muon method and improve the bound to $O(\min\{m, n\}\epsilon^{-\frac{5p-3}{2p-2}})$ when the norm is spectral and the Hessian is Lipschitz. Finally, we incorporate practical heuristics into our transported method and evaluate it across multiple architectures and model sizes, showing its compatibility with neural network training.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Beyond Morphology: Quantifying the Diagnostic Power of Color Features in Cancer Classification
Farnaz Kheiri, Shahryar Rahnamayan and Masoud Makrehchi
In histopathology, human experts primarily rely on color as a means of enhancing contrast to interpret tissue morphology, whereas machine
vision models process color as raw statistical information. This distinction raises a fundamental question: to what extent can pixel intensity alone, independent of structural and morphological cues, support cancer classification? To address this question, we systematically evaluated the standalone discriminative power of global color features while deliberately excluding all morphological information. Specifically, we extracted statistical color moments and discretized RGB and HSV color histograms, and assessed their performance across ten diverse experimental settings using classical machine learning classifiers. Our results demonstrate that color features alone can achieve strong performance in binary diagnostic tasks (e.g., benign versus malignant), with classification accuracies reaching up to 89%. This performance is likely attributable to global chromatic shifts associated with malignancy. Importantly, these simple color-based representations consistently outperformed random baselines by a substantial margin, indicating that raw color distributions encode a non-random and diagnostically relevant signal for cancer detection. Consequently, this study suggests that simple, computationally efficient color features can serve as an effective pre-screening tool. By identifying samples with strong chromatic indicators of malignancy, these lightweight models could function as a first-pass triage system, reducing the computational burden on complex deep learning architectures.
Read the full abstract ▾
Machine Learning
Offline Contextual Bandits in the Presence of New Actions
Ren Kishimoto, Tatsuhiro Shimizu, Kazuki Kawamura et al. (+6 more)
Automated decision-making
algorithms drive applications such as recommendation systems and search engines. These
algorithms often rely on off-policy contextual bandits or off-policy learning (OPL). Conventionally, OPL selects actions that maximize the expected reward from an existing action set. However, in many real-world scenarios, actions, such as news articles or
video content, change continuously, and the action space evolves over time after data collection. We define actions introduced after deploying the logging policy as new actions and focus on OPL with new actions. Existing OPL methods identify optimal actions from the existing set effectively but cannot learn and select new actions because no relevant data are logged. To address this limitation, we propose a new OPL method that leverages action features. We first introduce the Local Combination PseudoInverse (LCPI)
estimator for the policy gradient, generalizing the PseudoInverse
estimator initially proposed for off-policy evaluation of slate bandits. LCPI controls the trade-off between reward-modeling condition and the condition for data collection regarding the action features, capturing the interaction effects among different dimensions of action features. Furthermore, we propose a generalized
algorithm called Policy Optimization for Effective New Actions (PONA), which integrates LCPI, a component specialized for new action selection, with Doubly Robust (DR), which excels at learning within existing actions. We define PONA as a weighted sum of the LCPI and DR
estimators, optimizing both the selection of existing and new actions, and allowing the proportion of new action selections to be adjusted by the weight parameter. Through extensive experiments, we demonstrate that PONA efficiently selects new actions while maintaining the overall policy performance as opposed to most existing methods that cannot select new actions.
Read the full abstract ▾
All authors Ren Kishimoto, Tatsuhiro Shimizu, Kazuki Kawamura, Takanori Muroi, Yusuke Narita, Yuki Sasamoto, Kei Tateno, Takuma Udagawa, Yuta Saito
Machine Learning
DiPRL: Learning Discrete Programmatic Policies via Architecture Entropy Regularization
Chengpeng Hu, Yingqian Zhang and Hendrik Baier
Programmatic reinforcement learning (PRL) offers an interpretable alternative to deep reinforcement learning by representing policies as human-readable and -editable programs. While gradient-based methods have been developed to optimize continuous relaxations of programs, they face a significant performance drop when converting the continuous relaxations back into discrete programs. Post-hoc discretization can discard optimized branches and parameters in a program, which results in a collapse of policy expressivity and lowered task performance, leading in turn to a need for additional fine-tuning. To overcome these limitations, we propose Differentiable Discrete Programmatic Reinforcement Learning (DiPRL), a method that learns programmatic policies that become nearly discrete during training, avoiding a separate post-hoc fine-tuning stage. We first analyze the inherent risks of performance drop introduced by post-hoc discretization of gradient-based methods. Then, we introduce programmatic architecture
entropy regularization, which enables smooth, differentiable training that encourages convergence toward a discrete program. DiPRL maintains the efficiency of gradient-based optimization while mitigating the risks of post-hoc discretization. Our experiments across multiple discrete and continuous RL tasks demonstrate that DiPRL can achieve strong performance via interpretable programmatic policies.
Read the full abstract ▾
Machine Learning
DBES: A Systematic Benchmark and Metric Suite for Evaluating Expert Specialization in Large-Scale MoEs
Jing Wang, Hongxuan Lu, Jazze Young et al. (+2 more)
Expert specialization in Mixture-of-Experts (MoE) models remains poorly understood, with traditional evaluations conflating architectural load-balancing with functional specialization. We introduce DBES, a comprehensive diagnostic framework combining a multi-domain
benchmark with five theoretically grounded metrics: Routing Specialization, Normalized Effective Rank, Domain Isolation, Routing Stiffness Score, and N-gram Expertise measures. Critical findings demonstrate distinct specialization paradigms across models: Qwen-series exhibit modular specialization with high domain isolation, while DeepSeek and GLM employ distributed collaboration. However, we emphasize that specialization is a diagnostic dimension, necessary but not sufficient for downstream performance. Most crucially, interventional evidence validates the actionability of these metrics: by using DBES to identify high-specialization expert paths during domain-specific post-training, we achieved 66% to 94.48% improvement in specialized domains with only 15% of original training resources, demonstrating that these diagnostic tools can be converted into concrete optimization operators. This work provides the first systematic methodology for evaluating expert specialization independently of accuracy metrics, offering crucial insights for the design and post-training optimization of next-generation MoE systems.
Read the full abstract ▾
All authors Jing Wang, Hongxuan Lu, Jazze Young, Shu Wang, Zhimin Xin
Machine Learning
Modality vs. Morphology: A Framework for Time Series Classification for Biological Signals
Jordan Tschida, Matthew Yohe, Edward Kane et al. (+10 more)
Time series classification (TSC) of biological signals has progressed from handcrafted, modality-specific approaches to deep architectures capable of representing the diverse waveform structures of underlying physiological processes (i.e., morphology). This review introduces a unified morphology–modality framework that connects waveform structure to a methodological design, revealing how spikes, bursts, oscillations, slow drift, and hierarchical rhythms inform model design. By analyzing electroencephalography, electromyography, electrocardiography, photoplethysmography, and ocular modalities (electrooculography, pupillometry, eye-tracking), the review demonstrates how morphology determines preprocessing and modeling strategies. Integrating evidence across these biological signals, the framework reveals that morphology, not model class, most strongly determines performance and interpretability. This provides insight into why deep models succeed when their inductive biases align with underlying waveform dynamics. This review also identifies future work including morphological data augmentation and evaluation metrics to improve generalization. Together, these insights position morphology-aware modeling as a unifying principle for developing generalizable, interpretable, and physiologically meaningful TSC models across biological signals.
Read the full abstract ▾
All authors Jordan Tschida, Matthew Yohe, Edward Kane, Gavin Jager, Emma J. Reid, Tony G. Allen, Mark Story, Leanne Thompson, Joe Hoskins, Brandon Schreiber, Stan Seiferth, Scott Dolvin, David Cornett
Machine Learning · Cross-listed from stat.CO
AI4BayesCode: From Natural Language Descriptions to Validated Modular Stateful Bayesian Samplers
Jungang Zou, Alex Ziyu Jiang and Qixuan Chen
Coding and computation remain major bottlenecks in Markov chain Monte Carlo (MCMC) workflows, especially as modern sampling
algorithms have become increasingly complex and existing probabilistic programming systems remain limited in model support, extensibility, and composability. We introduce AI4BayesCode, an extensible LLM-driven system that translates natural-
language Bayesian model descriptions into runnable, validated MCMC samplers. To improve reliability, AI4BayesCode adopts a modular design that decomposes models into modular sampling blocks and maps each block to a built-in sampling component, reducing the need to implement complex sampling
algorithms from scratch. Reliability is further improved through pre-generation validation of model specifications and post-generation validation of generated sampler code. AI4BayesCode also introduces a novel recursively stateful coding paradigm for MCMC, allowing modular sampling components, potentially developed by different contributors, to be composed coherently within larger MCMC procedures. We develop a
benchmark suite to evaluate AI4BayesCode for sampler-generation. Experiments show that AI4BayesCode can implement a wide range of
Bayesian models from natural-
language descriptions alone. As an open-ended system, its capability can continue to expand with improvements in the underlying AI
agent and the addition of new built-in blocks. The software is available at https://ai4bayescode.com/.
Read the full abstract ▾
Machine Learning
GAMMA: Global Bit Allocation for Mixed-Precision Models under Arbitrary Budgets
Zhangyang Yao, Haiyan Zhao, Haoyu Wang et al. (+1 more)
Mixed-precision quantization improves the budget–accuracy trade-off for large
language models (LLMs) by allocating more bits to sensitive modules. However, automating this allocation at LLM scale faces a unique combination of constraints: learnable approaches require quantization-aware training, which is infeasible for billion-parameter models; training-free alternatives rely on static proxy metrics that miss cross-module interactions and must be recomputed per target budget; and search-based methods are expensive without guaranteeing exact budget compliance. We propose GAMMA, a quantizer-agnostic framework that learns module-wise precision preferences entirely within a post-training pipeline. GAMMA optimizes a teacher-forced hidden-state reconstruction objective under an augmented Lagrangian constraint, and projects the learned preferences into exact budget-feasible discrete assignments via integer programming. A key property is score reuse: because the learned preferences encode a stable sensitivity ranking rather than budget-specific weights, a single training run serves arbitrary deployment targets by re-solving only the integer program, reducing per-budget adaptation from hours to a few minutes. Across Llama and Qwen models (8B–32B), GAMMA outperforms both fixed-precision baselines (up to +12.99 Avg.) and search-based mixed-precision methods (up to +7.00 Avg.), and can match fixed 3-bit quality at 2.5-bit average precision, enabling deployment at substantially smaller
memory footprints.
Read the full abstract ▾
All authors Zhangyang Yao, Haiyan Zhao, Haoyu Wang, Xu Han
Machine Learning · Cross-listed from cs.CR
Prompt2Fingerprint: Plug-and-Play LLM Fingerprinting via Text-to-Weight Generation
Sixu Chen, Xiang Chen, Hongyao Yu et al. (+5 more)
The widespread deployment and redistribution of large
language models (LLMs) have made model provenance tracking a critical challenge. While existing LLM fingerprinting methods, particularly active approaches that embed identity signals via fine-tuning, achieve high accuracy and robustness, they suffer from significant scalability bottlenecks. These methods typically treat fingerprint injection as an independent, one-off optimization task rather than a reusable capability, necessitating separate, resource-intensive training for every new identity. This incurs prohibitive computational costs and deployment delays. To address this, we propose
Prompt2Fingerprint (P2F), the first framework that reformulates fingerprinting as a conditional parameter generation task. By leveraging a specialized generator, P2F maps textual descriptions directly to low-rank parameter increments in a single forward pass, enabling plug-and-play LLM fingerprint injection without further model retraining. Our experiments demonstrate that P2F maintains high fingerprint accuracy, harmlessness, and robustness while significantly reducing computational overhead, offering a scalable and instant solution for LLM ownership management.
Read the full abstract ▾
All authors Sixu Chen, Xiang Chen, Hongyao Yu, Jiaxin Hong, Hao Fang, Shuoyang Sun, Bin Chen, Shu-Tao Xia
Machine Learning · Cross-listed from stat.ML
Flowing with Confidence
Friso de Kruiff, Dario Coscia, Max Welling et al. (+1 more)
Generative models can produce nonsensical text, unrealistic images, and unstable materials faster than simulation or human review can absorb; without per-sample confidence, trust erodes. Existing fixes run $k$ ensembles or
stochastic trajectories at $k\times$ compute, measuring variability between models, not model confidence. We propose Flow Matching with Confidence (FMwC). FMwC injects input-dependent multiplicative noise at selected layers, propagates its variance through the network in closed form, and integrates it along the ODE trajectory, yielding a per-sample confidence score at standard sampling cost. The score supports multiple uses: filtering improves image quality and thermodynamic stability of
crystals; editing rewinds trajectories to the points where the model commits and redirects them; and adaptive stepping concentrates ODE compute where the flow is ambiguous. We find that the confidence score correlates with the magnitude of the divergence of the learned velocity field, which gives us a window to understand the generative process, opening up
surgical forms of guidance that target the moments that matter, new sampling
algorithms and interpretability of generative models.
Read the full abstract ▾
All authors Friso de Kruiff, Dario Coscia, Max Welling, Erik Bekkers
Machine Learning · Cross-listed from stat.ML
Shallow ReLU$^s$ Networks in $L^p$-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization
Weizhao Li, Fanghui Liu and Lei Shi
This paper studies approximation by shallow ReLU$^s$ networks, $\sigma_s(t)=\max\{0,t\}^s$, together with their generalization behavior under $\ell_1$ path-norm control. For the $L^p$-type integral spaces $\widetilde{\mathcal{F}}_{p,\tau_d,s}$, $1\le p\le2$, spherical harmonic analysis yields approximation bounds for shallow networks. In particular, when $\tau_d$ is the uniform measure and $1\le p<2$, the approximation rate is $O\!\left(m^{-\frac{p(2s+2d+1)-2d}{2dp}}\right)$ for $1\le p\le p^*$ and $O\!\left(m^{-\frac{p(4s+3d-1)-2d+2}{4dp}}\right)$ for $p^*<p<2$, where $p^*=\frac{2d+2}{d+3}$. Approximation bounds for Sobolev spaces $W^{\alpha,p}$, $1\le p<2$, are obtained through
embeddings into spectral Barron spaces. For nonparametric regression with sub-Gaussian noise, path-norm-regularized shallow ReLU$^s$ networks achieve minimax-optimal rates $O\!\left(n^{-\frac{d+2s+1}{2d+2s+1}}\log n\right)$ over $\mathscr{B}_s$ and $O\!\left(n^{-\frac{2\alpha}{2\alpha+d}}\log n\right)$ over $W^{\alpha,\infty}$, with matching lower bounds up to logarithmic factors.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
When Fireflies Cluster; Enhancing Automatic Clustering via Centroid-Guided Firefly Optimization
MKA Ariyaratne, Azwirman Gusrialdi, Yury Nikulin et al. (+1 more)
This work presents a novel variant of the Firefly
Algorithm (FA) for data clustering, addressing limitations of traditional methods like K-Means that struggle with non-uniform cluster shapes, densities, and the need for pre-defining the number of clusters. The proposed
algorithm introduces a centroid movement strategy and a multi-objective fitness function that balances compactness, separation, and a novel TSP-based
navigation penalty. It automatically estimates the optimal number of clusters and dynamically adjusts cluster boundaries. Application to robotic sensor networks highlights its practical value, with experiments showing improved clustering quality and reduced intra-cluster path distances compared to K-Means. These results confirm the
algorithm's robustness in complex spatial clustering tasks, with potential for future extensions to higher-dimensional and adaptive scenarios.
Read the full abstract ▾
All authors MKA Ariyaratne, Azwirman Gusrialdi, Yury Nikulin, Jaakko Peltonen
Machine Learning
Adaptive Experimentation for Censored Survival Outcomes
Yuxin Wang, Dennis Frauen, Jonas Schweisthal et al. (+3 more)
Adaptive experimentation enables efficient estimation of causal effects, but existing methods are not designed for survival data with censoring, where event times are only partially observed (e.g., overall survival in cancer trials but with dropout). In this paper, we develop a novel framework for adaptive experimentation to estimate causal effects under right censoring. For this, we derive the semiparametric efficiency bound for the average survival effect curve as a function of the treatment allocation policy and thereby obtain a closed-form efficiency-optimal allocation policy. The policy generalizes classical Neyman allocation to survival settings by prioritizing patient strata where both event and censoring dynamics induce high uncertainty. Building on this, we propose the Adaptive Survival
Estimator (ASE), an adaptive framework that learns the allocation policy and estimates the average survival effect curve sequentially. Our framework has three main benefits: (i) it accommodates arbitrary machine learning models for nuisance estimation; (ii) it is guided by a closed-form efficiency-optimal allocation policy; and (iii) it admits strong theoretical guarantees, including asymptotic normality via a martingale central limit
theorem. We demonstrate our framework across various numerical experiments to show consistent efficiency gains over uniform randomization and censoring-agnostic baselines.
Read the full abstract ▾
All authors Yuxin Wang, Dennis Frauen, Jonas Schweisthal, Maresa Schröder, Emil Javurek, Stefan Feuerriegel
Machine Learning
Scheduling That Speaks: An Interpretable Programmatic Reinforcement Learning Framework
Chengpeng Hu, Yingqian Zhang and Hendrik Baier
Deep reinforcement learning (DRL) has recently emerged as a promising approach to solve combinatorial optimization problems such as job shop scheduling. However, the policies learned by DRL are typically represented by deep neural networks (DNNs), whose opaque neural architectures and non-interpretable policy decisions can lead to critical trust and usability concerns for human decision makers. In addition, the computational requirements of DNNs can further hinder practical deployment in resource constrained environments. In this work, we propose ProRL, a novel interpretable programmatic reinforcement learning framework that achieves high-performance scheduling with human-readable and editable programmatic policies (i.e., programs). We first introduce a domain-specific
language for scheduling (DSL-S) to represent scheduling strategies as structured programs. ProRL then explores the program space defined by DSL-S using local search to identify incomplete programs, which are subsequently completed by learning their parameters via
Bayesian optimization. ProRL learns which scheduling heuristic rules to select, and hence, it naturally incorporates existing heuristics already used in industrial scenarios. Experiments on widely used
benchmark instances demonstrate the strong performance of ProRL against existing heuristics and DRL baselines. Furthermore, ProRL performs well under strongly constrained computational resources, such as training with only 100 episodes. Our code is available at https://github.com/HcPlu/ProRL.
Read the full abstract ▾
Machine Learning
Modelling Customer Trajectories with Reinforcement Learning for Practical Retail Insights
Ken Ming Lee, Paul Barde, Maxime C. Cohen et al. (+1 more)
Understanding customer movement within retail spaces is essential for optimizing store layouts. Real-world trajectory data can provide highly accurate insights, but collecting it is costly and often infeasible for many retailers. Heuristics such as Travelling Salesman Problem (TSP) and Probabilistic Nearest Neighbours (PNN) are commonly used as inexpensive approximations, but actual customer trajectories deviate by an average of 28% from shortest paths, highlighting a tradeoff between accuracy and practicality. We propose an
agent-based modelling framework that casts customer trajectory prediction as a maximum
entropy reinforcement learning (RL) problem, balancing reward maximization with stochasticity to better reflect customers with bounded rationality. Using real-world trajectory data from a convenience store, we show that RL-generated trajectories align more closely with customer behaviour than TSP and PNN, providing more accurate estimates of impulse purchase rates and shelf traffic densities. Furthermore, only RL-based predictions yield repositioning decisions for impulse products that align with those derived from actual trajectory data, resulting in comparable estimated profit gains. Our work demonstrates that RL provides a practical, behaviourally grounded alternative that bridges the gap between oversimplified heuristics and data-intensive approaches, making accurate layout optimization more accessible. To encourage further research, the source code is available on GitHub.
Read the full abstract ▾
All authors Ken Ming Lee, Paul Barde, Maxime C. Cohen, Derek Nowrouzezahrai
Machine Learning · Cross-listed from cs.CV
What's Holding Back Latent Visual Reasoning?
André G. Viveiros, Nuno Gonçalves, André F. T. Martins et al. (+1 more)
Humans can approach complex visual problems by mentally simulating intermediate visual steps, rather than
reasoning through
language alone. Inspired by this, several works on
Vision-
Language Models have recently explored chain-of-thought
reasoning with continuous latent
tokens as intermediate visual imagination steps. In this work, we investigate how recent models leverage such latent
tokens. Surprisingly, we find that model accuracy is unaffected when latent
tokens are replaced by uninformative dummy
tokens. This indicates that latent
tokens play a minimal causal role in the model's final prediction. To better understand this phenomenon, we analyze both the training signal provided by oracle latent representations and the quality of the latent
tokens generated at inference time. Our experiments reveal two crucial issues holding back latent visual
reasoning: First, in most existing
datasets, oracle latent
tokens provide limited additional information beyond the original image and do not substantially simplify the task, leading models to ignore them during training and effectively bypassing them at inference time. When fine-tuned on a diagnostic
dataset, in which latent
tokens provide sufficient support for the final prediction, we show that models can causally rely on them. Second, the latent
tokens produced at inference time deviate from their corresponding oracle representations, collapsing to a narrow region and preventing benefits even when the model relies on them. Overall, our findings suggest that future progress in latent visual
reasoning depends on two key pillars: high-quality
datasets with informative intermediate steps and more precise latent
token prediction.
Read the full abstract ▾
All authors André G. Viveiros, Nuno Gonçalves, André F. T. Martins, Matthias Lindemann
Machine Learning
Heterogeneous Tasks Offloading in Vehicular Edge Computing: A Federated Meta Deep Reinforcement Learning Approach
Yaorong Huang, Jingtao Luo and Xuechao Wang
Vehicular edge computing (VEC) enables latency-sensitive vehicular applications by offloading computation-intensive tasks to nearby edge servers. However, real-world vehicular workloads are typically modeled as heterogeneous directed acyclic graph (DAG) tasks with complex dependency structures, making joint offloading and resource allocation highly challenging. Moreover, distributed MEC deployment raises
privacy concerns when collaboratively training learning-based policies. In this paper, we propose a
Federated Meta Deep Reinforcement Learning framework with GAT-Seq2Seq modeling (FedMAGS) for heterogeneous task offloading in VEC systems. The proposed approach leverages Graph
Attention Networks to capture DAG dependencies, a Seq2Seq-based policy to generate structured offloading decisions, and
federated meta-learning to enable fast adaptation across distributed MEC servers without sharing raw data. Extensive simulations demonstrate that FedMAGS achieves faster convergence, lower execution delay, and better scalability compared with state-of-the-art baselines. In addition, the
federated design preserves data
privacy while reducing communication overhead, making the framework well suited for dynamic and large-scale VEC environments.
Read the full abstract ▾
Machine Learning
Text2CAD-Bench: A Benchmark for LLM-based Text-to-Parametric CAD Generation
Liang Wang, Heng Meng, Zekai Xiang et al. (+4 more)
Text-to-CAD generation aims to create parametric CAD models from natural
language, enabling rapid prototyping and intuitive design workflows. However, existing
benchmarks focus on basic primitives and simple sketch-extrude sequences, lacking advanced features essential for real-world applications and covering only traditional mechanical parts. We introduce Text2CAD-Bench, the first
benchmark systematically evaluating text-to-CAD across
geometric complexity and application diversity. Our
benchmark comprises 600 human-curated examples spanning four levels: L1-L2 cover fundamental geometry with standard features, L3 introduces complex
topology and freeform surfaces, and L4 extends to real-world domains beyond mechanical parts. Each example pairs dual-style
prompts –
geometric descriptions mimicking non-expert users, and procedural sequences aligned with expert-level conventions. Evaluating mainstream general LLMs and domain-specific models, we find that current models perform reasonably on basic geometry but degrade substantially on complex
topology and advanced features. We release our
benchmark to drive progress in text-to-CAD research.
Read the full abstract ▾
All authors Liang Wang, Heng Meng, Zekai Xiang, Jin Liu, Pingyi Zhou, Litao Chen, Yongqiang Tang
Machine Learning
Generative Adversarial Learning from Deterministic Processes
Joris C. Kühl and Hanno Gottschalk
Physical AI is being successfully applied to data which does not follow the traditional paradigm of independent and identically distributed (i.i.d.) samples. In fact, physical AI is often trained on data which is not random at all, and is instead derived from chaotic dynamical systems like
turbulence. We aim to explain the empirical success of these methods using the example of generative
adversarial networks (GANs), whose statistical learning theory under the i.i.d. assumption is generally well understood. We prove that it is possible, using an infinite-dimensional model of generative
adversarial learning (GAL), to learn the invariant distribution of a sufficiently chaotic dynamical system from a single deterministically evolving time series of its states or measurements thereof, and give explicit rates for the convergence to the solution in terms of the Jensen-Shannon divergence.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Generalized Functional ANOVA: A Complete Theoretical Framework
Baptiste Ferrere, Nicolas Bousquet, Fabrice Gamboa et al. (+1 more)
The functional ANOVA provides a fundamental representation of square-integrable multivariate functions into main effects and higher-order interactions. For independent inputs, the components belong to mutually orthogonal
Hilbert subspaces and admit an explicit representation. For dependent inputs, however, the components are only hierarchically orthogonal: although existence and uniqueness results are available, the
Hilbert subspaces underlying the generalized decomposition have remained implicit. We resolve this representation problem for continuous inputs supported on a bounded hyperrectangle whose joint density is bounded above and away from zero. We introduce a distribution-adapted family of functions and prove that it forms a Riesz basis of the $L^2$ space, thereby guaranteeing a unique, stable, and unconditionally convergent representation. We then show that, for every coalition of variables, the corresponding block of this basis exactly characterizes the
Hilbert subspace containing the functional ANOVA component. Our construction recovers the classical orthogonal decomposition under input independence. As a direct consequence, computing the generalized functional ANOVA reduces to estimating coefficients in an explicit, distribution-adapted basis. Finally, as a proof of concept, we introduce an elementary, fast and model-agnostic
estimator based on our theoretical results. Experiments on synthetic and real-world
datasets illustrate its connections with established tabular explanation methods and show that low order components often capture most of the signal in the model output.
Read the full abstract ▾
All authors Baptiste Ferrere, Nicolas Bousquet, Fabrice Gamboa, Jean-Michel Loubes
Machine Learning · Cross-listed from cs.CL
EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective
Yuyao Wang, Zhongjian Zhang, Mo Chi et al. (+7 more)
Recent
benchmarks for Large
Language Model (LLM)
agents mainly evaluate
reasoning, planning, and execution. However,
memory is also essential for
agents, as it enables them to store, update, and retrieve information over time. This ability remains under-evaluated, largely because existing
benchmarks do not provide a systematic way to assess
memory mechanisms. In this paper, we study
agent memory from a self-evolving perspective and introduce EvoMemBench, a unified
benchmark organized along two axes:
memory scope (in-episode vs. cross-episode) and
memory content (knowledge-oriented vs. execution-oriented). We compare 15 representative
memory methods with strong long-context baselines under a standardized protocol. Results show that current
memory systems are still far from a general solution: long-context baselines remain highly competitive,
memory helps most when the current context is insufficient or tasks are difficult, and no single
memory form works consistently across all settings.
Retrieval-based methods remain strong for knowledge-intensive settings, whereas procedural and long-term
memory methods are more effective for execution-oriented tasks when their stored experience matches the task structure. We hope EvoMemBench facilitates future research on more effective
memory systems for LLM-based
agents. Our
dataset and code are available at https://github.com/DSAIL-
Memory/EvoMemBench.
Read the full abstract ▾
All authors Yuyao Wang, Zhongjian Zhang, Mo Chi, Kaichi Yu, Yuhan Li, Miao Peng, Bing Tong, Chen Zhang, Yan Zhou, Jia Li
Machine Learning
Spherical Harmonic Optimal Transport: Application to Climate Models Comparisons
Pierre Houédry, Iskander Legheraba, Léo Buecher et al. (+1 more)
Optimal transport provides a powerful framework for comparing measures while respecting the geometry of their support, but comes with an expensive computational cost, hindering its potential application to real world use cases. On
manifolds, convolutional
algorithms based on the heat kernel have been proposed to alleviate this cost, but their theoretical properties remain largely unexplored. We establish that the heat kernel cost converges to the optimal transport cost as time vanishes in the balanced and unbalanced cases. In the specific case of the 2-sphere $\mathbb{S}^2$, we ensure that the associated Sinkhorn divergences retains the desirable
geometric and analytic properties of classical optimal transport discrepancies. Moreover, we leverage the harmonic structure of the sphere to derive a fast Sinkhorn
algorithm, requiring only $\mathcal{O}(n)$
memory and $\mathcal{O}(n^{3/2})$ time per iteration, with fully dense GPU-friendly operations. We validate its computational efficiency on synthetic data, and discuss its potential use in the evaluation of global
climate models, providing both spatial and seasonal insights into models performances.
Read the full abstract ▾
All authors Pierre Houédry, Iskander Legheraba, Léo Buecher, Nicolas Courty
Machine Learning
Graph Hierarchical Recurrence for Long-Range Generalization
Stefano Carotti, Marco Pacini, Alessio Gravina et al. (+3 more)
Graph Neural Networks and Graph
Transformers have become central to graph learning, combining expressive representation learning with sample-efficient inductive biases. Yet they remain fundamentally limited when predictions depend on correlations between distant graph regions. We address this limitation with Graph Hierarchical Recurrence (GHR), a novel framework that jointly operates on the input graph and a pooled hierarchical abstraction. We also show that existing models degrade more sharply under out-of-range generalization, where test instances require interactions across distances exceeding those observed during training. Despite its minimal design, GHR consistently strengthens every tested message-passing backbone, yielding robust performance on long-range dependencies and particularly pronounced gains in out-of-range regimes. Across a broad suite of long-range
benchmarks, GHR achieves state-of-the-art or competitive results on multiple tasks, establishing hierarchical recurrence as an effective mechanism for extending graph models beyond their observed interaction range.
Read the full abstract ▾
All authors Stefano Carotti, Marco Pacini, Alessio Gravina, Davide Bacciu, Bruno Lepri, Sebastiano Bontorin
Machine Learning
TabH2O: A Unified Foundation Model for Tabular Prediction
Pascal Pfeiffer, Dmitry Gordeev, Mathias Müller et al. (+6 more)
We present TabH2O, a foundation model for tabular data that performs classification and regression in a single forward pass via in-context learning. TabH2O builds on the TabICL architecture with several key modifications: (1) unified training, a single model handles both classification and regression via a dual-head architecture, eliminating the need for separate models and reducing total pretraining cost; (2) single-stage pretraining, training stability improvements (bounded scalable softmax, inter-stage normalization, learnable residual scaling, logit soft-capping) eliminate the need for multi-stage curriculum learning, enabling training with full-length sequences from the start; and (3) noise-aware pretraining, synthetic
datasets include explicit noise dimensions to teach the model robustness to irrelevant features. We evaluate TabH2O v1.1 (29.2M parameters) on the TALENT
benchmark (300
datasets), where it achieves an average rank of 2.54 out of 6 evaluated methods, outperforming tuned CatBoost (4.00), H2O AutoML (4.28), LightGBM (4.98), and TabPFN v2.6 (2.80), and competitive compared to TabICL v2 (2.13), while placing in the top-3 on 80% of the testing
datasets across classification and regression tasks. We also achieve state-of-the-art results on TabArena. Moreover, our single model is well-suited to tabular
datasets up to roughly 500,000 rows and 100 features.
Read the full abstract ▾
All authors Pascal Pfeiffer, Dmitry Gordeev, Mathias Müller, Laura Fink, Joan Salvà Soler, Mark Landry, Branden Murray, Marcos V. Conde, Sri Satish Ambati
Machine Learning
Generating Physically Consistent Molecules with Energy-Based Models
Christoph Griesbacher, Lea Bogensperger, Andreas Habring et al. (+1 more)
Molecules in
equilibrium follow a Boltzmann distribution, making the underlying energy landscape a physically grounded modeling objective. However, such landscapes are difficult to learn from data and, once learned, hard to sample from.
Diffusion and flow-matching models sidestep these difficulties by learning a time-conditional score or transport field between noise and data, losing the energy inductive bias in exchange for a more tractable training objective. We introduce EBMol, an energy-based model (EBM) that restores this inductive bias by learning an atom-additive scalar potential without explicit simulation during training. Our method employs a flow-inspired Restoring Field Matching objective to approximate the energy landscape. We adopt the Mirror-Langevin
algorithm for sampling, enabling unified updates of atomic positions and types, and incorporate parallel tempering for inference-time compute scaling. EBMol is the first EBM for 3D molecular generation to achieve state-of-the-art performance on QM9 and GEOM-Drugs. Moreover, we show that the learned energy landscape serves as a principled quality metric for ranking and filtering configurations, and demonstrate controllable generation without retraining through shape-steered sampling via potential composition and zero-shot linker design.
Read the full abstract ▾
All authors Christoph Griesbacher, Lea Bogensperger, Andreas Habring, Thomas Pock
Machine Learning
Beyond Square Roots: A Memory-Efficient Explicit Factorization for Multi-Epoch Private Learning
Nikita P. Kalinin, Aki Rehn, Joel Daniel Andersson et al. (+2 more)
Correlated-noise mechanisms are among the most promising approaches for improving the utility of differentially private model training, but rigorous guarantees require explicit, analyzable factorizations, and practical deployment requires
memory efficiency. Recent works have developed banded inverse factorizations, which address both requirements by exploiting a banded structure in the correlation
matrix. The bandwidth controls the size of the noise buffer used to correlate noise across iterations, and thus governs the tradeoff between utility and
memory cost. At the two ends of this tradeoff, DP-$\lambda$CGD achieves high
memory efficiency by using only a one-step noise buffer, but this limits its utility gains, while the banded inverse square root (BISR) factorization exploits larger correlation windows and is asymptotically optimal for large bandwidths but performs poorly at low bandwidths. To bridge this gap, we introduce $\gamma$-BIFR, a unified generalization of both factorizations. In the low-
memory, low-bandwidth regime, $\gamma$-BIFR improves RMSE, amplified RMSE, and private training performance, while yielding tighter theoretical guarantees for multi-participation error in multi-epoch training.
Read the full abstract ▾
All authors Nikita P. Kalinin, Aki Rehn, Joel Daniel Andersson, Antti Honkela, Christoph H. Lampert
Machine Learning
Beyond Inference-Time Search: Reinforcement Learning Synthesizes Reusable Solvers
Soheyl Massoudi, Gabriel Apaza, Milad Habibi et al. (+1 more)
Large
language models (LLMs) typically approach combinatorial optimization as an inference-time procedure, solving each instance separately through sampling, search, or repeated prompting. We ask whether reinforcement learning can instead shift part of this
reasoning cost into the weights of a code LLM, so that the model synthesizes a reusable solver for an entire problem family. We study this question on Synergistic Dependency Selection (SDS), a controlled variant of constrained Quadratic Knapsack designed to expose a specific failure mode: local signals and strict feasibility constraints make greedy heuristics attractive but unreliable. Under identical scaffolding, Best-of-64 base-model sampling saturates at an approximately 28.7% gap to the global Virtual Best Solver (VBS); code audits show that the base model often retrieves Simulated Annealing templates but misimplements the Metropolis acceptance rule. We fine-tune Qwen2.5-Coder-14B-Instruct with Group Relative Policy Optimization (GRPO) using a feasibility-gated reward and light structural scaffolding. The resulting policy converges to a constraint-aware Simulated Annealing template in 99.8% of feasible SDS outputs, achieves a 5.0% gap to that VBS, and is 91 times cheaper in post-generation execution/search cost than cumulative Best-of-64 evaluation. A compile-once check shows that one best frozen solver per seed remains highly competitive when reused unchanged across the SDS test set, while an additional-domain evaluation on Job Shop Scheduling provides narrower but positive evidence that the scaffold transfers beyond SDS. Negative ablations reveal the limits of this recipe: standard stabilizers degrade performance, a soft feasibility gate fails, and results remain sensitive to reward normalization and domain-specific design choices.
Read the full abstract ▾
All authors Soheyl Massoudi, Gabriel Apaza, Milad Habibi, Mark Fuge
Machine Learning · Cross-listed from cs.RO
Dynamic robotic cloth folding with efficient Koopman operator-based model predictive control
Edoardo Caldarelli, Franco Coltraro, Adrià Colomé et al. (+2 more)
Robotic cloth folding is a challenging task, particularly when considering dynamic folding tasks, which aim at folding cloth by fast motions that leverage its dynamics. When subject to such fast motions, the complexity of cloth dynamics hinders both system identification and planning of folding trajectories, resulting in a difficult simulation-to-reality transfer when using physical models of cloth. Compared to the dexterity that humans exhibit when performing folding tasks, robotic approaches usually employ small garments with quite rigid dynamics, and are either too slow, or fast but imprecise, requiring several attempts to achieve a reasonably good fold. In this paper, we tackle these challenges by generating fast folding trajectories with a novel model predictive controller, integrating physics-based simulation of cloth dynamics and efficient, kernel-based Koopman operator regression. Koopman operator regression, an increasingly popular machine learning technique for nonlinear system identification, is used to obtain a linear model for the cloth being folded. Such a surrogate model, trained with data from a high-fidelity, physics-based cloth simulator, can then be employed within a suitable model predictive control
algorithm, in place of the costly, nonlinear one, to efficiently generate folding trajectories to be executed by a robotic manipulator. Both in simulated and real-
robot experiments, we show how the linearization supplied by the Koopman operator-based model can be employed to efficiently generate fast folding trajectories to unseen poses, without sacrificing folding accuracy.
Read the full abstract ▾
All authors Edoardo Caldarelli, Franco Coltraro, Adrià Colomé, Lorenzo Rosasco, Carme Torras
Machine Learning · Cross-listed from stat.ML
On Stability and Decomposition of Sample Quantiles under Heavy-Tailed Distributions
Choudur Lakshminarayan
We study sample
quantiles of distributions indexed by estimated parameters, with a on Value-at-Risk related to linear projections of financial returns that whose underlying probability law is heavy-tailed. In this setting, the projection direction and the empirical
quantile threshold are estimated from the data, so the standard Bahadur representation under a fixed distribution does not separate the distinct sources of instability. A canonical starting point is Bahadur's representation, which expresses the sample
quantile through the empirical distribution function plus a remainder term \cite{bahadur1966}. Empirical-process theory provides a usable scaffolding through the mechanics of half-spaces, symmetric differences, and Glivenko–Cantelli uniform convergence. They yield stability bounds, but absorb changes in projection direction and changes in
quantile threshold into a single symmetric-difference measure. Interestingly, a global uniform-convergence requirement is imposed on what is intrinsically a local
quantile-stability problem. This paper introduces a Q-Q orthogonality formulation for separating projection-direction and
quantile-threshold effects. The object of interest is the difference between the empirical
quantile computed using the estimated projection direction and the population
quantile computed at the reference projection direction. We decompose this difference into three terms, $\hat q_{\alpha}(\hat w)-q_{\alpha}(w_0)=D_1+D_2+D_3$. Here, $D_1$ measures the population
quantile movement induced by perturbing the projection direction, $D_2$ measures the empirical
quantile fluctuation with the projection direction held fixed, and $D_3$ is the Bahadur-type remainder.
Read the full abstract ▾