Machine Learning · Cross-listed from cs.AI
Norm Enforcement for AI Agents: Robustly Shaping Behavior in Multi-Agent Systems
Yaowen Ye and Jacob Steinhardt
AI
agents are increasingly deployed in shared environments where they pursue diverse goals and compete for rewards. This multi-
agent competition can lead to behaviors that serve individual gains at collective cost – for instance, marketing
agents may post misleading content as a result of competing for engagement on social media. Human societies address such problems through norms that constrain acceptable behavior, supported by enforcement mechanisms that detect and penalize violations. Motivated by this, we study norm enforcement mechanisms for
language model
agents. We find that simple enforcement mechanisms are exploited by misaligned
agents for competitive advantage, even when they are not explicitly trained or prompted to do so. We thus turn our
attention to designing more robust mechanisms, and identify two key ingredients: estimating each
agent's reliability over time, and updating this estimate with escalating penalties for repeated misbehavior. Across three simulated environments and a variety of
agent populations, mechanisms built on these principles resist exploitation, while still penalizing norm violations at comparable or lower cost than baselines. Our results position norm enforcement mechanisms as scalable levers for shaping
agents' behavior, but only when designed to anticipate becoming part of the system they govern. Our code and data are available at https://yaowenye.com/norm-enforcement.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Knowledge-Constrained Shape Optimization with a Mixture-of-Experts Neural Operator for High-Confidence Design
Wenhao Fan, Yuanwei Bin, Jianghan Gu et al. (+4 more)
Engineering shape optimization faces challenges in both expert-dependent problem setup and surrogate-model reliability. In practical aerodynamic design, optimization settings such as editable regions, deformation ranges, and design-preservation constraints are typically specified manually by experienced engineers, while surrogate-based optimization may become unreliable for heterogeneous geometry databases and out-of-distribution designs. To address these challenges, we propose a knowledge-constrained shape-optimization framework that translates knowledge-based constraints and user intent into quantifiable parameters of DFFD-based deformation operators, enabling engineering-aware and controllable constrained optimization. We further develop a Mixture-of-Experts Neural Operator (MoE-NO) to improve drag prediction and trend consistency over heterogeneous aerodynamic
datasets. Based on the MoE-NO encoder and Mahalanobis distance, an uncertainty-estimation strategy is introduced to detect out-of-distribution geometries and selectively trigger physics-solver feedback for local sample enrichment. Experiments on in-house MPV, SUV, and Sedan
datasets show that MoE-NO achieves a test-set MAPE of $1.16\%$ and a trend-prediction accuracy of $94.34\%$, outperforming the best baseline results of $1.52\%$ and $90.34\%$, respectively. Vehicle shape-optimization experiments further yield CFD-validated drag coefficient reductions of approximately $4\%$ to $10\%$.
Read the full abstract ▾
All authors Wenhao Fan, Yuanwei Bin, Jianghan Gu, Wenfa Luo, Jiao Xiang, Yuntian Chen, Shiyi Chen
Machine Learning · Cross-listed from stat.ML
EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins
Joshua Pickard, Wei Qi, Na Li et al. (+4 more)
Sepsis is a leading cause of mortality, yet optimal treatment policies remain contested. Existing reinforcement learning (RL) approaches learn fixed strategies for sepsis treatment, limiting adaptability to changing clinical objectives during inference. We propose EHRMPC, a framework that decouples learning patient dynamics from optimizing treatment by training a patient digital twin in the form of a generative electronic health record (EHR) model. The digital twin predicts clinical trajectories under interventions and enables model predictive control (MPC) to optimize treatments via inference-time planning over simulations. We evaluate EHR-MPC on a multicenter ICU sepsis cohort spanning 8 hospitals in the Mass General Brigham health system using both off-policy importance sampling and on-policy simulation-based evaluation. Relative to RL baselines, EHR-MPC achieves comparable off-policy performance and improved simulation performance. Unlike RL, this work frames sepsis treatment optimization as inference-time control over learned patient dynamics, establishing a general framework for decision making with generative clinical models.
Read the full abstract ▾
All authors Joshua Pickard, Wei Qi, Na Li, Ann Woolley, Lisa Cosimi, Roy Kishony, Deborah Hung
Machine Learning · Cross-listed from cs.RO
SASGeo: Stability-Aware Semantic Map Localization for GNSS-Denied UAVs – A Framework and Synthetic Proof of Concept
Natalia Trukhina and Vadim Vashkelis
GNSS-denied unmanned aerial vehicles require occasional absolute position fixes to bound the drift of visual-inertial odometry. Cross-view image
retrieval can provide such fixes, but raw appearance is sensitive to season, illumination, viewpoint, map age, and sensor modality. We propose \sas, a semantic map-localization framework that represents the environment through persistent structures such as roads, buildings, waterways, railways, intersections, and field boundaries. The method combines semantic raster
alignment, relational graph evidence, feature stability and geographic distinctiveness, explicit positive/contradictory/unknown observations, and integrity-aware rejection of ambiguous fixes. Unlike a broad architecture-only proposal, this paper specifies concrete weighting and decision models and reports a reproducible synthetic proof of concept. In 220 randomized
retrieval trials with rotation, scale changes, partial crops, occlusion, simulated map changes, and hard semantic decoys, a global semantic descriptor achieved 58.6% Recall@1, while spatial semantic matching variants achieved 94.5-95.5%. Wilson 95% intervals separate the global descriptor from the spatial variants but overlap among the spatial variants, so the experiment supports semantic geometry rather than a definitive benefit from each proposed module. The preliminary experiment does not validate real-flight
navigation; rather, it demonstrates that structured semantic geometry can discriminate locations under controlled cross-view perturbations and identifies the harder aliasing, map-aging, and rejection tests required next.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
The Regularization Parameter: Sparse Precision Matrix Estimation
Aryan Eftekhari, Daniel Sergio Vega, Ernst-Jan Camiel Wit et al. (+1 more)
Sparse precision
matrix estimation provides an interpretable and computationally efficient framework for modeling conditional dependencies in high-dimensional, low-sample-size data. A recurring challenge is appropriately selecting the regularization parameter that controls
estimator sparsity and strikes a balance between underfitting and overfitting. We propose a closed-form,
matrix-valued regularization parameter derived from the sampling distribution of the first-order optimality conditions of the $\ell_1$-regularized Gaussian maximum-likelihood
estimator. By prescribing the probability that each nonzero entry of the
estimator satisfies its optimality condition under resampling, we eliminate the need for cross-validation. The resulting regularization parameter is shown to attain asymptotic scaling properties that, under standard conditions, provide consistency and sparsistency of the
estimator. On synthetic Gaussian and non-Gaussian
datasets, as well as real-world gene microarray and neuroimaging applications, the proposed approach achieves estimation accuracy comparable to cross-validation, delivers superior support recovery, and reduces runtime by several orders of magnitude.
Read the full abstract ▾
All authors Aryan Eftekhari, Daniel Sergio Vega, Ernst-Jan Camiel Wit, Olaf Schenk
Machine Learning · Cross-listed from cs.SD
A Self-Supervised Approach for Minimal-Annotation Hydroacoustic Data Exploration
Pierre-Yves Raumer, Axel Marmoret, Dorian Cazau et al. (+6 more)
Passive hydroacoustic monitoring often generates large volumes of continuous recordings that are only partially exploited due to the cost of manual annotation. Supervised detection methods perform well but require large labeled
datasets, seldom available for rare signals or understudied environments. This work proposes a self-supervised exploration pipeline to address this limitation in low-frequency settings. A Masked AutoEncoder (MAE) is pre-trained on a reconstruction pretext task, then used to extract patch-level representations from spectrograms. Within each spectrogram, adjacent informative patches are aggregated into event-level
embeddings, enabling the disentanglement of overlapping events. These
embeddings are then clustered at the
dataset scale using the dimension reduction
algorithm UMAP and the clustering
algorithm HDBSCAN to identify hydroacoustic patterns. The pipeline was applied to a multi-year hydroacoustic
dataset collected near Mayotte Island, Indian Ocean, containing marine mammal vocalizations, seismo-volcanic signals, and anthropogenic noise. The 317 clusters were manually mapped to 15 hydroacoustic classes or noise in less than one hour. The method was evaluated in two ways. Quantitatively, when used as a classifier, it achieved performance comparable to two existing detectors. Qualitatively, it recovered known seasonal patterns of marine mammal acoustic activity. It also identified patterns of previously unstudied signals, thereby demonstrating its practical value.
Read the full abstract ▾
All authors Pierre-Yves Raumer, Axel Marmoret, Dorian Cazau, Anatole Gros-Martial, Richard Dreo, Maelle Torterotot, Sara Bazin, Flore Samaran, Jean-Yves Royer
Machine Learning · Cross-listed from cs.CV
Gen4U: Unifying Video Generation and Understanding via Diffusion
Michael King, Aravindh Mahendran, Matthew Koichi Grimes et al. (+5 more)
Prior work suggests that
diffusion representations capture low-level geometry but struggle with high-level semantics. We demonstrate that state-of-the-art
video diffusion models overcome this limitation. By systematically probing their intermediate activations using recent mutual-kNN
alignment metrics, we reveal a highly structured latent space where visual representations evolve across both network depth and noise levels. We show that while moderate noise levels yield linearly separable global semantics, fine-grained details persist at lower noise levels but become spatially scattered, requiring
attention mechanisms to decode. Building on these insights, we introduce Gen4U (Generation for Understanding), a framework that repurposes these generative representations with a single forward pass. Our experiments establish that frozen, large-scale
video diffusion models function as highly competitive
video encoders across a wide spectrum of tasks, spanning semantic and non-semantic objectives (
video classification, depth estimation, camera pose estimation, image and
video captioning). Bypassing fine-tuning, Gen4U unifies the generation and understanding paradigms, achieving strong perception performance while fully preserving the model's ability to generate high-quality
video.
Read the full abstract ▾
All authors Michael King, Aravindh Mahendran, Matthew Koichi Grimes, Fedor Kitashov, Adham Elarabawy, Pedro Velez, Maks Ovsjanikov, Viorica Pătrăucean
Machine Learning
Josip Jukić and Ivan Titov
On-policy
distillation provides dense teacher supervision on a
language model's own trajectories. In self-
distillation with privileged context, this supervision comes from the model itself, conditioned on a hint or solution trace hidden from the student. When the teacher's preferences hinge on privileged information, it can assign higher probability to continuations the student cannot infer from its own context. Matching these preferences throughout training can induce predictive drift and degrade out-of-distribution (OOD)
reasoning. We propose GeoSD, a self-
distillation method that controls this drift through two complementary
geometric terms. A Hellinger loss weights each teacher preference by the student–teacher overlap, reducing the influence of
tokens to which the student assigns low probability. Because these influences can still accumulate, a Fisher–Rao penalty regulates predictive distance from a copy of the student refreshed periodically during training. Both terms compare next-
token distributions in Fisher–Rao geometry and are jointly optimized with a preconditioner motivated by the natural gradient. Across three model families, GeoSD retains strong in-distribution gains while improving average mathematical OOD accuracy by 5.7–8.6 points over the base model. OOD gains hold across five model scales from 1.7B to 32B and transfer to code generation, where GeoSD improves code accuracy by 1.9 points on average despite distilling on mathematics alone. Our analysis of mathematical
reasoning shows that standard matching rapidly concentrates probability mass at high-
entropy states and that its samples confidently agree on incorrect answers. In contrast, GeoSD preserves alternative
token mass and reduces false consensus.
Read the full abstract ▾
Machine Learning
A Gold-Standard Study of What Makes a Lightweight Game-Playing Agent Strong
Nima Kelidari, Mohammadsaeed Haghi and Mahdi Salmani
Reinforcement learning
agents for imperfect-information card games are only as strong as the opponents they train against, and they are hard to grade, since they beat a random opponent over 99 percent of the time and only tie copies of themselves. So we build a strong, fixed, rule-based expert for Gin Rummy and use it only as a yardstick, never for training. It beats every
agent we trained 70 to 99 percent of the time. Across more than a hundred runs, we isolate what makes a lightweight
agent stronger. Trust region updates, a well-aimed reward, a curriculum of tougher opponents, warm starting, and keeping the best checkpoint all help, and stacking them lifts a self-play champion from about 30 to 36 percent against the expert. Several ideas did not pay off. Short-term and longer-term reward shaping, learned state
embeddings, imitation and DAgger, and a live large
language model opponent were each unhelpful, too slow, or too heavy to train at scale. Comparing MLP, convolutional, set-based,
attention, and recurrent encoders shows that extra capacity does little to break the ceiling, suggesting the limit is information rather than network size. We add standard baselines (neural fictitious self-play and information set Monte Carlo search) and confirm the approach carries over to Leduc Hold'em, where the optimum is computable. The result is a lightweight, game-agnostic recipe that trains competitive
agents without training on the expert, for any game a small model can handle, reported with robust statistics and released as a reusable package.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Tensor Train Diffusion: Leveraging Low-Rank Structures for High-Dimensional Score-Based Sampling
Robert Gruhlke, Julius Berner, David Sommer et al. (+1 more)
Diffusion models offer a powerful framework for sampling from complex probability densities by learning to reverse a noising process. A common approach involves solving for the time-reversed
stochastic differential equation (SDE), which requires the score function of the evolving sample distribution. The logarithm of this distribution's density is governed by a Hamilton-Jacobi-Bellman (HJB) type partial differential equation (PDE). However, current methods for solving this PDE, such as PINNs or trajectory-based techniques, often suffer from long training times and significant sensitivity to hyperparameter tuning. In this work, we introduce a novel and efficient solver for the underlying HJB equation based on the functional tensor train (FTT) format. The FTT representation leverages latent low-rank structures to efficiently approximate high-dimensional functions, enabling both model compression and rapid computation. By integrating this efficient representation with a backward-in-time iterative scheme derived from backward
stochastic differential equations (BSDEs), we develop a fast, robust and accurate sampling method. Our approach overcomes primary bottlenecks of existing techniques, enabling high-fidelity sampling from challenging target distributions with improved efficiency.
Read the full abstract ▾
All authors Robert Gruhlke, Julius Berner, David Sommer, Lorenz Richter
Machine Learning
LEMUR 2: Unlocking Neural Network Diversity for AI
Tolgay Atinc Uzun, Waleed Khalid, Saif U Din et al. (+17 more)
Existing NAS
benchmarks (e.g., NAS-Bench, NATS-Bench) cover only narrow, task-specific regions of the architectural design space and lack cross-domain or deployment-aware evaluation. LEMUR 2 introduces a large-scale, extensible framework unifying generative, evaluative, and deployment pipelines to unlock neural-network diversity. It comprises over 14,000 distinct architectures and more than 750,000 structured training records documenting model performance, hyperparameters, and task outcomes. These models were produced through AST-based code mutation, genetic and reinforcement-learning
evolution, generation of fractal architectures, and synthesis guided by a Large
Language Model (LLM). This includes deep models generated with the
retrieval-augmented system NN-RAG, which derived and used architectural motifs from over 900 PyTorch modules extracted from public repositories. LEMUR 2 further employs NN-VR and NN-Lite pipelines for automated deployment and latency benchmarking on heterogeneous mobile and Unity-based VR platforms, providing real-device performance metadata. It spans
multimodal tasks, image captioning, text-to-image synthesis, and
language modeling, supporting cross-domain analysis of architectural transferability. By linking diverse architectures, tasks, and deployment data, LEMUR 2 provides the data foundation for LLM fine-tuning and coupling diverse architectural origins with large-scale, cross-platform empirical validation. This
dataset defines a new basis for reproducible and data-driven AI design, advancing the emerging paradigm of LLM-driven AutoML and architectural generalization across modalities and hardware.
Read the full abstract ▾
All authors Tolgay Atinc Uzun, Waleed Khalid, Saif U Din, Sai Revanth Mulukuledu, Akashdeep Singh, Chandini Vysyaraju, Raghuvir Duvvuri, Avi Goyal, Yashkumar Rajeshbhai Lukhi, Muhammad A. Hussain, Krunal Jesani, Usha Shrestha, Yash Mittal, Roman Kochnev, Pritam Kadam, Mohsin Ikram, Harsh R. Moradiya, Alice Arslanian, Dmitry Ignatov, Radu Timofte
Machine Learning
Yiğit Berkay Uslu, Samar Hadou, Sergio Rozada et al. (+2 more)
Sampling
stochastic signals supported on a graph underlies many graph machine learning tasks, including recommender systems,
forecasting in financial
markets, and wireless network optimization. In these settings, the target signals are realizations of unknown conditional distributions. However, prevailing approaches rely mostly on intricate, application-tailored designs that often regress to a conditional mean instead of sampling from the conditional law. This paper unifies such problems as conditional graph signal generative modeling and tackles them with a single denoising
diffusion framework. We learn a reverse
diffusion process, parametrized by graph neural networks (GNNs), that draws graph signals conditioned directly on the graph
topology and on node-feature side information. The reverse process is realized by a novel architecture, the U-Graph Neural Network (U-GNN), which generalizes the image-convolutional U-Net to graph-structured signals. The U-GNN performs multi-resolution encoder–decoder processing in which pooling and unpooling reduce to a learned node selection, expressed by nested selection
matrices, and a zero-padded lifting of coarse signals back to the full node set. The graph convolutions are carried out on the original graph, with a stride that sets their hop reach, so the U-GNN bypasses explicit graph coarsening at every resolution. We demonstrate our method on two generative tasks: stock price
forecasting and optimal wireless resource allocation, with extensive numerical results in both domains.
Read the full abstract ▾
All authors Yiğit Berkay Uslu, Samar Hadou, Sergio Rozada, Shirin Saeedi Bidokhti, Alejandro Ribeiro
Machine Learning · Cross-listed from cs.CL
Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs
Albert Zeyer, Ralf Schlüter and Hermann Ney
Speech-to-text
alignment means finding the temporal boundaries of each word in the audio. Some models provide such an
alignment directly and others do not. Connectionist temporal classification (CTC) and transducer models have an
alignment by construction, whereas
attention-based encoder-decoders (AED) and
speech large
language models (LLMs) do not, and their word timings are usually read off the
attention weights instead. All of these signals live on the encoder frame grid, which bounds their temporal precision. We study a generic gradient-based
alignment that applies to any differentiable ASR model. We take the gradient of each teacher-forced
token log probability with respect to the input, reduce it to a per-frame saliency, and decode the resulting
matrix into word boundaries with a single dynamic-programming pass. The method needs no training, no model modification and no
alignment heads, works across all model families including the
speech LLMs, and aligns on the input grid rather than on the coarser encoder grid. We evaluate it on sixteen models from four families, on read (TIMIT) and spontaneous (Buckeye)
speech, each against the model's own native or
attention-based
alignment. We find that the gradient yields a usable
alignment for every model, that it is usually somewhat behind a strong native aligner but better where the native
alignment is weak, as for the streaming models, and that its main disadvantage is the cost of one backward pass per
token.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
CaLiSym: Learning Symplectic Dynamics of Real-World Systems through Structured Canonical Lifts
Aristotelis Papatheodorou, Pranav Vaidhyanathan, Natalia Ares et al. (+2 more)
Physics-informed learning promises data-efficient and stable dynamics prediction, yet its strongest
geometric guarantees have largely remained confined to closed conservative systems. This excludes robotic systems of interest, where actuation, dissipation, and constraints exchange energy and momentum with the environment. We introduce CaLiSym, a lightweight framework that extends
symplectic learning to such systems by changing where the
geometric prior is imposed. Rather than enforcing symplecticity on the measured state, CaLiSym embeds the state and its ports into a lifted phase space, where the dynamics evolve through a
symplectic map. The lift is explicit and algebraic, requiring neither recurrent latent states,
transformer decoders, implicit optimization, nor inference-time numerical integration. We instantiate the framework with SympNet predictors and introduce GRB-SympNet, a B-spline variant combining approximation with exact
symplectic structure. Experiments on a controlled dissipative double pendulum, a real-world quadrotor, and a contact-constrained real-world quadruped demonstrate the lowest out-of-distribution autoregressive rollout error across systems, improving by up to 69.5% while using fewer parameters and up to 85x fewer floating-point operations per step than sequence-model baselines. The lifted dynamics preserve the
symplectic form to numerical precision, extending
symplectic learning beyond conservative mechanics toward real-world robotics.
Read the full abstract ▾
All authors Aristotelis Papatheodorou, Pranav Vaidhyanathan, Natalia Ares, Ioannis Havoutis, Gerard J. Milburn
Machine Learning · Cross-listed from cs.CL
Ad Headline Generation using Self-Critical Masked Language Model
Yashal Shakti Kanungo, Sumit Negi and Aruna Rajan
For any E-commerce website it is a nontrivial problem to build enduring advertisements that attract shoppers. It is hard to pass the creative quality bar of the website, especially at a large scale. We thus propose a programmatic solution to generate product advertising headlines using retail content. We propose a state of the art application of Reinforcement Learning (RL) Policy gradient methods on
Transformer based Masked
Language Models. Our method creates the advertising headline by jointly conditioning on multiple products that a seller wishes to advertise. We demonstrate that our method outperforms existing
Transformer and LSTM + RL methods in overlap metrics and quality audits. We also show that our model-generated headlines outperform human submitted headlines in terms of both grammar and creative quality as determined by audits.
Read the full abstract ▾
Machine Learning
What Predicts Correctness in Text-to-SQL? A Selective-Prediction Study
Robert Richardson
Evaluating uncertainty in AI-generated SQL queries requires estimating whether a query is correct, where correct means it executes to the same result as a human-written reference. We study which signals predict correctness on hard multi-table text-to-SQL, using AUROC to measure how well each ranks correct queries above incorrect ones. On BIRD and Spider, black-box signals such as string, structural, and execution self-consistency, a schema-relevance score, and query executability all fall between about 0.61 and 0.68 AUROC, with string self-consistency strongest at 0.675; white-box log-probability is similar (0.67). The signals that move past this ceiling are verification-based: an LLM judge scores from 0.72 (GPT-4o-mini) to 0.78 (Claude). Judges from different providers make different errors, so a two-provider ensemble reaches 0.82 AUROC with a well-calibrated probability (expected calibration error 0.03) and supports useful abstention frontiers (for example, answering 27% of questions at 24% selective risk) where self-consistency offers no valid low-risk subset. The pattern holds across two
benchmarks, two generators, and two judge providers. We also ask whether a verifier can be trained. Fine-tuned verifiers, both encoder and generative, reach about 0.77 to 0.79 AUROC in-distribution but fall to about 0.66 on unseen schemas; scaling to 7B, adding schema diversity, distilling a strong judge's rationales, and cross-
benchmark training all fail to close that gap. Cross-schema transfer appears to track model scale and
reasoning rather than fine-tuning. In practice, correctness uncertainty for text-to-SQL lives in
reasoning-based signals: a fine-tuned verifier is a good in-domain tool, but a verifier that generalizes across schemas currently means a large frozen
reasoning model.
Read the full abstract ▾
Machine Learning
Enhancing deep learning models for time series classification via knowledge distillation
Javidan Abdullayev, Maxime Devanne, Jonathan Weber et al. (+1 more)
Deep learning has achieved remarkable success in various domains including time series analysis, computer
vision and natural
language processing. However, high computational and
memory demands of state-of-the-art architectures pose challenges for deployment in resource-limited environments. Knowledge
Distillation (KD) addresses this by transferring knowledge from a large teacher model to a smaller, more efficient student model while maintaining competitive performance. In this work, we investigate the effectiveness of KD for Time Series Classification (TSC) across three architectures: the classical Fully Convolutional Network (FCN), the convolutional Inception model and the
transformer-based ConvTran model. We evaluate our approach on UCR Archive, the largest
benchmark repository of time series
datasets, by modifying architectural components such as convolutional filters, Inception modules and
attention heads across the three architectures. Our results consistently show that KD most effectively benefits student models of intermediate complexity across all three architectures, with the distilled FCN student reducing parameters by a factor of 38, the distilled Inception student achieving nearly the same performance as the teacher with 42% fewer parameters and the distilled ConvTran student with 2
attention heads showing the most significant improvement through
distillation. To encourage further research and reproducibility, we provide our implementation at https://github.com/MSD-IRIMAS/KD-4-TSC.
Read the full abstract ▾
All authors Javidan Abdullayev, Maxime Devanne, Jonathan Weber, Germain Forestier
Machine Learning
On Explicit Super-Expressive Approximation for Neural Networks
Feng-Lei Fan, Ze-Yu Li, Chen-Yu Wang et al. (+1 more)
In this work, we investigate the fixed-architecture neural network approximation with explicit parameter bounds and elementary activations. While prior work demonstrated super-expressive approximation using fixed-size networks, they lack quantitative and non-asymptotic characterizations of parameter magnitude with respect to the approximation error. We resolve this issue by introducing the Chinese Remainder
Theorem as a constructive encoding mechanism. For Lipschitz continuous functions on $[0,1]^D$, we construct a width-$\max\{D,4\}$, depth-$5$ network with explicit parameter-error trade-offs. For Hölder-smooth functions in $C^{r,\gamma}_A\left([0,1]^D\right)$, our fixed network of width $\max\{2D,\ D+5N+1\}$ and depth $r + 9$ achieves the parameter magnitude $\mathcal{P}$ bounded by $\log_2 \mathcal{P}=\mathcal{O}\bigl(\varepsilon^{-2D/(r+\gamma)}\log(1/\varepsilon)\bigr)$. This is the dual result compared to those in the parameter-bounded and architecture-unbounded paradigm.
Read the full abstract ▾
All authors Feng-Lei Fan, Ze-Yu Li, Chen-Yu Wang, Jian-Jun Wang
Machine Learning
Efficient Bayesian Deep Ensembles via Analytic Predictive Inference
Sina Aghaee Dabaghan Fard, Marie Maros and Jaesung Lee
We introduce an efficient
Bayesian deep ensemble method for predictive regression designed to enhance interpretability while maintaining competitive predictive performance and computational efficiency. Our method combines the statistical rigor of
Bayesian inference with the scalability of deep ensembles, providing calibrated uncertainty estimates that enable its use not only for standalone prediction but also as a component within broader learning systems. To achieve these goals, our work relies on three key design components: (i) low-dimensional ensemble representation: predictions are expressed as a combination of a small number of trained neural predictors, enabling scalable inference whose cost depends on ensemble size rather than
dataset size; (ii) closed-form
Bayesian aggregation: ensemble predictions are combined using
Bayesian linear regression, yielding interpretable posterior weights and calibrated uncertainty without approximate inference; and (iii) Independent ensemble training: multiple neural networks are trained separately, producing diverse predictive representations that improve robustness and uncertainty calibration. Empirical results on standard regression
benchmarks demonstrate that the proposed approach achieves competitive predictive performance while maintaining reliable uncertainty estimates across settings.
Read the full abstract ▾
Machine Learning
Efficient Long-Horizon Learning for Learned Optimization
Xiaolong Huang, Benjamin Thérien, James Harrison et al. (+1 more)
Learned optimization aims to improve upon hand-designed optimizers (e.g., Adam and
Muon) by meta-learning small neural network optimizers over a distribution of tasks. While recent work has greatly advanced the architectural design and inductive biases of learned optimizers (LOs), their meta-training remains biased toward short-unroll learning on particular tasks, resulting in redundant computation and leaving LOs often unable to compete with hand-designed optimizers. We introduce Efficient Long-hOrizon (ELO) learning, an efficient meta-training
algorithm that (1) reallocates wasted meta-training compute to longer failure regimes, achieving efficient long-horizon learning, and (2) enforces decoupled progressive expert supervision, providing stable meta-learning signals that additionally improve the generalization of LOs. Our empirical study evaluates ELO for meta-training both element-wise and
matrix-based LOs. Across downstream
language modeling (GPT-2 and GPT-3 on FineWeb) and image classification (ViT-B/16, ResNet-50 on ImageNet-1K) tasks, ELO substantially improves the long-unroll performance and out-of-distribution generalization of the base LOs. In particular, ELO-Celo2 outperforms well-tuned AdamW and
Muon on GPT-2 (30M and 124M)
language pretraining. Notably, all ELO baselines require less than 7 H100 GPU-hours for meta-training.
Read the full abstract ▾
All authors Xiaolong Huang, Benjamin Thérien, James Harrison, Eugene Belilovsky
Machine Learning · Cross-listed from cs.IT
AirPASS: Over-the-Air Federated Learning via Pinching Antenna Systems
Seyed Mohammad Azimi-Abarghouyi and Christopher G. Brinton
This paper investigates over-the-air
federated learning (AirFL) in wireless systems where the access point is equipped with a multi-waveguide pinching
antenna system (PASS). We adopt the widely studied learning-oriented AirFL formulation, which seeks to maximize the number of selected devices while keeping the aggregation distortion below a prescribed threshold. The resulting joint optimization of device selection, receive beamforming, and pinching-
antenna placement is highly nonconvex due to the intricate coupling among these system variables. To address this challenge, we develop AirPASS, an alternating optimization framework with two main components: a homotopy-Riemannian margin-consolidation method for device selection and receive beamforming under a fixed PASS configuration, and a homotopy-assisted geometry optimization method for updating the pinching-
antenna positions under fixed selected devices and beamformer.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SE
Deployment Risk Assessment Using Diff-Aware Features: A Case Study at Prime Video
Mayur Kurup, Hyunjae Suh, Swathi Vaidyanathan et al. (+3 more)
At Amazon
Prime Video, we face the critical operational challenge of managing code deployments during live events and rapid feature releases without causing service outages. Current change control approaches use blanket deployment freezes that block all changes regardless of risk, creating significant developer toil. While prior research has explored risky change predictors, these rely on developer-specific metadata or extensive historical data, raising
privacy concerns and limiting applicability to new projects. We introduce a framework centered on diff-aware features, characteristics derived directly from code modifications. Our key contribution is the systematic identification of which quantitative metrics (code-level and change-level metrics) and qualitative indicators (coding style violations, change type classification) are necessary for risk prediction. We employ LLMs as multi-
language feature extractors, demonstrating their effectiveness for code analysis beyond generation tasks and eliminating the need for
language-specific tooling. We evaluated our framework on two
datasets:
Prime Video's production environment and the public ApacheJIT
dataset. Our best-performing model achieves an average recall of 0.83 and F1 score of 0.81 across both
datasets for detecting risky code changes. Notably, ablation analysis reveals that change-level volume metrics (e.g., lines added/deleted) are noisy predictors, while structural code complexity provides a substantially stronger risk signal. These results demonstrate that thoughtful feature curation enables effective change risk assessment across different programming
languages and organizational contexts while avoiding
privacy concerns.
Read the full abstract ▾
All authors Mayur Kurup, Hyunjae Suh, Swathi Vaidyanathan, Pranesh Vyas, Srinidhi Madabhushi, Yegor Silyutin
Machine Learning
Trees from Marginals: Autoregressive drafting with factorized priors
Yuma Oda, Ryan Mathieu, Roman Knyazhitskiy et al. (+1 more)
Speculative decoding greatly increases the interactivity of autoregressive
language models by trading off computation for extra
tokens generated in a single forward pass. Factorized draft models are especially efficient because they predict future-
token marginals in parallel, but their independence assumption causes acceptance rates to degrade sharply as the speculative budget grows. We analyze this limitation and introduce Weaver, a lightweight autoregressive adapter that constructs proposal trees from the top-K marginals of a factorized drafter. Weaver restores conditional dependencies between proposed
tokens while avoiding a full-vocabulary projection. To support fast verification for models with Gated Delta Net layers, we derive a rollback-free tree-verification
algorithm and implement optimized CUDA kernels in SGLang. By combining these model and systems contributions we achieve a 4.08-fold speedup over autoregressive decoding, and outperform a highly optimized DFlash baseline by 16.3%.
Read the full abstract ▾
All authors Yuma Oda, Ryan Mathieu, Roman Knyazhitskiy, Artur Chakhvadze
Machine Learning · Cross-listed from math.OC
Optimization Geometrodynamics: Variational Reduction and Interaction Curvature
Zavier Li
Adaptive optimizers carry hidden states that change how visible gradients become parameter motion. We develop optimization geometrodynamics as a variational theory of this hidden geometry. Infimal pushforward eliminates all hidden states realizing the same action and composes across optimizer hierarchies. Under smooth nondegeneracy, it yields hidden susceptibility and the Schur-complement
curvature seen after relaxation. For affine pre-reduction perturbations, the induced interaction
curvature is the negative-semidefinite operator $-G^*H^{-1}G$, whose mixed entries integrate to finite mechanism contrasts. Our main realization is the determinant-one affine-invariant SPD action map $P\mapsto PA$. We prove a global analytic bundle with closed totally geodesic fibers and a unique analytic nearest-controller section. A strongly convex fiber
theorem and an explicit logarithmic action residual give a globally linearly convergent solver from every feasible initializer, together with nonasymptotic value, controller-distance, and residual bounds and observable posterior stopping certificates. A conditional inexact result propagates supplied rigorous residual-error and radius majorants. The dense spectral kernel is confined to an active subspace of dimension $r\le 2m$, yielding an explicit spectral-arithmetic operation bound and a strict dimensional reduction when $r<d$. For nested shape-normalized quadratic actions, canonical multi-secant projections satisfy an exact CAT(0) Pythagorean decrease and recover the determinant-one inverse Hessian shape at the sharp rank threshold $d-1$, provided the scalar gauge $c_H=(\det H)^{1/d}$ is known. These results turn the action bundle into an exact iterative computation with posterior certificates and a finite-identification theory.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CE
Macroeconomic Message Passing for Anticipating Foreign Exchange Regime Changes: A Deep Logical Learning Approach using Graph Tsetlin Machines
Christian Blakely and Melanie Gilmore
This paper introduces a graph-theoretic approach for predicting
market regimes in foreign exchange (FX) currency prices. Specifically, the proposed model incorporates exogenous macroeconomic variables to update localized node features via message-passing operations. Utilizing the Graph Tsetlin Machine (GraphTM) framework, we empirically demonstrate the efficacy of this approach in anticipating
market regimes for the US Dollar and Japanese Yen currency pair (USD/JPY). By representing multivariate macroeconomic drivers and technical indicators as hypervectorized directed multigraphs, the GraphTM leverages structured message passing to construct deep, interpretable logical clauses capable of recognizing complex sub-graph patterns.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Review
Inkyu Sa, Chanoh Park, Hea-Min Lee et al. (+2 more)
Vision Language Action (VLA) models unify visual perception, natural-
language understanding, and action generation within a single foundation model, allowing a
robot to follow instructions such as fold the towel or fly to the red building directly from camera images. Because VLAs inherit world knowledge from internet-scale pre-training, they have become the dominant framework for learning-based manipulation, with bimanual coordination serving as the most demanding testbed: two arms with 7 degrees of freedom each must move in concert to fold, assemble, and reorient objects. Unmanned aerial robotics faces a structurally similar challenge: a drone must coordinate thrust, attitude, and increasingly gripper commands from visual observations under strict latency and payload constraints. This review covers 183 contributions spanning 2017-2026 and organized along seven dimensions: VLA architectures, training recipes, action representations, bimanual coordination (2022-2026), unmanned aerial vehicle (
UAV)
navigation and control (2017-2026),
language grounding, and cross-cutting concerns including
memory and world models. We show that the coordination strategies, training recipes, and action representations developed for bimanual VLAs transfer to unmanned aerial systems and identify fourteen research directions across both domains.
Read the full abstract ▾
All authors Inkyu Sa, Chanoh Park, Hea-Min Lee, Donghee Noh, Ho Seok Ahn
Machine Learning · Cross-listed from cs.CV
SPEAR: A Simulator for Photorealistic Embodied AI Research
Mike Roberts, Renhan Wang, Rushikesh Zawar et al. (+10 more)
Interactive simulators have become powerful tools for training
embodied agents and generating synthetic visual data, but existing photorealistic simulators suffer from limited generality, programmability, and rendering speed. We address these limitations by introducing SPEAR: A Simulator for Photorealistic
Embodied AI Research. At its core, SPEAR is a Python library that can connect to, and programmatically control, any Unreal Engine (UE) application via a modular plugin architecture. SPEAR exposes over 14K unique UE functions to Python, representing an order-of-magnitude increase in programmable functionality over existing UE-based simulators. Additionally, a single SPEAR instance can render 1920x1080 photorealistic beauty images directly into a user's NumPy array at 73 frames per second - an order of magnitude faster than existing UE plugins - while also providing ground truth image modalities that are not available in any existing UE-based simulator (e.g., a non-diffuse intrinsic image decomposition, material IDs, and physically based shading parameters). Finally, SPEAR introduces an expressive high-level programming model that enables users to specify complex graphs of UE work with arbitrary data dependencies among work items, and to execute these graphs deterministically within a single UE frame. We demonstrate the utility of SPEAR through a diverse collection of example applications: controlling multiple
embodied agents with distinct action spaces (e.g., humans, cars, and
robots) across several in-the-wild UE projects; rendering photorealistic city-scale environments; manipulating UE's procedural content generation systems; rendering synchronized multi-view images of detailed human faces; coordinating an interactive co-simulation with the MuJoCo physics simulator; and editing scenes with natural
language via an AI coding assistant.
Read the full abstract ▾
All authors Mike Roberts, Renhan Wang, Rushikesh Zawar, Rachith Dey-Prakash, Quentin Leboutet, Stephan R. Richter, Matthias Müller, German Ros, Rui Tang, Stefan Leutenegger, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Vladlen Koltun
Machine Learning · Cross-listed from stat.ML
Heat-Kernel Entropy Profiles and Geometric Effective Sample Size for Weighted Measures on Manifolds
Kisung You and Boram Cho
Weighted empirical measures on compact
manifolds appear in importance sampling, particle approximations, posterior summaries, quadrature, and representation learning. Ordinary effective sample size and related weight summaries ignore the geometry of the support. We introduce heat-kernel
entropy profiles to measure nonuniformity after intrinsic
diffusion at a range of scales. For order-two Rényi
entropy, pairwise heat-kernel overlaps give an exact profile and a
geometric effective sample size. This effective sample size discounts nearby or duplicate particles. It approaches ordinary effective sample size as overlaps between distinct particles vanish. On compact boundaryless
manifolds, we establish profile monotonicity, gESS scale limits, deterministic-weight consistency, and a bounded-ratio result for self-normalized importance sampling. On spheres, the unlogged profile decomposes into spherical-harmonic energies. The first terms are squared mean-resultant and traceless-second-moment energies, which give vMF- and Bingham-type scalar summaries. Experiments identify antipodal, girdle,
multimodal, and duplicate-particle structures that weight-only and first-moment summaries miss.
Read the full abstract ▾
Machine Learning
Robust Human-AI Complementarity under Uncertainty
Yewon Byun and Bryan Wilder
Machine learning models are often intended to augment rather than replace human decision makers, by providing information that is complementary to human judgement. Yet, in practice, human decision makers routinely fail to realize such complementary gains, even when models provide useful signal. In this work, we study how asymmetric information about the quality of information available to a human decision maker vs. an AI impacts the ability of a decision maker to extract complementary value from AI predictions. We show that a key factor is the error correlation structure between human and AI predictions. In particular, when the AI's prediction errors are negatively correlated with those of the human, the decision maker can construct robust strategies which guarantee improvements in expected utility. We empirically investigate whether these conditions for complementarity arise in practice, using real-world
forecasting benchmarks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Pelican-VLA 0.5: Attending Before Acting Benefits Generalization
Zeyuan Ding, Wenhai Liu, Yang Xu et al. (+6 more)
In this report, we present Pelican-VLA 0.5, a unified VLA model that integrates
vision-
language understanding, future-frame generation, and action prediction within a single architecture. Pelican-VLA 0.5 achieves
attention-level generalization: without object annotations,
segmentation masks,
attention supervision, or task-specific fine-tuning, its action pathway already focuses on the manipulation-relevant object and contact region. This behavior persists across unseen scenes and unseen
robot embodiments, and is substantially stronger than in other open-source VLA baselines. We verify that this ability originates from the learnable Bottleneck
Token inserted between perception and action: by routing task-relevant visual information through a compact bottleneck, the
tokens interface induces manipulation-centric
attention during pre-training and remains effective across different policy structures, including a MoT-style architecture.
Read the full abstract ▾
All authors Zeyuan Ding, Wenhai Liu, Yang Xu, Jiayu Hu, Yinda Chen, Yi Zhang, Yong Dai, Jian Tang, Xiaozhu Ju