Machine Learning
Joint Spatiotemporal Spectral Neural Operators for Learning PDEs on Irregular Domains
Abdolmehdi Behroozi and Chaopeng Shen
Learning solution operators for partial differential equations (PDEs) on irregular and geometry-dependent domains remains a central challenge in scientific machine learning. While spectral methods provide strong inductive biases for modeling global interactions, they are typically limited to regular domains, and existing neural approaches often require domain warping, interpolation, or costly
geometric embeddings. We introduce the Graph Spectral Neural Operator (GSNO), a neural operator that combines spatial graph spectral decompositions with temporal
Fourier transforms through a unified space–time spectral kernel. This formulation enables globally coherent operator learning on non-Cartesian discretizations without domain warping or autoregressive rollouts. By replacing learned
geometric embeddings with a graph
Laplacian spectral basis, GSNO provides geometry-aware spectral learning with low parameter complexity. Across steady and unsteady PDE
benchmarks on irregular and geometry-dependent domains, GSNO achieves strong accuracy with reduced runtime and parameter counts, while demonstrating robust zero-shot generalization across mesh resolutions and geometry families.
Read the full abstract ▾
Machine Learning
Sensitivity-Constrained Neural Operators for Data-Efficient Forward and Inverse Modeling of Partial Differential Equation Systems
Abdolmehdi Behroozi, Chaopeng Shen, Daniel Kifer et al. (+1 more)
Neural operators provide fast surrogates for partial differential equation (PDE) solvers, but their reliability can degrade for high-dimensional spatial inputs and inverse or repeated inference. State-only training constrains solution values but not the learned input–output response. We study sensitivity-constrained neural operators (SC-NOs), which augment standard training with sampled solver-derived Jacobian supervision. Selected sensitivities from differentiable solvers or discrete adjoints are matched during training, allowing response information to be amortized across minibatches without imposing the full Jacobian at every update. We evaluate SC-NO on advection–
diffusion and RANS–Spalart–Allmaras
benchmarks, input-dimensionality scaling tests, long-horizon autoregressive rollout, and a shallow-water Tohoku tsunami source-inversion case. Sensitivity supervision improves forward prediction and yields larger gains in gradient-based inverse reconstruction of distributed fields. Scaling experiments show an improved accuracy–cost tradeoff for high-dimensional gridded inputs, while ablations indicate that state values and Jacobian information provide complementary supervision. In the tsunami case, SC-FNO reconstructs gridded seafloor deformation from sparse early gauge observations and forecasts subsequent wave propagation in a near-real-time proof-of-concept workflow. These results support sampled sensitivity supervision as a practical way to improve neural PDE surrogates when forward accuracy, inverse stability, robustness, and computational cost must be considered together.
Read the full abstract ▾
All authors Abdolmehdi Behroozi, Chaopeng Shen, Daniel Kifer, Kathryn Lawson
Machine Learning
Structural Hierarchy and Geometry in Molecular Representation Learning
David Sulu, Lorenzo Di Fruscia and Jana M. Weber
Molecular self-supervised learning uses chemical structures to guide which molecular
embeddings should be similar. We study whether explicitly encoding a molecule's Bemis-Murcko scaffold and using it to supervise the molecular
embedding changes what the model learns. We further test whether this effect depends on the
embedding geometry by comparing Euclidean and Lorentz contrastive objectives. Across two augmentation strengths, scaffold-supervised models consistently organize molecules according to both identical and structurally related scaffolds. The resulting
embeddings also improve molecular property prediction on several tasks, while the exact gains depend on the predicted property. The effect of scaffold supervision on molecular organization is stronger under Lorentz objectives, but neither geometry provides a consistent overall advantage. These results show that explicitly teaching the relation between a molecule and its structural core can reliably shape the organization of molecular
embedding space, while the extent of usefulness of this organization remains task dependent.
Read the full abstract ▾
Machine Learning
Towards an Expressivity-Normalized Energy-Demand Comparison of ANNs and SNNs
Miriam Kranzlmüller, Pascal Esser and Gitta Kutyniok
Spiking neural networks (SNNs) are often regarded as energy-efficient alternatives to artificial neural networks (ANNs), yet their advantage depends critically on both network architecture and data properties. We develop an analytical framework to compare fully-connected ReLU ANNs and integrate-and-fire SNNs for time-series data with respect to their theoretical energy efficiency at matched expressive capacity. By relating an inference-energy model to theoretical bounds on representational expressivity, we derive an expressivity-normalized efficiency ratio and explicit thresholds in network width, spike sparsity, and ANN depth scaling. Our analysis characterizes the regimes in which event-driven computation offsets the temporal overhead of SNNs, providing capacity-aware principles for designing energy-efficient temporal networks. It shows that ANNs exceed SNNs in expressivity-normalized efficiency only in specific regimes.
Read the full abstract ▾
Machine Learning
Partially Linear Autoencoders for Manifold Learning and Dimensionality Reduction
Louen Pottier, Louis Lesueur and Anders Thorin
Autoencoders are widely used for nonlinear dimensionality reduction and
manifold learning. While most common implementations rely on both nonlinear encoders and decoders, we investigate the specific role of the encoder and the extent to which it can be constrained to be linear without reducing accuracy. We conduct a comparative study on four autoencoder architectures: standard fully nonlinear autoencoders (AE), linear-encoder autoencoders (Lenc-AE), linear-decoder autoencoders (Ldec-AE), and fully linear autoencoders (LAE), evaluated on synthetic
manifolds, computational mechanics data sets, and real-world image data sets including MNIST. We demonstrate that imposing a linear encoder preserves most of the representational capacity of the autoencoder, provided the decoder remains nonlinear. In particular, Lenc-AE consistently outperforms both Ldec-AE and LAE, and achieves reconstruction quality comparable to fully nonlinear AE, while offering advantages in terms of parsimony and interpretability of the latent representation. These results suggest that the nonlinear decoder is the critical component for
manifold learning, rather than the encoder. A
geometric interpretation of this finding is developed, which identifies the precise conditions under which a linear encoder is sufficient, and the specific
manifold configurations that expose its limitations.
Read the full abstract ▾
Machine Learning
Uncertainty-Driven Replay Memory for Reinforcement Learning
Sheeraja Rajakrishnan, Alexander G. Ororbia, Travis Desell et al. (+1 more)
Uncertainty estimation provides promising capabilities for reinforcement learning (RL)
agents. Notably, estimating uncertainty can reduce the training time and enable
agents to obtain greater rewards over time by exploiting information related to whether an action would facilitate exploration of portions of an environment that are well-known versus those that are relatively unknown. In this work, we propose a novel formulation of the experience replay buffer commonly used in RL that we call uncertainty-driven replay
memory (UDRM), which entails an update scheme for internally stored
memories based on uncertainty estimates obtained by an RL model during training. In contrast to existing forms of RL, which typically use temporal difference error or the distribution of transitions to update the replay
memory buffer and train RL controllers, our scheme biases the
memory buffer to store more uncertain transitions that will improve an RL
agent's generalization throughout training. Experimental results demonstrate that our proposed uncertainty-aware replay buffer enables an RL
agent to obtain higher rewards during training compared to other existing uncertainty-aware RL frameworks.
Read the full abstract ▾
All authors Sheeraja Rajakrishnan, Alexander G. Ororbia, Travis Desell, Daniel E. Krutz
Machine Learning
Designing for the Next Click: Bandits for Real-Time Page Layout
Bhavtosh Rath, Harshith Narasimhamurthy, Bob Eisinger et al. (+3 more)
E-commerce platforms increasingly personalize user experiences through machine learning, yet page layout decisions remain dominated by static rules and manual curation. We present a scalable bandit-based system that optimizes product page layouts in real time while preserving human control over design intent. A contextual bandit model dynamically selects the most effective layout for each session using user, item, and category-level features. The system leverages a LinUCB-based policy to balance exploration and exploitation as it learns from live user interactions. The architecture is designed for seamless integration into large-scale web serving stacks, supporting low-latency inference and continuous model updates. The system was first tested on entry product pages. In online A/B deployments on a major retail platform, our approach achieved positive lifts in session-level performance metrics over a strong heuristic baseline. Our results demonstrate that contextual bandits can effectively optimize visual and structural aspects of product discovery for user engagement, providing a scalable path toward learning-to-design the web.
Read the full abstract ▾
All authors Bhavtosh Rath, Harshith Narasimhamurthy, Bob Eisinger, Cole Stiegler, Adnan Awow, Amit Pande
Machine Learning · Cross-listed from cs.CL
Influence-Directed Distillation: Solving the Diversity Bottleneck in Sampled-Token On-Policy Distillation
Run Yang, Runpeng Dai, Jie Sun et al. (+5 more)
Sampled-
token on-policy
distillation (OPD) efficiently transfers capabilities from teacher to student using student-generated
tokens, requiring teacher probabilities only for sampled
tokens. Yet it frequently suffers from diversity
distillation failure: the student's pass@1 improves while its pass@$k$ plateaus, failing to inherit the teacher's diversity. To explain this, we introduce First-Order Local
Entropy Influence, a signed first-order proxy that decouples each update's
entropy effect into the teacher–student log-probability gap and the student's local probability structure, and empirically links
entropy contraction to negative-influence positions. Motivated by this, we propose Influence-Directed Adaptive On-Policy
Distillation (IDA-OPD): rather than relying on costly full-vocabulary Forward-KL objectives, it preserves
entropy-expanding updates while replacing
entropy-contracting ones with divergence-adaptive advantage shrinkage, using only the teacher's sampled-
token log-probability. Experiments on
reasoning-oriented
distillation show IDA-OPD consistently improves pass@$k$, inheriting the teacher's diversity through
distillation, matches the strongest teacher-informed methods at strictly lower cost, and broadly maintains vanilla OPD's pass@1, all without full-vocabulary teacher information.
Read the full abstract ▾
All authors Run Yang, Runpeng Dai, Jie Sun, Jielei Zhang, Fan Zhou, Hongtu Zhu, Peiyi Li, Longwen Gao
Machine Learning · Cross-listed from econ.GN
The Price of Intelligence: A Quality-Adjusted Price Index for AI Services
Louis Yiven Zhu
Posted prices for AI inference have fallen steadily since 2024, yet the measured speed of that fall depends almost entirely on the method of measurement. This paper constructs quality-adjusted price indices for the AI inference
market from public data. The panel assembles 21,024 posted-price observations across 3,208 models and 86 providers and joins them to 4,605
benchmark scores through a latent quality index estimated from
benchmark response patterns, so the quality ladder of the hedonic tradition is built here from evaluations in place of product characteristics. Measured by the matched-model methods that statistical agencies apply to software, inference prices fell at 0.10 log points a year. The quality-adjusted index fell at 0.73, so 87% of the decline is invisible to current methods, with direct consequences for measured competition, concentration and productivity in this
market. Counted per completed task, moreover, the buyer's price stopped falling.
Reasoning models raised
token consumption faster than
token prices fell, and the seller's and buyer's prices accordingly diverged. A pre-registered validity audit disciplines the quality measure and yields the sharpest result. Excluding contamination-flagged
benchmarks leaves model rankings intact at 0.998 yet moves the index by 0.49 log points a year, so the leaderboard-stability arguments standard in AI evaluation offer no defence of economic statistics built on
benchmarks. Prices, quality and the audit are fully reproducible from public sources at zero cost.
Read the full abstract ▾
Machine Learning
Structure Aware Neural Architecture Search for Mixture of Experts
Petr Babkin and Oleg Bakhteev
Neural Architecture Search (NAS) has so far rarely been applied to Mixture-of-Experts (MoE) models, and existing MoE designs leave the
alignment between experts and the structure of the data to emerge on its own. We propose an architecture search framework that makes this
alignment an explicit search variable: the assignment of data clusters to experts is optimised jointly with the per-expert architectures. We cast the joint problem as a cluster-aware likelihood maximisation, show that it coincides with the incomplete-data maximum likelihood of a latent-variable mixture, and solve it by a generalised Expectation-Maximisation procedure whose otherwise intractable expert-quality term is supplied by an adaptively refined surrogate. We prove that the iterates converge whenever the surrogate errors are summable, and that at every limit point no candidate the search produces improves the true objective. On a heterogeneous image-classification mixture the method recovers the underlying domain partition on 95% of clusters without ever observing domain labels, and on that
benchmark and a four-domain time-series
forecasting one alike it outperforms the MoE and NAS baselines that likewise use no label information.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantification
Kehan Long, Yiqi Zhao, Pol Mestres et al. (+3 more)
Uncertainty quantification from finite data is central to machine learning, optimization, and automation systems, where decisions must remain reliable under limited samples and test-time distribution shift. Conformal prediction (CP) and distributionally robust optimization (DRO) offer two complementary approaches: CP constructs data-dependent prediction sets with distribution-free finite-sample validity under exchangeability, while DRO optimizes worst-case performance over an ambiguity set around an empirical distribution. We develop a unified probabilistic perspective on CP and DRO by viewing both as ways to turn finite calibration data into a data-dependent
quantile estimator that a test score falls below with high probability. From this perspective, CP and DRO correct the empirical
quantile along two coordinates of the same family of
estimators: CP inflates the
quantile level, whereas DRO shifts the
quantile value through an ambiguity radius. Both methods provide the same calibration-conditional guarantee for the true distribution, requiring the target coverage to hold with high probability over the calibration sample. Their constructions differ, however: CP uses a closed-form, distribution-free level correction, while DRO uses a value-space correction whose certified radius depends on properties of the unknown distribution and additionally guarantees coverage uniformly over the ambiguity set. This distinction emerges in the tails of the score distribution. Because CP relies on sparse upper-tail order statistics of the calibration samples, its level inflation barely moves the
estimator when those samples are dense near the target
quantile but overshoots when they are sparse, whereas a well-chosen DRO radius corrects in value space and may avoid this overshoot.
Read the full abstract ▾
All authors Kehan Long, Yiqi Zhao, Pol Mestres, Lars Lindemann, Nikolay Atanasov, Jorge Cortés
Machine Learning
PruneShift: A Framework for Evaluating Decision Reliability in Structured Pruning
Hao Ye and Gaopeng Zhang
Structured
pruning uses surrogate objectives because direct task evaluation over every feasible mask is too expensive. Most evaluations report average surrogate error or rank correlation on broadly sampled masks. These summaries do not directly test the mask chosen by the surrogate. We introduce PruneShift, an evaluation framework that separates broad predictive fidelity, fidelity near selector outputs, and the quality of the selected
pruning decision. We first prove that Spearman and Kendall agreement can approach one while normalized selection regret remains maximal. We then derive sufficient conditions based on uniform error, selector suboptimality, decision margin, density ratio, and comparison mass. The analysis also yields a finite pool certificate with an explicit excess cost bound. Four studies test different links in this argument. External TextbookQA confirmation is heterogeneous: 7 of 20 simultaneous intervals favor the surrogate-selected mask, 6 favor its fixed comparator, and 7 cross zero. On a fixed Natural Questions pool, strict improvement holds in one of four settings. A controlled QQP experiment supports the proposed coverage mechanism in all 16 prespecified endpoints, although the sufficient bounds are conservative. Finally, a restricted OSSCAR reconstruction study on OPT-125M shows better local than broad fidelity in 68 of 75 primary endpoints. Independent fixed-mask confirmation is inconclusive in 24 of 25 endpoints and favors the comparator in one. These results show why predictive fit, decision reliability, and
pruning method quality require separate evidence.
Read the full abstract ▾
Machine Learning
ECA-BLS: An Efficient Complex-Augmented Broad Learning System
A. Rahaman, A. Quadir, M. Sajid et al. (+2 more)
Broad Learning System (BLS) is an efficient alternative to deep architectures due to its fast training, analytical learning, and strong generalization under limited data. However, existing BLS variants are confined to real-valued representations, restricting their ability to capture nonlinear interactions and second-order statistical dependencies inherent in real-world data. Notably, no prior BLS model fully exploits the complete second-order statistics that naturally emerge when data are embedded in the complex domain. To address this limitation, this paper introduces the first complex augmented Broad Learning System (CA-BLS), which transforms real-valued inputs into phase-encoded complex representations and adopts widely linear modeling to jointly leverage covariance and pseudo-covariance information via complex conjugate augmentation. This enables effective modeling of latent nonlinearities, coherence structures, and second-order dependencies inaccessible to conventional BLS formulations. To mitigate the additional computational cost of complex augmentation, an Efficient Complex Augmented BLS (ECA-BLS) is further developed, reformulating CA-BLS entirely in the real domain while preserving its exact decision function, achieving up to 75% fewer multiplications and over 60% fewer additions. A rigorous theoretical analysis proves the mathematical equivalence between CA-BLS and ECA-BLS, ensuring zero theoretical loss. Extensive experiments on 26
benchmark datasets from the UCI and KEEL repositories demonstrate that ECA-BLS consistently outperforms classical BLS and recent state-of-the-art randomized neural networks in accuracy, average rank, and statistical significance, establishing augmented second-order modeling as a critical and previously missing dimension of BLS research.
Read the full abstract ▾
All authors A. Rahaman, A. Quadir, M. Sajid, M. Akhtar, M. Tanveer
Machine Learning
GraM-Diff: A Unified Graph-Mamba Diffusion Framework for EEG-Based Alzheimer's Disease Data Generation and Diagnosis
M. Tanveer, Ayush Singh Rana, Sanskriti Jain et al. (+5 more)
Electroencephalography (
EEG) is a promising, non-invasive, and cost-effective modality for Alzheimer's disease (AD) detection, but deep learning methods are limited by small and imbalanced clinical
datasets. Generative augmentation offers a solution, yet existing approaches rely on inefficient class-specific models or fail to capture complex spatial and temporal
brain dynamics. To address this, we propose GraM-Diff, a unified classifier-guided Graph-Mamba
diffusion framework for
EEG synthesis. It embeds Graph Convolutional Networks within a
diffusion U-Net to model inter-electrode connectivity and Bidirectional Mamba state-space blocks for linear-complexity long-range temporal modeling. Latent-space classifier guidance lets a single model generate both healthy and pathological
EEG within a shared representation, avoiding fragmented per-cohort pipelines. Across four
EEG-based AD
benchmarks, synthetic augmentation improves classification, yields superior Context-FID and correlation scores over strong generative baselines, and enhances robustness in data-scarce settings.
Read the full abstract ▾
All authors M. Tanveer, Ayush Singh Rana, Sanskriti Jain, Arnav Kumar, Aryaman Tiwari, A. Rahaman, A. Quadir, M. Sajid
Machine Learning
Higher-Dimensional Rotary Position Embedding
Yixing Li, Ruobing Xie, Yudong Zhang et al. (+3 more)
Transformers rely on position
embedding mechanisms in long context modeling in most cases. Rotary Position
Embedding (RoPE) embeds positional information with independent 2D rotations, forming relative position terms in self-
attention. However, its pairwise, block-based, and decoupled structure limits deep mixing and robustness across channels. We propose HD-RoPE, which extends RoPE from independent 2D rotations to higher-dimensional rotations and introduces a Paley-I orthogonal basis to obtain balanced, isotropic, and dense phase mixing within each rotation subspace. This significantly enhances channel coupling and rotational degrees of freedom while maintaining orthogonal stability and the relative position closure property. Furthermore, HD-RoPE is easily optimized for engineering efficiency without introducing additional trainable parameters. We have conducted extensive evaluation results demonstrating that HD-RoPE achieves significant performance improvements over standard RoPE across various popular
benchmarks and in both long and short contexts.
Read the full abstract ▾
All authors Yixing Li, Ruobing Xie, Yudong Zhang, Yushi Bai, Samm Sun, Yu Cheng
Machine Learning · Cross-listed from stat.ML
Neural ODE enhanced linear mixed effect models for estimating complex association patterns of time-varying covariates with the marker trajectory
Zhe Aurore Li, Quentin Clairon, Cécilia Samieri et al. (+3 more)
Longitudinal cohort studies produce repeated data that enable the assessment of time-varying association patterns between exposures and health outcomes. Classical linear mixed-effects models (LMMs) can accommodate a large variety of association patterns while accounting for the irregularly spaced, partially observed measurement. But they require the analyst to pre-specify the functional form linking the exposure history to the outcome. We propose the Neural ODE-LMM, which embeds a Neural Ordinary Differential Equation (Neural ODE) within the linear mixed-effects framework: a learned vector field encodes
covariate trajectories into a continuous-time latent state that drives both the fixed- and random-effect design, while preserving the standard LMM observation model. This retains classical likelihood-based inference while learning complex, potentially cumulative,
covariate effects flexibly. All parameters are estimated by maximising a penalised marginal likelihood. To quantify
covariate effects, we introduce contrasts of counterfactual predictions that compare the expected outcome under alternative
covariate trajectories with variance estimated via the delta method. In simulations, the model recovers both instantaneous and cumulative-burden effects without prior specification of the functional form. Applied to the Trois-Cités (3C) cohort, a population-based study of 7{,}324 participants, the method reveals trajectory-dependent associations of BMI and fasting glucose with cognitive decline.
Read the full abstract ▾
All authors Zhe Aurore Li, Quentin Clairon, Cécilia Samieri, Rodolphe Thiébaut, Mélanie Prague, Cécile Proust-Lima
Machine Learning · Cross-listed from math.OC
Transformer-Based Flow Shop Scheduling Using MILP-Generated Training Data
Roderich Wallrath
Advances in machine learning (ML) have created new opportunities to complement traditional operations research (OR) methods. In particular,
transformer models can capture complex interactions in
token sequences by mapping
tokens into a high-dimensional
embedding space and propagating contextual information via
attention. This makes them a candidate to model non-permutation flow shop scheduling with secondary resources as a next-
token prediction task, where
tokens represent job-machine-secondary resource tuples. For training, mixed-integer linear programming (MILP)-generated schedules are tokenized and used as next-
token prediction data. During inference, partial
token sequences (prefixes) are randomly generated and completed by the trained
transformer through constrained decoding. A computational study is conducted on a flow shop with 8 jobs, 4 machines, and 3 secondary resources, where jobs are selected from a fixed pool of 20 jobs that is sampled during training and provides the candidates during prefix completion. The
transformer achieves better solution quality (smaller makespans) compared to a genetic
algorithm (GA), the NEH heuristic, and random search. It is outperformed only by the MILP model and the iterated greedy (IG) heuristic. The study concludes that
transformer models can, to some extent, learn patterns from MILP-optimized non-permutation flow shop schedules and that
transformer-based scheduling represents an interesting direction for future research, particularly in settings with a fixed, recurring job set.
Read the full abstract ▾
Machine Learning
Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents
Zongyue Li, Chengyue Yu, Lei Zang et al. (+3 more)
Early failure prediction is important for long-horizon
agents, as it enables timely intervention and can reduce inference and tool-use costs. Uncertainty quantification, such as verbal confidence and perplexity, offers a promising approach to detecting
agent failures; however, it has not been explored whether these signals retain their discriminative power during the intermediate stages of long-horizon execution. We evaluate mainstream uncertainty signals on deep-research tasks and find that verbal confidence reliably distinguishes failures at trajectory completion, achieving a mean AUROC of 0.85, whereas all evaluated signals offer limited predictive value earlier in execution, with none exceeding a mean AUROC of 0.60 at 50% trajectory progress. We identify an underlying mechanism explaining this gap: path switching, where
agents frequently abandon their current search direction in-trajectory, breaking the link between early signal and final outcome. These findings challenge the assumption that intermediate uncertainty can reliably guide early intervention. They also motivate a practical recommendation for
agent harnesses in deep-research settings: use final-step confidence to decide whether to restart, an approach that our experiments find more effective than in-trajectory intervention.
Read the full abstract ▾
All authors Zongyue Li, Chengyue Yu, Lei Zang, Chenyi Zhuang, Linjian Mo, Leilei Gan
Machine Learning
Creation begins with understanding: LLMs as strategy designers for privacy-preserving tabular data synthesis
Jinmeng Li, Quan Zhang, Hangting Ye et al. (+4 more)
Sharing tabular data in high-stakes domains is constrained by
privacy regulations. Synthetic data offer a promising alternative, but deep generative models are costly to train and difficult to audit, while LLM-based methods often serialize records as text, obscuring tabular structure and exposing sensitive data. We introduce Tabular Synthesis Strategy Designer (TabSSD), which uses an LLM to design synthesis procedures rather than directly generate records. TabSSD provides the LLM with tree-derived summaries of variable dependence rather than raw records, which produces Python programs for local execution and evaluation. Across twelve
datasets, TabSSD strikes a favourable balance among statistical fidelity, predictive utility, and empirical
privacy risk, achieving the best average rank across six metrics among ten methods. Moreover, it substantially reduces local computation and
token consumption relative to the compared methods. By enabling human-guided refinement and eliminating user-side model tuning, TabSSD lowers the expertise and infrastructure barriers to transparent tabular data synthesis.
Read the full abstract ▾
All authors Jinmeng Li, Quan Zhang, Hangting Ye, He Zhao, Firas Laakom, Dandan Guo, Jürgen Schmidhuber
Machine Learning
A Target-Centric Survey of Quantization-Aware Training
Jiamin Song, Mengjie Zhao, Zijing Wang et al. (+6 more)
The rapid development of LLMs incurs prohibitive
memory footprints and intensive computational demands. Quantization-Aware Training (QAT) techniques have emerged as a promising solution to address these challenges by explicitly simulating quantization effects during model training, yielding low-bit models that achieve accuracy comparable to their full-precision counterparts. In this work, we provide a target-centric survey of QAT, aimed at clarifying both its theoretical foundations and its evolving implementation landscape. We systematically review existing QAT methods through a target-centric taxonomy and synthesize cross-target differences in error characteristics, numerical formats, and strategy transferability. We further summarize QAT evaluation paradigms and discuss challenges in optimization and deployment, outlining potential directions for future research.
Read the full abstract ▾
All authors Jiamin Song, Mengjie Zhao, Zijing Wang, Yongkang Liu, Qian Li, Shi Feng, Feiliang Ren, Daling Wang, Hinrich Schütze
Machine Learning · Cross-listed from physics.soc-ph
ButterMamba: Butterworth-Enhanced Spatial-Temporal Mamba for Efficient Traffic Flow Prediction
Limiao Zhang, Yuhui Lu, Jie Gao et al. (+3 more)
Accurate traffic flow prediction is fundamental to intelligent transportation systems, playing a pivotal role in urban mobility optimization and smart city development. While Graph Neural Networks (GNNs) integrated with time series
forecasting have emerged as promising solutions, two critical limitations persist: (1) the quadratic complexity of
attention-based architectures hinders real-time deployment in large-scale networks, and (2) high-frequency noise in sensor data significantly degrades prediction reliability. These challenges are particularly acute in metropolitan scenarios where both computational efficiency and noise robustness are paramount. To address these limitations, we introduce ButterMamba, a novel and efficient framework based on State Space Models (SSMs). ButterMamba consists of two key components: (1) a Butterworth Spectral Filtering module that preprocesses the data by removing high-frequency noise, allowing the model to focus on significant underlying trends, and (2) a Spatial-Temporal State Mixer that uses a parallel Mamba architecture to efficiently capture both long-range temporal dependencies and complex spatial correlations across the road network. By decoupling noise filtering from spatial-temporal modeling, ButterMamba achieves superior predictive accuracy with linear computational complexity. Extensive experiments on three public
datasets demonstrate that ButterMamba not only outperforms existing state-of-the-art models in terms of prediction accuracy but also considerably reduces training time and
memory usage.
Read the full abstract ▾
All authors Limiao Zhang, Yuhui Lu, Jie Gao, Hao Jiang, Haiping Ma, Xingyi Zhang
Machine Learning
Reward-guided Fine-Tuning of One-Step Generative Models via Wasserstein Gradient Flow
Hoseong Hwang, Woorim Han, Joungin Chun et al. (+2 more)
To mitigate the time complexity of generative models, one-step generative models have recently emerged through direct mapping from noise to data in a single forward pass. However, the reward-guided fine-tuning method of one-step generative models remains largely unexplored. To address this, we consider one-step generators from an optimal transport view, investigating Wasserstein Gradient Flow (WGF) for modeling smooth and controlled distributional
evolution in probability space. We then propose a novel reward-guided fine-tuning of a one-step generative model via WGF. We derive a practical training method that requires no reward gradients, thereby handling both non-differentiable and differentiable rewards. Moreover, our method provides smooth and stable reward-guided distributional updates while mitigating reward hacking and mode collapse. Experiments on 2D synthetic data, CIFAR-10, and ImageNet 256$\times$256 with diverse rewards, including JPEG (in)compressibility, class probability, Black-and-White and CLIP
alignment, show that our method achieves better reward
alignment compared to baselines.
Read the full abstract ▾
All authors Hoseong Hwang, Woorim Han, Joungin Chun, Jinseong Park, Jaewoong Choi
Machine Learning
LLMODE: Aligning ODEs with LLMs via Gated Token Injection for Irregular Spatio-Temporal Forecasting
Di Zhang, Jingyang Zhang, Ziqian Wang et al. (+4 more)
Large
language models (LLMs) have shown promise for spatio-temporal
forecasting, but existing approaches often rely on regularly sampled
token sequences and struggle with irregular observations because of temporal asynchrony, representation-space misalignment, and limited context windows. We propose LLMODE, a
token-efficient framework for irregular spatio-temporal
forecasting with a frozen LLM backbone. LLMODE first uses a graph-aware ODE encoder to reconstruct irregular graph observations as a continuous-time latent trajectory. A Fixed-Budget Perceiver Resampler then compresses this variable-length trajectory into a fixed number of dynamic
memory tokens. In parallel, compact statistical descriptors are encoded and resampled into context
memory tokens. A dual-source gated cross-
attention module injects both
memories into the frozen LLM, enabling controlled utilization of external spatio-temporal evidence. Experiments on three real-world urban
datasets and two physical-dynamics
benchmarks show competitive overall performance, with clearer advantages under sparse or dynamically complex irregular sampling. Additional evaluations on unseen urban regions further demonstrate strong zero-shot generalization without adaptation.
Read the full abstract ▾
All authors Di Zhang, Jingyang Zhang, Ziqian Wang, Chi Zhang, Yikun Ban, Ziwei Zhang, Ruijie Wang
Machine Learning
Unsupervised Multi-Scale Gromov-Wasserstein Hypergraph Alignment
Lutz Oettershagen, Honglian Wang and Aristides Gionis
We study unsupervised hypergraph
alignment, where the goal is to infer node correspondences between two hypergraphs using only structural information, without node features, labels, seed matches, or side information. Direct higher-order formulations can represent hyperedge interactions faithfully, but they can be computationally demanding and cumbersome for non-uniform hypergraphs. Graph-reduction approaches introduce a different challenge: clique expansions keep the
alignment problem on the original node set but collapse all hyperedge evidence into one pairwise graph, whereas bipartite expansions preserve incidence structure but enlarge the problem from nodes to nodes plus hyperedges. We introduce FALCON (Filtration-based hypergrAph
aLignment via Cross-scale Optimal traNsport), an unsupervised optimal-transport framework for hypergraph
alignment. Instead of representing each hypergraph by a single collapsed clique graph, FALCON constructs a filtration-induced sequence of clique-based co-occurrence dissimilarity
matrices and jointly aligns all levels through one shared multi-scale Gromov–Wasserstein (GW) objective. The shared transport plan enforces a globally consistent node correspondence across filtration levels while avoiding the auxiliary hyperedge nodes introduced by bipartite expansion. Experiments on perturbation
benchmarks derived from real-world hypergraphs show that FALCON is robust to structural noise and in almost all cases outperforms strong graph- and hypergraph-
alignment baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.MA
Forward-Deployed Full-Stack Engineering for Autonomous Cloud MLOps
Sagar Srinivas Sakhinana and Venkataramana Runkana
Across industries, machine-learning systems support applications ranging from prediction and anomaly detection to
forecasting, optimization, and scheduling, yet operationalizing these systems requires coordinating application development, model pipelines, cloud infrastructure, security, deployment, monitoring, retraining, recovery, and rollback. We present an evidence-gated multi-
agent framework for transforming a natural-
language MLOps cloud engineering task into a verified repository and operational cloud deployment. The framework combines graph engineering, loop engineering, and
agent harness engineering. A stateful Graph Orchestrator coordinates specialized
agents for repository generation, review, execution, verification, release, and monitoring while governing workflow dependencies, evidence gates, retry bounds, recovery paths, and termination. Consequential lifecycle transitions proceed only when their required predicates are supported by verifiable execution or runtime evidence. Verification failures activate bounded reflection, repair, and re-verification, while runtime evidence of failure, drift, degradation, or policy violation can trigger bounded adaptation, recovery, or rollback.
Agent harness engineering constrains repository generation, review, and repair, artifact execution, and cloud operations through controlled capabilities and isolated execution environments. We realize the framework on Google Cloud Platform and evaluate repository completeness, controlled execution, evidence-gated transitions, cloud promotion, and bounded recovery. Our experimental results show that the framework prevents unsupported lifecycle transitions and drives each run toward either a verified operational deployment or an auditable terminal failure.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Cross-lingual Functional Vectors for Emotion Detection in Large Language Models
Jieying Xue, Phuong Minh Nguyen, Minh Le Nguyen et al. (+1 more)
Function vectors (FVs) have recently emerged as a promising mechanism for steering the behavior of large
language models (LLMs) by injecting task-specific latent direction representations derived from in-context demonstrations. While prior studies have shown that FVs can recover task behavior in structured in-context learning settings, their effectiveness on semantically complex tasks and their ability to generalize across
languages remain underexplored. We investigate the cross-lingual transferability of FVs using
multilingual multi-label
emotion recognition as a challenging semantic classification
benchmark. Specifically, we examine whether FVs extracted from a source
language can steer task behavior in another
language under both standard clean and perturbed zero-shot settings without providing demonstrations during inference. Across diverse cross-lingual settings, applying FVs substantially improves performance, suggesting that FVs capture
language-agnostic, task-relevant signals rather than purely
language-specific lexical patterns, and highlighting their potential as a lightweight and transferable mechanism for
multilingual task adaptation. We observe that each LLM exhibits a relatively stable optimal range of
attention heads for constructing effective FVs, and the pattern remains consistent across
languages. In addition, FVs can partially replicate the task-steering effects of standard few-shot in-context learning while avoiding the computational overhead of processing multiple demonstrations, making them effective for large-scale practical applications. Our code is available at https://github.com/yingjie7/cross_lingual_fvs.
Read the full abstract ▾
All authors Jieying Xue, Phuong Minh Nguyen, Minh Le Nguyen, Shogo Okada
Machine Learning
Wide Learning: Learning to Reach Evidence
Junzhou Chen
Machine learning is usually evaluated after an evidence interface has been fixed. A
dataset, sensor suite, query
language, action set, or experimental protocol determines which observations can be obtained, and learning is judged by what it extracts from them. We study a complementary capability. A learner's state can determine which evidence-generating experiments it can reliably realise under bounded resources, even when primitive affordances remain fixed. We call this learner-relative experiment family its effective epistemic reach, and use Wide Learning for task-relevant learning-induced changes in that family.We formalise effective reach relative to learner state, deployment budget, reliability threshold, and evaluation distribution. In a controlled construction, two hidden worlds have exactly the same public observation law. An informative diagnostic exists in a fixed five-primitive substrate. Before calibration, one address attempt realises it with probability at most $2^{-10} = 1/1024$, below a pre-specified 0.95 threshold; after calibration, held-out realisation is 1. Public-channel total variation is 0, whereas the realised diagnostic has total variation 1, and sealed binary risk moves from approximately 1/2 to 0. The construction establishes that learning can change effective epistemic reach even when primitive affordances and deployment resources are held fixed. It opens a complementary evaluation question for learning systems: not only what they infer from available evidence, but what informative evidence experience teaches them to bring within reach.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
$N_0$-Foundation: Towards the Age of Tactile Intelligence
NeoteAI Team and Fudan TEAI Team
We present $N_0$-Foundation, a paradigm for tactile-enabled
embodied manipulation, which integrates tactile sensing hardware, large-scale
multimodal data, tactile representation learning, and standardized evaluation. First, we engineer the infrastructure for scalable data collection, including a
vision-based tactile sensor, a tactile Universal Manipulation Interface (UMI), and a synchronized visuo-tactile data collection system supporting both
robot embodiments and UMI-based demonstrations. Leveraging this infrastructure, we construct NeoData, which contains more than 30000 hours of synchronized visual and tactile demonstrations, spanning six embodiments, 450 tasks, and billions of paired RGB and tactile frames collected through a mixture of real-
robot teleoperation and UMI-based demonstrations. To facilitate open research, we further release OpenNeoData, a 5000-hour open-source subset of NeoData. The
dataset addresses a central limitation of existing manipulation corpora, critical for deformable-object manipulation, precise assembly, delicate force control, and sustained surface interaction. Capitalizing on the large-scale, heterogeneous tactile measurements, we propose NeoForce, a visuo-tactile representation model that learn transferable tactile representations across different sensor designs. To enable systematic evaluation of tactile
embodied models built upon our infrastructure,
datasets and tactile representations, we further propose a comprehensive
benchmark, which combines the real-world NeoReal suite and the simulated NeoSim suite for standardized evaluation. Experiments across both suites show that policies benefit from the physical contact state rather than from the device-specific appearance of the tactile signal. We release the
dataset, the representation, and the
benchmark, aiming at supporting future work on tactile-enabled
embodied manipulation.
Read the full abstract ▾
Machine Learning
Adaptive Doubly Robust Off-Policy Evaluation for Ranking Policies under Diverse User Behavior
Kosuke Iguchi and Ren Kishimoto
Off-policy evaluation (OPE) of ranking policies is challenging be- cause selecting and ordering multiple items from a candidate set makes the number of possible rankings grow combinatorially with the number of candidates and the ranking length. Consequently, Inverse Propensity Scoring (IPS), whose importance weight is the full-ranking probability ratio under the evaluation and logging policies, can have excessive variance. Independent IPS (IIPS) and Reward Interaction IPS (RIPS) reduce variance by imposing fixed assumptions on how users browse rankings, but may introduce bias when those assumptions mismatch actual behavior. Adaptive Inverse Propensity Scoring (AIPS) addresses this trade-off by adap- tively marginalizing importance weights over the actions that affect each position-wise reward. It attains minimum variance within a class of unbiased IPS-based
estimators when the true user be- havior model is observed. However, its estimation accuracy may still degrade for longer rankings, and AIPS does not use a reward model for residual correction. We propose Adaptive Doubly Robust (ADR), which combines adaptive importance weighting with re- ward regression through a control-variate correction. We establish its unbiasedness when the true user behavior model is observed and characterize a sufficient condition under which it reduces vari- ance relative to AIPS. Across synthetic experiments with 10,000 simulations per condition, ADR improves mean squared error over AIPS and conventional ranking OPE
estimators across a range of logged-data sizes and ranking lengths.
Read the full abstract ▾
Machine Learning
On the Resilience of Text-to-Video Diffusion Models to Hardware Faults
Zachary Coalson, A M Aahad, Stella Doehring et al. (+2 more)
We present the first systematic study of the resilience of text-to-
video (T2V)
diffusion models under random hardware-level faults. While T2V models are widely used for automated
video generation due to their ability to produce high-quality, temporally coherent, and realistic
videos, their iterative denoising process and spatiotemporal dependencies introduce unique failure modes. We perform an extensive fault-injection study covering both computational and
memory faults across three T2V models and a representative
benchmark. Our results show that (1) a single fault can degrade overall performance by up to 3.7%, with semantic correctness more affected than perceptual quality; (2)
memory faults are more damaging than computational faults, high-order exponent bits are particularly vulnerable, and the widely-used bfloat16 is more susceptible than alternative formats; and (3) 7-28% of faults cause visible artifacts, including semantic changes such as added objects, suggesting that single faults are sufficient to alter output semantics. Our findings reveal reliability risks in deployed T2V systems and motivate further research on improving fault resilience. Code: \href{https://github.com/ztcoalson/T2V-Resilience}{https://github.com/ztcoalson/T2V-Resilience}.
Read the full abstract ▾
All authors Zachary Coalson, A M Aahad, Stella Doehring, Zane Ma, Sanghyun Hong