Machine Learning
jNO: A JAX Library for Neural Operator and Foundation Model Training
Leon Armbruster, Rathan Ramesh, Georg Kruse et al. (+1 more)
jNO (jax Neural Operators) is a JAX-native library for neural operators and foundation models with unified support for both data-driven and physics-informed training. Its core design is a tracing system in which domains, model calls, residuals, supervised losses, and diagnostics are written in one symbolic
language and compiled into one optimization pipeline. This allows users to move between operator regression, mesh-aware residual evaluation, and PDE-constrained training without restructuring the surrounding code. jNO also supports multi-model compositions, fine-grained control at parameter level (model, optimizer, and learning rate), hyperparameter tuning, and JAX-native workflows for translated PDE foundation-model families. The source repository is available at https://github.com/FhG-IISB/jNO.
Read the full abstract ▾
All authors Leon Armbruster, Rathan Ramesh, Georg Kruse, Christopher Straub
Machine Learning
Unsupervised Process Reward Models
Artyom Gadetsky, Maxim Kodryan, Siba Smarak Panigrahi et al. (+2 more)
Process Reward Models (PRMs) are a powerful mechanism for steering large
language model
reasoning by providing fine-grained, step-level supervision. However, this effectiveness comes at a significant cost: PRMs require expert annotations for every
reasoning step, making them costly and difficult to scale. Here, we propose a method for training unsupervised PRMs (uPRM) that requires no human supervision, neither at the level of step-by-step annotations nor through ground-truth verification of final answers. The key idea behind our approach is to define a scoring function, derived from LLM next-
token probabilities, that jointly assesses candidate positions of first erroneous steps across a batch of
reasoning trajectories. We demonstrate the effectiveness of uPRM across diverse scenarios: (i) uPRM achieves up to 15% absolute accuracy improvements over the LLM-as-a-Judge in identifying first erroneous steps on the ProcessBench
dataset; (ii) as a verifier for test-time scaling, uPRM performs comparably to supervised PRMs and outperforms the majority voting baseline by up to 6.9%, and (iii) when used as a reward signal in reinforcement learning, uPRM enables more robust policy optimization throughout training compared to a supervised PRM trained using ground-truth labels. Overall, our results open a path toward scalable reward modeling for complex
reasoning tasks.
Read the full abstract ▾
All authors Artyom Gadetsky, Maxim Kodryan, Siba Smarak Panigrahi, Hang Guo, Maria Brbic
Machine Learning
Stable Long-Horizon PDE Forecasting via Latent Structured Spectral Propagators
Xiaoxiao Lu, Ye Yuan and Jiahao Shi
Long-horizon
forecasting of time-dependent partial differential equations (PDEs) is critical for characterizing the sustained
evolution of physical systems. While neural operators have emerged as efficient surrogates, they typically learn implicit finite-time transitions from discrete observations. When deployed autoregressively, such propagators often suffer from rapid error accumulation and dynamic drift. To address this, we propose a neural
forecasting framework that reformulates PDE rollout as learning a Structured Spectral Propagator (SSP) in a propagation-oriented latent space. Following an analysis-propagation-synthesis design, our framework: (i) maps physical states into a shared, time-consistent spatial representation; (ii) projects this space into a compact propagation state to isolate recurrent dynamics from fine-grained spatial details, thereby decoupling reconstruction fidelity from rollout regularity; and (iii) evolves retained spectral modes using a frequency-conditioned linear backbone complemented by a nonlinear spectral closure to account for truncated interactions. This explicit structuring endows the propagator with a strong inductive bias for coherent modal
evolution. Extensive experiments demonstrate that SSP significantly outperforms state-of-the-art baselines, reducing relative $L_2$ errors by up to 48.9% and exhibiting improved stability in temporal extrapolation beyond the supervised horizon.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
APEX: Audio Prototype EXplanations for Classification Tasks
Piotr Kawa, Kornel Howil, Piotr Borycki et al. (+3 more)
Explainable AI (XAI) has achieved remarkable success in image classification, yet the audio domain lacks equally mature solutions. Current methods apply
vision-based attribution techniques to spectrograms, overlooking fundamental differences between visual and acoustic signals. While prototype
reasoning is promising, acoustic similarity remains multidimensional. We introduce APEX (Audio Prototype EXplanations), a post-hoc framework for interpreting pre-trained audio classifiers. Crucially, APEX requires no fine-tuning of the original backbone and strictly preserves output invariance. APEX disentangles explanations into four perspectives: Square-based prototypes to localize transient events, Time-based for temporal patterns, Frequency-based highlighting spectral bands, and Time-Frequency-based integrating both. This yields intuitive, example-based explanations that respect acoustic properties, providing greater semantic clarity than standard gradient-based methods.
Read the full abstract ▾
All authors Piotr Kawa, Kornel Howil, Piotr Borycki, Miłosz Adamczyk, Przemysław Spurek, Piotr Syga
Machine Learning
Learning to Sparsify Stochastic Linear Bandits
Zhengmiao Wang, Ming Chi, Zhi-Wei Liu et al. (+2 more)
This paper addresses the problem of learning to sparsify
stochastic linear bandits, where a decision-maker sequentially selects actions from a high-dimensional space subject to a sparsity constraint on the number of nonzero elements in the action vector. The key challenge lies in minimizing cumulative regret while tackling the potential NP-hardness of finding optimal sparse actions due to the inherent combinatorial structure of the problem. We propose an adaptively phased exploration and exploitation algorithmic framework, utilizing ordinary least squares for parameter learning and specialized subroutines for sparse action selection. When the action set is a Euclidean ball, optimal sparse actions can be efficiently computed, enabling us to establish a $\tilde{\mathcal{O}}(d\sqrt{T})$ regret, where $d$ is the dimension of the action vector and $T$ is the time horizon length. For general convex and compact action sets where finding optimal sparse actions is intractable, we employ a greedy subroutine. For general strongly convex action sets, we derive a $\tilde{\mathcal{O}}(d \sqrt{T})$ $\alpha$-regret; for general compact sets lacking strong convexity, we establish a $\tilde{\mathcal{O}}(d T^{2/3})$ $\alpha$-regret, where $\alpha$ pertains to the approximation ratio of the greedy
algorithm. Finally, we validate the performance of our
algorithms using extensive experiments including an application to recommendation system.
Read the full abstract ▾
All authors Zhengmiao Wang, Ming Chi, Zhi-Wei Liu, Lintao Ye, Carla Fabiana Chiasserini
Machine Learning · Cross-listed from stat.ML
PFN-TS: Thompson Sampling for Contextual Bandits via Prior-Data Fitted Networks
Yan Shuo Tan, Kenyon Ng, Ruizhe Deng et al. (+3 more)
Thompson sampling is a widely used strategy for contextual bandits: at each round, it samples a reward function from a
Bayesian posterior and acts greedily under that sample. Prior-data fitted networks (PFNs), such as TabPFN v2+ and TabICL v2, are attractive candidates for this purpose because they approximate
Bayesian posterior predictive distributions in a single forward pass. However, PFNs predict noisy future rewards, while Thompson sampling requires uncertainty over the latent mean reward function. We propose PFN-TS, a Thompson sampling
algorithm that converts PFN posterior predictives into mean-reward samples using a subsampled predictive central limit
theorem. The method estimates posterior variance from a
geometric grid of $O(\log n)$
dataset prefixes rather than the full $O(n)$ predictive sequence used in previous predictive-sequence approaches, and reuses TabICL's cached representations across rounds. We prove consistency of the subsampled variance
estimator and give a
Bayesian regret bound that decomposes PFN-TS regret into exact posterior-sampling regret under the PFN prior plus approximation terms. Empirically, PFN-TS achieves the best average rank across nonlinear synthetic and OpenML classification-to-bandit
benchmarks, remains competitive on linear and BART-generated rewards, and attains the highest estimated policy value in an offline mobile-health evaluation. Code is available at https://anonymous.4open.science/r/PFN_TS-36ED/.
Read the full abstract ▾
All authors Yan Shuo Tan, Kenyon Ng, Ruizhe Deng, Sumetha Loganathan, Qiong Zhang, Bibhas Chakraborty
Machine Learning
Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networks
Bum Jun Kim and Gnankan Landry Regis N'guessan
Physics-informed neural networks (PINNs) train a single neural approximation by minimizing multiple physics- and data-derived losses, but the gradients of these losses often interfere and can stall optimization. Existing remedies typically treat this pathology either through scalar loss balancing or full-parameter-space gradient surgery, leaving it unclear which intervention is most appropriate. We show that PINN gradient conflict is not a uniform failure mode with one universal remedy. Instead, we identify distinct PINN gradient-conflict regimes, each associated with a different intervention class. Persistent directional conflict may require separate loss-indexed parameter subspaces, magnitude imbalance often favors scalar reweighting, and low or transient conflict may require no extra mitigation. To select between scalar reweighting and a lightweight architectural intervention, we propose a diagnostic-first framework. It profiles a 1000-step unmodified PINN run and, when intervention is warranted, uses one low-rank adapter per loss to create explicit loss-indexed parameter subspaces attached to a shared PINN trunk, providing each loss with a direct gradient pathway. Across more than 60 PDE configurations, including forward, inverse, multi-physics, parameter-varying, and high-dimensional problems up to 50D, persistent directional conflict dominates standard forward $K=3$
benchmarks and a natural $K=4$ thermoelastic system, where adapters combined with reweighting yield significant improvements. In contrast, $K=3$ inverse problems and natural $K=5$ and $K=6$ multi-physics systems are largely magnitude-dominated and often favor reweighting alone, while full-parameter-space gradient surgery can fail on heterogeneous parameter spaces.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NI
GELATO: Generative Entropy- and Lyapunov-based Adaptive Token Offloading for Device-Edge Speculative LLM Inference
Zengzipeng Tang, Yuxuan Sun, Wei Chen et al. (+2 more)
The recent growth of on-device Large
Language Model (LLM) inference has driven significant interest in device-edge collaborative LLM inference. As a promising architecture, Speculative Decoding (SD) is increasingly adopted where a lightweight draft model rapidly generates candidate
tokens to be verified by a powerful target model. However, a fundamental challenge lies in achieving per-
token resource scheduling to effectively adapt SD paradigm to resource-constrained edge environment. This paper proposes a Generative
Entropy- and Lyapunov-based Adaptive
Token Offloading framework, named GELATO, to maximize decoding throughput under energy constraints in a device-edge collaborative SD system. Specifically, an outer drift-plus-penalty loop makes online decisions to establish a reference drafting budget, managing long-term energy-throughput trade-off. Further, a nested
entropy-driven generation mechanism executes early exiting to adapt to per-
token dynamic generative uncertainty. Theoretical analysis establishes a rigorous performance bound on long-term throughput for GELATO. Extensive evaluations demonstrate that GELATO achieves a globally optimal tradeoff, outperforming state-of-the-art distributed SD architectures by 64.98% in
token throughput and reducing energy consumption by 47.47% under resource-constrained environments, while preserving LLM decoding quality.
Read the full abstract ▾
All authors Zengzipeng Tang, Yuxuan Sun, Wei Chen, Jianwen Ding, Bo Ai
Machine Learning
Complex-Valued Phase-Coherent Transformer
Leona Hioki
Complex-valued
Transformers have largely inherited softmax
attention from real-valued architectures. However, row-normalised
token competition is not necessarily aligned with phase-preserving computation. In this paper, we introduce the Phase-Coherent
Transformer (PCT), which applies a real-valued, element-independent, smooth gate to L2-normalised complex query-key similarities. PCT replaces
token competition with
token-non-competing
attention and is designed to preserve phase information across layers. Across mid-scale
benchmarks spanning long-range
memory, hierarchical long-range
reasoning, positional
retrieval, phase-based
memory and superposition, and image classification, PCT shows strong generalisation across task categories. Under parameter-fair comparison, PCT consistently outperforms both the standard softmax
Transformer and its direct complex-valued counterpart. Moreover, even on tasks traditionally considered difficult for complex-valued neural networks, such as NIAH and LRA-Text, PCT remains competitive with Multiscreen, the strongest real-valued NN baseline in our comparison. Experiments introducing gates that deliberately violate the PCT conditions show that the design is not incidental: smooth gates that preserve negatively aligned phase components remain strong, whereas gates that delete such components collapse on long-range
retrieval, and gates whose outputs become excessively large suffer clear performance degradation. PCT also shows no depth-related accuracy collapse across the tested depth range. These results support introducing multi-layer phase-coherent structure into
attention as a promising design principle for achieving generalisation in complex-valued
Transformers.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Rethinking Constraint Awareness for Efficient State Embedding of Neural Routing Solver
Canhong Yu, Changliang Zhou, Rongsheng Chen et al. (+2 more)
Heavy-Encoder-Light-Decoder (HELD) neural routing solvers have emerged as a promising paradigm due to their broad applicability across multiple vehicle routing problems (VRPs). However, they typically struggle with VRP variants with complex constraints. To address this limitation, this paper systematically revisits existing neural solvers from the perspective of the generation mechanism for state
embeddings (i.e., query vector prior to compatibility calculation) during decoding. We identify that current mechanisms restrict the observation space during
attention computation, introducing a key bottleneck to achieving high-quality solutions. Through detailed empirical analysis, we demonstrate the necessity of preserving a global observation space. To overcome the constraint-agnostic drawback inherent to global observation spaces, we propose a simple yet powerful Constraint-Aware Residual Modulation (CARM) module. By adaptively modulating the context
embedding with constraint-relevant variables, CARM effectively enhances constraint awareness, enabling the neural solver to fully leverage the global observation space and generate an efficient state
embedding. Extensive experimental results across two single-task and five multi-task neural routing solvers confirm that the CARM module consistently boosts baseline performance. Notably, solvers equipped with our CARM achieve substantial improvements in scaling to large-scale instances and in generalizing to unseen VRP variants. These findings provide valuable insights for the architectural design of neural routing solvers.
Read the full abstract ▾
All authors Canhong Yu, Changliang Zhou, Rongsheng Chen, Zhenkun Wang, Yu Zhou
Machine Learning
Explainability of Recurrent Neural Networks for Enhancing P300-based Brain-Computer Interfaces
Christian Oliva, Vinicio Changoluisa, Francisco B Rodríguez et al. (+1 more)
Brain-Computer Interfaces (BCIs) based on P300 event-related potentials offer promising applications in health, education, and assistive technologies. However, challenges related to inter- and intra-subject variability and the explainability of Deep Learning (DL) models limit their practical deployment. In this work, we present the Post-Recurrent Module (PRM), an additional layer designed to improve both performance and transparency, incorporated into a Recurrent Neural Network (RNN) architecture for classifying P300 signals from
EEG data. Our approach enables a dual analysis of spatio-temporal signals through both global and local explainability techniques, allowing us not only to identify the most relevant
brain regions and critical time intervals involved in classification, but also to interpret model decisions in terms of spatio-temporal
EEG patterns consistent with well-stablished neurophysiological descriptions of the P300. Experimental results show a 9% improvement in performance over state of the art, while also revealing the importance of inter- and intra-subject variability, in
alignment with established neuroscience literature. By making model decisions transparent and efficient, we present a framework for explainable
EEG-based models. This framework is not limited to more efficient P300 detection, but can be generalized to a wide range of
EEG-based tasks. Its ability to identify key spatial and temporal features makes it suitable for applications such as motor imagery, steady-state visual evoked potentials, and even cognitive workload assessment.
Read the full abstract ▾
All authors Christian Oliva, Vinicio Changoluisa, Francisco B Rodríguez, Luis F Lago-Fernández
Machine Learning
Refresh-Scaling the Memory of Balanced Adam
Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre et al. (+2 more)
Recent evidence suggests that Adam performs robustly when its momentum parameters are tied, $\beta_1=\beta_2$, reducing the optimizer to a single remaining parameter. However, how this parameter should be set remains poorly understood. We argue that, in balanced Adam, $\beta$ should not be treated as a dimensionless constant: it defines a statistical
memory horizon $H_\beta=(1-\beta)^{-1}$. In terms of the effective learning horizon $T_{\mathrm{ES}}$, estimated from the validation trajectory, we study the refresh count $R_\beta=(1-\beta)T_{\mathrm{ES}}$, which measures how many times Adam renews its internal statistics during the useful phase of training. Across 11
vision and
language experiments, we find that choosing $\beta$ so that $R_\beta\approx1000$ selects different $\beta$ values depending on the training scale, yet improves robustness over the best fixed-beta baseline. Compared with the strongest fixed choice $\beta=0.944$, the refresh rule improves worst-case robustness, reducing the maximum relative gap in validation loss by 33.4%, while bringing all 11 runs within 1% of their validation oracle. These results suggest that the remaining hyperparameter of balanced Adam is more naturally viewed as a
memory-scale variable than as a fixed constant. This provides a simple budget-aware perspective on optimizer scaling and opens a path toward treating Adam's momentum as part of the learning dynamics rather than as a static default.
Read the full abstract ▾
All authors Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre, Manuel F. Dolz, Enrique S. Quintana-Ortí
Machine Learning
SymADiT: Symmetry-Aware Crystal Generation with Latent Flow Matching
Anmar Karmush, Cedric Mathieu Brandenburg, Soheil Ershadrad et al. (+3 more)
Tackling the task of
crystal generation, we aim to enhance the previously proposed,
symmetry-agnostic, All-atom
Diffusion Transformer (ADiT) by introducing SymADiT, a
symmetry-aware variant. To do so, we use a representation of
crystals based on Wyckoff positions. We follow ADiT and perform generative modelling in latent space, adapted to our
symmetry-aware representation. By forcing the output of the generative model to adhere to the
symmetry restrictions imposed by the generated
crystal's space group and each atom's Wyckoff position, the generated
crystals exhibit more realistic
symmetry properties. We
benchmark our method against both
symmetry-aware and
symmetry-agnostic models for
crystal generation and show competitive performance, generating stable, symmetric, and physically plausible
crystals with only an off-the-shelf
Transformer architecture. Our code is available at https://github.com/Karmush21/SymADiT.
Read the full abstract ▾
All authors Anmar Karmush, Cedric Mathieu Brandenburg, Soheil Ershadrad, Johanna Rosen, Michael Felsberg, Filip Ekström Kelvinius
Machine Learning
CFSPMNet: Cross-subject Fourier-guided Spatial-Patch Mamba Network for EEG Motor Imagery Decoding in Stroke Patients
Xiangkai Wang, Yun Zhao, Dongyi He et al. (+5 more)
Motor imagery electroencephalography (MI-
EEG) decoding offers a non-invasive route for post-stroke rehabilitation, but cross-patient use remains difficult because pathological neural reorganization changes task-related
EEG dynamics, aperiodic activity, local excitability, cross-regional coordination, and trial-level
brain-state context. This makes source-learned MI representations unreliable for unseen patients. To address this problem, we propose CFSPMNet, a cross-patient adaptation framework that models post-stroke MI-
EEG as latent neural-state organization. CFSPMNet combines a
Fourier-Reorganized State Mamba Network (FRSM) with Shared-Private Prototype Matching (SPPM). FRSM represents each trial as a latent physiological
token sequence, reorganizes
token states in the
Fourier domain, and uses
Fourier-derived trial context to guide Mamba state-space propagation. SPPM improves target pseudo-label updating by combining semantic confidence with shared-private physiological consistency, filtering confident but physiologically inconsistent target predictions. Leave-one-subject-out experiments on two stroke MI-
EEG datasets show that CFSPMNet outperforms representative CNN-,
Transformer-, Mamba-, and adaptation-based baselines, achieving average accuracies of 68.23% on XW-Stroke and 73.33% on 2019-Stroke, with gains of 5.63 and 8.25 percentage points over the strongest competitors. Ablation, sensitivity, feature-
alignment, pseudo-label selection, and neurophysiological visualization analyses further support the roles of
Fourier-domain
token-state reorganization and calibrated pseudo-label updating. These results suggest that latent neural-state modeling can improve rehabilitation-oriented cross-patient BCI decoding. Code is available at https://github.com/wxk1224/CFSPMNet.
Read the full abstract ▾
All authors Xiangkai Wang, Yun Zhao, Dongyi He, Qingling Xia, Gen Li, Xinlai Xing, Yuchi Pan, Bin Jiang
Machine Learning · Cross-listed from cs.CL
GLiNER-Relex: A Unified Framework for Joint Named Entity Recognition and Relation Extraction
Ihor Stepanov, Oleksandr Lukashov, Mykhailo Shtopko et al. (+1 more)
Joint named entity recognition (NER) and relation extraction (RE) is a fundamental task in natural
language processing for constructing knowledge graphs from unstructured text. While recent approaches treat NER and RE as separate tasks requiring distinct models, we introduce GLiNER-Relex, a unified architecture that extends the GLiNER framework to perform both entity recognition and relation extraction in a single model. Our approach leverages a shared bidirectional
transformer encoder to jointly represent text, entity type labels, and relation type labels, enabling zero-shot extraction of arbitrary entity and relation types specified at inference time. GLiNER-Relex constructs entity pair representations from recognized spans and scores them against relation type
embeddings using a dedicated relation scoring module. We evaluate our model on four standard relation extraction
benchmarks: CoNLL04, DocRED, FewRel, and CrossRE, and demonstrate competitive performance against both specialized relation extraction models and large
language models, while maintaining the computational efficiency characteristic of the GLiNER family. The model is released as an open-source Python package with a simple inference API that allows users to specify arbitrary entity and relation type labels at inference time and obtain both entities and relation triplets in a single call. All models and code are publicly available.
Read the full abstract ▾
All authors Ihor Stepanov, Oleksandr Lukashov, Mykhailo Shtopko, Vivek Kalyanarangan
Machine Learning
TopoU-Net: a U-Net architecture for topological domains
Gaurav Gaurav, Ibrahem ALJabea, Yaroslav Zakomornyy et al. (+4 more)
Many modern
datasets mix points, edges, regions, groups, objects, events, hyperedges, and relations. Yet neural architectures often force such data into grids, graphs, or sequences, obscuring higher-order structure and making encoder-decoder designs domain-specific. We view U-Net not as a grid-specific architecture, but as a hierarchical encoder-decoder principle: representation spaces, transport maps between levels, and skip connections between matched levels. Combinatorial complexes naturally supply these ingredients through
cells, incidences, and ranks. We introduce TopoU-Net, a rank-path U-Net for topological domains. Given a path from an input rank to a bottleneck rank and back, the encoder lifts cochains upward along incidence maps, the decoder transports them downward, and skip connections merge features at matched ranks. Rank replaces spatial scale: choosing paths through nodes, edges, faces, hyperedges, or global
cells becomes the central architectural decision. A key quantity is the bottleneck support ratio, the number of
cells at the bottleneck relative to the number of
cells at the input rank. This ratio is fixed by the complex and chosen path rather than by arbitrary pooling, and it clarifies when skip connections are optional, useful, or structurally important. Across node classification, graph classification, hypergraph node classification, mesh classification, and image reconstruction, TopoU-Net provides a reusable encoder-decoder template for higher-order structured data. Among the evaluated baselines, it achieves the strongest mean accuracy on six of eight node-classification
datasets and four of five hypergraph
datasets, with the largest gains on heterophilic graphs. Ablations show that removing skip connections is most damaging under severe bottleneck compression.
Read the full abstract ▾
All authors Gaurav Gaurav, Ibrahem ALJabea, Yaroslav Zakomornyy, Eric Frank, Mohamed Elhamdadi, Theodore Papamarkou, Mustafa Hajij
Machine Learning · Cross-listed from eess.AS
PoDAR: Power-Decoupled Audio Representation for Generative Modeling
Alejandro Luebs, Mithilesh Vaidya, Ishaan Kumar et al. (+5 more)
The performance of audio latent
diffusion models is primarily governed by generator expressivity and the modelability of the underlying latent space. While recent research has focused primarily on the former, as well as improving the reconstruction fidelity of audio codecs, we demonstrate that latent modelability can be significantly improved through explicit factor decoupling. We present PoDAR (Power-Decoupled Audio Representation), a framework that utilizes a randomized power augmentation and latent consistency objective to decouple signal power from invariant semantic content. This factorization makes the latent space easier to model, which both accelerates the convergence of downstream generative models and improves final overall performance. When applied to a Stable Audio 1.0 VAE with an F5-TTS generator, PoDAR achieves about a 2x acceleration in convergence to match baseline performance, while increasing final speaker similarity by 0.055 and UTMOS by 0.22 on the LibriSpeech-PC
dataset. Furthermore, isolating power into dedicated channels enables the application of CFG exclusively to power-invariant content, effectively extending the stable guidance regime to higher scales.
Read the full abstract ▾
All authors Alejandro Luebs, Mithilesh Vaidya, Ishaan Kumar, Sumukh Badam, Stephen W. Bailey, Matthew Bendel, Jose Sotelo, Xingzhe He
Machine Learning
Unlocking air traffic flow prediction through microscopic aircraft-state modeling
Bin Wang, Anqi Liu, Jiangtao Zhao et al. (+8 more)
Short-term air traffic flow prediction in terminal airspace is essential for proactive air traffic management. Existing approaches predominantly model traffic flow as aggregated time series. However, traffic dynamics are governed by aircraft states and their interactions in continuous airspace. Such aggregation obscures fine-grained information, including aircraft kinematics, boundary interactions, and control-intent cues. Here we present AeroSense, a state-to-flow modeling paradigm that predicts future traffic flow directly from instantaneous airspace situations represented as dynamic sets of aircraft states derived from ADS-B trajectories. By establishing an end-to-end mapping from microscopic aircraft states to future regional traffic flow, AeroSense preserves aircraft-level dynamics while naturally accommodating varying traffic density, and avoids reliance on historical look-back windows. Experiments on a large-scale real-world
dataset show that AeroSense exhibits strong predictive accuracy and robustness compared with time series-based
forecasting approaches, without requiring exhaustive hyperparameter tuning. These findings suggest that aircraft-state situation modeling provides a promising alternative to conventional time-series
forecasting in air traffic flow management.
Read the full abstract ▾
All authors Bin Wang, Anqi Liu, Jiangtao Zhao, Yanyong Huang, Hina Birahmani, Peilan He, Guiyuan Jiang, Feng Hong, Yanwei Yu, Yuanyuan Hou, Tianrui Li
Machine Learning · Cross-listed from cs.CL
Ruhan Wang, Chengkai Huang, Zhiyong Wang et al. (+6 more)
Large
language models (LLMs) exhibit strong
reasoning capabilities when guided by high-quality demonstrations, yet such data is often distributed across organizations that cannot centralize it due to regulatory, proprietary, or institutional constraints. We study
federated reasoning, where a server improves multi-step
reasoning by coordinating with heterogeneous clients holding private demonstrations, without centralized training or raw data sharing. The key challenge is that client reliability is query-dependent, while the server cannot inspect client data to determine which contributions are trustworthy. To address this, we propose Uncertainty-Aware
Federated Reasoning (FERA), a training-free framework based on iterative server-client co-refinement. Across communication rounds, clients generate
reasoning traces with lightweight uncertainty estimates, and the server synthesizes them into improved
reasoning that is redistributed as context for the next round, progressively improving both server outputs and client-side
reasoning. Within each round, Uncertainty-Aware Self-Critique Aggregation (UA-SCA) resolves conflicts among heterogeneous client traces through query-dependent trust weighting and structured cross-client verification. Rather than simply discarding low-quality traces, UA-SCA revises flawed
reasoning steps to recover useful information. We provide theoretical guarantees showing that the proposed iterative protocol converges and that uncertainty-aware weighting accelerates convergence. Experiments on multiple
reasoning benchmarks show that FERA consistently outperforms both
federated training and training-free baselines, achieving progressively higher accuracy across rounds while maintaining communication and computational efficiency.
Read the full abstract ▾
All authors Ruhan Wang, Chengkai Huang, Zhiyong Wang, Junda Wu, Rui Wang, Tong Yu, Julian McAuley, Lina Yao, Dongruo Zhou
Machine Learning · Cross-listed from eess.IV
A Stability Benchmark of Generative Regularizers for Inverse Problems
Alexander Denker, Johannes Hertrich and Sebastian Neumayer
Generative (
diffusion) priors demonstrate remarkable performance in addressing inverse problems in imaging. Yet, for scientific and medical imaging, it is crucial that reconstruction techniques remain stable and reliable under imperfect settings. Typical definitions of stability encompass the notion of ”convergent regularization”, robustness to out-of-distribution data, and to inaccuracies in the forward operator or noise model. We evaluate these properties numerically. Furthermore, we
benchmark generative approaches against modern optimization-based methods inspired by the widely used variational techniques. Our results give insights for which settings and applications generative priors can deliver state-of-the-art reconstructions, and on those in which they fall short or may even be problematic.
Read the full abstract ▾
Machine Learning
Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization
Huilin Zhou, Jian Zhao, Yilu Zhong et al. (+7 more)
Red teaming is critical for uncovering vulnerabilities in Large
Language Models (LLMs). While automated methods have improved scalability, existing approaches often rely on static heuristics or
stochastic search, rendering them brittle against advanced safety
alignment. To address this, we introduce Metis, a framework that reformulates jailbreaking as inference-time policy optimization within an
adversarial Partially Observable Markov Decision Process (POMDP). Metis employs a self-evolving metacognitive loop to perform causal diagnosis of a target's defense logic and leverages structured feedback as a semantic gradient to refine its policy, offering enhanced interpretability through transparent
reasoning traces. Extensive evaluations across 10 diverse models demonstrate that Metis achieves the strongest average Attack Success Rate (ASR) among compared methods at 89.2%, maintaining high efficacy on resilient frontier models (e.g., 76.0% on O1 and 78.0% on GPT-5-chat) where traditional baselines exhibit substantial performance degradation. By replacing redundant exploration with directed optimization, Metis reduces
token costs by an average of 8.2x and up to 11.4x. Our analysis reveals that current defenses remain vulnerable to internally-steered, closed-loop
reasoning trajectories under the tested settings, highlighting a critical need for next-generation defenses capable of
reasoning about safety dynamically during inference.
Read the full abstract ▾
All authors Huilin Zhou, Jian Zhao, Yilu Zhong, Zhen Liang, Xiuyuan Chen, Yuchen Yuan, Tianle Zhang, Chi Zhang, Lan Zhang, Xuelong Li
Machine Learning
Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of View
Jinping Wang, Zixin Tong, Zhiwu Xie et al. (+1 more)
Loss reweighting is a widely used strategy for long-tailed classification, but existing reweighting strategies often rely on heuristics and rarely define a well-specified target. Inspired by Neural Collapse (NC), the ideal simplex Equiangular Tight Frame (ETF) terminal geometry suggests equal per-class average loss as a reasonable target for reweighting. Based on the ideal equal loss objective, we consider loss reweighting as an inverse problem and propose an inverse-view reweighting strategy that infers class weights dynamically to match this ideal objective. Empirically, NC metrics suggest our method can effectively reduce the loss imbalance coefficient and closer
alignment with NC geometry while consistently outperforming strong long-tailed baselines on different
datasets.
Read the full abstract ▾
All authors Jinping Wang, Zixin Tong, Zhiwu Xie, Zhiqiang Gao
Machine Learning · Cross-listed from cs.CV
PixelFlowCast: Latent-Free Precipitation Nowcasting via Pixel Mean Flows
Yufeng Zhu, Chunlei Shi, Yongchao Feng et al. (+1 more)
Precipitation nowcasting aims to forecast short-term radar echo sequences for extreme weather warning, where both prediction fidelity and inference efficiency are critical for real-world deployment. However,
diffusion-based models, despite their strong generative capability, suffer from slow inference due to multi-step sampling trajectories, limiting their practical usability. Conditional Flow Matching (CFM) improves efficiency via straightened trajectories, but relies on latent space compression, which inevitably discards high-frequency physical details and degrades fine-grained prediction quality. To address these limitations, we propose PixelFlowCast, a two-stage probabilistic
forecasting framework that achieves both high-efficiency and high-fidelity prediction without latent compression. Specifically, in the first stage, a deterministic model first produces coarse forecasts to capture global
evolution trends. In the subsequent stage, the proposed KANCondNet extracts deep spatiotemporal
evolution features to provide accurate conditional guidance. Based on this, a latent-free, few-step Pixel Mean Flows (PMF) predictor employs an $x$-prediction mechanism to generate high-quality predictions, effectively preserving fine-grained structures while maintaining fast inference. Experiments on the publicly available SEVIR
dataset demonstrate that PixelFlowCast outperforms existing mainstream methods in both prediction accuracy and inference efficiency, particularly for long sequence
forecasting, highlighting its strong potential for real-world operational deployment.
Read the full abstract ▾
All authors Yufeng Zhu, Chunlei Shi, Yongchao Feng, Dan Niu
Machine Learning
Adaptive Action Chunking via Multi-Chunk Q Value Estimation
Yongjae Shin, Jongseong Chae, Seongmin Kim et al. (+2 more)
Action chunking emerged as a pivotal technique in imitation learning, enabling policies to predict cohesive action sequences rather than single actions. Recently, this approach has expanded to reinforcement learning (RL), enhancing behavioral consistency and reducing bootstrapping errors in value function estimation. However, existing methods rely on a fixed chunk length, creating a performance bottleneck as the optimal length varies across states and tasks. In this paper, we propose Adaptive Action CHunking (ACH), a novel offline-to-online RL
algorithm that dynamically modulates chunk length during both training and inference. To find the optimal chunk length for a dynamically varying current state, we simultaneously estimate action-values for all candidate chunk lengths in a single forward pass, using a
Transformer-based architecture. Our mechanism allows the
agent to select the most effective chunk length adaptively based on the current state. Evaluated on 34 challenging tasks, ACH consistently outperforms fixed-length baselines, demonstrating superior generalization and learning efficiency in complex environments.
Read the full abstract ▾
All authors Yongjae Shin, Jongseong Chae, Seongmin Kim, Jongeui Park, Youngchul Sung
Machine Learning
TrajDLM: Topology-Aware Block Diffusion Language Model for Trajectory Generation
Wilson Wongso, Lihuan Li, Arian Prabowo et al. (+4 more)
Generating high-fidelity synthetic Web of Things (WoT) telemetry, such as GPS trajectories, is increasingly important for transportation, urban planning, and location-aware services, particularly when
privacy concerns restrict access to real-world mobility data. Existing trajectory generation methods face a trade-off between efficiency and road network fidelity: continuous-space methods generate trajectories efficiently but ignore road
topology, while
topology-aware methods rely on search-based autoregressive decoding that limits generation speed. We propose TrajDLM, a
topology-aware trajectory generation framework based on block
diffusion language models. TrajDLM represents trajectories as sequences of discrete road segments, combining a block
diffusion backbone, graph-based road network representations, and
topology-constrained sampling to generate realistic trajectories with topologically valid road transitions. To address the tendency of supervised models to favor near-shortest routes, we introduce TrajRFT, a rejection-sampling fine-tuning stage that improves route fidelity by training on self-generated routes selected for their similarity to real trajectories. Across three city-scale vehicle telemetry
datasets, TrajDLM closely matches real-world mobility in population-level distributions, local route similarity, and route diversity, while generating trajectories up to $7.5\times$ faster than the strongest baseline. It also demonstrates zero-shot transfer to unseen transport modes. These results establish block
diffusion with
topology-constrained sampling and rejection-sampling fine-tuning as an efficient approach to generating realistic synthetic mobility telemetry for location-aware Web services and WoT applications.
Read the full abstract ▾
All authors Wilson Wongso, Lihuan Li, Arian Prabowo, Xiachong Lin, Baiyu Chen, Hao Xue, Flora D. Salim
Machine Learning
The two clocks and the innovation window: When and how generative models learn rules
Binxu Wang, Emma Lucia Byrnes Finn and Bingbin Liu
Generative models trained on finite data face a fundamental tension: their score-matching or next-
token objective converges to the empirical training distribution rather than the population distribution we seek to learn. Using rule-valid synthetic tasks, we trace this tension across two training timescales: $\tau_{\mathrm{rule}}$, the step at which generations first become rule-valid, and $\tau_{\mathrm{mem}}$, the step at which models begin reproducing training samples. Focusing on parity and extending to other binary rules and combinatorial puzzles, we characterize how these two clocks, $\tau_{\mathrm{rule}}$ and $\tau_{\mathrm{mem}}$, depend on key aspects of the learning setup. Specifically, we show that $\tau_{\mathrm{rule}}$ increases with rule complexity and decreases with model capacity, while $\tau_{\mathrm{mem}}$ is approximately invariant to the rule and scales nearly linearly with
dataset size $N$. We define the innovation window as the interval $[\tau_{\mathrm{rule}}, \tau_{\mathrm{mem}}]$. This window widens with increasing $N$ and narrows with rule complexity, and may vanish entirely when $\tau_{\mathrm{rule}} \geq \tau_{\mathrm{mem}}$. The same two-clock structure arises in both
diffusion (DiT) and autoregressive (GPT) models, with architecture-dependent offsets. Dissecting the learned score of DiT models reveals a corresponding
evolution of the optimization landscapes, where rule-valid samples' basins expand substantially around $\tau_{\mathrm{rule}}$, while training samples' basins begin to dominate around $\tau_{\mathrm{mem}}$. Together, these results yield a unified and predictive account of when and how generative models exhibit genuine innovation.
Read the full abstract ▾
Machine Learning
The Value of Mechanistic Priors in Sequential Decision Making
Itai Shufaro, Gal Benor and Shie Mannor
Hybrid mechanistic models, physical priors with learned residuals, promise to reduce the data required for good decisions, but have no computable criterion to test this. We characterize the value of mechanistic priors in sequential decision-making within both asymptotic and burn-in regimes. To formalize this, we introduce the mechanistic information of a model: the mutual information between the model's recommended policy $\hat{\pi}$ and the true optimal policy $\pi^*$, bounded via a centered, occupancy-weighted bias. In the asymptotic regime (large $N$), matched bounds reveal that
Bayesian regret scales with the residual
entropy $H_{\mathrm{mech}}$, delivering a theoretical sample complexity reduction of $H(\mu)/H_{\mathrm{mech}}$ compared to an uninformed baseline. We further provide a computable pre-trial model certificate. Complementarily, in the clinically relevant burn-in regime (small $N$), we establish a lower bound on the penalty incurred by confidently wrong priors. We demonstrate both the asymptotic and burn-in bounds on an illustrative in-silico 5-fluorouracil (5-FU) chemotherapy plant whose structure follows published FOLFOX pharmacokinetics. The hybrid prior reduces cumulative regret by $1.79\times$ relative to standard body-surface-area dosing and $1.85\times$ relative to an uninformed learner, and remains below both under every calibration bias tested. Finally, we show that priors sensitive to distribution shift can lose half of their mechanistic information under a small distribution shift, motivating physically grounded priors for safety-critical applications.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Differentially Private Sampling from Distributions via Wasserstein Projection
Shokichi Takakura, Seng Pei Liew and Satoshi Hasegawa
In this paper, we study the problem of sampling from a distribution under the constraint of differential
privacy (DP). Prior works measure the utility of DP sampling with density ratio-based measures such as KL divergence. However, such formulations suffer from two key limitations: 1) they fail to capture the
geometric structure of the support, and 2) they are not applicable when the supports of the distributions differ. To deal with these issues, we develop a novel framework for DP sampling with Wasserstein distance as the utility measure. In this formulation, we propose Wasserstein Projection Mechanism (WPM), a minimax optimal mechanism based on Wasserstein projection. Furthermore, we develop efficient
algorithms for computing the proposed mechanisms approximately and provide convergence guarantees.
Read the full abstract ▾
Machine Learning
Anchor-guided Hypergraph Condensation with Dual-level Discrimination
Fan Li, Xiaoyang Wang, Chen Chen et al. (+1 more)
The increasing prevalence of large-scale hypergraphs poses significant computational challenges for hypergraph neural network (HNN) training. To address this, hypergraph condensation (HGC) distills large real hypergraphs into compact yet informative synthetic ones, beyond graph condensation (GC) methods limited to pairwise relations. However, existing HGC methods rely on decoupled training architectures, where structure generators are pre-trained on the original hypergraph but not jointly optimized with condensed features during refinement, resulting in misaligned structures that degrade downstream utility. Moreover, trajectory-based optimization incurs substantial computational overhead in refinement, limiting condensation efficiency. To tackle these issues, we propose Anchor-guided HyperGraph Condensation with Dual-level Discrimination (AHGCDD), which consists of three key components: (1) a node initialization module based on Heat Kernel PageRank (HKPR) to encode structural knowledge into feature semantics; (2) an anchor-guided hyperedge synthesis strategy for joint optimization of condensed features and structure; (3) a theoretically grounded dual-level discrimination objective for utility-preserving condensation without redundant HNN training. Extensive experiments demonstrate the superior effectiveness and efficiency of AHGCDD.
Read the full abstract ▾
All authors Fan Li, Xiaoyang Wang, Chen Chen, Wenjie Zhang
Machine Learning
Continual Harness: Online Adaptation for Self-Improving Foundation Agents
Seth Karten, Joel Zhang, Tersoo Upaa Jr et al. (+5 more)
Coding harnesses such as Claude Code and OpenHands wrap foundation models with tools,
memory, and planning, but no equivalent exists for
embodied agents' long-horizon partial-observability decision-making. We first report our Gemini Plays Pokemon (GPP) experiments. With iterative human-in-the-loop harness refinement, GPP became the first AI system to complete Pokemon Blue, Yellow Legacy on hard mode, and
Crystal without a lost battle. In the hardest stages, the
agent itself began iterating on its strategy through long-context
memory, surfacing emergent self-improvement signals alongside human-in-the-loop refinement. Continual Harness removes the human fully from this loop: a reset-free self-improving harness for
embodied agents that formalizes and automates what we observed. Starting from only a minimal environment interface, the
agent alternates between acting and refining its own
prompt, sub-
agents, skills, and
memory, drawing on any past trajectory data.
Prompt-optimization methods require episode resets; Continual Harness adapts online within a single run. On Pokemon Red and Emerald across frontier models, Continual Harness starting from scratch substantially reduces button-press cost relative to the minimalist baseline and recovers a majority of the gap to a hand-engineered expert harness, with capability-dependent gains, despite starting from the same raw interface with no curated knowledge, no hand-crafted tools, and no domain scaffolding. We then close the loop with the model itself: an online process-reward co-learning loop, in which an open-source
agent's rollouts through the refining harness are relabeled by a frontier teacher and used to update the model, drives sustained in-game milestone progress on Pokemon Red without resetting the environment between training iterations.
Read the full abstract ▾
All authors Seth Karten, Joel Zhang, Tersoo Upaa Jr, Ruirong Feng, Wenzhe Li, Chengshuai Shi, Chi Jin, Kiran Vodrahalli