Machine Learning
SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning
Guangyuan Wang, Mads Toftrup, Sebastian Loeschcke et al. (+2 more)
Physics-informed neural networks (PINNs) often face ill-conditioned objectives that limit high-accuracy training. Dense quasi-Newton methods improve local conditioning but require expensive optimizer state, while Kronecker-factored methods such as SOAP scale to larger networks but rely on periodic basis updates. We introduce \method, which augments SOAP-style preconditioning with a scalar secant-energy correction adapted to Kronecker geometry and an adaptive basis update followed by variance-state downscaling. We characterize the directional secant matching induced by the scalar correction and give a bound on variance-state mismatch across basis changes. Across eight PDE
benchmarks, \method attains the lowest final residual on six, including Burgers and Boussinesq, while SOAP-family baselines perform better on Gray-Scott and Ginzburg-Landau. On Boussinesq, \method reaches a residual of $10^{-5}$ in 4.1 hours with 9.2 GB peak VRAM, while Adam does not reach this target within 14 hours. Three-seed $L^2$ and $H^1$ errors on four representative PDEs support the link between lower residuals and improved solution accuracy. These results position \method as a scalable option for stiff, high-accuracy physics-informed training, rather than a uniform replacement for existing optimizers.
Read the full abstract ▾
All authors Guangyuan Wang, Mads Toftrup, Sebastian Loeschcke, Yixuan Wang, Anima Anandkumar
Machine Learning
Does Latent Planning Survive Point Clouds? Action-Conditioned JEPA World Models for Geometric Observations and Goals
Fabio F. Oberweger, Michael Schwingshackl and Markus Murschitz
Latent action world models let
agents plan new behaviors at test time by predicting how actions change the environment, and joint-
embedding predictive architectures (JEPAs) do so by
forecasting future latent states rather than pixels. Yet nearly all such models see the world through a camera, even though robotic manipulation is fundamentally
geometric: in robotics goals for manipulation are traditionally specified by target object poses, not by images of the object once placed. We ask whether latent planning survives a shift from appearance to geometry, on the observation side as well as on the goal specifications side. To answer this, we extend the stable-worldmodel evaluation platform with simulated
LiDAR-style raycast point clouds as a new sensor modality, and adapt three JEPA designs to point clouds: a frozen-encoder model built on Utonia features, a distribution-prior model based on LeWM, and an action-sensitive model based on Delta-JEPA. We further introduce a goal-encoding mechanism that constructs the goal latent from the current latent and a 3D target pose, removing the need for goal images or goal point clouds. A comparative evaluation of the different anti-collapse mechanisms shows that point-cloud world models can match their image-based counterparts, demonstrating that the modality shift from appearance to geometry is achievable. All models are released as open weights with open-source training and inference code, to make world-model planning accessible for
LiDAR-driven and pose-directed robotic tasks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
Content Exploration Beyond the Feed: Creator Supply and the Shared Corpus
Yuanyuan Shen, Yiren Yan, Wenjie Li et al. (+1 more)
Industrial recommenders give new content initial views through budgeted exploration, then use early performance to decide further delivery. On many short-
video platforms, exploration is the primary way new
videos reach viewers. Viewer-side tests measure consumption; the published budget objectives we review omit creator response. We analyze four experiments on a major short-
video platform. An eight-month creator ablation finds production exploration raises
videos posted per creator by 8.55% and creators posting at least once by 7.10% relative to a minimal floor. A budget-matched reallocation raises creator participation with no detectable short-run viewer-side change. A year-long viewer ablation finds 1.74% more
video views but 2.13% less view time. A delivered view creates immediate feed value, can trigger organic take-up, and can induce creator supply. Take-up and supply replenish a shared corpus, creating two measurement limits. Viewer-side A/B tests cancel the corpus effect when both arms consume the same corpus. Giving each arm its own corpus avoids cancellation, but turnover still controls the horizon. If the corpus turns over at rate w per posting cycle, a t-cycle experiment expresses at most wt of the eventual corpus effect. More users reduce noise but do not speed turnover. Before the corpus path visibly bends, data cannot distinguish a modest fast effect from an arbitrarily large slow one, so a valid confidence interval may lack a finite upper endpoint. As predicted, the three-week co-diverted experiment cannot determine the sign of the eventual corpus effect. Within the window, it identifies the direct feed effect, and an exploratory cohort analysis detects organic lift after exploration ends. The experiments establish a positive creator response, measure the gross corpus flow visible within three weeks, and show the design and duration needed to identify total value.
Read the full abstract ▾
All authors Yuanyuan Shen, Yiren Yan, Wenjie Li, Chunhui Zhu
Machine Learning
Behavioral Latency as Weak Event-Time Supervision for EEG Reaction-Time Decoding
Anuar Aimoldin, Ayana Mussabayeva, Yedige Mussabayev et al. (+2 more)
Single-trial
EEG analyses are often organized around events and latencies, yet
EEG-based reaction-time (RT) prediction is posed as scalar regression on a fixed stimulus-locked window. RT is treated as a window-level label rather than timing evidence about response-relevant dynamics. Here we reformulate trial-wise RT decoding as event-time posterior modeling. Instead of predicting RT directly, the model estimates a posterior over response-relevant event times, $p(t_{\mathrm{event}}\mid X)$, and uses its mean as the RT estimate. This treats behavioral latency as a weak observation of latent response-relevant timing. We evaluate this formulation on the Healthy
Brain Network contrast change detection
EEG task under a subject-disjoint, release-separated protocol. Across five seeds, distributional event-time supervision consistently improves held-out RT prediction relative to scalar regression and temporal-readout controls. Controlled objective comparisons isolate supervision of the event-time distribution, rather than expectation-based readout alone, as the source of this gain. Architecture controls show that the effect persists across four temporal backbones and is not explained by model scale. Beyond point prediction, posterior geometry characterizes concentration, target
alignment, and interval behavior, while observation-noise calibration separates latent concentration from predictive uncertainty over RT. Shifted-crop inference probes shortcut use versus temporal localization. Matched shift-jitter improves robustness, increases mean sensitivity, and moves predictions more often in the expected crop-relative direction. Sensitivity remains below ideal crop-relative localization, leaving a clear equivariance gap. Together, these results establish event-time posterior modeling as a probabilistic and interpretable formulation for linking single-trial
EEG dynamics to behavioral timing.
Read the full abstract ▾
All authors Anuar Aimoldin, Ayana Mussabayeva, Yedige Mussabayev, Xue Liu, Kun Zhang
Machine Learning · Cross-listed from cs.CV
Polis: 3D Self-Supervision at City Scale
Alexander Rusnak, Sophia Kovalenko, Jingru Wang et al. (+3 more)
Reliable semantic representations derived from city-scale 3D models are increasingly important for urban analysis, infrastructure monitoring, autonomous systems, and heritage conservation. However, urban scenes of large spatial extent captured through aerial surveying differ substantially from the indoor, object-level, and self-driving
LiDAR data used to pretrain most 3D self-supervised models. We introduce Polis, to our knowledge the first application of Sketched Isotropic Gaussian Regularization (SIGReg) as an objective for a native point cloud encoder, and evaluate it through a frozen-feature
benchmark spanning fourteen city- and building-scale corpora. Polis combines geometrically matched cosine invariance, SIGReg, and VICReg-style anti-collapse terms with a 12.8k-scene outdoor pretraining mixture and
gravity-preserving spatial view sampling. Controlled ablations show that this objective outperforms student–teacher architecture alternatives, as well as Polis versions without anti-collapse terms, on the same representative outdoor corpus. On three pretraining-disjoint city
datasets, Polis reaches $23.8\%$ mean mIoU versus $16.3\%$ for the next-best encoder under high-capacity frozen probing, and $17.3\%$ versus $16.1\%$ at a matched point and voxel budget. The same city-scale lead holds on
datasets whose training sets were seen in pretraining. On localized terrestrial captures with fine-grained facade and streetscape labels, the ranking reverses. Our results show that distributionally-regularized joint
embedding architectures can be successful on challenging city-scale 3D scenes, and that transfer improves when self-supervision is designed for the capture geometry and spatial context of this domain while also revealing the limits of this specialization.
Read the full abstract ▾
All authors Alexander Rusnak, Sophia Kovalenko, Jingru Wang, Ismail Moudden, Xiru Wang, Frédéric Kaplan
Machine Learning
One Capability or Many? Structural and Predictive Tests of Benchmark Validity Disagree About Economic Benchmarks for Frontier AI
Louis Yiven Zhu
Frontier-model leaderboards now rank systems on economic
benchmarks, and those rankings inform what organisations buy and what regulators scrutinise. Whether such
benchmarks measure a capability distinct from general test-taking is a question of construct validity that a structural test and a predictive test can answer in opposite ways. We show that they do on a hash-pinned snapshot of a frontier leaderboard with 421 model configurations across twelve
benchmarks, four of them economic, of which 103 configurations carry all three sparsely scored economic
benchmarks and 96 carry all twelve; four hypotheses and their thresholds were fixed before analysis, and every deviation from the plan is reported. The first factor of a three-factor extraction carries 74.5% of common variance and tracks release date (R^2 = 0.505), and date adjustment lowers its share by 14.9 points. Under the dimensionality rule fixed in advance the economic
benchmarks form no factor of their own. A leave-one-
benchmark-out test with factors re-estimated inside every fold nevertheless finds that a multi-factor representation predicts held-out economic scores better than a single general index (pooled Delta-MSE 0.037, 95% bootstrap interval [0.019, 0.055]) under the linear learners that fit best, an advantage that reverses for tree learners. Under the linear learners the same representation also predicts the eight other
benchmarks better, so the battery carries predictive structure that one index misses and the economic
benchmarks share it without forming a distinct factor. Construct validity should therefore be assessed by predictive tests alongside structural ones. We give a two-test protocol for
benchmark builders and release the pinned data, the analysis plan and the code.
Read the full abstract ▾
Machine Learning
Scalable Clinical Data Infrastructure and Comparative ML Evaluation for Hospitalisation Risk Prediction in Elderly Patients with Multiple Long-Term Conditions using CPRD
Asra Aslam, Volodymyr Chapman, Maurice M. O'Connell et al. (+9 more)
Deep learning architectures are increasingly proposed for patient trajectory modeling in electronic health records (EHRs), yet their advantage over simpler, more interpretable models is rarely subjected to rigorous empirical scrutiny in real-world clinical settings. We present a comprehensive patient timeline pipeline applied to elderly patients in CPRD Aurum, incorporating 260 clinical conditions classified via a three-tier automated framework including specialised detection logic for 17 complex conditions. Using this infrastructure, we
benchmark Temporal Graph Convolutional Neural Networks (TG-CNN) against Logistic Regression with LASSO regularisation and Random Forests for predicting 12-month all-cause emergency hospitalisation risk, motivated by (but not filtered to) the elevated risk of adverse drug reactions. Under cross-validation, TG-CNN achieves a marginally higher mean AUC-ROC than LASSO (0.712 vs. 0.705), whereas on the held-out test set LASSO achieves the highest discrimination of three models (AUC-ROC 0.733, versus 0.710 for Random Forest and 0.702 for TG-CNN). We show, that discrimination alone is an incomplete criterion for clinical deployment: after Platt calibration, LASSO is the only model with an acceptable calibration slope (0.817), while Random Forest (0.759) and, TG-CNN (0.391) remain substantially miscalibrated. We argue that LASSO, not the highest-discriminating model, is the model best suited to direct clinical deployment. We present lessons for the machine learning and healthcare community regarding data infrastructure, model selection, and value of calibration and interpretability in high-stakes decision support.
Read the full abstract ▾
All authors Asra Aslam, Volodymyr Chapman, Maurice M. O'Connell, Aseel S. Abuzour, Michael Abaho, Danushka Bollegala, Gary Leeming, Eduard Shantsila, Andrew Clegg, Lauren E. Walker, Iain Edward Buchan, Samuel D. Relton
Machine Learning
Where Induction Runs Out: Description-Length Difficulty and the Memorisation Gap in Integer-Sequence Benchmarks
Sabilashan Ganeshan
Integer sequences from the On-Line Encyclopedia of Integer Sequences (OEIS) are increasingly used to
benchmark mathematical
reasoning in
language models. We ask what such
benchmarks actually measure, using an exactly computable reference learner: two-part minimum description length (MDL) over the class of P-recursive (holonomic) recurrences, evaluated on every prefix of a sequence as terms arrive. Three findings follow. First, MDL difficulty is a parameter count. The discovery point nd, the first prefix length at which a symbolic hypothesis beats verbatim storage, is predicted almost exactly by a combinatorial identifiability bound on the selected operator's order and degree. It is invariant to term magnitude: scaling Fibonacci over twelve orders of magnitude leaves nd unchanged, because a hypothesis must encode its own initial conditions and the magnitude cancels. Second, at scale the learner exhibits a regime our curated corpus could not produce even once: across 20,000 OEIS sequences, 89.98% of those that fit a recurrence on some prefix fit none at full length. We call this the wilderness – induction acquires a theory, loses it, and never recovers. Third, evaluating three
language models on sequences stratified by these MDL regimes refuted our pre-registered hypothesis: models do not confabulate where MDL reports no theory, but hedge appropriately. Confident errors are inverted, concentrating on the easy stratum, where apparent competence tracks recognition of the sequence rather than induction of its rule. OEIS-derived
benchmarks therefore substantially measure memorisation, and MDL supplies a cheap, contamination-free difficulty signal they currently lack. Code and data are released.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
An Integrated Vision-and-Language Pretraining (VLP) and Visual Question Answering (VQA) model to Automate Nondestructive Evaluation Image Analysis
Mehrdad Shafiei Dizaji and Hoda Azari
An AI-based approach called ChatNDE Figure to Caption is introduced, which aims to automate the interpretation of NDE images using deep learning and natural
language processing (NLP). A
Vision-and-
Language Pretraining (VLP) strategy is developed to help the model learn how to connect visual features with meaningful
language. Basically, we built a large NDE image
dataset, trained the model using annotated examples, and then evaluated how well it performed using BLEU scores to compare its output to expert written descriptions. So, the system combines a ResNet50 model to extract important features from the images and a GPT2
language model to turn those features into natural sounding text. Even though the accuracy of the model has been low the generated caption results have been solid so far, the captions were shorter but mentioned some important features of images what human experts would say, which shows the model is learning to pick up on key details. Also, a Visual Question Answering (VQA) model is used as part of the system. VQA models are designed to take an image and a question about that image (like Is there a crack? or Where is the defect located?) and generate a useful answer. By adding this layer, the platform will not just describe what it sees, it can also respond to specific questions, making it even more interactive and helpful for inspectors in the field. This whole approach is a big step toward speeding up NDE workflows, reducing human error, and making the technology more accessible.
Read the full abstract ▾
Machine Learning · Cross-listed from math.HO
Explanations, Prompts, and Formalizations: Arguments for New Norms in LLM-Enabled Mathematical Research
Axel Boldt
As several mathematical
conjectures have recently been settled using large
language models (LLMs), the mathematical community has formulated norms and recommendations regarding the publishing of such results. These norms do not cover the disclosure of the
prompts and precise software setup used to obtain those results, nor do they require that results be formalized in a manner that allows for machine verification. I argue that both of these are essential. In addition, since LLM-obtained results may be hard to understand, human authors have the responsibility to invent intuitive explanations.
Read the full abstract ▾
Machine Learning
Fully Distributed GNE Algorithms for Multi-Robot Placement without Consensus on Multipliers
Shao-An Yin, Mingyi Hong and Nicola Elia
Recent machine learning research has increasingly focused on
equilibrium analysis in non-cooperative games rather than solely on optimal solutions. Many such problems involve shared constraints and can be formulated as Generalized Nash
Equilibrium Problems (GNEPs). For strongly monotone games, existing methods compute consensus-based variational GNEs (v-GNEs) by exchanging Lagrange multipliers. We propose a fully distributed continuous-time
algorithm for shared linear equality constraints that converges without multiplier exchange and reaches any GNE, reducing communication overhead and improving
privacy. Discrete-time schemes are also provided, and the method is validated on a multi-
robot placement task.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
FiLM-GPNet: Geometry-Aware Pseudo-Supervised Phase Restoration with Zero-Shot Generalization for Large Temporal InSAR Stacks
Getnet Demil, Muhammad Farhan Humayun, Tomi Westerlund et al. (+2 more)
The growing availability of dense commercial Synthetic Aperture Radar (SAR) time series enables temporal Interferometric SAR (InSAR) analysis, but fixed classical filters fail under heterogeneous acquisition geometries, degrading phase quality and temporal consistency. We propose FiLM-GPNet, a geometry-conditioned network for wrapped-phase restoration that explicitly adapts to acquisition differences using Feature-wise Linear Modulation (FiLM) and a 7D per-pair geometry descriptor. The model is trained with pseudo-supervision from Goldstein-filtered interferograms and regularized by interferometric physics via triplet-closure consistency, while also estimating per-pixel aleatoric uncertainty. Experiments on three Capella Spotlight stacks from the IEEE GRSS 2026 Data Fusion Contest show that FiLM-GPNet reduces temporal residual by 68% (Hawaii) and 66% (Western Australia) relative to the Goldstein baseline, alongside closure error reductions of 10% and 13%, respectively. In Western Australia, it further improves unwrapping success rate by 7.7 percentage points and Digital Elevation Model (DEM) Normalized Median Absolute Deviation (NMAD) by 31%. The model also shows strong zero-shot generalization to a geographically and geometrically distinct third stack (Los Angeles) without retraining, supporting geometry-conditioned restoration as an effective alternative to fixed classical filtering across heterogeneous stacks.
Read the full abstract ▾
All authors Getnet Demil, Muhammad Farhan Humayun, Tomi Westerlund, Jukka Heikkonen, Mourad Oussalah
Machine Learning · Cross-listed from cs.AI
Evaluating Tiny Recursive Models Across Training for Code Generation
Anjani Sirivella, Aanisha Newaz and Glaucia Melo
Code generation increasingly relies on large
transformer models, whose capability advances with scale. Yet such a scale is costly, creating demand for small models, especially where data is limited. Recursive models address this by reusing a single block to add depth rather than stacking independent layers. Such models are typically evaluated by teacher-forced fit (next-
token loss on ground-truth prefixes) or task accuracy, at a single checkpoint, whereas code is produced by free-running generation, where the model extends its own output. Whether a teacher-forced advantage survives free-running generation, and whether it holds across training, remains open. To study both, we compare a ~28M-parameter autoregressive Tiny Recursive Model (TRM-AR) on natural-
language-to-Python code generation against parameter-matched and depth-matched controls, tracking fit and generation across 40 epochs and three seeds. The fit ranking between the recursive model and the depth-matched control reverses twice. Selecting each checkpoint by validation loss and examining the trajectory yields a consistent comparison. At equal parameters, TRM-AR fits, generates, and generalizes better than the parameter-matched control while recovering approximately 45% of the validation-loss gap and 57% of the generation-quality gap between the two controls, at roughly 175 times the per-step cost of the parameter-matched control. However, at equal effective depth, the larger
transformer fits and generates better at its validation optimum, suggesting TRM-AR's advantage lies in resistance to overfitting, not greater capability. These findings suggest that recursive code generation models should be evaluated jointly on fit and generation across the training trajectory rather than at a single checkpoint.
Read the full abstract ▾
Machine Learning
Unlearning on Spatio-Temporal Graphs through Subgraph Virtual Edge Reconstruction
Qiming Guo, Wenbo Sun, Chen Pan et al. (+2 more)
Spatio-temporal graphs are widely used in modeling complex dynamic processes such as temporal
forecasting, molecular dynamics, and healthcare monitoring. Recently, stringent
privacy regulations such as GDPR and CCPA have introduced significant new challenges for existing spatio-temporal graph models, requiring complete unlearning of unauthorized data. Since each node in a spatio-temporal graph diffuses information globally across both spatial and temporal dimensions, existing unlearning methods primarily designed for static graphs and localized data removal cannot efficiently erase a single node without incurring costs nearly equivalent to full model retraining. To address this, we propose CallosumNet, a spatio-temporal graph unlearning framework biologically inspired by the corpus callosum structure. CallosumNet makes two key technical contributions: (1) it reconstructs subgraphs using biologically-inspired virtual edges; and (2) it restores interlinked spatio-temporal dependencies among subgraphs via a lightweight meta-graph integration layer. Empirical results on four diverse real-world
datasets show that CallosumNet achieves complete unlearning while maintaining accuracy very close to the gold model. The code is publicly available at https://github.com/wenlu-lab/STGraphUnlearning.
Read the full abstract ▾
All authors Qiming Guo, Wenbo Sun, Chen Pan, Ye Wang, Wenlu Wang
Machine Learning · Cross-listed from cs.PF
Spectral Analysis for Sparse Matrix Computation: Insights and Potential
Ruifeng Zhang and Xipeng Shen
Sparse computations are fundamental to scientific computing, graph analytics, and machine learning, yet their performance is highly sensitive to the diverse sparsity and patterns. This is because cache reuse,
memory coalescing, and load balancing depend critically on the sparsity patterns. This work gives the first known exploration of the connections between sparse
matrix computation and spectral analysis by treating sparse
matrices as two-dimensional signals and analyzing their frequency-domain representations through Fast
Fourier Transform. We show that spectral signatures uncover global structural characteristics that are not sufficiently captured by conventional spatial statistics and provide complementary information for understanding sparse computation performance. Experiments on incorporating spectral features into machine-learning-based SpMV format selection demonstrate the usefulness of such spectral analysis over a state-of-the-art spatial-only model. By uncovering the principled connections between spectral characteristics and sparse
matrix computations, this work introduces a novel analytical perspective into sparse computation, and provides a new approach to enhancing the current sparse structure characterization and optimization. On pruned LLM decoding, adding spectral features improves kernel selection and yields 1.035–1.245$\times$ kernel speedups.
Read the full abstract ▾
Machine Learning
Spatial Entropy based Partitioning for Spatiotemporal Graph Unlearning
Qiming Guo, Wenbo Sun, Ye Wang et al. (+1 more)
Spatiotemporal graphs underpin applications such as traffic
forecasting, weather
forecasting, and healthcare monitoring.
Privacy regulations such as the GDPR and the CCPA require the complete removal of unauthorized data from trained models, but achieving this on a spatiotemporal graph is difficult: because information propagates globally through both spatial and temporal message passing, fully erasing a node's influence forces costly full-graph retraining. ST-graph unlearning requires both exactness and efficiency. We propose IsleNet, which uses spatial-
entropy-guided partitioning to create balanced, locally coherent subgraphs and reconnects them with lightweight virtual edges. Upon an unlearning request, only the affected subgraph encoder and virtual-edge layer are retrained, ensuring exact removal with low cost. Experiments on four real-world
benchmarks show that IsleNet attains up to 94% of full-graph accuracy while reducing unlearning time by up to an order of magnitude. Our code is publicly available at https://github.com/wenlu-lab/STGraphUnlearning.
Read the full abstract ▾
All authors Qiming Guo, Wenbo Sun, Ye Wang, Wenlu Wang
Machine Learning · Cross-listed from cs.AI
APPSolver: Adaptive Patch Partitioning for Point-Wise Ship Flow Prediction on Unstructured Meshes
Wenhua Huo, Fenglei Han, Wangyuan Zhao et al. (+4 more)
Large non-uniform point sets make direct
attention-based surrogate modeling costly for ship hydrodynamics. We introduce APPSolver, a point-wise flow-prediction framework built around Adaptive Patch Partitioning (APP), a deterministic quadtree representation for fixed two-dimensional horizontal slices extracted from ship CFD simulations. APP assigns finer patches near the hull and coarser patches farther away, downsamples patch contents, and recovers predictions to the full reference point set. Under a corrected protocol that constructs natural $(t,t+1)$ pairs before splitting, reuses training-set normalization statistics, and reports three model seeds, learned tokenizers are more accurate than APP-
Transformer, and a persistence baseline has lower one-step MAE on all three ShipBench hulls. The supported benefit of APP is therefore computational rather than universal predictive superiority: on a representative DTC input, APP-
Transformer requires 1.815 GFLOPs and 1.309 ms per model forward, while a matched ablation shows that adaptive partitioning reduces MAE by 16.4-24.9% relative to a uniform partition augmented with learned slicing. Condition encoders provide setting-dependent gains in leave-one-hull-out evaluation, but the current absolute next-state objective does not establish accurate long-horizon dynamics. These results characterize APP as a compact spatial representation with an explicit accuracy–efficiency trade-off. Code is available at https://github.com/wenhuahuo/APPSolver .
Read the full abstract ▾
All authors Wenhua Huo, Fenglei Han, Wangyuan Zhao, Xiao Peng, Chunhui Wang, Jialin Wu, Jiayi Han
Machine Learning
Information-Based Calibration of Uncertainty Quantification in Product-of-Experts Gaussian Process Models
Yean Hoon Ong, Paolo Barucca, Wei Pan et al. (+1 more)
Gaussian process (GP) regression with a single global GP (GP-glo) incurs cubic computational cost, limiting scalability to large
datasets. Product-of-experts GP models (GP-pro), which combine local GP models to capture global correlations, alleviate this computational burden. However, training local experts on disjoint data subsets can lead to overestimated posterior variances. We propose GP-pro-c, a product-of-experts GP model that calibrates these variances using an information-based method. The method exploits the monotonicity and submodularity of information gain in GPs to define a calibration ratio that reduces the posterior variance of individual local GP models. We evaluate GP-pro-c using negative log-likelihood (NLL), root mean squared error (RMSE), and expected normalised calibration error (ENCE). Experiments on four synthetic functions and six regression
datasets show that GP-pro-c achieves average reductions of 2.3% in NLL and 12.0% in ENCE compared with the uncalibrated GP-pro model. The proposed method mitigates posterior variance overestimation while maintaining predictive accuracy and reducing computational complexity. GP-pro-c provides a promising approach for uncertainty estimation in scalable GP models and may serve as a useful surrogate model for
Bayesian optimisation with high-dimensional and large-scale data.
Read the full abstract ▾
All authors Yean Hoon Ong, Paolo Barucca, Wei Pan, Jun Wang
Machine Learning · Cross-listed from cs.CL
Learning Simple Test-Time Environments for LLM Web Agents
Junxuan Li, Zijun Liu, Ziyi Huang et al. (+4 more)
Large
language model (LLM)
agents have demonstrated remarkable proficiency in manually constructed environments, yet their performance frequently collapses when transitioned to complex real-world settings. Existing research largely attribute this degradation to the compositional generalization gaps in LLMs on combinations of multiple simple, well-structured environments. In this work, we propose that LLM web
agents can learn simple environment observations at test time. Specifically, we introduce trial steps for
agents to decompose a complex environment observation into sub-modules, and implement a label-free learning method, Test-Time Environment Decomposition (TTED), to adapt
agent behaviors with experience during inference. Our empirical evaluations demonstrate the framework's efficacy across both synthetic and realistic
benchmarks, showing (1) experience gains acquired within simpler sub-environments can be effectively composed to improve performance in the full one, and (2) test-time training on sub-environments can significantly enhance the compositional generalization of
agents in real-world web automation tasks. We also provide key insights in the design of the label-free learning
algorithm. As more complex environments are accessed by LLM
agents, we believe learning environment decomposition skills at test time will be critical for robust real-world deployment.
Read the full abstract ▾
All authors Junxuan Li, Zijun Liu, Ziyi Huang, Peng Li, Yuzhou Liu, Ming Yan, Yang Liu
Machine Learning
MEL: Coordinate-Preserving EEG Tokenization for fMRI Translation
Xiangyu Liu, Zeting Yan, Zhitong Yin et al. (+2 more)
Translating electroencephalography (
EEG) into functional
magnetic resonance imaging (fMRI) is important for medical neuroimaging, clinical
brain-state monitoring, and
multimodal neural decoding, because it aims to infer spatially organized hemodynamic activity from fast and accessible electrophysiological recordings. Existing
EEG-to-fMRI studies mainly pursue stronger decoders, but the problem is also constrained by a representation-interface mismatch: fMRI responses are delayed, temporally integrated, and spatially distributed, whereas generic
EEG encodings often entangle temporal lag, channel identity, and frequency-band structure. We propose Multi-band
EEG Latent-state Tokenization (MEL), a coordinate-preserving
EEG representation framework that anchors each target fMRI response to its preceding
EEG history and organizes it into lag-channel-frequency neural-state
tokens. By explicitly capturing hemodynamic latency and spectral-spatial dynamics, MEL aligns fMRI-pertinent
EEG representations with capacity-controlled readouts without depending entirely on model scaling. Experiments on VU
EEG-fMRI
benchmarks and external Oddball data show that MEL improves prediction over strong NeuroBOLT baselines. Ablations and controls further indicate that the gains come from structured
EEG representation rather than leakage, shortcut statistics, or decoder capacity.
Read the full abstract ▾
All authors Xiangyu Liu, Zeting Yan, Zhitong Yin, Boyang Li, Xi Zhang
Machine Learning
A Spectral Identifiability Threshold for Dissipative Rate Recovery from Truncated Liouvillian Spectra
Yujun Ji and Somyajit Chakraborty
Open
quantum systems lose energy and phase coherence through different dissipative processes, but these processes can produce overlapping dynamical signatures. The Liouvillian spectrum summarizes how such a system relaxes, yet it is not obvious how much of that spectrum is needed to distinguish the underlying dissipation rates. We study this question for amplitude damping and dephasing in a six-
qubit Lindblad model whose spectrum can be derived analytically. We retain only the slowest non-steady spectral modes and ask how many are required before each dissipative rate becomes recoverable. We show that population modes contain no dephasing information, which creates a lower bound of D = 2^n retained modes for uniform dephasing identifiability in the relevant rate regime. The measured recovery threshold reaches this bound at n = 4,5,6, while n = 3 remains above it. At n = 6, least squares achieves a mean joint absolute error of order 10^-9, compared with 4.355 x 10^-4 for four tabular learning methods. Robustness tests show that this advantage weakens when the spectra are perturbed and when a transverse field breaks the commuting structure. These results show that the amount and structure of retained spectral information can determine whether dissipative parameters are recoverable, independently of the
estimator used. The present conclusions apply to noise-free simulator spectra rather than measurement-derived spectra.
Read the full abstract ▾
Machine Learning
When Do Larger Batches Help Scale LLM Reinforcement Learning?
Ziniu Li, Jinbo Wang, Guanhua Huang et al. (+3 more)
Larger batches reduce the variance of
stochastic gradients per update and are therefore often expected to accelerate training. Yet whether this statistical benefit translates into lower wall-clock time-to-target remains unclear, because each update consumes more samples and may take longer to execute. We study this tradeoff in reinforcement learning for large
language models. We separate its algorithmic and systems effects by comparing learning and execution along their natural axes. At the algorithmic level, we compare configurations at equal cumulative sample counts while retuning batch-dependent hyperparameters. Over a bounded range of batch sizes, this procedure yields an approximately batch-size-invariant family whose members follow similar sample-indexed learning trajectories. At the systems level, we exploit the computational asymmetry between rollout generation and training: autoregressive generation is often
memory-bandwidth-bound at low concurrency, whereas training work scales approximately with the number of processed
tokens. Combining these two views yields a direct decision rule: a larger-batch configuration reduces time-to-target only when its throughput gain exceeds its samples-to-target penalty. Experiments with GRPO and PPO support both sides of this decomposition. At the algorithmic level, square-root learning-rate scaling with Adam produces approximately batch-size-invariant learning curves over a bounded range of batch sizes. At the systems level, larger batches improve generation throughput by up to 2.29x on fixed hardware. In GRPO, combining higher throughput with learning-rate retuning reduces time-to-target by up to 29%, whereas increasing the batch without retuning is slower despite its higher throughput.
Read the full abstract ▾
All authors Ziniu Li, Jinbo Wang, Guanhua Huang, Feiyuan Zhang, Pengbo Li, Alex Chen
Machine Learning · Cross-listed from stat.CO
Signed random Fourier features for fast density estimation with indefinite kernels
Xie Wang, Nicolas Langrené and Wen Chen
Kernel density estimation (KDE) is one of the most fundamental statistical
estimators of density functions. Its direct implementation on a
dataset of $N$ points incurs an $\mathcal{O}(N^{2})$ computational cost, which is prohibitive for large-scale
datasets. Kernel approximation techniques can be applied to bring the computational cost down to $\mathcal{O}(N)$. The random
Fourier features (RFF) technique, based on sampling from the spectral density of the kernel function, has become popular to speed up kernel
estimators for machine learning applications. Unfortunately, it is restricted to positive definite kernels, while the majority of kernel functions popular in KDE, such as the
parabolic kernel, do not satisfy this property. To overcome this limitation, this article introduces the signed random
Fourier features (SRFF) technique. It is a generalization of RFF compatible with indefinite kernels whose inverse
Fourier transform is absolutely integrable. The motivation for introducing this method is to speed up KDE in the case of multivariate compact kernels, which are generally not positive definite. We detail how to implement SRFF for both product kernels and isotropic kernels. For the class of Kuttner-Golubov kernels $K(\boldsymbol{x}_{i},\boldsymbol{x}_{j})=(1-\left\Vert \boldsymbol{x}_{i}-\boldsymbol{x}_{j}\right\Vert ^{\alpha})^{\beta}\mathbf{1}_{\{\left\Vert \boldsymbol{x}_{i}-\boldsymbol{x}_{j}\right\Vert \leq1\}}$ where $\boldsymbol{x}_{i}\in\mathbb{R}^{d}$, $\boldsymbol{x}_{j}\in\mathbb{R}^{d}$, $\alpha>0$, $\beta>0$, which includes the triangular,
parabolic, biweight, triweight, and other kernel functions of interest for KDE as particular examples, we provide an explicit acceptance-rejection
algorithm to sample from its signed spectral density. Our numerical tests on a
dataset of one million points confirm the computational efficiency and accuracy of SRFF for large-scale KDE.
Read the full abstract ▾
Machine Learning
Adaptive Multi-Branching for Shallow Decision Tree Induction
Hanul Park, Jeonghoon Choi, Juseong Kim et al. (+2 more)
Decision trees are attractive for tabular prediction tasks because each prediction follows an interpretable sequence of feature-threshold tests. Under a strict maximum-depth budget, however, conventional binary trees can be under-expressive, since each internal node makes only a single threshold decision. We study shallow-depth tree induction, where the goal is to improve accuracy while keeping root-to-leaf paths short. We propose the Multi-Branch Neural Decision Tree with Adaptive
Pruning (MBNDT), a single axis-aligned tree trained end-to-end with differentiable multi-way splits. Each internal node learns ordered thresholds over a selected feature and a branch mask that adapts its effective arity, and the trained model is converted to a deterministic single-path tree for inference. Across 21 OpenML binary-classification
benchmarks, MBNDT achieves the best average rank and mean balanced accuracy among depth-constrained single-tree baselines; a controlled ablation isolates multi-way splitting as the source of the gain. These gains come with an explicit trade-off: MBNDT realizes more leaves than the other single-tree baselines, making it best suited when accuracy under short, bounded decision paths is prioritized over minimal global tree size.
Read the full abstract ▾
All authors Hanul Park, Jeonghoon Choi, Juseong Kim, Sanghun Sel, Giltae Song
Machine Learning · Cross-listed from cs.NI
Sense Once, Serve Many: Common-Trace Factorized Constrained PPO for Online Sensing-Session Consolidation in Multi-Tenant ISAC Networks
Dang-Dung Vu
Integrated sensing and communication (
ISAC) networks can serve compatible requests through shared sensing sessions, but consolidation couples admission, reuse, profile selection, sensing service-level agreements (SLAs), communication quality of service (QoS), and future commitments. We formulate this problem as a constrained Markov decision process and propose Common-Trace Factorized Constrained Proximal Policy Optimization (CT-PPO). During training,
stochastic policy replicas share the same primitive workload trace; leave-one-out discounted Monte Carlo return contrasts provide reward credit to applicable actor factors, while constraint credit remains factor/prefix-specific. Across five training seeds and matched workloads, CT-PPO achieves the highest mean macro return, exceeding matched Joint-Credit PPO (JC-PPO) by 0.934 (95% confidence interval [0.702, 1.164]) and SLA-Aware Greedy by 1.847; versus JC-PPO, it reduces sensing-resource cost by 6.277 and raises accepted requests per created session by 0.0806. A four-way ablation shows that the factorized surrogate alone yields no detectable macro-return gain, whereas adding common-trace reward credit produces the dominant improvement. Without retraining, CT-PPO retains a return advantage at low, nominal, and high arrival loads, with the strongest gain under clustered arrivals. Deployment uses public observations and hard masks; CT-PPO's extra parameters are training-side, its actor footprint matches JC-PPO, and actor-only CPU latency is effectively unchanged.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NI
A-MADiff: Attention-Guided Multi-Agent DRL with Diffusion Policies for Memory-Aware Task Orchestration in Mobile AIGC Networks
Chongzhi Wu, Zhengtao Li, Jiawen Kang et al. (+4 more)
Artificial Intelligence-Generated Content (AIGC) services employ Generative AI (GenAI) models to automatically generate diverse content. Mobile AIGC networks host GenAI models on edge-located AIGC Service Providers (ASPs) to deliver low-latency and personalized AIGC services for mobile users. However, AIGC inference tasks typically occupy GPU
memory until task completion, causing GPU
memory exhaustion at serving ASPs and triggering out-of-
memory failures rather than merely increasing service latency. Existing studies on AIGC task orchestration have largely overlooked GPU
memory feasibility constraints. To address this issue, we develop a cooperative multi-
agent orchestration framework, in which each edge node is equipped with a scheduling
agent to route tasks to local ASPs or neighboring edge nodes. Since scheduling
agents make decisions based only on local observations, while peer offloading couples their resource states and long-term utilities, we formulate the orchestration process as a cooperative Decentralized Partially Observable Markov Decision Process (Dec-POMDP). To solve the Dec-POMDP, we propose an \underline{A}ttention-guided \underline{M}ulti-\underline{A}gent deep reinforcement learning
algorithm with \underline{Diff}usion policies (A-MADiff) under the centralized training with a decentralized execution paradigm. A-MADiff employs
diffusion-based decentralized actors to generate multi-modal preferences over feasible orchestration actions, and an
attention-guided centralized critic to estimate per-
agent values from cross-
agent states under GPU
memory heterogeneity. Numerical results demonstrate that A-MADiff significantly improves the cumulative reward over the state-of-the-art baseline.
Read the full abstract ▾
All authors Chongzhi Wu, Zhengtao Li, Jiawen Kang, Jinbo Wen, Xiaohuan Li, Maomao Zhang, Ekram Hossain
Machine Learning · Cross-listed from cs.CV
QCell: Recombining and Aligning Cell Queries for Overlapping Instance Segmentation
Yaroslav Prytula, Anton Popov and Dmytro Fishman
Instance
segmentation of overlapping
cells in microscopy remains challenging due to semi-transparent structures that produce weak boundaries and mixed visual evidence in overlap regions. Existing methods address this through local regions of interest or shape priors but lack global
reasoning across overlapping objects. We present QCell, a novel query-based model that de-overlaps
cell instances in microscopy scenes. Our approach combines (i) an instance recombination module that decomposes and recombines query representations in latent space, enabling the model to reason about complete object structure under overlap, and (ii) a contrastive query
alignment objective that combines distinctive instance feature learning and separation of overlapping
cell queries. We additionally introduce a new Organoid
dataset benchmark for overlapping
cell segmentation. We show that QCell outperforms state-of-the-art methods across multiple
benchmarks, achieving +2.2 AP and +2.7 AJI on ISBI2014. Code is available at https://github.com/SlavkoPrytula/QCell
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Validating FKG.in: Soundness Assessment in LLM-Augmented Indian Food Knowledge
Saransh Kumar Gupta, Armaan Shah, Lipika Dey et al. (+2 more)
The online culinary ecosystem is increasingly populated by recipe content generated, modified, or summarized by Large
Language Models (LLMs). While often plausible, such outputs may contain hallucinated ingredients, misrepresented quantities, or culturally implausible combinations, limiting their suitability for downstream applications and knowledge graph construction. In this paper, we present a semi-automated soundness assessment workflow for validating structured recipe data extracted and augmented by LLMs from informal culinary sources. Developed as part of FKG(.in), a knowledge graph of Indian food, the pipeline identifies and addresses common failure modes, including structural inconsistencies, semantic and logical incoherence, and deviations from the source text, through a multi-stage process combining formal grammars, vocabulary-based checks, statistical heuristics, Set
Transformer-based coherence modeling, and
retrieval-based verification. Although evaluated on Indian recipes, the proposed methods are applicable to broader
multilingual and multicultural culinary domains. We provide a practical, auditable, and application-agnostic framework for validating LLM-augmented recipe data, thereby strengthening the foundations of machine-readable food knowledge infrastructures in the era of LLM-generated content.
Read the full abstract ▾
All authors Saransh Kumar Gupta, Armaan Shah, Lipika Dey, Partha Pratim Das, Ramesh Jain
Machine Learning
RL-FAT: Reinforcement Learning for Fair Adversarial Training
Tejaswini Medi, Levan Mikeladze and Margret Keuper
Deep neural networks remain highly vulnerable to
adversarial perturbations, and
adversarial training (AT) has become a widely used approach for improving robustness. However, improvements in average robust accuracy often mask substantial class-wise disparities: while some classes become more robust, others may remain disproportionately vulnerable under attack. This imbalance raises an important
adversarial fairness concern, particularly in
vision tasks where reliable robustness is expected across all categories. To address this challenge, we propose RL-FAT, a reinforcement-learning-inspired fair
adversarial training framework that uses policy-gradient based feedback from
adversarial predictions. RL-FAT interprets the prediction distribution as a policy and combines correctness-based prediction rewards with class-wise value estimates to compute class-specific advantages for policy-gradient optimization. This enables the model to adaptively focus on class-wise misclassification. Furthermore, we introduce a
fairness-emphasis
adversarial loss that assigns stronger training pressure to classes with high
adversarial loss, thereby mitigating class-wise robustness disparity. By combining reinforcement-driven adaptation with
fairness-emphasis regularization, RL-FAT improves
adversarial robustness while promoting a more balanced robustness distribution across classes. Extensive experiments demonstrate that our method achieves competitive robust accuracy and substantially reduces class-wise robustness imbalance compared with standard
adversarial training baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
AdaVLA: Adaptive Step Flow Matching for Training-free Acceleration of Vision-Language-Action Models
Sunghwan Han, Youngtae Han and Youngmin Yi
Vision-
Language-Action (VLA) models, built upon
Vision-
Language Models (VLMs), have significantly enhanced robotic capabilities by leveraging internet-scale knowledge and
multimodal reasoning. However, the intensive computational overhead of VLAs constrains on-device deployment, hindering real-time responses to environmental changes. While various acceleration techniques have been proposed, they often rely on fine-tuning or access to training
datasets, which are frequently unavailable due to
privacy and proprietary concerns. Moreover, although flow-matching-based VLAs have emerged as efficient alternatives to standard
diffusion models, current acceleration efforts largely target VLM inference costs, failing to address the iterative ODE solving process inherent in flow matching inference. To address these limitations, we propose AdaVLA, an online, training-free adaptive framework for fast yet accurate flow-matching-based
Vision-
Language-Action models. We introduce a novel metric derived from the flow matching trajectory
curvature to quantify action generation confidence during inference. This metric enables the dynamic reduction of inference steps and the adaptive adjustment of MLP
pruning ratios through an efficiently computed importance evaluation, requiring no access to training data. Experimental results on the LIBERO
benchmark using a Jetson AGX Orin device demonstrate that our method achieves $1.87\times$ and $2.24\times$ speedups for $\pi_{0.5}$ and X-VLA, respectively, with negligible degradation in success rates. Furthermore, we validate the robustness of our approach on real-world robotic tasks using SmolVLA.
Read the full abstract ▾