Machine Learning
Advancing Optimal Subset Oracle via Learning Relaxation of Neural Set Functions
Yongquan Shi, Zijing Ou, Shiping Wang et al. (+1 more)
Learning neural set functions is pivotal to a wide range of important applications, including compound selection in AI-driven drug discovery and product recommendation. Recent work has introduced optimal subset oracles to implicitly learn set functions under practical weakly supervised settings, where model parameters are optimized through mean-field variational inference. However, these frameworks rely on Monte Carlo sampling to estimate gradients of the evidence lower bound when updating the variational distribution. Repeated sampling across iterations incurs substantial computational overhead, while the resulting stochasticity can destabilize the optimization trajectory. In this work, we reinterpret the evidence lower bound as a continuous relaxation of the set function and learn a surrogate objective that replaces sampling-based ELBO gradient estimation during variational optimization. The learned surrogate provides stable and efficient gradients throughout the continuous domain, thereby reducing computational overhead and accelerating inference. Furthermore, we establish an approximation guarantee for the proposed framework under submodular maximization and characterize its connection to variational free energy. Experiments on a variety of real-world tasks demonstrate consistent improvements over existing baselines.
Read the full abstract ▾
All authors Yongquan Shi, Zijing Ou, Shiping Wang, Yatao Bian
Machine Learning · Cross-listed from cs.CV
Training-Free Off-Screen Player Imputation for Broadcast-Based Spatial Football Analytics
Seongjin Choi
Spatial football metrics such as pitch control assume access to the positions of all 22 players, yet the most widely available source of positional data – the broadcast main camera – shows only 10-16 of them at any moment. We quantify the resulting distortion with an open, reproducible
benchmark: a simulated broadcast viewport applied to open full-pitch tracking data (Metrica Sports; three matches, one held out from method development). Ignoring off-screen players – the visible-only baseline implied whenever a
video-based game-state-reconstruction (GSR) pipeline adds no imputation layer – inflates hidden-zone pitch-control error to 25.1-26.9 percentage points and a mean absolute control-share error of 11.1-13.4 points across the three matches. We then evaluate a ladder of training-free, online imputation baselines that use only observations from the match being analysed. The best overall on these decision-relevant metrics, role-anchored centroid voting (each visible player votes for the full-team centroid by subtracting its running role offset, attenuating the viewport-induced subset bias), roughly halves hidden-zone error (to 12.2-13.8 points) and cuts control-share error to 28-48% of the ignore policy at every viewport width from 36 m to 60 m in all three matches. For occlusions <=9.6 s – the regime of the closest learned prior work – it reaches binwise median position errors of 3.3-8.9 m; but 50-57% of hidden-player observations lie beyond that regime. Integrated end-to-end into a broadcast-
video GSR pipeline, imputation moves a downstream possession-quality score (Space-Creation Index) by 15.6 and 17.2 points on two real World Cup broadcast windows, flipping the verdict class in one.
Read the full abstract ▾
Machine Learning
Condition-Stratified Robustness Analysis of Post-Hoc Calibration Methods for Probabilistic Classifiers
Gurdeep Singh Virdee
Post-hoc calibration is widely adopted to correct probability estimates from trained classifiers, yet most evaluations report aggregate performance without testing whether that performance holds across distinct operating conditions within a single
dataset. We present a pre-registered, condition-stratified robustness analysis comparing temperature scaling (TEMP) and isotonic regression (ISO) across four controlled conditions (C1–C4). Four hypothesis groups are evaluated: discrimination deltas with Holm-corrected multiplicity control (H1), Brier score differences (H2), calibration slope outcomes (H3), and AUROC differences under best-condition setups (H4). TEMP-minus-ISO discrimination deltas remain small across all conditions (-0.0155 to 0.0139), with Holm-adjusted p-values of 0.9895 everywhere. TEMP Brier differences are consistently negative (C1: -0.0002 through C4: -0.0074), while ISO shows sign reversals. TEMP calibration slopes stay closer to unity in every condition (range 0.7597–0.9493) than ISO slopes (0.1364–0.2726). AUROC differences shift from near zero in C1 (-0.0004) to positive in C4 (0.0264). These results establish that in-
dataset robustness is condition-dependent and metric-specific. No claim of external transportability is made.
Read the full abstract ▾
Machine Learning
Random Label Prediction Heads for Studying Memorization in Deep Neural Networks
Marlon Becker, Jonas Konrad, Luis Garcia Rodriguez et al. (+1 more)
We introduce a straightforward yet effective method to empirically study memorization in deep neural networks for classification tasks. Our approach augments each training sample with auxiliary random labels, which are then predicted by a random label prediction head (RLP-head). RLP-heads can be attached at arbitrary depths of a network, predicting random labels from the corresponding intermediate representation and thereby enabling analysis of how memorization capacity evolves across layers. By interpreting the RLP-head performance as an empirical estimate of Rademacher complexity, we obtain a direct measure of both sample-level memorization and model capacity. We leverage this random label accuracy metric to analyze generalization and overfitting in different models and
datasets. Building on this approach, we further propose a novel regularization technique based on the output of the RLP-head, which demonstrably reduces memorization. Interestingly, our experiments reveal that reducing memorization can either improve or impair generalization, depending on the
dataset and training setup. These findings challenge the traditional assumption that overfitting is equivalent to memorization and suggest new hypotheses to reconcile these seemingly contradictory results. The source code is available at https://github.com/MarlonBecker/RandomLabelHeads
Read the full abstract ▾
All authors Marlon Becker, Jonas Konrad, Luis Garcia Rodriguez, Benjamin Risse
Machine Learning · Cross-listed from cs.CC
Tropical Circuits with Scalar Multiplication Gates
Christoph Hertrich and Moritz Stargalla
We study tropical
circuits with scalar multiplication gates, that is, algebraic
circuits whose gates implement $\max$, $+$, or multiplication with a positive constant. For such
circuits, we prove exponential size lower bounds for computing maximum weight directed spanning trees and maximum weight bipartite perfect matchings. As a corollary, we obtain an exponential size separation between monotone and non-monotone maxout neural networks, which generalize the popularly used ReLU neural networks. One conclusion from this is that neural network models with enforced convexity constraints, such as input-convex neural networks (ICNNs), sometimes need to be exponentially larger than their unrestricted counterparts in order to express the same functions.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Adaptive Routing for Efficient Diffusion Transformer-Based PNI Prediction
Youngung Han, Dohyun Kweon, Kyeonghun Kim et al. (+15 more)
Perineural invasion (PNI) is a critical prognostic factor in cholangiocarcinoma. However, its preoperative prediction from
magnetic resonance imaging (MRI) remains challenging due to subtle imaging features that extend beyond tumor boundaries into surrounding regions. Conventional convolutional neural networks are limited in capturing long-range spatial dependencies.
Transformer-based architectures improve global modeling of volumetric MRI by aggregating spatially distributed contextual cues, yet capturing subtle and noise-sensitive patterns in peritumoral regions remains challenging.
Diffusion-based classifiers offer an alternative formulation by leveraging denoising-based class scoring to better capture such subtle patterns. However, these approaches introduce substantial computational overhead due to the combination of
transformer-based modeling and iterative denoising processes. To address these challenges, we formulate PNI prediction as a
diffusion-based classification problem and implement the denoising network using a
transformer-based representation. To improve computational efficiency, we introduce adaptive routing across
attention heads, spatial
tokens, and MLP width. Experimental results demonstrate that the proposed approach achieves an AUC of 0.731 with 257.57 GFLOPs.
Read the full abstract ▾
All authors Youngung Han, Dohyun Kweon, Kyeonghun Kim, Hyunsu Go, Jina Jeong, Suah Park, Induk Um, Junga Kim, Anna Jung, Yului Jeong, Sungha Park, Jinyong Jun, Pa Hong, Woo Kyoung Jeong, Won Jae Lee, Ken Ying-Kai Liao, Hyuk-Jae Lee, Nam-Joon Kim
Machine Learning · Cross-listed from cs.AI
Learning Residual Kinematic Corrections for Continuous Neural Decoding via Reinforcement Learning
Jiamian Li, Niall McShane, Attila Korik et al. (+6 more)
Decoding continuous three-dimensional (3D) motor imagery (MI) using non-invasive electroencephalography (
EEG)-based
brain–computer interfaces (BCIs) remains challenging due to signal variability and residual decoding errors. Deep learning architectures such as convolutional neural network–long short-term
memory (CNN–LSTM) models can capture spatial and temporal dynamics for continuous kinematic decoding; however, systematic residual errors persist in predicted trajectories. We propose a two-stage decoding framework that applies reinforcement learning (RL) to perform residual kinematic correction on the outputs of a CNN–LSTM decoder (CNN–LSTM–RL). The RL
agent is trained offline without direct
EEG input and instead operates on predicted kinematic trajectories to optimize movement accuracy relative to target trajectories. Decoding performance was quantified using Pearson correlation coefficients ($r$) and Root Mean Square Errors (RMSE) along the $x, y$, and $z$ axes. Compared to CNN–LSTM applied alone, CNN–LSTM–RL improved the mean correlation from $0.5076$ to $0.7181$ ($p = 0.0005$) in 2D and from $0.6420$ to $0.7780$ ($p = 0.0059$) in VR, with relative gains of $41.5\%$ and $21.2\%$, respectively. Correspondingly, RMSE was reduced from $0.0890$ to $0.0532$ (2D, $p < 0.0001$) and from $0.0714$ to $0.0441$ (VR, $p < 0.0001$), representing relative reductions of $40.2\%$ and $38.2\%$. These findings demonstrate that this scalable framework enhances 3D BCI MI decoding by correcting kinematic errors via offline residual RL without extra neural data, advancing neurorehabilitation, prosthetics, and virtual interaction.
Read the full abstract ▾
All authors Jiamian Li, Niall McShane, Attila Korik, Naomi du Bois, Karl McCreadie, Leen Jabban, Benjamin Metcalfe, Özgür Şimşek, Damien Coyle
Machine Learning
AutoMatBench: An Automatic Optimization Toolkit for the Acceleration of Material Properties Prediction Benchmarking
Hongxiao Li and Wanling Gao
Material property prediction (MPP) infers key properties from chemical composition and structure, accelerating the discovery and optimization of novel materials. In the realm of MPP, MatBench is a widely accepted benchmarking tool that defines over ten significant problems and provides the paradigm of performance evaluation for AI prediction models. Even though MatBench works well in benchmarking the performances of prediction models on in-distribution (ID) tasks and
datasets, it lacks the ability to reflect their performances on out-of-distribution (OOD) material data, resulting failure in new material discovery. By combining the pipelines of MatBench and the existing researches on OOD performance evaluation, this study enables a huge space of benchmarking configurations, comprehensively reflecting the performances, abilities, and disadvantages of various AI prediction models. This work reports that the discrepancy of performances at different configuration values is huge and can be illustrated with prior knowledge and novel insights, therefore consideration of causal effect of configurations on performance results is necessary. In case of the impossibility of enumerative benchmarking at every configuration, this work further proposes AutoMatBench, an automatic toolkit with
Bayesian optimization. Experiments with AutoMatBench reports that, within twelve steps of optimization, the similar results with MatBench and former OOD research can be accessed while more than half of the cost are saved. Besides, this tool also yields more essential findings on MPP benchmarking, positively contributing to the cost and efficiency of new material discovery.
Read the full abstract ▾
Machine Learning
DAG-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms
Yikang Chen, Zhengkang Guan, Haoyuan Qian et al. (+5 more)
Causal discovery from observational tabular data remains fundamentally challenging, primarily due to the heterogeneity of underlying causal mechanisms and the high-dimensional combinatorial search space of Directed Acyclic Graphs (DAGs). In this paper, we propose DAG-FM, a novel foundation model architecture that amortizes causal discovery. Unlike direct
matrix prediction, DAG-FM decomposes the causal discovery process into two auto-regressive stages using two specialized
Transformer-based sub-modules: a leaf-node predictor and a parent-node predictor. To effectively model complex row-column interactions, we adopt a robust tabular interaction block to output feature-wise representations. Crucially, to handle diverse and unknown Functional Causal Model (FCM) assumptions in real-world scenarios, we introduce Mixture-of-Leaf-Experts (MoLE), allowing the model to dynamically route and adapt to identifiable mechanism families. Through an iterative inference
algorithm, DAG-FM seamlessly extracts causal orderings and constructs valid DAGs. Extensive experiments demonstrate that DAG-FM achieves state-of-the-art performance on both synthetic
benchmarks and complex real-world
datasets, significantly outperforming traditional classical
algorithms and recent foundation models in both accuracy and scalability.
Read the full abstract ▾
All authors Yikang Chen, Zhengkang Guan, Haoyuan Qian, Xingxuan Zhang, Peng Cui, Yi Yang, Fei Wu, Kun Kuang
Machine Learning
CDFM: Towards a General-Purpose Causal Discovery Foundation Model
Jie Qiao, Ruichu Cai, Zijian Li et al. (+6 more)
Causal discovery, the process of recovering underlying causal structures from observational data, is a fundamental pursuit across scientific disciplines. Over the past decades, numerous
algorithms have been developed to tackle this challenge through workflows tailored to the specific causal mechanisms underlying each type of
dataset, demonstrating effectiveness across a wide range of applications. However, as the volume and heterogeneity of real-world data continue to grow, this
dataset-specific approach inevitably leads to a fragmented, test-driven paradigm that struggles to scale to the demands of modern scientific discovery. To address this, we formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework for zero-shot structural inference. To ensure reliable generalization across unknown domains, we first investigate the theoretical boundaries of causal identifiability, revealing the indispensable role of causal prior mechanisms in this process. Building on these insights, we formulate a principled variational framework that treats unknown causal mechanisms as latent variables and mathematically decomposes the intractable marginal likelihood into distinct, tractable learning modules. The variational decomposition provides a conceptual design principle for the architecture design of CDFM, while comprehensive causal knowledge guides the large-scale synthesis of our pretraining data. By pretraining on a massive, highly diverse space of synthetic structural causal models, CDFM successfully internalizes complex statistical asymmetries. Extensive experiments demonstrate that CDFM consistently outperforms traditional
algorithms, driving a paradigm shift toward a general-purpose causal discovery foundation model.
Read the full abstract ▾
All authors Jie Qiao, Ruichu Cai, Zijian Li, Weilin Chen, Pengfei Hua, Boyan Xu, Zhengming Chen, Zhifeng Hao, Peng Cui
Machine Learning
SCOPE-RL: Optimizing Reasoning Paths Before and After Success
Xiaojian Liu, Han Xu, Jianqiang Xia et al. (+6 more)
Reinforcement learning with verifiable rewards (RLVR) optimizes LLMs using sparse verifiable final-answer rewards. This sparse anchor reliably verifies whether a trajectory succeeds but provides no direct feedback on the
reasoning path that produced it. Before success, prerequisite progress on hard problems receives no reward signal; after success, outcome rewards cannot distinguish well-organized correct trajectories from redundant or locally flawed ones. We introduce SCOPE-RL (Scaffolded Chain Optimization with Process Efficiency), a two-stage framework that densifies this anchor while retaining the GRPO update: Adaptive Scaffolded RL adds prefix-decomposed verifiable rewards on answer-hidden sub-question chains before success, and Quality-Aware Process RL applies correctness-gated process-shape rewards to refine correct trajectories after success. An expert-validated Step-Quality Evaluation Protocol evaluates useful-step density, error localization, and
token efficiency beyond final-answer accuracy. On Qwen3-8B-Instruct trained on DAPO-Math and Big-Math, SCOPE-RL improves average accuracy by up to 11.2 pp and reduces
reasoning tokens by up to 27.1% over outcome-only GRPO; the gains hold under GSPO and on Qwen3-0.6B-Instruct, indicating that reward-signal densification is complementary to policy-update-level RLVR advances. Code and data are available at https://github.com/tokencraft-lab/SCOPE-RL.
Read the full abstract ▾
All authors Xiaojian Liu, Han Xu, Jianqiang Xia, Zhixuan Li, Ke Xu, Yiwei Dai, Xinran Chen, Changwo Wu, Yuchen Li
Machine Learning
Proxy OPD: On-Policy Distillation with Transferable Relative Proxy Update
Daocheng Fu, Rong Wu, Yu Yang et al. (+7 more)
Post-training for large
language models typically couples policy exploration with model optimization, hindering the reuse of high-reward behaviors from policy exploration. While on-policy
distillation alleviates this by consolidating independently optimized experts, its reliance on matching absolute expert distributions can yield suboptimal supervision, especially when the target model possesses a different prior or already surpasses the expert's capabilities. To alleviate this, we introduce Proxy OPD (P-OPD), an asynchronous post-training framework that transfers reward-induced policy improvements rather than absolute policy distributions. P-OPD first optimizes a proxy policy via reward feedback. It then extracts the relative distributional changes between the proxy's initial and optimized states, transferring these directional updates through the target model's own on-policy trajectories while retaining the target policy as the reference. This decoupled formulation requires the proxy to provide merely a useful direction of improvement rather than superior absolute capability, enabling update signals from older or weaker proxies to remain highly effective. Systematic experiments on Qwen3-family models across mathematical
reasoning and code generation demonstrate that P-OPD consistently enhances already strong target models. Furthermore, transfer intensity can be dynamically modulated through signal scaling, making the extracted update signals seamlessly reusable across diverse model variants and training configurations. These results establish relative policy updates as highly reusable, adjustable assets for scalable, reward-based post-training.
Read the full abstract ▾
All authors Daocheng Fu, Rong Wu, Yu Yang, Jianbiao Mei, Licheng Wen, Pinlong Cai, Xuemeng Yang, Yong Liu, Botian Shi, Yu Qiao
Machine Learning
IG-GAN: A Generative Adversarial Network for Aerodynamic Data Generation Based on Intrinsic Geometry
Ying Yan, Liwei Hu and Xiaoming Zhang
Existing generative models learn data distributions in flat Euclidean space. However, most data in our real world are
manifolds embedded in high dimensional Euclidean space. Therefore, we propose an intrinsic-geometry-based generative
adversarial network (IG-GAN) for data generation in the field of aerodynamics. The generator of the IG-GAN represents aerodynamic data as a piecewise smooth
manifold constructed by Bézier surfaces, and the generator tries to learn the coefficients of each Bézier surface to further combine multiple Bézier surfaces into a smooth
manifold automatically. The discriminator in the IG-GAN is a radial-basis-function based discriminator (RBF-D). Experimental results show that IG-GAN achieves lower predicted Mean Squared Errors (MSEs) than those of three baselines. Specifically, on the Burgers' equation
dataset, IG-GAN reduces the predicted MSE of velocity u by 97.41% compared with state of the art SSL-
Transformer. Additionally, on the ONERA M6 aircraft
dataset, IG-GAN reduces the overall MSE of nine aerodynamic coefficients by 82.95% compared with SSL-
Transformer.
Read the full abstract ▾
Machine Learning
Agentic Skill Optimization over Lie Algebroids
Sridhar Mahadevan
Agentic systems increasingly improve themselves by editing skills:
prompts, rubrics, plans, tool contracts, examples, validators, and traces. Skill edits are not independent coordinates in a vector space: they are local repairs to structured artifacts whose effects are observed only after rollout, validation, and critique. Distinct edits can have the same immediate visible effect while differing in routing context, template state, guardrail scope, or future composability. The order of edits can matter as well: repairing a schema before a normalization rule need not be equivalent to applying the same edits in the reverse order. This paper introduces a new framework for skill optimization called LASKO, for Lie Algebroid SKill Optimization. LASKO models typed, anchored Markdown skills as the base category and available edit policies as sections of a controlled Lie algebroid with anchor $\rho$. The anchor maps an edit policy to its visible Markdown effect; the kernel $\ker(\rho)$ represents latent template, routing, or implementation structure; and the algebroid bracket measures noncommuting edit composition. As shown in the paper, LASKO achieves order-of-magnitude speedups in skill optimization in our preliminary
benchmark results, primarily because it substitutes inexpensive Lie-bracket screening tests that run in microseconds, before investing in expensive validations that require running large
language models. On a causal extraction from natural
language task, LASKO achieved a speedup of almost $15 \times$ compared to a brute-force approach that validated all edits by running them through a DeepSeek V3.1 4-bit model with 671B parameters.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IT
Compound Interference Recognition for LR-FHSS Satellite IoT Uplinks via Multi-Domain Instance Fusion
H. Xu, B. He, S. Wang et al. (+1 more)
Long range-frequency hopping spread spectrum (LR-FHSS) is a promising uplink physical layer for massive low Earth orbit satellite Internet of Things, where low power terminals report short packets from wide area regions with limited terrestrial infrastructure. However, satellite IoT links are exposed to external interference, and the coexistence of multiple interference components can severely degrade receiver reliability and complicate interference mitigation. Existing recognition methods either focus on single interference scenarios or treat each compound interference combination as an independent class, leading to limited generalization or poor scalability. To address this problem, this paper formulates LR-FHSS uplink compound interference recognition as a multi-instance multi-label learning problem and proposes a multi-domain instance fusion method. The proposed method fuses local instances from the time-frequency and frequency domains and aggregates their predictions for bag-level multi-label recognition. A
dataset construction pipeline is developed based on the US915 LR-FHSS configuration and incorporates shadowed-Rician fading and time-varying Doppler to emulate practical satellite communication conditions. Considering the difficulty of obtaining labeled compound interference samples in practice, single-to-compound generalization and few-shot compound interference adaptation are investigated as two practical receiver deployment scenarios. Experimental results show that the proposed method improves the overall exact accuracy over the strongest baseline by 14.71 percentage points in single-to-compound generalization and by 14.81 percentage points in few-shot compound interference adaptation for $r=1$.
Read the full abstract ▾
All authors H. Xu, B. He, S. Wang, Y. Jiang
Machine Learning
HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models
Aznaur Aliev, Carlos Hinojosa, Abdelrahman Eldesokey et al. (+3 more)
Safety
alignment in large
language models can be fragile under fine-tuning, as even benign task adaptation may increase harmful compliance. Existing defenses mainly follow two directions: they either intervene during or after fine-tuning through retraining or weight modification, which can be costly and may hurt task performance, or they use model-agnostic safety classifiers, which may miss failures specific to a given fine-tuned checkpoint. These limitations motivate a post hoc, model-specific, and non-invasive approach to safety restoration. To meet these requirements, we propose HyperSafe, a framework that restores safety behavior by generating a model-specific Safe Side Network (SSN) for each fine-tuned checkpoint. HyperSafe uses layer-wise activation fingerprints to capture how fine-tuning changes the model's inner representations. With a small set of given calibration
prompts, the hypernetwork maps these fingerprints to the parameters of the \ssn{} in a single forward pass. The generated \ssn{} runs alongside the frozen fine-tuned model and performs
prompt-level safety classification: harmful
prompts are routed to refusal, while safe
prompts are answered by the original fine-tuned model. Thus, HyperSafe requires no gradient updates, no safety data at deployment time, and no modification to the deployed model weights. We evaluate HyperSafe on two model families, Qwen2-7B and LLaMA-3-8B, across multiple safety
benchmarks. HyperSafe reduces harmful response rates from 19-31% to below 1% on every held-out checkpoint, while keeping downstream task accuracy within 1% of the fine-tuned baseline on average. Code is available at https://github.com/nokronim/project-safety-remedy.
Read the full abstract ▾
All authors Aznaur Aliev, Carlos Hinojosa, Abdelrahman Eldesokey, Bang An, Bernard Ghanem, Yibo Yang
Machine Learning · Cross-listed from cs.CV
Towards Efficient Convolutional Neural Network for Embedded Hardware via Multi-Dimensional Pruning
Hao Kong, Di Liu, Xiangzhong Luo et al. (+5 more)
In this paper, we propose TECO, a multi-dimensional
pruning framework to collaboratively prune the three dimensions (depth, width, and resolution) of convolutional neural networks (CNNs) for better execution efficiency on embedded hardware. In TECO, we first introduce a two-stage importance evaluation framework, which efficiently and comprehensively evaluates each
pruning unit according to both the local importance inside each dimension and the global importance across different dimensions. Based on the evaluation framework, we present a heuristic
pruning algorithm to progressively prune the three dimensions of CNNs towards the optimal trade-off between accuracy and efficiency. Experiments on multiple
benchmarks validate the advantages of TECO over existing state-of-the-art (SOTA) approaches. The code and pre-trained models are available at https://github.com/ntuliuteam/Teco.
Read the full abstract ▾
All authors Hao Kong, Di Liu, Xiangzhong Luo, Shuo Huai, Ravi Subramaniam, Christian Makaya, Qian Lin, Weichen Liu
Machine Learning · Cross-listed from stat.AP
Climate-Invariant Conformal Prediction Intervals for Multi-Horizon Solar and Wind Forecasting
Shreedhar Gangwar, Abhinav Bains and Banalaxmi Brahma
Reliable uncertainty quantification is essential for integrating
solar and wind generation into modern power systems, where operators must weigh risk rather than act on point forecasts alone. Existing probabilistic methods, however, often either lack finite-sample validity or require per-site recalibration, so a single model rarely transfers across the diverse climates of a dispersed generation fleet. This paper proposes a heteroscedastic, asymmetric, group-conditional split-conformal framework built on a bootstrap-diverse XGBoost ensemble, producing prediction intervals that adapt in width to local difficulty while retaining distribution-free coverage guarantees. A single fixed specification, with no per-site or per-horizon tuning, is evaluated across four climatologically distinct sites spanning both hemispheres, at horizons of 1 to 12 hours, for both
solar irradiance and wind speed. The framework holds near-nominal coverage on both targets and reduces the Interval Score by up to 35% relative to competitive baselines, with the calibration and sharpness of its intervals shown to be properties of the method rather than of site-specific tuning.
Read the full abstract ▾
Machine Learning
Event-based Neural Decoding for Neuroprosthetic Motor Control
Khaleelulla Khan Nazeer, Sirine Arfa, Matthias Jobst et al. (+2 more)
A substantial number of patients experience diminished mobility due to disabilities, diseases, or accidents. Although modern prostheses, powered by deep neural networks, hold the promise of significantly enhancing the quality of life for these individuals, their widespread adoption is hindered by significant latency, energy consumption, and spatial requirements. Wired connections to external high-performance processors restrict patient mobility, while wireless connections limit the volume of information that can be transmitted to these processors. Spiking neural networks offer the potential for compressed communication and low-power inference, yet they often lag behind state-of-the-art deep learning models in various applications. In this study, we propose a high-performance neural decoding method that effectively balances task performance and efficiency. An eventbased gated recurrent unit generates a sparse communication pattern with graded spikes, surpassing classical spiking neural networks in terms of task performance. Utilising an efficient training method and sparse inference, our model presents new opportunities for on-device neural decoding.
Read the full abstract ▾
All authors Khaleelulla Khan Nazeer, Sirine Arfa, Matthias Jobst, Richard George, Christian Mayr
Machine Learning
Velocity Scheduled Flow Matching
Vitalii Bondar
Flow matching trains a neural network to regress the conditional velocity along a linear interpolant between noise and data, and the number of network evaluations~(NFE) sets the cost of sampling. The straight-line interpolant carries an implicit choice: the sample moves at constant speed throughout the trajectory. We relax this choice and introduce Velocity Scheduled Flow Matching~(VSFM), which replaces the conditional target $x_1 - x_0$ with $v(t)(x_1 - x_0)$ for any nonnegative profile $v:[0,1]\to\mathbb{R}_{\geq 0}$ satisfying $\int_0^1 v\,dt = 1$. We study six
polynomial profiles drawn from motion planning. The first use of VSFM is at inference time: a pretrained linear flow-matching model can be sampled under any admissible profile by integrating its ODE on a non-uniform $\tau$-schedule, with no retraining and no additional computation; on CIFAR-10 this lowers FID by up to $19.8\%$. Training from scratch under a braking profile gives a further reduction of $17.4\%$ at $4$~NFE. Both gains follow from the local truncation error of the Euler integrator on the induced grid.
Read the full abstract ▾
Machine Learning
On Generalizing Preference-based Reinforcement Learning to Multi-Objective Problems
Simone Drago, Marco Mussi, Leonardo Bianconi et al. (+1 more)
We study reinforcement learning (RL) from pairwise human-generated trajectory comparisons. We generalize preference-based RL by formalizing a novel setting in which the expert can also label trajectory pairs as incomparable when neither trajectory "dominates" the other. We introduce the learning problem and the desiderata that a model should satisfy, showing that, when multiple objectives are present, no compliant model admits a negative log-likelihood that is convex in the utility difference. Then, we propose a novel Bradley-Terry-inspired rationality model that enables incomparability and infers a multi-dimensional reward function, and discuss its compliance with the desiderata. We upper bound the divergence between the true and learned comparison distributions when a
dataset is available, separating the rate of approach to the optimum from a non-vanishing term due to non-convexity. Then, we study how learned parameters affect downstream guarantees, highlighting the impact of invariance on Pareto optimality.
Read the full abstract ▾
All authors Simone Drago, Marco Mussi, Leonardo Bianconi, Alberto Maria Metelli
Machine Learning
Physics-Aware Conditional SetGAN for Spatially Consistent Multi-User TR 38.901 Channel Generation
Mauro Gonzalo Tarazona-Levano, David Lopez-Perez, Nicola Piovesan et al. (+1 more)
TR 38.901-based channel models such as Sionna are reliable, but generating many multi-user channel realizations remains expensive. This paper asks a practical question: can a trained generative model produce multi-user TR 38.901 channels faster than Sionna without losing the spatial correlations imposed by user geometry? To answer this question, we propose a physics-aware, geometry-conditioned SetGAN trained on Sionna reference data. The method separates large-scale received power from normalized small-scale fading, compresses the latter with principal component analysis, and learns the conditional channel distribution in a latent space while preserving geometry-dependent correlations. On the UMa/NLoS
benchmark, the model keeps the received-power distributions close to the reference, with about 0.41 dB Wasserstein distance, and reproduces spatial-consistency profiles with mean deviations below 0.03 on median curves versus distance. In addition, it reduces elapsed generation time by a factor of 3.45 and CPU-total cost by a factor of 6.15 relative to Sionna under matched user positions in the fixed-position CPU-vs-CPU
benchmark. These results show that a trained generative model can substantially accelerate TR 38.901 channel generation without breaking the spatial consistency needed to evaluate multi-user systems.
Read the full abstract ▾
All authors Mauro Gonzalo Tarazona-Levano, David Lopez-Perez, Nicola Piovesan, David Gomez-Barquero
Machine Learning · Cross-listed from cs.AI
StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure
Wenyi Wu, Sibo Zhu, Kun Zhou et al. (+3 more)
Recent advances in large
language models (LLMs) and
vision-
language models (VLMs) have enabled increasingly capable digital
agents for computer use. However, real-world tasks are often long-horizon and involve evolving contexts containing accumulated observations, intermediate edits, failed attempts, and partially completed executions. Existing
agents typically operate over raw interaction history, making task progress difficult to interpret, verify, and recover, which ultimately limits reliable long-horizon execution. In this paper, we argue that addressing this challenge requires explicitly structuring both the
agent's state and workflow around a unified causal representation of task progress. We present StructAgent, a state-centered framework that introduces a unified state for maintaining compact, verifiable task progress and a structured workflow that regulates progress through verifier-backed state transitions. Building on this design, StructAgent further enables explicit progress checkpointing, evidence-driven task completion, targeted failure recovery, and tool-supported execution, while ensuring that all progress updates remain grounded in verification. Extensive experiments demonstrate that StructAgent consistently improves a wide range of LLM and VLM backbones on long-horizon computer-use tasks. On OSWorld-Verified, it improves Qwen3.5-9B from 27.0% to 46.9% success rate and Qwen3.5-27B from 31.6% to 62.2%, while achieving a new open-source state of the art of 78.9% with MiniMax-M3. Moreover, the same framework generalizes beyond desktop environments to Minecraft, demonstrating the generality of our design.
Read the full abstract ▾
All authors Wenyi Wu, Sibo Zhu, Kun Zhou, Aayush Salvi, Zixuan Song, Biwei Huang
Machine Learning
Surprisingly Simple and Effective Multi-Domain Graph Foundation Model through Graph-to-Table Alignment
Chunyu Hu, Tianyin Liao, Ge Lan et al. (+4 more)
Graph Foundation Models (GFMs) have emerged as a promising paradigm for learning transferable representations across diverse graph domains. Recent advancements in GFMs have been largely dominated by two paradigms: Graph Neural Network and Large
Language Model (LLM) based methods. However, these methods often face a fundamental dilemma between training with limited data and a heavy reliance on textual attributes. Tabular foundation models (TFMs) offer a potential alternative, as node features and representations can be naturally organized in a tabular form. However, how to enable TFMs to effectively capture structural information of graphs remains largely unexplored. The key challenge is to learn a graph-to-table
alignment mechanism that enables graph structural understanding for TFMs. To address this, we propose GTAlign, a surprisingly simple yet effective Graph-to-Table
Alignment framework for text-free Graph Foundation Model. Specifically, we first pretrain a graph encoder that maps diverse graphs into a unified latent space to capture domain-agnostic graph representations. To further bridge the gap between graph
topology and the tabular representation space, we propose community-guided continual pre-training, where pseudo-labels derived from graph community are used to construct few-shot prediction episodes. Lastly, we adapt the graph encoder for an unseen target domain and perform in-context inference. Extensive experiments on five
benchmark datasets demonstrate that GTAlign significantly outperforms state-of-the-art baselines on both node and graph classification, offering a simple, effective, and text-free GFM model. Code will be released upon acceptance.
Read the full abstract ▾
All authors Chunyu Hu, Tianyin Liao, Ge Lan, Xingxuan Zhang, Jianxin Li, Peng Cui, Ziwei Zhang
Machine Learning · Cross-listed from cs.DC
Decomposing Runtime, Kernel, and Quantization Speedups via a Matched FP16 Intermediate: A Hardware-Conditioned Case Study on Four NVIDIA RTX A5000 GPUs
Weijia Han and Lisha Qu
Reported serving speedups from quantized kernels typically bundle the weight format, the kernel, and the inference runtime into one number. We present an attribution study on four NVIDIA RTX A5000 GPUs, 24 GiB each, on a single host with NVLink-bridged pairs. A matched intermediate stack that keeps the faster runtime without the quantized kernel splits the full speedup into a runtime part and a kernel and quantization part. Under matched greedy decoding the full stack reaches $2.58\times$ end to end, with the runtime change accounting for about two thirds of that gain on a logarithmic scale; across three similar model families the kernel and quantization part moves by at most 1.5%. Sharding one instance across all four cards falls well below doubling: a profiler trace attributes about 80% of the per
token shortfall to coordination, and an NVLink versus PCIe control on the same hardware shows similar realized bandwidth on both links, pointing away from link bandwidth as the cause. Whether to run one sharded instance or several independent ones depends on the workload and the model, with the ranking reversing on the larger model: the smaller model splits between sharding and multiple instances by workload, while the larger model favors two paired instances on every workload. Quantization extends sustainable concurrent users roughly four times past a reproducible half precision
memory cliff. Differences in sampling mode and
prompt pool between the two stacks are documented as threats to validity.
Read the full abstract ▾
Machine Learning
BackgroundMellow: A Multi-Modal Cohesive Framework for Narrative-Driven Rich Cinematic Soundscape Generation
Ajitesh Jamulkar and Aritra Hazra
Generating immersive, synchronized and cinematic audio for long-form textual narratives remains a significant challenge in multi-modal AI. While current Text-to-Audio (TTA) frameworks successfully synthesize isolated sound effects, they struggle with narrative cohesion, temporal
alignment, and cinematic emotional depth. We present BackgroundMellow, a framework that treats story-to-audio generation as a precise orchestration and signal processing problem. This framework is enabled without ground-truth through a master-specialist
agent architecture that decomposes text into precise and multi-layered audio cues, generates each category of sounds with suitable specialist model, and superimposes the soundscapes to create a unified and aligned audio segment. Our pipeline is built over Tango2 latent
diffusion model for environmental synthesis alongside a novel Cinematic BGM Retriever mined from professional soundtracks. To automate the sound mixing process, we use an NLP based module that predicts precise audio parameters, like start time, duration, and relative loudness, based on the narrative timeline. We further empirically evaluate and show the efficacy of the proposed framework leveraging nearest-neighbor
retrieval against a curated
dataset of YouTube cinematic trailers to measure temporal synchronization, coverage, and spectral richness.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Characterising AI Models for Cataloguing
Miguel Arana-Catania and Neil Jefferies
The creation of digital collections involves not only the digitisation of content, but also the creation of catalogue records for it. This often-overlooked task requires slow and costly expert manual work. In this project, we have evaluated the application of AI models to this task, comparing different implementations and models. This work includes a qualitative and quantitative evaluation of the experiments carried out, as well as recommendations on the use of AI models that go beyond the specific use case.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Inter-Stop Energy Prediction and Causal Driver Quantification for Dual-Source Trolleybuses via a Time-Aware Tabular Deep Learning Architecture
Wentao Zeng, Zijian Huang, Yiming Bie et al. (+2 more)
Dual-source trolleybuses alternate between overhead catenary supply and on-board battery operation, creating energy-use patterns driven by route attributes, high-frequency trajectories, and hourly weather. Existing models struggle to represent these heterogeneous inputs and rarely explain the causal drivers of consumption. This paper proposes a time-aware tabular deep learning framework for inter-stop energy management. Periodic time encoding is integrated into a parameter-efficient batch-ensemble backbone to jointly learn static and sequential features, while
Bayesian optimization with tree-structured density estimation tunes hyperparameters. To move beyond prediction, a three-layer causal explanation pipeline combines feature attribution for marginal effects, a linear non-Gaussian acyclic model for causal direction discovery, and a meta-learner for net average treatment effects. Experiments on the Zurich trolleybus
dataset enriched with meteorological records achieve a MAPE of 6.52% and R of 0.982, outperforming ten statistical, tree-ensemble, and deep learning baselines. Ablation results show that periodic time encoding contributes most to the accuracy gain. Causal analysis identifies regenerative braking ratio and average speed as the strongest energy-saving factors, while coasting distance is the main driver of excess consumption. The findings offer actionable thresholds for vehicle technology, driving behavior, capacity allocation, and catenary network planning.
Read the full abstract ▾
All authors Wentao Zeng, Zijian Huang, Yiming Bie, Jiabin Wu, Jun Gong
Machine Learning
PRISM Edit: One Vector for All Temporal Answers
Chen Huang, Qi Zheng, Ruiqin Zheng et al. (+2 more)
Model editing keeps large
language models (LLMs) up to date without retraining, but temporal facts expose a limitation of the prevailing locate-and-edit paradigm: an update is not always a replacement. When a fact changes, the new answer should become current while the old answer may remain correct in historical time contexts. Building on this insight, we use causal tracing to show that LLMs already support this distinction via a two-stage internal computation: early MLP layers retrieve a time-agnostic subject representation, and later layers modulate it with temporal context to yield the time-correct answer. Motivated by this finding, we introduce PRISM Edit, which optimizes a single polysemous representation across temporal contexts and leverages the model's inherent modulation pathway to route it to temporally correct predictions without requiring any architectural modification. We evaluate on TimeConflict, a newly introduced temporal editing
benchmark, and on temporally augmented CounterFact. PRISM Edit improves multiple core metrics over the best baseline, most notably +23.3 Temporal Consistency (TC) and +33.7 Current Relative-time Score (CRS) on LLaMA-3, while being more than 2x faster. Code and data are publicly available at https://github.com/CheerCHuang/PRISM-Edit.
Read the full abstract ▾
All authors Chen Huang, Qi Zheng, Ruiqin Zheng, Long Zeng, Yuantong Xu
Machine Learning
SPARC-Net: A Spectral, Causality-Aware, and Hard-Constrained Physics-Informed Architecture for Stiff and Shock-Dominated Partial Differential Equations
Divyavardhan Singh, Dimple Sonone, Hammad Mohammad et al. (+1 more)
Physics-Informed Neural Networks (PINNs) provide a meshless approach for solving partial differential equations (PDEs), but suffer severe degradation in stiff and shock-dominated problems, where small PDE residuals can correspond to globally inaccurate solutions. We show these failures are multi-causal, arising from the concurrent interplay of (i) spectral bias against sharp features, (ii) imbalanced multi-term optimization and loss-weight collapse, (iii) violation of temporal causality, and (iv) under-resolved collocation. We present SPARC-Net, a unified architecture and training framework that jointly addresses all four pathologies. SPARC-Net leverages an adaptive multi-scale spectral encoder with a learnable spectral gate, a gated residual backbone, adaptive activations, and a hard-constraint output ansatz that exactly enforces initial and boundary conditions, structurally eliminating loss-weight collapse. Training employs stabilized gradient-norm loss balancing, floored causality-respecting residual weighting, and residual-based adaptive collocation (RAD). Validated against exact analytic and high-order spectral reference solutions across four canonical
benchmarks – viscous Burgers', Allen-Cahn, convection (beta=30), and reaction – SPARC-Net yields substantial improvements over vanilla PINNs: relative L2 error drops from 1.47e-1 to 1.14e-1 on Burgers' (22% reduction), 9.93e-1 to 5.78e-2 on Allen-Cahn (94% reduction), and 9.82e-1 to 3.54e-3 on reaction (100% reduction). A characteristic-coordinate encoder for
hyperbolic transport further reduces convection error from 5.14e-1 to 9.88e-5 (100% reduction). We report five-seed mean +/- standard deviation errors, Wilcoxon significance tests, full ablation studies, hyperparameter sensitivities, an extension to the 2D heat equation, and comparisons against parameter-matched baselines.
Read the full abstract ▾
All authors Divyavardhan Singh, Dimple Sonone, Hammad Mohammad, Kishor Upla