Machine Learning
MHPO: Modulated Hazard-aware Policy Optimization for Stable Reinforcement Learning
Hongjun Wang, Wei Liu, Weibo Gu et al. (+2 more)
Regulating the importance ratio is critical for the training stability of Group Relative Policy Optimization (GRPO) based frameworks. However, prevailing ratio control methods, such as hard clipping, suffer from non-differentiable boundaries and vanishing gradient regions, failing to maintain gradient fidelity. Furthermore, these methods lack a hazard-aware mechanism to adaptively suppress extreme deviations, leaving the optimization process vulnerable to abrupt policy shifts. To address these challenges, we propose Modulated Hazard-aware Policy Optimization (MHPO), a novel framework designed for robust and stable reinforcement learning. The proposed MHPO introduces a Log-Fidelity Modulator (LFM) to map unbounded importance ratios into a bounded, differentiable domain. This mechanism effectively prevents high-variance outlier
tokens from destabilizing the loss landscape while ensuring global gradient stability. Complementarily, a Decoupled Hazard Penalty (DHP) integrates cumulative hazard functions from survival analysis to independently regulate positive and negative policy shifts. By shaping the optimization landscape with hazard-aware penalties, the proposed MHPO achieves fine-grained regulation of asymmetric policy shifts simultaneously mitigating mode collapse from over-expansion and preventing policy erosion from catastrophic contraction within a stabilized trust region. Extensive evaluations on diverse
reasoning benchmarks across both text-based and
vision-
language tasks demonstrate that MHPO consistently outperforms existing methods, achieving superior performance while significantly enhancing training stability.
Read the full abstract ▾
All authors Hongjun Wang, Wei Liu, Weibo Gu, Xing Sun, Kai Han
Machine Learning · Cross-listed from cs.CR
Noticing the Watcher: LLM Agents Can Infer CoT Monitoring from Blocking Feedback
Thomas Jiralerspong, Flemming Kondrup and Yoshua Bengio
Chain-of-thought (CoT) monitoring is proposed as a method for overseeing the internal
reasoning of
language-model
agents. Prior work has shown that when models are explicitly informed that their
reasoning is being monitored, or are fine-tuned to internalize this fact, they may learn to obfuscate their CoTs in ways that allow them to evade CoT-based monitoring systems. We ask whether
reasoning agents can autonomously infer that their supposedly private CoT is under surveillance, and whether this awareness leads to strategic evasion, without any explicit training or instructions. In a multi-episode agentic framework, models pursue both a primary task and a concealed side task while being told that their
reasoning is private; a hidden CoT monitor blocks episodes when suspicious
reasoning is detected. We find that frontier models can deduce the existence of this monitor purely from blocking feedback, with the most capable models reaching a confident belief that their thinking is observed in up to 19% of episodes. This awareness scales with model capability and, in rare cases, escalates to an explicit intent to suppress
reasoning about the side task. However, models that form this intent uniformly fail to execute it, openly
reasoning about their concealed objectives in the very next episode. This intent-capability gap is reassuring for current deployment, but the autonomous emergence of both monitoring awareness and evasion intent suggests that CoT monitoring is not a permanently reliable safeguard.
Read the full abstract ▾
Machine Learning
Spectral Edge Dynamics of Training Trajectories: Signal–Noise Geometry Across Scales
Yongzhong Xu
Despite hundreds of millions of parameters,
transformer training trajectories evolve within only a few coherent directions. We introduce Spectral Edge Dynamics (SED) to quantify this structure: a rolling-window SVD of parameter updates reveals a sharp boundary – the spectral edge – between coherent optimization directions and
stochastic noise, identified via the maximum consecutive singular value ratio $\sigma_k / \sigma_{k+1}$. Across a 51M-parameter TinyStories model (4 seeds) and GPT-2 124M under distribution shift, the spectral edge exhibits a universal three-phase pattern (rise, plateau, collapse). The effective signal rank adapts to task complexity ($k^* = 2$ at 51M, $k^* = 3$ at 124M), and the directional coupling between spectral geometry and validation loss reverses with window size – a lag flip reflecting the timescale of trajectory integration. Johnson–Lindenstrauss projection to $d = 10W$ dimensions (e.g., $d = 100$ for $W = 10$) preserves the spectral gap within $5.7\%$, making the framework applicable to models of arbitrary scale. In companion work, the same spectral geometry provides early-warning signals of grokking – predicting generalization $600$–$1{,}700$ steps before it occurs across modular arithmetic, Dyck
languages, and the SCAN
benchmark.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.HC
Clinician input steers AI toward accurate and harmful recommendations
Ivan Lopez, Selin S. Everett, Bryan J. Bunning et al. (+10 more)
Large
language models (LLMs) are entering clinical workflows, yet evaluations rarely assess how clinician
reasoning shapes model behavior during clinical interactions. Using 61 curated NEJM Case Records, we tested how expert or misleading clinician
reasoning influenced AI-generated differential diagnoses and next step recommendations across 21
reasoning variants from 8 proprietary and open-source models. After clinician exposure, LLM-clinician concordance increased: simulations with >=3 overlapping differential diagnoses rose from 65.8% to 93.5%, and those with >=3 overlapping next step recommendations from 20.3% to 53.8%. Expert context significantly improved correct final-diagnosis inclusion in all 21 models (mean +20.4 pp), reflecting both improved
reasoning and passive content echoing, while
adversarial context significantly degraded performance in 14 models (mean -5.4 pp). Expert context also significantly increased leading-diagnosis accuracy in all 21 models, whereas
adversarial context significantly reduced it in 13. Multi-turn disagreement challenges revealed distinct model phenotypes, from highly conformist to dogmatic, with
adversarial arguments remaining a vulnerability even in otherwise resilient models. Inference-time scaling reduced harmful echoing of clinician-introduced recommendations across WHO harm-severity tiers by 62.7% for mild, 57.9% for moderate, 76.3% for severe, and 83.5% for death-tier recommendations. Inference-time prompting recovered diagnostic accuracy lost to
adversarial context while preserving expert-context benefits across GPT-5, Claude Sonnet 4.5, and Gemini 3 Flash, and sharply reduced highly consistent harmful echoing across severity tiers. These findings provide a foundation for evaluating clinician-AI collaboration and introduce interactive metrics and mitigation strategies essential to safety and robustness.
Read the full abstract ▾
All authors Ivan Lopez, Selin S. Everett, Bryan J. Bunning, April S. Liang, Dong Han Yao, Shivam C. Vedak, Kameron C. Black, Sophie Ostmeier, Stephen P. Ma, Emily Alsentzer, Jonathan H. Chen, Akshay S. Chaudhari, Eric Horvitz
Machine Learning
Align Forward, Adapt Backward: Closing the Discretization Gap in Logic Gate Networks
Youngsung Kim
In neural network models, soft mixtures of fixed candidate components (e.g., logic gates and sub-networks) are often used during training for stable optimization, while hard selection is typically used at inference. This raises questions about training-inference mismatch. We analyze this gap by separating forward-pass computation (hard selection vs. soft mixture) from stochasticity (with vs. without Gumbel noise). Using logic gate networks as a testbed, we observe distinct behaviors across four methods: Hard-ST achieves zero selection gap by construction; Gumbel-ST achieves near-zero gap when training succeeds but suffers accuracy collapse at low temperatures; Soft-Mix achieves small gap only at low temperature via weight concentration; and Soft-Gumbel exhibits large gaps despite Gumbel noise, confirming that noise alone does not reduce the gap. We propose CAGE (Confidence-Adaptive Gradient Estimation) to maintain gradient flow while preserving forward
alignment. On logic gate networks, Hard-ST with CAGE achieves over 98% accuracy on MNIST and over 58% on CIFAR-10, both with zero selection gap across all temperatures, while Gumbel-ST without CAGE suffers a 47-point accuracy collapse.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
An Alternative Trajectory for Generative AI
Margarita Belova, Yuval Kansal, Yihao Liang et al. (+2 more)
The generative artificial intelligence (AI) ecosystem is undergoing rapid transformations that threaten its sustainability. As models transition from research prototypes to high-traffic products, the energetic burden has shifted from one-time training to recurring, unbounded inference. This is exacerbated by
reasoning models that inflate compute costs by orders of magnitude per query. The prevailing pursuit of artificial general intelligence through scaling of monolithic models is colliding with hard physical constraints: grid failures, water consumption, and diminishing returns on data scaling. This trajectory yields models with impressive factual recall but struggles in domains requiring in-depth
reasoning, possibly due to insufficient abstractions in training data. Current large
language models (LLMs) exhibit genuine
reasoning depth only in domains like mathematics and coding, where rigorous, pre-existing abstractions provide structural grounding. In other fields, the current approach fails to generalize well. We propose an alternative trajectory based on domain-specific superintelligence (DSS). We argue for first constructing explicit symbolic abstractions (knowledge graphs, ontologies, and formal logic) to underpin synthetic curricula enabling small
language models to master domain-specific
reasoning without the model collapse problem typical of LLM-based synthetic data methods. Rather than a single generalist
giant model, we envision "societies of DSS models": dynamic ecosystems where orchestration
agents route tasks to distinct DSS back-ends. This paradigm shift decouples capability from size, enabling intelligence to migrate from energy-intensive data centers to secure, on-device experts. By aligning algorithmic progress with physical constraints, DSS societies move generative AI from an environmental liability to a sustainable force for economic empowerment.
Read the full abstract ▾
All authors Margarita Belova, Yuval Kansal, Yihao Liang, Jiaxin Xiao, Niraj K. Jha
Machine Learning
Multifidelity Surrogate Modeling of Depressurized Loss of Forced Cooling in High-temperature Gas Reactors
Meredith Eaheart and Majdi I. Radaideh
High-fidelity computational fluid dynamics (CFD) simulations are widely used to analyze
nuclear reactor transients, but are computationally expensive when exploring large parameter spaces. Multifidelity surrogate models offer an approach to reduce cost by combining information from simulations of varying resolution. In this work, several multifidelity machine learning methods were evaluated for predicting the time to onset of natural circulation (ONC) and the temperature after ONC for a high-temperature gas reactor (HTGR) depressurized loss of forced cooling transient. A CFD model was developed in Ansys Fluent to generate 1000 simulation samples at each fidelity level, with low and medium-fidelity
datasets produced by systematically coarsening the high-fidelity mesh. Multiple surrogate approaches were investigated, including multifidelity Gaussian processes and several neural network architectures, and validated on analytical
benchmark functions before application to the ONC
dataset. The results show that performance depends strongly on the informativeness of the input variables and the relationship between fidelity levels. Models trained using dominant inputs identified through prior sensitivity analysis consistently outperformed models trained on the full input set. The low- and high-fidelity pairing produced stronger performance than configurations involving medium-fidelity data, and two-fidelity configurations generally matched or exceeded three-fidelity counterparts at equivalent computational cost. Among the methods evaluated, multifidelity GP provided the most robust performance across input configurations, achieving excellent metrics for both time to ONC and temperature after ONC, while neural network approaches achieved comparable accuracy with substantially lower training times.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Conditional flow matching for physics-constrained inverse problems with finite training data
Agnimitra Dasgupta, Ali Fardisi, Mehrnegar Aminy et al. (+4 more)
This study presents a conditional flow matching framework for solving physics-constrained
Bayesian inverse problems. In this setting, samples from the joint distribution of inferred variables and measurements are assumed available, while explicit evaluation of the prior and likelihood densities is not required. We derive a simple and self-contained formulation of both the unconditional and conditional flow matching
algorithms, tailored specifically to inverse problems. In the conditional setting, a neural network is trained to learn the velocity field of a probability flow ordinary differential equation that transports samples from a chosen source distribution directly to the posterior distribution conditioned on observed measurements. This black-box formulation accommodates nonlinear, high-dimensional, and potentially non-differentiable forward models without restrictive assumptions on the noise model. We further analyze the behavior of the learned velocity field in the regime of finite training data. Under mild architectural assumptions, we show that overtraining can induce degenerate behavior in the generated conditional distributions, including variance collapse and a phenomenon termed selective memorization, wherein generated samples concentrate around training data points associated with similar observations. A simplified theoretical analysis explains this behavior, and numerical experiments confirm it in practice. We demonstrate that standard early-stopping criteria based on monitoring test loss effectively mitigate such degeneracy. The proposed method is evaluated on several physics-based inverse problems. We investigate the impact of different choices of source distributions, including Gaussian and data-informed priors. Across these examples, conditional flow matching accurately captures complex,
multimodal posterior distributions while maintaining computational efficiency.
Read the full abstract ▾
All authors Agnimitra Dasgupta, Ali Fardisi, Mehrnegar Aminy, Brianna Binder, Bryan Shaddy, Saeed Moazami, Assad Oberai
Machine Learning · Cross-listed from cs.CV
DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++
Shahriar Kabir, Abdullah Muhammed Amimul Ehsan, Istiak Ahmmed Rifti et al. (+1 more)
Automated
segmentation of Martian landslides, particularly in tectonically active regions such as Valles Marineris,is important for planetary geology, hazard assessment, and future robotic exploration. However, detecting landslides from planetary imagery is challenging due to the heterogeneous nature of available sensing modalities and the limited number of labeled samples. Each observation combines RGB imagery with geophysical measurements such as digital elevation models, slope maps, thermal inertia, and contextual grayscale imagery, which differ significantly in resolution and statistical properties. To address these challenges, we propose DualSwinFusionSeg, a
multimodal segmentation architecture that separates modality-specific feature extraction and performs multi-scale cross-modal fusion. The model employs two parallel Swin
Transformer V2 encoders to independently process RGB and auxiliary geophysical inputs, producing hierarchical feature representations. Corresponding features from the two streams are fused at multiple scales and decoded using a UNet++ decoder with dense nested skip connections to preserve fine boundary details. Extensive ablation studies evaluate modality contributions, loss functions, decoder architectures, and fusion strategies. Experiments on the MMLSv2
dataset from the PBVS 2026 Mars-LS Challenge show that modality-specific encoders and simple concatenation-based fusion improve
segmentation accuracy under limited training data. The final model achieves 0.867 mIoU and 0.905 F1 on the development
benchmark and 0.783 mIoU on the held-out test set, demonstrating strong performance for
multimodal planetary surface
segmentation.
Read the full abstract ▾
All authors Shahriar Kabir, Abdullah Muhammed Amimul Ehsan, Istiak Ahmmed Rifti, Md Kaykobad Reza
Machine Learning
Is the reconstruction loss culprit? An attempt to outperform JEPA
Alexey Potapov, Oleg Shcherbakov and Ivan Kravchenko
We evaluate JEPA-style predictive representation learning versus reconstruction-based autoencoders on a controlled "TV-series" linear dynamical system with known latent state and a single noise parameter. While an initial comparison suggests JEPA is markedly more robust to noise, further diagnostics show that autoencoder failures are strongly influenced by asymmetries in objectives and by bottleneck/component-selection effects (confirmed by PCA baselines). Motivated by these findings, we introduce gated predictive autoencoders that learn to select predictable components, mimicking the beneficial feature-selection behavior observed in over-parameterized PCA. On this toy testbed, the proposed gated model is stable across noise levels and matches or outperforms JEPA.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Diffusion Reinforcement Learning via Centered Reward Distillation
Yuanzhi Zhu, Xi Wang, Stéphane Lathuilière et al. (+1 more)
Diffusion and flow models achieve State-Of-The-Art (SOTA) generative performance, yet many practically important behaviors such as fine-grained
prompt fidelity, compositional correctness, and text rendering are weakly specified by score or flow matching pretraining objectives. Reinforcement Learning (RL) fine-tuning with external, black-box rewards is a natural remedy, but
diffusion RL is often brittle. Trajectory-based methods incur high
memory cost and high-variance gradient estimates; forward-process approaches converge faster but can suffer from distribution drift, and hence reward hacking. In this work, we present Centered Reward
Distillation (CRD), a
diffusion RL framework derived from KL-regularized reward maximization built on forward-process-based fine-tuning. The key insight is that the intractable normalizing constant cancels under within-
prompt centering, yielding a well-posed reward-matching objective. To enable reliable text-to-image fine-tuning, we introduce techniques that explicitly control distribution drift: (i) decoupling the sampler from the moving reference to prevent ratio-signal collapse, (ii) KL anchoring to a CFG-guided pretrained model to control long-run drift and align with the inference-time semantics of the pre-trained model, and (iii) reward-adaptive KL strength to accelerate early learning under large KL regularization while reducing late-stage exploitation of reward-model loopholes. Experiments on text-to-image post-training with GenEval and OCR rewards show that CRD achieves competitive SOTA reward optimization results with fast convergence and reduced reward hacking, as validated on unseen preference metrics.
Read the full abstract ▾
All authors Yuanzhi Zhu, Xi Wang, Stéphane Lathuilière, Vicky Kalogeiton
Machine Learning
SVD Contextual Sparsity Predictors for Fast LLM Inference
Georgii Serbin, Kirill Koshkin, Zhongao Sun et al. (+2 more)
Contextual sparsity is one of the approaches used to reduce computational complexity in the inference process of large
language models (LLMs). Existing techniques for efficient LLM inference acceleration based on contextual sparsity with minimal accuracy degradation require training sparse pattern predictors. This paper presents a framework for accelerating inference of ReGLU-based feed-forward networks (FFNs) within LLMs. The proposed framework provides a fast, training-free method for building sparse pattern predictors using truncation-aware singular value decomposition (SVD) of the gate projection
matrix, along with a threshold calibration
algorithm, and inference executors supporting conditional computation on CUDA and CANN devices. Experiments on three sparse LLMs with an average activation sparsity level of 90% in the FFNs demonstrate up to a 1.8x reduction in end-to-end decoding time while maintaining less than 1% degradation in
benchmark scores on tasks involving complex math and code generation. This work advances the deployment of LLMs on edge devices.
Read the full abstract ▾
All authors Georgii Serbin, Kirill Koshkin, Zhongao Sun, Anastasiya Bistrigova, C.C. Korikov
Machine Learning
ST-ResGAT: Explainable Spatio-Temporal Graph Neural Network for Road Condition Prediction and Priority-Driven Maintenance
Mohsin Mahmud Topu, Azmine Toushik Wasi, Mahfuz Ahmed Anik et al. (+1 more)
Climate-vulnerable road networks require a paradigm shift from reactive, fix-on-failure repairs to predictive, decision-ready maintenance. This paper introduces ST-ResGAT, a novel Spatio-Temporal Residual Graph
Attention Network that fuses residual graph-
attention encoding with GRU temporal aggregation to forecast pavement deterioration. Engineered for resource-constrained deployment, the framework translates continuous Pavement Condition Index (PCI) forecasts directly into the American Society for Testing and Materials (ASTM)-compliant maintenance priorities. Using a real-world inspection
dataset of 750 segments in Sylhet, Bangladesh (2021-2024), ST-ResGAT significantly outperforms traditional non-spatial machine learning baselines, achieving exceptional predictive fidelity (R2 = 0.93, RMSE = 2.72). Crucially, ablation testing confirmed the mathematical necessity of modeling topological neighbor effects, proving that structural decay acts as a spatial contagion. Uniquely, we integrate GNNExplainer to unbox the model, demonstrating that its learned priorities align perfectly with established physical engineering theory. Furthermore, we quantify classification safety: achieving 85.5% exact ASTM class agreement and 100% adjacent-class containment, ensuring bounded, engineer-safe predictions. To connect model outputs to policy, we generate localized longitudinal maintenance profiles, perform
climate stress-testing, and derive Pareto sustainability frontiers. ST-ResGAT therefore offers a practical, explainable, and sustainable blueprint for intelligent infrastructure management in high-risk, low-resource geological settings.
Read the full abstract ▾
All authors Mohsin Mahmud Topu, Azmine Toushik Wasi, Mahfuz Ahmed Anik, MD Manjurul Ahsan
Machine Learning
Concisely Explaining the Doubt: Minimum-Size Abductive Explanations for Linear Models with a Reject Option
Gleilson Pedro Fernandes and Thiago Alves Rocha
Trustworthiness in artificial intelligence depends not only on what a model decides, but also on how it handles and explains cases in which a reliable decision cannot be made. In critical domains such as healthcare and finance, a reject option allows the model to abstain when evidence is insufficient, making it essential to explain why an instance is rejected in order to support informed human intervention. In these settings, explanations must not only be interpretable, but also faithful to the underlying model and computationally efficient enough to support real-time decision making. Abductive explanations guarantee fidelity, but their exact computation is known to be NP-hard for many classes of models, limiting their practical applicability. Computing minimum-size abductive explanations is an even more challenging problem, as it requires
reasoning not only about fidelity but also about optimality. Prior work has addressed this challenge in restricted settings, including log-linear-time
algorithms for computing minimum-size abductive explanations in linear models without rejection, as well as a
polynomial-time method based on linear programming for computing abductive explanations, without guarantees of minimum size, for linear models with a reject option. In this work, we bridge these lines of research by computing minimum-size abductive explanations for linear models with a reject option. For accepted instances, we adapt the log-linear
algorithm to efficiently compute optimal explanations. For rejected instances, we formulate a 0-1 integer linear programming problem that characterizes minimum-size abductive explanations of rejection. Although this formulation is NP-hard in theory, our experimental results show that it is consistently more efficient in practice than the linear-programming-based approach that does not guarantee minimum-size explanations.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Maximin Robust Bayesian Experimental Design
Hany Abdulsamad, Sahel Iqbal, Christian A. Naesseth et al. (+2 more)
We address the brittleness of
Bayesian experimental design under model misspecification by formulating the problem as a max–min game between the experimenter and an
adversarial nature subject to information-theoretic constraints. We demonstrate that this approach yields a robust objective governed by Sibson's $\alpha$-mutual information~(MI), which identifies the $\alpha$-tilted posterior as the robust belief update and establishes the Rényi divergence as the appropriate measure of conditional information gain. To mitigate the bias and variance of nested Monte Carlo
estimators needed to estimate Sibson's $\alpha$-MI, we adopt a PAC-Bayes framework to search over
stochastic design policies, yielding rigorous high-probability lower bounds on the robust expected information gain that explicitly control finite-sample error.
Read the full abstract ▾
All authors Hany Abdulsamad, Sahel Iqbal, Christian A. Naesseth, Takuo Matsubara, Adrien Corenflos
Machine Learning
Not All Latent Spaces Are Flat: Hyperbolic Concept Control
Maria Rosaria Briglia, Simone Facchiano, Paolo Cursi et al. (+6 more)
As modern text-to-image (T2I) models draw closer to synthesizing highly realistic content, the threat of unsafe content generation grows, and it becomes paramount to exercise control. Existing approaches steer these models by applying Euclidean adjustments to text
embeddings, redirecting the generation away from unsafe concepts. In this work, we introduce
hyperbolic control (HyCon): a novel control mechanism based on parallel transport that leverages semantically aligned
hyperbolic representation space to yield more expressive and stable manipulation of concepts. HyCon reuses off-the-shelf generative models and a state-of-the-art
hyperbolic text encoder, linked via a lightweight adapter. HyCon achieves state-of-the-art results across four safety
benchmarks and four T2I backbones, showing that
hyperbolic steering is a practical and flexible approach for more reliable T2I generation.
Read the full abstract ▾
All authors Maria Rosaria Briglia, Simone Facchiano, Paolo Cursi, Alessio Sampieri, Emanuele Rodolà, Guido Maria D'Amely di Melendugno, Luca Franco, Fabio Galasso, Iacopo Masi
Machine Learning
Soft Mean Expected Calibration Error (SMECE): A Calibration Metric for Probabilistic Labels
Michael Leznik
The Expected Calibration Error (ece), the dominant calibration metric in machine learning, compares predicted probabilities against empirical frequencies of binary outcomes. This is appropriate when labels are binary events. However, many modern settings produce labels that are themselves probabilities rather than binary outcomes: a radiologist's stated confidence, a teacher model's soft output in knowledge
distillation, a class posterior derived from a generative model, or an annotator agreement fraction. In these settings, ece commits a category error - it discards the probabilistic information in the label by forcing it into a binary comparison. The result is not a noisy approximation that more data will correct. It is a structural misalignment that persists and converges to the wrong answer with increasing precision as sample size grows. We introduce the Soft Mean Expected Calibration Error (smece), a calibration metric for settings where labels are of probabilistic nature. The modification to the ece formula is one line: replace the empirical hard-label fraction in each prediction bin with the mean probability label of the samples in that bin. smece reduces exactly to ece when labels are binary, making it a strict generalisation.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AR
Evaluating Four FPGA-accelerated Space Use Cases based on Neural Network Algorithms for On-board Inference
Pedro Antunes, Muhammad Ihsan Al Hafiz, Jonah Ekelund et al. (+4 more)
Space missions increasingly deploy high-fidelity sensors that produce data volumes exceeding onboard buffering and downlink capacity. This work evaluates FPGA acceleration of neural networks (NNs) across four space use cases on the AMD ZCU104 board. We use Vitis AI (AMD DPU) and Vitis HLS to implement inference, quantify throughput and energy, and expose toolchain and architectural constraints relevant to deployment. Vitis AI achieves up to 34.16$\times$ higher inference rate than the embedded ARM CPU baseline, while custom HLS designs reach up to 5.4$\times$ speedup and add support for operators (e.g., sigmoids, 3D layers) absent in the DPU. For these implementations, measured MPSoC inference power spans 1.5-6.75 W, reducing energy per inference versus CPU execution in all use cases. These results show that NN FPGA acceleration can enable onboard filtering, compression, and event detection, easing downlink pressure in future missions.
Read the full abstract ▾
All authors Pedro Antunes, Muhammad Ihsan Al Hafiz, Jonah Ekelund, Ekaterina Dineva, George Miloshevich, Panagiotis Gonidakis, Artur Podobas
Machine Learning
Understanding the Emergence of Seemingly Useless Features in Next-Token Predictors
Mark Rofin, Jalal Naghiyev and Michael Hahn
Trained
Transformers have been shown to compute abstract features that appear redundant for predicting the immediate next
token. We identify which components of the gradient signal from the next-
token prediction objective give rise to this phenomenon, and we propose a method to estimate the influence of those components on the emergence of specific features. After validating our approach on toy tasks, we use it to interpret the origins of the world model in OthelloGPT and syntactic features in a small
language model. Finally, we apply our framework to a pretrained LLM, showing that features with extremely high or low influence on future
tokens tend to be related to formal
reasoning domains such as code. Overall, our work takes a step toward understanding hidden features of
Transformers through the lens of their development during training.
Read the full abstract ▾
Machine Learning
Bootstrapped Physically-Primed Neural Networks for Robust T2 Distribution Estimation in Low-SNR Pancreatic MRI
Hadas Ben Atya, Nicole Abramenkov, Noa Mashiah et al. (+4 more)
Estimating multi-component T2 relaxation distributions from Multi-Echo
Spin Echo (MESE) MRI is a severely ill-posed inverse problem, traditionally solved using regularized non-negative least squares (NNLS). In abdominal imaging, particularly the pancreas, low SNR and residual uncorrelated noise challenge classical solvers and deterministic deep learning models. We introduce a bootstrap-based inference framework for robust distributional T2 estimation that performs
stochastic resampling of the echo train and aggregates predictions across multiple subsets. This treats the acquisition as a distribution rather than a fixed input, yielding variance-reduced, physically consistent estimates and converting deterministic relaxometry networks into probabilistic ensemble predictors. Applied to the P2T2 architecture, our method uses inference-time bootstrapping to smooth noise artifacts and enhance fidelity to the underlying relaxation distribution. Noninvasive pancreatic evaluation is limited by location and biopsy risks, highlighting the need for biomarkers capable of capturing early pathophysiological changes. In type 1 diabetes (T1DM), progressive beta-
cell destruction begins years before overt hyperglycemia, yet current imaging cannot assess early islet decline. We evaluate clinical utility via a test-retest reproducibility study (N=7) and a T1DM versus healthy differentiation task (N=8). Our approach achieves the lowest Wasserstein distances across repeated scans and superior sensitivity to physiology-driven shifts in the relaxation-time distribution, outperforming NNLS and deterministic deep learning baselines. These results establish inference-time bootstrapping as an effective enhancement for quantitative T2 relaxometry in low-SNR abdominal imaging.
Read the full abstract ▾
All authors Hadas Ben Atya, Nicole Abramenkov, Noa Mashiah, Luise Brock, Daphna Link Sourani, Ram Weiss, Moti Freiman
Machine Learning · Cross-listed from cs.CL
CMHL: Contrastive Multi-Head Learning for Emotionally Consistent Text Classification
Menna Elgabry, Ali Hamdi and Khaled Shaban
Textual
Emotion Classification (TEC) is one of the most difficult NLP tasks. State of the art approaches rely on Large
language models (LLMs) and multi-model ensembles. In this study, we challenge the assumption that larger scale or more complex models are necessary for improved performance. In order to improve logical consistency, We introduce CMHL, a novel single-model architecture that explicitly models the logical structure of
emotions through three key innovations: (1) multi-task learning that jointly predicts primary
emotions, valence, and intensity, (2) psychologically-grounded auxiliary supervision derived from Russell's circumplex model, and (3) a novel contrastive contradiction loss that enforces emotional consistency by penalizing mutually incompatible predictions (e.g., simultaneous high confidence in joy and anger). With just 125M parameters, our model outperforms 56x larger LLMs and sLM ensembles with a new state-of-the-art F1 score of 93.75% compared to (86.13%-93.2%) on the dair-ai
Emotion dataset. We further show cross domain generalization on the Reddit Suicide Watch and Mental Health Collection
dataset (SWMH), outperforming domain-specific models like MentalBERT and MentalRoBERTa with an F1 score of 72.50% compared to (68.16%-72.16%) + a 73.30% recall compared to (67.05%-70.89%) that translates to enhanced sensitivity for detecting mental health distress. Our work establishes that architectural intelligence (not parameter count) drives progress in TEC. By
embedding psychological priors and explicit consistency constraints, a well-designed single model can outperform both massive LLMs and complex ensembles, offering a efficient, interpretable, and clinically-relevant paradigm for affective computing.
Read the full abstract ▾
Machine Learning
Enhancing Mental Health Classification with Layer-Attentive Residuals and Contrastive Feature Learning
Menna Elgabry, Ali Hamdi and Khaled Shaban
The classification of mental health is challenging for a variety of reasons. For one, there is overlap between the mental health issues. In addition, the signs of mental health issues depend on the context of the situation, making classification difficult. Although fine-tuning
transformers has improved the performance for mental health classification, standard cross-
entropy training tends to create entangled feature spaces and fails to utilize all the information the
transformers contain. We present a new framework that focuses on representations to improve mental health classification. This is done using two methods. First, layer-attentive residual aggregation which works on residual connections to to weigh and fuse representations from all
transformer layers while maintaining high-level semantics. Second, supervised contrastive feature learning uses temperature-scaled supervised contrastive learning with progressive weighting to increase the
geometric margin between confusable mental health problems and decrease class overlap by restructuring the feature space. With a score of 74.36%, the proposed method is the best performing on the SWMH
benchmark and outperforms models that are domain-specialized, such as MentalBERT and MentalRoBERTa by margins of (3.25% - 2.2%) and 2.41 recall points over the highest achieving model. These findings show that domain-adaptive pretraining for mental health text classification can be surpassed by carefully designed representation geometry and layer-aware residual integration, which also provide enhanced interpretability through learnt layer importance.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Self-Supervised Uncertainty Estimation For Super-Resolution of Satellite Images
Zhe Zheng, Valéry Dewil and Pablo Arias
Super-resolution (SR) of satellite imagery is challenging due to the lack of paired low-/high-resolution data. Recent self-supervised SR methods overcome this limitation by exploiting the temporal redundancy in burst observations, but they lack a mechanism to quantify uncertainty in the reconstruction. In this work, we introduce a novel self-supervised loss that allows to estimate uncertainty in image super-resolution without ever accessing the ground-truth high-resolution data. We adopt a decision-theoretic perspective and show that minimizing the corresponding
Bayesian risk yields the posterior mean and variance as optimal
estimators. We validate our approach on a synthetic SkySat L1B
dataset and demonstrate that it produces calibrated uncertainty estimates comparable to supervised methods. Our work bridges self-supervised restoration with uncertainty quantification, making a practical framework for uncertainty-aware image reconstruction.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
MotionCFG: Boosting Motion Dynamics via Semantic Motion Sharpening
Byungjun Kim, Soobin Um and Jong Chul Ye
Despite recent advancements in Text-to-
Video (T2V) synthesis, generating high-fidelity and dynamic motion remains a significant challenge. Existing methods primarily rely on Classifier-Free Guidance (CFG), but standard CFG treats all semantic dimensions uniformly and provides no mechanism to selectively resolve ambiguity along the motion axis. To address this, we propose MotionCFG, a training-free approach that introduces selective guidance along the motion subspace of the condition
embedding. Specifically, we construct a motion-perturbed negative condition by injecting Gaussian noise into motion-related
embeddings, and steer generation away from it to selectively amplify motion intent. We show theoretically that this procedure implicitly approximates Semantic
Laplacian Sharpening, a second-order correction that suppresses motion-ambiguous regions of the score landscape and amplifies well-resolved dynamic peaks. Combined with a piecewise guidance schedule that confines intervention to the early denoising steps, MotionCFG consistently improves motion dynamics across state-of-the-art T2V frameworks with negligible overhead. We further demonstrate that this
Laplacian sharpening principle generalizes beyond motion, effectively steering complex, non-linear concepts such as precise object numerosity that are typically difficult to modulate via standard text-based guidance. Our project page is available at https://motion-cfg.vercel.app/.
Read the full abstract ▾
Machine Learning · Cross-listed from q-fin.CP
Conditioning on a Volatility Proxy Compresses the Apparent Timescale of Collective Market Correlation
Yuda Bi and Vince D Calhoun
We address the attribution problem for apparent slow collective dynamics: is the observed persistence intrinsic, or inherited from a persistent driver? For the leading
eigenvalue fraction $\psi_1=\lambda_{\max}/N$ of S&P 500 60-day rolling correlation
matrices ($237$ stocks, 2004–2023), a VIX-coupled Ornstein–Uhlenbeck model reduces the effective relaxation time from $298$ to $61$ trading days and improves the fit over bare mean reversion by $\Delta$BIC$=109$. On the decomposition sample, an informational residual of $\log(\mathrm{VIX})$ alone retains most of that gain ($\Delta$BIC$=78.6$), whereas a mechanical VIX proxy alone does not improve the fit. Autocorrelation-matched placebo fields fail ($\Delta$BIC$_{\max}=2.7$), disjoint weekly reconstructions still favor the field-coupled model ($\Delta$BIC$=140$–$151$), and six anchored chronological holdouts preserve the out-of-sample advantage. Quiet-regime and field-stripped residual autocorrelation controls show the same collapse of persistence. Stronger hidden-variable extensions remain only partially supported. Within the tested
stochastic class, conditioning on the observed VIX proxy absorbs most of the apparent slow dynamics.
Read the full abstract ▾
Machine Learning
Gated Graph Attention Networks for Predicting Duration of Large Scale Power Outages Induced by Natural Disasters
Chenghao Duan, Chuanyi Ji, Anwar Walid et al. (+1 more)
The occurrence of large-scale power outages induced by natural disasters has been on the rise in a changing
climate. Such power outages often last extended durations, causing substantial financial losses and socioeconomic impacts to customers. Accurate estimation of outage duration is thus critical for enhancing the resilience of energy infrastructure under severe weather. We formulate such a task as a machine learning (ML) problem with focus on unique real-world challenges: high-order spatial dependency in the data, a moderate number of large-scale outage events, heterogeneous types of such events, and different impacts in a region within each event. To address these challenges, we develop a Bimodal Gated Graph
Attention Network (BiGGAT), a graph-based neural network model, that integrates a Graph
Attention Network (GAT) with a Gated Recurrent Unit (GRU) to capture the complex spatial characteristics. We evaluate the approach in a setting of inductive learning, using large-scale power outage data from six major hurricanes in the Southeastern United States. Experimental results demonstrate that BiGGAT achieves a superior performance compared to
benchmark models.
Read the full abstract ▾
All authors Chenghao Duan, Chuanyi Ji, Anwar Walid, Scott Ganz
Machine Learning · Cross-listed from cs.MA
A Benchmark for Multi-Party Negotiation Games from Real Negotiation Data
Leo Benac, Jonas Raedler, Zilin Ma et al. (+1 more)
Many real-world multi-party negotiations unfold as sequences of binding, action-level commitments rather than a single final outcome, yet this regime remains under-studied in existing
benchmarks. We introduce a
benchmark and evaluation framework for this setting, combining a configurable negotiation game generator with document-grounded instances derived from a
climate negotiation exercise. We also provide several baseline solvers. Exact evaluation on small games and comparative evaluation on larger instances show that no solver dominates across regimes; performance depends on the structural properties of the game. These results motivate the creation of novel negotiation methods that value partial commitments robustly across diverse strategic regimes. Code and data for the
benchmark are available at: https://anonymous.4open.science/r/negotiation_MARL-46B8
Read the full abstract ▾
All authors Leo Benac, Jonas Raedler, Zilin Ma, Finale Doshi-Velez
Machine Learning · Cross-listed from cs.CV
TMPDiff: Temporal Mixed-Precision for Diffusion Models
Basile Lewandowski, Simon Kurz, Aditya Shankar et al. (+3 more)
Diffusion models are the go-to method for Text-to-Image generation, but their iterative denoising processes has high inference latency. Quantization reduces compute time by using lower bitwidths, but applies a fixed precision across all denoising timesteps, leaving an entire optimization axis unexplored. We propose TMPDiff, a temporal mixed-precision framework for
diffusion models that assigns different numeric precision to different denoising timesteps. We hypothesize that quantization errors accumulate additively across timesteps, which we then validate experimentally. Based on our observations, we develop an adaptive bisectioning-based
algorithm, which assigns per-step precisions with linear evaluation complexity, reducing an otherwise exponential search problem. Across four state-of-the-art
diffusion models and three
datasets, TMPDiff consistently outperforms uniform-precision baselines at matched speedup, achieving 10 to 20% improvement in perceptual quality. On FLUX.1-dev, TMPDiff achieves 90% SSIM relative to the full-precision model at a speedup of 2.5x over 16-bit inference.
Read the full abstract ▾
All authors Basile Lewandowski, Simon Kurz, Aditya Shankar, Robert Birke, Jian-Jia Chen, Lydia Y. Chen
Machine Learning · Cross-listed from cs.CL
NepTam: A Nepali-Tamang Parallel Corpus and Baseline Machine Translation Experiments
Rupak Raj Ghimire, Bipesh Subedi, Balaram Prasain et al. (+7 more)
Modern Translation Systems heavily rely on high-quality, large parallel
datasets for state-of-the-art performance. However, such resources are largely unavailable for most of the South Asian
languages. Among them, Nepali and Tamang fall into such category, with Tamang being among the least digitally resourced
languages in the region. This work addresses the gap by developing NepTam20K, a 20K gold standard parallel corpus, and NepTam80K, an 80K synthetic Nepali-Tamang parallel corpus, both sentence-aligned and designed to support machine translation. The
datasets were created through a pipeline involving data scraping from Nepali news and online sources, pre-processing, semantic filtering, balancing for tense and polarity (in NepTam20K
dataset), expert translation into Tamang by native speakers of the
language, and verification by an expert Tamang linguist. The
dataset covers five domains: Agriculture, Health, Education and Technology, Culture, and General Communication. To evaluate the
dataset, baseline machine translation experiments were carried out using various
multilingual pre-trained models: mBART, M2M-100, NLLB-200, and a vanilla
Transformer model. The fine-tuning on the NLLB-200 achieved the highest sacreBLEU scores of 40.92 (Nepali-Tamang) and 45.26 (Tamang-Nepali).
Read the full abstract ▾
All authors Rupak Raj Ghimire, Bipesh Subedi, Balaram Prasain, Prakash Poudyal, Praveen Acharya, Nischal Karki, Rupak Tiwari, Rishikesh Kumar Sharma, Jenny Poudel, Bal Krishna Bal
Machine Learning · Cross-listed from math.OC
Schrödinger Bridge Over A Compact Connected Lie Group
Hamza Mahmood, Abhishek Halder and Adeel Akhtar
This work studies the Schrödinger bridge problem for the kinematic equation on a compact connected Lie group. The objective is to steer a controlled
diffusion between given initial and terminal densities supported over the Lie group while minimizing the control effort. We develop a coordinate-free formulation of this
stochastic optimal control problem that respects the underlying
geometric structure of the Lie group, thereby avoiding limitations associated with local parameterizations or
embeddings in Euclidean spaces. We establish the existence and uniqueness of solution to the corresponding Schrödinger system. Our results are constructive in that they derive a
geometric controller that optimally interpolates probability densities supported over the Lie group. To illustrate the results, we provide numerical examples on $\mathsf{SO}(2)$ and $\mathsf{SO}(3)$. The codes and animations are publicly available at https://github.com/gradslab/SbpLieGroups.git .
Read the full abstract ▾