Machine Learning
Measuring Effective Data Resolution in Guided Diffusion Posteriors
Ridham Patel, Defu Cao, Jiacheng Pang et al. (+1 more)
Guided
diffusion samplers are increasingly used to reconstruct physical fields from sparse observations, but standard diagnostics do not say how much of the reconstruction was actually determined by the data. We introduce effective data resolution for black-box generative posteriors: a comparison between the resolution warranted by the inverse problem, $\mathrm{dof}_{\mathrm{ref}}$, and the resolution realised by the sampler, $\mathrm{dof}_{\mathrm{samp}}$. A perturbation
estimator measures $\mathrm{dof}_{\mathrm{samp}}$ and the spatial map $R(x,x)$ from sampler queries alone. We validate the
estimator against exact references and use it to study guided
diffusion. The resulting measurements show that guidance weight can strongly alter apparent information transfer, that mean, spread and resolution are not jointly corrected by one weight even with an exact prior and score, and that resolution fidelity does not follow reliably from the apparent principledness of a guidance rule.
Read the full abstract ▾
All authors Ridham Patel, Defu Cao, Jiacheng Pang, Yan Liu
Machine Learning
MaDeL: Manifold-Decomposed Feature Losses for Generative Modeling
Beomsu Kim, Jong Chul Ye and Kwanyoung Kim
Generative models are often trained with isotropic objectives such as mean-squared error. For data concentrated near a low-dimensional
manifold, however, such losses conflate displacement along the
manifold, which may represent valid variation, with displacement away from it, which produces invalid samples. This mismatch is especially problematic in sparse, highly constrained domains, where ambient-space regression can encourage off-
manifold interpolation. We ask whether a generative objective can distinguish
manifold-parallel variation from
manifold-orthogonal deviation directly from data, without explicitly estimating the
manifold. We introduce a
manifold-decomposed feature loss (MaDeL) that learns complementary representations from corrupted observations: one is trained to recover the clean sample, while the other is trained to recover the corruption. We show that, under a feature bottleneck, their Jacobians align with the tangent and normal spaces, exactly for linear
manifolds and locally for smooth
manifolds. Together, these representations define an anisotropic objective that separately measures intrinsic variation and off-
manifold deviation. Across synthetic, Earth and
climate science, and torsion-angle
benchmarks, MaDeL improves support recovery and average angular $W_1$ under single-step sampling; on
protein backbones, it reduces steric clashes across one- and few-step sampling budgets.
Read the full abstract ▾
Machine Learning
Specific Algorithmic Interpretability of Neural Networks: A Case Study on Textures
Yanglin Zhang, Anneke von Seeger, Gilad Lerman et al. (+1 more)
We develop a principled framework for constructing neural networks whose specific parameter realizations admit an explicit algorithmic interpretation. Existing
algorithm-inspired architectures can explain the computational structure of a network, yet after standard training the learned parameters need not retain a clear relation to the motivating
algorithm. We address this gap as follows. First, we model each data point as a sample of a class-dependent
stochastic process and assume that statistics of this process can be estimated from a single sample and these statistics are sufficient to distinguish the classes. We then construct a neural network whose initial parameters exactly implement an
algorithm for estimating these statistics, making the network fully interpretable. To account for mismatch between the idealized model and real data, we fine-tune this network while controlling its deviation from the algorithmic initialization. The trained network hence roughly retains the interpretation of the initial network. A PAC-
Bayesian analysis yields a uniform generalization bound whose complexity term scales with the fine-tuning radius, providing a statistical motivation for our approach. We instantiate the framework for texture classification using the scattering transform to estimate the discriminative statistics.
Read the full abstract ▾
All authors Yanglin Zhang, Anneke von Seeger, Gilad Lerman, Ron Levie
Machine Learning · Cross-listed from cs.AI
TimeNet: An Extensible Unified Data Infrastructure for Next-Generation Temporal Foundation Models
Martin Maritsch, Timo Stoffregen, Thomas Kaar et al. (+36 more)
Temporal Foundation Models (TFMs) aim to generalize across domains,
datasets, and tasks. Yet, their development remains constrained by fragmented, task-specific data formats, annotations, and processing pipelines. We introduce TimeNet, an open-source data standard and scalable infrastructure that decouples temporal data from task definitions and represents signals, metadata, annotations, and supervision in a shared, extensible data model. TimeNet supports
multimodal signals with regular, irregular, or ordinal time axes and expresses different task families (including classification, forecasting, temporal localization, question answering, generation, and editing) as reusable views over the same recordings. This shared representation enables heterogeneous time-series
datasets to be combined for large-scale model training across domains, modalities, and tasks. We demonstrate TimeNet by transcoding
datasets with 1.5M task instances spanning diverse domains, modalities, temporal scales, and forms of supervision, while retaining practical I/O performance relative to native formats. TimeNet enables an existing TFN training pipeline to support joint training on a configurable number of heterogeneous
datasets through configuration changes alone. We show this capability by training TFM across multiple
datasets and tasks, obtaining a 14% F1 score improvement compared with models trained on individual
datasets. These results show that TimeNet provides the data and systems foundation needed to move beyond task- and
dataset-specific TFMs toward models that can learn jointly across heterogeneous domains, modalities, temporal scales, and forms of supervision from a common data model.
Read the full abstract ▾
All authors Martin Maritsch, Timo Stoffregen, Thomas Kaar, Behsad Riemer, Maxwell A. Xu, Max Rosenblattl, Juncheng Liu, Nicolas Zumarraga, Yu Yvonne Wu, Denys Herasymuk, Sparsh Rastogi, Hyungjun Yoon, Bosong Huang, Arvind Pillai, Dmytro Lopushanskyy, Tony Chen, Robin Deuber, Yichen Liu, Shvat Messica, Dan Li, Jian Lou, Yuwei Zhang, Jaeho Kim, Renée Rosillo Garcia, Fan Wu, Robert Müller, Elgar Fleisch, Flora D. Salim, Dimitris Spathis, Yuzhe Yang, Aaqib Saeed, Daniel McDuff, Ming Jin, Markus Kreft, Kevin O'Sullivan, Robert Jakob, Azul Garza, Paul Schmiedmayer, Patrick Langer
Machine Learning
CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations
Jeahan Jung, Heechang Kim, Hyomin Shin et al. (+1 more)
Despite the growing popularity of physics-informed neural networks (PINNs), their applicability in the long-time integration of partial differential equations (PDEs) remains constrained. We argue that this problem stems from the lack of consideration of temporal causality in the original PINN formulation, resulting in a bias towards satisfying governing equations at later times before learning the initial condition and hence leading to erroneous solutions. To this end, we propose a novel method that seamlessly integrates temporal causality into the training process. Drawing inspiration from classical numerical methods where the temporal causality is reflected, we divide the time domain into nonoverlapping subintervals, assign a unique neural network to each subinterval, and construct a loss function founded on the integral form of PDEs within these subintervals. The proposed networks undergo sequential training, beginning with the initial time step. Our method demonstrates significant improvement in accuracy for long-time simulations of various PDE problems where the original PINN method fails while it requires less computational cost and
memory compared to the PINN method. A parallelization
algorithm is provided to further enhance the computational efficiency, showing a significant speedup for solving time-dependent PDEs.
Read the full abstract ▾
All authors Jeahan Jung, Heechang Kim, Hyomin Shin, Minseok Choi
Machine Learning · Cross-listed from cs.CL
Saying, Not Knowing: Aggressively GGUF-Quantized Small Language Models Still Write Rare Words They Can No Longer Define
Saurabh Kumar Singh, Yogeshwar Singh Dadwhal and Malhar Vedak
Post-training quantization to the GGUF format's mixed-precision K-quants is commonly how open-weight
language models reach consumer hardware, yet its effect on fine-grained lexical competence is uncharacterized. We audit 27 quantized artifacts across 13 families and four architecture backbones, 0.35B-14B parameters, evaluated down their published ladder to Q2_K (about 2.6 bits per weight), on 429 frequency-validated rare English words under two probes: surface inclusion of a
prompt-supplied word and its one-sentence definition, scored by a tiered multi-synonym matcher, its error measured by a blind LLM-judge census of every definition, with human verification. Three regimes emerge at Q2: total collapse into unusable builds, severe semantic dissociation in sub-2B models, and mostly robust preservation above about 3B. In every sub-2B artifact, definitions fall 20-67% below the artifact's baseline, typically several times the inclusion loss. Two controls separate rarity from task difficulty: within the rare set, loss rises with rarity in six of seven sub-2B artifacts, and on a 100-word common-word set rare words lose more than common words in all eight, significantly in six. Tokenizer vocabulary size does not predict the damage (Spearman rho=0.12); parameter count dominates (rho=0.72), confirmed within five of six same-tokenizer families. Q4_K_M remains lexically clean at >=1B. The damage is frequency-graded, provider-dependent, and not calibrated by WikiText-2 perplexity: across nine artifact-matched ladders, near-identical Q2 penalties (44.7%/47.6%) separate an artifact keeping its definitions (3.6%) from one losing them (43.6%). Aggressively quantized small models can keep generating fluent text while no longer knowing what it means, risking hardware-constrained deployments in domains where semantics carries consequences. Validation must be per artifact.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
JASPER: Special Session on Joint Reliability And Security Assessment of SPlit Computing for Edge Robustness
Enrico Magliano, Giuseppe Esposito, Amir Hossein Shahdadian et al. (+13 more)
Split Computing (SC) enables efficient deployment of Deep Neural Networks (DNNs) by partitioning inference between edge devices and cloud servers. However, intermediate feature representations are simultaneously exposed to hardware faults and
adversarial attacks, which are traditionally evaluated independently. This paper presents a unified framework for the joint assessment of reliability and security in Split Computing. First, reliability is characterized through
neuron-level fault injection using the Mean Relative Accuracy Degradation (MRAD) while security through feature-map-aware
adversarial attacks simulations using the Attack Success Rate (ASR). Based on these complementary analyses, the Joint Vulnerability Score (JVS) is introduced, along with a confidence-aware extension that jointly captures prediction errors and confidence degradation. The framework is evaluated on ten Split Computing configurations based on ResNet-50 trained on ILSVRC-2012. Experimental results show substantial differences across compression strategies, with MRAD ranging from 44.3% to 61.2% under fault injection, while
adversarial attacks achieve up to 98.8% ASR. Furthermore, the proposed joint metrics reveal vulnerability trends that remain hidden when reliability and security are analyzed independently, providing a more comprehensive methodology for designing dependable Split Computing systems.
Read the full abstract ▾
All authors Enrico Magliano, Giuseppe Esposito, Amir Hossein Shahdadian, Rama Mounika Kodamanchili, Juan David Guerrero Balaguera, Juan Esteban Rodriguez Condia, Annachiara Ruospo, Roberta Siciliano, Stefano Di Carlo, Maksim Jenihhin, Marco Levorato, Alessandro Savino, Matteo Sonza Reorda, Christian Herglotz, Michael Hübner, Mahdi Taheri
Machine Learning · Cross-listed from stat.ML
Gaussian Flow Dynamics: Simulation-Free Neural SDE Learning Beyond One-Time Marginals
Grigory Bartosh and Christian A. Naesseth
Simulation-free training of latent
Stochastic Differential Equations (SDEs) relies on a variational posterior process whose one-time marginals are tractable, typically Gaussian. Such marginals, however, do not determine the underlying dynamics: many processes share the same marginals while differing in their temporal structure, and existing parameterizations fix this structure implicitly, which restricts the posterior family and biases the learned model. We introduce Gaussian flow dynamics, which construct
stochastic processes directly from smoothly evolving Gaussian marginals while making the marginal-preserving, or gauge, degrees of freedom explicit and parameterizable. The construction admits state-dependent
diffusion coefficients and recovers every linear SDE with additive noise and a non-degenerate Gaussian initial distribution. Building on it, we propose Gauge Matching, a simulation-free method for latent SDE learning that combines Gaussian flow dynamics with the SDE Matching objective. Gauge Matching costs at most quadratically in the latent dimension per step, like SDE Matching, but learns the temporal structure of the posterior beyond its one-time marginals. It comes within a nat of Helmholtz-SDE, which computes the gauge from the prior Jacobian at cubic cost, on the linear
benchmark where the exact posterior is known, matches it on nonlinear systems, and applies where Helmholtz-SDE does not, to state-dependent noise.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Largest Rashomon sets of decision trees for robust contextual optimization
Lorenzo Bonasera and David Pisinger
Many decision trees fit the same data almost equally well, yet they can route a query point to different leaves and induce different local empirical distributions. We study decisions that meet prescribed cost, shortage or risk targets despite this predictive multiplicity. We propose the joint Rashomon and robustness optimization framework for optimal decision trees. It jointly selects an operational decision and the largest Rashomon set of trees, so that the targets hold under the local empirical distribution that every tree in this set induces at the query point. We specialize the framework to the regression setting, and we show that a tree affects the decision only through the training observations sharing the query leaf, which we call its query neighborhood. As a result, the robust problem involves only finitely many distinct constraints, which can be examined in order of increasing estimation loss. We develop a constraint generation
algorithm that combines query-path pricing with dynamic programming to identify violating neighborhoods without enumerating trees. On synthetic newsvendor instances, the
algorithm typically needs few neighborhoods and runs substantially faster than full neighborhood enumeration. On restaurant demand data, the robust orders increase the mean tolerated excess estimation loss by 17.6% and reduce the empirical conditional value-at-risk of the worst 10% of realized costs by 8.3% relative to the sample average approximation orders of the optimal tree, while the mean cost difference is not statistically significant. An interpretability analysis further shows how the retained neighborhoods explain the decision and its robustness limit.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Frame-Level Temporal Alignment for Human-to-Robot Visual Adaptation
Xizhe Zhang, Jingfeng Zhang, Zirun Zhou et al. (+1 more)
Transferring visual representations pretrained on human
videos to
robot manipulation requires learning reliable correspondences between human and
robot demonstrations. However, paired demonstrations can differ in execution rate and in the proportion of non-key frames that do not directly reflect task progress. Frames at the same relative timestamp may therefore represent different task stages, which can cause correspondence learning to fail. To address these issues, we propose Frame-Level Temporal
Alignment (FLTA), a framework that uses two temporal priors to adapt visual encoders pretrained on human
videos for
robot manipulation. It learns shared task-progress representations without frame-level correspondence annotations, allowing frames at different relative temporal positions to match. A global progress prior combines normalized temporal positions with visual similarity to construct a soft correspondence target. A local temporal order prior penalizes backward transitions while allowing stays and varying forward rates to accommodate execution-rate differences. With ResNet-50 and ViT encoders, our method achieves relative improvements of 46.93% and 65.96%, respectively, over the best baselines in average simulation success rates and achieves higher task success rates on real-world manipulation tasks. These results also suggest that effective human-
robot adaptation depends less on the number of parameters updated than on which parameters are selected. Our project page is available at https://rtx5090ultra.github.io/FLTA-Project-Page/.
Read the full abstract ▾
All authors Xizhe Zhang, Jingfeng Zhang, Zirun Zhou, Hong Jia
Machine Learning
A multi-stage probabilistic framework to estimate gas-fired generator performance during extreme winter weather
Sajjad Uddin Mahmud and Anamika Dubey
Extreme winter weather has repeatedly disrupted gas-fired power generation in the United States, yet the plant-level data needed to systematically quantify outage risk remain proprietary. Using publicly available weather and electricity demand data together with anonymized generator contingency records from the North American Electric Reliability Corporation (NERC), we develop a three-stage
Bayesian probabilistic framework for estimating winter-driven generator performance. Applied to New York State (2013–2022), the framework sequentially estimates: the hourly probability of a generator contingency event, the expected net available capacity conditioned on an event occurring, and the event duration. Colder conditions and higher electricity demand are associated with higher failure probability, lower retained capacity, and longer event duration. Under the most severe observed stress conditions, estimated mean hourly event probability reaches 24% , while expected mean net available capacity falls to 13% of nameplate rating. Full outage events have a median duration of 12.7 hours, while partial derating event duration increases from 2.4 to 7.1 hours with capacity loss severity. The proposed framework establishes a transferable baseline that utilities with access to plant-level records can directly extend to obtain more precise reliability estimates for operational planning and resource adequacy assessment.
Read the full abstract ▾
Machine Learning
A Bird's-Eye View of Iterative Reward Design
Logan Mondal Bhamidipaty, Lauren Robson, Linda Petrini et al. (+2 more)
Designing effective reward functions in RL typically requires substantial expertise and trial and error. Recent work automates this process with LLM-based systems that generate and iteratively improve reward code using policy feedback. However, these methods are often hard to compare because they differ in implementation details, feedback assumptions, and evaluation environments. To address this, we introduce a
Benchmark for Iterative Reward Design (BIRD) that expresses existing methods in a unified configuration and evaluation space. This lets us compare
algorithms directly, ablate individual design choices, and prototype new components under matched feedback conditions and policy-training budgets. Across MuJoCo, Meta-World, Assistax, and HumanoidBench, we identify a small set of simple design choices that consistently improve performance. Combining these choices yields significantly better performance than the evaluated methods from prior work. Our results highlight the strength of simple baselines and motivate further study of when additional algorithmic complexity improves iterative reward design. Code is available at https://github.com/safety-research/bird.
Read the full abstract ▾
All authors Logan Mondal Bhamidipaty, Lauren Robson, Linda Petrini, Shengrui Lyu, Kamal Ndousse
Machine Learning
Checkable NTK Positivity and Finite-Width Gradient Descent for Scalar- and Vector-Valued PINNs with Strong-Form, Weak-Form, and Nonlocal Linear Constraints
Zifan Lyu
We give checkable positive-definiteness criteria for the limiting neural tangent kernel (NTK) and high-probability finite-width gradient-descent guarantees for scalar- or vector-valued physics-informed neural networks (PINNs) with linear constraints. The constraints may be strong-form differential rows of any fixed finite order, including coupled systems, with any linear initial or boundary conditions; weak-form residual and boundary functionals; or finite-measure nonlocal observations such as integral, nonlocal-
diffusion, and Dirac-data rows, all in any dimension. For each class, positive definiteness of the limiting NTK is equivalent to a rank condition on a finite coefficient, functional, or moment
matrix of the fixed design: a certificate computed before training that detects structural zero modes. The model hypotheses are those of an ordinary two-layer PINN, a smooth nonpolynomial activation with bounded symmetric initialization, and are met by standard choices such as $\tanh$ with uniform initialization. Given a certificate, explicit width and step-size conditions ensure, with high probability, that the empirical NTK retains at least half the limiting gap and that full-batch gradient descent decreases the training loss geometrically at every iteration. Controlled experiments check the certificates and the finite-width mechanism.
Read the full abstract ▾
Machine Learning
Hierarchical Credit Assignment for RLVR on Fused Gromov-Wasserstein Geometry
Qi Yu, Ruizhong Qiu, Zhichen Zeng et al. (+6 more)
Reinforcement learning with verifiable rewards (RLVR) has been shown to improve the
reasoning capability of large
language models (LLMs) across diverse
reasoning tasks. However, group-based RLVR methods, such as GRPO, assign a uniform advantage to all
tokens within rollouts of the same outcome. While existing works refine credit assignment of GRPO based on local signals such as
token locations or
entropy, they often fail to capture the global semantic novelty of a
reasoning behavior relative to the current policy. In this work, we propose a hierarchical credit assignment approach for group-based RLVR methods, called HarA, which identifies and encourages semantically novel
reasoning behaviors during RLVR. HarA represents each sampled rollout as a distribution over the hidden states and locations of
tokens, and computes the Fused Gromov-Wasserstein (FGW) barycenters of all rollouts with the same outcome, capturing the internal
reasoning patterns in the latent space under the current policy. The semantic novelty of a
reasoning element can then be measured by its contribution to the FGW distance between the current rollout and the barycenter. While solving the FGW formulation is expensive, we introduce an anchor-guided linearization that turns it into a Wasserstein formulation solvable via the Sinkhorn
algorithm efficiently. By reweighing
token-level advantage of group-based RLVR methods based on the novelty signals, HarA highlights novel
reasoning behaviors at flexible granularities to encourage fine-grained LLM exploration. Extensive experiments across three group-based RLVR methods show that our plug-and-play method effectively enhances the exploration of LLMs, outperforming existing methods across diverse
reasoning benchmarks.
Read the full abstract ▾
All authors Qi Yu, Ruizhong Qiu, Zhichen Zeng, Xuying Ning, Yanjun Zhao, Dongqi Fu, Yinglong Xia, Hong Li, Hanghang Tong
Machine Learning · Cross-listed from eess.SY
A KKL Observer Perspective on Reservoir Computing
Anastasia Bizyaeva, Fernando Castaños and Jaime A. Moreno
Reservoir computing (RC) is a machine learning technique for data-driven modeling of dynamics for forecasting and control, primarily studied in computer science and physics literature with promising applications in neural network learning, physical computing, and neuroscience. Why reservoirs learn and how to choose good reservoir architectures are considered important open questions. We show that the RC problem is mathematically an extension of a classic problem in systems and control theory, the Kazantzis-Kravaris-Luenberger (KKL) observer design problem. As a consequence, many of the questions considered open for RC stand to benefit from a large body of theory in the mature KKL literature, non-exhaustively including on questions of
embedding, transverse stability, local and global uniqueness guarantees, and effective data-driven solution constructions. Elaborating on this connection, we show that the surprising forecasting ability of reservoirs is in fact a direct consequence of the well-known observer internal model principle, derive an upper bound on the prediction error over a fixed forecast horizon, and provide a partial explanation for why linear readout training in RC works reasonably well. This work illustrates how classical ideas from systems and control can provide strong theoretical backing and open new questions for modern machine learning methods.
Read the full abstract ▾
Machine Learning
S$^3$N: A Spherical Spiral Scanning Network for Weather Forecasting
Fan Yan, Chen Hui, Weisi Lin et al. (+4 more)
Machine learning-based weather prediction (MLWP) has achieved strong performance in global weather forecasting. Recent Hierarchical Equal Area isoLatitude Pixelation (HEALPix)-based methods use the HEALPix (HP) grid to avoid area distortion near the poles of conventional latitude-longitude (LL) grids. However, existing HP-based approaches often use pointwise mapping methods and process HP pixels within separate base faces or local windows. Consequently, the mapping may introduce reconstruction errors and cross-face communication depends on handcrafted boundary handling or shifted windows. We propose the Spherical Spiral Scanning Network (S$^3$N) to address both limitations. First, L2Proj provides a bidirectional method for mapping atmospheric fields between the LL and HP grids through an $L^2$ projection of their continuous finite-element representations. Second, the
Attention-Guided Quad-Spiral State-Space Scanning (AQSS) block uses cross-latitude
attention to guide selective state-space updates along four global pole-to-pole spiral paths. This design enables continuous information propagation across HP base-face boundaries without additional boundary-processing mechanisms. Experiments show that S$^3$N achieves better results at 4-, 7-, and 10-day lead times, and exhibits slower error growth in long-range forecasting.
Read the full abstract ▾
All authors Fan Yan, Chen Hui, Weisi Lin, Haiqi Zhu, Feng Jiang, Sun-Yuan Kung, Wei Zhang
Machine Learning · Cross-listed from physics.soc-ph
FLAT: Smoothing the Rugged Landscape for Learnable, Sample-Efficient Traffic Calibration
Haopeng Deng, Shuo He and Dayuan Wang
Calibrating microscopic traffic models for digital twins is an expensive black-box optimization problem: tuning car-following and lane-changing parameters requires a full simulation run, affording only a tight budget per recalibration window. Matching raw trajectories yields a rugged objective that sparse surrogates cannot learn, reducing sequential acquisition to near-random probing. We present FLAT, which couples what to optimize with where to sample next. An eight-dimensional behavioral fingerprint smooths the parameter-error landscape, making the objective learnable from a few dozen samples; annealed lower-confidence-bound (LCB) acquisition then spends each remaining run where it most reduces error. The surrogate, interchangeable among a Gaussian process (GP), random forest (RF), or multi-layer-perceptron (MLP) ensemble, plugs into the same LCB loop. Across six heterogeneous real-world scenes, FLAT-GP achieves the lowest scene-averaged behavioral error, winning 6/6 scenes against SPSA, GA, and CMA-ES and 5/6 against TPE under the matched budget. Some baselines need up to 4.4 times more simulations to match. Ablations show objective choice shifts final behavioral error by 81% on average, removing sequential LCB raises the six-scene mean by 20%, and surrogate choice shifts it by at most 4.2%, confirming gains trace to objective geometry and sequential allocation rather than surrogate capacity.
Read the full abstract ▾
Machine Learning
LyapuFlow: Controlling Generative Flows with Lyapunov Feedback for Inverse Problems
Minseon Gwak, Hans Hao-Hsun Hsu, Danielle C. Maddix et al. (+1 more)
Pretrained flow models are now widely used as generative priors in science and
vision, where inference-time guidance enables test-time constraints without retraining. Existing methods use projection, posterior sampling, or iterative optimization of the generative trajectory. We propose LyapuFlow, an alternative based on Lyapunov feedback control. At each sampling step, LyapuFlow predicts the terminal sample towards which the current flow is evolving, and evaluates the constraint violation on this prediction. Then, we compute the minimum-norm control that satisfies a prescribed Lyapunov decrease condition. The resulting control remains inactive when the uncontrolled dynamics already reduce the constraint violation at the prescribed rate. Otherwise, it provides a corrective update within a feedback trust region that prevents the control from dominating the pretrained dynamics. We demonstrate LyapuFlow in both data and latent spaces, outperforming alternatives spanning different mechanisms for test-time constraint enforcement in scientific machine learning and image inverse problems.
Read the full abstract ▾
All authors Minseon Gwak, Hans Hao-Hsun Hsu, Danielle C. Maddix, N. Benjamin Erichson
Machine Learning
Modeling Deletion Requests in Machine Unlearning
Christian Cianfarani and Aloni Cohen
Machine unlearning is seen as a promising approach to enable users to exercise the "right to erasure" in the context of AI models. We ask how users might influence the behavior of models when exercising this right. We define two types of behaviors that users might adopt when requesting the deletion of their data: adaptivity and collectivity. Drawing connections between the goals of users in this context and results in
stochastic optimization, we demonstrate theoretical gaps between the potential effects of groups of users who do and do not display these behaviors. We then show how techniques from data valuation might be used to design deletion requesters that can significantly alter model behavior in realistic settings. In experiments on computer
vision tasks, we demonstrate the differential effects of different models of user behavior and attempt to isolate the impacts of adaptivity and collectivity.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SY
ML-OPF-Bench: Benchmarking Machine Learning for Optimal Power Flow
Xinyi Liu, Xuan He, Danny H.K. Tsang et al. (+1 more)
Machine Learning (ML) methods promise a fast solution process for Optimal Power Flow (OPF). While inconsistent test cases, implementations, and evaluation metrics across existing studies make it challenging to determine which algorithmic advances are most critical for real-world deployment. To this end, we propose ML-OPF-Bench, a unified
benchmark for AC- and DC-OPF that evaluates representative ML
algorithms under a consistent pipeline, stress-tests them across system sizes, distribution shifts, and resource budgets, and ranks them with a multi-objective framework. We find that prediction accuracy alone is not a reliable indicator of operational feasibility. Under heavily loaded, congested conditions, even the strongest in-distribution performers lose their advantage, while feasibility is maintained largely by post-processing that enforces the target constraints rather than by the underlying pure ML predictor. Data scaling shows that prediction accuracy and constraint violations follow different trajectories, whereas compute scaling shows that returns diminish and that larger models do not consistently perform better. These results expose critical trade-offs among ML methods' speed, accuracy, and feasibility, and offer practical guidance for future ML-OPF design. We open-source the
benchmark as an extensible Python package for integrating new learning-based OPF
algorithms and evaluating them under the same standard as the existing baselines.
Read the full abstract ▾
All authors Xinyi Liu, Xuan He, Danny H.K. Tsang, Yize Chen
Machine Learning
What to Preserve in Recursive Computation: A Local Predictive Sufficiency Principle
Peilin Wang, Feng Shiyang, Hongfu Gao et al. (+4 more)
Recursive computation repeatedly compresses or reuses intermediate states, creating a simple tension: information that must remain useful across longer recursive paths is also exposed to more opportunities for loss before reaching the final prediction. Existing reconstruction or local-prediction objectives provide tractable supervision, but do not ensure that the retained information remains sufficient for subsequent recursive computation. We identify local predictive sufficiency with recursive predictive closure: controlling local predictive deficiencies at individual interfaces controls the resulting discrepancy at the root. We then turn this principle into a tractable training procedure. Starting from a variational characterization, we derive finite predictive tests and an empirical predictive deficiency that measures predictive value retained across compression. Its predictive sensitivities define margin-relaxed half-space constraints on parameter updates, and we project the host optimizer's proposed update onto their intersection only when predictive preservation would otherwise be violated. Across temporal graphs,
language memory,
vision-
language-action control, and recursive self-improvement, the method matches or improves the corresponding host models under matched compression budgets, with larger gains under heavier recursive or
memory demands, while better preserving predictive information across successive transformations. Crucially, the same task-agnostic predictive-preservation principle is instantiated across all four settings through host-compatible interventions while keeping the endpoint task, backbone, and evaluation protocol fixed. These results establish predictive preservation at recursive interfaces as a general training principle for recursive compression.
Read the full abstract ▾
All authors Peilin Wang, Feng Shiyang, Hongfu Gao, Cencheng Zhao, Di Yuan, Hui Chen, Guiguang Ding
Machine Learning
Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
Jinyuan Li, Chengsong Huang, Langlin Huang et al. (+4 more)
Self-evolving
reasoning models learn from their own generated questions, yet repeated self-training can lead to performance collapse. In this paper, we investigate why performance deteriorates over successive rounds and how to sustain self-
evolution. Our analysis identifies two recurring quality problems in self-generated questions: invalid questions and repeated variants of the same mathematical questions. First, invalid questions become more prevalent across rounds, and answer-consistency filtering further increases their proportion in training data. Second, existing question diversity controls based on lexical similarity can miss mathematically equivalent questions expressed in different ways, which leads to question diversity collapse in later training rounds. Building on these findings, we introduce R-Quest, which uses question validity and novelty feedback to guide self-
evolution. We first train the solver to recognize and reject invalid questions, then use its judgments to guide questioner rewards and filter solver training data. To avoid question repetition, we use a frozen base model to compare sampled question pairs and provide novelty feedback. Empirically, our method consistently achieves the highest average performance on 12
benchmarks in mathematical
reasoning, general-domain
reasoning, and code generation across two model families. Additionally, R-Quest maintains stable performance gains over ten rounds of self-
evolution, peaking in the final round and outperforming R-Zero by 17.32 points.
Read the full abstract ▾
All authors Jinyuan Li, Chengsong Huang, Langlin Huang, Donghong Cai, Shiping Gao, Yuyi Yang, Jiaxin Huang
Machine Learning · Cross-listed from eess.SY
Latent Safety Filters: When a Lossy Encoder Admits a Transferable Certificate
Johannes Mootz, Zahra Nili Ahmadabadi and Reza Akhavian
Latent safety filters certify safety on a learned low-dimensional representation of the state, enabling constraints that resist analytic description. Because the encoder is lossy, a filter can report safe while the physical state is unsafe, with no detectable model error. Existing transfer conditions leave the effect of discarded safety information implicit. We ask when a lossy encoder admits a safety certificate that transfers to the physical system, and show the answer is governed by the detectability of the discarded safety-relevant dynamics. We construct a system whose latent model is exact and whose latent signals always report safe, while the physical state becomes arbitrarily unsafe. For this system no certificate exists and no monitor downstream of the encoder can detect the failure. When the discarded dynamics contract, a latent barrier certifies true safety up to two explicit margins, one for the latent-model error and one for the variation of safety across states the encoder cannot distinguish. In the linear case and under boundedness and non-degeneracy conditions, every calibrated barrier transfers with a finite margin when the safety-relevant subspace is detectable, and none does otherwise. On learned cartpole encoders, the model error does not indicate for which representations the estimated bound is non-vacuous, while the second margin does.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Local Fisher Information Enables Sparse Causal Discovery
Byeongguk Kang, Donghyeon Lee, Euijong Song et al. (+1 more)
Sparse causal discovery calls for methods that exploit graph structure without estimating high-dimensional densities. We introduce Fisher Information Completion Search (FiCS), a source-first
algorithm for additive noise models that uses one local Fisher score for both ordering and parent selection. Under regularity and nonconstant-parent conditions, we prove that a node's local Fisher information equals the noise Fisher information exactly when the conditioning set contains all parents, provided that it contains no descendants. This Fisher parent completion identifies the parent set as the unique minimal Fisher completion. With a maximum conditioning set size $q$ at least the maximum indegree $d$, population FiCS queries marginals of at most $q+1$ variables and recovers the true directed acyclic graph under a positive ordering margin. Bounded conditioning also has a population advantage: reducing $q$ toward $d$ cannot decrease, and can strictly increase, the ordering margin. A growing non-Gaussian family separates local Fisher selection from conditional-variance and leaf-first Fisher ordering. For the regularized kernel Stein
estimator, we establish high-dimensional DAG consistency under $q\{1+\log(p/q)\}+\log p=o(n)$, uniform Fisher separation, local approximation, and compatible ridge and parent penalty parameters. Experiments show the strongest gains when $n$ is small relative to $p$, quantify the effect of the conditioning size, and demonstrate competitive reference-graph recovery on three real-data
benchmarks.
Read the full abstract ▾
All authors Byeongguk Kang, Donghyeon Lee, Euijong Song, Gunwoong Park
Machine Learning · Cross-listed from cs.CR
CyTReX: Explainable AI-Based Cybersecurity Threat Reasoning Framework for DER Networks
Damilola Popoola, Souradeep Bhattacharya and Manimaran Govindarasu
Distributed Energy Resource (DER) environments rely on network communication protocols to coordinate control commands, measurements, and device states across edge assets and cloud systems. Edge anomaly detection systems (ADS) monitor this traffic to identify deviations from normal communication behavior, flagging suspicious flows for further investigation. When the ADS flags abnormal network traffic, a single attack label is often insufficient for operational response: the label reports the detector's selected class but does not expose alternative threat interpretations that may warrant investigation. This paper presents Cybersecurity Threat
Reasoning with Explainable Artificial Intelligence (CyTReX), an evidence-grounded threat
reasoning framework for DER security that transforms network-level anomaly alerts into ranked, analyst-facing threat hypotheses designed to support Security Operations Center (SOC) triage and investigation. CyTReX constrains large
language model (LLM)
reasoning through a structured evidence packet, defined as a consolidated record of detection outputs, model explanations, and cyber threat intelligence (CTI) context. The evidence packet integrates edge-layer anomaly detection evidence, cloud
reasoning layer attack interpretation, Shapley Additive Explanations (SHAP) network-feature attributions, surrogate decision rules, and Model Context Protocol (MCP)-enabled CTI enrichment. This ensures that every ranked hypothesis and attack-tree branch is traceable to explicit evidence rather than free-form LLM inference, and that incomplete or conflicting evidence is communicated rather than suppressed. Evaluation across five configurations shows that additional
reasoning components improve hypothesis specificity, evidence traceability, and analytical grounding, with the complete pipeline providing the richest evidence-grounded
reasoning context.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Amortized Score-Hamiltonian Policy Iteration: A Grid-Free Scheme for Relaxed Stochastic Control Problems
Qi Feng and Gu Wang
We develop an amortized, grid-free implementation of continuous Langevin dynamics based policy-value iteration for
entropy-regularized, infinite-horizon relaxed
stochastic control problems. The improvement rate of the exact iteration is a discounted aggregate of relative Fisher information between the policy and the Gibbs law of its
Hamiltonian. The associated score residual is the velocity with which the control's Langevin dynamics transport its law. We project this velocity onto a conditional sampler shared across states, instead of one Langevin dynamics per state, and the value dynamics onto a parametric critic, estimating both projections at sampled states to obtain coupled actor–critic flows. The score loss measures the actor's agreement with the current critic, while the policy-evaluation residual measures the critic's agreement with the actor. We also derive gradient and Hessian residuals, including a Feynman–Kac representation for the gradient equation, to control errors not detected by the projected value iteration. An exact decomposition of the HJB residual combines these errors into a policy-suboptimality bound under verification and logarithmic Sobolev assumptions. In the linear-quadratic class, both projections are exact and recover the pointwise iteration, and we provide numerical experiments on general models to demonstrate the coupled actor–critic learning in high-dimensions.
Read the full abstract ▾
Machine Learning
First-Order Steering: Translating Weight Adaptation into Activation Steering
Sri Pranav Kunda, Alexander Kurz, Tomas Dominik et al. (+1 more)
Activation steering exploits interpretable directions in the residual stream to enable inference-time manipulation of model behavior. Composing steering vectors to apply multiple target behaviors simultaneously is important in various fields-including AI
alignment and safety-but remains a challenge for existing activation steering methods. In contrast, prior work in model merging shows that target behaviors represented by learned weight adaptations can be combined with high accuracy. A method that translates weight adaptations into activation steering vectors could therefore extend prior work in model merging to generate composable steering vectors that better enable simultaneous inference-time behavioral control. For this, we introduce First-Order Steering, a formulation of activation steering as a first-order approximation of weight update
matrices parameterized by a vector of steering strengths, and establish theoretical bounds on the approximation error of first-order steering. We then develop a novel model merging procedure, HeRD-Merging, which minimizes the first-order approximation error terms to enable higher first-order steering accuracy. Together, our method produces steering vectors that control both individual and composed behaviors more accurately than existing activation steering methods. Furthermore, HeRD-Merging matches the performance of conventional model-merging baselines, while producing weight adaptations that admit more accurate first-order steering vectors.
Read the full abstract ▾
All authors Sri Pranav Kunda, Alexander Kurz, Tomas Dominik, Uri Maoz
Machine Learning
Do RUL explanations hold up? Faithfulness and stability of attributions on C-MAPSS
Manh Hien Nguyen, Ngoc Thanh Nguyen, Isabella Mendoza Cortes et al. (+5 more)
Deep remaining-useful-life (RUL) models on NASA C-MAPSS are now routine, and so are heatmaps that colour sensors and timesteps. A heatmap that looks mechanical is not the same as an explanation an engineer can act on. We train three standard architectures - a 1D CNN, an LSTM, and a small
Transformer encoder - on the official FD001 and FD003 splits with the piecewise RUL cap of 125 cycles and the official PHM08 asymmetric score. We then attach three attribution maps (Integrated Gradients, occlusion, last-layer
attention) and evaluate them with the checks the XAI-for-PdM literature still under-reports: deletion/insertion faithfulness, Spearman stability under sensor-scale noise, agreement across training seeds, and cosine consistency inside RUL bins. Prediction error is a prerequisite, not the claim. The headline is which explanation method moves the RUL output when its top
cells are removed, and which map survives a 5% input perturbation. Integrated Gradients and occlusion are similarly faithful on the LSTM;
Transformer attention is cheap and temporally smooth but weakly faithful. All three maps are almost unchanged under 5% input noise, yet IG/occlusion agree only moderately across two LSTM seeds - stability to sensor jitter is not the same as stability to retraining. A secondary tabular check on the AI4I 2020 failure
dataset shows the same deletion pattern for tree importances. We recommend occlusion or IG for any C-MAPSS-style report that will be read by a maintenance engineer, and we treat raw
attention weights as a visualisation only.
Read the full abstract ▾
All authors Manh Hien Nguyen, Ngoc Thanh Nguyen, Isabella Mendoza Cortes, Tam Khuat, Thanh Pham, Nhat Quang Tran, Ushik Shrestha Khwakhali, Loan Do
Machine Learning
Integrated Imputation-Classification for Supervised Learning with Missing Data
Yue Liu, Ben Liang, Ali Tizghadam et al. (+1 more)
We study supervised classification problems with missing feature values. Existing approaches often decouple imputation from classification, producing imputations that may be plausible but uninformative for prediction. Instead, we propose the Integrated Imputation and Classification Network (IICN), which jointly trains an imputer and an ${(n{+}1)}$-classdiscriminator adversarially with a single class supervised classification objective, where the discriminator learns to distinguish among the $n$ true classes and an additional “imputed" class. We prove that at the global optimum, the imputer and discriminator together implement marginalization over missing coordinates and yield a Bayes-optimal classifier. We evaluate IICN on FashionMNIST, CIFAR-10, and tabular
datasets with naturally occurring missingness. IICN outperforms classical impute-then-classify pipelines and recent generative baselines, showing strong robustness and accuracy in challenging settings.
Read the full abstract ▾
All authors Yue Liu, Ben Liang, Ali Tizghadam, Ilijc Albanese
Machine Learning
Rethinking Self-Distillation for Multi-Teacher Capability Merging
Roy Xie, Dan Friedman, Feng Nan et al. (+7 more)
Combining capabilities of multiple expert models trained starting from the same base checkpoint has become increasingly common in frontier
language-model post-training. Recent trends suggest that multi-teacher on-policy
distillation (MOPD) outperforms conventional off-policy methods. However, despite the higher inference and environment interaction costs incurred by MOPD, we find that much of its reported accuracy gain is due to certain training design choices and hyperparameter optimization, rather than the
algorithm itself. We conduct a controlled self-
distillation study across two multi-teacher settings, four models, and eleven
benchmarks, comparing off-policy methods, namely supervised fine-tuning (SFT) and soft-label
distillation, to hybrid teacher-prefix
distillation and MOPD. We found that all four methods achieve nearly identical accuracy. However, MOPD uses $14.8$–$23.1\times$ SFT's training GPU-hours. We also revisit four recently published comparisons between on-policy and off-policy
distillation and find that the reported on-policy gains shrink substantially when SFT baselines are trained on rejection-sampled teacher trajectories and use independently tuned hyperparameters. As a training-free alternative, we also find that simple weight-merging methods can recover expert capabilities with minutes of CPU merging time, although their accuracy degrades as model size decreases and task interference increases. Overall, our results question recent gains reported due to MOPD and suggest careful tuning of more efficient off-policy baselines as a viable alternative.
Read the full abstract ▾
All authors Roy Xie, Dan Friedman, Feng Nan, Yukun Huang, Zhichao Xu, Chengjiu Zhang, Jun Xu, Manaal Faruqui, Vivek Rathod, Bhuwan Dhingra