Machine Learning
Assured autonomy: How operations research powers and orchestrates generative AI systems
Tinglong Dai, David Simchi-Levi, Michelle Xiao Wu et al. (+1 more)
Generative artificial intelligence (GenAI) is shifting from conversational assistants toward agentic systems – autonomous decision-making systems that sense, decide, and act within operational workflows. This shift creates an autonomy paradox: as GenAI systems are granted greater operational autonomy, they should, by design, embody more formal structure, more explicit constraints, and stronger tail-risk discipline. We argue that
stochastic generative models can be fragile in operational domains unless paired with mechanisms that provide verifiable feasibility, robustness to distribution shift, and stress testing under high-consequence scenarios. To address this challenge, we develop a conceptual framework for assured autonomy grounded in operations research (OR), built on two complementary approaches. First, flow-based generative models frame generation as deterministic transport characterized by an ordinary differential equation, enabling auditability, constraint-aware generation, and connections to optimal transport, robust optimization, and sequential decision control. Second, operational safety is formulated through an
adversarial robustness lens: decision rules are evaluated against worst-case perturbations within uncertainty or ambiguity sets, making unmodeled risks part of the design. This framework clarifies how increasing autonomy shifts OR's role from solver to guardrail to system architect, with responsibility for control logic, incentive protocols, monitoring regimes, and safety boundaries. These elements define a research agenda for assured autonomy in safety-critical, reliability-sensitive operational domains.
Read the full abstract ▾
All authors Tinglong Dai, David Simchi-Levi, Michelle Xiao Wu, Yao Xie
Machine Learning
Causify DataFlow: A Framework For High-performance Machine Learning Stream Computing
Giacinto Paolo Saggese and Paul Smith
We present DataFlow, a computational framework for building, testing, and deploying high-performance machine learning systems on unbounded time-series data. Traditional data science workflows assume finite
datasets and require substantial reimplementation when moving from batch prototypes to streaming production systems. This gap introduces causality violations, batch boundary artifacts, and poor reproducibility of real-time failures. DataFlow resolves these issues through a unified execution model based on directed acyclic graphs (DAGs) with point-in-time idempotency: outputs at any time t depend only on a fixed-length context window preceding t. This guarantee ensures that models developed in batch mode execute identically in streaming production without code changes. The framework enforces strict causality by automatically tracking knowledge time across all transformations, eliminating future-peeking bugs. DataFlow supports flexible tiling across temporal and feature dimensions, allowing the same model to operate at different frequencies and
memory profiles via configuration alone. It integrates natively with the Python data science stack and provides fit/predict semantics for online learning, caching and incremental computation, and automatic parallelization through DAG-based scheduling. We demonstrate its effectiveness across domains including financial trading, IoT, fraud detection, and real-time analytics.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.ET
Exploring the Potential of Spiking Neural Networks in UWB NLOS Identification
Youdong Zhang, Xu He and Xiaolin Meng
Ultrawideband (UWB) nonline-of-sight (NLOS) identification is critical for accurate localization, yet prevailing deep learning solutions impose computational demands incompatible with low-cost edge devices. This letter explores the potential of spiking neural networks (SNNs) for this task and develops a fully unsupervised SNN solution. Against supervised deep learning baselines, our approach achieves comparable accuracy (~82%) with only 4.7K synaptic operations (SOPs) per inference, representing a three-order-of-magnitude improvement in efficiency. These results establish that SNNs can serve as a viable neuromorphic alternative for edge UWB applications.
Read the full abstract ▾
Machine Learning
DUALFloodGNN: Physics-informed Graph Neural Network for Operational Flood Modeling
Carlo Malapad Acosta, Herath Mudiyanselage Viraj Vidura Herath, Jia Yu Lim et al. (+3 more)
Flood models inform strategic disaster management by simulating the spatiotemporal hydrodynamics of flooding. While physics-based numerical flood models are accurate, their substantial computational cost limits their use in operational settings where rapid predictions are essential. Models designed with graph neural networks (GNNs) provide both speed and accuracy while having the ability to process unstructured spatial domains. Given its flexible input and architecture, GNNs can be leveraged alongside physics-informed techniques with ease, significantly improving interpretability and generalizability. We introduce a novel flood GNN architecture, DUALFloodGNN, which embeds physical constraints at both global and local scales through explicit loss terms. The model jointly predicts water volume at nodes and flow along edges through a shared message-passing framework. To improve performance for autoregressive inference, model training is conducted with a multi-step loss enhanced with dynamic curriculum learning. Compared with standard GNN architectures and state-of-the-art GNN flood models, DUALFloodGNN achieves substantial improvements in predicting multiple hydrologic variables (e.g., water volume, flow, and depth) while maintaining high computational efficiency. The model is open sourced at https://github.com/acostacos/dual_flood_gnn. The
dataset is open sourced at https://hdl.handle.net/2123/35293 with the DOI 10.25910/9xav-0s86.
Read the full abstract ▾
All authors Carlo Malapad Acosta, Herath Mudiyanselage Viraj Vidura Herath, Jia Yu Lim, Abhishek Saha, Sanka Rasnayaka, Lucy Marshall
Machine Learning · Cross-listed from cs.CL
HGMEM: Hypergraph-based Working Memory to Improve Multi-step RAG for Long-Context Complex Relational Modeling
Chulun Zhou, Chunkang Zhang, Guoxin Yu et al. (+4 more)
Multi-step
retrieval-augmented generation (RAG) has become a widely adopted strategy for enhancing large
language models (LLMs) on tasks that demand global comprehension and intensive
reasoning. Although many RAG systems incorporate a working
memory to consolidate information, existing designs primarily function as a passive storage for isolated facts. This static nature overlooks crucial high-order correlations among primitive facts, thereby limiting models' capacity for multi-step
reasoning and resulting in fragmented
reasoning and weak global sense-making within extended contexts. We introduce HGMem, a hypergraph-based working
memory system, extending the concept of
memory beyond simple storage into a dynamic, expressive structure for complex
reasoning and global understanding. In our approach,
memory is represented as a hypergraph where hyperedges correspond to distinct
memory units, enabling the progressive formation of high-order interactions within
memory. This mechanism connects facts and thoughts around the focal problem, evolving the
memory into an integrated and situated knowledge structure that provides strong propositions for deeper
reasoning. We evaluate HGMem on several challenging global sense-making
benchmarks. Extensive experiments and in-depth analyses demonstrate that our method consistently improves multi-step RAG and substantially outperforms strong baseline systems across diverse
datasets.
Read the full abstract ▾
All authors Chulun Zhou, Chunkang Zhang, Guoxin Yu, Fandong Meng, Jie Zhou, Wai Lam, Mo Yu
Machine Learning · Cross-listed from stat.ML
Implicit geometric regularization in flow matching via density weighted Stein operators
Shinto Eguchi
Flow Matching (FM) has emerged as a powerful paradigm for continuous normalizing flows, yet standard FM implicitly performs an unweighted $L^2$ regression over the entire ambient space. In high dimensions, this leads to a fundamental inefficiency: the vast majority of the integration domain consists of low-density “void” regions where the target velocity fields are often chaotic or ill-defined. In this paper, we propose {$\gamma$-Flow Matching ($\gamma$-FM)}, a density-weighted variant that aligns the regression geometry with the underlying probability flow. While density weighting is desirable, naive implementations would require evaluating the intractable target density. We circumvent this by introducing a Dynamic Density-Weighting strategy that estimates the target density directly from training particles. This approach allows us to dynamically downweight the regression loss in void regions without compromising the simulation-free nature of FM. Theoretically, we establish that $\gamma$-FM minimizes the transport cost on a statistical
manifold endowed with the $\gamma$-Stein metric. Spectral analysis further suggests that this geometry induces an implicit Sobolev regularization, effectively damping high-frequency oscillations in void regions. Empirically, $\gamma$-FM significantly improves vector field smoothness and sampling efficiency on high-dimensional latent
datasets, while demonstrating intrinsic robustness to outliers.
Read the full abstract ▾
Machine Learning
DivQAT: Enhancing Robustness of Quantized Convolutional Neural Networks against Model Extraction Attacks
Kacem Khaled, Felipe Gohring de Magalhães and Gabriela Nicolescu
Convolutional Neural Networks (CNNs) and their quantized counterparts are vulnerable to extraction attacks, posing a significant threat of IP theft. Yet, the robustness of quantized models against these attacks is little studied compared to large models. Previous defenses propose to inject calculated noise into the prediction probabilities. However, these defenses are limited since they are not incorporated during the model design and are only added as an afterthought after training. Additionally, most defense techniques are computationally expensive and often have unrealistic assumptions about the victim model that are not feasible in edge device implementations and do not apply to quantized models. In this paper, we propose DivQAT, a novel
algorithm to train quantized CNNs based on Quantization Aware Training (QAT) aiming to enhance their robustness against extraction attacks. To the best of our knowledge, our technique is the first to modify the quantization process to integrate a model extraction defense into the training process. Through empirical validation on
benchmark vision datasets, we demonstrate the efficacy of our technique in defending against model extraction attacks without compromising model accuracy. Furthermore, combining our quantization technique with other defense mechanisms improves their effectiveness compared to traditional QAT.
Read the full abstract ▾
Machine Learning
Improved Balanced Classification with Theoretically Grounded Loss Functions
Corinna Cortes, Mehryar Mohri and Yutao Zhong
The balanced loss is a widely adopted objective for multi-class classification under class imbalance. By assigning equal importance to all classes, regardless of their frequency, it promotes
fairness and ensures that minority classes are not overlooked. However, directly minimizing the balanced classification loss is typically intractable, which makes the design of effective surrogate losses a central question. This paper introduces and studies two advanced surrogate loss families: Generalized Logit-Adjusted (GLA) loss functions and Generalized Class-Aware weighted (GCA) losses. GLA losses generalize Logit-Adjusted losses, which shift logits based on class priors, to the broader general cross-
entropy loss family. GCA loss functions extend the standard class-weighted losses, which scale losses inversely by class frequency, by incorporating class-dependent confidence margins and extending them to the general cross-
entropy family. We present a comprehensive theoretical analysis of consistency for both loss families. We show that GLA losses are Bayes-consistent, but only $H$-consistent for complete (i.e., unbounded) hypothesis sets. Moreover, their $H$-consistency bounds depend inversely on the minimum class probability, scaling at least as $1/\mathsf p_{\min}$. In contrast, GCA losses are $H$-consistent for any hypothesis set that is bounded or complete, with $H$-consistency bounds that scale more favorably as $1/\sqrt{\mathsf p_{\min}}$, offering significantly stronger theoretical guarantees in imbalanced settings. We report the results of experiments demonstrating that, empirically, both the GCA losses with calibrated class-dependent confidence margins and GLA losses can greatly outperform straightforward class-weighted losses as well as the LA losses. GLA generally performs slightly better in common
benchmarks, whereas GCA exhibits a slight edge in highly imbalanced settings.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CY
Statistical Guarantees in the Search for Less Discriminatory Algorithms
Chris Hays, Ben Laufer, Solon Barocas et al. (+1 more)
U.S. discrimination law can impose liability on firms that fail to adopt a less discriminatory alternative (LDA): a decision policy that achieves the same business objectives while reducing disparate impact on legally protected groups. Recent scholarship argues that this doctrine has direct implications for algorithmic decision-making in high-stakes domains such as employment, lending, and housing, potentially obligating firms to search for "less discriminatory
algorithms" (Black et al., 2024). Regulators have at times encouraged proactive LDA searches, reinforcing the expectation of a good-faith effort to identify equally performant models with lower disparate impact. Model multiplicity makes such searches plausible: retraining with different random seeds can yield models with comparable predictive performance but materially different disparate impacts. Yet firms cannot retrain indefinitely, raising a central question: when is the search sufficient to demonstrate good faith? We formalize LDA search under multiplicity as an optimal stopping problem in which a developer seeks to produce evidence that further search is unlikely to yield meaningful improvements. Our main contribution is an adaptive stopping
algorithm that provides a high-probability upper bound on the best disparate-impact gains attainable through continued retraining, enabling developers to certify (e.g., to a court) that additional search is unlikely to help. We also show how stronger distributional assumptions over the model space can yield tighter bounds, and we validate the approach on real-world credit and housing
datasets.
Read the full abstract ▾
All authors Chris Hays, Ben Laufer, Solon Barocas, Manish Raghavan
Machine Learning · Cross-listed from cond-mat.stat-mech
Assessing generative modeling approaches for free energy estimates in condensed matter
Maximilian Schebek, Jiajun He, Emil Hoffmann et al. (+3 more)
The accurate estimation of free energy differences between two states is a long-standing challenge in molecular simulations. Traditional approaches generally rely on sampling multiple intermediate states to ensure sufficient overlap in phase space and are, consequently, computationally expensive. Boltzmann Generators and related generative-model-based methods have recently addressed this challenge by learning a direct probability density transform between two states. However, it remains unclear which approach provides the best trade-off between efficiency, accuracy, and scalability. In this work, we review and
benchmark selected generative approaches for condensed-matter systems, including discrete and continuous normalizing flows for targeted free energy perturbation and FEAT (Free Energy
Estimators with Adaptive Transport) combined with the escorted Jarzynski equality, using coarse-grained monatomic ice and Lennard-Jones solids as
benchmark systems. All models yield highly accurate free energy estimates and, depending on the system, may require fewer energy evaluations than traditional methods. Continuous flows and FEAT are most efficient in energy evaluations, whereas discrete flows have substantially lower inference cost. By releasing all data together with our results, we enable future benchmarking of free energy estimation methods in condensed-phase systems.
Read the full abstract ▾
All authors Maximilian Schebek, Jiajun He, Emil Hoffmann, Yuanqi Du, Frank Noé, Jutta Rogal
Machine Learning · Cross-listed from stat.ML
Soft Fitted Q-Iteration without Bellman Completeness: Occupancy Reweighting and Temperature Annealing
Lars van der Laan and Nathan Kallus
Fitted \(Q\)-iteration (FQI) is a standard regression-based method for optimal control in offline reinforcement learning, but its stability under function approximation often relies on Bellman completeness, which requires Bellman images of the fitted class to remain in the class. We study Kullback–Leibler (KL)-regularized, or soft, FQI relative to a fixed reference policy without this assumption. Our key insight is that soft control locally inherits the contraction of policy evaluation in a discounted-occupancy norm. At the soft-optimal fixed point, the linearization of the soft Bellman operator is exactly the Bellman operator for the soft-optimal policy, which contracts in its discounted-occupancy norm; projection in the same norm preserves this contraction. Standard soft FQI instead projects under the offline state-action distribution and need not preserve this property. Motivated by this observation, we propose occupancy-reweighted soft FQI, which retains standard Bellman targets and least-squares updates while reweighting regressions by discounted-occupancy ratios induced by the current soft policy. Under \(Q\)-function realizability and local regularity, we establish local contraction and finite-sample convergence with estimated ratios, without Bellman completeness. We then use temperature annealing to convert the local result into global convergence from arbitrary initialization: sufficiently high temperature provides a globally contractive starting regime, while gradual cooling connects successive local contraction regions to any prescribed positive target temperature. Under an action-gap margin condition, switching at a fixed positive temperature to hard FQI with refreshed occupancy weights also yields population and finite-sample convergence to the unregularized optimum.
Read the full abstract ▾
Machine Learning
Interactive Machine Learning: From Theory to Scale
Yinglun Zhu
Machine learning has achieved remarkable success across a wide range of applications, yet many of its most effective methods rely on access to large amounts of labeled data or extensive online interaction. In practice, acquiring high-quality labels and making decisions through trial-and-error can be expensive, time-consuming, or risky, particularly in large-scale or high-stakes settings. This dissertation studies interactive machine learning, in which the learner actively influences how information is collected or which actions are taken, using past observations to guide future interactions. We develop new algorithmic principles and establish fundamental limits for interactive learning along three dimensions: active learning with noisy data and rich model classes, sequential decision making with large action spaces, and model selection under partial feedback. Our results include the first computationally efficient active learning
algorithms achieving exponential label savings without low-noise assumptions; the first efficient, general-purpose contextual bandit
algorithms whose guarantees are independent of the size of the action space; and the first tight characterizations of the fundamental cost of model selection in sequential decision making. Overall, this dissertation advances the theoretical foundations of interactive learning by developing
algorithms that are statistically optimal and computationally efficient, while also providing principled guidance for deploying interactive learning methods in large-scale, real-world settings.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Tensor computing application programming interface: An application-oriented, lightweight API for portable high-performance tensor network computations
Rong-Yang Sun, Tomonori Shirakawa, Hidehiko Kohshiro et al. (+2 more)
Tensor networks (TNs) are a central computational tool in
quantum science and artificial intelligence. However, the lack of a unified software interface across tensor-computing frameworks severely limits the portability of TN applications, coupling algorithmic development to specific hardware and software back ends. To address this challenge, we introduce the Tensor Computing Application Programming Interface (TCAPI)—an application-oriented, lightweight API designed to enable framework-independent, high-performance TN applications, currently specified for C++ and Python. TCAPI provides well-defined type-system conventions that abstract tensor objects, together with a minimal yet expressive set of core functions for essential tensor manipulations and tensor linear-
algebra operations, with operation semantics shared across
language specifications. Through numerical demonstrations of representative tensor-network applications implemented in C++ using TCAPI, we show that code written against TCAPI can be migrated seamlessly across heterogeneous hardware and software platforms while achieving performance comparable to native framework implementations. We further provide open-source implementations of TCAPI based on Cytnx and ITensor in C++, and on NumPy in Python, demonstrating its practicality and ease of integration with existing tensor-computing frameworks.
Read the full abstract ▾
All authors Rong-Yang Sun, Tomonori Shirakawa, Hidehiko Kohshiro, D. N. Sheng, Seiji Yunoki
Machine Learning
Constraint Breeds Generalization: Temporal Dynamics as an Inductive Bias
Xia Chen
Conventional deep learning prioritizes unconstrained optimization, yet biological systems operate under strict metabolic constraints. We propose that these physical constraints shape dynamics to function not as limitations, but as a temporal inductive bias that breeds generalization. Through a phase-space analysis of signal propagation, we reveal a fundamental asymmetry: expansive dynamics amplify noise, whereas proper dissipative dynamics compress phase space that aligns with the network's spectral bias, compelling the abstraction of invariant features. This condition can be imposed externally via input encoding, or intrinsically through the network's own temporal dynamics. Both pathways require architectures capable of temporal integration and proper constraints to decode induced invariants, whereas static architectures fail to capitalize on temporal structure. Through comprehensive evaluations across supervised classification, unsupervised reconstruction, and zero-shot reinforcement learning, we demonstrate that a critical "transition" regime maximizes generalization capability. These findings establish dynamical constraints as a distinct class of inductive bias, suggesting that robust AI development requires not only scaling and removing limitations, but computationally mastering the temporal characteristics that naturally promote generalization.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SP
A multimodal Transformer for InSAR-based ground deformation forecasting with cross-site generalization across Europe
Wendong Yao, Binhua Huang and Soumyabrata Dev
Near-real-time regional-scale monitoring of ground deformation is increasingly required to support urban planning, critical infrastructure management, and natural hazard mitigation. While Interferometric Synthetic Aperture Radar (InSAR) and continental-scale services such as the European Ground Motion Service (EGMS) provide dense observations of past motion, predicting the next observation remains challenging due to the superposition of long-term trends, seasonal cycles, and occasional abrupt discontinuities (e.g., co-seismic steps), together with strong spatial heterogeneity. In this study we propose a
multimodal patch-based
Transformer for single-step, fixed-interval next-epoch nowcasting of displacement maps from EGMS time series (resampled to a 64x64 grid over 100 km x 100 km tiles). The model ingests recent displacement snapshots together with (i) static kinematic indicators (mean velocity, acceleration, seasonal amplitude) computed in a leakage-safe manner from the training window only, and (ii) harmonic day-of-year encodings. On the eastern Ireland tile (E32N34), the STGCN is strongest in the displacement-only setting, whereas the
multimodal Transformer clearly outperforms CNN-LSTM, CNN-LSTM+Attn, and
multimodal STGCN when all models receive the same
multimodal inputs, achieving RMSE = 0.90 mm and $R^2$ = 0.97 on the test set with the best threshold accuracies.
Read the full abstract ▾
Machine Learning
Rethinking Dense Linear Transformations: Stagewise Pairwise Mixing (SPM) for Near-Linear Training in Neural Networks
Peter Farag
Dense linear layers are a dominant source of computational and parametric cost in modern machine learning models, despite their quadratic complexity and often being misaligned with the compositional structure of learned representations. We introduce Stagewise Pairwise Mixers (SPM), a structured linear operator that replaces dense
matrices with a composition of sparse pairwise-mixing stages. An SPM layer implements a global linear transformation in $O(nL)$ time with $O(nL)$ parameters, where $L$ is typically constant or $log_2n$, and admits exact closed-form forward and backward computations. SPM is designed as a drop-in replacement for dense linear layers in feedforward networks, recurrent architectures,
attention mechanisms, etc. We derive complete forward and backward expressions for two parameterizations: an orthogonal norm-preserving rotation-based variant and a fully general $2 \times 2$ mixing variant. Beyond computational savings, the stagewise structure of SPM induces an explicit compositional inductive bias that constrains model capacity and improves generalization when aligned with task structure. We present proof-of-concept experiments demonstrating substantial reductions in wall-clock cost and improved accuracy on structured learning problems, while retaining competitive performance on real-world
benchmarks.
Read the full abstract ▾