Machine Learning
Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Training
Kevin Wang, Hongqian Niu and Didong Li
As artificial intelligence (AI)-generated content proliferates, models are increasingly trained on their own outputs, risking progressive degradation or collapse. In this article, we provide the first positive, rigorous theoretical results, to the best of our knowledge, showing that under model-agnostic mild conditions, the model converges to the true data-generating distribution. The convergence rate is the minimum of the model's intrinsic rate and the fraction of real data at each training iteration, revealing a phase transition between data-limited and model-limited regimes. We further show that, for biased real data, correcting the bias prevents the persistence and amplification of early bias over training iteration. Extensive experiments across simulations, real images and texts validate our theoretical framework, establishing quantitative conditions for long-term AI stability in contaminated environments.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SY
MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets
Nelson Salazar-Pena, Alejandra Tabares and Andres Gonzalez-Mancera
This paper introduces a novel, open-source MARL simulation framework for studying implicit cooperation in LEMs, modeled as a decentralized partially observable Markov decision process and implemented as a Gymnasium environment for MARL. Our framework features a modular
market platform with plug-and-play clearing mechanisms, physically constrained
agent models (including battery storage), a realistic grid network, and a comprehensive analytics suite to evaluate emergent coordination. The main contribution is a novel method to foster implicit cooperation, where
agents' observations and rewards are enhanced with system-level key performance indicators to enable them to independently learn strategies that benefit the entire system and aim for collectively beneficial outcomes without explicit communication. Through representative case studies (available in a dedicated GitHub repository in https://github.com/salazarna/marlem, we show the framework's ability to analyze how different
market configurations (such as varying storage deployment) impact system performance. This illustrates its potential to facilitate emergent coordination, improve
market efficiency, and strengthen grid stability. The proposed simulation framework is a flexible, extensible, and reproducible tool for researchers and practitioners to design, test, and validate strategies for future intelligent, decentralized energy systems.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SY
Harnessing Implicit Cooperation: A Multi-Agent Reinforcement Learning Approach Towards Decentralized Local Energy Markets
Nelson Salazar-Pena, Alejandra Tabares and Andres Gonzalez-Mancera
This paper proposes implicit cooperation, a framework enabling decentralized
agents to approximate optimal coordination in local energy
markets without explicit peer-to-peer communication. We formulate the problem as a decentralized partially observable Markov decision problem that is solved through a multi-
agent reinforcement learning task in which
agents use stigmergic signals (key performance indicators at the system level) to infer and react to global states. Through a 3x3 factorial design on an IEEE 34-node
topology, we evaluated three training paradigms (CTCE, CTDE, DTDE) and three
algorithms (PPO, APPO, SAC). Results identify APPO-DTDE as the optimal configuration, achieving a coordination score of 91.7% relative to the theoretical centralized
benchmark (CTCE). However, a critical trade-off emerges between efficiency and stability: while the centralized
benchmark maximizes allocative efficiency with a peer-to-peer trade ratio of 0.6, the fully decentralized approach (DTDE) demonstrates superior physical stability. Specifically, DTDE reduces the variance of grid balance by 31% compared to hybrid architectures, establishing a highly predictable, import-biased load profile that simplifies grid regulation. Furthermore, topological analysis reveals emergent spatial clustering, where decentralized
agents self-organize into stable trading communities to minimize congestion penalties. While SAC excelled in hybrid settings, it failed in decentralized environments due to
entropy-driven instability. This research proves that stigmergic signaling provides sufficient context for complex grid coordination, offering a robust,
privacy-preserving alternative to expensive centralized communication infrastructure.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
AI-Generated Measurements for Identification and Inference with Missing Data: A Weak Shadow Variable Approach
Hongyu Chen, David Simchi-Levi and Ruoxuan Xiong
Across business and social science applications, outcomes are often missing in ways that depend on the unobserved outcomes themselves. In service systems, for example, whether a customer submits a rating depends on the rating they would have provided. Such missing-not-at-random (MNAR) mechanisms make population quantities difficult to identify without strong assumptions on the observation process. Meanwhile, rich unstructured data, such as customer interaction histories, are increasingly available and can be used to construct structured measurements using tools such as large
language models (LLMs). In this work, we develop an assumption-lean partial identification framework that uses such measurements as weak shadow variables, defined as outcome-informative proxies that are conditionally independent of missingness given the true outcome and observed
covariates. Importantly, they need not accurately predict missing outcomes or satisfy the completeness requirement in the classical shadow variable literature. For identification, we characterize sharp bounds on population quantities through a pair of linear programs. For estimation and inference, we propose a localized penalized
estimator that remains feasible under sampling error, and a subsampling
algorithm for constructing confidence intervals. In semi-synthetic experiments using real customer-service dialogues, weak-shadow-variable intervals are about 89% narrower than those without auxiliary information, while their midpoints have around 41% lower estimation error than classical MNAR methods.
Read the full abstract ▾
Machine Learning
Extracting and Analyzing Rail Crossing Behavior Signatures from Videos using Tensor Methods
Dawon Ahn, Het Patel, Aemal Khattak et al. (+2 more)
Railway crossings present complex safety challenges where driver behavior varies by location, time, and conditions. Traditional approaches analyze crossings individually, limiting the ability to identify shared behavioral patterns across locations. We propose a multi-view tensor decomposition framework that captures behavioral similarities across three temporal phases: Approach (warning activation to gate lowering), Waiting (gates down to train passage), and Clearance (train passage to gate raising). We analyze railway crossing
videos from multiple locations using TimeSformer
embeddings to represent each phase. By constructing phase-specific similarity
matrices and applying non-negative symmetric CP decomposition, we discover latent behavioral components with distinct temporal signatures. Our tensor analysis reveals that crossing location appears to be a stronger determinant of behavior patterns than time of day, and that approach-phase behavior provides particularly discriminative signatures. Visualization of the learned component space confirms location-based clustering, with certain crossings forming distinct behavioral clusters. This automated framework enables scalable pattern discovery across multiple crossings, providing a foundation for grouping locations by behavioral similarity to inform targeted safety interventions.
Read the full abstract ▾
All authors Dawon Ahn, Het Patel, Aemal Khattak, Jia Chen, Evangelos E. Papalexakis
Machine Learning · Cross-listed from cs.CL
CLAA: Cross-Layer Attention Aggregation for Accelerating LLM Prefill
Bradley McDanel, Steven Li and Harshit Khaitan
The prefill stage in long-context LLM inference remains a computational bottleneck. Recent
token-ranking heuristics accelerate inference by selectively processing a subset of semantically relevant
tokens. However, existing methods suffer from unstable
token importance estimation, often varying between layers. Evaluating
token-ranking quality independently from heuristic-specific architectures is challenging. To address this, we introduce an Answer-Informed Oracle, which defines ground-truth
token importance by measuring
attention from generated answers back to the
prompt. This oracle reveals that existing heuristics exhibit high variance across layers: rankings can degrade sharply at specific layers, a failure mode invisible to end-to-end
benchmarks. The diagnosis suggests a simple fix: aggregate scores across layers rather than relying on any single one. We implement this as Cross-Layer
Attention Aggregation (CLAA), which closes the gap to the oracle upper bound and reduces Time-to-First-
Token (TTFT) by up to 39% compared to the Full KV Cache baseline.
Read the full abstract ▾
Machine Learning
Multi-Objective Alignment of Language Models for Personalized Psychotherapy
Mehrab Beikzadeh, Yasaman Asadollah Salmanpour, Ashima Suvarna et al. (+4 more)
Mental health disorders affect over 1 billion people worldwide, yet access to care remains limited by workforce shortages and cost constraints. While AI systems show therapeutic promise, current
alignment approaches optimize objectives independently, failing to balance patient preferences with clinical safety. We survey 335 individuals with lived mental health experience to collect preference rankings across therapeutic dimensions, then develop a multi-objective
alignment framework using direct preference optimization. We train reward models for six criteria – empathy, safety, active listening, self-motivated change, trust/rapport, and patient autonomy – and systematically compare multi-objective approaches against single-objective optimization, supervised fine-tuning, and parameter merging. Multi-objective DPO (MODPO) achieves superior balance (77.6% empathy, 62.6% safety) compared to single-objective optimization (93.6% empathy, 47.8% safety), and therapeutic criteria outperform general communication principles by 17.2%. Blinded clinician evaluation confirms MODPO is consistently preferred, with LLM-evaluator agreement comparable to inter-clinician reliability.
Read the full abstract ▾
All authors Mehrab Beikzadeh, Yasaman Asadollah Salmanpour, Ashima Suvarna, Sriram Sankararaman, Matteo Malgaroli, Majid Sarrafzadeh, Saadia Gabriel
Machine Learning
MoE-Spec: Expert Budgeting for Efficient Speculative Decoding
Bradley McDanel, Steven Li, Sruthikesh Surineni et al. (+1 more)
Speculative decoding accelerates Large
Language Model (LLM) inference by verifying multiple drafted
tokens in parallel. However, for Mixture-of-Experts (MoE) models, this parallelism introduces a severe bottleneck: large draft trees activate many unique experts, significantly increasing
memory pressure and diminishing speedups from speculative decoding relative to autoregressive decoding. Prior methods reduce speculation depth when MoE verification becomes expensive. We propose MoE-Spec, a training-free verification-time expert budgeting method that decouples speculation depth from
memory cost by enforcing a fixed expert capacity limit at each layer, loading only the experts that contribute most to verification and dropping the long tail of rarely used experts that drive bandwidth overhead. Experiments across multiple model scales and
datasets show that this method yields 10–30% higher throughput than state-of-the-art speculative decoding baselines (EAGLE-3) at comparable quality, with flexibility to trade accuracy for further latency reductions through tighter budgets.
Read the full abstract ▾
All authors Bradley McDanel, Steven Li, Sruthikesh Surineni, Harshit Khaitan
Machine Learning
AI-CARE: Carbon-Aware Reporting Evaluation Metric for AI Models
KC Santosh, Srikanth Baride and Rodrigue Rizk
As machine learning (ML) continues its rapid expansion, the environmental cost of model training and inference has become a critical societal concern. Existing
benchmarks overwhelmingly focus on standard performance metrics such as accuracy, BLEU, or mAP, while largely ignoring energy consumption and carbon emissions. This single-objective evaluation paradigm is increasingly misaligned with the practical requirements of large-scale deployment, particularly in energy-constrained environments such as mobile devices, developing regions, and
climate-aware enterprises. In this paper, we propose AI-CARE, an evaluation tool for reporting energy consumption, and carbon emissions of ML models. In addition, we introduce the carbon-performance tradeoff curve, an interpretable tool that visualizes the Pareto frontier between performance and carbon cost. We demonstrate, through theoretical analysis and empirical validation on representative ML workloads, that carbon-aware benchmarking changes the relative ranking of models and encourages architectures that are simultaneously accurate and environmentally responsible. Our proposal aims to shift the research community toward transparent, multi-objective evaluation and align ML progress with global sustainability goals. The tool and documentation are available at https://github.com/USD-AI-ResearchLab/ai-care.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NE
Heuristic Search as Language-Guided Program Optimization
Mingxin Yu, Ruixiao Yang and Chuchu Fan
Large
Language Models (LLMs) have advanced Automated Heuristic Design (AHD) in combinatorial optimization (CO) in the past few years. However, existing discovery pipelines often require extensive manual trial-and-error or reliance on domain expertise to adapt to new or complex problems. This stems from tightly coupled internal mechanisms that limit systematic improvement of the LLM-driven design process. To address this challenge, we propose a structured framework for LLM-driven AHD that explicitly decomposes the heuristic discovery process into modular stages: a forward pass for evaluation, a backward pass for analytical feedback, and an update step for program refinement. This separation provides a clear abstraction for iterative refinement and enables principled improvements of individual components. We validate our framework across four diverse real-world CO domains, where it consistently outperforms baselines, achieving up to $0.17$ improvement in QYI on unseen test sets. Finally, we show that several popular AHD methods are restricted instantiations of our framework. By integrating them in our structured pipeline, we can upgrade the components modularly and significantly improve their performance.
Read the full abstract ▾
Machine Learning
MolCrystalFlow: Molecular Crystal Structure Prediction via Flow Matching
Cheng Zeng, Harry W. Sullivan, Thomas Egg et al. (+8 more)
Molecular
crystal structure prediction represents a grand challenge in computational chemistry due to large sizes of constituent molecules and complex intra- and intermolecular interactions. While generative modeling has revolutionized structure discovery for molecules, inorganic solids, and metal-organic frameworks, extending such approaches to fully periodic molecular
crystals is still elusive. Here, we present MolCrystalFlow, a flow-based generative model for molecular
crystal structure prediction. The framework disentangles intramolecular complexity from intermolecular packing by
embedding molecules as rigid bodies and jointly learning the
lattice matrix, molecular orientations, and centroid positions. Centroids and orientations are represented on their native Riemannian
manifolds, allowing geodesic flow construction and graph neural network operations that respects
geometric symmetries. We
benchmark our model against a state-of-the-art generative model (MOFFlow) for large-size periodic
crystals and a rule-based structure generation method (Genarris) on two open-source molecular
crystal datasets. MolCrystalFlow outperforms MOFFlow while achieving competitive performance against Genarris. We also demonstrate an integration of MolCrystalFlow model with universal machine learning potential to accelerate molecular
crystal structure prediction, paving the way for data-driven generative discovery of molecular
crystals.
Read the full abstract ▾
All authors Cheng Zeng, Harry W. Sullivan, Thomas Egg, Maya M. Martirossyan, Philipp Höllmer, Jirui Jin, Richard G. Hennig, Adrian Roitberg, Stefano Martiniani, Ellad B. Tadmor, Mingjie Liu
Machine Learning · Cross-listed from quant-ph
Edge-Local and Qubit-Efficient Quantum Graph Learning for the NISQ Era
Armin Ahmadkhaniha and Jake Doliskani
Graph neural networks (GNNs) are a powerful framework for learning representations from graph-structured data, but their direct implementation on near-term
quantum hardware remains challenging due to
circuit depth, multi-
qubit interactions, and
qubit scalability constraints. In this work, we introduce a hybrid
quantum graph learning architecture designed explicitly for unsupervised learning in the noisy intermediate-scale
quantum (NISQ) regime. Our approach combines a variational
quantum feature extraction layer with an edge-local and
qubit-efficient
quantum message-passing mechanism inspired by the
Quantum Alternating Operator Ansatz (QAOA) framework. The message-passing operation is decomposed into pairwise interactions along graph edges using standard single- and two-
qubit gates. For a graph with $N$ nodes and $n$-
qubit feature registers, this reduces the number of
qubits required at one time from $Nn$ to at most $2n$. We train the model using the Deep Graph Infomax objective to perform unsupervised node representation learning. The external class labels are not used during graph construction or training and are used only to evaluate the learned
embeddings. Experiments on the Cora citation network and the Phase 3 release of the 1000 Genomes Project show that the
quantum edge interaction contributes to the quality of the learned node representations.
Read the full abstract ▾
Machine Learning
Geometry-Aware Uncertainty Quantification via Conformal Prediction on Manifolds
Marzieh Amiri Shahbazi and Ali Baheri
Conformal prediction gives finite-sample coverage guarantees for regression, but most standard constructions are designed for Euclidean output spaces. When the response lies on a Riemannian
manifold, Euclidean residuals and coordinate-based regions can ignore the geometry that defines meaningful error. We propose adaptive geodesic conformal prediction, a simple framework that builds nonconformity scores from geodesic distances and normalizes them with a cross-validated estimate of local prediction difficulty. On the sphere, this produces geodesic caps whose area is independent of position, while their radii still adapt to heteroscedastic noise. In both a synthetic sphere experiment and an IGRF-14 geomagnetic field
forecasting task, the adaptive method preserves valid marginal coverage, reduces variation in conditional coverage, and improves worst-case coverage relative to non-adaptive and coordinate-based baselines.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Towards Efficient Constraint Handling in Neural Solvers for Routing Problems
Jieyi Bi, Zhiguang Cao, Jianan Zhou et al. (+5 more)
Neural solvers have achieved impressive progress in addressing simple routing problems, particularly excelling in computational efficiency. However, their advantages under complex constraints remain nascent, for which current constraint-handling schemes via feasibility masking or implicit feasibility awareness can be inefficient or inapplicable for hard constraints. In this paper, we present Construct-and-Refine (CaR), the first general and efficient constraint-handling framework for neural routing solvers based on explicit learning-based feasibility refinement. Unlike prior construction-search hybrids that target reducing optimality gaps through heavy improvements yet still struggle with hard constraints, CaR achieves efficient constraint handling by designing a joint training framework that guides the construction module to generate diverse and high-quality solutions well-suited for a lightweight improvement process, e.g., 10 steps versus 5k steps in prior work. Moreover, CaR presents the first use of construction-improvement-shared representation, enabling potential knowledge sharing across paradigms by unifying the encoder, especially in more complex constrained scenarios. We evaluate CaR on typical hard routing constraints to showcase its broader applicability. Results demonstrate that CaR achieves superior feasibility, solution quality, and efficiency compared to both classical and neural state-of-the-art solvers.
Read the full abstract ▾
All authors Jieyi Bi, Zhiguang Cao, Jianan Zhou, Wen Song, Yaoxin Wu, Jie Zhang, Yining Ma, Cathy Wu
Machine Learning · Cross-listed from cs.SD
Adnan El Assadi, Isaac Chung, Chenghao Xiao et al. (+15 more)
We introduce the Massive Audio
Embedding Benchmark (MAEB), a large-scale
benchmark covering 30 tasks across
speech, music, environmental sounds, and cross-modal audio-text
reasoning in 100+
languages. We evaluate 50+ models and find that no single model dominates across all tasks: contrastive audio-text models excel at environmental sound classification (e.g., ESC50) but score near random on
multilingual speech tasks (e.g., SIB-FLEURS), while
speech-pretrained models show the opposite pattern. Clustering remains challenging for all models, with even the best-performing model achieving only modest results. We observe that models excelling on acoustic understanding often perform poorly on linguistic tasks, and vice versa. We also show that the performance of audio encoders on MAEB correlates highly with their performance when used in audio large
language models. MAEB is derived from MAEB+, a collection of 98 tasks. MAEB is designed to maintain task diversity while reducing evaluation cost, and it integrates into the MTEB ecosystem for unified evaluation across text, image, and audio modalities. We release MAEB and all 98 tasks along with code and a leaderboard at https://github.com/
embeddings-
benchmark/mteb.
Read the full abstract ▾
All authors Adnan El Assadi, Isaac Chung, Chenghao Xiao, Roman Solomatin, Animesh Jha, Rahul Chand, Silky Singh, Kaitlyn Wang, Ali Sartaz Khan, Marc Moussa Nasser, Sufen Fong, Pengfei He, Alan Xiao, Ayush Sunil Munot, Aditya Shrivastava, Artem Gazizov, Niklas Muennighoff, Kenneth Enevoldsen
Machine Learning · Cross-listed from physics.med-ph
Imaging-Derived Coronary Fractional Flow Reserve: Advances in Physics-Based, Machine Learning, and Physics-Informed Methods
Tanxin Zhu, Emran Hossen, Chen Zhao et al. (+4 more)
Purpose of Review Imaging derived fractional flow reserve (FFR) is rapidly evolving beyond conventional computational fluid dynamics (CFD) based pipelines toward machine learning (ML), deep learning (DL), and physics informed approaches that enable fast, wire free, and scalable functional assessment of coronary artery stenosis. This review synthesizes recent advances in computed
tomography (CT)- and angiography-based FFR measurement, with particular emphasis on emerging physics-informed neural networks and neural operators (PINNs and PINOs), as well as key considerations for their clinical translation. Recent Findings ML/DL approaches have markedly improved automation and computational speed, enabling prediction of pressure and FFR from anatomical descriptors or angiographic contrast dynamics. However, their real-world performance and generalizability can remain variable and sensitive to domain shift, due to multi-center heterogeneity, interpretability challenges, and differences in acquisition protocols and image quality. Physics informed learning introduces conservation structure and boundary condition consistency into model training, improving generalizability and reducing dependence on dense supervision while maintaining rapid inference. Recent evaluation trends increasingly highlight deployment oriented metrics, including calibration, uncertainty quantification, and quality control gatekeeping, as essential for safe clinical use. Summary The field is converging toward imaging derived FFR methods that are faster, more automated, and more reliable. While ML/DL offers substantial efficiency gains, physics informed frameworks such as PINNs and PINOs may provide a more robust balance between speed and physical consistency. Prospective multi center validation and standardized evaluation will be critical to support broad and safe clinical adoption.
Read the full abstract ▾
All authors Tanxin Zhu, Emran Hossen, Chen Zhao, Jingfeng Jiang, Michele Esposito, Jiguang Sun, Weihua Zhou
Machine Learning
Jayadev Billa
Neural networks gain capabilities during training, but the internal changes that precede capability acquisition are not well understood. In particular, the relationship between
geometric change and behavioral change, and the effect of task difficulty and model scale on that relationship, is unclear. We track
geometric measures and linear probes across six
transformer sizes (405K–151M parameters), eight algorithmic tasks (144 task$\times$level$\times$model combinations), and three Pythia
language models (160M–2.8B). Across all settings, representations first collapse to a low-dimensional state, then recover, and only then does behavioral performance improve. Linear probes show that the model's hidden states already contain task-relevant information before the model can act on it. The collapse floor is task-specific, the collapse propagates top-down through the network, and of the
geometric measures tested, only \rankme reliably precedes capability acquisition for hard tasks. Whether this precursor is detectable depends on task difficulty relative to model capacity. For hard tasks, there is a clear gap: geometry changes first, behavior follows. For easy tasks, the model learns so quickly that both happen simultaneously and no precursor is detectable. On Pythia-2.8B, a logical deduction task that is genuinely hard for the model shows a precursor gap of ${\sim}$49K training steps, while easy
benchmarks show none. This suggests that
geometric patterns observed in small proxy models can persist at larger scale when the task remains difficult relative to model capacity.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation
Aleksandr Beznosikov, Georgiy Kormakov, Alexander Grigorievskiy et al. (+7 more)
The objective of Vertical
Federated Learning (VFL) is to collectively train a model using features available on different devices while sharing the same users. This paper focuses on the saddle point reformulation of the VFL problem via the classical Lagrangian function. We first demonstrate how this formulation can be solved using deterministic methods. More importantly, we explore various
stochastic modifications to adapt to practical scenarios, such as employing compression techniques for efficient information transmission, enabling partial participation for asynchronous communication, and utilizing coordinate selection for faster local computation. We show that the saddle point reformulation plays a key role and opens up possibilities to use mentioned extension that seem to be impossible in the standard minimization formulation. Convergence estimates are provided for each
algorithm, demonstrating their effectiveness in addressing the VFL problem. Additionally, alternative reformulations are investigated, and numerical experiments are conducted to validate performance and effectiveness of the proposed approach.
Read the full abstract ▾
All authors Aleksandr Beznosikov, Georgiy Kormakov, Alexander Grigorievskiy, Mikhail Rudakov, Ruslan Nazykov, Alexander Rogozin, Anton Vakhrushev, Andrey Savchenko, Martin Takáč, Alexander Gasnikov
Machine Learning
Verifier-Constrained Flow Expansion for Discovery Beyond the Data
Riccardo De Santi, Kimon Protopapas, Ya-Ping Hsieh et al. (+1 more)
Flow and
diffusion models are typically pre-trained on limited available data (e.g., molecular samples), covering only a fraction of the valid design space (e.g., the full molecular space). As a consequence, they tend to generate samples from only a narrow portion of the feasible domain. This is a fundamental limitation for scientific discovery applications, where one typically aims to sample valid designs beyond the available data distribution. To this end, we address the challenge of leveraging access to a verifier (e.g., an atomic bonds checker), to adapt a pre-trained flow model so that its induced density expands beyond regions of high data availability, while preserving samples validity. We introduce formal notions of strong and weak verifiers and propose algorithmic frameworks for global and local flow expansion via probability-space optimization. Then, we present Flow Expander (FE), a scalable mirror descent scheme that provably tackles both problems by verifier-constrained
entropy maximization over the flow process noised state space. Next, we provide a thorough theoretical analysis of the proposed method, and state convergence guarantees under both idealized and general assumptions. Ultimately, we empirically evaluate our method on both illustrative, yet visually interpretable settings, and on a molecular design task showcasing the ability of FE to expand a pre-trained flow model increasing conformer diversity while preserving validity.
Read the full abstract ▾
All authors Riccardo De Santi, Kimon Protopapas, Ya-Ping Hsieh, Andreas Krause
Machine Learning · Cross-listed from cs.SE
ReLoop: Structured Modeling and Behavioral Verification for Reliable LLM-Based Optimization
Junbo Jacob Lian, Yujun Sun, Huiling Chen et al. (+3 more)
Large
language models (LLMs) can translate natural-
language problem descriptions into optimization code, but the code is prone to silent failures: it executes and returns a solver-feasible solution while encoding a semantically incorrect formulation. On compositional problems, the resulting feasibility-correctness gap reaches 90 percentage points. We introduce ReLoop, which combines two mechanisms. Structured generation decomposes code production into a four-stage
reasoning chain (understand, formalize, synthesize, verify) to reduce formulation errors during generation. Behavioral verification detects the errors that remain by testing whether the formulation responds correctly to solver-based parameter perturbation, a signal that comes from the solver rather than from LLM self-review and requires no ground truth. The two mechanisms address different error structures: structured generation gives the largest gain on compositional problems (+8.5pp accuracy on RetailOpt-190 with Claude Opus 4.6), and behavioral verification gives its largest gain on localized defects (+4.4pp on MAMO-ComplexLP). With diagnostic execution recovery, ReLoop reaches 100% executable code on Claude Opus 4.6, and relative to direct generation it raises or preserves every reported metric of the three chat-tuned foundation models on all three
benchmarks. For the narrowly fine-tuned SFT model we test, the chain-of-thought
prompt conflicts with its learned output format and lowers its accuracy on MAMO-ComplexLP; we document and analyze this interaction. We release RetailOpt-190, 190 compositional retail optimization scenarios in which several constraints interact.
Read the full abstract ▾
All authors Junbo Jacob Lian, Yujun Sun, Huiling Chen, Chaoyu Zhang, Hanzhang Qin, Chung-Piaw Teo
Machine Learning
Fast Online Learning with Gaussian Prior-Driven Hierarchical Unimodal Thompson Sampling
Tianchi Zhao, He Liu, Hongyin Shi et al. (+1 more)
We study a type of Multi-Armed Bandit (MAB) problems in which arms with a Gaussian reward feedback are clustered. Such an arm setting finds applications in many real-world problems, for example, mmWave communications and portfolio management with risky assets, as a result of the universality of the Gaussian distribution. Based on the Thompson Sampling
algorithm with Gaussian prior (TSG)
algorithm for the selection of the optimal arm, we propose our Thompson Sampling with Clustered arms under Gaussian prior (TSCG) specific to the 2-level hierarchical structure. We prove that by utilizing the 2-level structure, we can achieve a lower regret bound than we do with ordinary TSG. In addition, when the reward is Unimodal, we can reach an even lower bound on the regret by our Unimodal Thompson Sampling
algorithm with Clustered Arms under Gaussian prior (UTSCG). Each of our proposed
algorithms are accompanied by theoretical evaluation of the upper regret bound, and our numerical experiments confirm the advantage of our proposed
algorithms.
Read the full abstract ▾
All authors Tianchi Zhao, He Liu, Hongyin Shi, Jinliang Li
Machine Learning
B-DENSE: Branching For Dense Ensemble Network Supervision Efficiency
Cherish Puniani, Tushar Kumar, Arnav Bendre et al. (+2 more)
Inspired by non-
equilibrium thermodynamics,
diffusion models have achieved state-of-the-art performance in generative modeling. However, their iterative sampling nature results in high inference latency. While recent
distillation techniques accelerate sampling, they discard intermediate trajectory steps. This sparse supervision leads to a loss of structural information and introduces significant discretization errors. To mitigate this, we propose B-DENSE, a novel framework that leverages multi-branch trajectory
alignment. We modify the student architecture to output $K$-fold expanded channels, where each subset corresponds to a specific branch representing a discrete intermediate step in the teacher's trajectory. By training these branches to simultaneously map to the entire sequence of the teacher's target timesteps, we enforce dense intermediate trajectory
alignment. Consequently, the student model learns to navigate the solution space from the earliest stages of training, demonstrating superior image generation quality compared to baseline
distillation frameworks.
Read the full abstract ▾
All authors Cherish Puniani, Tushar Kumar, Arnav Bendre, Gaurav Kumar, Shree Singhi
Machine Learning
R$^2$Energy: A Large-Scale Benchmark for Robust Renewable Energy Forecasting under Diverse and Extreme Conditions
Zhi Sheng, Yuan Yuan, Guozhen Zhang et al. (+1 more)
The rapid expansion of renewable energy, particularly wind and
solar power, has made reliable
forecasting critical for power system operations. While recent deep learning models have achieved strong average accuracy, the increasing frequency and intensity of
climate-driven extreme weather events pose severe threats to grid stability and operational security. Consequently, developing robust
forecasting models that can withstand volatile conditions has become a paramount challenge. In this paper, we present R$^2$Energy, a large-scale
benchmark for NWP-assisted renewable energy
forecasting. It comprises over 10.7 million high-fidelity hourly records from 902 wind and
solar stations across four provinces in China, providing the diverse meteorological conditions necessary to capture the wide-ranging variability of renewable generation. We further establish a standardized, leakage-free
forecasting paradigm that grants all models identical access to future Numerical Weather Prediction (NWP) signals, enabling fair and reproducible comparison across state-of-the-art representative
forecasting architectures. Beyond aggregate accuracy, we incorporate regime-wise evaluation with expert-aligned extreme weather annotations, uncovering a critical “robustness gap” typically obscured by average metrics. This gap reveals a stark robustness-complexity trade-off: under extreme conditions, a model's reliability is driven by its meteorological integration strategy rather than its architectural complexity. R$^2$Energy provides a principled foundation for evaluating and developing
forecasting models for safety-critical power system applications.
Read the full abstract ▾
All authors Zhi Sheng, Yuan Yuan, Guozhen Zhang, Yong Li
Machine Learning
Adaptive Semi-Supervised Training of P300 ERP-BCI Speller System with Minimum Calibration Effort
Shumeng Chen, Jane E. Huggins and Tianwen Ma
A P300 ERP-based
Brain-Computer Interface (BCI) speller is an assistive communication tool. It searches for the P300 event-related potential (ERP) elicited by target stimuli, distinguishing it from the neural responses to non-target stimuli embedded in electroencephalogram (
EEG) signals. Conventional methods require a lengthy calibration procedure to construct the binary classifier, which reduced overall efficiency. Thus, we proposed a unified framework with minimum calibration effort such that, given a small amount of labeled calibration data, we employed an adaptive semi-supervised EM-GMM
algorithm to update the binary classifier. We evaluated our method based on character-level prediction accuracy, information transfer rate (ITR), and BCI utility. We applied calibration on training data and reported results on testing data. Our results indicate that, out of 15 participants, 9 participants exceed the minimum character-level accuracy of 0.7 using either on our adaptive method or the
benchmark, and 7 out of these 9 participants showed that our adaptive method performed better than the
benchmark. The proposed semi-supervised learning framework provides a practical and efficient alternative to improve the overall spelling efficiency in the real-time BCI speller system, particularly in contexts with limited labeled data.
Read the full abstract ▾
Machine Learning · Cross-listed from astro-ph.CO
MadEvolve: Evolutionary Optimization of Cosmological Algorithms with Large Language Models
Tianyi Li, Shihui Zang and Moritz Münchmeyer
We develop a general framework to discover scientific
algorithms and apply it to three problems in computational cosmology. Our code, MadEvolve, is similar to Google's AlphaEvolve, but places a stronger emphasis on free parameters and their optimization. Our code starts with a baseline human
algorithm implementation, and then optimizes its performance metrics by making iterative changes to its code. As a further convenient feature, MadEvolve automatically generates a report that compares the input
algorithm with the evolved
algorithm, describes the algorithmic innovations and lists the free parameters and their function. Our code supports both auto-differentiable, gradient-based parameter optimization and gradient-free optimization methods. We apply MadEvolve to the reconstruction of cosmological initial conditions, 21cm foreground contamination reconstruction and effective baryonic physics in N-body simulations. In all cases, we find substantial improvements over the base
algorithm. We make MadEvolve and our three tasks publicly available at madevolve.org.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Can Vision-Language Models See Squares? Text-Recognition Mediates Spatial Reasoning Across Three Model Families
Yuval Levental
We present a simple experiment that exposes a fundamental limitation in
vision-
language models (VLMs): the inability to accurately localize filled
cells in binary grids when those
cells lack textual identity. We generate fifteen 15x15 grids with varying density (10.7%-41.8% filled
cells) and render each as two image types – text symbols (. and #) and filled squares without gridlines – then ask three frontier VLMs (Claude Opus, ChatGPT 5.2, and Gemini 3 Thinking) to transcribe them. In the text-symbol condition, Claude and ChatGPT achieve approximately 91%
cell accuracy and 84% F1, while Gemini achieves 84% accuracy and 63% F1. In the filled-squares condition, all three models collapse to 60-73% accuracy and 29-39% F1. Critically, all conditions pass through the same visual encoder – the text symbols are images, not tokenized text. The text-vs-squares F1 gap ranges from 34 to 54 points across models, demonstrating that VLMs behave as if they possess a high-fidelity text-recognition pathway for spatial
reasoning that dramatically outperforms their native visual pathway. Each model exhibits a distinct failure mode in the squares condition – systematic under-counting (Claude), massive over-counting (ChatGPT), and template
hallucination (Gemini) – but all share the same underlying deficit: severely degraded spatial localization for non-textual visual elements.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Visual Memory Injection Attacks for Multi-Turn Conversations
Christian Schlarmann and Matthias Hein
Generative large
vision-
language models (LVLMs) have recently achieved impressive performance gains, and their user base is growing rapidly. However, the security of LVLMs, in particular in a long-context multi-turn setting, is largely underexplored. In this paper, we consider the realistic scenario in which an attacker uploads a manipulated image to the web/social media. A benign user downloads this image and uses it as input to the LVLM. Our novel stealthy Visual
Memory Injection (VMI) attack is designed such that on normal
prompts the LVLM exhibits nominal behavior, but once the user gives a triggering
prompt, the LVLM outputs a specific prescribed target message to manipulate the user, e.g. for
adversarial marketing or political persuasion. Compared to previous work that focused on single-turn attacks, VMI is effective even after a long multi-turn conversation with the user. We demonstrate our attack on several recent open-weight LVLMs. This article thereby shows that large-scale manipulation of users is feasible with perturbed images in multi-turn conversation settings, calling for better robustness of LVLMs against these attacks. We release the source code at https://github.com/chs20/visual-
memory-injection
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
A Study on Real-time Object Detection using Deep Learning
Ankita Bose, Jayasravani Bhumireddy and Naveen N
Object detection has compelling applications over a range of domains, including human-computer interfaces, security and
video surveillance,
navigation and road traffic monitoring, transportation systems, industrial automation healthcare, the world of Augmented Reality (AR) and Virtual Reality (VR), environment monitoring and activity identification. Applications of real time object detection in all these areas provide dynamic analysis of the visual information that helps in immediate decision making. Furthermore, advanced deep learning
algorithms leverage the progress in the field of object detection providing more accurate and efficient solutions. There are some outstanding deep learning
algorithms for object detection which includes, Faster R CNN(Region-based Convolutional Neural Network),Mask R-CNN, Cascade R-CNN, YOLO (You Only Look Once), SSD (Single Shot Multibox Detector), RetinaNet etc. This article goes into great detail on how deep learning
algorithms are used to enhance real time object recognition. It provides information on the different object detection models available, open
benchmark datasets, and studies on the use of object detection models in a range of applications. Additionally, controlled studies are provided to compare various strategies and produce some illuminating findings. Last but not least, a number of encouraging challenges and approaches are offered as suggestions for further investigation in both relevant deep learning approaches and object recognition.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation
Zier Mensch, Lars Holdijk, Samuel Duffield et al. (+4 more)
Stochastic-gradient MCMC methods enable scalable
Bayesian posterior sampling but often suffer from sensitivity to minibatch size and gradient noise. To address this, we propose
Stochastic Gradient
Lattice Random Walk (SGLRW), an extension of the
Lattice Random Walk discretization. Unlike conventional
Stochastic Gradient Langevin Dynamics (SGLD), SGLRW introduces
stochastic noise only through the off-diagonal elements of the update covariance; this yields greater robustness to minibatch size while retaining asymptotic correctness. Furthermore, as comparison we analyze a natural analogue of SGLD utilizing gradient clipping. Experimental validation on
Bayesian regression and classification demonstrates that SGLRW remains stable in regimes where SGLD fails, including in the presence of heavy-tailed gradient noise, and matches or improves predictive performance.
Read the full abstract ▾
All authors Zier Mensch, Lars Holdijk, Samuel Duffield, Maxwell Aifer, Patrick J. Coles, Max Welling, Miranda C. N. Cheng
Machine Learning · Cross-listed from cond-mat.mtrl-sci
A fully differentiable framework for training proxy Exchange Correlation Functionals for periodic systems
Rakshit Kumar Singh, Aryan Amit Barsainyan and Bharath Ramsundar
Density Functional Theory (DFT) is widely used for first-principles simulations in chemistry and materials science, but its computational cost remains a key limitation for large systems. Motivated by recent advances in ML-based exchange-correlation (XC) functionals, this paper introduces a differentiable framework that integrates machine learning models into density functional theory (DFT) for solids and other periodic systems. The framework defines a clean API for neural network models that can act as drop in replacements for conventional exchange-correlation (XC) functionals and enables gradients to flow through the full self-consistent DFT workflow. The framework is implemented in Python using a PyTorch backend, making it fully differentiable and easy to use with standard deep learning tools. We integrate the implementation with the DeepChem library to promote the reuse of established models and to lower the barrier for experimentation. In initial
benchmarks against established electronic structure packages (GPAW and PySCF), our models achieve relative errors on the order of 5-10%.
Read the full abstract ▾