Machine Learning · Cross-listed from eess.SP
A Multimodal Deep Learning Framework for Predicting ICU Deterioration: Integrating ECG Waveforms with Clinical Data and Clinician Benchmarking
Juan Miguel López Alcaraz, Xicoténcatl López Moran, Erick Dávila Zaragoza et al. (+4 more)
Artificial intelligence holds strong potential to support clinical decision making in intensive care units where timely and accurate risk assessment is critical. However, many existing models focus on isolated outcomes or limited data types, while clinicians integrate longitudinal history, real time physiology, and heterogeneous clinical information. To address this gap, we developed MDS ICU, a unified
multimodal machine learning framework that fuses routinely collected data including demographics, biometrics, vital signs, laboratory values, ECG waveforms,
surgical procedures, and medical device usage to provide continuous predictive support during ICU stays. Using 63001 samples from 27062 patients in MIMIC IV, we trained a deep learning architecture that combines structured state space S4 encoders for ECG waveforms with multilayer perceptron RealMLP encoders for tabular data to jointly predict 33 clinically relevant outcomes spanning mortality, organ dysfunction, medication needs, and acute deterioration. The model achieved strong discrimination with AUROCs of 0.90 for 24 hour mortality, 0.92 for sedative administration, 0.97 for invasive mechanical ventilation, and 0.93 for coagulation dysfunction. Calibration analysis showed close agreement between predicted and observed risks, with consistent gains from ECG waveform integration. Comparisons with clinicians and large
language models showed that model predictions alone outperformed both, and that providing model outputs as decision support further improved their performance. These results demonstrate that
multimodal AI can deliver clinically meaningful risk stratification across diverse ICU outcomes while augmenting rather than replacing clinical expertise, establishing a scalable foundation for precision critical care decision support.
Read the full abstract ▾
All authors Juan Miguel López Alcaraz, Xicoténcatl López Moran, Erick Dávila Zaragoza, Claas Händel, Richard Koebe, Wilhelm Haverkamp, Nils Strodthoff
Machine Learning
Quan Minh Nguyen, Min-Seon Kim, Hoang M. Ngo et al. (+3 more)
Membership inference attacks (MIAs) pose a serious
privacy threat in
federated learning (FL). While MIAs have been extensively studied in standard FL, the recent shift toward
federated fine-tuning introduces new and largely unexplored attack surfaces. In this work, we show that
federated prompt-tuning, which adapts pre-trained foundation models using lightweight input prefixes, exposes a novel and effective vector for membership inference. We propose PromptMIA, a membership inference attack tailored to
federated prompt-tuning, in which a malicious server introduces adversarially crafted
prompts and exploits their updates during collaborative training to determine whether a target data point belongs to a client's private
dataset. We formalize this threat via a security game and demonstrate that PromptMIA achieves consistently high attack advantage across diverse
benchmark datasets, substantially outperforming current SOTA
federated MIAs. We also provide a theoretical lower bound on the attack advantage that explains the observed empirical behavior. Finally, we show that existing MIA defenses are often ineffective against PromptMIA, highlighting the need for defense mechanisms specifically tailored to
prompt-tuning in
federated settings.
Read the full abstract ▾
All authors Quan Minh Nguyen, Min-Seon Kim, Hoang M. Ngo, Trong Nghia Hoang, Hyuk-Yoon Kwon, My T. Thai
Machine Learning
KASER: Knowledge-Aligned Student Error Simulator for Open-Ended Coding Tasks
Zhangqi Duan, Nigel Fernandez and Andrew Lan
Open-ended tasks, such as coding problems that are common in computer science education, provide detailed insights into student knowledge. However, training large
language models (LLMs) to simulate and predict possible student errors in their responses to these problems can be challenging: they often suffer from mode collapse and fail to fully capture the diversity in syntax, style, and solution approach in student responses. In this work, we present KASER (Knowledge-Aligned Student Error Simulator), a novel approach that aligns errors with student knowledge. We propose a training method based on reinforcement learning using a hybrid reward that reflects three aspects of student code prediction: i) code similarity to the ground-truth, ii) error matching, and iii) code prediction diversity. On two real-world
datasets, we perform two levels of evaluation and show that: At the per-student-problem pair level, our method outperforms baselines on code and error prediction; at the per-problem level, our method outperforms baselines on error coverage and simulated code diversity.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DS
Lower Bounds for the Algorithmic Complexity of Learned Indexes
Luis Alberto Croquevielle, Roman Sokolovskii and Thomas Heinis
Learned index structures aim to accelerate queries by training machine learning models to approximate the rank function associated with a database attribute. While effective in practice, their theoretical limitations are not fully understood. We present a general framework for proving lower bounds on query time for learned indexes, expressed in terms of their space overhead and parameterized by the model class used for approximation. Our formulation captures a broad family of learned indexes, including most existing designs, as piecewise model-based predictors. We solve the problem of lower bounding query time in two steps: first, we use probabilistic tools to control the effect of sampling when the database attribute is drawn from a probability distribution. Then, we analyze the approximation-theoretic problem of how to optimally represent a cumulative distribution function with approximators from a given model class. Within this framework, we derive lower bounds under a range of modeling and distributional assumptions, paying particular
attention to the case of piecewise linear and piecewise constant model classes, which are common in practical implementations. Our analysis shows how tools from approximation theory, such as quantization and Kolmogorov widths, can be leveraged to formalize the space-time tradeoffs inherent to learned index structures. The resulting bounds illuminate core limitations of these methods.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Cross-Border Data Security and Privacy Risks in Large Language Models and IoT Systems
Chalitha Handapangoda
The reliance of Large
Language Models and Internet of Things systems on massive, globally distributed data flows creates systemic security and
privacy challenges. When data traverses borders, it becomes subject to conflicting legal regimes, such as the EU's General Data Protection Regulation and China's Personal Information Protection Law, compounded by technical vulnerabilities like model memorization. Current static encryption and data localization methods are fragmented and reactive, failing to provide adequate, policy-aligned safeguards. This research proposes a Jurisdiction-Aware,
Privacy-by-Design architecture that dynamically integrates localized encryption, adaptive differential
privacy, and real-time compliance assertion via cryptographic proofs. Empirical validation in a multi-jurisdictional simulation demonstrates this architecture reduced unauthorized data exposure to below five percent and achieved zero compliance violations. These security gains were realized while maintaining model utility retention above ninety percent and limiting computational overhead. This establishes that proactive, integrated controls are feasible for secure and globally compliant AI deployment.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Pragya: An AI-Based Semantic Recommendation System for Sanskrit Subhasitas
Tanisha Raorane and Prasenjit Kole
Sanskrit Subhasitas encapsulate centuries of cultural and philosophical wisdom, yet remain underutilized in the digital age due to linguistic and contextual barriers. In this work, we present Pragya, a
retrieval-augmented generation (RAG) framework for semantic recommendation of Subhasitas. We curate a
dataset of 200 verses annotated with thematic tags such as motivation, friendship, and compassion. Using sentence
embeddings (IndicBERT), the system retrieves top-k verses relevant to user queries. The retrieved results are then passed to a generative model (Mistral LLM) to produce transliterations, translations, and contextual explanations. Experimental evaluation demonstrates that semantic
retrieval significantly outperforms keyword matching in precision and relevance, while user studies highlight improved accessibility through generated summaries. To our knowledge, this is the first attempt at integrating
retrieval and generation for Sanskrit Subhasitas, bridging cultural heritage with modern applied AI.
Read the full abstract ▾
Machine Learning
CEDAR: Context Engineering for Agentic Data Science
Rishiraj Saha Roy, Chris Hinze, Luzian Hahn et al. (+1 more)
We demonstrate CEDAR, an application for automating data science (DS) tasks with an agentic setup. Solving DS problems with LLMs is an underexplored area that has immense
market value. The challenges are
manifold: task complexities, data sizes, computational limitations, and context restrictions. We show that these can be alleviated via effective context engineering. We first impose structure into the initial
prompt with DS-specific input fields, that serve as instructions for the agentic system. The solution is then materialized as an enumerated sequence of interleaved plan and code blocks generated by separate LLM
agents, providing a readable structure to the context at any step of the workflow. Function calls for generating these intermediate texts, and for corresponding Python code, ensure that data stays local, and only aggregate statistics and associated instructions are injected into LLM
prompts. Fault tolerance and context management are introduced via iterative code generation and smart history rendering. The viability of our agentic data scientist is demonstrated using canonical Kaggle challenges.
Read the full abstract ▾
All authors Rishiraj Saha Roy, Chris Hinze, Luzian Hahn, Fabian Kuech
Machine Learning · Cross-listed from cs.AI
Object-Centric World Models Meet Monte Carlo Tree Search
Rodion Vakhitov, Leonid Ugadiarov and Aleksandr Panov
In this paper, we introduce ObjectZero, a novel reinforcement learning (RL)
algorithm that leverages the power of object-level representations to model dynamic environments more effectively. Unlike traditional approaches that process the world as a single undifferentiated input, our method employs Graph Neural Networks (GNNs) to capture intricate interactions among multiple objects. These objects, which can be manipulated and interact with each other, serve as the foundation for our model's understanding of the environment. We trained the
algorithm in a complex setting teeming with diverse, interactive objects, demonstrating its ability to effectively learn and predict object dynamics. Our results highlight that a structured world model operating on object-centric representations can be successfully integrated into a model-based RL
algorithm utilizing Monte Carlo Tree Search as a planning module.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
UMLoc: Uncertainty-Aware Map-Constrained Inertial Localization with Quantified Bounds
Mohammed S. Alharbi and Shinkyu Park
Inertial localization is particularly valuable in GPS-denied environments such as indoors. However, localization using only Inertial Measurement Units (IMUs) suffers from drift caused by motion-process noise and sensor biases. This paper introduces Uncertainty-aware Map-constrained Inertial Localization (UMLoc), an end-to-end framework that jointly models IMU uncertainty and map constraints to achieve drift-resilient positioning. UMLoc integrates two coupled modules: (1) a Long Short-Term
Memory (LSTM)
quantile regressor, which estimates the specific
quantiles needed to define 68%, 90%, and 95% prediction intervals serving as a measure of localization uncertainty and (2) a Conditioned Generative
Adversarial Network (CGAN) with cross-
attention that fuses IMU dynamic data with distance-based floor-plan maps to generate geometrically feasible trajectories. The modules are trained jointly, allowing uncertainty estimates to propagate through the CGAN during trajectory generation. UMLoc was evaluated on three
datasets, including a newly collected 2-hour indoor
benchmark with time-aligned IMU data, ground-truth poses and floor-plan maps. Results show that the method achieves a mean drift ratio of 5.9% over a 70 m travel distance and an average Absolute Trajectory Error (ATE) of 1.36 m, while maintaining calibrated prediction bounds.
Read the full abstract ▾
Machine Learning
Understanding and inverse design of implicit bias in stochastic learning: a geometric perspective
Nicola Aladrah, Emanuele Ballarin, Matteo Biagetti et al. (+3 more)
Can we design a model such that its
stochastic training favours a desired class of solutions without enforcing an explicit penalty? Under suitable conditions, the interplay between
symmetries of a model's weight parametrization and
stochastic training favours particular solutions, inducing an implicit bias. Building on this mechanism, we develop a framework for inverse-designing such biases by constructing novel parametrizations and their associated
symmetries. We show how holomorphic functions make this construction and calculation simple and explicit. Specifically, we introduce a new parametrization that biases learned weights toward the binary values $\{-1,+1\}$. Numerical experiments confirm the theoretical predictions. They also show that our parametrization reproduces the same preference induced by an explicitly regularized model without adding a penalty to the training loss.
Read the full abstract ▾
All authors Nicola Aladrah, Emanuele Ballarin, Matteo Biagetti, Alessio Ansuini, Alberto d'Onofrio, Fabio Anselmi
Machine Learning · Cross-listed from cs.CL
Detecting LLM-Generated Text with Performance Guarantees
Hongyi Zhou, Jin Zhu, Ying Yang et al. (+1 more)
Large
language models (LLMs) such as GPT, Claude, Gemini, and Grok have been deeply integrated into our daily life. They now support a wide range of tasks – from dialogue and email drafting to assisting with teaching and coding, serving as search engines, and much more. However, their ability to produce highly human-like text raises serious concerns, including the spread of fake news, the generation of misleading governmental reports, and academic misconduct. To address this practical problem, we train a classifier to determine whether a piece of text is authored by an LLM or a human. Our detector is deployed on an online CPU-based platform https://huggingface.co/spaces/stats-powered-ai/StatDetectLLM, and contains three novelties over existing detectors: (i) it does not rely on auxiliary information, such as watermarks or knowledge of the specific LLM used to generate the text; (ii) it more effectively distinguishes between human- and LLM-authored text; and (iii) it enables statistical inference, which is largely absent in the current literature. Empirically, our classifier achieves higher classification accuracy compared to existing detectors, while maintaining type-I error control, high statistical power, and computational efficiency.
Read the full abstract ▾
All authors Hongyi Zhou, Jin Zhu, Ying Yang, Chengchun Shi
Machine Learning
Softly Induced Functional Simplicity: Implications for Neural Network Generalisation, Robustness, and Distillation
Maciej Glowacki
Learning robust and generalisable abstractions from high-dimensional input data is a central challenge in machine learning and its applications to high-energy physics (HEP). Solutions of lower functional complexity are known to produce abstractions that generalise more effectively and are more robust to input perturbations. In complex hypothesis spaces, inductive biases make such solutions learnable by shaping the loss geometry during optimisation. In a HEP classification task, we show that a soft
symmetry respecting inductive bias creates approximate degeneracies in the loss, which we identify as pseudo-Goldstone modes. We quantify functional complexity using metrics derived from first principles Hessian analysis and via compressibility. Our results demonstrate that solutions of lower complexity give rise to abstractions that are more generalisable, robust, and efficiently distillable.
Read the full abstract ▾
Machine Learning
Hellinger Multimodal Variational Autoencoders
Huyen Vo and Isabel Valera
Multimodal variational autoencoders (VAEs) are widely used for weakly supervised generative learning with multiple modalities. Predominant methods aggregate unimodal inference distributions using either a product of experts (PoE), a mixture of experts (MoE), or their combinations to approximate the joint posterior. In this work, we revisit
multimodal inference through the lens of probabilistic opinion pooling, an optimization-based approach. We start from Hölder pooling with $\alpha=0.5$, which corresponds to the unique symmetric member of the $\alpha\text{-divergence}$ family, and derive a moment-matching approximation, termed Hellinger. We then leverage such an approximation to propose HELVAE, a
multimodal VAE that avoids sub-sampling, yielding an efficient yet effective model that: (i) learns more expressive latent representations as additional modalities are observed; and (ii) empirically achieves better trade-offs between generative coherence and quality, outperforming state-of-the-art
multimodal VAE models.
Read the full abstract ▾
Machine Learning
Mosaic: Unlocking Over 30$\times$ Context Length for Diffusion LLMs Inference via Global Memory Planning and Dynamic Peak Taming
Liang Zheng, Bowen Shi, Yitao Hu et al. (+10 more)
Diffusion-based large
language models (dLLMs) have emerged as a promising alternative to autoregressive models, leveraging simultaneous denoising to enable global planning and iterative refinement. These properties make dLLMs attractive for long-context generation. However, deploying dLLMs faces a prohibitive
memory barrier, as existing inference systems are inefficient for the
diffusion paradigm. We observe that current inference systems are misaligned with dLLMs. Unlike autoregressive models, whose
memory footprint is dominated by the KV-Cache, dLLMs are bottlenecked by transient activations rematerialized per step. Moreover, generic
memory reuse mechanisms lack the global visibility to handle dynamic
memory peaks of dLLMs, which alternate between logits and feed-forward networks. To address these challenges, we present Mosaic, a
memory-efficient inference system that shifts dLLM execution from local, static management to a global, dynamic paradigm. Mosaic integrates (i) a mask-only logits kernel eliminating redundant activation materialization, (ii) a lazy chunking optimizer using online heuristics to tame dynamic
memory peaks, and (iii) a global
memory manager leveraging virtual addressing to mitigate
memory fragmentation. Evaluations show that Mosaic reduces the
memory peak-to-average ratio by 2.71$\times$ on average and increases the maximum inference sequence length on identical hardware by 15.30-32.34$\times$. Crucially, Mosaic is training-free and preserves exact model outputs, while reducing end-to-end latency by 2.5%-55.4%. Our code is publicly available at https://github.com/flashserve/Mosaic.
Read the full abstract ▾
All authors Liang Zheng, Bowen Shi, Yitao Hu, Jiawei Zhang, Ruofan Li, Guotao Yang, Zhixin Zhao, Zhengchao Wang, Sheng Chen, Wenxin Li, Dezhi Ran, Tao Xie, Keqiu Li
Machine Learning · Cross-listed from cs.CL
Mechanism-Faithful Queueing Simulation Model Translation with Large Language Model Support
Jun-Qi Chen, Kun Zhang, Rui Zheng et al. (+1 more)
Queueing simulation studies often require substantial manual effort to translate conceptual system descriptions into executable programs and to verify that the implemented mechanisms match the intended queueing logic. Although large
language models (LLMs) may produce executable scripts, executability alone is insufficient when arrival, routing, interruption, or reporting logic is wrong. This study presents a simulation-oriented support framework for SimPy-based queueing model translation. We propose a category-template framework for mechanism coverage with a staged adaptation workflow that targets structured event logic and common simulation-specific failure modes. On held-out task instances, the adapted models improve executability, output-format compliance, and instruction-mechanism consistency across basic, behavioral, and networked queueing settings, so the generated scripts are more reliable as queueing simulation scripts. Error analysis shows better preservation of routing semantics and interruption-resume logic, while also exposing remaining weaknesses in multi-node transfer and residual-service updates. Overall, the results suggest that the proposed framework can act as a simulation-faithful generator for more standardized and reproducible queueing model construction.
Read the full abstract ▾
All authors Jun-Qi Chen, Kun Zhang, Rui Zheng, Ying Zhong
Machine Learning · Cross-listed from eess.SY
Self-Organizing Dual-Buffer Adaptive Clustering Experience Replay (SODACER) for Safe Reinforcement Learning in Optimal Control
Roya Khalili Amirabadi, Mohsen Jalaeian Farimani and Omid Solaymani Fard
This paper proposes a novel reinforcement learning framework, named Self-Organizing Dual-buffer Adaptive Clustering Experience Replay (SODACER), designed to achieve safe and scalable optimal control of nonlinear systems. The proposed SODACER mechanism consisting of a Fast-Buffer for rapid adaptation to recent experiences and a Slow-Buffer equipped with a self-organizing adaptive clustering mechanism to maintain diverse and non-redundant historical experiences. The adaptive clustering mechanism dynamically prunes redundant samples, optimizing
memory efficiency while retaining critical environmental patterns. The approach integrates SODACER with Control Barrier Functions (CBFs) to guarantee safety by enforcing state and input constraints throughout the learning process. To enhance convergence and stability, the framework is combined with the Sophia optimizer, enabling adaptive second-order gradient updates. The proposed SODACER-Sophia's architecture ensures reliable, effective, and robust learning in dynamic, safety-critical environments, offering a generalizable solution for applications in robotics, healthcare, and large-scale system optimization. The proposed approach is validated on a nonlinear Human Papillomavirus (HPV) transmission model with multiple control inputs and safety constraints. Comparative evaluations against random and clustering-based experience replay methods demonstrate that SODACER achieves faster convergence, improved sample efficiency, and a superior bias-variance trade-off, while maintaining safe system trajectories, validated via the Friedman test.
Read the full abstract ▾
Machine Learning
Short-term electricity load forecasting with multi-frequency reconstruction diffusion
Qi Dong, Rubing Huang, Ling Zhou et al. (+3 more)
Diffusion models have emerged as a powerful method in various applications. However, their application to Short-Term Electricity Load
Forecasting (STELF) – a typical scenario in energy systems – remains largely unexplored. Considering the nonlinear and fluctuating characteristics of the load data, effectively utilizing the powerful modeling capabilities of
diffusion models to enhance STELF accuracy remains a challenge. This paper proposes a novel
diffusion model with multi-frequency reconstruction for STELF, referred to as the Multi-Frequency-Reconstruction-based
Diffusion (MFRD) model. The MFRD model achieves accurate load
forecasting through four key steps: (1) The original data is combined with the decomposed multi-frequency modes to form a new data representation; (2) The
diffusion model adds noise to the new data, effectively reducing and weakening the noise in the original data; (3) The reverse process adopts a denoising network that combines Long Short-Term
Memory (LSTM) and
Transformer to enhance noise removal; and (4) The inference process generates the final predictions based on the trained denoising network. To validate the effectiveness of the MFRD model, we conducted experiments on two data platforms: Australian Energy
Market Operator (AEMO) and Independent System Operator of New England (ISO-NE). The experimental results show that our model consistently outperforms the compared models.
Read the full abstract ▾
All authors Qi Dong, Rubing Huang, Ling Zhou, Dave Towey, Jinyu Tian, Jianzhou Wang
Machine Learning
Improving Day-Ahead Grid Carbon Intensity Forecasting by Joint Modeling of Local-Temporal and Cross-Variable Dependencies Across Different Frequencies
Bowen Zhang, Hongda Tian, Adam Berry et al. (+1 more)
Accurate
forecasting of the grid carbon intensity factor (CIF) is critical for enabling demand-side management and reducing emissions in modern electricity systems. Leveraging multiple interrelated time series, CIF prediction is typically formulated as a multivariate time series
forecasting problem. Despite advances in deep learning-based methods, it remains challenging to capture the fine-grained local-temporal dependencies, dynamic higher-order cross-variable dependencies, and complex multi-frequency patterns for CIF
forecasting. To address these issues, we propose a novel model that integrates two parallel modules: 1) one enhances the extraction of local-temporal dependencies under multi-frequency by applying multiple wavelet-based convolutional kernels to overlapping patches of varying lengths; 2) the other captures dynamic cross-variable dependencies under multi-frequency to model how inter-variable relationships evolve across the time-frequency domain. Evaluations on four representative electricity
markets from Australia, featuring varying levels of renewable penetration, demonstrate that the proposed method outperforms the state-of-the-art models. An ablation study further validates the complementary benefits of the two proposed modules. Designed with built-in interpretability, the proposed model also enables better understanding of its predictive behavior, as shown in a case study where it adaptively shifts
attention to relevant variables and time intervals during a disruptive event.
Read the full abstract ▾
All authors Bowen Zhang, Hongda Tian, Adam Berry, A. Craig Roussac
Machine Learning · Cross-listed from cs.HC
Pareto-Optimal Model Selection for Low-Cost, Single-Lead EMG Control in Embedded Systems
Carl Vincent Ladres Kho
Consumer-grade biosensors offer a cost-effective alternative to medical-grade electromyography (EMG) systems, reducing hardware costs from thousands of dollars to approximately $13. However, these low-cost sensors introduce significant signal instability and motion artifacts. Deploying machine learning models on resource-constrained edge devices like the ESP32 presents a challenge: balancing classification accuracy with strict latency (<100ms) and
memory (<320KB) constraints. Using a single-subject
dataset comprising 1,540 seconds of raw data (1.54M data points, segmented into ~1,300 one-second windows), I evaluate 18 model architectures, ranging from statistical heuristics to deep transfer learning (ResNet50) and custom hybrid networks (MaxCRNN). While my custom "MaxCRNN" (Inception + Bi-LSTM +
Attention) achieved the highest safety (99% Precision) and robustness, I identify Random Forest (74% accuracy) as the Pareto-optimal solution for embedded control on legacy microcontrollers. I demonstrate that reliable, low-latency EMG control is feasible on commodity hardware, with Deep Learning offering a path to near-perfect reliability on modern Edge AI accelerators.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matching
Jinyuan Chang, Chenguang Duan, Yuling Jiao et al. (+3 more)
Inference-time
alignment for
diffusion models aims to adapt a pre-trained reference
diffusion model toward a target distribution without retraining the reference score network, thereby preserving the generative capacity of the reference model while enforcing desired properties at inference time. A central mechanism for achieving such
alignment is guidance, which modifies the sampling dynamics through an additional drift term. In this work, we introduce variationally stable Doob's matching, a novel framework for provable guidance estimation grounded in Doob's $h$-transform. Our approach formulates guidance as the gradient of the logarithm of an underlying Doob's $h$-function and employs gradient-regularized regression to simultaneously estimate both the $h$-function and its gradient, resulting in a consistent
estimator of the guidance. Theoretically, we establish non-asymptotic convergence rates for the estimated guidance. Moreover, we analyze the resulting controllable
diffusion processes and prove non-asymptotic convergence guarantees for the generated distributions in the Wasserstein-2 distance. Finally, we show that variationally stable guidance
estimators are adaptive to unknown low dimensionality, effectively mitigating the curse of dimensionality under low-dimensional linear subspace assumptions. Numerical experiments demonstrate the effectiveness of our proposed method and show the advantages compared to the baselines.
Read the full abstract ▾
All authors Jinyuan Chang, Chenguang Duan, Yuling Jiao, Ruoxuan Li, Yi Xu, Jerry Zhijian Yang
Machine Learning
A novel RF-enabled Non-Destructive Inspection Method through Machine Learning and Programmable Wireless Environments
Stavros Tsimpoukis, Dimitrios Tyrovolas, Sotiris Ioannidis et al. (+4 more)
Contemporary industrial Non-Destructive Inspection (NDI) methods require sensing capabilities that operate in occluded, hazardous, or access restricted environments. Yet, the current visual inspection based on optical cameras offers limited quality of service to that respect. In that sense, novel methods for workpiece inspection, suitable, for smart manufacturing are needed. Programmable Wireless Environments (PWE) could help towards that direction, by redefining the wireless
Radio Frequency (RF) wave propagation as a controllable inspector entity. In this work, we propose a novel approach to Non-Destructive Inspection, leveraging an RF sensing pipeline based on RF wavefront encoding for retrieving workpiece-image entries from a designated database. This approach combines PWE-enabled RF wave manipulation with machine learning (ML) tools trained to produce visual outputs for quality inspection. Specifically, we establish correlation relationships between RF wavefronts and target industrial assets, hence yielding a
dataset which links wavefronts to their corresponding images in a structured manner. Subsequently, a Generative
Adversarial Network (GAN) derives visual representations closely matching the database entries. Our results indicate that the proposed method achieves an SSIM 99.5% matching score in visual outputs, paving the way for next-generation quality control workflows in industry.
Read the full abstract ▾
All authors Stavros Tsimpoukis, Dimitrios Tyrovolas, Sotiris Ioannidis, Maria Kafesaki, Ian F. Akyildiz, George K. Karagiannidis, Christos K. Liaskos
Machine Learning
Neural Nonmyopic Bayesian Optimization in Dynamic Cost Settings
Sang T. Truong, Duc Q. Nguyen, Willie Neiswanger et al. (+4 more)
Bayesian optimization (BO) is a common framework for optimizing black-box functions, yet most existing methods assume static query costs and rely on myopic acquisition strategies. We introduce LookaHES, a nonmyopic BO framework designed for dynamic, history-dependent cost environments, where evaluation costs vary with prior actions, such as travel distance in spatial tasks or edit distance in sequence design. LookaHES combines a multi-step variant of $H$-
Entropy Search with pathwise sampling and neural policy optimization, enabling long-horizon planning beyond twenty steps without the exponential complexity of existing nonmyopic methods. The key innovation is the integration of neural policies, including large
language models, to effectively navigate structured, combinatorial action spaces such as
protein sequences. These policies amortize lookahead planning and can be integrated with domain-specific constraints during rollout. Empirically, LookaHES outperforms strong myopic and nonmyopic baselines across nine synthetic
benchmarks from two to eight dimensions and two real-world tasks: geospatial optimization using NASA night-light imagery and
protein sequence design with constrained
token-level edits. In short, LookaHES provides a general, scalable, and cost-aware solution for robust long-horizon optimization in complex decision spaces, which makes it a useful tool for researchers in machine learning, statistics, and applied domains. Our implementation is available at https://github.com/sangttruong/nonmyopia.
Read the full abstract ▾
All authors Sang T. Truong, Duc Q. Nguyen, Willie Neiswanger, Ryan-Rhys Griffiths, Stefano Ermon, Nick Haber, Sanmi Koyejo
Machine Learning
ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking
Qiang Zhang, Boli Chen, Fanrui Zhang et al. (+14 more)
Reinforcement learning has substantially improved the performance of LLM
agents on tasks with verifiable outcomes, but it still struggles on open-ended
agent tasks with vast solution spaces (e.g., complex travel planning). Due to the absence of objective ground-truth for these tasks, current RL
algorithms largely rely on reward models that assign scalar scores to individual responses. We contend that such pointwise scoring suffers from an inherent discrimination collapse: the reward model struggles to distinguish subtle advantages among different trajectories, resulting in scores within a group being compressed into a narrow range. Consequently, the effective reward signal becomes dominated by noise from the reward model, leading to optimization stagnation. To address this, we propose ArenaRL, a reinforcement learning paradigm that shifts from pointwise scalar scoring to intra-group relative ranking. ArenaRL introduces a process-aware pairwise evaluation mechanism, employing multi-level rubrics to assign fine-grained relative scores to trajectories. Additionally, we construct an intra-group
adversarial arena and devise a tournament-based ranking scheme to obtain stable advantage signals. Empirical results confirm that the built seeded single-elimination scheme achieves nearly equivalent advantage estimation accuracy to full pairwise comparisons with O(N^2) complexity, while operating with only O(N) complexity, striking an optimal balance between efficiency and precision. Furthermore, to address the lack of full-cycle
benchmarks for open-ended
agents, we build Open-Travel and Open-DeepResearch, two high-quality
benchmarks featuring a comprehensive pipeline covering SFT, RL training, and multi-dimensional evaluation. Extensive experiments show that ArenaRL substantially outperforms standard RL baselines, enabling LLM
agents to generate more robust solutions for complex real-world tasks.
Read the full abstract ▾
All authors Qiang Zhang, Boli Chen, Fanrui Zhang, Ruixue Ding, Shihang Wang, Qiuchen Wang, Yinfeng Huang, Haonan Zhang, Rongxiang Zhu, Pengyong Wang, Ailin Ren, Xin Li, Pengjun Xie, Jiawei Liu, Ning Guo, Jingren Zhou, Zheng-Jun Zha
Machine Learning
Deriving Decoder-Free Sparse Autoencoders from First Principles
Alan Oursland
Gradient descent on log-sum-exp (LSE) objectives performs implicit expectation–maximization (EM): the gradient with respect to each component output equals its responsibility. The same theory predicts collapse without volume control analogous to the log-determinant in Gaussian mixture models. We instantiate the theory in a single-layer encoder with an LSE objective and InfoMax regularization for volume control. Experiments confirm the theory's predictions. The gradient–responsibility identity holds exactly; LSE alone collapses; variance prevents dead components; decorrelation prevents redundancy. The model exhibits EM-like optimization dynamics in which lower loss does not correspond to better features and adaptive optimizers offer no advantage. The resulting decoder-free model learns interpretable mixture components, confirming that implicit EM theory can prescribe architectures.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.SY
Hybrid LSTM-UKF Framework: Ankle Angle and Ground Reaction Force Estimation
Mundla Narasimhappa and Praveen Kumar
Accurate prediction of joint kinematics and kinetics is essential for advancing gait analysis and developing intelligent assistive systems such as prosthetics and exoskeletons. This study presents a hybrid LSTM-UKF framework for estimating ankle angle and ground reaction force (GRF) across varying walking speeds. A
multimodal sensor fusion strategy integrates force plate data, knee angle, and GRF signals to enrich biomechanical context. Model performance was evaluated using RMSE and $R^2$ under subject-specific validation. The LSTM-UKF consistently outperformed standalone LSTM and UKF models, achieving up to 18.6% lower RMSE for GRF prediction at 3 km/h. Additionally, UKF integration improved robustness, reducing ankle angle RMSE by up to 22.4% compared to UKF alone at 1 km/h. These results underscore the effectiveness of hybrid architectures for reliable gait prediction across subjects and walking conditions.
Read the full abstract ▾
Machine Learning
StablePDENet: Enhancing Neural Operator Stability through Physics-Informed Residual-Sensitivity Regularization
Chutian Huang, Chang Ma, Kaibo Wang et al. (+1 more)
Learning solution operators for differential equations with neural networks has shown great potential in scientific computing, but ensuring their stability under input perturbations remains a critical challenge. We introduce the StablePDENet, a physics-informed
adversarial training method that regularizes the residual sensitivity with respect to an input perturbation. The operator learning task is formulated as a min–max optimization problem, where the inner model searches admissible input perturbations by physics-based projected-gradient adversary, while the outer problem combines the attacked physics loss with a normalized residual-sensitivity penalty. Moreover, residual-sensitivity regularization is included to ensure that the local Lipschitz constant of the learned operator is a more accurate approximation to that of the exact operator. We evaluate the StablePDENet on several
benchmark problems. Compared with PI-DeepONet and its adversarially trained variant, StablePDENet achieves higher accuracy under
adversarial input perturbations while maintaining competitive accuracy on clean inputs. The numerical results also demonstrate that the StablePDENet can effectively improve the generalization accuracy for operator learning. The Helmholtz study further distinguishes learned-model sensitivity from amplification intrinsic to an ill-conditioned solution operator. The results support residual-sensitivity regularization as a practical route to more stable and physically consistent neural PDE operators.
Read the full abstract ▾
All authors Chutian Huang, Chang Ma, Kaibo Wang, Yang Xiang
Machine Learning · Cross-listed from cs.CL
PRISP: Privacy-Safe Few-Shot Personalization via Lightweight Adaptation
Junho Park, Dohoon Kim and Taesup Moon
Large
language model (LLM) personalization aims to adapt general-purpose models to individual users. Most existing methods, however, are developed under data-rich and resource-abundant settings, often incurring
privacy risks. In contrast, realistic personalization typically occurs after deployment under (i) extremely limited user data, (ii) constrained computational resources, and (iii) strict
privacy requirements. We propose PRISP, a lightweight and
privacy-safe personalization framework tailored to these constraints. PRISP leverages a Text-to-LoRA hypernetwork to generate task-aware LoRA parameters from task descriptions, and enables efficient user personalization by optimizing a small subset of task-aware LoRA parameters together with minimal additional modules using few-shot user data. Experiments on a few-shot variant of the LaMP
benchmark demonstrate that PRISP achieves strong overall performance compared to prior approaches, while reducing computational overhead and eliminating
privacy risks.
Read the full abstract ▾
Machine Learning
Gecko: An Efficient Neural Architecture Inherently Processing Sequences with Arbitrary Lengths
Xuezhe Ma, Shicheng Wen, Linghao Jin et al. (+11 more)
Designing a unified neural network to efficiently and inherently process sequential data with arbitrary lengths is a central and challenging problem in sequence modeling. The design choices in
Transformer, including quadratic complexity and weak length extrapolation, have limited their ability to scale to long sequences. In this work, we propose Gecko, a neural architecture that inherits the design of Mega and Megalodon (exponential moving average with gated
attention), and further introduces multiple technical components to improve its capability to capture long range dependencies, including timestep decay normalization, sliding chunk
attention mechanism, and adaptive working
memory. In a controlled pretraining comparison with Llama2 and Megalodon in the scale of 7 billion parameters and 2 trillion training
tokens, Gecko achieves better efficiency and long-context scalability. Gecko reaches a training loss of 1.68, significantly outperforming Llama2-7B (1.75) and Megalodon-7B (1.70), and landing close to Llama2-13B (1.67). Notably, without relying on any context-extension techniques, Gecko exhibits inherent long-context processing and
retrieval capabilities, stably handling sequences of up to 4 million
tokens and retrieving information from contexts up to $4\times$ longer than its
attention window. Code: https://github.com/XuezheMax/gecko-llm
Read the full abstract ▾
All authors Xuezhe Ma, Shicheng Wen, Linghao Jin, Bilge Acun, Ruihang Lai, Bohan Hou, Will Lin, Hao Zhang, Songlin Yang, Ryan Lee, Mengxi Wu, Jonathan May, Luke Zettlemoyer, Carole-Jean Wu
Machine Learning · Cross-listed from stat.ML
Physics-informed Gaussian Process Regression in Solving Eigenvalue Problem of Linear Operators
Tianming Bai and Jiannan Yang
Applying Physics-Informed Gaussian Process Regression to the
eigenvalue problem $(\mathcal{L}-\lambda)u = 0$ poses a fundamental challenge, where the null source term results in a trivial predictive mean and a degenerate marginal likelihood. Drawing inspiration from system identification, we construct a transfer function-type indicator for the unknown
eigenvalue/eigenfunction using the physics-informed Gaussian Process posterior. We demonstrate that the posterior covariance is only non-trivial when $\lambda$ corresponds to an
eigenvalue of the partial differential operator $\mathcal{L}$, reflecting the existence of a non-trivial eigenspace, and any sample from the posterior lies in the eigenspace of the linear operator. We demonstrate the effectiveness of the proposed approach through several numerical examples with both linear and non-linear
eigenvalue problems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
PixRec: Leveraging Visual Context for Next-Item Prediction in Sequential Recommendation
Sayak Chakrabarty and Souradip Pal
Large
Language Models (LLMs) have recently shown strong potential for usage in sequential recommendation tasks through text-only models, which combine advanced
prompt design, contrastive
alignment, and fine-tuning on downstream domain-specific data. While effective, these approaches overlook the rich visual information present in many real-world recommendation scenarios, particularly in e-commerce. This paper proposes PixRec - a
vision-
language framework that incorporates both textual attributes and product images into the recommendation pipeline. Our architecture leverages a
vision-
language model backbone capable of jointly processing image-text sequences, maintaining a dual-tower structure and mixed training objective while aligning multi-modal feature projections for both item-item and user-item interactions. Using the Amazon Reviews
dataset augmented with product images, our experiments demonstrate $3\times$ and 40% improvements in top-rank and top-10 rank accuracy over text-only recommenders respectively, indicating that visual features can help distinguish items with similar textual descriptions. Our work outlines future directions for scaling multi-modal recommenders training, enhancing visual-text feature fusion, and evaluating inference-time performance. This work takes a step toward building software systems utilizing visual information in sequential recommendation for real-world applications like e-commerce.
Read the full abstract ▾