Machine Learning · Cross-listed from cs.AI
AgentSM: Semantic Memory for Agentic Text-to-SQL
Asim Biswal, Chuan Lei, Xiao Qin et al. (+3 more)
Recent advances in LLM-based Text-to-SQL have achieved remarkable gains on public
benchmarks such as BIRD and Spider. Yet, these systems struggle to scale in realistic enterprise settings with large, complex schemas, diverse SQL dialects, and expensive multi-step
reasoning. Emerging agentic approaches show potential for adaptive
reasoning but often suffer from inefficiency and instability-repeating interactions with databases, producing inconsistent outputs, and occasionally failing to generate valid answers. To address these challenges, we introduce
Agent Semantic
Memory (AgentSM), an agentic framework for Text-to-SQL that builds and leverages interpretable semantic
memory. Instead of relying on raw scratchpads or vector
retrieval, AgentSM captures prior execution traces-or synthesizes curated ones-as structured programs that directly guide future
reasoning. This design enables systematic reuse of
reasoning paths, which allows
agents to scale to larger schemas, more complex questions, and longer trajectories efficiently and reliably. Compared to state-of-the-art systems, AgentSM achieves higher efficiency by reducing average
token usage and trajectory length by 25% and 35%, respectively, on the Spider 2.0
benchmark. It also improves execution accuracy, reaching a state-of-the-art accuracy of 44.8% on the Spider 2.0 Lite
benchmark.
Read the full abstract ▾
All authors Asim Biswal, Chuan Lei, Xiao Qin, Aodong Li, Balakrishnan Narayanaswamy, Tim Kraska
Machine Learning · Cross-listed from cs.CR
Balancing Security and Privacy: The Pivotal Role of AI in Modern Healthcare Systems
Binu V P, Deepthy K Bhaskar and Minimol B
As digital threats continue to grow, organizations must find ways to enhance security while protecting user
privacy. This paper explores how artificial intelligence (AI) plays a crucial role in achieving this balance. AI technologies can improve security by detecting threats, monitoring systems, and automating responses. However, using AI also raises
privacy concerns that need careful consideration.We examine real-world examples from the healthcare sector to illustrate how organizations can implement AI solutions that strengthen security without compromising patient
privacy. Additionally, we discuss the importance of creating transparent AI systems and adhering to
privacy regulations.Ultimately, this paper provides insights and recommendations for integrating AI into healthcare security practices, helping organizations navigate the challenges of modern management while keeping patient data safe.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Performance-guided Reinforced Active Learning for Object Detection
Zhixuan Liang, Xingyu Zeng, Rui Zhao et al. (+1 more)
Active learning (AL) strategies aim to train high-performance models with minimal labeling efforts, only selecting the most informative instances for annotation. Current approaches to evaluating data informativeness predominantly focus on the data's distribution or intrinsic information content and do not directly correlate with downstream task performance, such as mean average precision (mAP) in object detection. Thus, we propose Performance-guided (i.e. mAP-guided) Reinforced Active Learning for Object Detection (MGRAL), a novel approach that leverages the concept of expected model output changes as informativeness. To address the combinatorial explosion challenge of batch sample selection and the non-differentiable correlation between model performance and selected batches, MGRAL skillfully employs a reinforcement learning-based sampling
agent that optimizes selection using policy gradient with mAP improvement as reward. Moreover, to reduce the computational overhead of mAP estimation with unlabeled samples, MGRAL utilizes an unsupervised way with fast look-up tables, ensuring feasible deployment. We evaluate MGRAL's active learning performance on detection tasks over PASCAL VOC and COCO
benchmarks. Our approach demonstrates the highest AL curve with convincing visualizations, establishing a new paradigm in reinforcement learning-driven active object detection.
Read the full abstract ▾
All authors Zhixuan Liang, Xingyu Zeng, Rui Zhao, Ping Luo
Machine Learning
Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing
Xinyu Wang, Sicheng Lyu, Yu Gu et al. (+4 more)
Model editing updates a pre-trained LLM with new facts or rules without retraining while preserving unrelated behavior. In real deployment, edits arrive as long streams, creating a plasticity-stability dilemma: repeated locate-then-edit "hard writes" can accumulate interference over time, while rigid preservation constraints may protect only explicitly constrained directions, allowing past edits or unconstrained behaviors to deviate. We propose RLSEdit, a recursive least-squares editor for long sequential editing. RLSEdit formulates editing as an online quadratic optimization with soft constraints, minimizing a cumulative key-value fitting objective together with two regularizers that control deviation from the pre-trained weights and from a designated anchor mapping. This objective admits an efficient Woodbury-based online recursion, with per-edit cost independent of history length and scaling only with the current edit size. We further provide deviation bounds and an asymptotic characterization of the adherence-preservation trade-off in the many-edits regime. Experiments on CounterFact and ZsRE across multiple model families show stable scaling to 10K edits, outperforming strong baselines in both edit success and holistic stability, while retaining early edits and preserving general capabilities on GLUE and held-out
reasoning/code
benchmarks.
Read the full abstract ▾
All authors Xinyu Wang, Sicheng Lyu, Yu Gu, Jerry Huang, Peng Lu, Yufei Cui, Xiao-Wen Chang
Machine Learning · Cross-listed from cs.CR
Connect the Dots: Knowledge Graph-Guided Crawler Attack on Retrieval-Augmented Generation Systems
Mengyu Yao, Ziqi Zhang, Ning Luo et al. (+5 more)
Stealing attacks pose a persistent threat to the intellectual property of deployed machine-learning systems.
Retrieval-augmented generation (RAG) intensifies this risk by extending the attack surface beyond model weights to knowledge base that often contains IP-bearing assets such as proprietary runbooks, curated domain collections, or licensed documents. Recent work shows that multi-turn questioning can gradually steal corpus content from RAG systems, yet existing attacks are largely heuristic and often plateau early. We address this gap by formulating RAG knowledge-base stealing as an adaptive
stochastic coverage problem (ASCP), where each query is a
stochastic action and the goal is to maximize the conditional expected marginal gain (CMG) in corpus coverage under a query budget. Bridging ASCP to real-world black-box RAG knowledge-base stealing raises three challenges: CMG is unobservable, the natural-
language action space is intractably large, and feasibility constraints require stealthy queries that remain effective under diverse architectures. We introduce RAGCrawler, a knowledge graph-guided attacker that maintains a global attacker-side state to estimate coverage gains, schedule high-value semantic anchors, and generate non-redundant natural queries. Across four corpora and four generators with BGE retriever, RAGCrawler achieves 66.8% average coverage (up to 84.4%) within 1,000 queries, improving coverage by 44.90% relative to the strongest baseline. It also reduces the queries needed to reach 70% coverage by at least 4.03x on average and enables surrogate reconstruction with answer similarity up to 0.699. Our attack is also scalable to retriever switching and newer RAG techniques like query rewriting and multi-query
retrieval. These results highlight urgent needs to protect RAG knowledge assets.
Read the full abstract ▾
All authors Mengyu Yao, Ziqi Zhang, Ning Luo, Shaofei Li, Yifeng Cai, Xiangqun Chen, Yao Guo, Ding Li
Machine Learning · Cross-listed from cs.SD
Bridging the Perception Gap: A Lightweight Coarse-to-Fine Architecture for Edge Audio Systems
Hengfan Zhang, Yueqian Lin, Hai Helen Li et al. (+1 more)
Deploying Audio-
Language Models (Audio-LLMs) on edge infrastructure exposes a persistent tension between perception depth and computational efficiency. Lightweight local models tend to produce passive perception - generic summaries that miss the subtle evidence required for multi-step audio
reasoning - while indiscriminate cloud offloading incurs unacceptable latency, bandwidth cost, and
privacy risk. We propose CoFi-
Agent (Tool-Augmented Coarse-to-Fine
Agent), a hybrid architecture targeting edge servers and gateways. It performs fast local perception and triggers conditional forensic refinement only when uncertainty is detected. CoFi-
Agent runs an initial single-pass on a local 7B Audio-LLM, then a cloud controller gates difficult cases and issues lightweight plans for on-device tools such as temporal re-listening and local ASR. On the MMAR
benchmark, CoFi-
Agent improves accuracy from 27.20% to 53.60%, while achieving a better accuracy-efficiency trade-off than an always-on investigation pipeline. Overall, CoFi-
Agent bridges the perception gap via tool-enabled, conditional edge-cloud collaboration under practical system constraints.
Read the full abstract ▾
All authors Hengfan Zhang, Yueqian Lin, Hai Helen Li, Yiran Chen
Machine Learning
Dualformer: Time-Frequency Dual Domain Learning for Long-term Time Series Forecasting
Jingjing Bai and Yoshinobu Kawahara
Transformer-based models, despite their promise for long-term time series
forecasting (LTSF), suffer from an inherent low-pass filtering effect that limits their effectiveness. This issue arises due to undifferentiated propagation of frequency components across layers, causing a progressive attenuation of high-frequency information crucial for capturing fine-grained temporal variations. To address this limitation, we propose Dualformer, a principled dual-domain framework that rethinks frequency modeling from a layer-wise perspective. Dualformer introduces three key components: (1) a dual-branch architecture that concurrently models complementary temporal patterns in both time and frequency domains; (2) a hierarchical frequency sampling module that allocates distinct frequency bands to different layers, preserving high-frequency details in lower layers while modeling low-frequency trends in deeper layers; and (3) a periodicity-aware weighting mechanism that dynamically balances contributions from the dual branches based on the harmonic energy ratio of inputs, supported theoretically by a derived lower bound. This design enables structured frequency modeling and adaptive integration of time-frequency features, effectively preserving high-frequency information and enhancing generalization. Extensive experiments conducted on eight widely used
benchmarks demonstrate Dualformer's robustness and superior performance, particularly on heterogeneous or weakly periodic data. Our code is publicly available at https://github.com/Akira-221/Dualformer.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
TempoNet: Learning Realistic Communication and Timing Patterns for Network Traffic Simulation
Kristen Moore, Diksha Goel, Cody James Christopher et al. (+5 more)
Realistic network traffic simulation is critical for evaluating intrusion detection systems, stress-testing network protocols, and constructing high-fidelity environments for cybersecurity training. While attack traffic can often be layered into training environments using red-teaming or replay methods, generating authentic benign background traffic remains a core challenge – particularly in simulating the complex temporal and communication dynamics of real-world networks. This paper introduces TempoNet, a novel generative model that combines multi-task learning with multi-mark temporal point processes to jointly model inter-arrival times and all packet- and flow-header fields. TempoNet captures fine-grained timing patterns and higher-order correlations such as host-pair behavior and seasonal trends, addressing key limitations of GAN-, LLM-, and
Bayesian-based methods that fail to reproduce structured temporal variation. TempoNet produces temporally consistent, high-fidelity traces, validated on real-world
datasets. Furthermore, we show that intrusion detection models trained on TempoNet-generated background traffic perform comparably to those trained on real data, validating its utility for real-world security applications.
Read the full abstract ▾
All authors Kristen Moore, Diksha Goel, Cody James Christopher, Zhen Wang, Minjune Kim, Ahmed Ibrahim, Ahmad Mohsin, Seyit Camtepe
Machine Learning
Integrating Knowledge Distillation Methods: A Sequential Multi-Stage Framework
Yinxi Tian, Changwu Huang, Ke Tang et al. (+1 more)
Knowledge
distillation (KD) transfers knowledge from large teacher models to compact student models, enabling efficient deployment on resource constrained devices. While diverse KD methods, including response based, feature based, and relation based approaches, capture different aspects of teacher knowledge, integrating multiple methods or knowledge sources is promising but often hampered by complex implementation, inflexible combinations, and catastrophic forgetting, which limits practical effectiveness. This work proposes SMSKD (Sequential Multi Stage Knowledge
Distillation), a flexible framework that sequentially integrates heterogeneous KD methods. At each stage, the student is trained with a specific
distillation method, while a frozen reference model from the previous stage anchors learned knowledge to mitigate forgetting. In addition, we introduce an adaptive weighting mechanism based on the teacher true class probability (TCP) that dynamically adjusts the reference loss per sample to balance knowledge retention and integration. By design, SMSKD supports arbitrary method combinations and stage counts with negligible computational overhead. Extensive experiments show that SMSKD consistently improves student accuracy across diverse teacher student architectures and method combinations, outperforming existing baselines. Ablation studies confirm that stage wise
distillation and reference model supervision are primary contributors to performance gains, with TCP based adaptive weighting providing complementary benefits. Overall, SMSKD is a practical and resource efficient solution for integrating heterogeneous KD methods.
Read the full abstract ▾
All authors Yinxi Tian, Changwu Huang, Ke Tang, Xin Yao
Machine Learning · Cross-listed from quant-ph
Machine Failure Detection Based on Projected Quantum Models
Larry Bowden, Qi Chu, Bernard Cena et al. (+4 more)
Detecting machine failures promptly is of utmost importance in industry for maintaining efficiency and minimizing downtime. This paper introduces a failure detection
algorithm based on
quantum computing and a statistical change-point detection approach. Our method leverages the potential of projected
quantum feature maps to enhance the precision of anomaly detection in machine monitoring systems. We empirically validate our approach on
benchmark multi-dimensional time series
datasets as well as on a real-world
dataset comprising IoT sensor readings from operational machines, ensuring the practical relevance of our study. The
algorithm was executed on IBM's 133-
qubit Heron
quantum processor, demonstrating the feasibility of integrating
quantum computing into industrial maintenance procedures. The presented results underscore the effectiveness of our
quantum-based failure detection system, showcasing its capability to accurately identify anomalies in noisy time series data. This work not only highlights the potential of
quantum computing in industrial diagnostics but also paves the way for more sophisticated
quantum algorithms in the realm of predictive maintenance.
Read the full abstract ▾
All authors Larry Bowden, Qi Chu, Bernard Cena, Kentaro Ohno, Bob Parney, Deepak Sharma, Mitsuharu Takeori
Machine Learning
An Empirical Study on Ensemble-Based Transfer Learning Bayesian Optimisation with Mixed Variable Types
Natasha Trinkle, Huong Ha and Jeffrey Chan
Bayesian optimisation is a sample efficient method for finding a global optimum of expensive black-box objective functions. Historic
datasets from related problems can be exploited to help improve performance of
Bayesian optimisation by adapting transfer learning methods to various components of the
Bayesian optimisation pipeline. In this study we perform an empirical analysis of various ensemble-based transfer learning
Bayesian optimisation methods and pipeline components. We expand on previous work in the literature by contributing some specific pipeline components, and three new real-time transfer learning
Bayesian optimisation
benchmarks. In particular we propose to use a weighting strategy for ensemble surrogate model predictions based on regularised regression with weights constrained to be positive, and a related component for handling the case when transfer learning is not improving
Bayesian optimisation performance. We find that in general, two components that help improve transfer learning
Bayesian optimisation performance are warm start initialisation and constraining weights used with ensemble surrogate model to be positive.
Read the full abstract ▾
Machine Learning
Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors
Zhiwei Zhang, Fei Zhao, Rui Wang et al. (+6 more)
Large
language models (LLMs) can call tools effectively, yet they remain brittle in multi-turn execution: after a tool-call error, smaller models often fall into repetitive invalid re-invocations instead of interpreting the feedback and recovering. This failure mode persists because current training paradigms do not explicitly teach models how to recover from execution errors. In particular, standard reinforcement learning (RL) collapses rich failure experience into sparse negative rewards, while pre-collected error-correction
datasets become mismatched to the policy's evolving failure modes. To bridge this gap, we propose Fission-GRPO, a framework that converts execution errors into on-policy corrective supervision within the RL training loop. Our core mechanism fissions each failed trajectory into a new training instance by augmenting it with diagnostic feedback from a fine-tuned Error Simulator, then resampling multiple recovery rollouts on-policy. This enables the model to learn from the precise errors it makes during exploration, rather than from static, pre-collected error cases. On BFCL v4 Multi-Turn, Fission-GRPO improves the error recovery rate of Qwen3-8B by 5.7% absolute and overall accuracy by 4.0% (from 42.75% to 46.75%), outperforming both RL baselines and specialized tool-use
agents. The method further generalizes to TAU-Bench and TAU2-Bench, achieving leading results across most settings with gains up to +17.4%.
Read the full abstract ▾
All authors Zhiwei Zhang, Fei Zhao, Rui Wang, Zezhong Wang, Bin Liang, Jiakang Wang, Yao Hu, Shaosheng Cao, Kam-Fai Wong
Machine Learning
Closing the Gap on the Sample Complexity of 1-Identification
Zitian Li and Wang Chi Cheung
The 1-identification problem is a fundamental pure-exploration problem in multi-armed bandits. An
agent aims to determine whether there exists an arm whose mean reward exceeds a known threshold $\mu_0$, or to output None otherwise. The
agent must guarantee correctness with probability at least $1-\delta$, while minimizing the expected number of arm pulls $\mathbb{E}[\tau]$. We study the 1-identification problem and make two main contributions. First, for instances with at least one qualified arm, we derive a new lower bound on $\mathbb{E}[\tau]$ via a novel optimization formulation. Second, we propose a new
algorithm and establish upper bounds that match the lower bounds up to
polynomial logarithmic factors uniformly over all instances. Our result complements the analysis of $\mathbb{E}\tau$ when there are multiple qualified arms, which is an open problem in the literature.
Read the full abstract ▾
Machine Learning
When Sharpening Becomes Collapse: Sampling Bias and Semantic Coupling in RL with Verifiable Rewards
Mingyuan Fan, Weiguang Han, Daixin Wang et al. (+3 more)
Reinforcement Learning with Verifiable Rewards (RLVR) is a central paradigm for turning large
language models (LLMs) into reliable problem solvers, especially in logic-heavy domains. Despite its empirical success, it remains unclear whether RLVR elicits novel capabilities or merely sharpens the distribution over existing knowledge. We study this by formalizing over-sharpening, a phenomenon where the policy collapses onto limited modes, suppressing valid alternatives. At a high level, we discover finite-batch updates intrinsically bias learning toward sampled modes, triggering a collapse that propagates globally via semantic coupling. To mitigate this, we propose inverse-success advantage calibration to prioritize difficult queries and distribution-level calibration to diversify sampling via a
memory network. Empirical evaluations validate that our strategies can effectively improve generalization.
Read the full abstract ▾
All authors Mingyuan Fan, Weiguang Han, Daixin Wang, Cen Chen, Zhiqiang Zhang, Jun Zhou
Machine Learning
Neural Nonlinear Shrinkage of Covariance Matrices for Minimum Variance Portfolio Optimization
Liusha Yang, Siqi Zhao and Shuqi Chai
This paper introduces a neural network-based nonlinear shrinkage
estimator of covariance
matrices for the purpose of minimum variance portfolio optimization. It is a hybrid approach that integrates statistical estimation with machine learning. Starting from the Ledoit-Wolf (LW) shrinkage
estimator, we decompose the LW covariance
matrix into its
eigenvalues and eigenvectors, and apply a lightweight
transformer-based neural network to learn a nonlinear
eigenvalue shrinkage function. Trained with portfolio risk as the loss function, the resulting precision
matrix (the inverse covariance
matrix)
estimator directly targets portfolio risk minimization. By conditioning on the sample-to-dimension ratio, the approach remains scalable across different sample sizes and asset universes. Empirical results on stock daily returns from Standard & Poor's 500 Index (S&P500) demonstrate that the proposed method consistently achieves lower out-of-sample realized risk than
benchmark approaches. This highlights the promise of integrating structural statistical models with data-driven learning.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Data-Free Privacy-Preserving for LLMs via Model Inversion and Selective Unlearning
Xinjie Zhou, Zhihui Yang, Lechao Cheng et al. (+2 more)
Large
language models (LLMs) exhibit powerful capabilities but risk memorizing sensitive personally identifiable information (PII) from their training data, posing significant
privacy concerns. While machine unlearning techniques aim to remove such data, they predominantly depend on access to the training data. This requirement is often impractical, as training data in real-world deployments is commonly proprietary or inaccessible. To address this limitation, we propose Data-Free Selective Unlearning (DFSU), a novel
privacy-preserving framework that removes sensitive PII from an LLM without requiring its training data. Our approach first synthesizes pseudo-PII through
language model inversion, then constructs
token-level
privacy masks for these synthetic samples, and finally performs
token-level selective unlearning via a contrastive mask loss within a low-rank adaptation (LoRA) subspace. Extensive experiments on the AI4
Privacy PII-Masking
dataset using Pythia models demonstrate that our method effectively removes target PII while maintaining model utility.
Read the full abstract ▾
All authors Xinjie Zhou, Zhihui Yang, Lechao Cheng, Sai Wu, Gang Chen
Machine Learning · Cross-listed from cs.CL
Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow
Yangyang Zhong, Yanmei Gu, Zhengqing Zang et al. (+14 more)
Masked
Diffusion Language Models (MDLMs) promise parallel
token generation and arbitrary-order decoding, yet it remains unclear to what extent current models truly realize these capabilities. We characterize MDLM behavior along two dimensions – parallelism strength and generation order – using Average Finalization Parallelism (AFP) and Kendall's tau. We evaluate eight mainstream MDLMs (up to 100B parameters) on 58
benchmarks spanning knowledge,
reasoning, and programming. The results show that MDLMs still lag behind comparably sized autoregressive models, mainly because parallel probabilistic modeling weakens inter-
token dependencies. Meanwhile, MDLMs exhibit adaptive decoding behavior: their parallelism and generation order vary significantly with the task domain, the stage of
reasoning, and whether the output is correct. On tasks that require "backward information" (e.g., Sudoku), MDLMs adopt a solution order that tends to fill easier Sudoku blanks first, highlighting their advantages. Finally, we provide theoretical motivation and design insights supporting a Generate-then-Edit paradigm, which mitigates dependency loss while retaining the efficiency of parallel decoding.
Read the full abstract ▾
All authors Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao
Machine Learning
Deep Learning for Perishable Inventory Systems with Human Knowledge
Xuan Liao, Zhenkang Peng and Ying Rong
Managing perishable products with limited lifetimes is a fundamental challenge in inventory management, as poor ordering decisions can quickly lead to stockouts or excessive waste. We study a perishable inventory system with random lead times in which both the demand process and the lead time distribution are unknown. We consider a practical setting where orders are placed using limited historical data together with observed
covariates and current system states. To improve learning efficiency under limited data, we adopt a marginal cost accounting scheme that assigns each order a single lifetime cost and yields a unified loss function for end-to-end learning. This enables training a deep learning-based policy that maps observed
covariates and system states directly to order quantities. We develop two end-to-end variants: a purely black-box approach that outputs order quantities directly (E2E-BB), and a structure-guided approach that embeds the projected inventory level (PIL) policy, capturing inventory effects through explicit computation rather than additional learning (E2E-PIL). We further show that the objective induced by E2E-PIL is homogeneous of degree one, enabling a boosting technique from operational data analytics (ODA) that yields an enhanced policy (E2E-BPIL). Experiments on synthetic and real data establish a robust performance ordering: E2E-BB is dominated by E2E-PIL, which is further improved by E2E-BPIL. Using an excess-risk decomposition, we show that
embedding heuristic policy structure reduces effective model complexity and improves learning efficiency with only a modest loss of flexibility. More broadly, our results suggest that deep learning-based decision tools are more effective and robust when guided by human knowledge, highlighting the value of integrating advanced analytics with inventory theory.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.NI
MapViT: A Two-Stage ViT-Based Framework for Real-Time Radio Quality Map Prediction in Dynamic Environments
Cyril Shih-Huan Hsu, Xi Li, Lanfranco Zanzi et al. (+3 more)
Recent advancements in mobile and wireless networks are unlocking the full potential of robotic autonomy, enabling
robots to take advantage of ultra-low latency, high data throughput, and ubiquitous connectivity. However, for
robots to navigate and operate seamlessly, efficiently and reliably, they must have an accurate understanding of both their surrounding environment and the quality of
radio signals. Achieving this in highly dynamic and ever-changing environments remains a challenging and largely unsolved problem. In this paper, we introduce MapViT, a two-stage
Vision Transformer (ViT)-based framework inspired by the success of pre-train and fine-tune paradigm for Large
Language Models (LLMs). MapViT is designed to predict both environmental changes and expected
radio signal quality. We evaluate the framework using a set of representative Machine Learning (ML) models, analyzing their respective strengths and limitations across different scenarios. Experimental results demonstrate that the proposed two-stage pipeline enables real-time prediction, with the ViT-based implementation achieving a strong balance between accuracy and computational efficiency. This makes MapViT a promising solution for energy- and resource-constrained platforms such as mobile
robots. Moreover, the geometry foundation model derived from the self-supervised pre-training stage improves data efficiency and transferability, enabling effective downstream predictions even with limited labeled data. Overall, this work lays the foundation for next-generation digital twin ecosystems, and it paves the way for a new class of ML foundation models driving multi-modal intelligence in future 6G-enabled systems.
Read the full abstract ▾
All authors Cyril Shih-Huan Hsu, Xi Li, Lanfranco Zanzi, Zhiheng Yang, Chrysa Papagianni, Xavier Costa Pérez
Machine Learning · Cross-listed from cs.ET
Enhanced Convergence in p-bit Based Simulated Annealing with Partial Deactivation for Large-Scale Combinatorial Optimization Problems
Naoya Onizawa and Takahiro Hanyu
This article critically investigates the limitations of the simulated annealing
algorithm using probabilistic bits (pSA) in solving large-scale combinatorial optimization problems. The study begins with an in-depth analysis of the pSA process, focusing on the issues resulting from unexpected oscillations among p-bits. These oscillations hinder the energy reduction of the Ising model and thus obstruct the successful execution of pSA in complex tasks. Through detailed simulations, we unravel the root cause of this energy stagnation, identifying the feedback mechanism inherent to the pSA operation as the primary contributor to these disruptive oscillations. To address this challenge, we propose two novel
algorithms, time average pSA (TApSA) and stalled pSA (SpSA). These
algorithms are designed based on partial deactivation of p-bits and are thoroughly tested using Python simulations on maximum cut
benchmarks that are typical combinatorial optimization problems. On the 16
benchmarks from 800 to 5,000 nodes, the proposed methods improve the normalized cut value from 0.8% to 98.4% on average in comparison with the conventional pSA.
Read the full abstract ▾
Machine Learning
BanditLP: Large-Scale Stochastic Optimization for Personalized Recommendations
Phuc Nguyen, Benjamin Zelditch, Joyce Chen et al. (+2 more)
We present BanditLP, a scalable multi-stakeholder contextual bandit framework that unifies neural Thompson Sampling for learning objective-specific outcomes with a large-scale linear program for constrained action selection at serving time. The methodology is application-agnostic, compatible with arbitrary neural architectures, and deployable at web scale, with an LP solver capable of handling billions of variables. Experiments on public
benchmarks and synthetic data show consistent gains over strong baselines. We apply this approach in LinkedIn's email marketing system and demonstrate business win, illustrating the value of integrated exploration and constrained optimization in production.
Read the full abstract ▾
All authors Phuc Nguyen, Benjamin Zelditch, Joyce Chen, Rohit Patra, Changshuai Wei
Machine Learning
Learning Neural Operators from Partial Observations via Latent Autoregressive Modeling
Jingren Hou, Hong Wang, Pengyu Xu et al. (+3 more)
Real-world scientific applications frequently encounter incomplete observational data due to sensor limitations, geographic constraints, or measurement costs. Although neural operators significantly advanced PDE solving in terms of computational efficiency and accuracy, their underlying assumption of fully-observed spatial inputs severely restricts applicability in real-world applications. We introduce the first systematic framework for learning neural operators from partial observation. We identify and formalize two fundamental obstacles: (i) the supervision gap in unobserved regions that prevents effective learning of physical correlations, and (ii) the dynamic spatial mismatch between incomplete inputs and complete solution fields. Specifically, our proposed Latent Autoregressive Neural Operator(LANO) introduces two novel components designed explicitly to address the core difficulties of partial observations: (i) a mask-to-predict training strategy that creates artificial supervision by strategically masking observed regions, and (ii) a Physics-Aware Latent Propagator that reconstructs solutions through boundary-first autoregressive generation in latent space. Additionally, we develop POBench-PDE, a dedicated and comprehensive
benchmark designed specifically for evaluating neural operators under partial observation conditions across three PDE-governed tasks. LANO achieves state-of-the-art performance with 18–69$\%$ relative L2 error reduction across all
benchmarks under patch-wise missingness with less than 50$\%$ missing rate, including real-world
climate prediction. Our approach effectively addresses practical scenarios involving up to 75$\%$ missing rate, to some extent bridging the existing gap between idealized research settings and the complexities of real-world scientific computing.
Read the full abstract ▾
All authors Jingren Hou, Hong Wang, Pengyu Xu, Chang Gao, Huafeng Liu, Liping Jing
Machine Learning
Beyond validation loss: Clinically-tailored optimization metrics improve a model's clinical performance
Charles B. Delahunt, Courosh Mehanian, Daniel E. Shea et al. (+1 more)
A key task in ML is to optimize models at various stages, e.g. by choosing hyperparameters or picking a stopping point. A traditional ML approach is to use validation loss, i.e. to apply the training loss function on a validation set to guide these optimizations. However, ML for healthcare has a distinct goal from traditional ML: Models must perform well relative to specific clinical requirements, vs. relative to the loss function used for training. These clinical requirements can be captured more precisely by tailored metrics. Since many optimization tasks do not require the driving metric to be differentiable, they allow a wider range of options, including the use of metrics tailored to be clinically-relevant. In this paper we describe two controlled experiments which show how the use of clinically-tailored metrics provide superior model optimization compared to validation loss, in the sense of better performance on the clinical task. The use of clinically-relevant metrics for optimization entails some extra effort, to define the metrics and to code them into the pipeline. But it can yield models that better meet the central goal of ML for healthcare: strong performance in the clinic.
Read the full abstract ▾
All authors Charles B. Delahunt, Courosh Mehanian, Daniel E. Shea, Matthew P. Horning
Machine Learning
RDumb++: Drift-Aware Continual Test-Time Adaptation
Himanshu Mishra
Continual Test-Time Adaptation (CTTA) seeks to update a pretrained model during deployment using only the incoming, unlabeled data stream. Although prior approaches such as Tent, EATA etc. provide meaningful improvements under short evolving shifts, they struggle when the test distribution changes rapidly or over extremely long horizons. This challenge is exemplified by the CCC
benchmark, where models operate over streams of 7.5M samples with continually changing corruption types and severities. We propose RDumb++, a principled extension of RDumb that introduces two drift-detection mechanisms i.e
entropy-based drift scoring and KL-divergence drift scoring, together with adaptive reset strategies. These mechanisms allow the model to detect when accumulated adaptation becomes harmful and to recover before prediction collapse occurs. Across CCC-medium with three speeds and three seeds (nine runs, each containing one million samples), RDumb++ consistently surpasses RDumb, yielding approx 3% absolute accuracy gains while maintaining stable adaptation throughout the entire stream. Ablation experiments on drift thresholds and reset strengths further show that drift-aware resetting is essential for preventing collapse and achieving reliable long-horizon CTTA.
Read the full abstract ▾