Machine Learning · Cross-listed from cs.NE
Meta-Learning-Assisted Constraint Relaxation for Constrained Black-Box Optimization
Sijie Ma, Zeyuan Ma, Yue-Jiao Gong et al. (+1 more)
Constraint handling is central to constrained black-box optimization (BBO), where objective improvement and feasibility restoration often provide conflicting search signals. Existing $\epsilon$-relaxation methods are simple and effective, but their relaxation schedules are usually fixed or manually designed for a limited range of problems. To address this limitation, this letter proposes MeCO, a meta-learning-assisted optimizer that learns an adaptive $\epsilon$-relaxation policy for constrained BBO. MeCO couples a SHADE optimizer with a Double Deep Q-Network controller. At each optimization step, the controller observes compact population and constraint features and selects a scalar action, which is decoded into a relaxation vector for the candidate comparison rule. The policy is trained across constrained BBO instances and then deployed on held-out problems without problem-specific tuning. Experiments on the CEC2017 constrained
benchmark, 16
UAV path-planning tasks and eight real-world engineering problems provide evidence that MeCO transfers across held-out
benchmark functions, higher dimensions, and an application-domain setting. Ablation and behavior analyses further clarify the roles of constraint-related state features, action scaling, reward shaping, and meta-training.
Read the full abstract ▾
All authors Sijie Ma, Zeyuan Ma, Yue-Jiao Gong, Ran Cheng
Machine Learning
Physiology as Language: Translating Respiration to Sleep EEG
Kaiwen Zha, Chao Li, Hao He et al. (+7 more)
This paper introduces a novel cross-physiology translation task: synthesizing sleep electroencephalography (
EEG) from respiration signals. To address the significant complexity gap between the two modalities, we propose a waveform-conditional generative framework that preserves fine-grained respiratory dynamics while constraining the
EEG target space through discrete tokenization. Trained on over 28,000 individuals, our model achieves a 7% Mean Absolute Error in
EEG spectrogram reconstruction. Beyond reconstruction, the synthesized
EEG supports downstream tasks with performance comparable to ground truth
EEG on age estimation (MAE 5.0 vs. 5.1 years), sex detection (AUROC 0.81 vs. 0.82), and sleep staging (Accuracy 0.84 vs. 0.88), significantly outperforming baselines trained directly on breathing. Finally, we demonstrate that the framework generalizes to contactless sensing by synthesizing
EEG from wireless
radio-frequency reflections, highlighting the feasibility of remote, non-contact neurological assessment during sleep.
Read the full abstract ▾
All authors Kaiwen Zha, Chao Li, Hao He, Peng Cao, Tianhong Li, Ali Mirzazadeh, Ellen Zhang, Jong Woo Lee, Yoon Kim, Dina Katabi
Machine Learning
NEST: Nested Event Stream Transformer for Sequences of Multisets
Minghui Sun, Haoyu Gong, Xingyu You et al. (+3 more)
Event stream data often exhibit hierarchical structure in which multiple events co-occur, resulting in a sequence of multisets (i.e., bags of events). In electronic health records (EHRs), for example, medical events are grouped into a sequence of clinical encounters with well-defined temporal structure, but the order and timing of events within each encounter may be unknown or unreliable. Most existing foundation models (FMs) for event stream data flatten this hierarchy into a one-dimensional sequence, leading to (i) computational inefficiency associated with dense
attention and learning spurious within-set relationships, and (ii) lower-quality set-level representations from heuristic post-training pooling for downstream tasks. Here, we show that preserving the original hierarchy in the FM architecture provides a useful inductive bias that improves both computational efficiency and representation quality. We then introduce Nested Event Stream
Transformer (NEST), a FM for event streams comprised of sequences of multisets. Building on this architecture, we formulate Masked Set Modeling (MSM), an efficient paradigm that promotes improved set-level representation learning. Experiments on real-world multiset sequence data show that NEST captures real-world dynamics while improving both pretraining efficiency and downstream performance.
Read the full abstract ▾
All authors Minghui Sun, Haoyu Gong, Xingyu You, Jillian Hurst, Benjamin Goldstein, Matthew Engelhard
Machine Learning
Contrastive Learning for Privacy Enhancements in Industrial Internet of Things
Lin Liu, Rita Machacy and Simi Kuniyilh
The Industrial Internet of Things (IIoT) integrates intelligent sensing, communication, and analytics into industrial environments, including manufacturing, energy, and critical infrastructure. While IIoT enables predictive maintenance and cross-site optimization of modern industrial control systems, such as those in manufacturing and energy, it also introduces significant
privacy and confidentiality risks due to the sensitivity of operational data. Contrastive learning, a self-supervised representation learning paradigm, has recently emerged as a promising approach for
privacy-preserving analytics by reducing reliance on labeled data and raw data sharing. Although contrastive learning-based
privacy-preserving techniques have been explored in the Internet of Things (IoT) domain, this paper offers a comprehensive review of these techniques specifically for
privacy preservation in Industrial Internet of Things (IIoT) systems. It emphasizes the unique characteristics of industrial data, system architectures, and various application scenarios. Additionally, the paper discusses solutions and open challenges and outlines future research directions.
Read the full abstract ▾
Machine Learning
Minerva: Reinforcement Learning with Verifiable Rewards for Cyber Threat Intelligence LLMs
Md Tanvirul Alam, Aritran Piplai, Ionut Cardei et al. (+2 more)
Cyber threat intelligence (CTI) analysts routinely convert noisy, unstructured security artifacts into standardized, automation-ready representations. Although large
language models (LLMs) show promise for this task, existing approaches remain brittle when producing structured CTI outputs and have largely relied on supervised fine-tuning (SFT). In contrast, CTI standards and community-maintained resources define canonical identifiers and schemas that enable deterministic verification of model outputs. We leverage this structure to study reinforcement learning with verifiable rewards (RLVR) for CTI tasks. We introduce Minerva, a unified
dataset and training pipeline spanning multiple CTI subtasks, each paired with task-specific verifiers that score structured outputs and identifier predictions. To address reward sparsity during rollout, we propose MinervaRL, an auxiliary self-training mechanism that generates additional verified trajectories and distills them back into the model. Averaged across four backbones and 12 CTI
benchmarks, MinervaRL improves the mean score by 15.8 percentage points over the corresponding base models and by 4.3 points over GRPO. The project page is available at https://maveryn.github.io/minerva-tmlr/.
Read the full abstract ▾
All authors Md Tanvirul Alam, Aritran Piplai, Ionut Cardei, Nidhi Rastogi, Peter J Worth Jr
Machine Learning
Partition of Unity Neural Networks for Interpretable Classification with Explicit Class Regions
Akram Aldroubi
We introduce Partition of Unity Neural Networks (PUNNs), a neural-network architecture for multiclass classification based on the classical mathematical notion of a partition of unity. The starting point is the observation that the characteristic functions of ideal class regions form a partition of unity. PUNNs replace these discontinuous indicators by learned continuous functions \[ h_1,\ldots,h_C:\mathcal X\to[0,1] \] whose sum is identically one and whose values are interpreted directly as class probabilities. The partition functions are generated through a recursive family of input-dependent gates. This construction guarantees nonnegative class probabilities summing to one without a separate normalization layer such as softmax, while providing an explicit ordered factorization of each probability in terms of the gate values. The resulting gate trace gives an interpretable representation of how individual class probabilities are formed. The framework also allows multiple partition functions to represent a single class and admits both neural-network and geometry-informed realizations of the gates. We prove that PUNNs are dense in the space of continuous probability maps from compact subsets of $\mathbb R^d$ into the probability simplex. Thus, the recursive partition-of-unity structure retains universal approximation of continuous probabilistic classifiers, including maps whose components may vanish. Numerical experiments on synthetic
datasets, MNIST, and CIFAR-100 illustrate the learned partitions, the effect of class ordering, and the use of multiple partition components per class. We also develop shape-informed gates that incorporate
geometric information directly; when the chosen geometry is well matched to the class regions, these models achieve comparable accuracy with substantially fewer trainable parameters.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
PCBSchemaGen: Reward-Guided LLM Code Synthesis for Printed Circuit Boards (PCB) Schematic Design with Structured Verification
Huanghaohe Zou, Peng Han, Emad Nazerian et al. (+3 more)
Most LLM code-synthesis
benchmarks rely on unit tests as the reward oracle, but PCB schematic design has none: correctness is defined by structured physical constraints over real IC packages and pin-level assignments, per-task golden references are unavailable, and SPICE simulation does not validate schematic-level correctness. We introduce PCBSchemaGen, a training-free inference-time framework that turns a frozen LLM into a verifiable, repairable PCB schematic generator. The framework induces a domain schema from IC datasheets to ground LLM decoding, pairs it with a deterministic 5-layer continuous-reward verifier with pin-level error localization, and refines candidates through a Thompson Sampling arm-acquiring bandit. We evaluate on 2 PCB
benchmarks covering 227 real-IC tasks across 22 unified
circuit domains, including a public-schematic-derived suite that serves as a fully held-out generalization test (verifier, KG library, and
prompts frozen before any evaluation). Under our framework, an open-weight 31B model (Gemma-4-31B) passes 81.3% of PCBBench tasks on average, and the same framework transfers across both
benchmarks with zero verifier code changes; a Circuitron-style inference-time prompting baseline on the same Gemma-4-31B backbone collapses on hard system-level designs. This suggests inference-time refinement under a deterministic structural verifier is a general recipe for reference-free LLM code synthesis in domains without unit-test oracles. Our
benchmarks and deterministic verifier are publicly available at https://github.com/HZou9/PCBSchemaGen_v2.
Read the full abstract ▾
All authors Huanghaohe Zou, Peng Han, Emad Nazerian, Mafu Zhang, Zhicheng Guo, Alex Q. Huang
Machine Learning
OD-Gear: Online Decomposition and Group Sampling for Expert-Guided Adversarial Routing in Scalable Capacitated Vehicle Routing
Dongbin Jiao, Zisheng Chen, Xianyi Wang et al. (+3 more)
Solving large-scale capacitated vehicle routing problems (CVRP) is hindered by the high complexity of classical heuristics and the limited generalization of neural solvers. To bridge this gap, we propose OD-Gear, an expert-guided
adversarial framework that integrates hybrid genetic search (HGS) and online barycenter clustering (BCC) decomposition with group-relative optimization. OD-Gear internalizes expert heuristics into a graph
attention network (GAT)-based policy via high-fidelity knowledge
distillation. Our minimax
adversarial training distills divide-and-conquer strategies into dense surrogate rewards, while a group-sampling strategy exploits relative solution advantages to promote both diversity and quality. This architecture enables high-quality, clustering-free inference on massive graphs, effectively bypassing the overhead of traditional decomposition. Empirical results demonstrate that OD-Gear achieves state-of-the-art (SOTA) performance across most
benchmarks, remaining highly competitive at the 10,000-node scale. By providing heuristic-quality solutions with low-latency, OD-Gear offers a robust and scalable framework for large-scale CVRP.
Read the full abstract ▾
All authors Dongbin Jiao, Zisheng Chen, Xianyi Wang, Jintao Shi, Shengcai Liu, Shi Yan
Machine Learning
AREAL-DTA: Dynamic Tree Attention for Efficient Reinforcement Learning of Large Language Models
Jiarui Zhang, Yuchen Yang, Ran Yan et al. (+8 more)
Reinforcement learning (RL)-based post-training for large
language models (LLMs) is computationally expensive, as it generates many rollout sequences that frequently share long
token prefixes. Existing RL frameworks usually process these sequences independently during policy training, i.e., repeatedly recomputing identical prefixes in both the forward and backward passes of policy gradient computation, leading to substantial inefficiencies in computation resources and
memory usage. Although prefix sharing naturally induces a tree structure over rollouts, packed tree-mask approaches scale poorly in RL settings. In this paper, we introduce AReaL-DTA, which efficiently exploits prefix sharing in RL training. AReaL-DTA employs a depth-first search (DFS)-based execution strategy that dynamically traverses the rollout prefix tree during both forward and backward computation, materializing only a single root-to-leaf path at a time. To further improve scalability, AReaL-DTA incorporates a load-balanced distributed batching mechanism that dynamically constructs and processes prefix trees across multiple GPUs. On $\tau^2$-bench, AReaL-DTA improves training throughput by up to $8.31\times$ over dense training and up to $1.70\times$ over sparse training. Our code is available at https://github.com/areal-project/AReaL/tree/feat/dta.
Read the full abstract ▾
All authors Jiarui Zhang, Yuchen Yang, Ran Yan, Zhiyu Mei, Liyuan Zhang, Daifeng Li, Wei Fu, Jiaxuan Gao, Shusheng Xu, Yi Wu, Binhang Yuan
Machine Learning
Quality-Diversity Optimization as Multi-Objective Optimization
Xi Lin, Ping Guo, Yilu Liu et al. (+2 more)
The Quality-Diversity (QD) optimization aims to discover a collection of high-performing solutions that simultaneously exhibit diverse behaviors within a user-defined behavior space. This paradigm has stimulated significant research interest and demonstrated practical utility in domains including
robot control, creative design, and
adversarial sample generation. A variety of QD
algorithms with distinct design principles have been proposed in recent years. Instead of proposing a new QD
algorithm, this work introduces a novel reformulation by casting the QD optimization as a multi-objective optimization (MOO) problem with a huge number of optimization objectives. By establishing this connection, we enable the direct adoption of well-established MOO methods, particularly set-based scalarization techniques, to solve QD problems through a collaborative search process. We further provide a theoretical analysis demonstrating that our approach inherits theoretical guarantees from MOO while providing desirable properties for the QD optimization. Experimental studies across several QD applications confirm that our method achieves performance competitive with state-of-the-art QD
algorithms.
Read the full abstract ▾
All authors Xi Lin, Ping Guo, Yilu Liu, Qingfu Zhang, Jianyong Sun
Machine Learning
Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitly
Hengchang Liu, Zhao Yang and Bing Su
Diffusion language models (DLMs) provide a bidirectional generation framework naturally suited for infilling, yet their performance is constrained by the pre-specified infilling length. In this paper, we reveal that DLMs possess an inherent ability to discover the correct infilling length. We identify two key statistical phenomena in the first-step denoising confidence: a local Oracle Peak that emerges near the ground-truth length and a systematic Length Bias that often obscures this signal. By leveraging this signal and calibrating the bias, our training-free method CAL (Calibrated Adaptive Length) enables DLMs to approximate the optimal length through an efficient search before formal decoding. Empirical evaluations demonstrate that CAL improves Pass@1 by up to 47.7% over fixed-length baselines and 40.5% over chat-based adaptive methods in code infilling, while boosting BLEU-2 and ROUGE-L by up to 8.5% and 9.9% in text infilling. These results demonstrate that CAL paves the way for robust DLM infilling without requiring any specialized training. Code is available at https://github.com/NiuHechang/Calibrated_Adaptive_Length.
Read the full abstract ▾
Machine Learning
Parallel Stochastic Gradient-Based Planning for World Models
Michael Psenka, Michael Rabbat, Aditi Krishnapriyan et al. (+2 more)
World models simulate environment dynamics from raw sensory inputs like
video. However, using them for planning can be challenging due to the vast and unstructured search space. We propose a robust and highly parallelizable planner that leverages the differentiability of the learned world model for efficient optimization, solving long-horizon control tasks from visual input. Our method treats states as optimization variables ("virtual states") with soft dynamics constraints, enabling parallel computation and easier optimization. To facilitate exploration and avoid local optima, we introduce stochasticity into the states. To mitigate sensitive gradients through high-dimensional
vision-based world models, we modify the gradient structure to descend towards valid plans while only requiring action-input gradients. Our planner, which we call GRASP (Gradient RelAxed
Stochastic Planner), can be viewed as a
stochastic version of a non-condensed or collocation-based optimal controller. We provide theoretical justification and experiments on
video-based world models, where our resulting planner outperforms existing planning
algorithms like the cross-
entropy method (CEM) and vanilla gradient-based optimization (GD) on long-horizon experiments, both in success rate and time to convergence.
Read the full abstract ▾
All authors Michael Psenka, Michael Rabbat, Aditi Krishnapriyan, Yann LeCun, Amir Bar
Machine Learning · Cross-listed from stat.ML
Persistent-Transient Policy Evaluation for Markov Chains via Minimal Peripheral Quotients
Yang Xu and Vaneet Aggarwal
We study fixed-policy evaluation for finite Markov chains that may be reducible and periodic. Classical evaluation methods with gain and bias decomposition are not always diagnostic: the gain records only invariant Cesàro averages, while persistent phase-dependent behavior is absorbed into the bias together with genuinely transient effects. We identify the real peripheral invariant subspace $\mathcal{K}(P)$ of the transition
matrix $P$ as the source of this ambiguity. Quotienting by $\mathcal{K}(P)$ is the minimal exact quotient that removes all non-decaying modes and makes the remaining dynamics strictly stable. After choosing a gauge projection $\Pi$ with kernel $\mathcal{K}(P)$, the reward admits a unique decomposition $r = g_\Pi^\star + (I-P)v_\Pi^\star$, where $g_\Pi^\star$ is a persistent regime profile and $v_\Pi^\star$ is a gauge-fixed transient component. An exact comparison with classical normalized gain and bias shows that the new pair reallocates the same information into a persistent profile and a gauge-fixed representative of the transient quotient. This decomposition reconstructs finite-horizon returns, recovers statewise average reward, admits a transient-cost interpretation, and yields a stable
estimator under a generative model.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Jin-Long Chen, Xin Li and Zhang-Qi Yin
Quantum phase transitions in many-body systems are fundamentally characterized by complex correlation structures, which pose computational challenges for conventional methods in large systems. To address this, we propose a hybrid
quantum-classical
attention model. This model uses an
attention mechanism, realized through swap tests and a parameterized
quantum circuit, to extract correlations within
quantum states and perform ground-state classification. Benchmarked on the cluster-Ising model with system sizes of 9 and 15
qubits, the model achieves high classification accuracy with less than 100 training data and demonstrates robustness against variations in the training set. Further analysis reveals that the model successfully captures phase-sensitive features and characteristic physical length scales, offering a scalable and data-efficient approach for
quantum phase recognition in complex many-body systems.
Read the full abstract ▾
Machine Learning
PAIR-Former: Budgeted Relational Multi-Instance Learning for Functional miRNA Target Prediction
Jiaqi Yin, Baiming Chen, Jia Fei et al. (+1 more)
Functional miRNA–mRNA targeting is a large-bag prediction problem where each transcript yields a heavy-tailed pool of candidate target sites (CTSs), yet only a pair-level label is observed. Prior methods use max-pooling over individual CTS scores, ignoring relational patterns among sites, but modeling these patterns is critical for accuracy. The challenge is that naive relational aggregation incurs $\mathcal{O}(n^2)$ cost, prohibitive when $n$ reaches thousands, yet a cheap scan alone discards the very interactions that drive functional repression. We formalize this tension as Budgeted Relational Multi-Instance Learning (BR-MIL), a new MIL problem where the compute budget $K$ is a first-class constraint such that at most $K$ instances per bag may receive expensive encoding and relational processing. We establish theoretical foundations for BR-MIL, proving that both approximation quality and generalization are governed by $K$ rather than the raw bag size $n$. Building on this theory, we propose PAIR-Former, which scans all candidates cheaply, selects $K$ diverse CTSs, and aggregates them via Set
Transformer. PAIR-Former achieves state-of-the-art performance, outperforming all reproduced baselines with F1$=0.840$ on miRAW (10-fold balanced CV) and $0.839$ on deepTargetPro in transfer evaluation, while achieving $0.793$ on the large-scale MTI
benchmark (420K pairs, $38\times$ larger), demonstrating that budgeted relational MIL scales where naive approaches fail. Additional results on CAMELYON16 and Musk2 further show that the proposed BR-MIL formulation extends beyond biological sequence modeling.
Read the full abstract ▾
All authors Jiaqi Yin, Baiming Chen, Jia Fei, Mingjun Yang
Machine Learning
Search Inspired Exploration in Reinforcement Learning
Georgios Sotirchos, Zlatan Ajanović and Jens Kober
Exploration in environments with sparse rewards remains a fundamental challenge in reinforcement learning (RL). Existing approaches such as curriculum learning and Go-Explore often rely on hand-crafted heuristics, while curiosity-driven methods risk converging to suboptimal policies. We propose Search-Inspired Exploration in Reinforcement Learning (SIERL), a novel method that actively guides exploration by setting sub-goals based on the
agent's learning progress. At the beginning of each episode, SIERL chooses a sub-goal from the frontier (the boundary of the
agent's known state space), before the
agent continues exploring toward the main task objective. The key contribution of our method is the sub-goal selection mechanism, which provides state-action pairs that are neither overly familiar nor completely novel. Thus, it assures that the frontier is expanded systematically and that the
agent is capable of reaching any state within it. Inspired by search, sub-goals are prioritized from the frontier based on estimates of cost-to-come and cost-to-go, effectively steering exploration towards the most informative regions. In experiments on challenging sparse-reward environments, SIERL outperforms dominant baselines in both achieving the main task goal and generalizing to reach arbitrary states in the environment.
Read the full abstract ▾
Machine Learning
LatentTrack: Sequential Weight Generation via Latent Filtering
Omer Haq
We introduce LatentTrack (LT), a sequential neural architecture for online probabilistic prediction under nonstationary dynamics. LT performs causal
Bayesian filtering in a low-dimensional latent space and uses a lightweight hypernetwork to generate predictive model parameters at each time step, enabling constant-time online adaptation without per-step gradient updates. At each time step, a learned latent model predicts the next latent distribution, which is updated via amortized inference using new observations, yielding a predict–generate–update filtering framework in function space. The formulation supports both structured (Markovian) and unstructured latent dynamics within a unified objective, while Monte Carlo inference over latent trajectories produces calibrated predictive mixtures with fixed per-step cost. Evaluated on long-horizon online regression using the Jena
Climate benchmark, LT consistently achieves lower negative log-likelihood and mean squared error than stateful sequential and static uncertainty-aware baselines, with competitive calibration, demonstrating that latent-conditioned function
evolution is an effective alternative to traditional latent-state modeling under distribution shift.
Read the full abstract ▾
Machine Learning
FedMOA: Federated GRPO for Personalized Reasoning LLMs under Heterogeneous Rewards
Ziyao Wang, Daeun Jung, Yexiao He et al. (+4 more)
Group Relative Policy Optimization (GRPO) has recently emerged as an effective approach for improving the
reasoning capabilities of large
language models through online multi-objective reinforcement learning. While personalization on private data is increasingly vital, traditional Reinforcement Learning (RL)
alignment is often
memory-prohibitive for on-device
federated learning due to the overhead of maintaining a separate critic network. GRPO's critic-free architecture enables feasible on-device training, yet transitioning to a
federated setting introduces systemic challenges: heterogeneous reward definitions, imbalanced multi-objective optimization, and high training costs. We propose FedMOA, a
federated GRPO framework for multi-objective
alignment under heterogeneous rewards. FedMOA stabilizes local training through an online adaptive weighting mechanism via hypergradient descent, which prioritizes primary
reasoning as auxiliary objectives saturate. On the server side, it utilizes a task- and accuracy-aware aggregation strategy to prioritize high-quality updates. Experiments on mathematical
reasoning and code generation
benchmarks demonstrate that FedMOA consistently outperforms
federated averaging, achieving accuracy gains of up to 2.2% while improving global performance, personalization, and multi-objective balance.
Read the full abstract ▾
All authors Ziyao Wang, Daeun Jung, Yexiao He, Guoheng Sun, Zheyu Shen, Myungjin Lee, Ang Li
Machine Learning
Stabilizing Decentralized Federated Fine-Tuning via Topology-Aware Alternating LoRA
Xiaoyu Wang, Xiaotian Li, Zhixiang Zhou et al. (+2 more)
Decentralized
federated learning (DFL), a serverless variant of
federated learning, poses unique challenges for parameter-efficient fine-tuning due to the factorized structure of low-rank adaptation (LoRA). Unlike linear parameters, decentralized aggregation of LoRA updates introduces
topology-dependent cross terms that can destabilize training under dynamic communication graphs. We propose TAD-LoRA, a
Topology-Aware Decentralized Low-Rank Adaptation framework that coordinates the updates and mixing of LoRA factors to control inter-client misalignment. We theoretically prove the convergence of TAD-LoRA under non-convex objectives, explicitly characterizing the trade-off between
topology-induced cross-term error and block-coordinate representation bias governed by the switching interval of alternative training. Experiments under various communication conditions validate our analysis, showing that TAD-LoRA achieves robust performance across different communication scenarios, remaining competitive in strongly connected
topologies and delivering clear gains under moderately and weakly connected
topologies, with particularly strong results on the MNLI
dataset.
Read the full abstract ▾
All authors Xiaoyu Wang, Xiaotian Li, Zhixiang Zhou, Chen Li, Yong Liu
Machine Learning · Cross-listed from cs.AI
Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Tasks
Jia Liang and Liangming Pan
Latent Chain-of-Thought aims to enable step-by-step computation without emitting long rationales, yet its internal mechanisms remain unclear. We study CODI, a continuous-thought teacher-student
distillation model, on strictly sequential
polynomial-iteration tasks with known intermediate states. Using logit-lens decoding, linear probes,
attention analysis, and activation patching, we localize intermediate-state representations and trace how they are routed to the final readout. In short-horizon, low-hop tasks, CODI forms faithful bridge states across latent-thought positions, while the final input follows a separate near-direct route; predictions arise through late fusion at the answer readout. As task depth and difficulty increase, however, CODI does not reliably sustain a full latent rollout: it either compresses computation into a partial late-intermediate pathway or, in harder regimes, loses the latent
reasoning signature altogether. To explain this transition, we show theoretically that the task's algebraic structure controls its effective
memory: compressible regimes support late-bottleneck
reasoning, while incompressible regimes preserve full-history dependence and destabilize latent rollouts. Overall, our results characterize when CODI-style latent-CoT yields faithful iterative computation versus compressed or shortcut strategies.
Read the full abstract ▾
Machine Learning
Towards Building Non-Fine-Tunable Foundation Models
Ziyao Wang, Nizhang Li, Pingzhi Li et al. (+3 more)
Open-sourcing foundation models (FMs) enables broad reuse but also exposes model trainers to economic and safety risks from unrestricted downstream fine-tuning. We address this problem by building non-fine-tunable foundation models: models that remain broadly usable in their released form while yielding limited adaptation gains under task-agnostic unauthorized fine-tuning. We propose Private Mask Pre-Training (PMP), a pre-training framework that concentrates representation learning into a sparse subnetwork identified early in training. The binary mask defining this subnetwork is kept private, and only the final dense weights are released. This forces unauthorized fine-tuning without access to the mask to update parameters misaligned with pretraining subspace, inducing an intrinsic mismatch between the fine-tuning objective and the pre-training geometry. We provide theoretical analysis showing that this mismatch destabilizes gradient-based adaptation and bounds fine-tuning gains. Empirical results on large
language models demonstrating that PMP preserves base model performance while consistently degrading unauthorized fine-tuning across a wide range of downstream tasks, with the strength of non-fine-tunability controlled by the mask ratio.
Read the full abstract ▾
All authors Ziyao Wang, Nizhang Li, Pingzhi Li, Guoheng Sun, Tianlong Chen, Ang Li
Machine Learning · Cross-listed from cs.CV
DISK: Dynamic Inference SKipping for World Models
Anugunj Naman, Gaibo Zhang, Ayushman Singh et al. (+1 more)
We present DISK, a training-free adaptive inference method for autoregressive world models. DISK coordinates two coupled
diffusion transformers for
video and ego-trajectory via dual-branch controllers with cross-modal skip decisions, preserving motion-appearance consistency without retraining. We extend higher-order latent-difference skip testing to the autoregressive chain-of-forward regime and propagate controller statistics through rollout loops for long-horizon stability. When integrated into closed-loop driving rollouts on 1500 NuPlan and NuScenes samples using an NVIDIA L40S GPU, DISK achieves 2x speedup on trajectory
diffusion and 1.6x speedup on
video diffusion while maintaining L2 planning error, visual quality (FID/FVD), and NAVSIM PDMS scores, demonstrating practical long-horizon
video-and-trajectory prediction at substantially reduced cost.
Read the full abstract ▾
All authors Anugunj Naman, Gaibo Zhang, Ayushman Singh, Yaguang Zhang
Machine Learning · Cross-listed from math.OC
Exact Instance Compression for Convex Empirical Risk Minimization via Color Refinement
Bryan Zhu and Ziang Chen
Empirical risk minimization (ERM) can be computationally expensive, with standard solvers scaling poorly even in the convex setting. We propose a novel lossless compression framework for convex ERM based on color refinement, extending prior work from linear programs and convex quadratic programs to a broad class of differentiable convex optimization problems. We develop concrete
algorithms for a range of models, including linear and
polynomial regression, binary and multiclass logistic regression, regression with elastic-net regularization, and kernel methods such as kernel ridge regression and kernel logistic regression. Numerical experiments on representative
datasets demonstrate the effectiveness of the proposed approach.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Topological Residual Asymmetry for Bivariate Causal Direction
Mouad El Bouchattaoui
Inferring causal direction from purely observational bivariate data is fragile: many methods commit to a direction even in ambiguous or near non-identifiable regimes. We propose Topological Residual Asymmetry (TRA), a geometry-based criterion for additive-noise models. TRA compares the shapes of two cross-fitted regressor-residual clouds after rank-based copula standardization: in the correct direction, residuals are approximately independent, producing a two-dimensional bulk, while in the reverse direction – especially under low noise – the cloud concentrates near a one-dimensional tube. We quantify this bulk-tube contrast using a 0D persistent-homology functional, computed efficiently from Euclidean MST edge-length profiles. We prove consistency in a triangular-array small-noise regime, extend the method to fixed noise via a binned variant (TRA-s), and introduce TRA-C, a confounding-aware abstention rule calibrated by a Gaussian-copula plug-in bootstrap. Extensive experiments across many challenging synthetic and real-data scenarios demonstrate the method's superiority.
Read the full abstract ▾
Machine Learning
LLMs as High-Dimensional Nonlinear Autoregressive Models with Attention: Training, Alignment and Inference
Vikram Krishnamurthy
Large
language models (LLMs) based on
transformer architectures are typically described through collections of architectural components and training procedures, obscuring their underlying computational structure. This review article provides a concise mathematical reference for researchers seeking an explicit, equation-level description of LLM training,
alignment, and generation. We formulate LLMs as high-dimensional nonlinear autoregressive models with
attention-based dependencies. The framework encompasses pretraining via next-
token prediction,
alignment methods such as reinforcement learning from human feedback (RLHF), direct preference optimization (DPO), rejection sampling fine-tuning (RSFT), and reinforcement learning from verifiable rewards (RLVR), as well as autoregressive generation during inference. Self-
attention emerges naturally as a repeated bilinear–softmax–linear composition, yielding highly expressive sequence models. This formulation enables principled analysis of
alignment-induced behaviors (including sycophancy), inference-time phenomena (such as
hallucination, in-context learning, chain-of-thought prompting, and
retrieval-augmented generation), and extensions like continual learning, while serving as a concise reference for interpretation and further theoretical development.
Read the full abstract ▾
Machine Learning
Open Materials Generation with Inference-Time Reinforcement Learning
Philipp Hoellmer and Stefano Martiniani
Continuous-time generative models for crystalline materials enable inverse materials design by learning to predict stable
crystal structures, but incorporating explicit target properties into the generative process remains challenging. Policy-gradient reinforcement learning (RL) provides a principled mechanism for aligning generative models with downstream objectives but typically requires access to the score, which has prevented its application to flow-based models that learn only velocity fields. We introduce Open Materials Generation with Inference-time Reinforcement Learning (OMatG-IRL), a policy-gradient RL framework that operates directly on the learned velocity fields and eliminates the need for the explicit computation of the score. OMatG-IRL leverages
stochastic perturbations of the underlying generation dynamics preserving the baseline performance of the pretrained generative model while enabling exploration and policy-gradient estimation at inference time. Using OMatG-IRL, we present the first application of RL to
crystal structure prediction (CSP). Our method enables effective reinforcement of an energy-based objective while preserving diversity through composition conditioning, and it achieves performance competitive with score-based RL approaches. Finally, we show that OMatG-IRL can learn time-dependent velocity-annealing schedules, enabling accurate CSP with order-of-magnitude improvements in sampling efficiency and, correspondingly, reduction in generation time. The OMatG-IRL code is included in a new release of the Open Materials Generation (OMatG) framework available at https://github.com/FERMat-ML/OMatG.
Read the full abstract ▾
Machine Learning
scBatchProx: Federated-Inspired Refinement for Stable Cell-Type Discriminability under Heterogeneous Batch Compositions
Quang-Huy Nguyen, Jiaqi Wang and Wei-Shinn Ku
Single-
cell integration workflows often construct low-dimensional
cell embeddings and then refine them with post-hoc methods to reduce batch effects. This refinement process can become unstable when
cell-type compositions vary across batches, with some populations underrepresented or absent in particular batches. The problem becomes more consequential in dynamic single-
cell data systems, where newly acquired batches can change both technical conditions and
cell-type composition. Such instability can reduce downstream
cell-type classification performance and weaken stability under imbalance perturbations. We introduce scBatchProx, a lightweight post-hoc refinement method for stabilizing single-
cell latent
embeddings in these heterogeneous and evolving settings. scBatchProx operates directly on precomputed
embeddings and treats each batch or study as a client in a
federated-inspired optimization procedure. A batch-conditioned FiLM adapter learns local latent updates, while proximal and identity-preserving regularization keep these updates conservative. Experiments on multi-batch and cross-study single-
cell datasets show that scBatchProx improves downstream
cell-type classification across different upstream
embeddings. In controlled imbalance perturbations, scBatchProx maintains more stable affected-
cell-type F1 when selected populations are downsampled or ablated from one batch. In cumulative retraining and continual integration settings, scBatchProx remains effective as new
datasets arrive over time. Together, these results suggest that conservative,
federated-inspired refinement can help maintain stable single-
cell embeddings as batch compositions change across
datasets and over time.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Shuffle and Joint Differential Privacy for Generalized Linear Contextual Bandits
Sahasrajit Sarmasarkar
We present the first
algorithms for generalized linear contextual bandits under shuffle differential
privacy and joint differential
privacy. While prior work on private contextual bandits has been restricted to linear reward models – which admit closed-form
estimators – generalized linear models (GLMs) pose fundamental new challenges: no closed-form
estimator exists, requiring private convex optimization;
privacy must be tracked across multiple evolving design
matrices; and optimization error must be explicitly incorporated into regret analysis. We address these challenges under two
privacy models and context settings. For
stochastic contexts, we design a shuffle-DP
algorithm achieving $\tilde{O}(d^{3/2}\sqrt{T \log T}/\sqrt{\varepsilon})$ regret in dominant term, differing from the non-private rate by a factor of $\sqrt{d/\varepsilon}$. For
adversarial contexts, we provide a joint-DP
algorithm with regret $\tilde{O}\!\big(d\sqrt{T} \log T + d^{3/4}\sqrt{T/\varepsilon}\,(\log T)\,(d + \log T)^{1/4}\big)$ – matching the non-private rate $\tilde{O}(d\sqrt{T} \log T)$ in the leading term, with
privacy contributing only an additive correction. Unlike prior work on locally private GLM bandits, our methods require no spectral assumptions on the context distribution beyond $\ell_2$ boundedness.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Vision-Language Models
Zhisheng Chen, Tingyu Wu, Zijie Zhou et al. (+7 more)
Memory is not merely a storage mechanism for intelligent systems, but a structure for organizing evidence and constraining belief. This is especially important for
multimodal reasoning, where retrieved evidence must be both query-relevant and visually consistent. However, current
memory systems for
vision-
language models (VLMs) remain largely positive-associative: they retrieve what is similar or previously observed, but lack an explicit way to remember what has been verified as absent or logically excluded. To this end, we propose PolarMem, a training-free polarized latent graph
memory framework for verifiable
vision-
language reasoning. PolarMem transforms frozen VLM perceptual signals into HAS, NOT\_HAS, and Uncertain
memory states through semantic consistency verification and adaptive distributional partitioning, and stores them in a polarized graph with distinct positive and negative
memory relations. During inference, a lexicographical logic-aware
retrieval protocol enforces logical consistency before semantic similarity, suppressing conflicting
memories before they enter the model context. Across eight frozen VLM backbones and six
multimodal benchmarks, PolarMem consistently improves
retrieval-intensive tasks and reduces
retrieval-level contradictions. These results highlight negative
memory as a key mechanism for building more reliable
multimodal memory systems. Our code is available at https://github.com/czs-ict/PolarMem.
Read the full abstract ▾
All authors Zhisheng Chen, Tingyu Wu, Zijie Zhou, Zhengwei Xie, Jinhan Li, Ziyan Weng, Liang Lin, Jingwei Song, Zikai Xiao, Yingwei Zhang
Machine Learning · Cross-listed from cs.CV
Toward Autonomous Laboratory Safety Monitoring with Vision Language Models: Learning to See Hazards Through Scene Structure
Trishna Chakraborty, Udita Ghosh, Aldair Ernesto Gongora et al. (+5 more)
Laboratories are prone to severe injuries from unsafe actions, yet continuous manual safety monitoring, beyond mandatory pre-lab safety training, is infeasible.
Vision-
language models (VLMs) offer promise for autonomous laboratory safety monitoring, but their effectiveness in realistic settings is unclear due to the lack of visual evaluation data. Safety incidents are mostly documented as text, and existing studies are largely limited to textual understanding. Since hazardous scenes are unsafe to stage or capture, synthetic visual data becomes the only practical way to study this problem. This motivates us to introduce a structured data generation pipeline that converts textual laboratory scenarios into aligned quadruples using LLM as scene graph architect and image generation model as renderer, yielding \dataset: a synthetic
dataset of 1,207 $\langle$image, scene graph, ground truth, hazard relevance mask$\rangle$ quadruples across 362 unique scenarios. Experiments on \dataset and seven open- and closed-source models show that VLMs perform effectively given a textual scene graph but degrade substantially in visual-only settings (i.e., where no scene graph is involved), indicating difficulty in extracting structured object relationships directly from pixels. We thus propose \saga, a post-training framework that leverages reinforcement learning to first structure the visual scene as a scene graph, and then predict hazards while aligning
attention with relevant image regions. Evaluations with Qwen 3.5 and Gemma 4 show that our method nearly doubles the $F_1$ score over the baseline on our synthetic
dataset and achieves an improvement of $\sim 3\times$ on a mini real-world
dataset, while preserving general visual understanding in visual question answering.
Read the full abstract ▾
All authors Trishna Chakraborty, Udita Ghosh, Aldair Ernesto Gongora, Ruben Glatt, Yue Dong, Jiachen Li, Amit K. Roy-Chowdhury, Chengyu Song