Machine Learning
OmniPatch: A Universal Adversarial Patch for ViT-CNN Cross-Architecture Transfer in Semantic Segmentation
Aarush Aggarwal, Akshat Tomar, Amritanshu Tiwari et al. (+1 more)
Robust semantic
segmentation is crucial for safe autonomous driving, yet deployed models remain vulnerable to black-box
adversarial attacks when target weights are unknown. Most existing approaches either craft image-wide perturbations or optimize patches for a single architecture, which limits their practicality and transferability. We introduce OmniPatch, a training framework for learning a universal
adversarial patch that generalizes across images and both ViT and CNN architectures without requiring access to target model parameters.
Read the full abstract ▾
All authors Aarush Aggarwal, Akshat Tomar, Amritanshu Tiwari, Sargam Goyal
Machine Learning
Evaluating Uplift Modeling under Structural Biases: Insights into Metric Stability and Model Robustness
Yuxuan Yang, Dugang Liu and Yiyan Huang
In personalized marketing, uplift models estimate the incremental effect of an intervention by modeling how customer behavior would change under alternative treatments using counterfactual analysis. However, real-world marketing data often exhibit various biases, such as selection bias, spillover effects, measurement error, and unobserved confounding. These biases can adversely affect both the accuracy of uplift estimation and the validity of evaluation metrics. Despite the importance of bias-aware assessment, there remains a lack of systematic studies evaluating how different models and metrics perform under such biased conditions. To bridge this gap, we design a systematic benchmarking framework. Unlike standard predictive tasks, real-world uplift
datasets inherently lack counterfactual ground truth. This limitation renders the direct validation of evaluation metrics infeasible and prevents the precise quantification of biases. Therefore, a semi-synthetic approach serves as a critical enabler for systematic benchmarking. This approach effectively bridges the gap by retaining real-world feature dependencies while providing the ground truth needed to isolate structural biases. Our investigations reveal that (i) uplift targeting and prediction can manifest as distinct objectives, where proficiency in one does not ensure efficacy in the other; (ii) while many models exhibit inconsistent performance under diverse biases, TARNet shows notable robustness, providing insights for subsequent model design; (iii) the stability of evaluation metrics is linked to their mathematical
alignment with the ATE, suggesting that ATE-approximating metrics yield more consistent model rankings under structural data imperfections. These findings suggest the need for more robust uplift models and evaluation metrics under real-world data imperfections.
Read the full abstract ▾
Machine Learning
Adversarial Attacks on Locally Private Graph Neural Networks
Matta Varun, Ajay Kumar Dhakar, Yuan Hong et al. (+1 more)
Graph neural network (GNN) is a powerful tool for analyzing graph-structured data. However, their vulnerability to
adversarial attacks raises serious concerns, especially when dealing with sensitive information. Local Differential
Privacy (LDP) offers a
privacy-preserving framework for training GNNs, but its impact on
adversarial robustness remains underexplored. This paper investigates
adversarial attacks on LDP-protected GNNs. We explore how the
privacy guarantees of LDP can be leveraged or hindered by
adversarial perturbations. The effectiveness of existing attack methods on LDP-protected GNNs are analyzed and potential challenges in crafting
adversarial examples under LDP constraints are discussed. Additionally, we suggest directions for defending LDP-protected GNNs against
adversarial attacks. This work investigates the interplay between
privacy and security in graph learning, highlighting the need for robust and
privacy-preserving GNN architectures.
Read the full abstract ▾
All authors Matta Varun, Ajay Kumar Dhakar, Yuan Hong, Shamik Sural
Machine Learning · Cross-listed from cs.AI
Multi-RF Fusion with Multi-GNN Blending for Molecular Property Prediction
Zacharie Bugaud
Multi-RF Fusion achieves a test ROC-AUC of 0.8476 +/- 0.0002 on ogbg-molhiv (10 seeds), placing #1 on the OGB leaderboard ahead of HyperFusion (0.8475 +/- 0.0003). The core of the method is a rank-averaged ensemble of 12 Random Forest models trained on concatenated molecular fingerprints (FCFP, ECFP, MACCS, atom pairs – 4,263 dimensions total), blended with deep-ensembled GNN predictions at 12% weight. Two findings drive the result: (1) setting max_features to 0.20 instead of the default sqrt(d) gives a +0.008 AUC gain on this scaffold split, and (2) averaging GNN predictions across 10 seeds before blending with the RF eliminates GNN seed variance entirely, dropping the final standard deviation from 0.0008 to 0.0002. No external data or pre-training is used.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DC
RoboECC: Multi-Factor-Aware Edge-Cloud Collaborative Deployment for VLA Models
Zihao Zheng, Hangyu Cao, Jiayu Chen et al. (+6 more)
Vision-
Language-Action (VLA) models are mainstream in
embodied intelligence but face high inference costs. Edge-Cloud Collaborative (ECC) deployment offers an effective fix by easing edge-device computing pressure to meet real-time needs. However, existing ECC frameworks are suboptimal for VLA models due to two challenges: (1) Diverse model structures hinder optimal ECC
segmentation point identification; (2) Even if the optimal split point is determined, changes in network bandwidth can cause performance drift. To address these issues, we propose a novel ECC deployment framework for various VLA models, termed RoboECC. Specifically, we propose a model-hardware co-aware
segmentation strategy to help find the optimal
segmentation point for various VLA models. Moreover, we propose a network-aware deployment adjustment approach to adapt to the network fluctuations for maintaining optimal performance. Experiments demonstrate that RoboECC achieves a speedup of up to 3.28x with only 2.55%~2.62% overhead.
Read the full abstract ▾
All authors Zihao Zheng, Hangyu Cao, Jiayu Chen, Sicheng Tian, Chenyue Li, Maoliang Li, Xinhao Sun, Guojie Luo, Xiang Chen
Machine Learning · Cross-listed from cs.IR
NDT: Non-Differential Transformer and Its Application to Sentiment Analysis
Soudeep Ghoshal, Himanshu Buckchash, Sarita Paudel et al. (+1 more)
From customer feedback to social media, understanding human sentiment in text is central to how machines can interact meaningfully with people. However, despite notable progress, accurately capturing sentiment remains a challenging task, which continues to motivate further research in this area. To this end, we introduce Non-Differential
Transformer (NDT). It is inspired by (but in contrast to) the state-of-the-art Differential
Transformer (DT) model. While standard
Transformers can struggle with irrelevant context, the sota DT model uses
attention map subtraction, potentially for noise cancellation. We explore an alternative motivation, hypothesizing that benefits may arise from enabling different
attention components to specialize on distinct concepts within the text, similar to multiplexing information channels or mixture models, rather than primarily canceling noise via subtraction. Guided by this concept-multiplexing (ConPlex) view, the specific architecture presented in this paper employs a purely additive strategy. It uses only positive weights, learned during training, to ensure constructive combination of these specialized
attention perspectives. This design choice explores positive only integration, though our broader framework also shows promise with less constrained linear combinations involving both positive and negative weights. Our model computes
attention via this positively weighted sum of multiple distinct
attention maps. This allows the model to constructively integrate diverse signals and potentially capture more complex contextual relationships. Competitive performance is achieved by the proposed model for Sentiment Analysis while tested on multiple
datasets. We conclude by presenting our results, challenges and future research agenda in this important area of research.
Read the full abstract ▾
All authors Soudeep Ghoshal, Himanshu Buckchash, Sarita Paudel, Rubén Ruiz-Torrubiano
Machine Learning · Cross-listed from eess.IV
mmWave-Diffusion:A Novel Framework for Respiration Sensing Using Observation-Anchored Conditional Diffusion Model
Yong Wang, Qifan Shen, Bao Zhang et al. (+4 more)
Millimeter-wave (mmWave) radar enables contactless respiratory sensing,yet fine-grained monitoring is often degraded by nonstationary interference from body micromotions.To achieve micromotion interference removal,we propose mmWave-
Diffusion,an observation-anchored conditional
diffusion framework that directly models the residual between radar phase observations and the respiratory ground truth,and initializes sampling within an observation-consistent neighborhood rather than from Gaussian noise-thereby aligning the generative process with the measurement physics and reducing inference overhead. The accompanying Radar
Diffusion Transformer (RDT) is explicitly conditioned on phase observations, enforces strict one-to-one temporal
alignment via patch-level dual positional encodings, and injects local physical priors through banded-mask multi-head cross-
attention, enabling robust denoising and interference removal in just 20 reverse steps. Evaluated on 13.25 hours of synchronized radar-respiration data, mmWave-
Diffusion achieves state-of-the-art waveform reconstruction and respiratory-rate estimation with strong generalization. Code repository:https://github.com/goodluckyongw/mmWave-
Diffusion.
Read the full abstract ▾
All authors Yong Wang, Qifan Shen, Bao Zhang, Zijun Huang, Chengbo Zhu, Shuai Yao, Qisong Wu
Machine Learning · Cross-listed from stat.ML
High-dimensional online learning via asynchronous decomposition: Non-divergent results, dynamic regularization, and beyond
Shixiang Liu, Zhifan Li, Hanming Yang et al. (+1 more)
Existing high-dimensional online learning methods often face the challenge that their error bounds, or per-batch sample sizes, diverge as the number of data batches increases. To address this issue, we propose an asynchronous decomposition framework that leverages summary statistics to construct a surrogate score function for current-batch learning. This framework is implemented via a dynamic-regularized iterative hard thresholding
algorithm, providing a computationally and
memory-efficient solution for sparse online optimization. We provide a unified theoretical analysis that accounts for both the streaming computational error and statistical accuracy, establishing that our
estimator maintains non-divergent error bounds and $\ell_0$ sparsity across all batches. Furthermore, the proposed
estimator adaptively achieves additional gains as batches accumulate, attaining the oracle accuracy as if the entire historical
dataset were accessible and the true support were known. These theoretical properties are further illustrated through an example of the generalized linear model.
Read the full abstract ▾
All authors Shixiang Liu, Zhifan Li, Hanming Yang, Jianxin Yin
Machine Learning
Neuronal Self-Adaptation Enhances Capacity and Robustness of Representation in Spiking Neural Networks
Zhuobin Yang, Liao Yu, Yeyao Bao et al. (+5 more)
Spiking Neural Networks (SNNs) are promising for energy-efficient, real-time edge computing, yet their performance is often constrained by the limited adaptability of conventional leaky integrate-and-fire (LIF)
neurons. Existing LIF models struggle with restricted information capacity and susceptibility to noise, leading to degraded accuracy and compromised robustness. Inspired by the dynamic self-regulation of biological potassium channels, we propose the Potassium-regulated LIF (KvLIF)
neuron model. KvLIF introduces an auxiliary conductance state that integrates membrane potential and spiking history to adaptively modulate neuronal excitability and reset dynamics. This design extends the dynamic response range of
neurons to varying input intensities and effectively suppresses noise-induced spikes. We extensively evaluate KvLIF on both static image and neuromorphic
datasets, demonstrating consistent improvements in classification accuracy and superior robustness compared to existing LIF models. Our work bridges biological plausibility with computational efficiency, offering a
neuron model that enhances SNN performance while maintaining suitability for low-power neuromorphic deployment.
Read the full abstract ▾
All authors Zhuobin Yang, Liao Yu, Yeyao Bao, Jian Zhang, Liangfu Lv, Daqing Guo, Xiaohong Li, Yunliang Zang
Machine Learning
Centrality-Based Pruning for Efficient Echo State Networks
Sudip Laudari
Echo State Networks (ESNs) are a reservoir computing framework widely used for nonlinear time-series prediction. However, despite their effectiveness, randomly initialized reservoirs often contain redundant nodes, leading to unnecessary computational overhead and reduced efficiency. In this work, we propose a graph centrality-based
pruning approach that interprets the reservoir as a weighted directed graph and removes structurally less important nodes using centrality measures. Experiments on Mackey-Glass time-series prediction and electric load
forecasting demonstrate that the proposed method can significantly reduce reservoir size while maintaining, and in some cases improving, prediction accuracy.
Read the full abstract ▾
Machine Learning · Cross-listed from q-bio.NC
Hierarchical Multiscale Structure-Function Coupling for Brain Connectome Integration
Jianwei Chen, Zhengyang Miao, Wenjie Cai et al. (+10 more)
Integrating structural and functional connectomes remains challenging because their relationship is non-linear and organized over nested modular hierarchies. We propose a hierarchical multiscale structure-function coupling framework for connectome integration that jointly learns individualized modular organization and hierarchical coupling across structural connectivity (SC) and functional connectivity (FC). The framework includes: (i) Prototype-based Modular Pooling (PMPool), which learns modality-specific multiscale communities by selecting prototypical ROIs and optimizing a differentiable modularity-inspired objective; (ii) an
Attention-based Hierarchical Coupling Module (AHCM) that models both within-hierarchy and cross-hierarchy SC-FC interactions to produce enriched hierarchical coupling representations; and (iii) a Coupling-guided Clustering loss (CgC-Loss) that regularizes SC and FC community assignments with coupling signals, allowing cross-modal interactions to shape community
alignment across hierarchies. We evaluate the model's performance across four cohorts for predicting
brain age, cognitive score, and disease classification. Our model consistently outperforms baselines and other state-of-the-art approaches across three tasks. Ablation and sensitivity analyses verify the contributions of key components. Finally, the visualizations of learned coupling reveal interpretable differences, suggesting that the framework captures biologically meaningful structure-function relationships.
Read the full abstract ▾
All authors Jianwei Chen, Zhengyang Miao, Wenjie Cai, Jiaxue Tang, Boxing Liu, Yunfan Zhang, Yuhang Yang, Hao Tang, Carola-Bibiane Schönlieb, Zaixu Cui, Du Lei, Shouliang Qi, Chao Li
Machine Learning
Breaking the $O(\sqrt{T})$ Cumulative Constraint Violation Barrier while Achieving $O(\sqrt{T})$ Static Regret in Constrained Online Convex Optimization
Haricharan Balasundaram, Karthick Krishna Mahendran and Rahul Vaze
The problem of constrained online convex optimization is considered, where at each round, once a learner commits to an action $x_t \in \mathcal{X} \subset \mathbb{R}^d$, a convex loss function $f_t$ and a convex constraint function $g_t$ that drives the constraint $g_t(x)\le 0$ are revealed. The objective is to simultaneously minimize the static regret and cumulative constraint violation (CCV) compared to the
benchmark that knows the loss functions and constraint functions $f_t$ and $g_t$ for all $t$ ahead of time, and chooses a static optimal action that is feasible with respect to all $g_t(x)\le 0$. In recent prior work Sinha and Vaze [2024],
algorithms with simultaneous regret of $O(\sqrt{T})$ and CCV of $O(\sqrt{T})$ or (CCV of $O(1)$ in specific cases Vaze and Sinha [2025], e.g. when $d=1$) have been proposed. It is widely believed that CCV is $\Omega(\sqrt{T})$ for all
algorithms that ensure that regret is $O(\sqrt{T})$ with the worst case input for any $d\ge 2$. In this paper, we refute this and show that the
algorithm of Vaze and Sinha [2025] simultaneously achieves regret of $O(\sqrt{T})$ regret and CCV of $O(T^{1/3})$ when $d=2$.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics
Aratrika Mustafi and Soumya Mukherjee
We propose a dense associative
memory for empirical measures (weighted point clouds). Stored patterns and queries are finitely supported probability measures, and
retrieval is defined by minimizing a Hopfield-style log-sum-exp energy built from the debiased Sinkhorn divergence. We derive
retrieval dynamics as a spherical Hellinger Kantorovich (SHK) gradient flow, which updates both support locations and weights. Discretizing the flow yields a deterministic
algorithm that uses Sinkhorn potentials to compute barycentric transport steps and a multiplicative simplex reweighting. Under local separation and PL-type conditions we prove basin invariance,
geometric convergence to a local minimizer, and a bound showing the minimizer remains close to the corresponding stored pattern. Under a random pattern model, we further show that these Sinkhorn basins are disjoint with high probability, implying exponential capacity in the ambient dimension. Experiments on synthetic Gaussian point-cloud
memories demonstrate robust recovery from perturbed queries versus a Euclidean Hopfield-type baseline.
Read the full abstract ▾
Machine Learning
Exponential Family Discriminant Analysis: Generalizing LDA-Style Generative Classification to Non-Gaussian Models
Anish Lakkapragada
We introduce Exponential Family Discriminant Analysis (EFDA), a unified generative framework that extends classical Linear Discriminant Analysis (LDA) beyond the Gaussian setting to any member of the exponential family. Under the assumption that each class-conditional density belongs to a common exponential family, EFDA derives closed-form maximum-likelihood
estimators for all natural parameters and yields a decision rule that is linear in the sufficient statistic, recovering LDA as a special case and capturing nonlinear decision boundaries in the original feature space. We prove that EFDA is asymptotically calibrated and statistically efficient under correct specification, and we generalise it to $K \geq 2$ classes and multivariate data. Through extensive simulation across five exponential-family distributions (Weibull, Gamma, Exponential, Poisson, Negative Binomial), EFDA matches the classification accuracy of LDA, QDA, and logistic regression while reducing Expected Calibration Error (ECE) by $2$-$6\times$, a gap that is structural: it persists for all $n$ and across all class-imbalance levels, because misspecified models remain asymptotically miscalibrated. We further prove and empirically confirm that EFDA's log-odds
estimator approaches the Cramér-Rao bound under correct specification, and is the only
estimator in our comparison whose mean squared error converges to zero. Complete derivations are provided for nine distributions. Finally, we formally verify all four theoretical propositions in Lean 4, using Aristotle (Harmonic) and OpenGauss (Math, Inc.) as proof generators, with all outputs independently machine-checked by AXLE (Axiom).
Read the full abstract ▾
Machine Learning
Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity
Zixuan Zhang, Kaixuan Huang, Tuo Zhao et al. (+2 more)
Diffusion models have become a leading framework in generative modeling, yet their theoretical understanding – especially for high-dimensional data concentrated on low-dimensional structures – remains incomplete. This paper investigates how
diffusion models learn such structured data, focusing on two key aspects: statistical complexity and influence of data
geometric properties. By modeling data as samples from a smooth Riemannian
manifold, our analysis reveals crucial decompositions of score functions in
diffusion models under different levels of injected noise. We also highlight the interplay of
manifold curvature with the structures in the score function. These analyses enable an efficient neural network approximation to the score function, built upon which we further provide statistical rates for score estimation and distribution learning. Remarkably, the obtained statistical rates are governed by the intrinsic dimension of data and the
manifold curvature. These results advance the statistical foundations of
diffusion models, bridging theory and practice for generative modeling on
manifolds.
Read the full abstract ▾
All authors Zixuan Zhang, Kaixuan Huang, Tuo Zhao, Mengdi Wang, Minshuo Chen
Machine Learning
CFNN: Continued Fraction Neural Network
Chao Wang, Xuancheng Zhou, Ruilin Hou et al. (+2 more)
Accurately characterizing non-linear functional
manifolds with singularities is a fundamental challenge in scientific computing. While Multi-Layer Perceptrons (MLPs) dominate, their spectral bias hinders resolving high-
curvature features without excessive parameters. We introduce Continued Fraction Neural Networks (CFNNs), integrating continued fractions with gradient-based optimization to provide a “rational inductive bias.” This enables capturing complex asymptotics and discontinuities with extreme parameter frugality. We provide formal approximation bounds demonstrating exponential convergence and stability guarantees. To address recursive instability, we develop three implementations: CFNN-Boost, CFNN-MoE, and CFNN-Hybrid.
Benchmarks show CFNNs consistently outperform MLPs in precision with one to two orders of magnitude fewer parameters, exhibiting up to a 47-fold improvement in noise robustness and physical consistency. By bridging black-box flexibility and white-box transparency, CFNNs establish a reliable “grey-box” paradigm for AI-driven scientific research.
Read the full abstract ▾
All authors Chao Wang, Xuancheng Zhou, Ruilin Hou, Xiaoyu Cheng, Ruiyi Ding
Machine Learning
Optimal low-rank stochastic gradient estimation for LLM training
Zehao Li, Tao Ren, Zishi Zhang et al. (+2 more)
Large
language model (LLM) training is often bottlenecked by
memory constraints and
stochastic gradient noise in extremely high-dimensional parameter spaces. Motivated by empirical evidence that many LLM gradient
matrices are effectively low-rank during training, we present an unbiased,
memory-efficient, low-rank
matrix estimator with the lowest variance that is applicable across common
stochastic gradient estimation paradigms. The core idea is to project a high-dimensional
stochastic gradient
estimator onto a random low-dimensional subspace and lift it back, reducing
memory while keeping the
estimator unbiased and controlling mean-squared error via an optimally designed projection distribution, including Haar–Stiefel projections. The projection distribution is derived by solving a constrained functional optimization problem, yielding an optimal random projector that guides
algorithm design. Empirically, the resulting low-rank gradient
estimators deliver both practical
memory savings and improved training behavior. In RoBERTa-large fine-tuning, our method attains the lowest peak GPU
memory among compared methods (e.g., 3.83GB versus 16.7GB for full BP) while remaining competitive in accuracy; in autoregressive LLM pretraining (LLaMA-20M/60M/100M), our method outperforms the traditional methods, supporting the benefit of the proposed optimal projection strategy.
Read the full abstract ▾
All authors Zehao Li, Tao Ren, Zishi Zhang, Xi Chen, Yijie Peng
Machine Learning · Cross-listed from stat.ML
LassoFlexNet: Flexible Neural Architecture for Tabular Data
Kry Yik Chau Lui, Cheng Chi, Kishore Basu et al. (+1 more)
Despite their dominance in
vision and
language, deep neural networks often underperform relative to tree-based models on tabular data. To bridge this gap, we incorporate five key inductive biases into deep learning: robustness to irrelevant features, axis
alignment, localized irregularities, feature heterogeneity, and training stability. We propose LassoFlexNet, an architecture that evaluates the linear and nonlinear marginal contribution of each input via Per-Feature
Embeddings, and sparsely selects relevant variables using a Tied Group Lasso mechanism. Because these components introduce optimization challenges that destabilize standard proximal methods, we develop a Sequential Hierarchical Proximal Adaptive Gradient optimizer with exponential moving averages (EMA) to ensure stable convergence. Across $52$
datasets from three
benchmarks, LassoFlexNet matches or outperforms leading tree-based models, achieving up to a $10$% relative gain, while maintaining Lasso-like interpretability. We substantiate these empirical results with ablation studies and theoretical proofs confirming the architecture's enhanced expressivity and structural breaking of undesired rotational invariance.
Read the full abstract ▾
All authors Kry Yik Chau Lui, Cheng Chi, Kishore Basu, Yanshuai Cao
Machine Learning
Beyond Token Eviction: Mixed-Dimension Budget Allocation for Efficient KV Cache Compression
Ruijie Miao, Zhiming Wang, Wang Li et al. (+4 more)
Key-value (KV) caching is widely used to accelerate
transformer inference, but its
memory cost grows linearly with input length, limiting long-context deployment. Existing
token eviction methods reduce
memory by discarding less important
tokens, which can be viewed as a coarse form of dimensionality reduction that assigns each
token either zero or full dimension. We propose MixedDimKV, a mixed-dimension KV cache compression method that allocates dimensions to
tokens at a more granular level, and MixedDimKV-H, which further integrates head-level importance information. Experiments on long-context
benchmarks show that MixedDimKV outperforms prior KV cache compression methods that do not rely on head-level importance profiling. When equipped with the same head-level importance information, MixedDimKV-H consistently outperforms HeadKV. Notably, our approach achieves comparable performance to full
attention on LongBench with only 6.25% of the KV cache. Furthermore, in the Needle-in-a-Haystack test, our solution maintains 100% accuracy at a 50K context length while using as little as 0.26% of the cache.
Read the full abstract ▾
All authors Ruijie Miao, Zhiming Wang, Wang Li, Shiwei Wu, Shufan Liu, Yanbing Jiang, Tong Yang
Machine Learning · Cross-listed from cs.RO
Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models
Zhilong Zhang, Haoxiang Ren, Yihao Sun et al. (+6 more)
Vision-
Language-Action (VLA) models show strong generalization for robotic control, but finetuning them with reinforcement learning (RL) is constrained by the high cost and safety risks of real-world interaction. Training VLA models in interactive world models avoids these issues but introduces several challenges, including pixel-level world modeling, multi-view consistency, and compounding errors under sparse rewards. Building on recent advances across large
multimodal models and model-based RL, we propose VLA-MBPO, a practical framework to tackle these problems in VLA finetuning. Our approach has three key design choices: (i) adapting unified
multimodal models (UMMs) for data-efficient world modeling; (ii) an interleaved view decoding mechanism to enforce multi-view consistency; and (iii) chunk-level branched rollout to mitigate error compounding. Theoretical analysis and experiments across simulation and real-world tasks demonstrate that VLA-MBPO significantly improves policy performance and sample efficiency, underscoring its robustness and scalability for real-world robotic deployment.
Read the full abstract ▾
All authors Zhilong Zhang, Haoxiang Ren, Yihao Sun, Yifei Sheng, Haonan Wang, Haoxin Lin, Zhichao Wu, Pierre-Luc Bacon, Yang Yu
Machine Learning
Bayesian Learning in Episodic Zero-Sum Games
Chang-Wei Yueh, Andy Zhao, Ashutosh Nayyar et al. (+1 more)
We study
Bayesian learning in episodic, finite-horizon zero-sum Markov games with unknown transition and reward models. We investigate a posterior
algorithm in which each player maintains a
Bayesian posterior over the game model, independently samples a game model at the beginning of each episode, and computes an
equilibrium policy for the sampled model. We analyze two settings: (i) Both players use the posterior sampling
algorithm, and (ii) Only one player uses posterior sampling while the opponent follows an arbitrary learning
algorithm. In each setting, we provide guarantees on the expected regret of the posterior sampling
agent. Our notion of regret compares the expected total reward of the learning
agent against the expected total reward under
equilibrium policies of the true game. Our main theoretical result is an expected regret bound for the posterior sampling
agent of order $O(HS\sqrt{ABHK\log(SABHK)})$ where $K$ is the number of episodes, $H$ is the episode length, $S$ is the number of states, and $A,B$ are the action space sizes of the two players. Experiments in a grid-world predator–prey domain illustrate the sublinear regret scaling and show that posterior sampling competes favorably with a fictitious-play baseline.
Read the full abstract ▾
All authors Chang-Wei Yueh, Andy Zhao, Ashutosh Nayyar, Rahul Jain
Machine Learning · Cross-listed from stat.ML
Interpretable Operator Learning for Inverse Problems via Adaptive Spectral Filtering: Convergence and Discretization Invariance
Hang-Cheng Dong, Pengcheng Cheng and Shuhuan Li
Solving ill-posed inverse problems necessitates effective regularization strategies to stabilize the inversion process against measurement noise. While classical methods like Tikhonov regularization require heuristic parameter tuning, and standard deep learning approaches often lack interpretability and generalization across resolutions, we propose SC-Net (Spectral Correction Network), a novel operator learning framework. SC-Net operates in the spectral domain of the forward operator, learning a pointwise adaptive filter function that reweights spectral coefficients based on the signal-to-noise ratio. We provide a theoretical analysis showing that SC-Net approximates the continuous inverse operator, guaranteeing discretization invariance. Numerical experiments on 1D integral equations demonstrate that SC-Net: (1) achieves the theoretical minimax optimal convergence rate ($O(\delta^{0.5})$ for $s=p=1.5$), matching theoretical lower bounds; (2) learns interpretable sharp-cutoff filters that outperform Oracle Tikhonov regularization; and (3) exhibits zero-shot super-resolution, maintaining stable reconstruction errors ($\approx 0.23$) when trained on coarse grids ($N=256$) and tested on significantly finer grids (up to $N=2048$). The proposed method bridges the gap between rigorous regularization theory and data-driven operator learning.
Read the full abstract ▾
Machine Learning
Generating from Discrete Distributions Using Diffusions: Insights from Random Constraint Satisfaction Problems
Alankrita Bhatt, Mukur Gupta, Germain Kolossov et al. (+1 more)
Generating data from discrete distributions is important for a number of application domains including text, tabular data, and genomic data. Several groups have recently used random $k$-satisfiability ($k$-SAT) as a synthetic
benchmark for new generative techniques. In this paper, we show that fundamental insights from the theory of random constraint satisfaction problems have observable implications (sometime contradicting intuition) on the behavior of generative techniques on such
benchmarks. More precisely, we study the problem of generating a uniformly random solution of a given (random) $k$-SAT or $k$-XORSAT formula. Among other findings, we observe that: $(i)$~Continuous
diffusions outperform masked discrete
diffusions; $(ii)$~Learned
diffusions can match the theoretical `ideal' accuracy; $(iii)$~Smart ordering of the variables can significantly improve accuracy, although not following popular heuristics.
Read the full abstract ▾
All authors Alankrita Bhatt, Mukur Gupta, Germain Kolossov, Andrea Montanari
Machine Learning
Neural collapse in the orthoplex regime
James Alcala, Rayna Andreeva, Vladimir A. Kobzar et al. (+4 more)
When training a neural network for classification, the feature vectors of the training set are known to collapse to the vertices of a regular simplex, provided the dimension $d$ of the feature space and the number $n$ of classes satisfies $n\leq d+1$. This phenomenon is known as neural collapse. For other applications like
language models, one instead takes $n\gg d$. Here, the neural collapse phenomenon still occurs, but with different emergent
geometric figures. We characterize these
geometric figures in the orthoplex regime where $d+2\leq n\leq 2d$. The techniques in our analysis primarily involve Radon's
theorem and convexity.
Read the full abstract ▾
All authors James Alcala, Rayna Andreeva, Vladimir A. Kobzar, Dustin G. Mixon, Sanghoon Na, Shashank Sule, Yangxinyu Xie
Machine Learning
Dong Liu, Yanxuan Yu, Ben Lengerich et al. (+1 more)
As long-context
language modeling becomes increasingly important, the cost of maintaining and attending to large Key/Value (KV) caches grows rapidly, becoming a major bottleneck in both training and inference. While prior works such as Multi-Query
Attention (MQA) and Multi-Latent
Attention (MLA) reduce
memory by sharing or compressing KV features, they often trade off representation quality or incur runtime overhead. We propose
Memory-Keyed
Attention (MKA), a hierarchical
attention mechanism that integrates multi-level KV caches (local, session, and long-term) and learns to route
attention across them dynamically. We further introduce Route-Fused MKA (FastMKA), a broadcast-routed variant that fuses
memory sources before
attention computation for improved efficiency. Experiments on different sequence lengths show that FastMKA achieves a favorable accuracy-efficiency trade-off: comparable perplexity to MLA while achieving up to 5x faster training throughput and 1.8x lower evaluation latency. These results highlight MKA as a practical and extensible framework for efficient long-context
attention.
Read the full abstract ▾
All authors Dong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu
Machine Learning
RECLAIM: Cyclic Causal Discovery Amid Measurement Noise
Muralikrishnna G. Sethuraman and Faramarz Fekri
Uncovering causal relationships is a fundamental problem across science and engineering. However, most existing causal discovery methods assume acyclicity and direct access to the system variables – assumptions that fail to hold in many real-world settings. For instance, in genomics, cyclic regulatory networks are common, and measurements are often corrupted by instrumental noise. To address these challenges, we propose RECLAIM, a causal discovery framework that natively handles both cycles and measurement noise. RECLAIM learns the causal graph structure by maximizing the likelihood of the observed measurements via expectation-maximization (EM), using residual normalizing flows for tractable likelihood computation. We consider two measurement models: (i) Gaussian additive noise, and (ii) a linear measurement system with additive Gaussian noise. We provide theoretical consistency guarantees for both the settings. Experiments on synthetic data and real-world
protein signaling
datasets demonstrate the efficacy of the proposed method.
Read the full abstract ▾
Machine Learning
LJ-Bench: Ontology-Based Benchmark for U.S. Crime
Hung Yun Tseng, Wuzhen Li, Blerina Gkotse et al. (+1 more)
The potential of Large
Language Models (LLMs) to provide harmful information remains a significant concern due to the vast breadth of illegal queries they may encounter. Unfortunately, existing
benchmarks only focus on a handful types of illegal activities, and are not grounded in legal works. In this work, we introduce an ontology of crime-related concepts grounded in the legal frameworks of Model Panel Code, which serves as an influential reference for criminal law and has been adopted by many U.S. states, and instantiated using Californian Law. This structured knowledge forms the foundation for LJ-Bench, the first comprehensive
benchmark designed to evaluate LLM robustness against a wide range of illegal activities. Spanning 76 distinct crime types organized taxonomically, LJ-Bench enables systematic assessment of diverse attacks, revealing valuable insights into LLM vulnerabilities across various crime categories: LLMs exhibit heightened susceptibility to attacks targeting societal harm rather than those directly impacting individuals. Our
benchmark aims to facilitate the development of more robust and trustworthy LLMs. The LJ-Bench
benchmark and LJ-Ontology, along with experiments implementation for reproducibility are publicly available at https://github.com/AndreaTseng/LJ-Bench.
Read the full abstract ▾
All authors Hung Yun Tseng, Wuzhen Li, Blerina Gkotse, Grigorios Chrysos