Machine Learning · Cross-listed from stat.ML
Generalized Robust Adaptive-Bandwidth Multi-View Manifold Learning in High Dimensions with Noise
Xiucai Ding, Chao Shen and Hau-Tieng Wu
Multiview
datasets are common in scientific and engineering applications, yet existing fusion methods offer limited theoretical guarantees, particularly in the presence of heterogeneous and high-dimensional noise. We propose Generalized Robust Adaptive-Bandwidth Multiview
Diffusion Maps (GRAB-MDM), a new kernel-based
diffusion geometry framework for integrating multiple noisy data sources. The key innovation of GRAB-MDM is a {view}-dependent bandwidth selection strategy that adapts to the geometry and noise level of each view, enabling a stable and principled construction of multiview
diffusion operators. Under a common-
manifold model, we establish asymptotic convergence results and show that the adaptive bandwidths lead to provably robust recovery of the shared intrinsic structure, even when noise levels and sensor dimensions differ across views. Numerical experiments demonstrate that GRAB-MDM significantly improves robustness and
embedding quality compared with fixed-bandwidth and equal-bandwidth baselines, and usually outperform existing
algorithms. The proposed framework offers a practical and theoretically grounded solution for multiview sensor fusion in high-dimensional noisy environments.
Read the full abstract ▾
Machine Learning
A swap-adversarial framework for domain generalization in pre- and post-stimulation Parkinsonian state classification
Seongwon Jin, Hanseul Choi, Sunggu Yang et al. (+2 more)
We propose a novel swap-
adversarial framework that mitigates high inter-subject variability and the high-dimensional low-sample-size problem in electrocorticography (ECoG) data. It achieves robust domain generalization across ECoG and electroencephalography (
EEG)-based
brain-computer interface
datasets. Our framework integrates (1) robust preprocessing, (2) inter-subject balanced channel swap (ISBCS) for cross-subject augmentation, and (3) domain-
adversarial learning (DAL) to suppress subject-specific bias. The ISBCS method is a bio-inspired channel swapping strategy that exchanges only functionally corresponding channels across subjects, guided by a
brain map, to mitigate inter-subject distribution differences. The DAL strategy encourages the model to learn task-relevant shared features. In addition, we introduce a new ECoG
dataset, the first reproducible
benchmark, which is constructed from long-term ECoG recordings of 6-hydroxydopamine-induced rat models and annotated with neural responses measured before and after electrical stimulation. We validate the effectiveness of this framework through extensive experiments under cross-subject, cross-session, and cross-
dataset settings. Our results show that the proposed framework exhibits the best and robust generalization performance both for our newly constructed ECoG and public
EEG datasets.
Read the full abstract ▾
All authors Seongwon Jin, Hanseul Choi, Sunggu Yang, Sungho Park, Jibum Kim
Machine Learning · Cross-listed from cs.CL
LHAW: Controllable Underspecification for Long-Horizon Tasks
George Pu, Michael S. Lee, Udari Madhushani Sehwag et al. (+6 more)
Long-horizon workflow
agents that operate effectively over extended periods are essential for truly autonomous systems. Their reliable execution critically depends on the ability to reason through ambiguous situations in which clarification seeking is necessary to ensure correct task execution. However, progress is limited by the lack of scalable, task-agnostic frameworks for systematically curating and measuring the impact of ambiguity across custom workflows. We address this gap by introducing LHAW (Long-Horizon Augmented Workflows), a modular,
dataset-agnostic synthetic pipeline that transforms any well-specified task into controllable underspecified variants by systematically removing information across four dimensions - Goals, Constraints, Inputs, and Context - at configurable severity levels. Unlike approaches that rely on LLM predictions of ambiguity, LHAW validates variants through empirical
agent trials, classifying them as outcome-critical, divergent, or benign based on observed terminal state divergence. We release 285 task variants from TheAgentCompany, SWE-Bench Pro and MCP-Atlas according to our taxonomy alongside formal analysis measuring how current
agents detect, reason about, and resolve underspecification across ambiguous settings. LHAW provides the first systematic framework for cost-sensitive evaluation of
agent clarification behavior in long-horizon settings, enabling development of reliable autonomous systems.
Read the full abstract ▾
All authors George Pu, Michael S. Lee, Udari Madhushani Sehwag, David J. Lee, Bryan Zhu, Yash Maurya, Mohit Raghavendra, Yuan Xue, Samuel Marc Denton
Machine Learning
Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Models
Jonathan Williams and Esin Tureci
Looped
Language Models (LoopLMs) perform multi-step latent
reasoning prior to
token generation and outperform conventional LLMs on
reasoning benchmarks at smaller parameter budgets. However, attempts to further improve LoopLM
reasoning with reinforcement learning have failed - standard objectives such as Group Relative Policy Optimization (GRPO) only assign credit to the final latent state, creating a fundamental mismatch with the model's internal computation. To resolve this, we introduce RLTT (Reward Latent Thought Trajectories), a reinforcement learning framework which distributes reward across the full latent
reasoning trajectory. RLTT provides dense, trajectory-level credit assignment without relying on external verifiers and can directly replace GRPO with negligible overhead. Across extensive experiments with Ouro-1.4B/2.6B-Thinking under identical training and inference conditions, RLTT yields statistically significant improvements over GRPO on challenging mathematical
reasoning benchmarks, improving mean accuracy over MATH-500, AIME24/26, and BeyondAIME by +5.8% on the 1.4B scale, and +10.9% on the 2.6B scale. Despite being trained exclusively on mathematics, RLTT also transfers effectively to non-mathematical
reasoning benchmarks, demonstrating the effectiveness of trajectory-level credit assignment for reinforcement learning in LoopLMs. Code is available at https://github.com/jonwill8/RLTT.git.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Co-jump: Cooperative Jumping with Quadrupedal Robots via Multi-Agent Reinforcement Learning
Shihao Dong, Yeke Chen, Zeren Luo et al. (+8 more)
While single-
agent legged locomotion has witnessed remarkable progress, individual
robots remain fundamentally constrained by physical actuation limits. To transcend these boundaries, we introduce Co-jump, a cooperative task where two quadrupedal
robots synchronize to execute jumps far beyond their solo capabilities. We tackle the high-impulse contact dynamics of this task under a decentralized setting, achieving synchronization without explicit communication or pre-specified motion primitives. Our framework leverages Multi-
Agent Proximal Policy Optimization (MAPPO) enhanced by a progressive curriculum strategy, which effectively overcomes the sparse-reward exploration challenges inherent in mechanically coupled systems. We demonstrate robust performance in simulation and successful transfer to physical hardware, executing multi-directional jumps onto platforms up to 1.5 m in height. Specifically, one of the
robots achieves a foot-end elevation of 1.1 m, which represents a 144% improvement over the 0.45 m jump height of a standalone quadrupedal
robot, demonstrating superior vertical performance. Notably, this precise coordination is achieved solely through proprioceptive feedback, establishing a foundation for communication-free collaborative locomotion in constrained environments.
Read the full abstract ▾
All authors Shihao Dong, Yeke Chen, Zeren Luo, Jiahui Zhang, Bowen Xu, Jinghan Lin, Yimin Han, Ji Ma, Zhiyou Yu, Yudong Zhao, Peng Lu
Machine Learning
Exponential Sample Complexity Separation between Flat and Hierarchical Agentic Theorem Provers
Sho Sonoda, Shunta Akiyama and Yuya Uezato
Agentic
theorem provers often introduce intermediate lemmas, proof sketches, or subgoal decompositions before returning to tactic-level search. This can look like an expensive detour: if proving lemmas is itself hard, why should a learned prover spend effort there? We give a statistical learning answer. Instead of worst-case proof complexity over all formulas, we study the biased data distribution produced by a teacher prover: initial
theorem states together with successful verified proof traces. We model proof search as a deterministic finite-horizon MDP and analyze offline imitation learning from those traces. The success bounds depend on the average length of teacher proofs, how predictable the teacher's next action is, and how accurately the student learns that local prediction problem. A flat student learns from fully inlined traces, so repeated subproofs appear many times in its training and test-time certificate. A hierarchical student instead predicts a reusable proof DAG and solves each shared block once. When flattening duplicates the same hard local argument exponentially many times, the sufficient-sample certificate produced by our bounds can be exponentially smaller for the hierarchical learner. This gives a concrete statistical mechanism by which reusable proof structure helps verifier-based
theorem proving.
Read the full abstract ▾
Machine Learning · Cross-listed from quant-ph
Privacy-Utility Tradeoffs in Quantum Information Processing
Theshani Nuradha, Sujeet Bhalerao and Felix Leditzky
When sensitive information is encoded in data, it is important to ensure the
privacy of information when attempting to learn useful information from the data. There is a natural tradeoff whereby increasing
privacy requirements may decrease the utility of a learning protocol. In the
quantum setting of differential
privacy, such tradeoffs between
privacy and utility have so far remained largely unexplored. In this work, we study optimal
privacy-utility tradeoffs for both generic and application-specific utility metrics when
privacy is quantified by $(\varepsilon,\delta)$-
quantum local differential
privacy. In the generic setting, we focus on optimizing fidelity and trace distance between the original state and the privatized state. We show that the depolarizing mechanism achieves the optimal utility for given
privacy requirements. We then study the specific application of learning the expectation of an observable with respect to an input state when only given access to privatized states. We derive a lower bound on the number of samples of privatized data required to achieve a fixed accuracy guarantee with high probability. To prove this result, we employ existing lower bounds on private
quantum hypothesis testing, thus showcasing the first operational use of them. We also devise private mechanisms that achieve optimal sample complexity with respect to the
privacy parameters and accuracy parameters, demonstrating that utility can be significantly improved for specific tasks in contrast to the generic setting. In addition, we show that the number of samples required to privately learn observable expectation values scales as $\Theta((\varepsilon \beta)^{-2})$, where $\varepsilon \in (0,1)$ is the
privacy parameter and $\beta$ is the accuracy tolerance. We conclude by initiating the study of private classical shadows, which promise useful applications for private learning tasks.
Read the full abstract ▾
Machine Learning
Learning Structure-Semantic Evolution Trajectories for Graph Domain Adaptation
Wei Chen, Xingyu Guo, Shuang Li et al. (+7 more)
Graph Domain Adaptation (GDA) aims to bridge distribution shifts between domains by transferring knowledge from well-labeled source graphs to given unlabeled target graphs. One promising recent approach addresses graph transfer by discretizing the adaptation process, typically through the construction of intermediate graphs or stepwise
alignment procedures. However, such discrete strategies often fail in real-world scenarios, where graph structures evolve continuously and nonlinearly, making it difficult for fixed-step
alignment to approximate the actual transformation process. To address these limitations, we propose DiffGDA, a
Diffusion-based GDA method that models the domain adaptation process as a continuous-time generative process. We formulate the
evolution from source to target graphs using
stochastic differential equations (SDEs), enabling the joint modeling of structural and semantic transitions. To guide this
evolution, a domain-aware network is introduced to steer the generative process toward the target domain, encouraging the
diffusion trajectory to follow an optimal adaptation path. We theoretically show that the
diffusion process converges to the optimal solution bridging the source and target domains in the latent space. Extensive experiments on 14 graph transfer tasks across 8 real-world
datasets demonstrate DiffGDA consistently outperforms state-of-the-art baselines.
Read the full abstract ▾
All authors Wei Chen, Xingyu Guo, Shuang Li, Yan Zhong, Zhao Zhang, Fuzhen Zhuang, Hongrui Liu, Libang Zhang, Guo Ye, Huimei He
Machine Learning
Enhancing Ride-Hailing Forecasting at DiDi with Multi-View Geospatial Representation Learning from the Web
Xixuan Hao, Guicheng Li, Daiqiang Wu et al. (+6 more)
The proliferation of ride-hailing services has fundamentally transformed urban mobility patterns, making accurate ride-hailing
forecasting crucial for optimizing passenger experience and urban transportation efficiency. However, ride-hailing
forecasting faces significant challenges due to geospatial heterogeneity and high susceptibility to external events. This paper proposes MVGR-Net(Multi-View Geospatial Representation Learning), a novel framework that addresses these challenges through a two-stage approach. In the pretraining stage, we learn comprehensive geospatial representations by integrating Points-of-Interest and temporal mobility patterns to capture regional characteristics from both semantic attribute and temporal mobility pattern views. The
forecasting stage leverages these representations through a
prompt-empowered framework that fine-tunes Large
Language Models while incorporating external events. Extensive experiments on DiDi's real-world
datasets demonstrate the state-of-the-art performance.
Read the full abstract ▾
All authors Xixuan Hao, Guicheng Li, Daiqiang Wu, Xusen Guo, Yumeng Zhu, Zhichao Zou, Peng Zhen, Yao Yao, Yuxuan Liang
Machine Learning
Low-Dimensional Execution Manifolds in Transformer Learning Dynamics: Evidence from Modular Arithmetic Tasks
Yongzhong Xu
We investigate the
geometric structure of learning dynamics in overparameterized
transformer models through carefully controlled modular arithmetic tasks. Our primary finding is that despite operating in high-dimensional parameter spaces ($d=128$),
transformer training trajectories rapidly collapse onto low-dimensional execution
manifolds of dimension $3$–$4$. This dimensional collapse is robust across random seeds and moderate task difficulties, though the orientation of the
manifold in parameter space varies between runs. We demonstrate that this
geometric structure underlies several empirically observed phenomena: (1) sharp
attention concentration emerges as saturation along routing coordinates within the execution
manifold, (2) SGD commutators are preferentially aligned with the execution subspace (up to $10\times$ random baseline) early in training, with $>92\%$ of non-commutativity confined to orthogonal staging directions and this
alignment decreasing as training converges, and (3) sparse autoencoders capture auxiliary routing structure but fail to isolate execution itself, which remains distributed across the low-dimensional
manifold. Our results suggest a unifying
geometric framework for understanding
transformer learning, where the vast majority of parameters serve to absorb optimization interference while core computation occurs in a dramatically reduced subspace. These findings have implications for interpretability, training curriculum design, and understanding the role of overparameterization in neural network learning.
Read the full abstract ▾
Machine Learning
Learning Adaptive Distribution Alignment with Neural Characteristic Function for Graph Domain Adaptation
Wei Chen, Xingyu Guo, Shuang Li et al. (+4 more)
Graph Domain Adaptation (GDA) transfers knowledge from labeled source graphs to unlabeled target graphs but is challenged by complex, multi-faceted distributional shifts. Existing methods attempt to reduce distributional shifts by aligning manually selected graph elements (e.g., node attributes or structural statistics), which typically require manually designed graph filters to extract relevant features before
alignment. However, such approaches are inflexible: they rely on scenario-specific heuristics, and struggle when dominant discrepancies vary across transfer scenarios. To address these limitations, we propose ADAlign, an Adaptive Distribution
Alignment framework for GDA. Unlike heuristic methods, ADAlign requires no manual specification of
alignment criteria. It automatically identifies the most relevant discrepancies in each transfer and aligns them jointly, capturing the interplay between attributes, structures, and their dependencies. This makes ADAlign flexible, scenario-aware, and robust to diverse and dynamically evolving shifts. To enable this adaptivity, we introduce the Neural Spectral Discrepancy (NSD), a theoretically principled parametric distance that provides a unified view of cross-graph shifts. NSD leverages neural characteristic function in the spectral domain to encode feature-structure dependencies of all orders, while a learnable frequency sampler adaptively emphasizes the most informative spectral components for each task via minimax paradigm. Extensive experiments on 10
datasets and 16 transfer tasks show that ADAlign not only outperforms state-of-the-art baselines but also achieves efficiency gains with lower
memory usage and faster training.
Read the full abstract ▾
All authors Wei Chen, Xingyu Guo, Shuang Li, Zhao Zhang, Yan Zhong, Fuzhen Zhuang, Deqing wang
Machine Learning · Cross-listed from cs.GT
Pricing Query Complexity of Multiplicative Revenue Approximation
Wei Tang, Yifan Wang and Mengxiao Zhang
We study the pricing query complexity of revenue maximization for a single buyer whose private valuation is drawn from an unknown distribution. In this setting, the seller must learn the optimal monopoly price by posting prices and observing only binary purchase decisions, rather than the realized valuations. Prior work has established tight query complexity bounds for learning a near-optimal price with additive error $\varepsilon$ when the valuation distribution is supported on $[0,1]$. However, our understanding of how to learn a near-optimal price that achieves at least a $(1-\varepsilon)$ fraction of the optimal revenue remains limited. In this paper, we study the pricing query complexity of the single-buyer revenue maximization problem under such multiplicative error guarantees in several settings. Observe that when pricing queries are the only source of information about the buyer's distribution, no
algorithm can achieve a non-trivial approximation, since the scale of the distribution cannot be learned from pricing queries alone. Motivated by this fundamental impossibility, we consider two natural and well-motivated models that provide "scale hints": (i) a one-sample hint, in which the
algorithm observes a single realized valuation before making pricing queries; and (ii) a value-range hint, in which the valuation support is known to lie within $[1, H]$. For each type of hint, we establish pricing query complexity guarantees that are tight up to polylogarithmic factors for several classes of distributions, including monotone hazard rate (MHR) distributions, regular distributions, and general distributions.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
GPU-Fuzz: Finding Memory Errors in Deep Learning Frameworks
Zihao Li, Hongyi Lu, Yanan Guo et al. (+3 more)
GPU
memory errors are a critical threat to deep learning (DL) frameworks, leading to crashes or even security issues. We introduce GPU-Fuzz, a fuzzer locating these issues efficiently by modeling operator parameters as formal constraints. GPU-Fuzz utilizes a constraint solver to generate test cases that systematically probe error-prone boundary conditions in GPU kernels. Applied to PyTorch, TensorFlow, and PaddlePaddle, we uncovered 13 unknown bugs, demonstrating the effectiveness of GPU-Fuzz in finding
memory errors.
Read the full abstract ▾
All authors Zihao Li, Hongyi Lu, Yanan Guo, Zhenkai Zhang, Shuai Wang, Fengwei Zhang
Machine Learning
Driving Reaction Trajectories via Latent Flow Matching
Yili Shen and Xiangliang Zhang
Recent advances in reaction prediction have achieved near-saturated accuracy on standard
benchmarks (e.g., USPTO), yet most state-of-the-art models formulate the task as a one-shot mapping from reactants to products, offering limited insight into the underlying reaction process. Procedural alternatives introduce stepwise generation but often rely on mechanism-specific supervision, discrete symbolic edits, and computationally expensive inference. In this work, we propose LatentRxnFlow, a new reaction prediction paradigm that models reactions as continuous latent trajectories anchored at the thermodynamic product state. Built on Conditional Flow Matching, our approach learns time-dependent latent dynamics directly from standard reactant-product pairs, without requiring mechanistic annotations or curated intermediate labels. While LatentRxnFlow achieves state-of-the-art performance on USPTO
benchmarks, more importantly, the continuous formulation exposes the full generative trajectory, enabling trajectory-level diagnostics that are difficult to realize with discrete or one-shot models. We show that latent trajectory analysis allows us to localize and characterize failure modes and to mitigate certain errors via gated inference. Furthermore,
geometric properties of the learned trajectories provide an intrinsic signal of epistemic uncertainty, helping prioritize reliably predictable reaction outcomes and flag ambiguous cases for additional validation. Overall, LatentRxnFlow combines strong predictive accuracy with improved transparency, diagnosability, and uncertainty awareness, moving reaction prediction toward more trustworthy deployment in high-throughput discovery workflows.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.GT
Online Generalized-Mean Welfare Maximization: Achieving Near-Optimal Regret from Samples
Zongjun Yang, Rachitesh Kumar and Christian Kroer
We study online fair allocation of $T$ sequentially arriving items among $n$
agents with heterogeneous preferences, with the objective of maximizing generalized-mean welfare, defined as the $p$-mean of
agents' time-averaged utilities, with $p\in (-\infty, 1)$. We first consider the i.i.d. arrival model and show that the pure greedy
algorithm – which myopically chooses the welfare-maximizing integral allocation – achieves $\widetilde{O}(1/T)$ average regret. Importantly, in contrast to prior work, our
algorithm does not require distributional knowledge and achieves the optimal regret rate using only the online samples. We then go beyond i.i.d. arrivals and investigate a nonstationary model with time-varying independent distributions. In the absence of additional data about the distributions, it is known that every online
algorithm must suffer $\Omega(1)$ average regret. We show that only a single historical sample from each distribution is sufficient to recover the optimal $\widetilde{O}(1/T)$ average regret rate, even in the face of arbitrary non-stationarity. Our
algorithms are based on the re-solving paradigm: they assume that the remaining items will be the ones seen historically in those periods and solve the resulting welfare-maximization problem to determine the decision in every period. Finally, we also account for distribution shifts that may distort the fidelity of historical samples and show that the performance of our re-solving
algorithms is robust to such shifts.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Unlocked Backpropagation using Wave Scattering
Christian Pehle and Jean-Jacques Slotine
Both the backpropagation
algorithm in machine learning and the maximum principle in optimal control theory are posed as a two-point boundary problem, resulting in a "forward-backward" lock. We derive a reformulation of the maximum principle in optimal control theory as a
hyperbolic initial value problem by introducing an additional "optimization time" dimension. We introduce counter-propagating wave variables with finite propagation speed and recast the optimization problem in terms of scattering relationships between them. This relaxation of the original problem can be interpreted as a physical system that equilibrates and changes its physical properties in order to minimize reflections. We discretize this continuum theory to derive a family of fully unlocked
algorithms suitable for training neural networks. Different parameter dynamics, including gradient descent, can be derived by demanding dissipation and minimization of reflections at parameter ports. These results also imply that any physical substrate that supports the scattering and dissipation of waves can be interpreted as solving an optimization problem.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Found-RL: foundation model-enhanced reinforcement learning for autonomous driving
Yansong Qu, Zihao Sheng, Zilin Huang et al. (+6 more)
Reinforcement Learning (RL) has emerged as a dominant paradigm for end-to-end autonomous driving (AD). However, RL suffers from sample inefficiency and a lack of semantic interpretability in complex scenarios. Foundation Models, particularly
Vision-
Language Models (VLMs), can mitigate this by offering rich, context-aware knowledge, yet their high inference latency hinders deployment in high-frequency RL training loops. To bridge this gap, we present Found-RL, a platform tailored to efficiently enhance RL for AD using foundation models. A core innovation is the asynchronous batch inference framework, which decouples heavy VLM
reasoning from the simulation loop, effectively resolving latency bottlenecks to support real-time learning. We introduce diverse supervision mechanisms: Value-Margin Regularization (VMR) and Advantage-Weighted Action Guidance (AWAG) to effectively distill expert-like VLM action suggestions into the RL policy. Additionally, we adopt high-throughput CLIP for dense reward shaping. We address CLIP's dynamic blindness via Conditional Contrastive Action
Alignment, which conditions
prompts on discretized speed/command and yields a normalized, margin-based bonus from context-specific action-anchor scoring. Found-RL provides an end-to-end pipeline for fine-tuned VLM integration and shows that a lightweight RL model can achieve near-VLM performance compared with billion-parameter VLMs while sustaining real-time inference (approx. 500 FPS). Code, data, and models will be publicly available at https://github.com/ys-qu/found-rl.
Read the full abstract ▾
All authors Yansong Qu, Zihao Sheng, Zilin Huang, Jiancong Chen, Yuhao Luo, Tianyi Wang, Yiheng Feng, Samuel Labi, Sikai Chen
Machine Learning
Analyzing Fairness of Neural Network Prediction via Counterfactual Dataset Generation
Brian Hyeongseok Kim, Jacqueline L. Mitchell and Chao Wang
Interpreting the inference-time behavior of deep neural networks remains a challenging problem. Existing approaches to counterfactual explanation typically ask: What is the closest alternative input that would alter the model's prediction in a desired way? In contrast, we explore counterfactual
datasets. Rather than perturbing the input, our method efficiently finds the closest alternative training
dataset, one that differs from the original
dataset by changing a few labels. Training a new model on this altered
dataset can then lead to a different prediction of a given test instance. This perspective provides a new way to assess
fairness by directly analyzing the influence of label bias on training and inference. Our approach can be characterized as probing whether a given prediction depends on biased labels. Since exhaustively enumerating all possible alternate
datasets is infeasible, we develop analysis techniques that trace how bias in the training data may propagate through the learning
algorithm to the trained network. Our method heuristically ranks and modifies the labels of a bounded number of training examples to construct a counterfactual
dataset, retrains the model, and checks whether its prediction on a chosen test case changes. We evaluate our approach on feedforward neural networks across over 1100 test cases from 7 widely-used
fairness datasets. Results show that it modifies only a small subset of training labels, highlighting its ability to pinpoint the critical training examples that drive prediction changes. Finally, we demonstrate how our counterfactual
datasets reveal connections between training examples and test cases, offering an interpretable way to probe
dataset bias.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
Compute Only Once: UG-Separation for Efficient Large Recommendation Models
Hui Lu, Zheng Chai, Shipeng Bai et al. (+15 more)
Driven by scaling laws, recommender systems increasingly rely on larger-scale models to capture complex feature interactions and user behaviors, but this trend also leads to prohibitive training and inference costs. While long-sequence models can reuse user-side computation through KV Caching, such reuse is difficult in TokenMixer-based dense feature interaction architectures, where user and group features are deeply entangled and mixed-up across layers. In this work, we present User-Group Separation (UG-Sep), an industrial large-scale framework that enables user-side computation reusable in TokenMixer-based dense interaction models for the first time. UG-Sep explicitly disentangles user-side and item-side information flows within
token-mixing layers, ensuring that a subset of
tokens preserves purely user-side representations across layers. This design allows the corresponding per-
token computations to be reused across multiple samples, significantly reducing redundant inference cost. To compensate for the potential expressive capacity loss induced by masking, we further propose an Information Compensation strategy that adaptively reconstructs suppressed user-item interactions. Moreover, as UG-Sep substantially reduces user-side FLOPs and exposes
memory-bound components, we incorporate W8A16 (8-bit weight, 16-bit activation) weight-only quantization to alleviate
memory bandwidth bottlenecks and achieve additional acceleration. We conduct extensive offline evaluations and large-scale online A/B experiments at ByteDance to validate the effectiveness of UG-Sep. Results show that UG-Sep reduces inference latency by up to 20% without causing adverse changes to online user experience and commercial metrics on multiple influential business scenarios compared to TokenMixer at ByteDance, including Douyin Feed Recommendation, Hongguo Feed Recommendation, Chuanshanjia Ads, and Qianchuan Ads.
Read the full abstract ▾
All authors Hui Lu, Zheng Chai, Shipeng Bai, Hao Zhang, Zhifang Fan, Kunmin Bai, Ke Sun, Yingwen Wu, Bingzheng Wei, Xiang Sun, Ziyan Gong, Tianyi Liu, Hua Chen, Deping Xie, Zhongkai Chen, Zhiliang Guo, Qiwei Chen, Yuchao Zheng
Machine Learning · Cross-listed from math.OC
Distributed Online Convex Optimization with Nonseparable Costs and Constraints
Zhaoye Pan, Haozhe Lei, Fan Zuo et al. (+2 more)
This paper studies distributed online convex optimization with time-varying coupled constraints, motivated by distributed online control in network systems. Most prior work assumes a separability condition: the global objective and coupled constraint functions are sums of local costs and individual constraints. In contrast, we study a group of
agents, networked via a communication graph, that collectively select actions to minimize a sequence of nonseparable global cost functions and to satisfy nonseparable long-term constraints based on full-information feedback and intra-
agent communication. We propose a distributed online primal-dual belief consensus
algorithm, where each
agent maintains and updates a local belief of the global collective decisions, which are repeatedly exchanged with neighboring
agents. Unlike the previous consensus primal-dual
algorithms under separability that ask
agents to only communicate their local decisions, our belief-sharing protocol eliminates coupling between the primal consensus disagreement and the dual constraint violation, yielding sublinear regret and cumulative constraint violation (CCV) bounds, both in $O({T}^{1/2})$, where $T$ denotes the time horizon. Such a result breaks the long-standing $O(T^{3/4})$ barrier for CCV and matches the lower bound of online constrained convex optimization, indicating the online learning efficiency at the cost of communication overhead.
Read the full abstract ▾
All authors Zhaoye Pan, Haozhe Lei, Fan Zuo, Zilin Bian, Tao Li
Machine Learning
A Multimodal Conditional Mixture Model with Distribution-Level Physics Priors
Jinkyo Han and Bahador Bahmani
Many scientific and engineering systems exhibit intrinsically
multimodal behavior arising from latent regime switching and non-unique physical mechanisms. In such settings, learning the full conditional distribution of admissible outcomes in a physically consistent and interpretable manner remains a challenge. While recent advances in machine learning have enabled powerful
multimodal generative modeling, their integration with physics-constrained scientific modeling remains nontrivial, particularly when physical structure must be preserved or data are limited. This work develops a physics-informed
multimodal conditional modeling framework based on mixture density representations. Mixture density networks (MDNs) provide an explicit and interpretable parameterization of
multimodal conditional distributions. Physical knowledge is embedded through component-specific regularization terms that penalize violations of governing equations or physical laws. This formulation naturally accommodates non-uniqueness and stochasticity while remaining computationally efficient and amenable to conditioning on contextual inputs. The proposed framework is evaluated across a range of scientific problems in which multimodality arises from intrinsic physical mechanisms rather than observational noise, including bifurcation phenomena in nonlinear dynamical systems,
stochastic partial differential equations, and atomistic-scale shock dynamics. In addition, the proposed method is compared with a conditional flow matching (CFM) model, a representative state-of-the-art generative modeling approach, demonstrating that MDNs can achieve competitive performance while offering a simpler and more interpretable formulation.
Read the full abstract ▾
Machine Learning
Constructing Industrial-Scale Optimization Modeling Benchmark
Zhong Li, Hongliang Lu, Tao Wei et al. (+5 more)
Optimization modeling underpins decision-making in logistics, manufacturing, energy, and finance, yet translating natural-
language requirements into correct optimization formulations and solver-executable code remains labor-intensive. Although large
language models (LLMs) have been explored for this task, evaluation is still dominated by toy-sized or synthetic
benchmarks, masking the difficulty of industrial problems with $10^{3}$–$10^{6}$ (or more) variables and constraints. A key bottleneck is the lack of
benchmarks that align natural-
language specifications with reference formulations/solver code grounded in real optimization models. To fill in this gap, we introduce MIPLIB-NL, built via a structure-aware reverse construction methodology from real mixed-integer linear programs in MIPLIB~2017. Our pipeline (i) recovers compact, reusable model structure from flat solver formulations, (ii) reverse-generates natural-
language specifications explicitly tied to this recovered structure under a unified model–data separation format, and (iii) performs iterative semantic validation through expert review and human–LLM interaction with independent reconstruction checks. This yields 223 one-to-one reconstructions that preserve the mathematical content of the original instances while enabling realistic natural-
language-to-optimization evaluation. Experiments show substantial performance degradation on MIPLIB-NL for systems that perform strongly on existing
benchmarks, exposing failure modes invisible at toy scale.
Read the full abstract ▾
All authors Zhong Li, Hongliang Lu, Tao Wei, Yuxuan Chen, Wenyu Liu, Yuan Lan, Fan Zhang, Zaiwen Wen
Machine Learning
A Unified Theory of Random Projection for Influence Functions
Pingbang Hu, Yuzheng Hu, Jiaqi W. Ma et al. (+1 more)
Influence functions and related data attribution scores take the form of $g^{\top}F^{-1}g^{\prime}$, where $F\succeq 0$ is a
curvature operator. In modern overparametrized models, forming or inverting $F\in\mathbb{R}^{d\times d}$ is prohibitive, motivating scalable influence computation via random projection with a sketch $P \in \mathbb{R}^{m\times d}$. This practice is commonly justified via the Johnson–Lindenstrauss (JL) lemma, which ensures approximate preservation of Euclidean geometry for a fixed
dataset. However, JL does not address how sketching behaves under inversion. Furthermore, there is no existing theory that explains how sketching interacts with other widely-used techniques, such as ridge regularization and structured
curvature approximations. We develop a unified theory characterizing when projection provably preserves influence functions. When $g,g^{\prime}\in\text{range}(F)$, we show that: 1) Unregularized projection: exact preservation holds iff $P$ is injective on $\text{range}(F)$, which necessitates $m\geq \text{rank}(F)$; 2) Regularized projection: ridge regularization fundamentally alters the sketching barrier, with approximation guarantees governed by the effective dimension of $F$ at the regularization scale; 3) Factorized influence: for Kronecker-factored
curvatures $F=A\otimes E$, the guarantees continue to hold for decoupled sketches $P=P_A\otimes P_E$, even though such sketches exhibit row correlations that violate i.i.d. assumptions. Beyond this range-restricted setting, we analyze out-of-range test gradients and quantify a leakage term that arises when test gradients have components in $\ker(F)$. This yields guarantees for influence queries on general test points. Overall, this work develops a novel theory that characterizes when projection provably preserves influence and provides principled guidance for choosing the sketch size in practice.
Read the full abstract ▾
All authors Pingbang Hu, Yuzheng Hu, Jiaqi W. Ma, Han Zhao
Machine Learning · Cross-listed from cs.IR
End-to-End Semantic ID Generation for Generative Advertisement Recommendation
Jie Jiang, Xinxun Zhang, Enming Zhang et al. (+8 more)
Generative Recommendation (GR) has excelled by framing recommendation as next-
token prediction. This paradigm relies on Semantic IDs (SIDs) to tokenize large-scale items into discrete sequences. Existing GR approaches predominantly generate SIDs via Residual Quantization (RQ), where items are encoded into
embeddings and then quantized to discrete SIDs. However, this paradigm suffers from inherent limitations: 1) Objective misalignment and semantic degradation stemming from the two-stage compression; 2) Error accumulation inherent in the structure of RQ. To address these limitations, we propose UniSID, a Unified SID generation framework for generative advertisement recommendation. Specifically, we jointly optimize
embeddings and SIDs in an end-to-end manner from raw advertising data, enabling semantic information to flow directly into the SID space and thus addressing the inherent limitations of the two-stage cascading compression paradigm. To capture fine-grained semantics, a multi-granularity contrastive learning strategy is introduced to align distinct items across SID levels. Finally, a summary-based ad reconstruction mechanism is proposed to encourage SIDs to capture high-level semantic information that is not explicitly present in advertising contexts. Experiments demonstrate that UniSID consistently outperforms state-of-the-art SID generation methods, yielding up to a 4.62% improvement in Hit Rate metrics across downstream advertising scenarios compared to the strongest baseline.
Read the full abstract ▾
All authors Jie Jiang, Xinxun Zhang, Enming Zhang, Yuling Xiong, Jun Zhang, Jingwen Wang, Huan Yu, Yuxiang Wang, Hao Wang, Xiao Yan, Jiawei Jiang
Machine Learning
Chamfer-Linkage for Hierarchical Agglomerative Clustering
Kishen N Gowda, Willem Fletcher, MohammadHossein Bateni et al. (+4 more)
Hierarchical Agglomerative Clustering (HAC) is a widely-used clustering method based on repeatedly merging the closest pair of clusters, where inter-cluster distances are determined by a linkage function. Unlike many clustering methods, HAC does not optimize a single explicit global objective; clustering quality is therefore primarily evaluated empirically, and the choice of linkage function plays a crucial role in practice. However, popular classical linkages, such as single-linkage, average-linkage and Ward's method show high variability across real-world
datasets and do not consistently produce high-quality clusterings in practice. In this paper, we propose Chamfer-linkage, a novel linkage function that measures the distance between clusters using the Chamfer distance, a popular notion of distance between point-clouds in machine learning and computer
vision. We argue that Chamfer-linkage satisfies desirable concept representation properties that other popular measures struggle to satisfy. Theoretically, we show that Chamfer-linkage HAC can be implemented in $O(n^2)$ time, matching the efficiency of classical linkage functions. Experimentally, we find that Chamfer-linkage consistently yields higher-quality clusterings than classical linkages such as average-linkage and Ward's method across a diverse collection of
datasets. Our results establish Chamfer-linkage as a practical drop-in replacement for classical linkage functions, broadening the toolkit for hierarchical clustering in both theory and practice.
Read the full abstract ▾
All authors Kishen N Gowda, Willem Fletcher, MohammadHossein Bateni, Laxman Dhulipala, D Ellis Hershkowitz, Rajesh Jayaram, Jakub Łącki
Machine Learning
LakeMLB: Data Lake Machine Learning Benchmark
Feiyu Pan, Tianbin Zhang, Aoqian Zhang et al. (+5 more)
Data lakes have become a fundamental platform for large-scale machine learning by enabling flexible management of heterogeneous data. Despite their growing importance, standardized
benchmarks for evaluating machine learning performance in data lake environments remain scarce. To address this gap, we present LakeMLB (Data Lake Machine Learning
Benchmark), the first
benchmark designed for multi-table machine learning in data lakes. LakeMLB focuses on two representative scenarios, Union and Join, and provides six real-world
datasets spanning diverse domains. It supports three representative multi-table learning paradigms: pre-training, data augmentation, and feature augmentation, together with standardized data splits and evaluation protocols. We conduct extensive experiments with state-of-the-art tabular learning methods and provide insights into their performance across different data lake scenarios. We release both
datasets and code to facilitate rigorous research on machine learning in data lake ecosystems; the
benchmark is available at https://github.com/zhengwang100/LakeMLB.
Read the full abstract ▾
All authors Feiyu Pan, Tianbin Zhang, Aoqian Zhang, Yu Sun, Zheng Wang, Lixing Chen, Li Pan, Jianhua Li
Machine Learning
Control Reinforcement Learning: Interpretable Token-Level Steering of LLMs via Sparse Autoencoder Features
Seonglae Cho, Zekun Wu and Adriano Koshiyama
Sparse autoencoders (SAEs) decompose
language model activations into interpretable features, but existing methods reveal only which features activate, not which change model outputs when amplified. We introduce Control Reinforcement Learning (CRL), which trains a policy to select SAE features for steering at each
token, producing interpretable intervention logs: the learned policy identifies features that change model outputs when amplified. Adaptive Feature Masking encourages diverse feature discovery while preserving singlefeature interpretability. The framework yields new analysis capabilities: branch point tracking locates
tokens where feature choice determines output correctness; critic trajectory analysis separates policy limitations from value estimation errors; layer-wise comparison reveals syntactic features in early layers and semantic features in later layers. On Gemma 2 2B across MMLU, BBQ, GSM8K, HarmBench, and XSTest, CRL achieves improvements while providing per-
token intervention logs. These results establish learned feature steering as a mechanistic interpretability tool that complements static feature analysis with dynamic intervention probes
Read the full abstract ▾
Machine Learning
A Dual-Stream Physics-Augmented Unsupervised Architecture for Runtime Embedded Vehicle Health Monitoring
Enzo Nicolas Spotorno and Antonio Augusto Medeiros Frohlich
Runtime quantification of vehicle operational intensity is essential for predictive maintenance and condition monitoring in commercial and heavy-duty fleets. Traditional metrics like mileage fail to capture mechanical burden, while unsupervised deep learning models detect statistical anomalies, typically transient surface shocks, but often conflate statistical stability with mechanical rest. We identify this as a critical blind spot: high-load steady states, such as hill climbing with heavy payloads, appear statistically normal yet impose significant drivetrain fatigue. To resolve this, we propose a Dual-Stream Architecture that fuses unsupervised learning for surface anomaly detection with macroscopic physics proxies for cumulative load estimation. This approach leverages low-frequency sensor data to generate a multi-dimensional health vector, distinguishing between dynamic hazards and sustained mechanical effort. Validated on a RISC-V embedded platform, the architecture demonstrates low computational overhead, enabling comprehensive, edge-based health monitoring on resource-constrained ECUs without the latency or bandwidth costs of cloud-based monitoring.
Read the full abstract ▾
Machine Learning
QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs
Kanghyun Noh, Jinheon Choi and Yulhwa Kim
Large
language models (LLMs) demand substantial computational and
memory resources, posing challenges for efficient deployment. Two complementary approaches have emerged to address these issues:
token-adaptive layer execution, which reduces floating-point operations (FLOPs) by selectively bypassing layers, and quantization, which lowers
memory footprint by reducing weight precision. However, naively integrating these techniques leads to additional accuracy degradation due to reduced redundancy in
token-adaptive models. We propose QTALE (Quantization-Robust
Token-Adaptive Layer Execution for LLMs), a novel framework that enables seamless integration of
token-adaptive execution with quantization while preserving accuracy. Conventional
token-adaptive methods reduce redundancy in two ways: (1) by limiting the diversity of training paths explored during fine-tuning, and (2) by lowering the number of parameters actively involved in inference. To overcome these limitations, QTALE introduces two key components: (1) a training strategy that ensures diverse execution paths are actively explored during fine-tuning, and (2) a post-training mechanism that allows flexible adjustment of the execution ratio at inference to reintroduce redundancy when needed. Experimental results show that QTALE enables seamless integration of
token-adaptive layer execution with quantization, while keeping the accuracy gap to quantization-only models below 0.5% on CommonsenseQA
benchmarks. By combining tokenadaptive execution for FLOPs reduction and quantization for
memory savings, QTALE provides an effective solution for efficient LLM deployment.
Read the full abstract ▾
Machine Learning
Breaking the Curse of Repulsion: Remoteness-Aware Control of Negative Off-Policy Updates
Yusen Huo, Changping Wang, Yangru Huang et al. (+2 more)
Off-policy policy optimization reuses historical behavior, including negative-advantage samples that suppress known failures. We show that repeated reuse can turn this useful signal into excessive repulsion: as the learner moves away from a historical negative action, subsequent updates make that action increasingly remote without necessarily reducing its update strength. Our aggregate theory characterizes the resulting transition from a stable displacement beyond the positive-only target to persistent drift and the loss of finite stable
equilibria; controlled strength sweeps show that an intermediate displacement can improve held-out reward. The relevant learner-relative coordinate is squared standardized distance for Gaussian policies and surprisal for categorical policies. We introduce Dynamic Remoteness-Aware Policy Optimization (DRPO), which leaves the negative update unchanged in the near field and exponentially attenuates its remote tail. DRPO restores eventual inward Gaussian drift for every fixed finite negative-to-positive mass ratio, yields an explicit ultimate-bound radius, and changes categorical support suppression from exponential to
polynomial probability decay. External diagnostics and controlled interventions isolate remoteness-dependent policy geometry as a source of negative-update amplification and show that selective tapering can remove its far-field effect without discarding useful local feedback.
Read the full abstract ▾
All authors Yusen Huo, Changping Wang, Yangru Huang, Jun Zhang, Jie Jiang