Machine Learning · Cross-listed from cs.CL
Temp-R1: A Unified Autonomous Agent for Complex Temporal KGQA via Reverse Curriculum Reinforcement Learning
Zhaoyan Gong, Zhiqiang Liu, Songze Li et al. (+7 more)
Temporal Knowledge Graph Question Answering (TKGQA) is inherently challenging, as it requires sophisticated
reasoning over dynamic facts with multi-hop dependencies and complex temporal constraints. Existing methods rely on fixed workflows and expensive closed-source APIs, limiting flexibility and scalability. We propose Temp-R1, the first autonomous end-to-end
agent for TKGQA trained through reinforcement learning. To address cognitive overload in single-action
reasoning, we expand the action space with specialized internal actions alongside external action. To prevent shortcut learning on simple questions, we introduce reverse curriculum learning that trains on difficult questions first, forcing the development of sophisticated
reasoning before transferring to easier cases. Our 8B-parameter Temp-R1 achieves state-of-the-art performance on MultiTQ and TimelineKGQA, improving 19.8% over strong baselines on complex questions. Our work establishes a new paradigm for autonomous temporal
reasoning agents. The code is available at https://github.com/zjukg/Temp-R1.
Read the full abstract ▾
All authors Zhaoyan Gong, Zhiqiang Liu, Songze Li, Xiaoke Guo, Yuanxiang Liu, Xinle Deng, Zhizhen Liu, Lei Liang, Huajun Chen, Wen Zhang
Machine Learning
TriPlay-RL: Tri-Role Self-Play Reinforcement Learning for LLM Safety Alignment
Zhewen Tan, Wenhan Yu, Jianfeng Si et al. (+9 more)
In recent years, safety risks associated with large
language models have become increasingly prominent, highlighting the urgent need to mitigate the generation of toxic and harmful content. The mainstream paradigm for LLM safety
alignment typically adopts a collaborative framework involving three roles: an attacker for
adversarial prompt generation, a defender for safety defense, and an evaluator for response assessment. In this paper, we propose a closed-loop reinforcement learning framework called TriPlay-RL that enables iterative and co-improving collaboration among three roles with near-zero manual annotation. Experimental results show that the attacker preserves high output diversity while achieving a 20%-50% improvement in
adversarial effectiveness; the defender attains 10%-30% gains in safety performance without degrading general
reasoning capability; and the evaluator continuously refines its fine-grained judgment ability through iterations, accurately distinguishing unsafe responses, simple refusals, and useful guidance. Overall, our framework establishes an efficient and scalable paradigm for LLM safety
alignment, enabling continuous co-
evolution within a unified learning loop.
Read the full abstract ▾
All authors Zhewen Tan, Wenhan Yu, Jianfeng Si, Tongxin Liu, Kaiqi Guan, Huiyan Jin, Jiawen Tao, Xiaokun Yuan, Duohe Ma, Xiangzheng Zhang, Tong Yang, Lin Sun
Machine Learning · Cross-listed from cs.AI
Think-Augmented Function Calling: Improving LLM Parameter Accuracy Through Embedded Reasoning
Lei Wei, Xiao Peng, Jinpeng Ou et al. (+1 more)
Large
language models (LLMs) have demonstrated remarkable capabilities in function calling for autonomous
agents, yet current mechanisms lack explicit
reasoning transparency during parameter generation, particularly for complex functions with interdependent parameters. While existing approaches like chain-of-thought prompting operate at the
agent level, they fail to provide fine-grained
reasoning guidance for individual function parameters. To address these limitations, we propose Think-Augmented Function Calling (TAFC), a novel framework that enhances function calling accuracy through explicit
reasoning at both function and parameter levels. Our method introduces a universal "think" parameter augmentation that enables models to articulate their decision-making process, with dynamic optimization for parameter descriptions to improve
reasoning quality. For complex parameters, TAFC automatically triggers granular
reasoning based on complexity scoring, ensuring appropriate justification for critical decisions. Additionally, we propose
reasoning-guided optimization to align generated
reasoning with human expectations. TAFC requires no architectural modifications to existing LLMs while maintaining full API compatibility. Evaluation on ToolBench across proprietary and open-source models demonstrates significant improvements in parameter generation accuracy and
reasoning coherence for multi-parameter functions, while providing enhanced interpretability for debugging AI
agent behaviors.
Read the full abstract ▾
All authors Lei Wei, Xiao Peng, Jinpeng Ou, Bin Wang
Machine Learning
What Do Learned Models Measure?
Indrė Žliobaitė
In many scientific and data-driven applications, machine learning models are increasingly used as measurement instruments, rather than merely as predictors of predefined labels. When the measurement function is learned from data, the mapping from observations to quantities is determined implicitly by the training distribution and inductive biases, allowing multiple inequivalent mappings to satisfy standard predictive evaluation criteria. We formalize learned measurement functions as a distinct focus of evaluation and introduce measurement stability, a property capturing invariance of the measured quantity across admissible realizations of the learning process and across contexts. We show that standard evaluation criteria in machine learning, including generalization error, calibration, and robustness, do not guarantee measurement stability. Through a real-world case study, we show that models with comparable predictive performance can implement systematically inequivalent measurement functions, with distribution shift providing a concrete illustration of this failure. Taken together, our results highlight a limitation of existing evaluation frameworks in settings where learned model outputs are identified as measurements, motivating the need for an additional evaluative dimension.
Read the full abstract ▾
Machine Learning · Cross-listed from cond-mat.str-el
Toward Scalable Normalizing Flows for the Hubbard Model
Janik Kreit, Andrea Bulgarelli, Lena Funcke et al. (+4 more)
Normalizing flows have recently demonstrated the ability to learn the Boltzmann distribution of the Hubbard model, opening new avenues for generative modeling in condensed matter physics. In this work, we investigate the steps required to extend such simulations to larger
lattice sizes and lower temperatures, with a focus on enhancing stability and efficiency. Additionally, we present the scaling behavior of
stochastic normalizing flows and non-
equilibrium Markov chain Monte Carlo methods for this fermionic system.
Read the full abstract ▾
All authors Janik Kreit, Andrea Bulgarelli, Lena Funcke, Thomas Luu, Dominic Schuh, Simran Singh, Lorenzo Verzichelli
Machine Learning
Neural Network Approximation: A View from Polytope Decomposition
ZeYu Li, ShiJun Zhang, TieYong Zeng et al. (+1 more)
Universal approximation theory offers a foundational framework to verify neural network expressiveness, enabling principled utilization in real-world applications. However, most existing theoretical constructions are established by uniformly dividing the input space into tiny hypercubes without considering the local regularity of the target function. In this work, we investigate the universal approximation capabilities of ReLU networks from a view of polytope decomposition, which offers a more realistic and task-oriented approach compared to current methods. To achieve this, we develop an explicit kernel
polynomial method to derive an universal approximation of continuous functions, which is characterized not only by the refined Totik-Ditzian-type modulus of continuity, but also by polytopical domain decomposition. Then, a ReLU network is constructed to approximate the kernel
polynomial in each subdomain separately. Furthermore, we find that polytope decomposition makes our approximation more efficient and flexible than existing methods in many cases, especially near singular points of the objective function. Lastly, we extend our approach to analytic functions to reach a higher approximation rate.
Read the full abstract ▾
All authors ZeYu Li, ShiJun Zhang, TieYong Zeng, FengLei Fan
Machine Learning
FGGM: Fisher-Guided Gradient Masking for Continual Learning
Chao-Hong Tan, Qian Chen, Wen Wang et al. (+6 more)
Catastrophic forgetting impairs the continuous learning of large
language models. We propose Fisher-Guided Gradient Masking (FGGM), a framework that mitigates this by strategically selecting parameters for updates using diagonal Fisher Information. FGGM dynamically generates binary masks with adaptive thresholds, preserving critical parameters to balance stability and plasticity without requiring historical data. Unlike magnitude-based methods such as MIGU, our approach offers a mathematically principled parameter importance estimation. On the TRACE
benchmark, FGGM shows a 9.6% relative improvement in retaining general capabilities over supervised fine-tuning (SFT) and a 4.4% improvement over MIGU on TRACE tasks. Additional analysis on code generation tasks confirms FGGM's superior performance and reduced forgetting, establishing it as an effective solution.
Read the full abstract ▾
All authors Chao-Hong Tan, Qian Chen, Wen Wang, Yukun Ma, Chong Zhang, Chong Deng, Qinglin Zhang, Xiangang Li, Jieping Ye
Machine Learning
Beyond Retention: Orchestrating Structural Safety and Plasticity in Continual Learning for LLMs
Fei Meng
Continual learning in Large
Language Models (LLMs) faces the critical challenge of balancing stability (retaining old knowledge) and plasticity (learning new tasks). While Experience Replay (ER) is a standard countermeasure against catastrophic forgetting, its impact across diverse capabilities remains underexplored. In this work, we uncover a critical dichotomy in ER's behavior: while it induces positive backward transfer on robust, unstructured tasks (e.g., boosting performance on previous NLP classification tasks through repeated rehearsal), it causes severe negative transfer on fragile, structured domains like code generation (e.g., a significant relative drop in coding accuracy). This reveals that ER trades structural integrity for broad consolidation. To address this dilemma, we propose Orthogonal Subspace Wake-up (OSW). OSW identifies essential parameter subspaces of previous tasks via a brief "wake-up" phase and enforces orthogonal updates for new tasks, providing a mathematically grounded "safety guarantee" for established knowledge structures. Empirical results across a diverse four-task sequence demonstrate that OSW uniquely succeeds in preserving fragile coding abilities where Replay fails, while simultaneously maintaining high plasticity for novel tasks. Our findings emphasize the necessity of evaluating structural safety alongside average retention in LLM continual learning.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Co-PLNet: A Collaborative Point-Line Network for Prompt-Guided Wireframe Parsing
Chao Wang, Xuanying Li, Cheng Dai et al. (+4 more)
Wireframe parsing aims to recover line segments and their junctions to form a structured
geometric representation useful for downstream tasks such as Simultaneous Localization and Mapping (SLAM). Existing methods predict lines and junctions separately and reconcile them post-hoc, causing mismatches and reduced robustness. We present Co-PLNet, a point-line collaborative framework that exchanges spatial cues between the two tasks, where early detections are converted into spatial
prompts via a Point-Line
Prompt Encoder (PLP-Encoder), which encodes
geometric attributes into compact and spatially aligned maps. A Cross-Guidance Line Decoder (CGL-Decoder) then refines predictions with sparse
attention conditioned on complementary
prompts, enforcing point-line consistency and efficiency. Experiments on Wireframe and YorkUrban show consistent improvements in accuracy and robustness, together with favorable real-time efficiency, demonstrating our effectiveness for structured geometry perception. Our code is available at https://github.com/GalacticHogrider/Co-PLNet.
Read the full abstract ▾
All authors Chao Wang, Xuanying Li, Cheng Dai, Jinglei Feng, Yuxiang Luo, Hao Qin, Yuqi Ouyang
Machine Learning
Tractable Gaussian Phase Retrieval with Heavy Tails and Adversarial Corruption with Near-Linear Sample Complexity
Santanu Das and Jatin Batra
Phase
retrieval is the classical problem of recovering a signal $x^* \in \mathbb{R}^n$ from its noisy phaseless measurements $y_i = \langle a_i, x^* \rangle^2 + \zeta_i$ (where $\zeta_i$ denotes noise, and $a_i$ is the sensing vector) for $i \in [m]$. The problem of phase
retrieval has a rich history, with a variety of applications such as optics, crystallography, heteroscedastic regression, astrophysics, etc. A major consideration in
algorithms for phase
retrieval is robustness against measurement errors. In recent breakthroughs in algorithmic robust statistics, efficient
algorithms have been developed for several parameter estimation tasks such as mean estimation, covariance estimation, robust principal component analysis (PCA), etc. in the presence of heavy-tailed noise and
adversarial corruptions. In this paper, we study efficient
algorithms for robust phase
retrieval with heavy-tailed noise when a constant fraction of both the measurements $y_i$ and the sensing vectors $a_i$ may be arbitrarily adversarially corrupted. For this problem, Buna and Rebeschini (AISTATS 2025) very recently gave an exponential time
algorithm with sample complexity $O(n \log n)$. Their
algorithm needs a robust spectral initialization, specifically, a robust estimate of the top eigenvector of a covariance
matrix, which they deemed to be beyond known efficient algorithmic techniques (similar spectral initializations are a key ingredient of a large family of phase
retrieval algorithms). In this work, we make a connection between robust spectral initialization and recent algorithmic advances in robust PCA, yielding the first
polynomial-time
algorithms for robust phase
retrieval with both heavy-tailed noise and
adversarial corruptions, in fact with near-linear (in $n$) sample complexity.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
TechING: Towards Real World Technical Image Understanding via VLMs
Tafazzul Nadeem, Bhavik Shangari, Manish Rai et al. (+2 more)
Professionals working in technical domain typically hand-draw (on whiteboard, paper, etc.) technical diagrams (e.g., flowcharts, block diagrams, etc.) during discussions; however, if they want to edit these later, it needs to be drawn from scratch. Modern day VLMs have made tremendous progress in image understanding but they struggle when it comes to understanding technical diagrams. One way to overcome this problem is to fine-tune on real world hand-drawn images, but it is not practically possible to generate large number of such images. In this paper, we introduce a large synthetically generated corpus (reflective of real world images) for training VLMs and subsequently evaluate VLMs on a smaller corpus of hand-drawn images (with the help of humans). We introduce several new self-supervision tasks for training and perform extensive experiments with various baseline models and fine-tune Llama 3.2 11B-instruct model on synthetic images on these tasks to obtain LLama-VL-TUG, which significantly improves the ROUGE-L performance of Llama 3.2 11B-instruct by 2.14x and achieves the best all-round performance across all baseline models. On real-world images, human evaluation reveals that we achieve minimum compilation errors across all baselines in 7 out of 8 diagram types and improve the average F1 score of Llama 3.2 11B-instruct by 6.97x.
Read the full abstract ▾
All authors Tafazzul Nadeem, Bhavik Shangari, Manish Rai, Gagan Raj Gupta, Ashutosh Modi
Machine Learning
Rethinking Cross-Modal Fine-Tuning: Optimizing the Interaction Between Feature Alignment and Target Fitting
Trong Khiem Tran, Manh Cuong Dao, Phi Le Nguyen et al. (+2 more)
Adapting pre-trained models to unseen feature modalities has become increasingly important due to the growing need for cross-disciplinary knowledge integration. A key challenge here is how to align the representation of new modalities with the most relevant parts of the pre-trained model's representation space to enable accurate knowledge transfer. This requires combining feature
alignment with target fine-tuning, but uncalibrated combinations can exacerbate misalignment between the source and target feature-label structures and reduce target generalization. Existing work, however, lacks a theoretical understanding of this critical interaction between feature
alignment and target fitting. To bridge this gap, we develop a principled framework that establishes a provable generalization bound on the target error, which explains the interaction between feature
alignment and target fitting through a novel concept of feature-label distortion. This bound offers actionable insights into how this interaction should be optimized for practical
algorithm design. The resulting approach achieves significantly improved performance over state-of-the-art methods across a wide range of
benchmark datasets.
Read the full abstract ▾
All authors Trong Khiem Tran, Manh Cuong Dao, Phi Le Nguyen, Thao Nguyen Truong, Trong Nghia Hoang
Machine Learning · Cross-listed from cs.CV
Facial Emotion Recognition on FER-2013 using an EfficientNetB2-Based Approach
Sahil Naik, Soham Bagayatkar and Pavankumar Singh
Detection of human
emotions based on facial images in real-world scenarios is a difficult task due to low image quality, variations in lighting, pose changes, background distractions, small inter-class variations, noisy crowd-sourced labels, and severe class imbalance, as observed in the FER-2013
dataset of 48x48 grayscale images. Although recent approaches using large CNNs such as VGG and ResNet achieve reasonable accuracy, they are computationally expensive and
memory-intensive, limiting their practicality for real-time applications. We address these challenges using a lightweight and efficient facial
emotion recognition pipeline based on EfficientNetB2, trained using a two-stage warm-up and fine-tuning strategy. The model is enhanced with AdamW optimization, decoupled weight decay, label smoothing (epsilon = 0.06) to reduce annotation noise, and clipped class weights to mitigate class imbalance, along with dropout, mixed-precision training, and extensive real-time data augmentation. The model is trained using a stratified 87.5%/12.5% train-validation split while keeping the official test set intact, achieving a test accuracy of 68.78% with nearly ten times fewer parameters than VGG16-based baselines. Experimental results, including per-class metrics and learning dynamics, demonstrate stable training and strong generalization, making the proposed approach suitable for real-time and edge-based applications.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.med-ph
Automated HER2 scoring with uncertainty quantification using lensfree holography and deep learning
Che-Yung Shen, Xilin Yang, Yuzhu Li et al. (+2 more)
Accurate assessment of human epidermal growth factor receptor 2 (HER2) expression is critical for breast cancer diagnosis, prognosis, and therapy selection; yet, most existing digital HER2 scoring methods rely on bulky and expensive optical systems. Here, we present a compact and cost-effective lensfree holography platform integrated with deep learning for automated HER2 scoring of immunohistochemically stained breast tissue sections. The system captures lensfree diffraction patterns of stained HER2 tissue sections under RGB
laser illumination and acquires complex field information over a sample area of ~1,250 mm^2 at an effective throughput of ~84 mm^2 per minute. To enhance diagnostic reliability, we incorporated an uncertainty quantification strategy based on
Bayesian Monte Carlo dropout, which provides autonomous uncertainty estimates for each prediction and supports reliable, robust HER2 scoring, with an overall correction rate of 30.4%. Using a blinded test set of 412 unique tissue samples, our approach achieved a testing accuracy of 84.9% for 4-class (0, 1+, 2+, 3+) HER2 classification and 94.8% for binary (0/1+ vs. 2+/3+) HER2 scoring with uncertainty quantification. Overall, this lensfree holography approach provides a practical pathway toward portable, high-throughput, and cost-effective HER2 scoring, particularly suited for resource-limited settings, where traditional digital pathology infrastructure is unavailable.
Read the full abstract ▾
All authors Che-Yung Shen, Xilin Yang, Yuzhu Li, Leon Lenk, Aydogan Ozcan
Machine Learning · Cross-listed from cs.AI
Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agents
Zhihan Liu, Lin Guan, Yixin Nie et al. (+6 more)
Generalist LLM
agents are often post-trained on a narrow set of environments but deployed across far broader, unseen domains. In this work, we investigate the challenge of agentic post-training when the eventual test domains are unknown. Specifically, we analyze which properties of reinforcement learning (RL) environments and modeling choices have the greatest influence on out-of-domain performance. First, we identify two environment axes that strongly correlate with cross-domain generalization: (i) state information richness, i.e., the amount of information for the
agent to process from the state, and (ii) planning complexity, estimated via goal reachability and trajectory length under a base policy. Notably, domain realism and text-level similarity are not the primary factors; for instance, the simple grid-world domain Sokoban leads to even stronger generalization in SciWorld than the more realistic ALFWorld. Motivated by these findings, we further show that increasing state information richness alone can already effectively improve cross-domain robustness. We propose a randomization technique, which is low-overhead and broadly applicable: add small amounts of distractive goal-irrelevant features to the state to make it richer without altering the task. Beyond environment-side properties, we also examine several modeling choices: (a) SFT warmup or mid-training helps prevent catastrophic forgetting during RL but undermines generalization to domains that are not included in the mid-training datamix; and (b) turning on step-by-step thinking during RL, while not always improving in-domain performance, plays a crucial role in preserving generalization.
Read the full abstract ▾
All authors Zhihan Liu, Lin Guan, Yixin Nie, Kai Zhang, Zhuoqun Hao, Lin Chen, Asli Celikyilmaz, Zhaoran Wang, Na Zhang
Machine Learning
PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVR
James Burgess, Jan N. Hansen, Duo Peng et al. (+5 more)
Search
agents are
language models (LMs) that reason and search knowledge bases (or the web) to answer questions; recent methods supervise only the final answer accuracy using reinforcement learning with verifiable rewards (RLVR). Most RLVR search
agents tackle general-domain QA, which limits their relevance to technical AI systems in science, engineering, and medicine. In this work we propose training
agents to search and reason over scientific papers – this tests technical question-answering, it is directly relevant to real scientists, and the capabilities will be crucial to future AI Scientist systems. Concretely, we release a search corpus of 16 million biomedical paper abstracts and construct a challenging factoid QA
dataset called PaperSearchQA with 60k samples answerable from the corpus, along with
benchmarks. We train search
agents in this environment to outperform non-RL
retrieval baselines; we also perform further quantitative analysis and observe interesting
agent behaviors like planning,
reasoning, and self-verification. Our corpus,
datasets, and
benchmarks are usable with the popular Search-R1 codebase for RLVR training and released on https://huggingface.co/collections/jmhb/papersearchqa. Finally, our data creation methods are scalable and easily extendable to other scientific domains.
Read the full abstract ▾
All authors James Burgess, Jan N. Hansen, Duo Peng, Yuhui Zhang, Alejandro Lozano, Min Woo Sun, Emma Lundberg, Serena Yeung-Levy
Machine Learning
HeterCSI: Channel-Adaptive Heterogeneous CSI Pretraining Framework for Generalized Wireless Foundation Models
Chenyu Zhang, Xinchen Lyu, Chenshan Ren et al. (+3 more)
Wireless foundation models promise transformative capabilities for channel state information (CSI) processing across diverse 6G network applications, yet face fundamental challenges due to the inherent dual heterogeneity of CSI across both scale and scenario dimensions. However, current pretraining approaches either constrain inputs to fixed dimensions or isolate training by scale, limiting the generalization and scalability of wireless foundation models. In this paper, we propose HeterCSI, a channel-adaptive pretraining framework that reconciles training efficiency with robust cross-scenario generalization via a new understanding of gradient dynamics in heterogeneous CSI pretraining. Our key insight reveals that CSI scale heterogeneity primarily causes destructive gradient interference, while scenario diversity actually promotes constructive gradient
alignment when properly managed. Specifically, we formulate heterogeneous CSI batch construction as a partitioning optimization problem that minimizes zero-padding overhead while preserving scenario diversity. To solve this, we develop a scale-aware adaptive batching strategy that aligns CSI samples of similar scales, and design a double-masking mechanism to isolate valid signals from padding artifacts. Extensive experiments on 12
datasets demonstrate that HeterCSI establishes a generalized foundation model without scenario-specific finetuning, achieving superior average performance over full-shot baselines. Compared to the state-of-the-art zero-shot
benchmark WiFo, it reduces NMSE by 7.19 dB, 4.08 dB, and 5.27 dB for CSI reconstruction, time-domain, and frequency-domain prediction, respectively. The proposed HeterCSI framework also reduces training latency by 53% compared to existing approaches while improving generalization performance by 1.53 dB on average.
Read the full abstract ▾
All authors Chenyu Zhang, Xinchen Lyu, Chenshan Ren, Shuhan Liu, Qimei Cui, Xiaofeng Tao
Machine Learning
Smooth, Sparse, and Stable: Finite-Time Exact Skeleton Recovery via Smoothed Proximal Gradients
Rui Wu and Yongjun Li
Continuous optimization has significantly advanced causal discovery, yet existing methods (e.g., NOTEARS) generally guarantee only asymptotic convergence to a stationary point. This often yields dense weighted
matrices that require arbitrary post-hoc thresholding to recover a DAG. This gap between continuous optimization and discrete graph structures remains a fundamental challenge. In this paper, we bridge this gap by proposing the Hybrid-Order Acyclicity Constraint (AHOC) and optimizing it via the Smoothed Proximal Gradient (SPG-AHOC). Leveraging the
Manifold Identification Property of proximal
algorithms, we provide a rigorous theoretical guarantee: the Finite-Time Oracle Property. We prove that under standard identifiability assumptions, SPG-AHOC recovers the exact DAG support (structure) in finite iterations, even when optimizing a smoothed approximation. This result eliminates structural ambiguity, as our
algorithm returns graphs with exact zero entries without heuristic truncation. Empirically, SPG-AHOC achieves state-of-the-art accuracy and strongly corroborates the finite-time identification theory.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Success Conditioning as Policy Improvement: The Optimization Problem Solved by Imitating Success
Daniel Russo
A widely used technique for improving policies is success conditioning, in which one collects trajectories, identifies those that achieve a desired outcome, and updates the policy to imitate the actions taken along successful trajectories. This principle appears under many names – rejection sampling with SFT, goal-conditioned RL, Decision
Transformers – yet what optimization problem it solves, if any, has remained unclear. We prove that success conditioning exactly solves a trust-region optimization problem, maximizing policy improvement subject to a $\chi^2$ divergence constraint whose radius is determined automatically by the data. This yields an identity: relative policy improvement, the magnitude of policy change, and a quantity we call action-influence – measuring how random variation in action choices affects success rates – are exactly equal at every state. Success conditioning thus emerges as a conservative improvement operator. Exact success conditioning cannot degrade performance or induce dangerous distribution shift, but when it fails, it does so observably, by hardly changing the policy at all. We apply our theory to the common practice of return thresholding, showing this can amplify improvement, but at the cost of potential misalignment with the true objective.
Read the full abstract ▾
Machine Learning
Learning Fair Domain Adaptation with Virtual Label Distribution
Yuguang Zhang, Lijun Sheng, Jian Liang et al. (+1 more)
Unsupervised Domain Adaptation (UDA) aims to mitigate performance degradation when training and testing data are sampled from different distributions. While significant progress has been made in enhancing overall accuracy, most existing methods overlook performance disparities across categories-an issue we refer to as category
fairness. Our empirical analysis reveals that UDA classifiers tend to favor certain easy categories while neglecting difficult ones. To address this, we propose Virtual Label-distribution-aware Learning (VILL), a simple yet effective framework designed to improve worst-case performance while preserving high overall accuracy. The core of VILL is an adaptive re-weighting strategy that amplifies the influence of hard-to-classify categories. Furthermore, we introduce a KL-divergence-based re-balancing strategy, which explicitly adjusts decision boundaries to enhance category
fairness. Experiments on commonly used
datasets demonstrate that VILL can be seamlessly integrated as a plug-and-play module into existing UDA methods, significantly improving category
fairness.
Read the full abstract ▾
All authors Yuguang Zhang, Lijun Sheng, Jian Liang, Ran He
Machine Learning · Cross-listed from cs.CV
Agentic Very Long Video Understanding
Aniket Rege, Arka Sadhu, Yuliang Li et al. (+5 more)
The advent of always-on personal AI assistants, enabled by all-day wearable devices such as smart glasses, demands a new level of contextual understanding, one that goes beyond short, isolated events to encompass the continuous, longitudinal stream of
egocentric video. Achieving this
vision requires advances in long-horizon
video understanding, where systems must interpret and recall visual and audio information spanning days or even weeks. Existing methods, including large
language models and
retrieval-augmented generation, are constrained by limited context windows and lack the ability to perform compositional, multi-hop
reasoning over very long
video streams. In this work, we address these challenges through EGAgent, an enhanced agentic framework centered on entity scene graphs, which represent people, places, objects, and their relationships over time. Our system equips a planning
agent with tools for structured search and
reasoning over these graphs, as well as hybrid visual and audio search capabilities, enabling detailed, cross-modal, and temporally coherent
reasoning. Experiments on the EgoLifeQA and
Video-MME (Long)
datasets show that our method achieves state-of-the-art performance on EgoLifeQA (57.5%) and competitive performance on
Video-MME (Long) (74.1%) for complex longitudinal
video understanding tasks. Code is available at https://github.com/facebookresearch/egagent.
Read the full abstract ▾
All authors Aniket Rege, Arka Sadhu, Yuliang Li, Kejie Li, Ramya Korlakai Vinayak, Yuning Chai, Yong Jae Lee, Hyo Jin Kim
Machine Learning
FP8-RL: A Practical and Stable Low-Precision Stack for LLM Reinforcement Learning
Zhaopeng Qiu, Shuang Yu, Jingqi Zhang et al. (+4 more)
Reinforcement learning (RL) for large
language models (LLMs) is increasingly bottlenecked by rollout (generation), where long output sequence lengths make
attention and KV-cache
memory dominate end-to-end step time. FP8 offers an attractive lever for accelerating RL by reducing compute cost and
memory traffic during rollout, but applying FP8 in RL introduces unique engineering and algorithmic challenges: policy weights change every step (requiring repeated quantization and weight synchronization into the inference engine) and low-precision rollouts can deviate from the higher-precision policy assumed by the trainer, causing train-inference mismatch and potential instability. This report presents a practical FP8 rollout stack for LLM RL, implemented in the veRL ecosystem with support for common training backends (e.g., FSDP/Megatron-LM) and inference engines (e.g., vLLM/SGLang). We (i) enable FP8 W8A8 linear-layer rollout using blockwise FP8 quantization, (ii) extend FP8 to KV-cache to remove long-context
memory bottlenecks via per-step QKV scale recalibration, and (iii) mitigate mismatch using importance-sampling-based rollout correction (
token-level TIS/MIS variants). Across dense and MoE models, these techniques deliver up to 44% rollout throughput gains while preserving learning behavior comparable to BF16 baselines.
Read the full abstract ▾
All authors Zhaopeng Qiu, Shuang Yu, Jingqi Zhang, Shuai Zhang, Xue Huang, Jingyi Yang, Junjie Lai
Machine Learning · Cross-listed from stat.ML
Exact Minimum-Volume Confidence Set Intersection for Multinomial Outcomes
Heguang Lin, Binhao Chen, Mengze Li et al. (+2 more)
Computation of confidence sets is central to data science and machine learning, serving as the workhorse of A/B testing and underpinning the operation and analysis of reinforcement learning
algorithms. Among all valid confidence sets for the multinomial parameter, minimum-volume confidence sets (MVCs) are optimal in that they minimize average volume, but they are defined as level sets of an exact p-value that is discontinuous and difficult to compute. Rather than attempting to characterize the geometry of MVCs directly, this paper studies a practically motivated decision problem: given two observed multinomial outcomes, can one certify whether their MVCs intersect? We present a certified, tolerance-aware
algorithm for this intersection problem. The method exploits the fact that likelihood ordering induces halfspace constraints in log-odds coordinates, enabling adaptive
geometric partitioning of parameter space and computable lower and upper bounds on p-values over each
cell. For three categories, this yields an efficient and provably sound
algorithm that either certifies intersection, certifies disjointness, or returns an indeterminate result when the decision lies within a prescribed margin. We further show how the approach extends to higher dimensions. The results demonstrate that, despite their irregular geometry, MVCs admit reliable certified decision procedures for core tasks in A/B testing.
Read the full abstract ▾
All authors Heguang Lin, Binhao Chen, Mengze Li, Daniel Pimentel-Alarcón, Matthew L. Malloy
Machine Learning
Enhance the Safety in Reinforcement Learning by ADRC Lagrangian Methods
Mingxu Zhang, Huicheng Zhang, Jiaming Ji et al. (+2 more)
Safe reinforcement learning (Safe RL) seeks to maximize rewards while satisfying safety constraints, typically addressed through Lagrangian-based methods. However, existing approaches, including PID and classical Lagrangian methods, suffer from oscillations and frequent safety violations due to parameter sensitivity and inherent phase lag. To address these limitations, we propose ADRC-Lagrangian methods that leverage Active Disturbance Rejection Control (ADRC) for enhanced robustness and reduced oscillations. Our unified framework encompasses classical and PID Lagrangian methods as special cases while significantly improving safety performance. Extensive experiments demonstrate that our approach reduces safety violations by up to 74%, constraint violation magnitudes by 89%, and average costs by 67%, establishing superior effectiveness for Safe RL in complex environments.
Read the full abstract ▾
All authors Mingxu Zhang, Huicheng Zhang, Jiaming Ji, Yaodong Yang, Ying Sun
Machine Learning · Cross-listed from stat.ML
Nonlinear multi-study sparse factor analysis
Gemma E. Moran and Anandi Krishnan
High-dimensional data often exhibit variation that can be captured by lower-dimensional factors. For high-dimensional data from multiple studies, one goal is to understand which underlying factors are common to all studies, and which factors are study-specific. As a particular example, we consider platelet gene expression data from patients in different disease groups. In this data, factors correspond to clusters of genes which are co-expressed; we may expect some clusters (or biological pathways) to be active for all diseases, while some clusters are only active for a specific disease. To learn these factors, we consider a nonlinear multi-study sparse factor model, which allows for both shared and study-specific factors. To fit this model, we propose a multi-study sparse variational autoencoder. The underlying model is sparse in that each observed feature (i.e. each dimension of the data) depends on a small subset of the latent factors. In the genomics example, this means each gene is active in only a few biological processes. We prove that the latent factor distributions are identifiable (up to element-wise transformations), and that the canonical shared and study-specific support structure is identifiable. Empirically, we demonstrate our method recovers meaningful factors in the platelet gene expression data.
Read the full abstract ▾
Machine Learning
Robust Learning of a Group DRO Neuron
Guyang Cao, Shuyao Li, Sushrut Karmalkar et al. (+1 more)
We study the problem of learning a single
neuron under standard squared loss in the presence of arbitrary label noise and group-level distributional shifts, for a broad family of
covariate distributions. Our goal is to identify a ”best-fit”
neuron parameterized by $\mathbf{w}_*$ that performs well under the most challenging reweighting of the groups. Specifically, we address a Group Distributionally Robust Optimization problem: given sample access to $K$ distinct distributions $\mathcal p_{[1]},\dots,\mathcal p_{[K]}$, we seek to approximate $\mathbf{w}_*$ that minimizes the worst-case objective over convex combinations of group distributions $\boldsymbol{\lambda} \in \Delta_K$, where the objective is $\sum_{i \in [K]}\lambda_{[i]}\,\mathbb E_{(\mathbf x,y)\sim\mathcal p_{[i]}}(\sigma(\mathbf w\cdot\mathbf x)-y)^2 - \nu d_f(\boldsymbol\lambda,\frac{1}{K}\mathbf1)$ and $d_f$ is an $f$-divergence that imposes (optional) penalty on deviations from uniform group weights, scaled by a parameter $\nu \geq 0$. We develop a computationally efficient primal-dual
algorithm that outputs a vector $\widehat{\mathbf w}$ that is constant-factor competitive with $\mathbf{w}_*$ under the worst-case group weighting. Our analytical framework directly confronts the inherent nonconvexity of the loss function, providing robust learning guarantees in the face of arbitrary label corruptions and group-specific distributional shifts. The implementation of the dual extrapolation update motivated by our algorithmic framework shows promise on LLM pre-training
benchmarks.
Read the full abstract ▾
All authors Guyang Cao, Shuyao Li, Sushrut Karmalkar, Jelena Diakonikolas
Machine Learning
Demystifying Data-Driven Probabilistic Medium-Range Weather Forecasting
Jean Kossaifi, Nikola Kovachki, Morteza Mardani et al. (+15 more)
The recent revolution in data-driven methods for weather
forecasting has lead to a fragmented landscape of complex, bespoke architectures and training strategies, obscuring the fundamental drivers of forecast accuracy. Here, we demonstrate that state-of-the-art probabilistic skill requires neither intricate architectural constraints nor specialized training heuristics. We introduce a scalable framework for learning multi-scale atmospheric dynamics by combining a directly downsampled latent space with a history-conditioned local projector that resolves high-resolution physics. We find that our framework design is robust to the choice of probabilistic
estimator, seamlessly supporting
stochastic interpolants,
diffusion models, and CRPS-based ensemble training. Validated against the Integrated
Forecasting System and the deep learning probabilistic model GenCast, our framework achieves statistically significant improvements on most of the variables. These results suggest scaling a general-purpose model is sufficient for state-of-the-art medium-range prediction, eliminating the need for tailored training recipes and proving effective across the full spectrum of probabilistic frameworks.
Read the full abstract ▾
All authors Jean Kossaifi, Nikola Kovachki, Morteza Mardani, Daniel Leibovici, Suman Ravuri, Ira Shokar, Edoardo Calvello, Mohammad Shoaib Abbas, Peter Harrington, Ashay Subramaniam, Noah Brenowitz, Boris Bonev, Wonmin Byeon, Karsten Kreis, Dale Durran, Arash Vahdat, Mike Pritchard, Jan Kautz
Machine Learning
AttenMIA: LLM Membership Inference Attack through Attention Signals
Pedram Zaree, Md Abdullah Al Mamun, Yue Dong et al. (+2 more)
Large
Language Models (LLMs) are increasingly deployed to enable or improve a multitude of real-world applications. Given the large size of their training data sets, their tendency to memorize training data raises serious
privacy and intellectual property concerns. A key threat is the membership inference attack (MIA), which aims to determine whether a given sample was included in the model's training set. Existing MIAs for LLMs rely primarily on output confidence scores or
embedding-based features, but these signals are often brittle, leading to limited attack success. We introduce AttenMIA, a new MIA framework that exploits self-
attention patterns inside the
transformer model to infer membership.
Attention controls the information flow within the
transformer, exposing different patterns for memorization that can be used to identify members of the
dataset. Our method uses information from
attention heads across layers and combines them with perturbation-based divergence metrics to train an effective MIA classifier. Using extensive experiments on open-source models including LLaMA-2, Pythia, and Opt models, we show that
attention-based features consistently outperform baselines, particularly under the important low-false-positive metric (e.g., achieving up to 0.996 ROC AUC & 87.9% TPR@1%FPR on the WikiMIA-32
benchmark with Llama2-13b). We show that
attention signals generalize across
datasets and architectures, and provide a layer- and head-level analysis of where membership leakage is most pronounced. We also show that using AttenMIA to replace other membership inference attacks in a data extraction framework results in training data extraction attacks that outperform the state of the art. Our findings reveal that
attention mechanisms, originally introduced to enhance interpretability, can inadvertently amplify
privacy risks in LLMs, underscoring the need for new defenses.
Read the full abstract ▾
All authors Pedram Zaree, Md Abdullah Al Mamun, Yue Dong, Ihsen Alouani, Nael Abu-Ghazaleh
Machine Learning
Beyond Static Datasets: Robust Offline Policy Optimization via Vetted Synthetic Transitions
Pedram Agand and Mo Chen
Offline Reinforcement Learning (ORL) holds immense promise for safety-critical domains like industrial robotics, where real-time environmental interaction is often prohibitive. A primary obstacle in ORL remains the distributional shift between the static
dataset and the learned policy, which typically mandates high degrees of conservatism that can restrain potential policy improvements. We present MoReBRAC, a model-based framework that addresses this limitation through Uncertainty-Aware latent synthesis. Instead of relying solely on the fixed data, MoReBRAC utilizes a dual-recurrent world model to synthesize high-fidelity transitions that augment the training
manifold. To ensure the reliability of this synthetic data, we implement a hierarchical uncertainty pipeline integrating Variational Autoencoder (VAE)
manifold detection, model sensitivity analysis, and Monte Carlo (MC) dropout. This multi-layered filtering process guarantees that only transitions residing within high-confidence regions of the learned dynamics are utilized. Our results on D4RL Gym-MuJoCo
benchmarks reveal significant performance gains, particularly in “random” and “suboptimal” data regimes. We further provide insights into the role of the VAE as a
geometric anchor and discuss the distributional trade-offs encountered when learning from near-optimal
datasets.
Read the full abstract ▾
Machine Learning
From LLMs to LRMs: Rethinking Pruning for Reasoning-Centric Models
Longwei Ding, Anhao Zhao, Fanghua Ye et al. (+2 more)
Large
language models (LLMs) are increasingly costly to deploy, motivating extensive research on model
pruning. However, most existing studies focus on instruction-following LLMs, leaving it unclear whether established
pruning strategies transfer to
reasoning-augmented models that explicitly generate long intermediate
reasoning traces. In this work, we conduct a controlled study of
pruning for both instruction-following ($\textbf{LLM-instruct}$) and
reasoning-augmented ($\textbf{LLM-think}$) models. To isolate the effects of
pruning, we align
pruning calibration and post-
pruning recovery data with each model's original training distribution, which we show yields more stable and reliable
pruning behavior. We evaluate static depth
pruning, static width
pruning, and dynamic
pruning across 17 tasks spanning classification, generation, and
reasoning. Our results reveal clear paradigm-dependent differences: depth
pruning outperforms width
pruning on classification tasks, while width
pruning is more robust for generation and
reasoning. Moreover, static
pruning better preserves
reasoning performance, whereas dynamic
pruning excels on classification and generation but remains challenging for long-chain
reasoning. These findings underscore the need for
pruning strategies that explicitly account for the distinct characteristics of
reasoning-augmented LLMs. Our code is publicly available at https://github.com/EIT-NLP/LRM-
Pruning.
Read the full abstract ▾
All authors Longwei Ding, Anhao Zhao, Fanghua Ye, Ziyang Chen, Xiaoyu Shen