Machine Learning
Amortized Reasoning Tree Search: Decoupling Proposal and Decision in Large Language Models
Zesheng Hong, Jiadong Yu and Hui Pan
Reinforcement Learning with Verifiable Rewards (RLVR) has established itself as the dominant paradigm for instilling rigorous
reasoning capabilities in Large
Language Models. While effective at amplifying dominant behaviors, we identify a critical pathology in this
alignment process: the systematic suppression of valid but rare (low-likelihood under the base model distribution)
reasoning paths. We theoretically characterize this phenomenon as a "Normalization Squeeze," where the interplay between mode-seeking policy gradients and finite sampling acts as a high-pass likelihood filter, driving the probability of rare correct traces to statistical extinction. To counteract this collapse without discarding the base model's latent diversity, we propose Amortized
Reasoning Tree Search (ARTS). Unlike standard approaches that force internalization via parameter updates, ARTS prioritizes deliberation by decoupling generation from verification. We introduce a Flow Matching objective that repurposes the verifier to estimate the conservation of probability flow, enabling robust
navigation through sparse, high-
entropy search spaces where traditional discriminative objectives fail. Extensive experiments on the MATH-500
benchmark demonstrate that ARTS achieves a performance of 74.6% (BoN@16), effectively matching fully fine-tuned policies (74.7%) without modifying the generative backbone. Crucially, on the long-tail subset where coupled RL optimization collapses to 0% pass@k, ARTS uniquely recovers significant performance, suggesting that disentangling verification from generation offers a more robust pathway for solving complex
reasoning tasks.
Read the full abstract ▾
Machine Learning
Vital Trace: Protocol-Constrained Patient-State Reasoning for Longitudinal Clinical Trajectories
Zhan Qu and Michael Färber
Longitudinal clinical
reasoning over electronic health records requires tracking evolving physiological measurements, laboratory results, and interventions across extended patient trajectories. Existing LLM-based clinical
reasoning systems often rely on repeatedly serializing patient histories or exchanging unconstrained textual
agent messages, leading to context drift, unstable
reasoning, and growing inference cost over long horizons. We present Vital Trace, a protocol-constrained multi-
agent framework for future clinical risk prediction over evolving ICU trajectories. Instead of maintaining unbounded textual histories, Vital Trace uses a compact persistent patient-state
memory together with staged
reasoning performed by four coordinated
agents: a Router, Reasoner, Auditor, and Steward. To support temporally coherent
reasoning, we introduce a manually curated Global Protocol containing physiological state-transition rules and a dynamic patient-state representation that tracks hemodynamic, respiratory, renal, metabolic, and inflammatory instability over time. We evaluate Vital Trace on MIMIC-IV and eICU using future vasopressor-support, respiratory-support, renal-support, and deterioration prediction tasks. Results show that structured protocol-constrained
reasoning improves temporal consistency, communication stability, calibration, and interpretability compared with free-form multi-
agent baselines while achieving strong predictive performance across long ICU trajectories.
Read the full abstract ▾
Machine Learning
FLAC: Maximum Entropy RL via Kinetic Energy Regularized Bridge Matching
Lei Lv, Yunfei Li, Yu Luo et al. (+2 more)
Iterative generative policies, such as
diffusion models and flow matching, offer superior expressivity for continuous control but complicate Maximum
Entropy Reinforcement Learning because their action log-densities are not directly accessible. To address this, we propose Field Least-Energy Actor-Critic (FLAC), a likelihood-free framework that regulates policy stochasticity by penalizing the kinetic energy of the velocity field. Our key insight is to formulate policy optimization as a Generalized Schrödinger Bridge (GSB) problem relative to a high-
entropy reference process (e.g., uniform). Under this view, the maximum-
entropy principle emerges naturally as staying close to a high-
entropy reference while optimizing return, without requiring explicit action densities. In this framework, kinetic energy serves as a physically grounded proxy for divergence from the reference: minimizing path-space energy bounds the deviation of the induced terminal action distribution. Building on this view, we derive an energy-regularized policy iteration scheme and a practical off-policy
algorithm that automatically tunes the kinetic energy via a Lagrangian dual mechanism. Empirically, FLAC achieves superior or comparable performance on high-dimensional
benchmarks relative to strong baselines, while avoiding explicit density estimation.
Read the full abstract ▾
All authors Lei Lv, Yunfei Li, Yu Luo, Fuchun Sun, Xiao Ma
Machine Learning
Risk Horizons: Structured Hypothesis Spaces for Longitudinal Clinical Prediction
Zhan Qu and Michael Färber
Predicting future clinical events from longitudinal electronic health records (EHRs) requires selecting plausible outcomes from a large and structured event space under sparse observations. While clinical coding systems provide hierarchical organization of events, cross-modal and temporal relationships are not explicitly specified and must instead be inferred from data, making prediction difficult for weakly observed longitudinal transitions. We introduce Risk Horizons, a geometry-aware framework for constructing patient-specific candidate spaces for multi-modal next-visit prediction. Risk Horizons combines deterministic coding hierarchies with data-driven lagged cross-modal associations, embeds the resulting clinical graph in
hyperbolic space, and retrieves candidate futures using directional risk cones. This reframes longitudinal prediction as ranking within a compact, clinically coherent hypothesis space rather than scoring an unconstrained vocabulary. Experiments on MIMIC-IV and eICU demonstrate competitive next-visit prediction performance, with consistently improved hierarchy consistency across diagnoses, procedures, and medications. Further analysis suggests that
hyperbolic structured candidate
retrieval is the primary driver of performance, while LLMs are effective as constrained inference-time rerankers operating over clinically grounded candidate sets.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Reliable Hierarchical Operating System Fingerprinting via Conformal Prediction
Rubén Pérez-Jove, Osvaldo Simeone, Alejandro Pazos et al. (+1 more)
Operating System (OS) fingerprinting is critical for network security, but conventional methods do not provide formal uncertainty quantification mechanisms. Conformal Prediction (CP) could be directly wrapped around existing methods to obtain prediction sets with guaranteed coverage. However, a direct application of CP would treat OS identification as a flat classification problem, ignoring the natural taxonomic structure of OSs and providing brittle point predictions. This work addresses these limitations by introducing and evaluating two distinct structured CP strategies: level-wise CP (L-CP), which calibrates each hierarchy level independently, and projection-based CP (P-CP), which ensures structural consistency by projecting leaf-level sets upwards. Our results demonstrate that, while both methods satisfy validity guarantees, they expose a fundamental trade-off between level-wise efficiency and structural consistency. L-CP yields tighter prediction sets suitable for human forensic analysis but suffers from taxonomic inconsistencies. Conversely, P-CP guarantees hierarchically consistent, nested sets ideal for automated policy enforcement, albeit at the cost of reduced efficiency at coarser levels.
Read the full abstract ▾
All authors Rubén Pérez-Jove, Osvaldo Simeone, Alejandro Pazos, Jose Vázquez-Naya
Machine Learning · Cross-listed from cs.CL
RAT-Bench: A Comprehensive Benchmark for Text Anonymization
Nataša Krčo, Zexi Yao, Matthieu Meeus et al. (+1 more)
Data containing personal information is increasingly used to train, fine-tune, or query Large
Language Models (LLMs). Text is typically scrubbed of identifying information prior to use, often with tools such as Microsoft's Presidio or Anthropic's PII purifier. These tools have traditionally been evaluated on their ability to remove specific identifiers (e.g., names), yet their effectiveness at preventing re-identification remains unclear. We introduce RAT-Bench, a comprehensive
benchmark for text anonymization tools based on re-identification risk. Using U.S. demographic statistics, we generate synthetic text containing various direct and indirect identifiers across domains,
languages, and difficulty levels. We evaluate a range of NER- and LLM-based text anonymization tools and, based on the attributes an LLM-based attacker is able to correctly infer from the anonymized text, we report the risk of re-identification in the U.S. population, while properly accounting for the disparate impact of identifiers. We find that, while capabilities vary widely, even the best tools are far from perfect in particular when direct identifiers are not written in standard ways and when indirect identifiers enable re-identification. Overall we find LLM-based anonymizers, including new iterative anonymizers, to provide a better
privacy-utility trade-off albeit at a higher computational cost. Importantly, we also find them to work well across
languages. We conclude with recommendations for future anonymization tools and will release the
benchmark and encourage community efforts to expand it, in particular to other geographies.
Read the full abstract ▾
All authors Nataša Krčo, Zexi Yao, Matthieu Meeus, Yves-Alexandre de Montjoye
Machine Learning
Can Neural Networks Provide Latent Embeddings for Telemetry-Aware Greedy Routing?
Andreas Boltres, Niklas Freymuth and Gerhard Neumann
Telemetry-Aware routing promises to increase efficacy and responsiveness to traffic surges in computer networks. Recent research leverages Machine Learning to deal with the complex dependency between network state and routing, but sacrifices explainability of routing decisions due to the black-box nature of the proposed neural routing modules. We propose Placer, a novel
algorithm using Message Passing Networks to transform network states into latent node
embeddings. These
embeddings facilitate quick greedy next-hop routing without directly solving the all-pairs shortest paths problem, and let us visualize how certain network events shape routing decisions.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Aspect-Based Sentiment Analysis for Future Tourism Experiences: A BERT-MoE Framework for Persian User Reviews
Hamidreza Kazemi Taskooh and Taha Zare Harofte
This study advances aspect-based sentiment analysis (ABSA) for Persian-
language user reviews in the tourism domain, addressing challenges of low-resource
languages. We propose a hybrid BERT-based model with Top-K routing and auxiliary losses to mitigate routing collapse and improve efficiency. The pipeline includes: (1) overall sentiment classification using BERT on 9,558 labeled reviews, (2) multi-label aspect extraction for six tourism-related aspects (host, price, location, amenities, cleanliness, connectivity), and (3) integrated ABSA with dynamic routing. The
dataset consists of 58,473 preprocessed reviews from the Iranian accommodation platform Jabama, manually annotated for aspects and sentiments. The proposed model achieves a weighted F1-score of 90.6% for ABSA, outperforming baseline BERT (89.25%) and a standard hybrid approach (85.7%). Key efficiency gains include a 39% reduction in GPU power consumption compared to dense BERT, supporting sustainable AI deployment in
alignment with UN SDGs 9 and 12. Analysis reveals high mention rates for cleanliness and amenities as critical aspects. This is the first ABSA study focused on Persian tourism reviews, and we release the annotated
dataset to facilitate future
multilingual NLP research in tourism.
Read the full abstract ▾
Machine Learning
Closing the Loop: A Control-Theoretic Framework for Provably Stable Time Series Forecasting with LLMs
Xingyu Zhang, Jingyao Wang, Zeen Song et al. (+2 more)
Large
Language Models (LLMs) have recently shown exceptional potential in time series
forecasting (TSF), leveraging their inherent sequential
reasoning capabilities to model complex temporal dynamics. Existing approaches typically employ an autoregressive generation strategy to adapt LLMs for TSF. However, we identify a theoretical flaw in this paradigm: during inference, the model operates in an open-loop manner, recursively consuming its own generated outputs. This leads to error accumulation, where minor early deviations cascade into significant rollout drift over long horizons. In this paper, we reformulate autoregressive
forecasting through the lens of control theory, proposing Feedback-driven LLM (F-LLM), a novel closed-loop framework. Unlike standard methods that passively propagate errors, F-LLM actively stabilizes the trajectory via a learnable residual
estimator functioning as a system observer. Furthermore, we provide a mathematical proof that, under explicit contraction assumptions, this closed-loop mechanism guarantees a uniformly bounded step-wise error sequence within the local surrogate dynamics. Extensive experiments demonstrate that F-LLM significantly mitigates error propagation, achieving good performance on time series
benchmarks. Our code is publicly available at https://github.com/Zh-XY22/F-LLM.
Read the full abstract ▾
All authors Xingyu Zhang, Jingyao Wang, Zeen Song, Changwen Zheng, Wenwen Qiang
Machine Learning
Hierarchical Successor Representation for Robust Transfer
Changmin Yu and Máté Lengyel
The successor representation (SR) provides a powerful framework for decoupling predictive dynamics from rewards, enabling rapid generalisation across reward configurations. However, the classical SR is limited by its inherent policy dependence: policies change due to ongoing learning, environmental non-stationarities, and changes in task demands, making established predictive representations obsolete. Furthermore, in topologically complex environments, SRs suffer from spectral
diffusion, leading to dense and overlapping features that scale poorly. Here we propose the Hierarchical Successor Representation (HSR) for overcoming these limitations. By incorporating temporal abstractions into the construction of predictive representations, HSR learns stable state features which are robust to task-induced policy changes. Applying non-negative
matrix factorisation (NMF) to the HSR yields a sparse, low-rank state representation that facilitates highly sample-efficient transfer to novel tasks in multi-compartmental environments. Further analysis reveals that HSR-NMF discovers interpretable topological structures, providing a policy-agnostic hierarchical map that effectively bridges model-free optimality and model-based flexibility. Beyond providing a useful basis for task-transfer, we show that HSR's temporally extended predictive structure can also be leveraged to drive efficient exploration, effectively scaling to large, procedurally generated environments.
Read the full abstract ▾
Machine Learning
Adaptive Structured Pruning of Convolutional Neural Networks for Time Series Classification
Javidan Abdullayev, Maxime Devanne, Cyril Meyer et al. (+3 more)
Deep learning models for Time Series Classification (TSC) have achieved strong predictive performance but their high computational and
memory requirements often limit deployment on resource-constrained devices. While structured
pruning can address these issues by removing redundant filters, existing methods typically rely on manually tuned hyperparameters such as
pruning ratios which limit scalability and generalization across
datasets. In this work, we propose Dynamic Structured
Pruning (DSP), a fully automatic, structured
pruning framework for convolution-based TSC models. DSP introduces an instance-wise sparsity loss during training to induce channel-level sparsity, followed by a global activation analysis to identify and prune redundant filters without needing any predefined
pruning ratio. This work tackles computational bottlenecks of deep TSC models for deployment on resource-constrained devices. We validate DSP on 128 UCR
datasets using two different deep state-of-the-art architectures: LITETime and InceptionTime. Our approach achieves an average compression of 58% for LITETime and 75% for InceptionTime architectures while maintaining classification accuracy. Redundancy analyses confirm that DSP produces compact and informative representations, offering a practical path for scalable and efficient deep TSC deployment.
Read the full abstract ▾
All authors Javidan Abdullayev, Maxime Devanne, Cyril Meyer, Ali Ismail-Fawaz, Jonathan Weber, Germain Forestier
Machine Learning · Cross-listed from cs.CV
Synthetic Craquelure Generation for Unsupervised Painting Restoration
Jana Cuch-Guillén, Antonio Agudo and Raül Pérez-Gonzalo
Cultural heritage preservation increasingly demands non-invasive digital methods for painting restoration, yet identifying and restoring fine craquelure patterns from complex brushstrokes remains challenging due to scarce pixel-level annotations. We propose a fully annotation-free framework driven by a domain-specific synthetic craquelure generator, which simulates realistic branching and tapered fissure geometry using Bézier trajectories. Our approach couples a classical morphological detector with a learning-based refinement module: a SegFormer backbone adapted via Low-Rank Adaptation (LoRA). Uniquely, we employ a detector-guided strategy, injecting the morphological map as an input spatial prior, while a masked hybrid loss and logit adjustment constrain the training to focus specifically on refining candidate crack regions. The refined masks subsequently guide an Anisotropic
Diffusion inpainting stage to reconstruct missing content. Experimental results demonstrate that our pipeline significantly outperforms state-of-the-art photographic restoration models in zero-shot settings, while faithfully preserving the original paint brushwork.
Read the full abstract ▾
Machine Learning
ADEPT: RL-Aligned Agentic Decoding of Emotion via Evidence Probing Tools – From Consensus Learning to Ambiguity-Driven Emotion Reasoning
Esther Sun, Bo-Hao Su, Abinay Reddy Naini et al. (+2 more)
Speech Large
Language Models (SLLMs) enable high-level
emotion reasoning but often produce ungrounded, text-biased judgments without verifiable acoustic evidence. In contrast, self-supervised
speech encoders such as WavLM provide strong acoustic representations yet remain opaque discriminative models with limited interpretability. To bridge this gap, we introduce ADEPT (Agentic Decoding of
Emotion via Evidence Probing Tools), a framework that reframes
emotion recognition as a multi-turn inquiry process rather than a single-pass prediction. ADEPT transforms an SLLM into an
agent that maintains an evolving candidate
emotion set and adaptively invokes dedicated semantic and acoustic probing tools within a structured pipeline of candidate generation, evidence collection, and adjudication. Crucially, ADEPT enables a paradigm shift from consensus learning to ambiguity-driven
emotion reasoning. Since human affect exhibits inherent complexity and frequent co-occurrence of
emotions, we treat minority annotations as informative perceptual signals rather than discarding them as noise. Finally, we integrate Group Relative Policy Optimization (GRPO) with an Evidence Trust Gate to explicitly couple tool-usage behaviors with prediction quality and enforce evidence-grounded
reasoning. Experiments show that ADEPT improves primary
emotion accuracy in most settings while substantially improving minor
emotion characterization, producing explanations grounded in auditable acoustic and semantic evidence.
Read the full abstract ▾
All authors Esther Sun, Bo-Hao Su, Abinay Reddy Naini, Shinji Watanabe, Carlos Busso
Machine Learning
Mixture of Predefined Experts: Maximizing Data Usage on Vertical Federated Learning
Jon Irureta, Gorka Azkune, Jon Imaz et al. (+2 more)
Vertical
Federated Learning (VFL) has emerged as a critical paradigm for collaborative model training in
privacy-sensitive domains such as finance and healthcare. However, most existing VFL frameworks rely on the idealized assumption of full sample
alignment across participants, a premise that rarely holds in real-world scenarios. To bridge this gap, this work introduces Split-MoPE, a novel framework that integrates Split Learning with a specialized Mixture of Predefined Experts (MoPE) architecture. Unlike standard Mixture of Experts (MoE), where routing is learned dynamically, MoPE uses predefined experts to process specific data
alignments, effectively maximizing data usage during both training and inference without requiring full sample overlap. By leveraging pretrained encoders for target data domains, Split-MoPE achieves state-of-the-art performance in a single communication round, significantly reducing the communication footprint compared to multi-round end-to-end training. Furthermore, unlike existing proposals that address sample misalignment, this novel architecture provides inherent robustness against malicious or noisy participants and offers per-sample interpretability by quantifying each collaborator's contribution to each prediction. Extensive evaluations on
vision (CIFAR-10/100) and tabular (Breast Cancer Wisconsin)
datasets demonstrate that Split-MoPE consistently outperforms state-of-the-art systems such as
LASER and Vertical SplitNN, particularly in challenging scenarios with high data missingness.
Read the full abstract ▾
All authors Jon Irureta, Gorka Azkune, Jon Imaz, Aizea Lojo, Javier Fernandez-Marques
Machine Learning
Physics-Informed Laplace Neural Operator for Solving Partial Differential Equations
Heechang Kim, Qianying Cao, Hyomin Shin et al. (+3 more)
Neural operators have emerged as fast surrogate solvers for parametric partial differential equations (PDEs). However, purely data-driven models often require extensive training data and can generalize poorly, especially in small-data regimes and under unseen (out-of-distribution) input functions that are not represented in the training data. To address these limitations, we propose the Physics-Informed Laplace Neural Operator (PILNO), which enhances the Laplace Neural Operator (LNO) by
embedding governing physics into training through PDE, boundary condition, and initial condition residuals. To improve expressivity, we first introduce an Advanced LNO (ALNO) backbone that retains a pole-residue transient representation while replacing the steady-state branch with an FNO-style
Fourier multiplier. To make physics-informed training both data-efficient and robust, PILNO further leverages (i) virtual inputs: an unlabeled ensemble of input functions spanning a broad spectral range that provides abundant physics-only supervision and explicitly targets out-of-distribution (OOD) regimes; and (ii) temporal-causality weighting: a time-decaying reweighting of the physics residual that prioritizes early-time dynamics and stabilizes optimization for time-dependent PDEs. Across four representative
benchmarks – Burgers' equation, Darcy flow, a reaction-
diffusion system, and a forced KdV equation – PILNO consistently improves accuracy in small-data settings (e.g., N_train <= 27), reduces run-to-run variability across random seeds, and achieves stronger OOD generalization with respect to input function statistics than purely data-driven baselines.
Read the full abstract ▾
All authors Heechang Kim, Qianying Cao, Hyomin Shin, Seungchul Lee, George Em Karniadakis, Minseok Choi
Machine Learning
QTabGAN: A Hybrid Quantum-Classical GAN for Tabular Data Synthesis
Subhangi Kumari, Rakesh Achutha and Vignesh Sivaraman
Synthesizing realistic tabular data is challenging due to heterogeneous feature types and high dimensionality. We introduce QTabGAN, a hybrid
quantum-classical generative
adversarial framework for tabular data synthesis. QTabGAN is especially designed for settings where real data are scarce or restricted by
privacy constraints. The model exploits the expressive power of
quantum circuits to learn complex data distributions, which are then mapped to tabular features using classical neural networks. We evaluate QTabGAN on multiple classification and regression
datasets and
benchmark it against leading state-of-the-art generative models. Experiments show that QTabGAN achieves up to 54.07% improvement across various classification
datasets and evaluation metrics, thus establishing a scalable
quantum approach to tabular data synthesis and highlighting its potential for
quantum-assisted generative modelling.
Read the full abstract ▾
Machine Learning
SWING: Unlocking Implicit Graph Representations for Graph Random Features
Alessandro Manenti, Avinava Dubey, Arijit Sehanobish et al. (+2 more)
We propose SWING: Space Walks for Implicit Network Graphs, a new class of
algorithms for computations involving Graph Random Features on graphs given by implicit representations (i-graphs), where edge-weights are defined as bi-variate functions of feature vectors in the corresponding nodes. Those classes of graphs include several prominent examples, such as: $\epsilon$-neighborhood graphs, used on regular basis in machine learning. Rather than conducting walks on graphs' nodes, those methods rely on walks in continuous spaces, in which those graphs are embedded. To accurately and efficiently approximate original combinatorial calculations, SWING applies customized Gumbel-softmax sampling mechanism with linearized kernels, obtained via random features coupled with importance sampling techniques. This
algorithm is of its own interest. SWING relies on the deep connection between implicitly defined graphs and
Fourier analysis, presented in this paper. SWING is accelerator-friendly and does not require input graph materialization. We provide detailed analysis of SWING and complement it with thorough experiments on different classes of i-graphs.
Read the full abstract ▾
All authors Alessandro Manenti, Avinava Dubey, Arijit Sehanobish, Cesare Alippi, Krzysztof Choromanski
Machine Learning · Cross-listed from cs.CV
Channel-Aware Probing for Multi-Channel Imaging
Umar Marikkar, Syed Sameed Husain, Muhammad Awais et al. (+1 more)
Training and evaluating
vision encoders on Multi-Channel Imaging (MCI) data remains challenging as channel configurations vary across
datasets, preventing fixed-channel training and limiting reuse of pre-trained encoders on new channel settings. Prior work trains MCI encoders but typically evaluates them via full fine-tuning, leaving probing with frozen pre-trained encoders comparatively underexplored. Existing studies that perform probing largely focus on improving representations, rather than how to best leverage fixed representations for downstream tasks. Although the latter problem has been studied in other domains, directly transferring those strategies to MCI yields weak results, even worse than training from scratch. We therefore propose Channel-Aware Probing (CAP), which exploits the intrinsic inter-channel diversity in MCI
datasets by controlling feature flow at both the encoder and probe levels. CAP uses Independent Feature Encoding (IFE) to encode each channel separately, and Decoupled Pooling (DCP) to pool within channels before aggregating across channels. Across three MCI
benchmarks, CAP consistently improves probing performance over the default probing protocol, matches fine-tuning from scratch, and largely reduces the gap to full fine-tuning from the same MCI pre-trained checkpoints. Code can be found in https://github.com/umarikkar/CAP.
Read the full abstract ▾
All authors Umar Marikkar, Syed Sameed Husain, Muhammad Awais, Sara Atito
Machine Learning
Leverage-Weighted Conformal Prediction
Shreyas Fadnavis
Split conformal prediction provides distribution-free prediction intervals with finite-sample marginal coverage, but produces constant-width intervals that overcover in low-variance regions and undercover in high-variance regions. Existing adaptive methods require training auxiliary models. We propose Leverage-Weighted Conformal Prediction (LWCP), which weights nonconformity scores by a function of the statistical leverage – the diagonal of the hat
matrix – deriving adaptivity from the geometry of the design
matrix rather than from auxiliary model fitting. We prove that LWCP preserves finite-sample marginal validity for any weight function; achieves asymptotically optimal conditional coverage at essentially no width cost when heteroscedasticity factors through leverage; and recovers the form and width of classical prediction intervals under Gaussian assumptions while retaining distribution-free guarantees. We further establish that randomized leverage approximations preserve coverage exactly with controlled width perturbation, and that vanilla CP suffers a persistent, sample-size-independent conditional coverage gap that LWCP eliminates. The method requires no hyperparameters beyond the choice of weight function and adds negligible computational overhead to vanilla CP. Experiments on synthetic and real data confirm the theoretical predictions, demonstrating substantial reductions in conditional coverage disparity across settings.
Read the full abstract ▾
Machine Learning
Trust the uncertain teacher: distilling dark knowledge via calibrated uncertainty
Jeonghyun Kim, SooKyung Kim, Richeng Xuan et al. (+1 more)
The core of knowledge
distillation lies in transferring the teacher's rich 'dark knowledge'-subtle probabilistic patterns that reveal how classes are related and the distribution of uncertainties. While this idea is well established, teachers trained with conventional cross-
entropy often fail to preserve such signals. Their distributions collapse into sharp, overconfident peaks that appear decisive but are in fact brittle, offering little beyond the hard label or subtly hindering representation-level transfer. This overconfidence is especially problematic in high-cardinality tasks, where the nuances among many plausible classes matter most for guiding a compact student. Moreover, such brittle targets reduce robustness under distribution shift, leaving students vulnerable to miscalibration in real-world conditions. To address this limitation, we revisit
distillation from a distributional perspective and propose Calibrated Uncertainty
Distillation (CUD), a framework designed to make dark knowledge more faithfully accessible. Instead of uncritically adopting the teacher's overconfidence, CUD encourages teachers to reveal uncertainty where it is informative and guides students to learn from targets that are calibrated rather than sharpened certainty. By directly shaping the teacher's predictive distribution before transfer, our approach balances accuracy and calibration, allowing students to benefit from both confident signals on easy cases and structured uncertainty on hard ones. Across diverse
benchmarks, CUD yields students that are not only more accurate, but also more calibrated under shift and more reliable on ambiguous, long-tail inputs.
Read the full abstract ▾
All authors Jeonghyun Kim, SooKyung Kim, Richeng Xuan, Hyunsoo Cho
Machine Learning · Cross-listed from cs.RO
Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution
Rui Cai, Jun Guo, Xinze He et al. (+20 more)
In this report, we introduce Xiaomi-Robotics-0, an advanced
vision-
language-action (VLA) model optimized for high performance and fast and smooth real-time execution. The key to our method lies in a carefully designed training recipe and deployment strategy. Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment
robot trajectories and
vision-
language data, endowing it with broad and generalizable action-generation capabilities while avoiding catastrophic forgetting of the visual-semantic knowledge of the underlying pre-trained VLM. During post-training, we propose several techniques for training the VLA model for asynchronous execution to address the inference latency during real-
robot rollouts. During deployment, we carefully align the timesteps of consecutive predicted action chunks to ensure continuous and seamless real-time rollouts. We evaluate Xiaomi-Robotics-0 extensively in simulation
benchmarks and on two challenging real-
robot tasks that require precise and dexterous bimanual manipulation. Results show that our method achieves state-of-the-art performance across all simulation
benchmarks. Moreover, Xiaomi-Robotics-0 can roll out fast and smoothly on real
robots using a consumer-grade GPU, achieving high success rates and throughput on both real-
robot tasks. To facilitate future research, code and model checkpoints are open-sourced at https://xiaomi-robotics-0.github.io
Read the full abstract ▾
All authors Rui Cai, Jun Guo, Xinze He, Piaopiao Jin, Jie Li, Bingxuan Lin, Futeng Liu, Wei Liu, Fei Ma, Kun Ma, Feng Qiu, Heng Qu, Yifei Su, Qiao Sun, Dong Wang, Donghao Wang, Yunhong Wang, Rujie Wu, Diyun Xiang, Yu Yang, Hangjun Ye, Yuan Zhang, Quanyun Zhou
Machine Learning
Flow Matching from Viewpoint of Proximal Operators
Kenji Fukumizu, Wei Huang, Han Bao et al. (+2 more)
We discuss the dynamics of Optimal Transport Conditional Flow Matching (OT-CFM) under the
manifold hypothesis, where the support of the target distribution is a low dimensional
manifold. We first derive an exact proximal expression of the vector field, using an extended Brenier potential for the OT; the vector field can be expressed by a denoiser, which is presented as the proximal operator associated with the potential function. Based on this explicit proximal form of the vector field, we analyze the
manifold stability for perturbations to initial points, using the Lyapunov exponents of the dynamics. By analyzing the local behavior with the tools in convex analysis, we show that the dynamics contract linearly in the perturbation of the normal directions to the
manifold, while perturbations in the tangential directions are preserved; this implies the stability of the
manifold structure to the perturbations. To justify the theoretical arguments on the population OT, we also establish the convergence of minibatch OT-CFM with neural network learning to population OT-CFM under regularity and stability assumptions, with rate exponents governed by the intrinsic dimension of the data
manifold. Experiments on synthetic data, MNIST, and CIFAR-10 support the predicted
geometric behavior.
Read the full abstract ▾
All authors Kenji Fukumizu, Wei Huang, Han Bao, Shuntuo Xu, Nisha Chandramoorthy
Machine Learning · Cross-listed from cs.CR
Fool Me If You Can: On the Robustness of Binary Code Similarity Detection Models against Semantics-preserving Transformations
Jiyong Uhm, Minseok Kim, Michalis Polychronakis et al. (+1 more)
Binary code analysis plays an essential role in cybersecurity, facilitating reverse engineering to reveal the inner workings of programs in the absence of source code. Traditional approaches, such as static and dynamic analysis, extract valuable insights from stripped binaries, but often demand substantial expertise and manual effort. Recent advances in deep learning have opened promising opportunities to enhance binary analysis by capturing latent features and disclosing underlying code semantics. Despite the growing number of binary analysis models based on machine learning, their robustness to
adversarial code transformations at the binary level remains underexplored. We evaluate the robustness of deep learning models for the task of binary code similarity detection (BCSD) under semantics-preserving transformations. The unique nature of machine instructions presents distinct challenges compared to the typical input perturbations found in other domains. We introduce asmFooler, a system that evaluates the resilience of BCSD models using a diverse set of
adversarial code transformations that preserve functional semantics. We construct a
dataset of 9,565 binary variants from 620 baseline samples by applying eight semantics-preserving transformations across six representative BCSD models. Our major findings highlight several key insights: i) model robustness relies on the processing pipeline, including code pre-processing, architecture, and feature selection; ii)
adversarial transformation effectiveness is bounded by a budget shaped by model-specific constraints like input size and instruction expressive capacity; iii) well-crafted transformations can be highly effective with minimal perturbations; and iv) such transformations efficiently disrupt model decisions (e.g., misleading to false positives or false negatives) by focusing on semantically significant instructions.
Read the full abstract ▾
All authors Jiyong Uhm, Minseok Kim, Michalis Polychronakis, Hyungjoon Koo
Machine Learning · Cross-listed from stat.ML
A Regularization-Sharpness Tradeoff for Linear Interpolators
Qingyi Hu and Liam Hodgkinson
The rule of thumb regarding the relationship between the bias-variance tradeoff and model size plays a key role in classical machine learning, but is now well-known to break down in the overparameterized setting as per the double descent curve. In particular, minimum-norm interpolating
estimators can perform well, suggesting the need for new tradeoff in these settings. Accordingly, we propose a regularization-sharpness tradeoff for overparameterized linear regression with an $\ell^p$ penalty. Inspired by the interpolating information criterion, our framework decomposes the selection penalty into a regularization term (quantifying the
alignment of the regularizer and the interpolator) and a
geometric sharpness term on the interpolating
manifold (quantifying the effect of local perturbations), yielding a tradeoff analogous to bias-variance. Building on prior analyses that established this information criterion for ridge regularizers, this work first provides a general expression of the interpolating information criterion for $\ell^p$ regularizers where $p \ge 2$. Subsequently, we extend this to the LASSO interpolator with $\ell^1$ regularizer, which induces stronger sparsity. Empirical results on real-world
datasets with random
Fourier features and
polynomials validate our theory, demonstrating how the tradeoff terms can distinguish performant linear interpolators from weaker ones.
Read the full abstract ▾
Machine Learning
SLA2: Sparse-Linear Attention with Learnable Routing and Quantization-aware Training
Jintao Zhang, Haoxu Wang, Kai Jiang et al. (+6 more)
Sparse-Linear
Attention (SLA) combines sparse and linear
attention to accelerate
diffusion models and has shown strong performance in
video generation. However, (i) SLA relies on a heuristic split that assigns computations to the sparse or linear branch based on
attention-weight magnitude, which can be suboptimal. Additionally, (ii) after formally analyzing the
attention error in SLA, we identify a mismatch between SLA and a direct decomposition into sparse and linear
attention. We propose SLA2, which introduces (I) a learnable router that dynamically selects whether each
attention computation should use sparse or linear
attention, (II) a more faithful and direct sparse-linear
attention formulation that uses a learnable ratio to combine the sparse and linear
attention branches, and (III) a sparse + low-bit
attention design, where low-bit
attention is introduced via quantization-aware fine-tuning to reduce quantization error. Experiments show that on
video diffusion models, SLA2 can achieve 97%
attention sparsity and deliver an 18.6x
attention speedup while preserving generation quality.
Read the full abstract ▾
All authors Jintao Zhang, Haoxu Wang, Kai Jiang, Kaiwen Zheng, Youhe Jiang, Ion Stoica, Jianfei Chen, Jun Zhu, Joseph E. Gonzalez
Machine Learning
Uncovering spatial tissue domains and cell types in spatial omics through cross-scale profiling of cellular and genomic interactions
Rui Yan, Xiaohan Xing, Xun Wang et al. (+3 more)
Cellular identity and function are linked to both their intrinsic genomic makeup and extrinsic spatial context within the tissue microenvironment. Spatial transcriptomics (ST) offers an unprecedented opportunity to study this, providing in situ gene expression profiles at single-
cell resolution and illuminating the spatial and functional organization of
cells within tissues. However, a significant hurdle remains: ST data is inherently noisy, large, and structurally complex. This complexity makes it intractable for existing computational methods to effectively capture the interplay between spatial interactions and intrinsic genomic relationships, thus limiting our ability to discern critical biological patterns. Here, we present CellScape, a deep learning framework designed to overcome these limitations for high-performance ST data analysis and pattern discovery. CellScape jointly models cellular interactions in tissue space and genomic relationships among
cells, producing comprehensive representations that seamlessly integrate spatial signals with underlying gene regulatory mechanisms. This technique uncovers biologically informative patterns that improve spatial domain
segmentation and supports comprehensive spatial cellular analyses across diverse transcriptomics
datasets, offering an accurate and versatile framework for deep analysis and interpretation of ST data.w
Read the full abstract ▾
All authors Rui Yan, Xiaohan Xing, Xun Wang, Zixia Zhou, Md Tauhidul Islam, Lei Xing
Machine Learning · Cross-listed from cs.CV
Multi-Task Learning with Additive U-Net for Image Denoising and Classification
Vikram Lakkavalli and Neelam Sinha
We investigate additive skip fusion in U-Net architectures for image denoising and denoising-centric multi-task learning (MTL). By replacing concatenative skips with gated additive fusion, the proposed Additive U-Net (AddUNet) constrains shortcut capacity while preserving fixed feature dimensionality across depth. This structural regularization induces controlled encoder-decoder information flow and stabilizes joint optimization. Across single-task denoising and joint denoising-classification settings, AddUNet achieves competitive reconstruction performance with improved training stability. In MTL, learned skip weights exhibit systematic task-aware redistribution: shallow skips favor reconstruction, while deeper features support discrimination. Notably, reconstruction remains robust even under limited classification capacity, indicating implicit task decoupling through additive fusion. These findings show that simple constraints on skip connections act as an effective architectural regularizer for stable and scalable multi-task learning without increasing model complexity.
Read the full abstract ▾
Machine Learning
Unifying Model-Free Efficiency and Model-Based Representations via Latent Dynamics
Jashaswimalya Acharjee and Balaraman Ravindran
We present Unified Latent Dynamics (ULD), a novel reinforcement learning
algorithm that unifies the efficiency of model-free methods with the representational strengths of model-based approaches, without incurring planning overhead. By
embedding state-action pairs into a latent space in which the true value function is approximately linear, our method supports a single set of hyperparameters across diverse domains – from continuous control with low-dimensional and pixel inputs to high-dimensional Atari games. We prove that, under mild conditions, the fixed point of our
embedding-based temporal-difference updates coincides with that of a corresponding linear model-based value expansion, and we derive explicit error bounds relating
embedding fidelity to value approximation quality. In practice, ULD employs synchronized updates of encoder, value, and policy networks, auxiliary losses for short-horizon predictive dynamics, and reward-scale normalization to ensure stable learning under sparse rewards. Evaluated on 80 environments spanning Gym locomotion, DeepMind Control (proprioceptive and visual), and Atari, our approach matches or exceeds the performance of specialized model-free and general model-based baselines – achieving cross-domain competence with minimal tuning and a fraction of the parameter footprint. These results indicate that value-aligned latent representations alone can deliver the adaptability and sample efficiency traditionally attributed to full model-based planning.
Read the full abstract ▾
Machine Learning
Dual-Granularity Contrastive Reward via Generated Episodic Guidance for Efficient Embodied RL
Xin Liu, Yixuan Li, Yuhui Chen et al. (+3 more)
Designing suitable rewards poses a significant challenge in reinforcement learning (RL), especially for
embodied manipulation. Trajectory success rewards are suitable for human judges or model fitting, but the sparsity severely limits RL sample efficiency. While recent methods have effectively improved RL via dense rewards, they rely heavily on high-quality human-annotated data or abundant expert supervision. To tackle these issues, this paper proposes Dual-granularity contrastive reward via generated Episodic Guidance (DEG), a novel framework to seek sample-efficient dense rewards without requiring human annotations or extensive supervision. Leveraging the prior knowledge of large
video generation models, DEG only needs a small number of expert
videos for domain adaptation to generate dedicated task guidance for each RL episode. Then, the proposed dual-granularity reward that balances coarse-grained exploration and fine-grained matching, will guide the
agent to efficiently approximate the generated guidance
video sequentially in the contrastive self-supervised latent space, and finally complete the target task. Extensive experiments on 18 diverse tasks across both simulation and real-world settings show that DEG can not only serve as an efficient exploration stimulus to help the
agent quickly discover sparse success rewards, but also guide effective RL and stable policy convergence independently.
Read the full abstract ▾
All authors Xin Liu, Yixuan Li, Yuhui Chen, Yuxing Qin, Haoran Li, Dongbin Zhao
Machine Learning · Cross-listed from cs.CL
Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats
Pengxiang Zhao, Hui-Ling Zhen, Xing Li et al. (+10 more)
As LLMs scale, low-bit floating-point formats like MXFP and NVFP4 offer new opportunities for precision and efficiency. In this work, we evaluate HiFloat (HiF8 and HiF4), a family of formats tailored for Ascend NPUs. Through rigorous comparison across weight-activation and KV-cache tasks, we provide three key insights: (1) INT8 suits narrow-range data, while floating-point formats excel with high-variance data; (2) in 4-bit regimes, HiF4's hierarchical scaling prevents the accuracy collapse seen in integer formats; and (3) HiFloat is fully compatible with state-of-the-art post-training quantization frameworks. Overall, HiFloat provides a solution for high-efficiency LLM inference on NPUs.
Read the full abstract ▾
All authors Pengxiang Zhao, Hui-Ling Zhen, Xing Li, Han Bao, Weizhe Lin, Zhiyuan Yang, Manyi Zhang, Yuanyong Luo, Ziwei Yu, Xin Wang, Mingxuan Yuan, Xianzhi Yu, Zhenhua Dong