Machine Learning
Bridge Matching Sampler: Scalable Sampling via Generalized Fixed-Point Diffusion Matching
Denis Blessing, Lorenz Richter, Julius Berner et al. (+2 more)
Sampling from unnormalized densities using
diffusion models has emerged as a powerful paradigm. However, while recent approaches that use least-squares `matching' objectives have improved scalability, they often necessitate significant trade-offs, such as restricting prior distributions or relying on unstable optimization schemes. By generalizing these methods as special forms of fixed-point iterations rooted in Nelson's relation, we develop a new method that addresses these limitations, called Bridge Matching Sampler (BMS). Our approach enables learning a
stochastic transport map between arbitrary prior and target distributions with a single, scalable, and stable objective. Furthermore, we introduce a damped variant of this iteration that incorporates a regularization term to mitigate mode collapse and further stabilize training. Empirically, we demonstrate that our method enables sampling at unprecedented scales while preserving mode diversity, achieving state-of-the-art results on complex synthetic densities and high-dimensional molecular
benchmarks.
Read the full abstract ▾
All authors Denis Blessing, Lorenz Richter, Julius Berner, Egor Malitskiy, Gerhard Neumann
Machine Learning · Cross-listed from cs.CL
CIRCUS: Circuit Consensus under Uncertainty via Stability Ensembles
Swapnil Parekh
Every mechanistic
circuit carries an invisible asterisk: it reflects not just the model's computation, but the analyst's choice of
pruning threshold. Change that choice and the
circuit changes, yet current practice treats a single pruned subgraph as ground truth with no way to distinguish robust structure from threshold artifacts. We introduce CIRCUS, which reframes
circuit discovery as a problem of uncertainty over explanations. CIRCUS prunes one attribution graph under B configurations, assigns each edge an empirical inclusion frequency s(e) in [0,1] measuring how robustly it survives across the configuration family, and extracts a consensus
circuit of edges present in every view. This yields a principled core/contingent/noise decomposition (analogous to posterior model-inclusion indicators in
Bayesian variable selection) that separates robust structure from threshold-sensitive artifacts, with negligible overhead. On Gemma-2-2B and Llama-3.2-1B, consensus
circuits are 40x smaller than the union of all configurations while retaining comparable influence-flow explanatory power, consistently outperform influence-ranked and random baselines, and are confirmed causally relevant by activation patching.
Read the full abstract ▾
Machine Learning
Phys-Diff: A Physics-Inspired Latent Diffusion Model for Tropical Cyclone Forecasting
Lei Liu, Xiaoning Yu, Kang Chen et al. (+4 more)
Tropical cyclone (TC)
forecasting is critical for disaster warning and emergency response. Deep learning methods address computational challenges but often neglect physical relationships between TC attributes, resulting in predictions lacking physical consistency. To address this, we propose Phys-Diff, a physics-inspired latent
diffusion model that disentangles latent features into task-specific components (trajectory, pressure, wind speed) and employs cross-task
attention to introduce prior physics-inspired inductive biases, thereby
embedding physically consistent dependencies among TC attributes. Phys-Diff integrates
multimodal data including historical cyclone attributes, ERA5 reanalysis data, and FengWu forecast fields via a
Transformer encoder-decoder architecture, further enhancing
forecasting performance. Experiments demonstrate state-of-the-art performance on global and regional
datasets.
Read the full abstract ▾
All authors Lei Liu, Xiaoning Yu, Kang Chen, Jiahui Huang, Tengyuan Liu, Hongwei Zhao, Bin Li
Machine Learning
FastBUS: A Fast Bayesian Framework for Unified Weakly-Supervised Learning
Ziquan Wang, Haobo Wang, Ke Chen et al. (+2 more)
Machine Learning often involves various imprecise labels, leading to diverse weakly supervised settings. While recent methods aim for universal handling, they usually suffer from complex manual pre-work, ignore the relationships between associated labels, or are unable to batch process due to computational design flaws, resulting in long running times. To address these limitations, we propose a novel general framework that efficiently infers latent true label distributions across various weak supervisions. Our key idea is to express the label brute-force search process as a probabilistic transition of label variables, compressing diverse weakly supervised DFS tree structures into a shared
Bayesian network. From this, we derived a latent probability calculation
algorithm based on generalized belief propagation and proposed two joint acceleration strategies: 1) introducing a low-rank assumption to approximate the transition
matrix, reducing time complexity; 2) designing an end-to-end state
evolution module to learn batch-scale transition
matrices, facilitating multi-category batch processing. In addition, the equivalence of our method with the EM
algorithm in most scenarios is further demonstrated. Extensive experiments show that our method achieves SOTA results under most weakly supervised settings, and achieves up to hundreds of times faster acceleration in running time compared to other general methods.
Read the full abstract ▾
All authors Ziquan Wang, Haobo Wang, Ke Chen, Lei Feng, Gang Chen
Machine Learning · Cross-listed from cs.CV
Multimodal Adaptive Retrieval Augmented Generation through Internal Representation Learning
Ruoshuang Du, Xin Sun, Qiang Liu et al. (+4 more)
Visual Question Answering systems face reliability issues due to
hallucinations, where models generate answers misaligned with visual input or factual knowledge. While
Retrieval Augmented Generation frameworks mitigate this issue by incorporating external knowledge, static
retrieval often introduces irrelevant or conflicting content, particularly in visual RAG settings where visually similar but semantically incorrect evidence may be retrieved. To address this, we propose
Multimodal Adaptive RAG (MMA-RAG), which dynamically assesses the confidence in the internal knowledge of the model to decide whether to incorporate the retrieved external information into the generation process. Central to MMA-RAG is a decision classifier trained through a layer-wise analysis, which leverages joint internal visual and textual representations to guide the use of reverse image
retrieval. Experiments demonstrated that the model achieves a significant improvement in response performance in three VQA
datasets. Meanwhile, ablation studies highlighted the importance of internal representations in adaptive
retrieval decisions. In general, the experimental results demonstrated that MMA-RAG effectively balances external knowledge utilization and inference robustness in diverse
multimodal scenarios.
Read the full abstract ▾
All authors Ruoshuang Du, Xin Sun, Qiang Liu, Bowen Song, Zhongqi Chen, Weiqiang Wang, Liang Wang
Machine Learning
Trinity: A Scenario-Aware Recommendation Framework for Large-Scale Cold-Start Users
Wenhao Zheng, Wang Lu, Fangshuang Tang et al. (+4 more)
Early-stage users in a new scenario intensify cold-start challenges, yet prior works often address only parts of the problem through model architecture. Launching a new user experience to replace an established product involves sparse behavioral signals, low-engagement cohorts, and unstable model performance. We argue that effective recommendations require the synergistic integration of feature engineering, model architecture, and stable model updating. We propose Trinity, a framework embodying this principle. Trinity extracts valuable information from existing scenarios while ensuring predictive effectiveness and accuracy in the new scenario. In this paper, we showcase Trinity applied to a billion-user Microsoft product transition. Both offline and online experiments demonstrate that our framework achieves substantial improvements in addressing the combined challenge of new users in new scenarios.
Read the full abstract ▾
All authors Wenhao Zheng, Wang Lu, Fangshuang Tang, Yiyang Lu, Jun Yang, Pengcheng Xiong, Yulan Yan
Machine Learning
Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence
Quoc Minh Nguyen, Trung Le, Jing Wu et al. (+2 more)
Fine-tuning-as-a-service introduces a threat to Large
Language Models' safety when service providers fine-tune their models on poisoned user-submitted
datasets, a process known as harmful fine-tuning attacks. In this work, we show that by regularizing the gradient contribution of harmful samples encountered during fine-tuning, we can effectively mitigate the impact of harmful fine-tuning attacks. To this end, we introduce Antibody, a defense strategy that first ensures robust safety
alignment for the model before fine-tuning, and then applies a safety-preservation learning
algorithm during fine-tuning. Specifically, in the
alignment stage before fine-tuning, we propose optimizing the model to be in a flat loss region with respect to harmful samples, which makes the safety
alignment more resilient to subsequent harmful fine-tuning. Then, in the fine-tuning stage, we design a fine-tuning
algorithm that applies a weighting scheme to all samples in each training batch to inhibit the model from learning from harmful samples while encouraging learning from benign samples. Experimental results demonstrate that Antibody successfully mitigates harmful fine-tuning attacks while boosting fine-tuning performance on the user-submitted
dataset.
Read the full abstract ▾
All authors Quoc Minh Nguyen, Trung Le, Jing Wu, Anh Tuan Bui, Mehrtash Harandi
Machine Learning
ESENSC: A Polynomial-Time Axiomatic Alternative to SHAP
Kazuhiro Hiraki, Shinichi Ishihara, Takumi Kongo et al. (+1 more)
We propose ESENSC, a computationally efficient and theoretically grounded alternative to SHAP. ESENSC is a deterministic, closed-form attribution rule that requires only linearly many coalition-value evaluations, satisfies the null-player property, and admits an axiomatic characterization based on efficiency, a restricted form of differential marginality, and explicit computational constraints.Empirically, on tabular prediction tasks using neural network and XGBoost models, ESENSC shows close agreement with exact SHAP in both attribution magnitude and feature rankings while requiring substantially less computation. Under the evaluated settings, it compares favorably with widely used sampling-based SHAP approximations. While exact SHAP requires exponentially many coalition evaluations, ESENSC scales linearly with the number of features. These results show that ESENSC offers a favorable trade-off between agreement with exact SHAP and computational cost, providing a practical and theoretically grounded alternative for feature attribution in high-dimensional applications.
Read the full abstract ▾
All authors Kazuhiro Hiraki, Shinichi Ishihara, Takumi Kongo, Junnosuke Shino
Machine Learning · Cross-listed from cs.CV
ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models
Riccardo de Lutio, Tobias Fischer, Yen-Yu Chang et al. (+7 more)
Per-scene optimization methods such as 3D Gaussian Splatting provide state-of-the-art novel view synthesis quality but extrapolate poorly to under-observed areas. Methods that leverage generative priors to correct artifacts in these areas hold promise but currently suffer from two shortcomings. The first is scalability, as existing methods use image
diffusion models or bidirectional
video models that are limited in the number of views they can generate in a single pass (and thus require a costly iterative
distillation process for consistency). The second is quality itself, as generators used in prior work tend to produce outputs that are inconsistent with existing scene content and fail entirely in completely unobserved regions. To solve these, we propose a two-stage pipeline that leverages two key insights. First, we train a powerful bidirectional generative model with a novel opacity mixing strategy that encourages consistency with existing observations while retaining the model's ability to extrapolate novel content in unseen areas. Second, we distill it into a causal auto-regressive model that generates hundreds of frames in a single pass. This model can directly produce novel views or serve as pseudo-supervision to improve the underlying 3D representation in a simple and highly efficient manner. We evaluate our method extensively and demonstrate that it can generate plausible reconstructions in scenarios where existing approaches fail completely. When measured on commonly benchmarked
datasets, we outperform all existing baselines by a wide margin, exceeding prior state-of-the-art methods by 1-3 dB PSNR.
Read the full abstract ▾
All authors Riccardo de Lutio, Tobias Fischer, Yen-Yu Chang, Yuxuan Zhang, Jay Zhangjie Wu, Xuanchi Ren, Tianchang Shen, Katarina Tothova, Zan Gojcic, Haithem Turki
Machine Learning
Heaviside Low-Rank Support Matrix Machine
Xianchao Xiu, Shenghao Sun, Xinrong Li et al. (+1 more)
Support
matrix machine (SMM) is an emerging classification framework that directly handles
matrix-structured observations, thereby avoiding the spatial correlations destroyed by vectorization. However, most existing SMM variants rely on convex or nonconvex surrogate loss functions, which may lead to high sensitivity to noise. To address this issue, we propose a novel Heaviside low-rank SMM model called HL-SMM, which leverages the Heaviside loss instead of the common hinge or ramp losses for robustness. Moreover, the low-rank constraint is adopted to accurately characterize the inherent global structure. In theory, we analyze the Karush-Kuhn-Tucker (KKT) points and rigorously prove the sufficient and necessary conditions. In
algorithms, we develop an effective proximal alternating minimization (PAM) scheme, where all subproblems have closed-form solutions. Extensive experiments on
benchmark datasets validate that the proposed HL-SMM achieves superior classification accuracy and robustness compared to state-of-the-art methods.
Read the full abstract ▾
All authors Xianchao Xiu, Shenghao Sun, Xinrong Li, Jiyuan Tao
Machine Learning
Dynamic Spatio-Temporal Graph Neural Network for Early Detection of Pornography Addiction in Adolescents Based on Electroencephalogram Signals
Achmad Ardani Prasha, Clavino Ourizqi Rachmadi, Sabrina Laila Mutiara et al. (+3 more)
Adolescent pornography addiction requires early detection based on objective neurobiological biomarkers because self-report is prone to subjective bias due to social stigma. Conventional machine learning has not been able to model dynamic functional connectivity of the
brain that fluctuates temporally during addictive stimulus exposure. This study proposes a state-of-the-art Dynamic Spatio-Temporal Graph Neural Network (DST-GNN) that integrates Phase Lag Index (PLI)-based Graph
Attention Network (GAT) for spatial modeling and Bidirectional Gated Recurrent Unit (BiGRU) for temporal dynamics. The
dataset consists of 14 adolescents (7 addicted, 7 healthy) with 19-channel
EEG across 9 experimental conditions. Leave-One-Subject-Out Cross Validation (LOSO-CV) evaluation shows F1-Score of 71.00%$\pm$12.10% and recall of 85.71%, a 104% improvement compared to baseline. Ablation study confirms temporal contribution of 21% and PLI graph construction of 57%. Frontal-central regions (Fz, Cz, C3, C4) are identified as dominant biomarkers with Beta contribution of 58.9% and Hjorth of 31.2%, while Cz-T7 connectivity is consistent as a trait-level biomarker for objective screening.
Read the full abstract ▾
All authors Achmad Ardani Prasha, Clavino Ourizqi Rachmadi, Sabrina Laila Mutiara, Hilman Syachr Ramadhan, Chareyl Reinalyta Borneo, Saruni Dwiasnati
Machine Learning
Analyzing Physical Adversarial Example Threats to Machine Learning in Election Systems
Khaleque Md Aashiq Kamal, Surya Eada, Aayushi Verma et al. (+4 more)
Developments in the machine learning voting domain have shown both promising results and risks. Trained models perform well on ballot classification tasks (> 99% accuracy) but are at risk from
adversarial example attacks that cause misclassifications. In this paper, we analyze an attacker who seeks to deploy
adversarial examples against machine learning ballot classifiers to compromise a U.S. election. We first derive a probabilistic framework for determining the number of
adversarial example ballots that must be printed to flip an election, in terms of the probability of each candidate winning and the total number of ballots cast. Second, it is an open question as to which type of
adversarial example is most effective when physically printed in the voting domain. We analyze six different types of
adversarial example attacks: l_infinity-APGD, l2-APGD, l1-APGD, l0 PGD, l0 + l_infinity PGD, and l0 + sigma-map PGD. Our experiments include physical realizations of 144,000
adversarial examples through printing and scanning with four different machine learning models. We empirically demonstrate an analysis gap between the physical and digital domains, wherein attacks most effective in the digital domain (l2 and l_infinity) differ from those most effective in the physical domain (l1 and l2, depending on the model). By unifying a probabilistic election framework with digital and physical
adversarial example evaluations, we move beyond prior close race analyses to explicitly quantify when and how
adversarial ballot manipulation could alter outcomes.
Read the full abstract ▾
All authors Khaleque Md Aashiq Kamal, Surya Eada, Aayushi Verma, Subek Acharya, Adrian Yemin, Benjamin Fuller, Kaleel Mahmood
Machine Learning
Benchmarking Few-shot Transferability of Pre-trained Models with Improved Evaluation Protocols
Xu Luo, Ji Zhang, Lianli Gao et al. (+2 more)
Few-shot transfer has been revolutionized by stronger pre-trained models and improved adaptation
algorithms.However, there lacks a unified, rigorous evaluation protocol that is both challenging and realistic for real-world usage. In this work, we establish FEWTRANS, a comprehensive
benchmark containing 10 diverse
datasets, and propose the Hyperparameter Ensemble (HPE) protocol to overcome the "validation set illusion" in data-scarce regimes. Our empirical findings demonstrate that the choice of pre-trained model is the dominant factor for performance, while many sophisticated transfer methods offer negligible practical advantages over a simple full-parameter fine-tuning baseline. To explain this surprising effectiveness, we provide an in-depth mechanistic analysis showing that full fine-tuning succeeds via distributed micro-adjustments and more flexible reshaping of high-level semantic presentations without suffering from overfitting. Additionally, we quantify the performance collapse of
multimodal models in specialized domains as a result of linguistic rarity using adjusted Zipf frequency scores. By releasing FEWTRANS, we aim to provide a rigorous "ruler" to streamline reproducible advances in few-shot transfer learning research. We make the FEWTRANS
benchmark publicly available at https://github.com/Frankluox/FewTrans.
Read the full abstract ▾
All authors Xu Luo, Ji Zhang, Lianli Gao, Heng Tao Shen, Jingkuan Song
Machine Learning · Cross-listed from cs.IT
Wireless Power Control Based on Large Language Models
Jiacheng Wang, Yucheng Sheng, Le Liang et al. (+2 more)
This paper investigates the power control problem in wireless networks by repurposing pre-trained large
language models (LLMs) as relational
reasoning backbones. In hyper-connected interference environments, traditional optimization methods face high computational cost, while standard message passing neural networks suffer from aggregation bottlenecks that can obscure critical high-interference structures. In response, we propose PC-LLM, a physics-informed framework that augments a pre-trained LLM with an interference-aware
attention bias. The proposed bias tuning mechanism injects the physical channel gain
matrix directly into the self-
attention scores, enabling explicit fusion of wireless
topology with pre-trained relational priors without retraining the backbone from scratch. Extensive experiments demonstrate that PC-LLM consistently outperforms both traditional optimization methods and state-of-the-art graph neural network baselines, while exhibiting exceptional zero-shot generalization to unseen environments. We further observe that
topology-relevant relational
reasoning is concentrated in shallow layers, whereas deeper layers encode task-irrelevant semantic noise. Motivated by this finding, we develop a lightweight adaptation strategy that reduces model depth by 50%, significantly lowering inference cost while preserving state-of-the-art spectral efficiency.
Read the full abstract ▾
All authors Jiacheng Wang, Yucheng Sheng, Le Liang, Hao Ye, Shi Jin
Machine Learning
Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Training
Xi Wang, Wenbo Lu and Shengjie Wang
Generative Flow Networks (GFlowNets) enable fine-tuning large
language models to approximate reward-proportional posteriors, but they remain prone to mode collapse, manifesting as prefix collapse and length bias. We attribute this to two factors: (i) weak credit assignment to early prefixes, and (ii) biased replay that induces a shifted, non-representative training flow distribution. We propose Rooted absorbed prefix Trajectory Balance RapTB, an objective that anchors subtrajectory supervision at the root and propagates terminal rewards to intermediate prefixes via absorbed suffix-based backups, providing dense prefix-level learning signals. To mitigate replay-induced distribution shift, we further introduce SubM, a submodular replay refresh strategy that promotes both high reward and diversity. Empirically, on tasks such as molecule generation with LLM using SMILES strings, RapTB combined with SubM consistently improves optimization performance and molecular diversity while preserving high validity. The code is released on https://github.com/ComDec/ChemGFN.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Neurosymbolic Learning for Advanced Persistent Threat Detection under Extreme Class Imbalance
Quhura Fathima, Neda Moghim, Mostafa Taghizade Firouzjaee et al. (+3 more)
The growing deployment of Internet of Things (IoT) devices in smart cities and industrial environments increases vulnerability to stealthy, multi-stage advanced persistent threats (APTs) that exploit wireless communication. Detection is challenging due to severe class imbalance in network traffic, which limits the effectiveness of traditional deep learning approaches and their lack of explainability in classification decisions. To address these challenges, this paper proposes a neurosymbolic architecture that integrates an optimized BERT model with logic tensor networks (LTN) for explainable APT detection in wireless IoT networks. The proposed method addresses the challenges of mobile IoT environments through efficient feature encoding that transforms network flow data into BERT-compatible sequences while preserving temporal dependencies critical for APT stage identification. Severe class imbalance is mitigated using focal loss, hierarchical classification that separates normal traffic detection from attack categorization, and adaptive sampling strategies. Evaluation on the SCVIC-APT2021
dataset demonstrates an operationally viable binary classification F1 score of 95.27% with a false positive rate of 0.14%, and a 76.75% macro F1 score for multi-class attack categorization. Furthermore, a novel explainability analysis statistically validates the importance of distinct network features. These results demonstrate that neurosymbolic learning enables high-performance, interpretable, and operationally viable APT detection for IoT network monitoring architectures.
Read the full abstract ▾
All authors Quhura Fathima, Neda Moghim, Mostafa Taghizade Firouzjaee, Christo K. Thomas, Ross Gore, Walid Saad
Machine Learning
ROKA: Robust Knowledge Unlearning against Adversaries
Jinmyeong Shin, Joshua Tapia, Nicholas Ferreira et al. (+3 more)
The need for machine unlearning is critical for data
privacy, yet existing methods often cause Knowledge Contamination by unintentionally damaging related knowledge. Such a degraded model performance after unlearning has been recently leveraged for new inference and
backdoor attacks. Most studies design
adversarial unlearning requests that require poisoning or duplicating training data. In this study, we introduce a new unlearning-induced attack model, namely indirect unlearning attack, which does not require data manipulation but exploits the consequence of knowledge contamination to perturb the model accuracy on security-critical predictions. To mitigate this attack, we introduce a theoretical framework that models neural networks as Neural Knowledge Systems. Based on this, we propose ROKA, a robust unlearning strategy centered on Neural Healing. Unlike conventional unlearning methods that only destroy information, ROKA constructively rebalances the model by nullifying the influence of forgotten data while strengthening its conceptual neighbors. To the best of our knowledge, our work is the first to provide a theoretical guarantee for knowledge preservation during unlearning. Evaluations on various large models, including
vision transformers, multi-modal models, and large
language models, show that ROKA effectively unlearns targets while preserving, or even enhancing, the accuracy of retained data, thereby mitigating the indirect unlearning attacks.
Read the full abstract ▾
All authors Jinmyeong Shin, Joshua Tapia, Nicholas Ferreira, Gabriel Diaz, Moayed Daneshyari, Hyeran Jeon
Machine Learning
Efficient Decoder Scaling Strategy for Neural Routing Solvers
Qing Luo, Fu Luo, Ke Li et al. (+1 more)
Construction-based neural routing solvers, typically composed of an encoder and a decoder, have emerged as a promising approach for solving vehicle routing problems. While recent studies suggest that shifting parameters from the encoder to the decoder enhances performance, most works restrict the decoder size to 1-3M parameters, leaving the effects of scaling largely unexplored. To address this gap, we conduct a systematic study comparing two distinct strategies: scaling depth versus scaling width. We synthesize these strategies to construct a suite of 12 model configurations, spanning a parameter range from 1M to ~150M, and extensively evaluate their scaling behaviors across three critical dimensions: parameter efficiency, data efficiency, and compute efficiency. Our empirical results reveal that parameter count is insufficient to accurately predict the model performance, highlighting the critical and distinct roles of model depth (layer count) and width (
embedding dimension). Crucially, we demonstrate that scaling depth yields superior performance gains to scaling width. Based on these findings, we provide and experimentally validate a set of design principles for the efficient allocation of parameters and compute resources to enhance the model performance.
Read the full abstract ▾
All authors Qing Luo, Fu Luo, Ke Li, Zhenkun Wang
Machine Learning
Weight Updates as Activation Shifts: A Principled Framework for Steering
Dyah Adila, John Cooper, Alexander Yun et al. (+2 more)
Activation steering promises to be an extremely parameter-efficient form of adaptation, but its effectiveness depends on critical design choices – such as intervention location and parameterization – that currently rely on empirical heuristics rather than a principled foundation. We establish a first-order equivalence between activation-space interventions and weight-space updates, deriving the conditions under which activation steering can replicate fine-tuning behavior. This equivalence yields a principled framework for steering design and identifies the post-block output as a theoretically-backed and highly expressive intervention site. We further explain why certain intervention locations outperform others and show that weight updates and activation updates play distinct, complementary functional roles. This analysis motivates a new approach – joint adaptation – that trains in both spaces simultaneously. Our post-block steering achieves accuracy within 0.2%-0.9%$ of full-parameter tuning, on average across tasks and models, while training only 0.04% of model parameters. It consistently outperforms prior activation steering methods such as ReFT and PEFT approaches including LoRA, while using significantly fewer parameters. Finally, we show that joint adaptation often surpasses the performance ceilings of weight and activation updates in isolation, introducing a new paradigm for efficient model adaptation.
Read the full abstract ▾
All authors Dyah Adila, John Cooper, Alexander Yun, Avi Trost, Frederic Sala
Machine Learning
Physics-Aware Learnability: From Set-Theoretic Independence to Operational Constraints
Jeongho Bang and Kyoungho Cho
Beyond binary classification, learnability can become a logically fragile notion: in EMX, even the class of all finite subsets of $[0,1]$ is learnable in some models of ZFC and not in others. We argue the paradox is operational. The standard definitions quantify over arbitrary set-theoretic learners that implicitly assume non-operational resources (infinite precision, unphysical data access, and non-representable outputs). We introduce physics-aware learnability (PL), which defines the learnability relative to an explicit access model – a family of admissible physical protocols. Finite-precision coarse-graining reduces continuum EMX to a countable problem, via an exact pushforward/pullback reduction that preserves the EMX objective, making the independence example provably learnable with explicit $(\epsilon,\delta)$ sample complexity. For
quantum data, admissible learners are exactly POVMs on $d$ copies, turning sample size into copy complexity and yielding Helstrom(-type) lower bounds. For finite no-signaling and
quantum models, PL feasibility becomes linear or semidefinite and is therefore decidable.
Read the full abstract ▾
Machine Learning
Exact and Asymptotically Complete Robust Verifications of Neural Networks via Ising Solvers
Wenxin Li, Wenchao Liu, Anzhe Jiang et al. (+6 more)
We present an Ising-compatible framework for formal neural-network robustness verification under bounded input perturbations. For piecewise activations (both continuous piecewise-linear and discrete piecewise-constant), the Exact Logarithmic Piecewise Model (Log-PW) provides an exact, sound, and complete formulation with a state-optimal logarithmic encoding, reducing the binary variables per
neuron from linear $O(\kappa)$ to information-theoretically minimal logarithmic complexity $O(\lceil \log_2 \kappa \rceil)$ via logarithmic SOS2 (Log-PWL) and SOS1 (Log-PWC) constraints. For general element-wise activations, the Asymptotic Step-Envelope Model (Step-Env) leverages Log-PWC to construct sound, refinable endpoint enclosures. An exact monotonic reduction based on Jordan decomposition further extends this framework to arbitrary non-monotonic activations. We further develop a hybrid Benders solver with output-sensitive iteration bounds, leveraging a Helly-based conflict compression theory to avoid generic exponential cut explosion. In particular, for shallow networks with fixed input dimension, this establishes an unconditionally
polynomial Benders iteration complexity in network size. Interval
pruning, certificate transfer for pruned networks, and layerwise classical–Ising partitioning further reduce
spin requirements. Experiments demonstrate exact certification fidelity for piecewise-linear networks and near-reference accuracy for non-linear activations under compact
spin budgets. On MNIST
benchmarks, Log-PW halves binary decision
spins over One-Hot formulations, while Helly-based cuts achieve up to a $13.41\times$ speedup and an $87.6\%$ reduction in master queries over standard baselines.
Read the full abstract ▾
All authors Wenxin Li, Wenchao Liu, Anzhe Jiang, Weihao Li, Chuan Wang, Qi Gao, Yin Ma, Hai Wei, Kai Wen
Machine Learning
USE: Uncertainty Structure Estimation for Robust Semi-Supervised Learning
Tsao-Lun Chen, Chien-Liang Liu, Tzu-Ming Harry Hsu et al. (+4 more)
In this study, a novel idea, Uncertainty Structure Estimation (USE), a lightweight,
algorithm-agnostic procedure that emphasizes the often-overlooked role of unlabeled data quality is introduced for Semi-supervised learning (SSL). SSL has achieved impressive progress, but its reliability in deployment is limited by the quality of the unlabeled pool. In practice, unlabeled data are almost always contaminated by out-of-distribution (OOD) samples, where both near-OOD and far-OOD can negatively affect performance in different ways. We argue that the bottleneck does not lie in algorithmic design, but rather in the absence of principled mechanisms to assess and curate the quality of unlabeled data. The proposed USE trains a proxy model on the labeled set to compute
entropy scores for unlabeled samples, and then derives a threshold, via statistical comparison against a reference distribution, that separates informative (structured) from uninformative (structureless) samples. This enables assessment as a preprocessing step, removing uninformative or harmful unlabeled data before SSL training begins. Through extensive experiments on imaging (CIFAR-100) and NLP (Yelp Review) data, it is evident that USE consistently improves accuracy and robustness under varying levels of OOD contamination. Thus, it can be concluded that the proposed approach reframes unlabeled data quality control as a structural assessment problem, and considers it as a necessary component for reliable and efficient SSL in realistic mixed-distribution environments.
Read the full abstract ▾
All authors Tsao-Lun Chen, Chien-Liang Liu, Tzu-Ming Harry Hsu, Tai-Hsien Wu, Chi-Cheng Fu, Han-Yi E. Chou, Shun-Feng Su
Machine Learning
TENG-BC: Time-Evolving Natural Gradient for High-Accuracy Neural PDE Solvers with General Boundary Conditions
Hongjie Jiang and Di Luo
Accurately solving time-dependent partial differential equations (PDEs) with neural networks remains challenging due to long-time error accumulation and the difficulty of enforcing general boundary conditions. We introduce TENG-BC, a high-precision neural PDE solver based on the Time-Evolving Natural Gradient, designed to perform under generic boundary constraints. At each time step, TENG-BC performs a boundary-aware optimization that jointly enforces interior dynamics and boundary conditions, accommodating Dirichlet, Neumann, Robin, and mixed types within a unified framework. This formulation admits a natural-gradient interpretation, enabling stable time
evolution without delicate penalty tuning. Across
benchmarks over
diffusion, transport, and nonlinear PDEs with various boundary conditions, TENG-BC achieves solver-level accuracy under comparable sampling budgets, outperforming conventional solvers and physics-informed neural network (PINN) baselines.
Read the full abstract ▾
Machine Learning
Hereditary Geometric Meta-RL: Nonlocal Generalization via Task Symmetries
Paul Nitschke and Shahriar Talebi
Meta-Reinforcement Learning (Meta-RL) commonly generalizes via smoothness in the task encoding. While this enables local generalization around each training task, it requires dense coverage of the task space and leaves richer task space structure untapped. In response, we develop a
geometric perspective that endows the task space with a "hereditary geometry" induced by the inherent
symmetries of the underlying system. Concretely, the
agent reuses a policy learned at the train time by transforming states and actions through actions of a Lie group. This converts Meta-RL into
symmetry discovery rather than smooth extrapolation, enabling the
agent to generalize to wider regions of the task space. We show that when the task space is inherited from the
symmetries of the underlying system, the task space embeds into a subgroup of those
symmetries whose actions are linearizable, connected, and compact–properties that enable efficient learning and inference at the test time. To learn these structures, we develop a differential
symmetry discovery method. This collapses functional invariance constraints and thereby improves numerical stability and sample efficiency over functional approaches. Empirically, on a two-dimensional
navigation task, our method efficiently recovers the ground-truth
symmetry and generalizes across the entire task space, while a common baseline generalizes only near training tasks.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SD
Fine-grained Soundscape Control for Augmented Hearing
Seunghyun Oh, Malek Itani, Aseem Gauri et al. (+1 more)
Hearables are becoming ubiquitous, yet their sound controls remain blunt: users can either enable global noise suppression or focus on a single target sound. Real-world acoustic scenes, however, contain many simultaneous sources that users may want to adjust independently. We introduce Aurchestra, the first system to provide fine-grained, real-time soundscape control on resource-constrained hearables. Our system has two key components: (1) a dynamic interface that surfaces only active sound classes and (2) a real-time, on-device multi-output extraction network that generates separate streams for each selected class, achieving robust performance for upto 5 overlapping target sounds, and letting users mix their environment by customizing per-class volumes, much like an audio engineer mixes tracks. We optimize the model architecture for multiple compute-limited platforms and demonstrate real-time performance on 6 ms streaming audio chunks. Across real-world environments in previously unseen indoor and outdoor scenarios, our system enables expressive per-class sound control and achieves substantial improvements in target-class enhancement and interference suppression. Our results show that the world need not be heard as a single, undifferentiated stream: with Aurchestra, the soundscape becomes truly programmable.
Read the full abstract ▾
All authors Seunghyun Oh, Malek Itani, Aseem Gauri, Shyamnath Gollakota
Machine Learning · Cross-listed from physics.geo-ph
Dual-space posterior sampling for Bayesian inference in constrained inverse problems
Ali Siahkoohi, Kamal Aghazade and Ali Gholami
Inverse problems constrained by partial differential equations are often ill-conditioned due to noisy, incomplete data or inherent non-uniqueness. A prominent example is full waveform inversion (FWI), which estimates Earth's subsurface properties by fitting seismic measurements subject to the wave equation, where ill-conditioning stems from noisy, band-limited, finite-aperture measurements and complex geological structures. A
Bayesian framework describes the solution more comprehensively: instead of a single estimate, a posterior distribution of plausible solutions characterizes the non-uniqueness and can be sampled to quantify uncertainty. However, no clear procedure exists for translating hard physical constraints, such as the wave equation, into priors amenable to existing sampling techniques. We address this by sampling the posterior in the dual space via an augmented Lagrangian formulation, which converts hard constraints into penalties suited to sampling
algorithms while enforcing them progressively through multiplier updates, so they are satisfied in the limit. We integrate the alternating direction method of multipliers (ADMM) with Stein variational gradient descent (SVGD), a particle-based sampler: the constraint is relaxed at each iteration and the multiplier updates progressively enforce its satisfaction. This enables posterior sampling under hard constraints while inheriting the favorable conditioning of dual-space solvers, where partial constraint relaxation permits productive updates even when the current model is far from the true solution. We validate the method on a stylized Rosenbrock conditional inference problem and on frequency-domain FWI for a Gaussian anomaly model and the Marmousi II
benchmark, demonstrating physically consistent uncertainty estimates and posterior contraction with increasing data coverage.
Read the full abstract ▾