Machine Learning · Cross-listed from eess.AS
[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmetic
Kwanghee Choi, Eunjung Yeo, Cheol Jun Cho et al. (+2 more)
Self-supervised
speech models (S3Ms) are known to encode rich phonetic information, yet how this information is structured remains underexplored. We conduct a comprehensive study across 96
languages to analyze the underlying structure of S3M representations, with particular
attention to phonological vectors. We first show that there exist linear directions within the model's representation space that correspond to phonological features. We further demonstrate that the scale of these phonological vectors correlate to the degree of acoustic realization of their corresponding phonological features in a continuous manner. For example, the difference between [d] and [t] yields a voicing vector: adding this vector to [p] produces [b], while scaling it results in a continuum of voicing. Together, these findings indicate that S3Ms encode
speech using phonologically interpretable and compositional vectors, demonstrating phonological vector arithmetic. All code and interactive demos are available at https://github.com/juice500ml/phonetic-arithmetic .
Read the full abstract ▾
All authors Kwanghee Choi, Eunjung Yeo, Cheol Jun Cho, David Harwath, David R. Mortensen
Machine Learning
HEHRGNN: A Unified Embedding Model for Knowledge Graphs with Hyperedges and Hyper-Relational Edges
Rajesh Rajagopalamenon and Unnikrishnan Cheramangalath
Knowledge Graph(KG) has gained traction as a machine-readable organization of real-world knowledge for analytics using artificial intelligence systems. Graph Neural Network(GNN), is proven to be an effective KG
embedding technique that enables various downstream tasks like link prediction, node classification, and graph classification. The focus of research in both KG
embedding and GNNs has been mostly oriented towards simple graphs with binary relations. However, real-world knowledge bases have a significant share of complex and n-ary facts that cannot be represented by binary edges. More specifically, real-world knowledge bases are often a mix of two types of n-ary facts - (i) that require hyperedges and (ii) that require hyper-relational edges. Though there are research efforts catering to these n-ary fact types, they are pursued independently for each type. We propose $H$yper$E$dge $H$yper-$R$elational edge $GNN$(HEHRGNN), a unified
embedding model for n-ary relational KGs with both hyperedges and hyper-relational edges. The two main components of the model are i)HEHR unified fact representation format, and ii)HEHRGNN encoder, a GNN-based encoder with a novel message propagation model capable of capturing complex graph structures comprising both hyperedges and hyper-relational edges. The experimental results of HEHRGNN on link prediction tasks show its effectiveness as a unified
embedding model, with inductive prediction capability, for link prediction across real-world
datasets having different types of n-ary facts. The model also shows improved link prediction performance over baseline models for hyperedge and hyper-relational
datasets.
Read the full abstract ▾
Machine Learning · Cross-listed from q-fin.RM
Could Large Language Models work as Post-hoc Explainability Tools in Credit Risk Models?
Wenxi Geng, Dingyuan Liu, Liya Li et al. (+1 more)
Large
language models (LLMs) have shown promise in translating model-based explanations into human-readable narratives. This study evaluates whether LLMs can serve as post-hoc explainability interfaces for credit risk models, focusing on their ability to preserve feature-importance rankings and generate autonomous explanations. Using a LendingClub
dataset, we compare LLM outputs with SHAP and coefficient-based attributions on three major LLMs, including GPT-4-turbo, Claude-Sonnet-4.5, and Gemini-2.5-Flash. Results indicate that LLMs reliably reproduce reference rankings under controlled
prompts but show limited
alignment when generating explanations autonomously. These findings suggest that LLMs are best deployed as narrative interfaces rather than substitutes for formal attribution methods in credit risk governance.
Read the full abstract ▾
All authors Wenxi Geng, Dingyuan Liu, Liya Li, Yiqing Wang
Machine Learning · Cross-listed from cs.CV
SceneTok: A Compressed, Diffusable Token Space for 3D Scenes
Mohammad Asim, Christopher Wewer and Jan Eric Lenssen
We present SceneTok, a novel tokenizer for encoding view sets of scenes into a compressed and diffusable set of unstructured
tokens. Existing approaches for 3D scene representation and generation commonly use 3D data structures or view-aligned fields. In contrast, we introduce the first method that encodes scene information into a small set of permutation-invariant
tokens that is disentangled from the spatial grid. The scene
tokens are predicted by a multi-view tokenizer given many context views and rendered into novel views by employing a light-weight rectified flow decoder. We show that the compression is 1-3 orders of magnitude stronger than for other representations while still reaching state-of-the-art reconstruction quality. Further, our representation can be rendered from novel trajectories, including ones deviating from the input trajectory, and we show that the decoder gracefully handles uncertainty. Finally, the highly-compressed set of unstructured latent scene
tokens enables simple and efficient scene generation in 5 seconds, achieving a much better quality-speed trade-off than previous paradigms.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Federated Measurement of Demographic Disparities from Quantile Sketches
Arthur Charpentier, Agathe Fernandes Machado, Olivier Côté et al. (+1 more)
Many
fairness goals are defined at a population level that misaligns with siloed data collection, which remains unsharable due to
privacy regulations. Horizontal
federated learning (FL) enables collaborative modeling across clients with aligned features without sharing raw data. We study
federated auditing of demographic parity through score distributions, measuring disparity as a Wasserstein–Frechet variance between sensitive-group score laws, and expressing the population metric in
federated form that makes explicit how silo-specific selection drives local-global mismatch. For the squared Wasserstein distance, we prove an ANOVA-style decomposition that separates (i) selection-induced mixture effects from (ii) cross-silo heterogeneity, yielding tight bounds linking local and global metrics. We then propose a one-shot, communication-efficient protocol in which each silo shares only group counts and a
quantile summary of its local score distributions, enabling the server to estimate global disparity and its decomposition, with $O(1/k)$ discretization bias ($k$
quantiles) and finite-sample guarantees. Experiments on synthetic data and COMPAS show that a few dozen
quantiles suffice to recover global disparity and diagnose its sources.
Read the full abstract ▾
All authors Arthur Charpentier, Agathe Fernandes Machado, Olivier Côté, François Hu
Machine Learning · Cross-listed from math.OC
Limits of Convergence-Rate Control for Open-Weight Safety
Domenic Rosati, Xijie Zeng, Hong Huang et al. (+4 more)
Open-weight foundation models can be fine-tuned for harmful purposes after release, yet no existing training resistance methods provide theoretical guarantees. Treating these interventions as convergence-rate control problems allows us to connect optimization speed to the spectral structure of model weights. We leverage this insight to develop a novel understanding of convergence rate control through spectral reparameterization and derive an
algorithm, SpecDef, that can both provably and empirically slow first- and second-order optimization in non-
adversarial settings. In
adversarial settings, we establish a fundamental limit on a broad class of convergence rate control methods including our own: an attacker with sufficient knowledge can restore fast convergence at a linear increase in model size. In order to overcome this limitation, future works will need to investigate methods that are not equivalent to controlling convergence rate.
Read the full abstract ▾
All authors Domenic Rosati, Xijie Zeng, Hong Huang, Sebastian Dionicio, Subhabrata Majumdar, Frank Rudzicz, Hassan Sajjad
Machine Learning
$(\alpha,\beta)$-Stability for Boosting Vector-Valued Prediction
Jian Qian and Shu Ge
Despite the widespread use of boosting in structured prediction, a general theoretical understanding of aggregation beyond scalar prediction remains incomplete. We study vector-valued prediction under a target divergence and identify a
geometric stability property under which aggregation amplifies weak guarantees into strong ones. We formalize this property as $(\alpha,\beta)$-stability by
geometric median and show how it supports a boosting framework based on exponential reweighting and
geometric-median aggregation. For vector-valued prediction, we characterize this stability property under several natural divergences: $\ell_1$ and $\ell_2$ distances for unconstrained vector-valued prediction, and TV, Hellinger, and KL for density estimation over finite probability vectors. Building on these results, we propose a generic boosting framework \geomedboost. Under a weak learner condition and $(\alpha,\beta)$-stability, we obtain exponential decay of the empirical divergence error, which then yields population guarantees through a generalization bound.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
TIACam: Text-Anchored Invariant Feature Learning with Auto-Augmentation for Camera-Robust Zero-Watermarking
Abdullah All Tanvir, Agnibh Dasgupta and Xin Zhong
Camera recapture introduces complex optical degradations, such as perspective warping, illumination shifts, and Moiré interference, that remain challenging for deep watermarking systems. We present TIACam, a text-anchored invariant feature learning framework with auto-augmentation for camera-robust zero-watermarking. The method integrates three key innovations: (1) a learnable auto-augmentor that discovers camera-like distortions through differentiable
geometric, photometric, and Moiré operators; (2) a text-anchored invariant feature learner that enforces semantic consistency via cross-modal
adversarial alignment between image and text; and (3) a zero-watermarking head that binds binary messages in the invariant feature space without modifying image pixels. This unified formulation jointly optimizes invariance, semantic
alignment, and watermark recoverability. Extensive experiments on both synthetic and real-world camera captures demonstrate that TIACam achieves state-of-the-art feature stability and watermark extraction accuracy, establishing a principled bridge between
multimodal invariance learning and physically robust zero-watermarking.
Read the full abstract ▾
Machine Learning
Hyperbolic Busemann Neural Networks
Ziheng Chen, Bernhard Schölkopf and Nicu Sebe
Hyperbolic spaces provide a natural geometry for representing hierarchical and tree-structured data due to their exponential volume growth. To leverage these benefits, neural networks require intrinsic and efficient components that operate directly in
hyperbolic space. In this work, we lift two core components of neural networks, Multinomial Logistic Regression (MLR) and Fully Connected (FC) layers, into
hyperbolic space via Busemann functions, resulting in Busemann MLR (BMLR) and Busemann FC (BFC) layers with a unified mathematical interpretation. BMLR provides compact parameters, a point-to-horosphere distance interpretation, batch-efficient computation, and a Euclidean limit, while BFC generalizes FC and activation layers with comparable complexity. Experiments on image classification, genome sequence learning, node classification, and link prediction demonstrate improvements in effectiveness and efficiency over prior
hyperbolic layers. The code is available at https://github.com/GitZH-Chen/HBNN.
Read the full abstract ▾
Machine Learning
VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learning
Joery A. de Vries, Jinke He, Yaniv Oren et al. (+3 more)
Optimally trading-off exploration and exploitation is the holy grail of reinforcement learning as it promises maximal data-efficiency for solving any task. Bayes-optimal
agents achieve this, but obtaining the belief-state and performing planning are both typically intractable. Although deep learning methods can greatly help in scaling this computation, existing methods are still costly to train. To accelerate this, this paper proposes a variational framework for learning and planning in Bayes-adaptive Markov decision processes that coalesces variational belief learning, sequential Monte-Carlo planning, and meta-reinforcement learning. In a single-GPU setup, our new method VariBASeD exhibits favorable scaling to larger planning budgets, improving sample- and runtime-efficiency over prior methods.
Read the full abstract ▾
All authors Joery A. de Vries, Jinke He, Yaniv Oren, Pascal R. van der Vaart, Mathijs M. de Weerdt, Matthijs T. J. Spaan
Machine Learning
Issues with Measuring Task Complexity via Random Policies in Robotic Tasks
Reabetswe M. Nkhumise, Mohamed S. Talamali and Aditya Gilra
Reinforcement learning (RL) has enabled major advances in fields such as robotics and natural
language processing. A key challenge in RL is measuring task complexity, which is essential for creating meaningful
benchmarks and designing effective curricula. While there are numerous well-established metrics for assessing task complexity in tabular settings, relatively few exist in non-tabular domains. These include (i) Statistical analysis of the performance of random policies via Random Weight Guessing (RWG), and (ii) information-theoretic metrics Policy Information Capacity (PIC) and Policy-Optimal Information Capacity (POIC), which are reliant on RWG. In this paper, we evaluate these methods using progressively difficult robotic manipulation setups, with known relative complexity, with both dense and sparse reward formulations. Our empirical results reveal that measuring complexity is still nuanced. Specifically, under the same reward formulation, PIC suggests that a two-link robotic arm setup is easier than a single-link setup - which contradicts the robotic control and empirical RL perspective whereby the two-link setup is inherently more complex. Likewise, for the same setup, POIC estimates that tasks with sparse rewards are easier than those with dense rewards. Thus, we show that both PIC and POIC contradict typical understanding and empirical results from RL. These findings highlight the need to move beyond RWG-based metrics towards better metrics that can more reliably capture task complexity in non-tabular RL with our task framework as a starting point.
Read the full abstract ▾
Machine Learning
Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training
Seyed Morteza Emadi
Attention scores in
transformers are bilinear forms $S_{ij} = x_i^\top M x_j / \sqrt{d_h}$ whose maximum magnitude governs overflow risk in low-precision training. We derive a rank-aware concentration inequality: when the interaction
matrix $M = W^Q W^{K\top}$ has rank $r \ll d$, tail probabilities for $\max_{i,j}|S_{ij}|$ decay as $\exp(-d^{2}\alpha^{2}/(\gamma r))$ rather than $\exp(-d\alpha^{2})$, where $\gamma > 1$. For
transformer attention where $r = d_h$, this yields $8$ to $28\times$ tighter concentration than rank-agnostic bounds in modern architectures. We apply this result to FP8 training, deriving geometry-aware scale factors that provide principled overflow guarantees without observing activations. The method computes per-layer scales from the spectral norm $\|W^Q W^{K\top}\|_2$ via implicit power iteration, includes a grouped query
attention formulation that avoids key expansion, and remains compatible with fused
attention kernels. Across GPT-2 XL to Llama-2-70B, geometry-aware scaling eliminates overflows in transient scenarios where delayed scaling fails, while achieving comparable downstream MMLU accuracy.
Read the full abstract ▾
Machine Learning
Exact Attention Sensitivity and the Geometry of Transformer Stability
Seyed Morteza Emadi
We develop a sensitivity analysis for
transformer attention in a geometry aligned with tokenwise computation. Our main result is the exact identity $\|J_\tau(u)\|_{\infty\to1}=\theta(p)/\tau$ for the Jacobian $J_\tau(u)$ of the tempered softmax $u\mapsto\mathrm{softmax}(u/\tau)$, where $\theta(p)=4\max_{S\subseteq[L]}p(S)(1-p(S))$ measures how evenly the
attention distribution can be bisected rather than how concentrated it is. We combine this identity with a block-$\infty$/RMS norm under which row-
stochastic attention mixing is nonexpansive. This yields a distribution-aware local Jacobian bound for multi-head
attention and a sequence-length-independent Lipschitz bound on bounded input sets, with explicit dependence on width, input magnitude, temperature, and projection norms. We also identify a structural distinction between normalization placements: a pre-LN residual-sublayer Jacobian contains an additive identity term, whereas a post-LN residual-sublayer Jacobian does not. A LayerNorm projection lemma gives a sufficient condition under which the LayerNorm-only term in the post-LN expansion contracts geometrically; the condition is not tested by our experiments. Across three Pre-LN early-training runs of $774$M-parameter models,
attention becomes substantially more concentrated while the median lower-bound certificate for $\theta(p)$ remains near one at every sampled layer and checkpoint. This certifies near-maximal exact sensitivity for at least half of the sampled rows within each layer. A minority of rows enters a dominant-atom regime with lower exact sensitivity, consistent with the deterministic relationship between $p_{\max}$ and $\theta(p)$.
Read the full abstract ▾
Machine Learning
L2G-Net: Local to Global Spectral Graph Neural Networks via Cauchy Factorizations
Samuel Fernández-Menduiña, Eduardo Pavez and Antonio Ortega
Despite their theoretical advantages, spectral methods based on the graph
Fourier transform (GFT) are seldom used in graph neural networks (GNNs) due to the cost of computing the eigenbasis and the lack of vertex-domain locality in the resulting representations. As a result, most GNNs rely on local approximations such as
polynomial Laplacian filters or message passing, which limit their ability to model long-range dependencies. In this paper, we introduce an exact factorization of the GFT into operators acting on subgraphs, which are then combined via a sequence of Cauchy
matrices. Building on this factorization, we propose a new class of spectral GNNs, termed L2G-Net (Local to Global Net). Unlike existing spectral methods, which are either fully global (when using the GFT) or local (when using
polynomial filters), L2G-Net operates by processing the spectral representations of subgraphs and then combining them via structured
matrices. Our
algorithm avoids full eigendecompositions, exploiting graph
topology to construct the factorization with quadratic complexity in the number of nodes, scaled by the maximum cut size between subgraphs. Experiments stressing long-range dependencies on large graphs show that L2G-Net scales to regimes out of reach for the standard GFT, and is competitive with state-of-the-art methods with orders of magnitude fewer learnable parameters.
Read the full abstract ▾
Machine Learning
Bayesian Lottery Ticket Hypothesis
Nicholas Kuhn, Arvid Weyrauch, Lars Heyen et al. (+3 more)
Bayesian neural networks (BNNs) are a useful tool for uncertainty quantification, but require substantially more computational resources than conventional neural networks. For non-
Bayesian networks, the Lottery Ticket Hypothesis (LTH) posits the existence of sparse subnetworks that can train to the same or even surpassing accuracy as the original dense network. Such sparse networks can lower the demand for computational resources at inference, and during training. The existence of the LTH and corresponding sparse subnetworks in BNNs could motivate the development of sparse training
algorithms and provide valuable insights into the underlying training process. Towards this end, we translate the LTH experiments to a
Bayesian setting using common computer
vision models. We investigate the defining characteristics of
Bayesian lottery tickets, and extend our study towards a transplantation method connecting BNNs with deterministic Lottery Tickets. We generally find that the LTH holds in BNNs, and winning tickets of matching and surpassing accuracy are present independent of model size, with degradation at very high sparsities. However, the
pruning strategy should rely primarily on magnitude, secondly on standard deviation. Furthermore, our results demonstrate that models rely on mask structure and weight initialization to varying degrees.
Read the full abstract ▾
All authors Nicholas Kuhn, Arvid Weyrauch, Lars Heyen, Achim Streit, Markus Götz, Charlotte Debus
Machine Learning · Cross-listed from cs.RO
Habilis-$\beta$: A Fast-Motion and Long-Lasting On-Device Vision-Language-Action Model
Tommoro Robotics: Jesoon Kang, Taegeon Park, Jisu An et al. (+20 more)
We introduce Habilis-$\beta$, a fast-motion and long-lasting on-device
vision-
language-action (VLA) model designed for real-world deployment. Current VLA evaluation remains largely confined to single-trial success rates under curated resets, which fails to capture the fast-motion and long-lasting capabilities essential for practical operation. To address this, we introduce the Productivity-Reliability Plane (PRP), which evaluates performance through Tasks per Hour (TPH) and Mean Time Between Intervention (MTBI) under a continuous-run protocol that demands both high-speed execution and sustained robustness. Habilis-$\beta$ achieves high performance by integrating
language-free pre-training on large-scale play data for robust interaction priors with post-training on cyclic task demonstrations that capture state drift across consecutive task iterations. The system further employs ESPADA for phase-adaptive motion shaping to accelerate free-space transit, utilizes rectified-flow
distillation to enable high-frequency control on edge devices, and incorporates classifier-free guidance (CFG) as a deployment-time knob to dynamically balance instruction adherence and learned interaction priors. In 1-hour continuous-run evaluations, Habilis-$\beta$ achieves strong performance under the PRP metrics, compared to $\pi_{0.5}$ in both simulation and real-world environments. In simulation, Habilis-$\beta$ achieves 572.6 TPH and 39.2 s MTBI (vs. 120.5 TPH and 30.5 s for $\pi_{0.5}$), while in a real-world
humanoid logistics workflow it achieves 124 TPH and 137.4 s MTBI (vs. 19 TPH and 46.1 s for $\pi_{0.5}$). Finally, Habilis-$\beta$ achieves the highest reported performance on the standard RoboTwin 2.0 leaderboard across representative tasks, validating its effectiveness in complex manipulation scenarios.
Read the full abstract ▾
All authors Tommoro Robotics: Jesoon Kang, Taegeon Park, Jisu An, Soo Min Kimm, Jaejoon Kim, Jinu Pahk, Byungju Kim, Junseok Lee, Namheon Baek, Sungwan Ha, Hojun Baek, Eduardo Ayerve Cruz, Wontae Kim, Junghyeon Choi, Yousuk Lee, Joonmo Han, Sunghyun Cho, Sunghyun Kwon, Soyoung Lee, Jun Ki Lee, Seung-Joon Yi, Byoung-Tak Zhang, Theo Taeyeong Kim
Machine Learning
SGNO: Spectral Generator Neural Operator for Long-Horizon PDE Prediction
Jiayi Li, Penghao Jiang, Baiyu Chen et al. (+5 more)
Neural operators efficiently simulate dynamical systems by repeatedly predicting the next state. However, small errors at each step can accumulate, causing long-term predictions to drift from the true trajectory. Long-term error reflects both the propagation of earlier prediction errors and new errors introduced at each step. How to reduce these errors is therefore a critical question. We propose the Spectral Generator Neural Operator (SGNO), which combines a linear estimate of the next state with a learned correction. The linear component uses consecutive states in the training data to fit a map from the current state to the next. A neural network corrects this prediction by learning changes that the linear model does not capture, including nonlinear interactions. Both components are trained jointly toward the same next-state target. Experiments across 13 periodic PDE configurations in one to three dimensions show that SGNO accurately learns state
evolution on linear tasks and reduces 100-step rollout error by 13-74% over the strongest baseline under the main comparison protocol on each nonlinear task. Longer-horizon extrapolation tests show lower late-window prediction errors on several nonlinear tasks. These results show that combining fitted propagation with learned state-dependent correction can improve long-horizon autoregressive prediction on the evaluated PDE tasks. Code is available at https://github.com/cruiseresearchgroup/SGNO.
Read the full abstract ▾
All authors Jiayi Li, Penghao Jiang, Baiyu Chen, Yifan Duan, Hira Saleem, Zhaonan Wang, Piotr Koniusz, Flora D. Salim
Machine Learning · Cross-listed from cs.SE
Operational Robustness of LLMs on Code Generation
Debalina Ghosh Paul, Hong Zhu and Ian Bayley
It is now common practice in software development for large
language models (LLMs) to be used to generate program code. It is desirable to evaluate the robustness of LLMs for this usage. This paper is concerned in particular with how sensitive LLMs are to variations in descriptions of the coding tasks. However, existing techniques for evaluating this robustness are unsuitable for code generation because the input data space of natural
language descriptions is discrete. To address this problem, we propose a robustness evaluation method called scenario domain analysis, which aims to find the expected minimal change in the natural
language descriptions of coding tasks that would cause the LLMs to produce incorrect outputs. We have formally proved the theoretical properties of the method and also conducted extensive experiments to evaluate the robustness of four state-of-the-art art LLMs: Gemini-pro, Codex, Llamma2 and Falcon 7B, and have found that we are able to rank these with confidence from best to worst. Moreover, we have also studied how robustness varies in different scenarios, including the variations with the topic of the coding task and with the complexity of its sample solution, and found that robustness is lower for more complex tasks and also lower for more advanced topics, such as multi-threading and data structures.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DC
Carbon-aware decentralized dynamic task offloading in MIMO-MEC networks via multi-agent reinforcement learning
Mubshra Zulfiqar, Muhammad Ayzed Mirza and Basit Qureshi
Massive internet of things microservices require integrating renewable energy harvesting into mobile edge computing (MEC) for sustainable eScience infrastructures. Spatiotemporal mismatches between
stochastic task arrivals and intermittent green energy along with complex inter-user interference in multi-
antenna (
MIMO) uplinks complicate real-time resource management. Traditional centralized optimization and off-policy reinforcement learning struggle with scalability and signaling overhead in dense networks. This paper proposes CADDTO-PPO, a carbon-aware decentralized dynamic task offloading framework based on multi-
agent proximal policy optimization. The multi-user
MIMO-MEC system is modeled as a Decentralized Partially Observable Markov Decision Process (DEC-POMDP) to jointly minimize carbon emissions and buffer latency and energy wastage. A scalable architecture utilizes decentralized execution with parameter sharing (DEPS), which enables autonomous IoT
agents to make fine-grained power control and offloading decisions based solely on local observations. Additionally, a carbon-first reward structure adaptively prioritizes green time slots for data transmission to decouple system throughput from grid-dependent carbon footprints. Finally, experimental results demonstrate CADDTO-PPO outperforms deep deterministic policy gradient (DDPG) and lyapunov-based baselines. The framework achieves the lowest carbon intensity and maintains near-zero packet overflow rates under extreme traffic loads. Architectural profiling validates the framework to demonstrate a constant $O(1)$ inference complexity and theoretical lightweight feasibility for future generation sustainable IoT deployments.
Read the full abstract ▾
Machine Learning
Vectorized Bayesian Inference for Latent Dirichlet-Tree Allocation
Zheng Wang and Nizar Bouguila
Latent Dirichlet Allocation (LDA) is a foundational model for discovering latent thematic structure in discrete data, but its Dirichlet prior cannot represent the rich correlations and hierarchical relationships often present among topics. We introduce the framework of Latent Dirichlet-Tree Allocation (LDTA), a generalization of LDA that replaces the Dirichlet prior with an arbitrary Dirichlet-Tree (DT) distribution. LDTA preserves LDA's generative structure but enables expressive, tree-structured priors over topic proportions. To perform inference, we develop universal mean-field variational inference and Expectation Propagation, providing tractable updates for all DT. We reveal the vectorized nature of the two inference methods through theoretical development, and perform fully vectorized, GPU-accelerated implementations. The resulting framework substantially expands the modeling capacity of LDA while maintaining scalability and computational efficiency.
Read the full abstract ▾
Machine Learning
From Few-Shot to Zero-Shot: Towards Generalist Graph Anomaly Detection
Yixin Liu, Shiyuan Li, Yu Zheng et al. (+4 more)
Graph anomaly detection (GAD) is critical for identifying abnormal nodes in graph-structured data from diverse domains, including cybersecurity and social networks. The existing GAD methods often focus on the learning paradigms of "one-model-for-one-
dataset", requiring
dataset-specific training for each
dataset to achieve optimal performance. However, this paradigm suffers from significant limitations, such as high computational and data costs, limited generalization and transferability to new
datasets, and challenges in
privacy-sensitive scenarios where access to full
datasets or sufficient labels is restricted. To address these limitations, we propose a novel generalist GAD paradigm that aims to develop a unified model capable of detecting anomalies on multiple unseen
datasets without extensive retraining/fine-tuning or
dataset-specific customization. To this end, we propose ARC, a few-shot generalist GAD method that leverages in-context learning and requires only a few labeled normal samples at inference time. Specifically, ARC consists of three core modules: a feature
Alignment module to unify and align features across
datasets, a Residual GNN encoder to capture
dataset-agnostic anomaly representations, and a cross-attentive in-Context learning module to score anomalies using few-shot normal context. Building on ARC, we further introduce ARC_zero for the zero-shot generalist GAD setting, which selects representative pseudo-normal nodes via a pseudo-context mechanism and thus enables fully label-free inference on unseen
datasets. Extensive experiments on 17 real-world graph
datasets demonstrate that both ARC and ARC_zero effectively detect anomalies, exhibit strong generalization ability, and perform efficiently under few-shot and zero-shot settings.
Read the full abstract ▾
All authors Yixin Liu, Shiyuan Li, Yu Zheng, Qingfeng Chen, Chengqi Zhang, Philip S. Yu, Shirui Pan
Machine Learning
CaliCausalRank: Calibrated Multi-Objective Ad Ranking with Robust Counterfactual Utility Optimization
Xikai Yang, Sebastian Sun, Yilin Li et al. (+3 more)
Ad ranking systems must simultaneously optimize multiple objectives including click-through rate (CTR), conversion rate (CVR), revenue, and user experience metrics. However, production systems face critical challenges: score scale inconsistency across traffic segments undermines threshold transferability, and position bias in click logs causes offline-online metric discrepancies. We propose CaliCausalRank, a unified framework that integrates training-time scale calibration, constraint-based multi-objective optimization, and robust counterfactual utility estimation. Our approach treats score calibration as a first-class training objective rather than post-hoc processing, employs Lagrangian relaxation for constraint satisfaction, and utilizes variance-reduced counterfactual
estimators for reliable offline evaluation. Experiments on the Criteo and Avazu
datasets demonstrate that CaliCausalRank achieves 1.1% relative AUC improvement, 31.6% calibration error reduction, and 3.2% utility gain compared to the best baseline (PairRank) while maintaining consistent performance across different traffic segments.
Read the full abstract ▾
All authors Xikai Yang, Sebastian Sun, Yilin Li, Yue Xing, Ming Wang, Yang Wang
Machine Learning · Cross-listed from cs.CR
MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs
Chun Yan Ryan Kan, Tommy Tran, Vedant Yadav et al. (+4 more)
Defending LLMs against
adversarial jailbreak attacks remains an open challenge. Existing defenses rely on binary classifiers that fail when
adversarial input falls outside the learned decision boundary, and repeated fine-tuning is computationally expensive while potentially degrading model capabilities. We propose MANATEE, an inference-time defense that uses density estimation over a benign representation
manifold. MANATEE learns the score function of benign hidden states and uses
diffusion to project anomalous representations toward safe regions–requiring no harmful training data and no architectural modifications. Experiments across Mistral-7B-Instruct, Llama-3.1-8B-Instruct, and Gemma-2-9B-it demonstrate that MANATEE reduce Attack Success Rate by up to 100% on certain
datasets, while preserving model utility on benign inputs.
Read the full abstract ▾
All authors Chun Yan Ryan Kan, Tommy Tran, Vedant Yadav, Ava Cai, Kevin Zhu, Ruizhe Li, Maheep Chaudhary
Machine Learning
GLaDiGAtor: Language-Model-Augmented Multi-Relation Graph Learning for Predicting Disease-Gene Associations
Osman Onur Kuzucu and Tunca Doğan
Understanding disease-gene associations is essential for unravelling disease mechanisms and advancing diagnostics and therapeutics. Traditional approaches based on manual curation and literature review are labour-intensive and not scalable, prompting the use of machine learning on large biomedical data. In particular, graph neural networks (GNNs) have shown promise for modelling complex biological relationships. To address limitations in existing models, we propose GLaDiGAtor (Graph Learning-bAsed DIsease-Gene AssociaTiOn pRediction), a novel GNN framework with an encoder-decoder architecture for disease-gene association prediction. GLaDiGAtor constructs a heterogeneous biological graph integrating gene-gene, disease-disease, and gene-disease interactions from curated databases, and enriches each node with contextual features from well-known
language models (ProtT5 for
protein sequences and BioBERT for disease text). In evaluations, our model achieves superior predictive accuracy and generalisation, outperforming 14 existing methods. Literature-supported case studies confirm the biological relevance of high-confidence novel predictions, highlighting GLaDiGAtor's potential to discover candidate disease genes. These results underscore the power of graph convolutional networks in biomedical informatics and may ultimately facilitate drug discovery by revealing new gene-disease links. The source code and processed
datasets are publicly available at https://github.com/HUBioDataLab/GLaDiGAtor.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.LO
Nazrin: An Atomic Neural Proof Automation Tactic in Lean 4
Leni Aniva, Iori Oikawa, David Dill et al. (+1 more)
In Machine-Assisted
Theorem Proving, a
theorem proving
agent searches for a sequence of expressions and tactics that can prove a statement in a proof assistant. In this work, we introduce several novel concepts and capabilities to address obstacles faced by machine-assisted
theorem proving. We first present a set of atomic tactics, a small finite set of tactics capable of proving any provable statement in Lean. We then introduce a transposing atomization
algorithm which turns arbitrary proof expressions into a series of atomic tactics. We next introduce the ExprGraph data structure, which provides a succinct representation for Lean expressions. Finally, we present the Nazrin Prover, short for Neural Atomizer for Inhabitation Problems, a graph neural network-based
theorem proving
agent using atomic tactics and ExprGraph. Nazrin circumvents many challenges faced by existing proving
agents by exclusively dispatching atomic tactics, and it is robust enough to both train and evaluate on consumer-grade hardware. We demonstrate the potential of tools like Nazrin using
theorems from Lean's standard library and from Mathlib.
Read the full abstract ▾
All authors Leni Aniva, Iori Oikawa, David Dill, Clark Barrett
Machine Learning · Cross-listed from stat.ML
Bounds and Identification of Joint Probabilities of Potential Outcomes and Observed Variables under Monotonicity Assumptions
Naoya Hashimoto, Yuta Kawakami and Jin Tian
Evaluating joint probabilities of potential outcomes and observed variables, and their linear combinations, is a fundamental challenge in causal inference. This paper addresses the bounding and identification of these probabilities in settings with discrete treatment and discrete ordinal outcome. We propose new families of monotonicity assumptions and formulate the bounding problem as a linear programming problem. We further introduce a new monotonicity assumption specifically to achieve identification. Finally, we present numerical experiments to validate our methods and demonstrate their application using real-world
datasets.
Read the full abstract ▾
Machine Learning
RadioGen3D: 3D Radio Map Generation via Adversarial Learning on Large-Scale Synthetic Data
Junshen Chen, Angzi Xu, Zezhong Zhang et al. (+3 more)
Radio maps are essential for efficient
radio resource management in future 6G and low-altitude networks. While deep learning (DL) techniques have emerged as an efficient alternative to conventional ray-tracing for
radio map estimation (RME), most existing DL approaches are confined to 2D near-ground scenarios. They often fail to capture essential 3D signal propagation characteristics and
antenna polarization effects, primarily due to the scarcity of 3D data and training challenges. To address these limitations, we present the RadioGen3D framework. First, we propose an efficient data synthesis method to generate high-quality 3D
radio map data. By establishing a parametric target model that captures 2D ray-tracing and 3D channel fading characteristics, we derive realistic coefficient combinations from minimal real measurements, enabling the construction of a large-scale synthetic
dataset,
Radio3DMix. Utilizing this
dataset, we propose a 3D model training scheme based on a conditional generative
adversarial network (cGAN), yielding a 3D U-Net capable of accurate RME under diverse input feature combinations. Experimental results demonstrate that RadioGen3D surpasses all baselines in both estimation accuracy and speed. Furthermore, fine-tuning experiments verify its strong generalization capability via successful knowledge transfer.
Read the full abstract ▾
All authors Junshen Chen, Angzi Xu, Zezhong Zhang, Shiyao Zhang, Junting Chen, Shuguang Cui
Machine Learning
HONEST-CAV: Hierarchical Optimization of Network Signals and Trajectories for Connected and Automated Vehicles with Multi-Agent Reinforcement Learning
Ziyan Zhang, Changxin Wan, Peng Hao et al. (+5 more)
This study presents a hierarchical, network-level traffic flow control framework for mixed traffic consisting of Human-driven Vehicles (HVs), Connected and Automated Vehicles (CAVs). The framework jointly optimizes vehicle-level eco-driving behaviors and intersection-level traffic signal control to enhance overall network efficiency and decrease energy consumption. A decentralized Multi-
Agent Reinforcement Learning (MARL) approach by Value Decomposition Network (VDN) manages cycle-based traffic signal control (TSC) at intersections, while an innovative Signal Phase and Timing (SPaT) prediction method integrates a Machine Learning-based Trajectory Planning
Algorithm (MLTPA) to guide CAVs in executing Eco-Approach and Departure (EAD) maneuvers. The framework is evaluated across varying CAV proportions and powertrain types to assess its effects on mobility and energy performance. Experimental results conducted in a 4*4 real-world network demonstrate that the MARL-based TSC method outperforms the baseline model (i.e., Webster method) in speed, fuel consumption, and idling time. In addition, with MLTPA, HONEST-CAV benefits the traffic system further in energy consumption and idling time. With a 60% CAV proportion, vehicle average speed, fuel consumption, and idling time can be improved/saved by 7.67%, 10.23%, and 45.83% compared with the baseline. Furthermore, discussions on CAV proportions and powertrain types are conducted to quantify the performance of the proposed method with the impact of automation and electrification.
Read the full abstract ▾
All authors Ziyan Zhang, Changxin Wan, Peng Hao, Kanok Boriboonsomsin, Matthew J. Barth, Yongkang Liu, Seyhan Ucar, Guoyuan Wu
Machine Learning
When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Models
Zhixiang Guo, Siyuan Liang, Andras Balogh et al. (+4 more)
Generative world models (WMs) are increasingly used to synthesize controllable, sensor-conditioned driving
videos, yet their reliance on physical priors exposes novel attack surfaces. In this paper, we present Physical-Conditioned World Model Attack (PhysCond-WMA), the first white-box world model attack that perturbs physical-condition channels, such as HDMap
embeddings and 3D-box features, to induce semantic, logic, or decision-level distortion while preserving perceptual fidelity. PhysCond-WMA is optimized in two stages: (1) a quality-preserving guidance stage that constrains reverse-
diffusion loss below a calibrated threshold, and (2) a momentum-guided denoising stage that accumulates target-aligned gradients along the denoising trajectory for stable, temporally coherent semantic shifts. Extensive experimental results demonstrate that our approach remains effective while increasing FID by about 9% on average and FVD by about 3.9% on average. Under the targeted attack setting, the attack success rate (ASR) reaches 0.55. Downstream studies further show tangible risk, which using attacked
videos for training decreases 3D detection performance by about 4%, and worsens open-loop planning performance by about 20%. These findings has for the first time revealed and quantified security vulnerabilities in generative world models, driving more comprehensive security checkers.
Read the full abstract ▾
All authors Zhixiang Guo, Siyuan Liang, Andras Balogh, Noah Lunberry, Rong-Cheng Tu, Mark Jelasity, Dacheng Tao
Machine Learning
A Prior-Aware Metric for Efficiently Distinguishing Memorization from Generalization in Large Language Models
Trishita Tiwari, Ari Trachtenberg and G. Edward Suh
Training data leakage from Large
Language Models (LLMs) raises serious concerns related to
privacy, security, and copyright compliance. A central challenge in assessing this risk is distinguishing prefix-specific memorization of training data from the generation of statistically common sequences. Existing approaches to measuring memorization often conflate these phenomena, labeling outputs as memorized even when they arise from generalization over common patterns. Counterfactual memorization and other related metrics \citep{zhang2023counterfactual, wang2025generalization, lesci2024causal} provide principled solutions, however, their reliance on retraining multiple baseline models or parsing through the training data makes them computationally impractical at scale. This work introduces Prior-Aware memorization, a theoretically grounded, lightweight and training-free criterion for identifying prefix-specific memorization in LLMs. The key idea is to evaluate whether a candidate suffix is strongly associated with its specific training prefix or whether it appears with high probability across many IID sampled sequences from the training data distribution due to statistical commonality. We correlate our metric with counterfactual memorization, and also evaluate it on the training corpora of two pre-trained models, LLaMA and OPT. Our results show that between 55% and 90% of sequences previously labeled as memorized fail our criterion and are consistent with statistical commonality
Read the full abstract ▾