Machine Learning
A Consensus-Bayesian Framework for Detecting Malicious Activity in Enterprise Directory Access Graphs
Pratyush Uppuluri, Shilpa Noushad and Sajan Kumar
This work presents a consensus-based
Bayesian framework to detect malicious user behavior in enterprise directory access graphs. By modeling directories as topics and users as
agents within a multi-level interaction graph, we simulate access
evolution using influence-weighted opinion dynamics. Logical dependencies between users are encoded in dynamic
matrices Ci, and directory similarity is captured via a shared influence
matrix W. Malicious behavior is injected as cross-component logical perturbations that violate structural norms of strongly connected components(SCCs). We apply theoretical guarantees from opinion dynamics literature to determine topic convergence and detect anomaly via scaled opinion variance. To quantify uncertainty, we introduce a
Bayesian anomaly scoring mechanism that evolves over time, using both static and online priors. Simulations over synthetic access graphs validate our method, demonstrating its sensitivity to logical inconsistencies and robustness under dynamic perturbation.
Read the full abstract ▾
Machine Learning
Group Contrastive Learning for Weakly Paired Multimodal Data
Aditya Gorla, Hugues Van Assel, Jan-Christian Huetter et al. (+4 more)
We present GROOVE, a semi-supervised multi-modal representation learning approach for high-content perturbation data where samples across modalities are weakly paired through shared perturbation labels but lack direct correspondence. Our primary contribution is GroupCLIP, a novel group-level contrastive loss that bridges the gap between CLIP for paired cross-modal data and SupCon for uni-modal supervised contrastive learning, addressing a fundamental gap in contrastive learning for weakly-paired settings. We integrate GroupCLIP with an on-the-fly backtranslating autoencoder framework to encourage cross-modally entangled representations while maintaining group-level coherence within a shared latent space. Critically, we introduce a comprehensive combinatorial evaluation framework that systematically assesses representation learners across multiple optimal transport aligners, addressing key limitations in existing evaluation strategies. This framework includes novel simulations that systematically vary shared versus modality-specific perturbation effects enabling principled assessment of method robustness. Our combinatorial benchmarking reveals that there is not yet an aligner that uniformly dominates across settings or modality pairs. Across simulations and two real single-
cell genetic perturbation
datasets, GROOVE performs on par with or outperforms existing approaches for downstream cross-modal matching and imputation tasks. Our ablation studies demonstrate that GroupCLIP is the key component driving performance gains. These results highlight the importance of leveraging group-level constraints for effective multi-modal representation learning in scenarios where only weak pairing is available.
Read the full abstract ▾
All authors Aditya Gorla, Hugues Van Assel, Jan-Christian Huetter, Heming Yao, Kyunghyun Cho, Aviv Regev, Russell Littman
Machine Learning
Understanding and Guiding Layer Placement in Parameter-Efficient Fine-Tuning of Large Language Models
Yichen Xu, Yuyang Liang, Shan Dai et al. (+3 more)
As large
language models (LLMs) continue to grow, the cost of full-parameter fine-tuning has made parameter-efficient fine-tuning (PEFT) the default strategy for downstream adaptation. Constraints from inference latency in scalable serving and fine-tuning cost in edge or rapid-deployment settings make the choice of which layers to fine-tune unavoidable. Yet current practice typically applies PEFT uniformly across all layers, with limited understanding or leverage of layer selection. This paper develops a unified projected residual view of PEFT on top of a frozen base model. Under a local quadratic approximation, layerwise adaptation is governed by three quantities: (i) the projected residual norm (resnorm), which measures how much correctable bias a layer can capture; (ii) the activation energy, which determines feature conditioning; and (iii) layer coupling, which quantifies how strongly residuals interact across layers. We show that, for squared loss and linear adapters, the resnorm equals a normalized gradient norm, activation energy controls ill-conditioning and noise amplification, and weak coupling yields approximately additive layerwise contributions. Building on these insights, we introduce the Layer Card, a reusable diagnostic that summarizes residual signal strength, compute cost, and performance for each layer of a given model. With an identical model and LoRA configuration, Layer Card-guided placement refines the choice of adapted layers to flexibly prioritize different objectives, such as maximizing performance or reducing fine-tuning cost. Moreover, on Qwen3-8B, we show that selectively adapting a subset of layers can achieve performance close to full-layer LoRA while substantially reducing fine-tuning cost and the number of adapter-augmented layers during inference, offering a more cost-performance-aware alternative to full-layer insertion.
Read the full abstract ▾
All authors Yichen Xu, Yuyang Liang, Shan Dai, Tianyang Hu, Tsz Nam Chan, Chenhao Ma
Machine Learning · Cross-listed from eess.SP
A Multi-Modal Foundational Model for Wireless Communication and Sensing
Vahid Yazdnian and Yasaman Ghasempour
Artificial intelligence is a key enabler for next-generation wireless communication and sensing. Yet, today's learning-based wireless techniques do not generalize well: most models are task-specific, environment-dependent, and limited to narrow sensing modalities, requiring costly retraining when deployed in new scenarios. This work introduces a task-agnostic, multi-modal foundational model for physical-layer wireless systems that learns transferable, physics-aware representations across heterogeneous modalities, enabling robust generalization across tasks and environments. Our framework employs a physics-guided self-supervised pretraining strategy incorporating a dedicated physical
token to capture cross-modal physical correspondences governed by electromagnetic propagation. The learned representations enable efficient adaptation to diverse downstream tasks, including massive multi-
antenna optimization, wireless channel estimation, and device localization, using limited labeled data. Our extensive evaluations demonstrate superior generalization, robustness to deployment shifts, and reduced data requirements compared to task-specific baselines.
Read the full abstract ▾
Machine Learning
PromptSplit: Revealing Prompt-Level Disagreement in Generative Models
Mehdi Lotfian, Mohammad Jalali and Farzan Farnia
Prompt-guided generative AI models have rapidly expanded across
vision and
language domains, producing realistic and diverse outputs from textual inputs. The growing variety of such models, trained with different data and architectures, calls for principled methods to identify which types of
prompts lead to distinct model behaviors. In this work, we propose PromptSplit, a kernel-based framework for detecting and analyzing
prompt-dependent disagreement between generative models. For each compared model pair, PromptSplit constructs a joint
prompt–output representation by forming tensor-product
embeddings of the
prompt and image (or text) features, and then computes the corresponding kernel covariance
matrix. We utilize the eigenspace of the weighted difference between these
matrices to identify the main directions of behavioral difference across
prompts. To ensure scalability, we employ a random-projection approximation that reduces computational complexity to $O(nr^2 + r^3)$ for projection dimension $r$. We further provide a theoretical analysis showing that this approximation yields an eigenstructure estimate whose expected deviation from the full-dimensional result is bounded by $O(1/r^2)$. Experiments across text-to-image, text-to-text, and image-captioning settings demonstrate that PromptSplit accurately detects ground-truth behavioral differences and isolates the
prompts responsible, offering an interpretable tool for detecting where generative models disagree.
Read the full abstract ▾
Machine Learning
Rational ANOVA Networks
Jusheng Zhang, Ningyuan Liu, Qinhan Lyu et al. (+2 more)
Deep neural networks typically treat nonlinearities as fixed primitives (e.g., ReLU), limiting both interpretability and the granularity of control over the induced function class. While recent additive models (like KANs) attempt to address this using splines, they often suffer from computational inefficiency and boundary instability. We propose the Rational-ANOVA Network (RAN), a foundational architecture grounded in functional ANOVA decomposition and Padé-style rational approximation. RAN models f(x) as a composition of main effects and sparse pairwise interactions, where each component is parameterized by a stable, learnable rational unit. Crucially, we enforce a strictly positive denominator, which avoids poles and numerical instability while capturing sharp transitions and near-singular behaviors more efficiently than
polynomial bases. This ANOVA structure provides an explicit low-order interaction bias for data efficiency and interpretability, while the rational parameterization significantly improves extrapolation. Across controlled function
benchmarks and
vision classification tasks (e.g., CIFAR-10) under matched parameter and compute budgets, RAN matches or surpasses parameter-matched MLPs and learnable-activation baselines, with better stability and throughput. Code is available at https://github.com/jushengzhang/Rational-ANOVA-Networks.git.
Read the full abstract ▾
All authors Jusheng Zhang, Ningyuan Liu, Qinhan Lyu, Jing Yang, Keze Wang
Machine Learning · Cross-listed from math.PR
Functional Stochastic Localization
Anming Gu, Bobby Shi and Kevin Tian
Eldan's
stochastic localization is a probabilistic construction that has proved instrumental to modern breakthroughs in high-dimensional geometry and the design of sampling
algorithms. Motivated by sampling under non-Euclidean geometries and the mirror descent
algorithm in optimization, we develop a functional generalization of Eldan's process that replaces Gaussian regularization with regularization by any positive integer multiple of a log-Laplace transform. We further give a mixing time bound on the Markov chain induced by our localization process, which holds if our target distribution satisfies a functional Poincaré inequality. Finally, we apply our framework to differentially private convex optimization in $\ell_p$ norms for $p \in [1, 2)$, where we improve state-of-the-art query complexities in a zeroth-order model.
Read the full abstract ▾
Machine Learning
Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models
Anish Sathyanarayanan, Aditya Nagarsekar and Aarush Rathore
Chain-of-thought (CoT) prompting is widely used as a
reasoning aid and is often treated as a transparency mechanism. Yet behavioral gains under CoT do not imply that the model's internal computation causally depends on the emitted
reasoning text, i.e. models may produce fluent rationales while routing decision-critical computation through latent pathways. We introduce a causal, layerwise audit of CoT faithfulness based on activation patching. Our key metric, the CoT Mediation Index (CMI), isolates CoT-specific causal influence by comparing performance degradation from patching CoT-
token hidden states against matched control patches. Across multiple model families (Phi, Qwen, DialoGPT) and scales, we find that CoT-specific influence is typically depth-localized into narrow ”
reasoning windows,” and we identify bypass regimes where CMI is near-zero despite plausible CoT text. We further observe that models tuned explicitly for
reasoning tend to exhibit stronger and more structured mediation than larger untuned counterparts, while Mixture-of-Experts models show more distributed mediation consistent with routing-based computation. Overall, our results show that CoT faithfulness varies substantially across models and tasks and cannot be inferred from behavior alone, motivating causal, layerwise audits when using CoT as a transparency signal.
Read the full abstract ▾
Machine Learning
eCP: Equivariant Conformal Prediction with pre-trained models
Nikolaos Bousias, Lars Lindemann and George Pappas
Conformal prediction, a post-hoc, distribution-free, finite-sample method of uncertainty quantification that offers formal coverage guarantees under the assumption of data exchangeability. Unfortunately, the resulting uncertainty regions can grow significantly in long horizon missions, rendering the statistical guarantees uninformative. To that end, we propose infusing CP with
geometric information via group-averaging of the pretrained predictor to distribute the non-conformity mass across the orbits. Each sample now is treated as a representative of an orbit, thus uncertainty can be mitigated by other samples entangled to it via the orbit inducing elements of the
symmetry group. Our approach provably yields contracted non-conformity scores in increasing convex order, implying improved exponential-tail bounds and sharper conformal prediction sets in expectation, especially at high confidence levels. We then propose an experimental design to test these theoretical claims in pedestrian trajectory prediction.
Read the full abstract ▾
Machine Learning
DeXposure-FM: A Time-series, Graph Foundation Model for Credit Exposures and Stability on Decentralized Financial Networks
Aijie Shu, Wenbin Wu, Gbenga Ibikunle et al. (+1 more)
Credit exposure in Decentralized Finance (DeFi) is often implicit and
token-mediated, creating a dense web of inter-protocol dependencies. Thus, a shock to one
token may result in significant and uncontrolled contagion effects. As the DeFi ecosystem becomes increasingly linked with traditional financial infrastructure through instruments, such as stablecoins, the risk posed by this dynamic demands more powerful quantification tools. We introduce DeXposure-FM, the first time-series, graph foundation model for measuring and
forecasting inter-protocol credit exposure on DeFi networks, to the best of our knowledge. Employing a graph-tabular encoder, with pre-trained weight initialization, and multiple task-specific heads, DeXposure-FM is trained on the DeXposure
dataset that has 43.7 million data entries, across 4,300+ protocols on 602 blockchains, covering 24,300+ unique
tokens. The training is operationalized for credit-exposure
forecasting, predicting the joint dynamics of (1) protocol-level flows, and (2) the
topology and weights of credit-exposure links. The DeXposure-FM is empirically validated on two machine learning
benchmarks; it consistently outperforms the state-of-the-art approaches, including a graph foundation model and temporal graph neural networks. DeXposure-FM further produces financial economics tools that support macroprudential monitoring and scenario-based DeFi stress testing, by enabling protocol-level systemic-importance scores, sector-level spillover and concentration measures via a forecast-then-measure pipeline. Empirical verification fully supports our financial economics tools. The model and code have been publicly available. Model: https://huggingface.co/EVIEHub/DeXposure-FM. Code: https://github.com/EVIEHub/DeXposure-FM.
Read the full abstract ▾
All authors Aijie Shu, Wenbin Wu, Gbenga Ibikunle, Fengxiang He
Machine Learning · Cross-listed from cs.AI
Monitorability as a Free Gift: How RLVR Spontaneously Aligns Reasoning
Zidi Xiong, Shan Chen and Himabindu Lakkaraju
As Large
Reasoning Models (LRMs) are increasingly deployed,
auditing their chain-of-thought (CoT) traces for safety becomes critical. Recent work has reported that monitorability–the degree to which CoT faithfully and informatively reflects internal computation–can appear as a "free gift" during the early stages of Reinforcement Learning with Verifiable Rewards (RLVR). We make this observation concrete through a systematic evaluation across model families and training domains. Our results show that this effect is not universal: monitorability improvements are strongly data-dependent. In particular, we demonstrate the critical role of data diversity and instruction-following data during RLVR training. We further show that monitorability is orthogonal to capability–improvements in
reasoning performance do not imply increased transparency. Through mechanistic analysis, we attribute monitorability gains primarily to response distribution sharpening (
entropy reduction) and increased
attention to the
prompt, rather than stronger causal reliance on
reasoning traces. We also reveal how monitorability dynamics vary with controlled training and evaluation difficulty. Together, these findings provide a holistic view of how monitorability emerges under RLVR, clarifying when gains are likely to occur and when they are not.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Fixed Budget is No Harder Than Fixed Confidence in Best-Arm Identification up to Logarithmic Factors
Kapilan Balagopalan, Yinan Li, Yao Zhao et al. (+4 more)
The best-arm identification (BAI) problem is one of the most fundamental problems in interactive machine learning, which has two
flavors: the fixed-budget setting (FB) and the fixed-confidence setting (FC). For $K$-armed bandits with a unique best arm, the optimal sample complexities for both settings have been settled down, and they match up to logarithmic factors. This
prompts an interesting research question about the generic, potentially structured BAI problems: is FB harder than FC or the other way around? In this paper, we show that FB is no harder than FC up to logarithmic factors. We do this constructively: we propose a novel
algorithm called FC2FB (fixed confidence to fixed budget), which is a meta
algorithm that takes in an FC
algorithm $\mathcal{A}$ and turn it into an FB
algorithm. We prove that FC2FB enjoys a sample complexity that matches, up to logarithmic factors, that of the sample complexity of $\mathcal{A}$. This means that the optimal FC sample complexity is an upper bound of the optimal FB sample complexity up to logarithmic factors. Our result not only reveals a fundamental relationship between FB and FC, but also has a significant implication: FC2FB combined with existing state-of-the-art FC
algorithms leads to improved sample complexity for a number of FB problems.
Read the full abstract ▾
All authors Kapilan Balagopalan, Yinan Li, Yao Zhao, Tuan Nguyen, Anton Daitche, Houssam Nassif, Kwang-Sung Jun
Machine Learning · Cross-listed from stat.ML
Statistical Guarantees for Reasoning Probes on Looped Boolean Circuits
Anastasis Kratsios, Giulia Livieri and A. Martina Neuman
We study the statistical behavior of
reasoning probes in a stylized model of iterative computation inspired by neural algorithmic
reasoning. The underlying computation is given by a looped Boolean
circuit whose graph is a perfect $\nu$-ary tree ($\nu\ge 2$), with outputs recursively fed back as inputs across computation rounds. A probe observes a sampled subset of internal nodes and seeks to infer the latent operation at each node, represented as a probability distribution over a finite set of admissible Boolean gates. This partial observability induces a transductive generalization problem on a structured computation graph. We show that when the probe is parameterized by a graph convolutional network and queries $N$ nodes, the worst-case generalization error decays at the optimal rate $\mathcal{O}(\sqrt{\log(2/\delta)}/\sqrt{N})$ with probability at least $1-\delta$. Our analysis combines metric
embedding techniques with tools from optimal transport. A key insight is that this rate is achievable independently of the size of the computation graph, enabled by a low-distortion one-dimensional snowflake
embedding of the induced graph metric. These results highlight a
geometric mechanism underlying statistical efficiency in probing structured, iterative computations.
Read the full abstract ▾
Machine Learning
Non-linear PCA via Evolution Strategies: a Novel Objective Function
Thomas Uriot and Elise Chung
Principal Component Analysis (PCA) is a powerful and popular dimensionality reduction technique. However, due to its linear nature, it often fails to capture the complex underlying structure of real-world data. While Kernel PCA (kPCA) addresses non-linearity, it sacrifices interpretability and struggles with hyperparameter selection. In this paper, we propose a robust non-linear PCA framework that unifies the interpretability of PCA with the flexibility of neural networks. Our method parametrizes variable transformations via neural networks, optimized using
Evolution Strategies (ES) to handle the non-differentiability of eigendecomposition. We introduce a novel, granular objective function that maximizes the individual variance contribution of each variable providing a stronger learning signal than global variance maximization. This approach natively handles categorical and ordinal variables without the dimensional explosion associated with one-hot encoding. We demonstrate that our method significantly outperforms both linear PCA and kPCA in explained variance across synthetic and real-world
datasets. At the same time, it preserves PCA's interpretability, enabling visualization and analysis of feature contributions using standard tools such as biplots. The code can be found on GitHub.
Read the full abstract ▾
Machine Learning
Temporal Validation Changes the Apparent Public-Health Utility of Under-Five Mortality Prediction in Bangladesh: A Four-Round DHS Machine-Learning Study
Md Muhtasim Munif Fahim, M. Monimul Huq, M. Sabiruzzaman et al. (+1 more)
Background: Under-five mortality in Bangladesh remains uneven despite national progress. DHS-based prediction models may guide targeted follow-up, but only if validation reflects future use. We examined how validation design changes apparent prediction performance. Methods: Four BDHS rounds (2011-2022; 33,962 children; 1,290 deaths) were analysed with a 26-feature pipeline and three model classes under four validation regimes, including cross-survey temporal validation (train 2011+2014, calibrate 2017, test 2022). A 32-unit ELU multilayer perceptron was selected via genetic-
algorithm neural architecture search. AUROC used 2,000 bootstrap resamples; screening utility used sensitivity, PPV, and number needed to screen (NNS) at fixed capacity. Results: Validation regime altered public-health interpretation more than model class. NAS MLP AUROC ranged from 0.669 (2022-only random) to 0.775 (pooled random), with temporal AUROC 0.730. At the top-10% temporal threshold, NAS identified 152/355 deaths in 2022 (sensitivity 42.8%, PPV 13.2%, NNS 7.6). NNS across designs ranged from 5.6 to 11.0. Conclusions: Validation-regime choice changed screening workload and apparent policy value more than architecture. Temporal validation supports defensible estimates of follow-up and referral demand; DHS child-mortality studies should report sensitivity, PPV, and NNS before programmatic use.
Read the full abstract ▾
All authors Md Muhtasim Munif Fahim, M. Monimul Huq, M. Sabiruzzaman, Md Rezaul Karim
Machine Learning · Cross-listed from stat.ML
Learning Multi-type heterogeneous interacting particle systems
Quanjun Lang, Xiong Wang, Fei Lu et al. (+1 more)
We propose a framework for the joint inference of network
topology, multi-type interaction kernels, and latent type assignments in heterogeneous interacting particle systems from multi-trajectory data. This learning task is a challenging non-convex mixed-integer optimization problem, which we address through a novel three-stage approach. First, we leverage shared structure across
agent interactions to recover a low-rank
embedding of the system parameters via
matrix sensing. Second, we identify discrete interaction types by clustering within the learned
embedding. Third, we recover the network weight
matrix and kernel coefficients through
matrix factorization and a post-processing refinement. We provide theoretical guarantees with estimation error bounds under a Restricted Isometry Property (RIP) assumption and establish conditions for the exact recovery of interaction types based on cluster separability. Numerical experiments on synthetic
datasets, including heterogeneous predator-prey systems, demonstrate that our method yields an accurate reconstruction of the underlying dynamics and is robust to noise.
Read the full abstract ▾
All authors Quanjun Lang, Xiong Wang, Fei Lu, Mauro Maggioni
Machine Learning
Representation Geometry as a Diagnostic for Out-of-Distribution Robustness
Ali Zia and Farid Hazratian
Robust generalization under distribution shift remains difficult to monitor and optimize in the absence of target-domain labels, as models with similar in-distribution accuracy can exhibit markedly different out-of-distribution (OOD) performance. While prior work has focused on training-time regularization and low-order representation statistics, little is known about whether the
geometric structure of learned
embeddings provides reliable post-hoc signals of robustness. We propose a geometry-based diagnostic framework that constructs class-conditional mutual k-nearest-neighbor graphs from in-distribution
embeddings and extracts two complementary invariants: a global spectral complexity proxy based on the reduced log-determinant of the normalized
Laplacian, and a local smoothness measure based on Ollivier–
Ricci curvature. Across multiple architectures, training regimes, and corruption
benchmarks, we find that lower spectral complexity and higher mean
curvature consistently predict stronger OOD accuracy across checkpoints. Controlled perturbations and topological analyses further show that these signals reflect meaningful representation structure rather than superficial
embedding statistics. Our results demonstrate that representation geometry enables interpretable, label-free robustness diagnosis and supports reliable unsupervised checkpoint selection under distribution shift.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Enhancing Mathematical Problem Solving in LLMs through Execution-Driven Reasoning Augmentation
Aditya Basarkar, Benyamin Tabarsi, Tiffany Barnes et al. (+1 more)
Mathematical problem solving is a fundamental
benchmark for assessing the
reasoning capabilities of artificial intelligence and a gateway to applications in education, science, and engineering where reliable symbolic
reasoning is essential. Although recent advances in multi-
agent LLM-based systems have enhanced their mathematical
reasoning capabilities, they still lack a reliably revisable representation of the
reasoning process. Existing
agents either operate in rigid sequential pipelines that cannot correct earlier steps or rely on heuristic self-evaluation that can fail to identify and fix errors. In addition, programmatic context can distract
language models and degrade accuracy. To address these gaps, we introduce Iteratively Improved Program Construction (IIPC), a
reasoning method that iteratively refines programmatic
reasoning chains and combines execution feedback with the native Chain-of-thought abilities of the base LLM to maintain high-level contextual focus. IIPC surpasses competing approaches in the majority of
reasoning benchmarks on multiple base LLMs. All code and implementations are released as open source.
Read the full abstract ▾
All authors Aditya Basarkar, Benyamin Tabarsi, Tiffany Barnes, Dongkuan Xu
Machine Learning · Cross-listed from cs.IT
Semantic Rate Distortion and Posterior Design: Compute Constraints, Multimodality, and Strategic Inference
Emrah Akyol
We study strategic Gaussian semantic compression under rate and compute constraints, where an encoder and decoder optimize distinct quadratic objectives. A latent Gaussian state generates a task dependent semantic variable, and the decoder best responds via MMSE estimation, reducing the encoder's problem to posterior covariance design under an information rate constraint. We characterize the strategic rate distortion function in direct, remote, and full information regimes, derive semantic waterfilling and rate constrained Gaussian persuasion solutions, and establish Gaussian optimality under misaligned objectives. We further show that architectural compute limits act as implicit rate constraints, yielding exponential improvements in semantic accuracy with model depth and inference time compute, while
multimodal observation eliminates the
geometric mean penalty inherent to remote encoding. These results provide information theoretic foundations for data and energy efficient AI and offer a principled interpretation of modern
multimodal language models as posterior design mechanisms under resource constraints.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Privacy utility trade offs for parameter estimation in degree heterogeneous higher order networks
Bibhabasu Mandal and Sagnik Nandy
In sensitive applications involving relational
datasets, protecting information about individual links from
adversarial queries is of paramount importance. In many such settings, the available data are summarized solely through the degrees of the nodes in the network. We adopt the $\beta$ model, which is the prototypical statistical model adopted for this form of aggregated relational information, and study the problem of minimax-optimal parameter estimation under both local and central differential
privacy constraints. We establish finite sample minimax lower bounds that characterize the precise dependence of the estimation risk on the network size and the
privacy parameters, and we propose simple
estimators that achieve these bounds up to constants and logarithmic factors under both local and central differential
privacy frameworks. Our results provide the first comprehensive finite sample characterization of
privacy utility trade offs for parameter estimation in $\beta$ models, addressing the classical graph case and extending the analysis to higher order hypergraph models. We further demonstrate the effectiveness of our methods through experiments on synthetic data and a real world communication network.
Read the full abstract ▾
Machine Learning
Grables: When Graph Structure Helps Tabular Prediction
Tamara Cucumides and Floris Geerts
Simple row-wise predictors that score each row independently remain competitive with far more expressive architectures on tabular
benchmarks, which are i.i.d. by construction. They cannot, however, represent natural targets on transactional, temporal, and relational tables, where labels depend on other rows. We show that row-wise prediction provably rules out targets driven by global counts, overlaps, and relational patterns. To make “using structure” precise across architectures, we introduce grables: a modular interface that separates how a table is lifted to a graph (constructor) from how predictions are computed on that graph (node predictor), placing gradient-boosted trees, neighbourhood aggregation,
transformer-style tabular models, and relational GNNs on a single axis and pinpointing where expressive power comes from. The framework yields a hierarchy that answers when structure is needed: row-local models suffice for extension-invariant targets, lightweight neighbourhood aggregation captures shallow inter-row signals, and explicit graph construction is needed for compositional patterns that fixed aggregations cannot express without exponential feature blow-up. Experiments on synthetic, transactional, and relational data confirm the predicted separations, but also show that
benchmark performance is a poor proxy for structural capability: row-local models score well on tasks they provably cannot represent, and a stress-test that suppresses row-local cues is needed to expose the gap. Conversely, access to cross-row context does not imply a model uses it. Feeding explicitly extracted inter-row structure to strong tabular learners yields gains exactly where the hierarchy predicts. The practical recipe is to add structure incrementally, and only where the target is extension-sensitive.
Read the full abstract ▾
Machine Learning
Autonomous AI Agents for Real-Time Affordable Housing Site Selection: Multi-Objective Reinforcement Learning Under Regulatory Constraints
Olaf Yunus Laitinen Imanov, Duygu Erisken, Derya Umut Kulali et al. (+2 more)
Affordable housing shortages affect billions, while land scarcity and regulations make site selection slow. We present AURA (Autonomous Urban Resource Allocator), a hierarchical multi-
agent reinforcement learning system for real-time affordable housing site selection under hard regulatory constraints (QCT, DDA, LIHTC). We model the task as a constrained multi-objective Markov decision process optimizing accessibility, environmental impact, construction cost, and social equity while enforcing feasibility. AURA uses a regulatory-aware state encoding 127 federal and local constraints, Pareto-constrained policy gradients with feasibility guarantees, and reward decomposition separating immediate costs from long-term social outcomes. On
datasets from 8 U.S. metros (47,392 candidate parcels), AURA attains 94.3% regulatory compliance and improves Pareto hypervolume by 37.2% over strong baselines. In a New York City 2026 case study, it reduces selection time from 18 months to 72 hours and identifies 23% more viable sites; chosen sites have 31% better transit access and 19% lower environmental impact than expert picks.
Read the full abstract ▾
All authors Olaf Yunus Laitinen Imanov, Duygu Erisken, Derya Umut Kulali, Taner Yilmaz, Rana Irem Turhan
Machine Learning · Cross-listed from eess.SY
C-IDS: Solving Contextual POMDP via Information-Directed Objective
Chongyang Shi, Michael Dorothy and Jie Fu
We study the policy synthesis problem in contextual partially observable Markov decision processes (CPOMDPs), where the environment is governed by an unknown latent context that induces distinct POMDP dynamics. Our goal is to design a policy that simultaneously maximizes cumulative return and actively reduces uncertainty about the underlying context. We introduce an information-directed objective that augments reward maximization with mutual information between the latent context and the
agent's observations. We develop the C-IDS
algorithm to synthesize policies that maximize the information-directed objective. We show that the objective can be interpreted as a Lagrangian relaxation of the linear information ratio and prove that the temperature parameter is an upper bound on the information ratio. Based on this characterization, we establish a sublinear
Bayesian regret bound over K episodes. We evaluate our approach on a continuous Light-Dark environment and show that it consistently outperforms standard POMDP solvers that treat the unknown context as a latent state variable, achieving faster context identification and higher returns.
Read the full abstract ▾
Machine Learning
WIND: Weather Inverse Diffusion for Zero-Shot Atmospheric Modeling
Michael Aich, Andreas Fürst, Florian Sestak et al. (+3 more)
Deep learning has revolutionized weather
forecasting, but many challenges remain, including
climate modeling. Moreover, the current landscape remains fragmented: highly specialized models are typically trained individually for distinct tasks. To unify this landscape, we introduce WIND, a single pre-trained foundation model capable of replacing specialized baselines across a vast array of tasks. Crucially, in contrast to previous atmospheric foundation models, we achieve this without any task-specific fine-tuning. To learn a robust, task-agnostic prior of the atmosphere, we pre-train WIND with a self-supervised
video reconstruction objective, utilizing an unconditional
video diffusion model to iteratively reconstruct atmospheric dynamics from a noisy state. At inference, we frame diverse domain-specific problems strictly as inverse problems and solve them via posterior sampling. This unified approach allows us to tackle highly relevant weather and
climate problems, including probabilistic
forecasting, spatial and temporal downscaling, reconstruction of spatial fields from sparse observations and enforcing global dry air mass conservation. We further demonstrate how WIND can be applied to explore extreme weather events under prescribed out-of-distribution thermodynamic perturbations. By combining generative
video modeling with inverse problem solving, WIND offers a computationally efficient alternative for AI-based atmospheric modeling.
Read the full abstract ▾
All authors Michael Aich, Andreas Fürst, Florian Sestak, Carlos Ruiz-Gonzalez, Niklas Boers, Johannes Brandstetter
Machine Learning
Online Vector Quantized Attention
Nick Alonso, Tomas Figliolia and Beren Millidge
Standard sequence mixing layers used in
language models struggle to balance efficiency and performance. Self-
attention performs well on long context tasks but has expensive quadratic compute and linear
memory costs, while linear
attention and SSMs use only linear compute and constant
memory but struggle with long context processing. In this paper, we develop a sequence mixing layer that aims to find a better compromise between
memory-compute costs and long-context processing, which we call online vector-quantized (OVQ)
attention. OVQ-
attention requires linear compute costs and constant
memory, but, unlike linear
attention and SSMs, it uses a sparse
memory update that allows it to greatly increase the size of its
memory state and, consequently,
memory capacity. We develop a theoretical basis for OVQ-
attention based on Gaussian mixture regression, and we test it on a variety of synthetic long context tasks and on long context
language modeling. OVQ-
attention shows significant improvements over linear
attention baselines and the original VQ-
attention, on which OVQ-
attention was inspired. It demonstrates competitive, and sometimes identical, performance to strong self-
attention baselines up 64k sequence length, despite using a small fraction of the
memory of full self-
attention.
Read the full abstract ▾
Machine Learning
SpecMD: A Comprehensive Study On Speculative Expert Prefetching
Duc Hoang, Ajay Jaiswal, Mohammad Samragh et al. (+1 more)
Mixture-of-Experts (MoE) models enable sparse expert activation, meaning that only a subset of the model's parameters is used during each inference. However, to translate this sparsity into practical performance, an expert caching mechanism is required. Previous works have proposed hardware-centric caching policies, but how these various caching policies interact with each other and different hardware specification remains poorly understood. To address this gap, we develop SpecMD, a standardized framework for benchmarking ad-hoc cache policies on various hardware configurations. Using SpecMD, we perform an exhaustive benchmarking of several MoE caching strategies, reproducing and extending prior approaches in controlled settings with realistic constraints. Our experiments reveal that MoE expert access is not consistent with temporal locality assumptions (e.g LRU, LFU). Motivated by this observation, we propose Least-Stale, a novel eviction policy that exploits MoE's predictable expert access patterns to reduce collision misses by up to $85\times$ over LRU. With such gains, we achieve over $88\%$ hit rates with up to $34.7\%$ Time-to-first-
token (TTFT) reduction on OLMoE at only $5\%$ or $0.6GB$ of VRAM cache capacity.
Read the full abstract ▾
All authors Duc Hoang, Ajay Jaiswal, Mohammad Samragh, Minsik Cho
Machine Learning · Cross-listed from cs.CV
SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
Azmine Toushik Wasi, Wahid Faisal, Abdur Rahman et al. (+12 more)
Spatial
reasoning is a fundamental aspect of human cognition, yet it remains a major challenge for contemporary
vision-
language models (VLMs). Prior work largely relied on synthetic or LLM-generated environments with limited task designs and puzzle-like setups, failing to capture the real-world complexity, visual noise, and diverse spatial relationships that VLMs encounter. To address this, we introduce SpatiaLab, a comprehensive
benchmark for evaluating VLMs' spatial
reasoning in realistic, unconstrained contexts. SpatiaLab comprises 1,400 visual question-answer pairs across six major categories: Relative Positioning, Depth & Occlusion, Orientation, Size & Scale, Spatial
Navigation, and 3D Geometry, each with five subcategories, yielding 30 distinct task types. Each subcategory contains at least 25 questions, and each main category includes at least 200 questions, supporting both multiple-choice and open-ended evaluation. Experiments across diverse state-of-the-art VLMs, including open- and closed-source models,
reasoning-focused, and specialized spatial
reasoning models, reveal a substantial gap in spatial
reasoning capabilities compared with humans. In the multiple-choice setup, InternVL3.5-72B achieves 54.93% accuracy versus 87.57% for humans. In the open-ended setting, all models show a performance drop of around 10-25%, with GPT-5-mini scoring highest at 40.93% versus 64.93% for humans. These results highlight key limitations in handling complex spatial relationships, depth perception,
navigation, and 3D geometry. By providing a diverse, real-world evaluation framework, SpatiaLab exposes critical challenges and opportunities for advancing VLMs' spatial
reasoning, offering a
benchmark to guide future research toward robust, human-aligned spatial understanding. SpatiaLab is available at: https://spatialab-
reasoning.github.io/.
Read the full abstract ▾
All authors Azmine Toushik Wasi, Wahid Faisal, Abdur Rahman, Mahfuz Ahmed Anik, Munem Shahriar, Mohsin Mahmud Topu, Sadia Tasnim Meem, Rahatun Nesa Priti, Sabrina Afroz Mitu, Md. Iqramul Hoque, Shahriyar Zaman Ridoy, Mohammed Eunus Ali, Majd Hawasly, Mohammad Raza, Md Rizwan Parvez
Machine Learning · Cross-listed from cs.CV
Phaedra: Learning High-Fidelity Discrete Tokenization for the Physical Science
Levi Lingsch, Georgios Kissas, Johannes Jakubik et al. (+1 more)
Tokens are discrete representations that allow modern deep learning to scale by transforming high-dimensional data into sequences that can be efficiently learned, generated, and generalized to new tasks. While foundational for image and
video generation, the application of
tokens to physical simulation remains nascent. Because existing tokenizers are designed for the perceptual requirements of natural images, they struggle with scientific data, which exhibits large dynamic ranges and requires exact preservation of physical and spectral properties. In this work, we investigate the performance of a suite of image tokenizers across metrics designed to measure PDE fidelity. Observing that these baselines struggle to simultaneously capture fine
geometric details and precise physical magnitudes, we propose Phaedra, a novel tokenizer inspired by classical shape-gain quantization and the paradigm of basis functions coupled with continuous coefficients. Phaedra acts as a highly effective nonlinear compression
algorithm, massively reducing
dataset footprints while maintaining physical fidelity. We demonstrate that Phaedra consistently improves reconstruction across diverse 2D gridded PDE solutions, generalizes robustly to unseen PDE types and real-world Earth observation data, and is competitive with continuous models in downstream proof-of-concept operator learning and masked autoencoding tasks.
Read the full abstract ▾
All authors Levi Lingsch, Georgios Kissas, Johannes Jakubik, Siddhartha Mishra
Machine Learning
Causal Discovery for Cross-Sectional Data Based on Super-Structure and Divide-and-Conquer
Wenyu Wang and Yaping Wan
This paper tackles a critical bottleneck in Super-Structure-based divide-and-conquer causal discovery: the high computational cost of constructing accurate Super-Structures–particularly when conditional independence (CI) tests are expensive and domain knowledge is unavailable. We propose a novel, lightweight framework that relaxes the strict requirements on Super-Structure construction while preserving the algorithmic benefits of divide-and-conquer. By integrating weakly constrained Super-Structures with efficient graph partitioning and merging strategies, our approach substantially lowers CI test overhead without sacrificing accuracy. We instantiate the framework in a concrete causal discovery
algorithm and rigorously evaluate its components on synthetic data. Comprehensive experiments on Gaussian
Bayesian networks, including magic-NIAB, ECOLI70, and magic-IRRI, demonstrate that our method matches or closely approximates the structural accuracy of PC and FCI while drastically reducing the number of CI tests. Further validation on the real-world China Health and Retirement Longitudinal Study (CHARLS)
dataset confirms its practical applicability. Our results establish that accurate, scalable causal discovery is achievable even under minimal assumptions about the initial Super-Structure, opening new avenues for applying divide-and-conquer methods to large-scale, knowledge-scarce domains such as biomedical and social science research.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Entropy-Aware Structural Alignment for Zero-Shot Handwritten Chinese Character Recognition
Qiuming Luo, Tao Zeng, Feng Li et al. (+3 more)
Zero-shot Handwritten Chinese Character Recognition (HCCR) aims to recognize unseen characters by leveraging radical-based semantic compositions. However, existing approaches often treat characters as flat radical sequences, neglecting the hierarchical
topology and the uneven information density of different components. To address these limitations, we propose an
Entropy-Aware Structural
Alignment Network that bridges the visual-semantic gap through information-theoretic modeling. First, we introduce an Information
Entropy Prior to dynamically modulate positional
embeddings via multiplicative interaction, acting as a saliency detector that prioritizes discriminative roots over ubiquitous components. Second, we construct a Dual-View Radical Tree to extract multi-granularity structural features, which are integrated via an adaptive Sigmoid-based gating network to encode both global layout and local spatial roles. Finally, a Top-K Semantic Feature Fusion mechanism is devised to augment the decoding process by utilizing the centroid of semantic neighbors, effectively rectifying visual ambiguities through feature-level consensus. Extensive experiments demonstrate that our method establishes new state-of-the-art performance, achieving an accuracy of 55.04% on the ICDAR 2013
dataset ($m=1500$), significantly outperforming existing CLIP-based baselines in the challenging zero-shot setting. Furthermore, the framework exhibits exceptional data efficiency, demonstrating rapid adaptability with minimal support samples, achieving 92.41% accuracy with only one support sample per class.
Read the full abstract ▾
All authors Qiuming Luo, Tao Zeng, Feng Li, Heming Liu, Rui Mao, Chang Kong