Machine Learning · Cross-listed from cs.AI
Interval Certifications for Multilayered Perceptrons via Lattice Traversal
Merkouris Papamichail, Konstantinos Varsos, Giorgos Flouris et al. (+1 more)
In this work we present a rigorous theoretical framework to a foundational problem of AI safety, namely
adversarial robustness. In particular, we show that the
adversarial robustness problem can be reduced to a
lattice traversal problem. Each element of this
lattice corresponds to an interval, i.e., an axis-aligned hyper-rectangle, containing an input point $\mathbf{x}$. Consider a multilayered perceptron classifier (MLP). An interval $I$ constitutes a sound certification if $\mathbf{x} \in I$ and $\mathbf{x}$ can be freely perturbed in $I$ without changing the MLP's prediction. Complementarily, an interval $I$ constitutes a complete certification if $\mathbf{x} \in I$ and when $\mathbf{x}$ moves outside of $I$ the MLP's prediction is guaranteed to change. While the sound certification problem corresponds to the well-studied
adversarial robustness, complete certifications have not been examined in the literature. We develop
lattice traversal operators, which we apply in a refine & verify iterative scheme. Using formal MLP verifiers, sound maximality and complete minimality are guaranteed. Moreover, we examine objective optimization problems. There we discover some interesting asymmetries. For complete certifications, the minimum solution is obtained in
polynomial oracle calls. This does not hold for sound certifications, where we prove strong intractability results. Additionally, we examine optimization problems in symmetric intervals (i.e., $\ell_\infty$-spheres), where we provide logarithmic
algorithms. Finally, we present an empirical evaluation, using the novel ParallelepipedoNN system.
Read the full abstract ▾
All authors Merkouris Papamichail, Konstantinos Varsos, Giorgos Flouris, João Marques-Silva
Machine Learning
Heterogeneous Graph Importance Scoring and Clustering with Automated LLM-based Interpretation
Takato Yasuno
Urban bridge networks are critical infrastructure whose disruption can cascade into severe impacts on transportation, emergency services, and economic activity. This paper presents a comprehensive methodology for assessing bridge importance through heterogeneous graph analysis, unsupervised clustering, and automated interpretation via large
language models (LLMs). Our approach addresses three fundamental challenges: (1) quantifying multi-dimensional bridge importance using only open data sources, (2) discovering functional bridge archetypes across different cities, and (3) generating policy-relevant interpretations automatically. We construct heterogeneous graphs from OpenStreetMap (OSM) data incorporating bridges, road networks, buildings, and public facilities. Five social impact indicators are computed: transit desert score, hospital access score, isolation risk score, supply chain impact score, and green space access score. These 52-dimensional feature vectors undergo dimensionality reduction via UMAP and density-based clustering via HDBSCAN. Discovered clusters are interpreted using temperature-optimized LLMs (Elyza8b, trained on construction domain corpus). (1) A complete open-data pipeline from OSM to actionable bridge importance rankings, (2) a five-indicator scoring methodology with 40$\times$ computational optimization, (3) a UMAP+HDBSCAN clustering framework validated on multi-city data, (4) an LLM interpretation methodology including temperature optimization and model selection rationale, and (5) transferability demonstration across cities via configuration-only adaptation.
Read the full abstract ▾
Machine Learning
Mitigating the reconstruction-detection trade-off in VAE-based unsupervised anomaly detection
Agathe Senellart, Maëlys Solal, Stéphanie Allassonnière et al. (+1 more)
Variational autoencoders are widely used for unsupervised anomaly detection. Model selection however remains an open-question: to remain fully unsupervised, hyperparameters are often chosen to minimize the reconstruction error on normal samples. In this paper, we reveal a trade-off between reconstruction quality and anomaly detection among $\beta$-VAE models. Models with constrained latent space reach higher detection metrics but lower reconstruction quality. We also assess the performance variability across random seeds and show it is linked to the distance between normal and abnormal latent distributions. From this analysis, we justify and investigate two methods to mitigate the reconstructiondetection tradeoff: beta-scheduling and the Sparse VAE. The latter especially shows an improvement in detection while maintaining high reconstruction quality.
Read the full abstract ▾
All authors Agathe Senellart, Maëlys Solal, Stéphanie Allassonnière, Ninon Burgos
Machine Learning
PRISM-CTG: A Foundation Model for Cardiotocography Analysis with Multi-View SSL
Sheng Wong, Ravi Shankar, Beth Albert et al. (+5 more)
Supervised deep learning models for automated CTG analysis are typically constrained by narrowly curated labelled
datasets and limited patient cohorts, leaving substantial volumes of physiologically informative clinical recordings untapped. To address this limitation, we propose Physiology-aware Representation Learning via Integrated Self-supervision and Metadata for CTG (PRISM-CTG), a clinically grounded self-supervised foundation model (FM) for CTG that leverages large-scale unlabelled recordings to learn transferable domain-level representations. PRISM-CTG is pretrained using a multi-view self-supervised framework that jointly optimises 3 complementary pretext objectives: random-projected guided masked signal reconstruction, clinical variable prediction, and feature classification. Each objective is associated with a dedicated task-specific
token, enabling specialised representation learning, while controlled cross-
attention facilitates information exchange across clinical context. By reframing patient metadata and domain knowledge, which are often underutilised in conventional training as prediction targets, Prism-CTG transforms readily available clinical information into additional supervisory targets that guide clinically meaningful representation learning. Extensive experiments across 7 downstream CTG tasks in both antepartum and intrapartum domains demonstrated that PRISM-CTG consistently outperforms in-domain and SSL baselines. Notably, PRISM-CTG demonstrated strong generalisation under external validation on 2
datasets, while achieving comparable performance to studies trained on substantially larger, privately labelled
datasets. To our knowledge, this is the first study to introduce large-scale FM for CTG that learns domain-level representations.
Read the full abstract ▾
All authors Sheng Wong, Ravi Shankar, Beth Albert, Hao Fei, Lin Li, Imane Ben M'Barek, Manu Vatish, Gabriel Davis Jones
Machine Learning
From Synthesis to Clinical Assistance: A Strategy-Aware Agent Framework for Autism Intervention based on Real Clinical Dataset
Junhong Lai, Shuzhong Lai, Yanhao Yu et al. (+5 more)
The development of AI-assisted Early Intensive Behavioral Intervention (EIBI) for Autism Spectrum Disorder (ASD) is severely constrained by data scarcity. Furthermore, while Applied Behavior Analysis (ABA) serves as the gold standard for clinical intervention, general-purpose Large
Language Models (LLMs) struggle to strictly adhere to its standardized procedures, often resulting in interactions that are linguistically fluent but strategically inconsistent. To address these challenges, we introduce ASDAgent, a strategy-aware framework designed to unify high-fidelity intervention dialogue synthesis and clinical decision support. ASDAgent incorporates two specialized components to solve distinct problems: (i) a DoctorAgent equipped with an Observe-Think-Act-Correct (O-T-A-C)
reasoning loop, which resolves the issue of strategy collapse in LLMs by making ABA execution explicit and controllable; and (ii) a ChildAgent that utilizes probabilistic behavior modeling to mitigate data homogeneity, simulating diverse and non-deterministic ASD response patterns. Experiments demonstrate that dialogues generated by ASDAgent closely mirror the strategy distribution of human therapists (KL divergence: 0.083). In real autism intervention, ASDAgent achieves nearly 80% strategic consistency with human experts. Moreover, we show that synthetic data produced by ASDAgent effectively distills professional clinical knowledge into small
language models (SLMs), significantly enhancing their therapeutic capabilities.
Read the full abstract ▾
All authors Junhong Lai, Shuzhong Lai, Yanhao Yu, Wanlin Chen, Chenyu Yan, Haifeng Li, Lin Yao, Yueming Wang
Machine Learning · Cross-listed from q-bio.QM
Dual Triangle Attention: Effective Bidirectional Attention Without Positional Embeddings
Logan Hallee and Jason P. Gleghorn
Bidirectional
transformers are the foundation of many sequence modeling tasks across natural, biological, and chemical
language domains, but they are permutation-invariant without explicit positional
embeddings. In contrast, unidirectional
attention inherently encodes positional information through its triangular mask, enabling models to operate without positional
embeddings altogether. Here, we introduce Dual Triangle
Attention, a novel bidirectional
attention mechanism that separates the query-key subspace of each
attention head into two complementary triangular masks: one that attends to past-and-self positions and one that attends to future-and-self positions. This design provides bidirectional context while maintaining the causal mask's implicit positional inductive bias in both directions. Using PyTorch's flex_
attention, Dual Triangle
Attention is implemented as a single compiled kernel call with no additional parameters beyond standard multi-head
attention. We evaluated Dual Triangle
Attention across three settings: (1) a synthetic argmax position probe, (2) masked
language modeling (MLM) on natural
language, and (3) MLM on
protein sequences. In the argmax task, both Dual Triangle
Attention and causal
attention learn positional information without explicit positional
embeddings, whereas standard bidirectional
attention cannot. In the MLM experiments, Dual Triangle
Attention with Rotary Positional
Embeddings (RoPE) achieved the best context extension performance and strong performance across the board. These findings suggest that Dual Triangle
Attention is a viable
attention mechanism for bidirectional
transformers, with or without positional
embeddings.
Read the full abstract ▾
Machine Learning
Multi-Label Phase Diagram Prediction in Complex Alloys via Physics-Informed Graph Attention Networks
Eunjeong Park and Amrita Basak
Accurate phase
equilibria are foundational to
alloy design because they encode the underlying thermodynamics governing stability, transformations, and processing windows. However, while the CALculation of Phase Diagrams (CALPHAD) provides a rigorous thermodynamic framework, exploring multicomponent composition-temperature space remains computationally expensive and is typically limited to sparse section. To enable rapid phase mapping and
alloy screening, we propose a physics-informed graph
attention network (GAT) that learns element-aware representations and couples them with thermodynamic constraints for multi-label phase-set prediction in the Ag-Bi-Cu-Sn
alloy system. Using about 25,000
equilibrium states generated with pycalphad, each composition-temperature point is represented as a four-node element graph with atomic fractions and elemental descriptors as node features. The model combines graph
attention, global pooling, and a multilayer perceptron to predict nine relevant phases. To improve physical consistency, we incorporate thermodynamic constraints, applied as training penalties or as an inference-time projection. Across six binary and three ternary subsystems, the baseline model achieves a macro-F1 score of 0.951 and 93.98% exact-set match, while physics-informed decoding improves robustness and raises exact-set accuracy to about 96% on dense in-domain grids. The surrogate also generalizes to an unseen ternary section with 99.32% exact-set accuracy and to a quaternary section at 700 {\deg}C with 91.78% accuracy. These results demonstrate that
attention-based graph learning coupled with thermodynamic constraint enforcement provides an effective and physically consistent surrogate for high-resolution phase mapping and extrapolative
alloy screening.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.HC
Functional Emotions or Situational Contexts? A Discriminating Test from the Mythos Preview System Card
Hiranya V. Peiris
The Claude Mythos Preview system card deploys
emotion vectors, sparse autoencoder (SAE) features, and activation verbalisers to study model internals during misaligned behaviour. The two primary toolkits are not jointly reported on the most
alignment-relevant episodes. This note identifies two hypotheses that are qualitatively consistent with the published results: that the
emotion vectors track functional
emotions that causally drive behaviour, or that they are a projection of a richer situational-context structure onto human emotional axes. The hypotheses can be distinguished by cross-referencing the two toolkits on episodes where only one is currently reported: most directly, applying
emotion probes to the strategic concealment episodes analysed only with SAE features. If
emotion probes show flat activation while SAE features are strongly active, the
alignment-relevant structure lies outside the
emotion subspace. Which hypothesis is correct determines whether
emotion-based monitoring will robustly detect dangerous model behaviour or systematically miss it.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Improving DNS Exfiltration Detection via Transformer Pretraining
Miloš Tomić, Aleksa Cvetanović and Predrag Tadić
We study whether in-domain pretraining of Bidirectional Encoder Representations from
Transformer (BERT) model improves subdomain-level detection of exfiltration at low false positive rates. While previous work mostly examines fine-tuned generic
Transformers, it does not aim to isolate the effect of pretraining on the downstream task of classification. To address this gap, we develop a controlled pipeline where we freeze operating points on validation and transfer them to the test set, thus enabling clean ablations across different label and pretraining budgets. Our results show significant improvements in the left tail of the Receiver Operating Characteristic (ROC) curve, especially against randomly initialized baseline. Additionally, within pretrained model variants, increasing the number of pretraining steps helps the most when more labeled data are available for fine-tuning.
Read the full abstract ▾
Machine Learning
ExecTune: Effective Steering of Black-Box LLMs with Guide Models
Vijay Lingam, Aditya Golatkar, Anwesan Pal et al. (+6 more)
For large
language models deployed through black-box APIs, recurring inference costs often exceed one-time training costs. This motivates composed agentic systems that amortize expensive
reasoning into reusable intermediate representations. We study a broad class of such systems, termed Guide-Core Policies (GCoP), in which a guide model generates a structured strategy that is executed by a black-box core model. This abstraction subsumes base, supervised, and advisor-style approaches, which differ primarily in how the guide is trained. We formalize GCoP under a cost-sensitive utility objective and show that end-to-end performance is governed by guide-averaged executability: the probability that a strategy generated by the guide can be faithfully executed by the core. Our analysis shows that existing GCoP instantiations often fail to optimize executability under deployment constraints, resulting in brittle strategies and inefficient computation. Motivated by these insights, we propose ExecTune, a principled training recipe that combines teacher-guided acceptance sampling, supervised fine-tuning, and structure-aware reinforcement learning to directly optimize syntactic validity, execution success, and cost efficiency. Across mathematical
reasoning and code-generation
benchmarks, GCoP with ExecTune improves accuracy by up to 9.2% over prior state-of-the-art baselines while reducing inference cost by up to 22.4%. It enables Claude Haiku 3.5 to outperform Sonnet 3.5 on both math and code tasks, and to come within 1.7% absolute accuracy of Sonnet 4 at 38% lower cost. Beyond efficiency, GCoP also supports modular adaptation by updating the guide without retraining the core.
Read the full abstract ▾
All authors Vijay Lingam, Aditya Golatkar, Anwesan Pal, Ben Vo, Narayanan Sadagopan, Alessandro Achille, Jun Huan, Anoop Deoras, Stefano Soatto
Machine Learning
STaR-DRO: Stateful Tsallis Reweighting for Group-Robust Structured Prediction
Samah Fodeh, Ganesh Puthiaraju, Elyas Irankhah et al. (+5 more)
Structured prediction with large
language models requires outputs that are label-accurate, ontology-constrained, structurally valid, and evidence-grounded under label imbalance and heterogeneous group difficulty. We present a unified framework for ontology-constrained generation. First, we introduce a modular
prompt-engineering architecture combining XML-style structure, expert disambiguation rules, chain-of-thought
reasoning, metadata-aware decision logic, schema contracts, and a self-validation gate. It targets recurrent in-context failures, including format drift, label ambiguity, evidence
hallucination, and metadata-conditioned confusion. Second, we propose
STaR-DRO, combining Tsallis mirror ascent, sparse entmax-style primal mapback, EMA-smoothed group-loss tracking, rescaled ascent signals, and bounded excess-only multipliers. Unlike conventional DRO, which relies on dense Shannon-
entropy exponentiated-gradient updates, can introduce high-variance
stochastic reweighting, assigns positive
adversarial mass to groups that are not persistently hard, and incurs costs through simplex competition,
STaR-DRO upweights only persistently hard groups without suppressing easier ones. We evaluate the framework on EPPC Miner, a clinically grounded high-stakes structured-prediction task requiring hierarchical label prediction and evidence-span extraction from patient-provider secure messages. Across 1B-70B Llama models,
prompt engineering improves zero-shot extraction, yielding an average label F1 gain of +14.46 and a Span F1 gain of +17.40. Building on supervised fine-tuning,
STaR-DRO further improves accuracy and robustness, increasing average label F1 by +1.08 and +2.20 while reducing mean groupwise validation cross-
entropy by 21.3% and 14.8% relative to SFT and standard DRO, respectively. These results advance reliable automated communication mining for patient-centered clinical care analysis.
Read the full abstract ▾
All authors Samah Fodeh, Ganesh Puthiaraju, Elyas Irankhah, Afshan Khan, Sreeraj Ramachandran, Linhai Ma, Srivani Talakokkul, Sarah Schellhorn
Machine Learning
HiFloat4 Format for Language Model Pre-training on Ascend NPUs
Mehran Taghian, Yunke Peng, Xing Huang et al. (+22 more)
Training large foundation models at low numerical precision is one of the most promising directions for reducing the compute and
memory cost of modern AI. Recent 4-bit floating-point formats such as MXFP4 and NVFP4 can be applied to linear GEMM operations in LLMs, but their limited dynamic range introduces numerical instability that prior work addresses by stacking stabilization mechanisms, typically executed at higher precision and partially eroding the efficiency gains that motivate FP4. In this work, we argue that numerical format design is itself a first-class lever for stable FP4 training, and present the first systematic study of FP4 LLM pretraining on energy-efficient Huawei Ascend NPUs. We compare the recently proposed HiFloat4 (HiF4) format against both MXFP4 and NVFP4, holding one recipe fixed across all three formats across dense (OpenPangu-1B, Llama3-8B) and Mixture-of-Experts (Qwen3-MoE-30B) architectures and executing all linear and expert GEMMs in FP4. At matched storage — NVFP4 and HiF4 both spend 4.5 bits per value — the three formats differ far more in what they require before they will train at all than in final accuracy. HiF4 reaches a relative loss of 1.55% with no stabilization, below fully stabilized MXFP4 (1.79%) and below NVFP4 carrying the per-tensor scaling it cannot train without (2.00%); NVFP4 diverges under every combination of
stochastic rounding and Hadamard transform we tried. Our results suggest that stable, accurate FP4 training does not require an ever-growing stack of stabilization techniques; it requires the right numerical format.
Read the full abstract ▾
All authors Mehran Taghian, Yunke Peng, Xing Huang, Yao Wang, Yaoyuan Wang, Wei Guo, Yuanyong Luo, Tianchi Hu, Junsong Wang, Xin Wang, Hu Liu, Yu Cheng, Ziwei Yu, Hongliang Li, Mehdi Rahimifar, Lei Yan, Xuefei Wang, Zhuang Ma, Lei Liu, Hui Yu, Anandharaju Durai Raju, Hoang Le, Hei Yi Mak, Tanzila Rahman, Shadan Golestan
Machine Learning · Cross-listed from cs.CV
SenBen: Sensitive Scene Graphs for Explainable Content Moderation
Fatih Cagatay Akyon and Alptekin Temizel
Content moderation systems classify images as safe or unsafe but lack spatial grounding and interpretability: they cannot explain what sensitive behavior was detected, who is involved, or where it occurs. We introduce the Sensitive
Benchmark (SenBen), the first large-scale scene graph
benchmark for sensitive content, comprising 13,999 frames from 157 movies annotated with Visual Genome-style scene graphs (25 object classes, 28 attributes including affective states such as pain, fear, aggression, and distress, 14 predicates) and 16 sensitivity tags across 5 categories. We distill a frontier VLM into a compact 241M student model using a multi-task recipe that addresses vocabulary imbalance in autoregressive scene graph generation through suffix-based object identity, Vocabulary-Aware Recall (VAR) Loss, and a decoupled Query2Label tag head with asymmetric loss, yielding a +6.4 percentage point improvement in SenBen Recall over standard cross-
entropy training. On grounded scene graph metrics, our student model outperforms all evaluated VLMs except Gemini models and all commercial safety APIs, while achieving the highest object detection and captioning scores across all models, at $7.6\times$ faster inference and $16\times$ less GPU
memory.
Read the full abstract ▾
Machine Learning
Loom: A Scalable Analytical Neural Computer Architecture
Mehmet Kerem Turkcan
We present Loom, a computer architecture that executes programs compiled from C inside a looped
transformer whose weights are derived analytically. The architecture implements a 22-opcode instruction set in 8
transformer layers. Each forward pass executes one instruction; the model is applied iteratively until the program counter reaches zero. The full machine state resides in a single tensor $X \in \mathbb{R}^{d \times n}$ of fixed size, and every step has fixed cost for fixed $d$ and $n$, independent of program length or execution history. The default configuration uses $d = 155$ and $n = 1024$, yielding 4.7 million parameters and 928 instruction slots. A compact configuration at $d = 146$ and $n = 512$ suffices for a 9$\times$9 Sudoku solver (284 instructions). The weights are program-independent: programs live in the state tensor, and the same fixed-weight model executes any compiled program. We make Loom source code publicly available at https://github.com/mkturkcan/Loom.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
R2G: A Multi-View Circuit Graph Benchmark Suite from RTL to GDSII
Zewei Zhou, Jiajun Zou, Jiajia Zhang et al. (+8 more)
Graph neural networks (GNNs) are increasingly applied to physical design tasks such as congestion prediction and wirelength estimation, yet progress is hindered by inconsistent
circuit representations and the absence of controlled evaluation protocols. We present R2G (RTL-to-GDSII), a multi-view
circuit-graph
benchmark suite that standardizes five stage-aware views with information parity (every view encodes the same attribute set, differing only in where features attach) over 30 open-source IP cores (up to $10^6$ nodes/edges). R2G provides an end-to-end DEF-to-graph pipeline spanning synthesis, placement, and routing stages, together with loaders, unified splits, domain metrics, and reproducible baselines. By decoupling representation choice from model choice, R2G isolates a confound that prior EDA and graph-ML
benchmarks leave uncontrolled. In systematic studies with GINE, GAT, and ResGatedGCN, we find: (i) view choice dominates model choice, with Test R$^2$ varying by more than 0.3 across representations for a fixed GNN; (ii) node-centric views generalize best across both placement and routing; and (iii) decoder-head depth (3–4 layers) is the primary accuracy driver, turning divergent training into near-perfect predictions (R$^2$$>$0.99). Code and
datasets are available at https://github.com/ShenShan123/R2G.
Read the full abstract ▾
All authors Zewei Zhou, Jiajun Zou, Jiajia Zhang, Ao Yang, Ruichao He, Haozheng Zhou, Ao Liu, Jiawei Liu, Leilei Jin, Shan Shen, Daying Sun
Machine Learning
Structural Evaluation Metrics for SVG Generation via Leave-One-Out Analysis
Haonan Zhu, Adrienne Deganutti, Elad Hirsch et al. (+1 more)
SVG generation is typically evaluated by comparing rendered outputs to reference images, which captures visual similarity but not the structural properties that make SVG editable, decomposable, and reusable. Inspired by the classical jackknife, we introduce element-level leave-one-out (LOO) analysis. The procedure renders the SVG with and without each element, which yields element-level signals for quality assessment and structural analysis. From this single mechanism, we derive (i) per-element quality scores that enable zero-shot artifact detection; (ii) element-concept attribution via LOO footprints crossed with VLM-grounded concept heatmaps; and (iii) four structural metrics: purity, coverage, compactness, and locality, which quantify SVG modularity from complementary angles. These metrics extend SVG evaluation from image similarity to code structure, enabling element-level diagnosis and comparison of how visual concepts are represented, partitioned, and organized within SVG code. Their practical relevance is validated on over 19,000 edits (5 types) across 5 generation systems and 3 complexity tiers.
Read the full abstract ▾
All authors Haonan Zhu, Adrienne Deganutti, Elad Hirsch, Purvanshi Mehta
Machine Learning
Smartwatch-Based Sitting Time Estimation in Real-World Office Settings
Olivia Zhang and Zhilin Zhang
Sedentary behavior poses a major public health risk, being strongly linked to obesity, cardiovascular disease, and other chronic conditions. Accurately estimating sitting time is therefore critical for monitoring and improving individual health. This work addresses the problem in real-world office settings, where signals from the inertial measurement units (IMU) on a smartwatch were collected from office workers during their daily routines. We propose a method that estimates sitting time from the IMU signals by introducing the use of rotation vector sequences, derived from Euler angles, as a novel representation of movement dynamics. Experiments on a 34-hour
dataset demonstrate that exploiting rotation vector sequences improves
algorithm performance, highlighting their potential for robust sitting time estimation in natural environments.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Policy-Aware Design of Large-Scale Factorial Experiments
Xin Wen, Xi Chen, Will Wei Sun et al. (+1 more)
Digital firms routinely run many online experiments on shared user populations. When product decisions are compositional, such as combinations of interface elements, flows, messages, or incentives, the number of feasible interventions grows combinatorially, while available traffic remains limited. Overlapping experiments can therefore generate interaction effects that are poorly handled by decentralized A/B testing. We study how to design large-scale factorial experiments when the objective is not to estimate every treatment effect, but to identify a high-performing policy under a fixed experimentation budget. We propose a two-stage design that centralizes overlapping experiments into a single factorial problem and models expected outcomes as a low-rank tensor. In the first stage, the platform samples a subset of intervention combinations, uses tensor completion to infer performance on untested combinations, and eliminates weak factor levels using estimated marginal contributions. In the second stage, it applies sequential halving to the surviving combinations to select a final policy. We establish gap-independent simple-regret bounds and gap-dependent identification guarantees showing that the relevant complexity scales with the degrees of freedom of the low-rank tensor and the separation structure across factor levels, rather than the full factorial size. In an offline evaluation based on a product-bundling problem constructed from 100 million Taobao interactions, the proposed method substantially outperforms one-shot tensor completion and unstructured best-arm
benchmarks, especially in low-budget and high-noise settings. These results show how centralized, policy-aware experimentation can make combinatorial product design operationally feasible at platform scale.
Read the full abstract ▾
All authors Xin Wen, Xi Chen, Will Wei Sun, Yichen Zhang
Machine Learning
Alleviating Community Fear in Disasters via Multi-Agent Actor-Critic Reinforcement Learning
Yashodhan D. Hakke, Almuatazbellah M. Boker, Lamine Mili et al. (+2 more)
During disasters, cascading failures across power grids, communication networks, and social behavior amplify community fear and undermine cooperation. Existing cyber-physical-social (CPS) models simulate these coupled dynamics but lack mechanisms for active intervention. We extend the CPS resilience model of Valinejad and Mili (2023) with control channels for three agencies, communication, power, and emergency management, and formulate the resulting system as a three-player non-zero-sum differential game solved via online actor-critic reinforcement learning. Simulations based on Hurricane Harvey data show 70% mean fear reduction with improved infrastructure recovery; cross-validation in the case of Hurricane Irma (without refitting) achieves 50% fear reduction, confirming generalizability.
Read the full abstract ▾
All authors Yashodhan D. Hakke, Almuatazbellah M. Boker, Lamine Mili, Michael von Spakovsky, Hoda Eldardiry
Machine Learning
$p1$: Better Prompt Optimization with Fewer Prompts
Zhaolin Gao, Yu Wang, Bo Liu et al. (+3 more)
Prompt optimization improves
language models without updating their weights by searching for a better system
prompt, but its effectiveness varies widely across tasks. We study what makes a task amenable to
prompt optimization. We show that the reward variance across different system
prompts can be decomposed into two components: variance among responses, which captures generation stochasticity, and variance among system
prompts, which captures differences in system
prompt quality.
Prompt optimization succeeds when variance among system
prompts is sufficiently large, but fails when variance among responses dominates the variance of the system
prompts. Surprisingly, we further show that scaling to more user
prompts can hurt optimization by reducing variance among system
prompts, especially on heterogeneous
datasets where different user
prompts favor different system
prompts. Motivated by this insight, we propose $p1$, a simple user
prompt filtering method that selects a small subset of user
prompts with high variance across candidate system
prompts. This subset of user
prompts allows one to distinguish a good system
prompt from a bad one, making system optimization easier. Experiments on
reasoning benchmarks show that $p1$ substantially improves
prompt optimization over training on the full
dataset and outperforms strong baselines such as GEPA. Notably, training on only two
prompts from AIME 24 yields a system
prompt that generalizes well to other
reasoning benchmarks.
Read the full abstract ▾
All authors Zhaolin Gao, Yu Wang, Bo Liu, Thorsten Joachims, Kianté Brantley, Wen Sun
Machine Learning · Cross-listed from cs.CR
Tracing the Chain: Deep Learning for Stepping-Stone Intrusion Detection
Nate Mathews, Nicholas Hopper and Matthew Wright
Stepping-stone intrusions (SSIs) are a prevalent network evasion technique in which attackers route sessions through chains of compromised intermediate hosts to obscure their origin. Effective SSI detection requires correlating the incoming and outgoing flows at each relay host at extremely low false positive rates – a stringent requirement that renders classical statistical methods inadequate in operational settings. We apply ESPRESSO, a deep learning flow correlation model combining a
transformer-based feature extraction network, time-aligned multi-channel interval features, and online triplet metric learning, to the problem of stepping-stone intrusion detection. To support training and evaluation, we develop a synthetic data collection tool that generates realistic stepping-stone traffic across five tunneling protocols: SSH, SOCAT, ICMP, DNS, and mixed multi-protocol chains. Across all five protocols and in both host-mode and network-mode detection scenarios, ESPRESSO substantially outperforms the state-of-the-art DeepCoFFEA baseline, achieving a true positive rate exceeding 0.99 at a false positive rate of $10^{-3}$ for standard bursty protocols in network-mode. We further demonstrate chain length prediction as a tool for distinguishing malicious from benign pivoting, and conduct a systematic robustness analysis revealing that timing-based perturbations are the primary vulnerability of correlation-based stepping-stone detectors.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Morphology-Conditioned World Model for Cross-Embodiment Quadrupedal Locomotion
Mohamad H. Danesh, Chenhao Li, Amin Abyaneh et al. (+5 more)
World models promise a paradigm shift in robotics, where an
agent learns the physics of its environment once and then acquires behaviors efficiently. Yet the learned dynamics models at their core are typically morphology locked. In legged locomotion, a dynamics model trained on an ANYmal-D quadruped fails on a Unitree Go1 because it overfits to one
robot's embodiment rather than capturing the locomotion dynamics shared across
robots, so even a small change in actuator dynamics or limb length forces retraining from scratch. However, if we formalize a
robot's unique physical traits into a morphology specification, a controller for a family of
robots can utilize this blueprint in two ways. It can feed the specification to a model-free policy, or it can feed the specification to a learned dynamics model and extract the policy in imagination. We argue for the second route and introduce the Quadrupedal World Model (QWM), which conditions a single generative dynamics model on scale-invariant physical features and trains policies entirely inside it, through a physical morphology encoder, an adaptive reward normalizer, and morphology conditioning in the latent dynamics. Holding the morphology information identical, a model-free policy matches QWM on the training cohort but degrades on unseen morphologies, while QWM transfers zero-shot with no fine-tuning, adaptation, or warm-up in such cases. To our knowledge, this is the first world model to demonstrate zero-shot cross-embodiment transfer within the quadrupedal family.
Read the full abstract ▾
All authors Mohamad H. Danesh, Chenhao Li, Amin Abyaneh, Anas Houssaini, Kirsty Ellis, Glen Berseth, Marco Hutter, Hsiu-Chin Lin
Machine Learning
Adaptive Simulation Experiment for LLM Policy Optimization
Mingjie Hu, Siyang Gao, Jian-qiang Hu et al. (+1 more)
Large
language models (LLMs) have significant potential to improve operational efficiency in operations management. Deploying these models requires specifying a policy that governs response quality, shapes user experience, and influences operational value. In this research, we treat LLMs as
stochastic simulators and propose a pairwise comparison-based adaptive simulation experiment framework for identifying the optimal policy from a finite set of candidates. We consider two policy spaces: an unstructured space with no parametric assumption, and a structured space in which the data are generated from a preference model. For both settings, we characterize the fundamental data requirements for identifying the optimal policy with high probability. In the unstructured case, we derive a closed-form expression for the optimal sampling proportions, together with a clear operational interpretation. In the structured case, we formulate a regularized convex program to compute the optimal proportions. We then develop an adaptive experimental procedure, termed LLM-PO, for both policy spaces, and prove that it identifies the optimal policy with the desired statistical guarantee while asymptotically attaining the fundamental data requirements. Numerical experiments demonstrate that LLM-PO consistently outperforms
benchmark methods and improves LLM performance.
Read the full abstract ▾
All authors Mingjie Hu, Siyang Gao, Jian-qiang Hu, Enlu Zhou
Machine Learning · Cross-listed from physics.ao-ph
CERBERUS: A Three-Headed Decoder for Vertical Cloud Profiles
Emily K. deJong, Nipun Gunawardena, Kevin Smalley et al. (+2 more)
Atmospheric clouds exhibit complex three-dimensional structure and microphysical details that are poorly constrained by the predominantly two-dimensional satellite observations available at global scales. This mismatch complicates data-driven learning and evaluation of cloud processes in weather and
climate models, contributing to ongoing uncertainty in atmospheric physics. We introduce CERBERUS, a probabilistic inference framework for generating vertical radar reflectivity profiles from geostationary satellite brightness temperatures, near-surface meteorological variables, and temporal context. CERBERUS employs a three-headed encoder-decoder architecture to predict a zero-inflated (ZI) vertically-resolved distribution of radar reflectivity. Trained and evaluated using ground-based Ka-band radar observations at the ARM Southern Great Plains site, CERBERUS recovers coherent structures across cloud regimes, generalizes to withheld test periods, and provides uncertainty estimates that reflect physical ambiguity, particularly in multilayer and dynamically complex clouds. These results demonstrate the value of distribution-based learning targets for bridging observational scales, introducing a path toward model-relevant synthetic observations of clouds.
Read the full abstract ▾
All authors Emily K. deJong, Nipun Gunawardena, Kevin Smalley, Hassan Beydoun, Peter Caldwell
Machine Learning · Cross-listed from quant-ph
Weak Adversarial Neural Pushforward Method for the Wigner Transport Equation
Andrew Qing He, Wei Cai and Sihong Shao
We extend the Weak
Adversarial Neural Pushforward Method to the Wigner transport equation governing the phase-space dynamics of
quantum systems. The central contribution is a structural observation: integrating the nonlocal pseudo-differential potential operator against plane-wave test functions produces a Dirac delta that exactly inverts the
Fourier transform defining the Wigner potential kernel, reducing the operator to a pointwise finite difference of the potential at two shifted arguments. This holds in arbitrary dimension, requires no truncation of the Moyal series, and treats the potential as a black-box function oracle with no derivative information. To handle the negativity of the Wigner quasi-probability distribution, we introduce a signed pushforward architecture that decomposes the solution into two non-negative phase-space distributions mixed with a learnable weight. The resulting method inherits the mesh-free, Jacobian-free, and scalable properties of the original framework while extending it to the
quantum setting.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CE
Accurate and Reliable Uncertainty Estimates for Deterministic Predictions Extensions to Under and Overpredictions
Rileigh Bandy, Enrico Camporeale, Andong Hu et al. (+2 more)
Computational models support high-stakes decisions across engineering and science, and practitioners increasingly seek probabilistic predictions to quantify uncertainty in such models. Existing approaches generate predictions either by sampling input parameter distributions or by augmenting deterministic outputs with uncertainty representations, including distribution-free and distributional methods. However, sampling-based methods are often computationally prohibitive for real-time applications, and many existing uncertainty representations either ignore input dependence or rely on restrictive Gaussian assumptions that fail to capture asymmetry and heavy-tailed behavior. Therefore, we extend the ACCurate and Reliable Uncertainty Estimate (ACCRUE) framework to learn input-dependent, non-Gaussian uncertainty distributions, specifically two-piece Gaussian and asymmetric Laplace forms, using a neural network trained with a loss function that balances predictive accuracy and reliability. Through synthetic and real-world experiments, we show that the proposed approach captures an input-dependent uncertainty structure and improves probabilistic forecasts relative to existing methods, while maintaining flexibility to model skewed and non-Gaussian errors.
Read the full abstract ▾
All authors Rileigh Bandy, Enrico Camporeale, Andong Hu, Thomas Berger, Rebecca Morrison
Machine Learning
IKKA: Inversion Classification via Critical Anomalies for Robust Visual Servoing
Darya Pavlenko
We introduce IKKA (Inversion Classification via Critical Anomalies), a topologically motivated weighting framework for robust visual servoing under distribution shift. Unlike conventional outlier handling, IKKA treats maverick points as structurally informative observations: points where small perturbations can induce qualitatively different control responses or class assignments. The method combines local extremality, boundary transversality, and multi-scale persistence into a single anomaly weight, W(x) = E(x) x T(x) x M(x), which modulates control updates near ambiguous decision regions. We instantiate IKKA in a CPU-only embedded visual-servoing pipeline on Raspberry Pi 4 and evaluate it across 230 reproducible runs under nominal and stress conditions. In stress scenarios involving dim illumination and transient occlusion, IKKA reduces the 95th-percentile lateral error by 24% relative to a hybrid baseline (0.124 to 0.094) while increasing throughput from 20.0 to 24.8 Hz. Non-parametric analysis confirms a large effect size (Cliff's delta = 0.79).
Read the full abstract ▾
Machine Learning
Adversarial Sensor Errors for Safe and Robust Wind Turbine Fleet Control
Julian Quick, Marcus Binder Nilsen, Andreas Bechmann et al. (+2 more)
Plant-level control is an emerging wind energy technology that presents opportunities and challenges. By controlling turbines in a coordinated manner via a central controller, it is possible to achieve greater wind power plant efficiency. However, there is a risk that measurement errors will confound the process, or even that hackers will alter the telemetry signals received by the central controller. This paper presents a framework for developing a safe plant controller by training it with an
adversarial agent designed to confound it. This necessitates training the adversary to confound the controller, creating a sort of circular logic or "Arms Race." This paper examines three broad training approaches for co-training the protagonist and adversary, finding that an Arms Race approach yields the best results. These initial results indicate that the Arms Race
adversarial training reduced worst-case performance degradation from 39% power loss to 7.9% power gain relative to a baseline operational strategy.
Read the full abstract ▾
All authors Julian Quick, Marcus Binder Nilsen, Andreas Bechmann, Tran Nguyen Le, Pierre-Elouan Mikael Rethore
Machine Learning
A Little Rank Goes a Long Way: Random Scaffolds with LoRA Adapters Are All You Need
Hananel Hazan, Yanbo Zhang, Benedikt Hartl et al. (+1 more)
How many of a neural network's parameters actually encode task-specific information? We investigate this question with LottaLoRA, a training paradigm in which every backbone weight is drawn at random and frozen; only low-rank LoRA adapters are trained. Across nine
benchmarks spanning diverse architecture families from single-layer classifiers to 900M parameter
Transformers low-rank adapters over frozen random backbones recover 96-100% of fully trained performance while training only 0.5-40% of the parameters. The task-specific signal therefore occupies a subspace orders of magnitude smaller than the full parameter count suggests. Three mechanistic findings underpin this result:(1) the frozen backbone is actively exploited when static the learned scaling~$\beta$ remains strictly positive across all architectures but when the scaffold is destabilized, the optimizer silences it and the LoRA factors absorb all task information; (2) the frozen backbone is preferable but interchangeable any random initialization works equally well, provided it remains fixed throughout training; and (3) the minimum LoRA rank at which performance saturates estimates the intrinsic dimensionality of the task, reminiscent of the number of components retained in Principal Component Analysis (PCA). The construction is formally analogous to Reservoir Computing unfolded along the depth axis of a feedforward network. Because the backbone is determined by a random seed alone, models can be distributed as adapters plus seed a footprint that grows with task complexity, not model size, so that storage and
memory savings compound as architectures scale.
Read the full abstract ▾
All authors Hananel Hazan, Yanbo Zhang, Benedikt Hartl, Michael Levin
Machine Learning · Cross-listed from physics.chem-ph
Active Learning for Generalizable Detonation Performance Prediction of Energetic Materials
R. Seaton Ullberg, Megan C. Davis, Jeremy N. Schroeder et al. (+5 more)
The discovery of new energetic materials is critical for advancing technologies from defense to private industry. However, experimental approaches remain slow and expensive while computational alternatives require accurate material property inputs that are often costly to obtain, limiting their ability to efficiently predict detonation performance across a vast chemical space. We address this challenge through an active learning strategy that integrates density functional theory calculations, thermochemical modeling, message-passing neural networks, and
Bayesian optimization. The resulting high-throughput workflow iteratively expands the training
dataset by selecting new molecules in a targeted manner that balances the exploration of broad chemical space with the exploitation of promising high-performing candidates. This approach yields the largest publicly available database of potential CHNO explosives drawn from an initial pool of more than 70 billion candidates and a generalizable surrogate model capable of accurately predicting detonation performance (R$^2$ > 0.98). Feature importance analysis on this largest-to-date
dataset reveals that oxygen balance is the dominant driver of detonation performance, complemented by contributions from local electronic structure, density, and the presence of specific functional groups. Cheminformatics analysis highlights how energetic materials with similar performance metrics tend to cluster in distinct chemical spaces offering a clearer direction for future synthesis studies. Together, the surrogate model, database, and resulting chemical insights provide a valuable foundation for high-throughput screening and targeted discovery of new energetic materials spanning diverse and previously unexplored regions of chemical space.
Read the full abstract ▾
All authors R. Seaton Ullberg, Megan C. Davis, Jeremy N. Schroeder, Andrew H. Salij, M. J. Cawkwell, Christopher J. Snyder, Wilton J. M. Kort-Kamp, Ivana Matanovic