Machine Learning · Cross-listed from cs.CR
Quantum Key Distribution Secured Federated Learning for Channel Estimation and Radar Spectrum Sensing in 6G Networks
Ferhat Ozgur Catak, Murat Kuzlu, Jungwon Seo et al. (+1 more)
This paper presents a
federated learning framework secured by
quantum key distribution (QKD) for wireless channel estimation and radar spectrum sensing in the next generation networks (NextG or Beyond 6G). A BB84-style protocol abstraction and pairwise additive masking are utilized to train clients' local models (CNN for channel estimation, U-Net for radar
segmentation) and upload only masked model updates. The server aggregates without observing plain parameters; an eavesdropper without QKD keys cannot recover individual updates. Experiments show that secure FL achieves NMSE of 0.216 for channel estimation and 92.1% accuracy with 0.72 mIoU for radar sensing. When an eavesdropper is present, QBER rises to $\sim$25% and all rounds abort as intended; reconstruction error remains below $10^{-5}$, confirming correct aggregation.
Read the full abstract ▾
All authors Ferhat Ozgur Catak, Murat Kuzlu, Jungwon Seo, Umit Cali
Machine Learning · Cross-listed from cs.AI
The Norm-Separation Delay Law of Grokking: A First-Principles Theory of Delayed Generalization
Truong Xuan Khanh, Truong Quynh Hoa, Luu Duc Trung et al. (+1 more)
Grokking – the sudden generalisation that appears long after a model has perfectly memorised its training data – has been widely observed but lacks a quantitative theory explaining the length of the delay. We show that grokking is a norm-driven representational phase transition in regularised training dynamics, and establish the Norm-Separation Delay Law: $T_{\mathrm{grok}} - T_{\mathrm{mem}} = \Theta(\gamma_{\mathrm{eff}}^{-1} \log(\|\theta_{\mathrm{mem}}\|^2 / \|\theta_{\mathrm{post}}\|^2))$, where $\gamma_{\mathrm{eff}}$ is the optimiser's effective contraction rate ($\gamma_{\mathrm{eff}} = \eta\lambda$ for SGD, $\gamma_{\mathrm{eff}} \ge \eta\lambda$ for AdamW). The upper bound follows from a discrete Lyapunov contraction argument; the matching lower bound from dynamical constraints of regularised first-order optimisation. Across 293 training runs spanning modular addition, modular multiplication, and sparse parity, we confirm three falsifiable predictions: inverse scaling with weight decay ($R^2 = 0.97$), inverse scaling with learning rate ($R^2 = 0.92$), and logarithmic dependence on the norm ratio (Pearson $r = 0.91$). A fourth finding reveals that grokking requires an optimiser capable of decoupling memorisation from contraction: SGD fails entirely at the same hyperparameters where AdamW reliably groks. These results reframe grokking not as a mysterious optimisation artefact but as a predictable consequence of norm separation between competing interpolating representations. We further derive a practical three-input
algorithm that predicts grokking delay at memorisation time with 34.6% mean absolute error (bootstrap 95% CI [30.0%, 39.4%], $N=60$ seeds), enabling principled early stopping.
Read the full abstract ▾
All authors Truong Xuan Khanh, Truong Quynh Hoa, Luu Duc Trung, Phan Thanh Duc
Machine Learning
RBF-Solver: A Multistep Sampler for Diffusion Probabilistic Models via Radial Basis Functions
Soochul Park, Yeon Ju Lee, SeongJin Yoon et al. (+3 more)
Diffusion probabilistic models (DPMs) are widely adopted for their outstanding generative fidelity, yet their sampling is computationally demanding.
Polynomial-based multistep samplers mitigate this cost by accelerating inference; however, despite their theoretical accuracy guarantees, they generate the sampling trajectory according to a predefined scheme, providing no flexibility for further optimization. To address this limitation, we propose RBF-Solver, a multistep
diffusion sampler that interpolates model evaluations with Gaussian radial basis functions (RBFs). By leveraging learnable shape parameters in Gaussian RBFs, RBF-Solver explicitly follows optimal sampling trajectories. At first order, it reduces to the Euler method (DDIM). At second order or higher, as the shape parameters approach infinity, RBF-Solver converges to the Adams method, ensuring its compatibility with existing samplers. Owing to the locality of Gaussian RBFs, RBF-Solver maintains high image fidelity even at fourth order or higher, where previous samplers deteriorate. For unconditional generation, RBF-Solver consistently outperforms
polynomial-based samplers in the high-NFE regime (NFE >= 15). On CIFAR-10 with the Score-SDE model, it achieves an FID of 2.87 with 15 function evaluations and further improves to 2.48 with 40 function evaluations. For conditional ImageNet 256 x 256 generation with the Guided
Diffusion model at a guidance scale 8.0, substantial gains are achieved in the low-NFE range (5-10), yielding a 16.12-33.73% reduction in FID relative to
polynomial-based samplers.
Read the full abstract ▾
All authors Soochul Park, Yeon Ju Lee, SeongJin Yoon, Jiyub Shin, Juhee Lee, Seongwoon Jo
Machine Learning
LUMINA: Laplacian-Unifying Mechanism for Interpretable Neurodevelopmental Analysis via Quad-Stream GCN
Minkyung Cha, Jooyoung Bae, Jaewon Jung et al. (+3 more)
Functional
Magnetic Resonance Imaging(fMRI) has now become a classic way for measuring
brain activity, and recent trend is shifting toward utilizing fMRI
brain data for AI-driven diagnosis. Given that the
brain functions as not a discrete but interconnected whole, Graph-based architectures represented by Graph Convolutional Network(GCN) has emerged as a dominant framework for such task, since they are capable of treating ROIs as dynamically interconnected nodes and extracting relational architecture between them. Ironically, however, it is the very nature of GCN's architecture that acts as an obstacle to its performance. The mathematical foundation of GCN, effective for capturing global regularities, acts as a tradeoff; by smoothing features across the connected nodes repeatedly, traditional GCN tend to blur out the contrastive dynamics that might be crucial in identifying certain neurological disorders. In order to break through this structural bottleneck, we propose LUMINA, a
Laplacian-Unifying Mechanism for Interpretable Neurodevelopmental Analysis. Our model is a Quad-Stream GCN that employs a bipolar RELU activation and a dual-spectrum graph
Laplacian filtering mechanism, thereby capturing heterogeneous dynamics that were often blurred out in conventional GCN. By doing so, we can preserve the diverse range and characteristics of neural connections in each fMRI data. Through 5-fold cross validation on the ADHD200(N=144) and ABIDE(N=579)
dataset, LUMINA demonstrates stable diagnostic performance in two of the most critical neurodevelopmental disorder in childhood, ADHD and ASD, outperforming existing models with an accuracy of 84.66% and 88.41% each.
Read the full abstract ▾
All authors Minkyung Cha, Jooyoung Bae, Jaewon Jung, Ping Shu Ho, Ka Chun Cheung, Namjoon Kim
Machine Learning · Cross-listed from cs.NI
Tureis: Transformer-based Unified Resilience for IoT Devices in Smart Homes
Alireza Borhani, Vafa Andalibi and Bahar Asgari
Smart-home IoT systems rely on heterogeneous sensor networks whose correctness shapes application behavior and the physical environment. However, these low-cost, resource-constrained sensors are highly prone to failure under real-world stressors. Prior methods often assume single-failure, single-resident settings, offer only failure detection rather than sensor-level localization, cover limited fault types and sensor modalities, require labels and human intervention, or impose overheads hindering edge deployment. To overcome these limitations, we propose Tureis, a self-supervised, context-aware method for failure detection and faulty-sensor localization in smart homes, designed for multi-failure, multi-resident edge settings. Tureis encodes heterogeneous binary and numeric sensor streams into compact bit-level features. It then trains a lightweight BERT-style
Transformer with sensor-wise masked reconstruction over short-horizon windows, capturing spatial and short-term temporal correlations without mixing unrelated events. This self-supervised objective removes the need for labels or curated semantics. Then, at run-time, Tureis converts reconstruction residuals into sensor-level failure evidence and uses an iterative isolate-and-continue loop that masks flagged sensors, allowing other failures to surface and enabling resilient, fine-grained localization. Across five
datasets with up to nine residents, Tureis improves single-failure localization F1 by +7.6%, +21.0%, and +25.0% over three strong baselines. In multi-failure scenarios with up to five faulty sensors, it further boosts localization F1 by +17.6% and +35.4% over two baselines, while the third does not extend to this setting. These gains come with minute-scale localization and an edge-friendly footprint, as a sub-megabyte model that processes each minute of data in a few milliseconds with ~0.5 GB peak
memory on a Raspberry Pi 5.
Read the full abstract ▾
Machine Learning
Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces
Karan Gupta, Pranav Vajreshwari, Yash Pandya et al. (+3 more)
Agentic systems operating over large tool ecosystems must plan and execute long-horizon workflows under weak or non-verifiable supervision. While frontier models mitigate these challenges through scale and large context budgets, small
language models (SLMs) remain brittle: eager tool loading saturates context, execution errors compound over time, and sparse rewards limit learning. We introduce ATLAS, a reinforcement finetuning framework that enables SLMs to operate effectively in large-scale toolspace environments by learning how to acquire context and how to execute actions. Our approach makes two key contributions. First, we treat context control and execution structure as learnable decisions, combining iterative tool loading with programmatic tool orchestration to bound context growth and stabilize long-horizon trajectories. Second, we propose rubric-based reinforcement finetuning, which decomposes task success into structured, task-aligned criteria and enables scalable training using small judge models. Across MCP
benchmarks, these design choices yield large and consistent gains over generic RL baselines, allowing a 4B SLM to approach frontier-
agent performance under far tighter parameter and context budgets.
Read the full abstract ▾
All authors Karan Gupta, Pranav Vajreshwari, Yash Pandya, Raghav Magazine, Akshay Nambi, Ahmed Awadallah
Machine Learning · Cross-listed from astro-ph.SR
Uncertainty-Aware Solar Flare Regression
Jinsu Hong, Chetraj Pandey and Berkay Aydin
Current
solar flare predictions often lack precise quantification of their reliability, resulting in frequent false alarms, particularly when dealing with
datasets skewed towards extreme events. To improve the trustworthiness of space weather
forecasting, it is crucial to establish confidence intervals for model predictions. Conformal prediction, a machine learning framework, presents a promising avenue for this purpose by constructing prediction intervals that ensure valid coverage in finite samples without making assumptions about the underlying data distribution. In this study, we explore the application of conformal prediction to regression tasks in space weather
forecasting. Specifically, we implement full-disk
solar flare prediction using images created from
magnetic field maps and adapt four pre-trained deep learning models to incorporate three distinct methods for constructing confidence intervals: conformal prediction,
quantile regression, and conformalized
quantile regression. Our experiments demonstrate that conformalized
quantile regression achieves higher coverage rates and more favorable average interval lengths compared to alternative methods, underscoring its effectiveness in enhancing the reliability of
solar weather
forecasting models.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
On the Generalization Capacities of MLLMs for Spatial Intelligence
Gongjie Zhang, Wenhao Li, Quanhao Qian et al. (+4 more)
Multimodal Large
Language Models (MLLMs) that directly process RGB inputs for tasks like 3D localization and
navigation have shown remarkable potential. However, we argue that these RGB-only approaches are fundamentally flawed in their ability to generalize across cameras. By ignoring camera parameters, they entangle an object's physical properties with the camera's perspective, creating an irresolvable ambiguity. We show this leads MLLMs to overfit to the training camera distribution, rather than learning true and generalizable 3D
geometric principles. To address this, we propose Camera-Aware MLLM framework for spatial MLLMs. It learns generalizable spatial
reasoning by: (i) injecting camera intrinsics via a dense
embedding that conditions each visual
token; (ii) introducing a camera-aware data augmentation strategy that synthetically varies camera parameters, forcing the model to disentangle camera properties from scene content; and (iii) distilling
geometric priors from a 3D
vision foundation model. Extensive experiments demonstrate that camera-aware MLLMs substantially outperform their naive counterparts, particularly in cross-camera generalization tests on spatially-grounded tasks, indicating that camera-awareness is not only beneficial but also a prerequisite for robust and generalizable spatial intelligence in MLLMs.
Read the full abstract ▾
All authors Gongjie Zhang, Wenhao Li, Quanhao Qian, Jiuniu Wang, Deli Zhao, Shijian Lu, Ran Xu
Machine Learning · Cross-listed from cond-mat.mtrl-sci
Predicting Atomistic Transitions with Transformers
Henry Tischler, Wenting Li, Qi Tang et al. (+2 more)
Accurate knowledge of the atomistic transition pathways in materials and material surfaces is crucial for many material science problems. However, conventional simulation techniques used to find these transitions are extremely computationally intensive. Even with large-scale, accelerated material simulations, the computational cost constrains the applicable domain in practice. Machine learning models, with the potential to learn the complex emergent behaviors governing atomistic transitions as a fast surrogate model, have great promise to predict transitions with a vastly reduced computational cost. Here, we demonstrate how
transformers can be trained to predict atomistic transitions in nano-clusters. We show how we evaluate physical validity of the predictions and how a multitude of additional, different microstates can be generated by slightly varying the data provided to the model.
Read the full abstract ▾
All authors Henry Tischler, Wenting Li, Qi Tang, Danny Perez, Thomas Vogel
Machine Learning
TML-Bench: Benchmark for Data Science Agents on Tabular ML Tasks
Mykola Pinchuk
Autonomous coding
agents can produce strong tabular baselines quickly on Kaggle-style tasks. Practical value depends on end-to-end correctness and reliability under time limits. This paper introduces TML-Bench, a tabular
benchmark for data science
agents on Kaggle-style tasks. This paper evaluates 10 OSS LLMs on four Kaggle competitions and three time budgets (240s, 600s, and 1200s). Each model is run five times per task and budget. A run is successful if it produces a valid submission and a private-holdout score on hidden labels that are not accessible to the
agent. This paper reports median performance, success rates, and run-to-run variability. MiniMax-M2.1 model achieves the best aggregate performance score on all four competitions under the paper's primary aggregation. Average performance improves with larger time budgets. Scaling is noisy for some individual models at the current run count. Code and materials are available at https://github.com/MykolaPinchuk/TML-bench/tree/master.
Read the full abstract ▾
Machine Learning
Score-Guided Proximal Projection: A Unified Geometric Framework for Rectified Flow Editing
Vansh Bansal and James G Scott
Rectified Flow (RF) models achieve state-of-the-art generation quality, yet controlling them for precise tasks – such as semantic editing or blind image recovery – remains a challenge. Current approaches bifurcate into inversion-based guidance, which suffers from "
geometric locking" by rigidly adhering to the source trajectory, and posterior sampling approximations (e.g., DPS), which are computationally expensive and unstable. In this work, we propose Score-Guided Proximal Projection (SGPP), a unified framework that bridges the gap between deterministic optimization and
stochastic sampling. We reformulate the recovery task as a proximal optimization problem, defining an energy landscape that balances fidelity to the input with realism from the pre-trained score field. We theoretically prove that this objective induces a normal contraction property, geometrically guaranteeing that out-of-distribution inputs are snapped onto the data
manifold, and it effectively reaches the posterior mode constrained to the
manifold. Crucially, we demonstrate that SGPP generalizes state-of-the-art editing methods: RF-inversion is effectively a limiting case of our framework. By relaxing the proximal variance, SGPP enables "soft guidance," offering a continuous, training-free trade-off between strict identity preservation and generative freedom.
Read the full abstract ▾
Machine Learning
MIRACL: A Diverse Meta-Reinforcement Learning for Multi-Objective Multi-Echelon Combinatorial Supply Chain Optimisation
Rifny Rachman, Josh Tingey, Richard Allmendinger et al. (+3 more)
Multi-objective reinforcement learning (MORL) is effective for multi-echelon combinatorial supply chain optimisation, where tasks involve high dimensionality, uncertainty, and competing objectives. However, its deployment in dynamic environments is hindered by the need for task-specific retraining and substantial computational cost. We introduce MIRACL (Meta multI-objective Reinforcement leArning with Composite Learning), a hierarchical Meta-MORL framework that allows for a few-shot generalisation across diverse tasks. MIRACL decomposes each task into structured subproblems for efficient policy adaptation and meta-learns a global policy across tasks using a Pareto-based adaptation strategy to encourage diversity in meta-training and fine-tuning. To our knowledge, this is the first integration of Meta-MORL with such mechanisms in combinatorial optimisation. Although validated in the supply chain domain, MIRACL is theoretically domain-agnostic and applicable to broader dynamic multi-objective decision-making problems. Empirical evaluations show that MIRACL outperforms conventional MORL baselines in simple to moderate tasks, achieving up to 10% higher hypervolume and 5% better expected utility. These results underscore the potential of MIRACL for robust, efficient adaptation in multi-objective problems.
Read the full abstract ▾
All authors Rifny Rachman, Josh Tingey, Richard Allmendinger, Wei Pan, Pradyumn Shukla, Bahrul Ilmi Nasution
Machine Learning · Cross-listed from cs.CV
Full Dynamic Range Sky-Modelling For Image Based Lighting
Ian J. Maquignaz
Accurate environment maps are a key component to modelling real-world outdoor scenes. They enable captivating visual arts, immersive virtual reality and a wide range of scientific and engineering applications. To alleviate the burden of physical-capture, physically-simulation and volumetric rendering, sky-models have been proposed as fast, flexible, and cost-saving alternatives. In recent years, sky-models have been extended through deep learning to be more comprehensive and inclusive of cloud formations, but recent work has demonstrated these models fall short in faithfully recreating accurate and photorealistic natural skies. Particularly at higher resolutions, DNN sky-models struggle to accurately model the 14EV+ class-imbalanced
solar region, resulting in poor visual quality and scenes illuminated with skewed light transmission, shadows and tones. In this work, we propose Icarus, an all-weather sky-model capable of learning the exposure range of Full Dynamic Range (FDR) physically captured outdoor imagery. Our model allows conditional generation of environment maps with intuitive user-positioning of
solar and cloud formations, and extends on current state-of-the-art to enable user-controlled texturing of atmospheric formations. Through our evaluation, we demonstrate Icarus is interchangeable with FDR physically captured outdoor imagery or parametric sky-models, and illuminates scenes with unprecedented accuracy, photorealism, lighting directionality (shadows), and tones in Image Based Lightning (IBL).
Read the full abstract ▾
Machine Learning
Revisiting the (Sub)Optimality of Best-of-N for Inference-Time Alignment
Ved Sriraman and Adam Block
Best-of-N (BoN) sampling is a widely used inference-time
alignment method for
language models, whereby N candidate responses are sampled from a reference model and the one with the highest predicted reward according to a learned reward model is selected. Despite its widespread practical use, recent theoretical work has suggested that it is statistically suboptimal and vulnerable to reward hacking, the process by which models exploit weaknesses in the learned reward model to achieve high estimated reward without genuinely improving performance. We revisit this question under assumptions that more closely reflect practice than that of prior work. In particular, in contradistinction to earlier analyses that focused on expected true reward, which may not be meaningful in many practical settings, we investigate how inference-time
alignment affects the win-rate, a pairwise comparison-based metric more closely aligned with how reward models are trained and evaluated in practice. We demonstrate that, under minimal conditions on the quality of the reference model and learned reward model, properly tuned BoN is both computationally and statistically optimal in achieving high win-rate, partially explaining its widespread practical success. Because BoN remains susceptible to reward-hacking in this setting, we propose a simple and practical variant that provably eliminates reward-hacking while maintaining optimal statistical performance. Finally, we show that prior approaches are provably suboptimal when considering win-rate, highlighting the importance of choosing appropriate objectives when analyzing inference-time
alignment methods.
Read the full abstract ▾
Machine Learning
Unsupervised domain adaptation for radioisotope identification in gamma spectroscopy
Peter Lalor, Ayush Panigrahy and Alex Hagen
Training machine learning models for radioisotope identification using gamma spectroscopy remains an elusive challenge for many practical applications, largely stemming from the difficulty of acquiring and labeling large, diverse experimental
datasets. Simulations can mitigate this challenge, but the accuracy of models trained on simulated data can deteriorate substantially when deployed to an out-of-distribution operational environment. In this study, we demonstrate that unsupervised domain adaptation (UDA) can improve the ability of a model trained on synthetic data to generalize to a new testing domain, provided unlabeled data from the target domain is available. Conventional supervised techniques are unable to utilize this data because the absence of isotope labels precludes defining a supervised classification loss. We compare a range of different UDA techniques, finding that feature
alignment strategies, particularly via maximum mean discrepancy (MMD) minimization or domain-
adversarial training, yield the most consistent improvement to testing scores. For instance, using a custom
transformer-based neural network, we achieve a testing accuracy of $0.904 \pm 0.022$ on an experimental LaBr$_3$ test set after performing unsupervised feature
alignment via MMD minimization, compared to $0.754 \pm 0.014$ before
alignment. Overall, our results highlight the potential of using UDA to adapt a radioisotope classifier trained on synthetic data for real-world deployment.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.ao-ph
The Rise of AI in Weather and Climate Information and its Impact on Global Inequality
Amirpasha Mozaffari, Amanda Duarte, Lina Teckentrup et al. (+8 more)
AI development's current trajectory risks automating and amplifying the North-South divide in the global
climate information system. Frontier models are built almost exclusively in the Global North, and this inequality continues through inputs, processes, and outputs, from biased training data to unrepresentative validation, disproportionately affecting vulnerable regions. Addressing these disparities requires a
Climate Digital Public Infrastructure, evaluation metrics centring well-being, and knowledge co-production to foster resilience rather than inequity.
Read the full abstract ▾
All authors Amirpasha Mozaffari, Amanda Duarte, Lina Teckentrup, Stefano Materia, Gina E. C. Charnley, Lluis Palma, Eulalia Baulenas Serra, Dragana Bojovic, Paula Checchia, Aude Carreric, Francisco Doblas-Reyes
Machine Learning · Cross-listed from stat.ME
Random Dot Product Graphs as Dynamical Systems: Limitations and Opportunities
Giulio Valentino Dalla Riva
Can we learn the differential equations governing the
evolution of a temporal network? We investigate this within Random Dot Product Graphs (RDPGs), where each network snapshot is generated from latent positions evolving under unknown dynamics. We identify three fundamental obstructions: gauge freedom from rotational ambiguity in latent positions, realizability constraints from the
manifold structure of the probability
matrix, and trajectory recovery artifacts from spectral
embedding. We develop a
geometric framework based on principal fiber bundles that formalizes these obstructions. We characterize invisible dynamics as exactly the skew-symmetric generators, and show the realizable tangent space has dimension $nd - d(d-1)/2$. An holonomy dichotomy emerges:
polynomial dynamics have commuting generators, stationary eigenvectors, and trivial holonomy, making gauge
alignment purely statistical;
Laplacian dynamics satisfy a non-commutativity criterion producing nontrivial holonomy, with
curvature weighted by $1/(\lambda_\iota + \lambda_\gamma)$ linking gauge sensitivity to the spectral gap. In $d=2$ this yields full restricted holonomy $\mathrm{SO}(2)$; for $d \ge 3$ generic full $\mathrm{SO}(d)$ remains conjectural. Cram'er–Rao lower bounds reveal that the same spectral gap controlling
curvature and injectivity simultaneously controls Fisher information, so
geometric and statistical difficulty are inextricable. We prove an identifiability principle: symmetric dynamics cannot absorb skew-symmetric gauge contamination, so dynamics structure can resolve gauge ambiguity. We demonstrate this constructively with anchor-based
alignment and a UDE pipeline recovering vector fields from noisy graph sequences. Yet finite-sample interactions between noise, gauge, and dynamics expressiveness remain beyond the asymptotic theory. We frame this gap as an open challenge.
Read the full abstract ▾
Machine Learning
Warm Starting State-Space Models with Automata Learning
William Fishell, Sam Nicholas Kouteili and Mark Santolucito
We prove that Moore machines can be exactly realized as state-space models (SSMs), establishing a formal correspondence between symbolic automata and these continuous machine learning architectures. These Moore-SSMs preserve both the complete symbolic structure and input-output behavior of the original Moore machine, but operate in Euclidean space. With this correspondence, we compare the training of SSMs with both passive and active automata learning. In recovering automata from the SYNTCOMP
benchmark, we show that SSMs require orders of magnitude more data than symbolic methods and fail to learn state structure. This suggests that symbolic structure provides a strong inductive bias for learning these systems. We leverage this insight to combine the strengths of both automata learning and SSMs in order to learn complex systems efficiently. We learn an adaptive arbitration policy on a suite of arbiters from SYNTCOMP and show that initializing SSMs with symbolically-learned approximations learn both faster and better. We see 2-5 times faster convergence compared to randomly initialized models and better overall model accuracies on test data. Our work lifts automata learning out of purely discrete spaces, enabling principled exploitation of symbolic structure in continuous domains for efficiently learning in complex settings.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DC
Parallelization Strategies for Dense LLM Deployment: Navigating Through Application-Specific Tradeoffs and Bottlenecks
Burak Topcu, Musa Oguzhan Cim, Poovaiah Palangappa et al. (+2 more)
Breakthroughs in the generative AI domain have fueled an explosion of large
language model (LLM)-powered applications, whose workloads fundamentally consist of sequences of inferences through
transformer architectures. Within this rapidly expanding ecosystem, dense LLMs–those that activate all model parameters for each
token generation–form the foundation for advanced expert-based variants. Dense models continue to dominate because of their strong generalization ability, scalability, ease of fine-tuning, and versatility across diverse tasks. In LLM inference systems, performance is mainly characterized by latency, response time, and throughput (i.e.,
tokens generated per unit of time). Latency and throughput are inherently coupled: optimizing for one often comes at the expense of the other. Moreover, batching strategies and parallelism configurations, which are essential when dense model parameters exceed device
memory capacity, can significantly affect both latency and overall system throughput. This paper (i) investigates the workloads of two representative dense LLMs–Llama-3.1-70B and Llama-3.1-405B, focusing in particular on intra-node parallelization schemes, (ii) analyzes how input characteristics, batching, and parallelism strategies influence latency flexibility and the latency-throughput tradeoff, and (iii) identifies key performance bottlenecks that inform design choices for meeting service-level agreements (SLAs) and sustaining inference quality. Our empirical evaluations reveal that Tensor Parallelism (TP) improves the latency objectives while Pipeline Parallelism (PP) is better-suited for throughput-oriented applications. We highlight that their hybrid usage by controlling the TP and PP degrees provides control over the latency-throughput interplay.
Read the full abstract ▾
All authors Burak Topcu, Musa Oguzhan Cim, Poovaiah Palangappa, Meena Arunachalam, Mahmut Taylan Kandemir
Machine Learning
Improved Scaling Laws via Weak-to-Strong Generalization in Random Feature Ridge Regression
Diyuan Wu, Lehan Chen, Theodor Misiakiewicz et al. (+1 more)
It is increasingly common in machine learning to use learned models to label data and then employ such data to train more capable models. The phenomenon of weak-to-strong generalization exemplifies the advantage of this two-stage procedure: a strong student is trained on imperfect labels obtained from a weak teacher, and yet the strong student outperforms the weak teacher. In this paper, we show that the potential improvement is substantial, in the sense that it affects the scaling law followed by the test error. Specifically, we consider students and teachers trained via random feature ridge regression (RFRR). Our main technical contribution is to derive a deterministic equivalent for the excess test error of the student trained on labels obtained via the teacher. Via this deterministic equivalent, we then identify regimes in which the scaling law of the student improves upon that of the teacher, unveiling that the improvement can be achieved both in bias-dominated and variance-dominated settings. Strikingly, the student may attain the minimax optimal rate regardless of the scaling law of the teacher – in fact, when the test error of the teacher does not even decay with the sample size.
Read the full abstract ▾
All authors Diyuan Wu, Lehan Chen, Theodor Misiakiewicz, Marco Mondelli
Machine Learning
Reinforcement Learning for Power-Flow Network Analysis
Alperen Ergur, Julia Lindberg and Vinny Miller
The power flow equations are non-linear multivariate equations that describe the relationship between power injections and bus voltages of electric power networks. Given a network
topology, we are interested in finding network parameters with many
equilibrium points. This corresponds to finding instances of the power flow equations with many real solutions. Current state-of-the art
algorithms in computational
algebra are not capable of answering this question for networks involving more than a small number of variables. To remedy this, we design a probabilistic reward function that gives a good approximation to this root count, and a state-space that mimics the space of power flow equations. We derive the average root count for a Gaussian model, and use this as a baseline for our RL
agents. The
agents discover instances of the power flow equations with many more solutions than the average baseline. This demonstrates the potential of RL for power-flow network design and analysis as well as the potential for RL to contribute meaningfully to problems that involve complex non-linear
algebra or geometry. \footnote{Author order alphabetic, all authors contributed equally.
Read the full abstract ▾
Machine Learning
The Value of Graph-based Encoding in NBA Salary Prediction
Junhao Su, David Grimsman and Christopher Archibald
Market valuations for professional athletes is a difficult problem, given the amount of variability in performance and location from year to year. In the National Basketball Association (NBA), a straightforward way to address this problem is to build a tabular data set and use supervised machine learning to predict a player's salary based on the player's performance in the previous year. For younger players, whose contracts are mostly built on draft position, this approach works well, however it can fail for veterans or those whose salaries are on the high tail of the distribution. In this paper, we show that building a knowledge graph with on and off court data,
embedding that graph in a vector space, and including that vector in the tabular data allows the supervised learning to better understand the landscape of factors that affect salary. We compare several graph
embedding algorithms and show that such a process is vital to NBA salary prediction.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On
Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna et al. (+1 more)
Reinforcement learning with verifiable rewards (RLVR) and Rubrics as Rewards (RaR) have driven strong gains in domains with clear correctness signals and even in subjective domains by synthesizing evaluation criteria from ideal reference answers. But many real-world tasks admit multiple valid outputs and lack the single ideal answer that rubric generation depends on. We identify this reference-free setting as a gap in current post-training methods and propose Implicit Error Counting (IEC) to fill it. Instead of checking what a response gets right against a rubric, IEC enumerates what it gets wrong, applying severity-weighted scores across task-relevant axes and converting them into calibrated per-aspect rewards. We show that naïve explicit enumeration is too noisy for stable optimization, and that two design choices: implicit score emission and group calibration are necessary to make error counting a reliable reward. As a case study, we validate IEC on virtual try-on (VTO), a domain that is simultaneously too constrained for holistic scoring and too permissive for rubric-based evaluation: subtle garment errors are unacceptable, yet many output variations are correct. We introduce Cascaded Error Counting (CEC) as an evaluation metric, which tracks human preferences well (60% top-1 vs. 30% others), and curate Mismatch-DressCode (MDressBench), a
benchmark with maximal attribute mismatch to stress-test reward designs. On MDressBench, IEC outperforms RaR across all metrics (CEC: 5.31 vs. 5.60 on flat references; 5.20 vs. 5.53 on non-flat). On VITON-HD and DressCode, IEC matches or surpasses six baselines on 6 of 8 perceptual metrics. These results suggest that when ideal answers are unavailable, counting errors provide a stronger signal than constructing rubrics.
Read the full abstract ▾
All authors Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna, Karim Bouyarmane
Machine Learning · Cross-listed from cs.CV
Making Reconstruction FID Predictive of Diffusion Generation FID
Tongda Xu, Mingwei He, Shady Abu-Hussein et al. (+6 more)
It is well known that the reconstruction FID (rFID) of a VAE is poorly correlated with the generation FID (gFID) of a latent
diffusion model. We propose interpolated FID (iFID), a simple variant of rFID that exhibits a strong correlation with gFID. Specifically, for each
dataset element, we retrieve its nearest neighbor in latent space, interpolate between their latent representations, decode the interpolated latent, and compute the FID between the decoded samples and the original
dataset. We provide an intuitive explanation for why iFID correlates well with gFID, and why reconstruction metrics can be negatively correlated with gFID, by connecting iFID to recent results on
diffusion generalization and
hallucination. Theoretically, we show that iFID evaluates decoded interpolations aligned with the ridge set around which
diffusion samples concentrate, thereby measuring a quantity closely related to
diffusion sample quality. Empirically, iFID is the first metric shown to strongly correlate with
diffusion gFID across diverse VAEs, achieving Pearson and Spearman correlations of approximately $0.85$. The project page is available at https://tongdaxu.github.io/pages/ifid.html.
Read the full abstract ▾
All authors Tongda Xu, Mingwei He, Shady Abu-Hussein, Jose Miguel Hernandez-Lobato, Chunhang Zheng, Kai Zhao, Chao Zhou, Ya-Qin Zhang, Yan Wang
Machine Learning
SHERLOC: Statistical Hypothesis Evaluation and Recovery for Learning Opponent Characteristics
Soyon Choi, Scott Alfeld and Meiyi Ma
When used in automated decision-making systems, machine learning (ML) models are vulnerable to data-manipulation attacks. Some defense mechanisms (e.g.,
adversarial regularization) directly affect the ML models while others (e.g., anomaly detection) act within the broader system. However, many defenses operate under a predefined threat model that assumes characteristics of the adversary, including their knowledge, capabilities, and objectives, are known and fixed. In practice, these characteristics may be unknown or evolve over time, creating a mismatch between the assumptions of the defender and the true attacker. In this paper, we study a new perspective on
adversarial defense: focusing on the attacker, rather than the attack. We present and demonstrate SHERLOC, a framework for
adversarial learning forensics – namely, a detective that learns about the attacker from observed attack(s). We prove that, without additional knowledge, the attacker is non-identifiable — multiple potential attackers would perform the same observed attack. This reveals an important distinction between reconstructing an attacker's behavior and recovering the attacker that generated it. We demonstrate empirically that SHERLOC is able to reconstruct a hypothesis attacker that is consistent with the observed attacks against various defender models (i.e., generalized least-squares linear regressors, logistic regressors, and
polynomial classifiers) across various security-critical domains. Finally, we show that these inferred attackers need not match the ground truth attacker exactly to have downstream utility. Defenses using the SHERLOC-inferred attackers achieve approximately 90-100% of robustness improvement that would be achieved if the defender had access to the ground-truth attacker.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
RACAS: Controlling Diverse Robots With a Single Agentic System
Dylan R. Ashley, Jan Przepióra, Yimeng Chen et al. (+5 more)
Many robotic platforms expose an API through which external software can command their actuators and read their sensors. However, transitioning from these low-level interfaces to high-level autonomous behaviour requires a complicated pipeline, whose components demand distinct areas of expertise. Existing approaches to bridging this gap either require retraining for every new embodiment or have only been validated across structurally similar platforms. We introduce RACAS (
Robot-Agnostic Control via Agentic Systems), a cooperative agentic architecture in which three LLM/VLM-based modules (Monitors, a Controller, and a
Memory Curator) communicate exclusively through natural
language to provide closed-loop
robot control. RACAS requires only a natural
language description of the
robot, a definition of available actions, and a task specification; no source code, model weights, or reward functions need to be modified to move between platforms. We evaluate RACAS on several tasks using a wheeled ground
robot, a recently published novel multi-jointed robotic limb, and an
underwater vehicle. RACAS consistently solved all assigned tasks across these radically different platforms, demonstrating the potential of agentic AI to substantially reduce the barrier to prototyping robotic solutions.
Read the full abstract ▾
All authors Dylan R. Ashley, Jan Przepióra, Yimeng Chen, Ali Abualsaud, Nurzhan Yesmagambet, Shinkyu Park, Eric Feron, Jürgen Schmidhuber
Machine Learning · Cross-listed from q-bio.NC
Behavior-dLDS: A decomposed linear dynamical systems model for neural activity partially constrained by behavior
Eva Yezerets, En Yang, Misha B. Ahrens et al. (+1 more)
Brain-wide recordings of large-scale networks of
neurons now provide an unprecedented view into how the
brain drives behavior. However,
brain activity contains both information directly related to behavior as well as the potential for many internal computations. Moreover, observable behavior is executed not only by the
brain, but also by the spinal cord and peripheral nervous system. Behavior is a coarse-grained product of neural activity, and we thus take the view that it can be best represented by lower-dimensional latent neural dynamics. Capturing this indirect relationship while disambiguating behavior-generating networks from internal computations running in parallel requires new modeling approaches that can embody the parallel and distributed nature of large-scale neural populations. We thus present behavior-decomposed linear dynamical systems (b-dLDS) to disentangle simultaneously recorded subsystems and identify how the latent neural subsystems relate to behavior. We demonstrate the ability of b-dLDS to decouple behavioral vs. internal computations on controlled, simulated data, showing improvements over a state-of-the-art model that uses behavior to supervise all dynamics based on behavior. We also demonstrate b-dLDS's interpretability benefits on a task-driven RNN
dataset featuring a nonlinear relationship between behavior and activations. We then show that b-dLDS can further scale up to tens of thousands of
neurons by applying our model to a large-scale recording of a zebrafish hindbrain during the complex positional homeostasis behavior, wherein b-dLDS highlights asymmetry in behavior-related dynamic connectivity networks.
Read the full abstract ▾
All authors Eva Yezerets, En Yang, Misha B. Ahrens, Adam S. Charles
Machine Learning · Cross-listed from cs.CV
From Decoupled to Coupled: Robustness Verification for Learning-based Keypoint Detection with Joint Specifications
Xusheng Luo and Changliu Liu
Keypoint detection underpins many
vision tasks, including pose estimation, viewpoint recovery, and 3D reconstruction, yet modern neural models remain vulnerable to small input perturbations. Despite its importance, formal robustness verification for keypoint detectors is largely unexplored due to high-dimensional inputs and continuous coordinate outputs. We propose the first coupled robustness verification framework for heatmap-based keypoint detectors that bounds the joint deviation across all keypoints, capturing their interdependencies and downstream task requirements. Unlike prior decoupled, classification-style approaches that verify each keypoint independently and yield conservative guarantees, our method verifies collective behavior. We formulate verification as a falsification problem using a mixed-integer linear program (MILP) that combines reachable heatmap sets with a polytope encoding joint deviation constraints. Infeasibility certifies robustness, while feasibility provides counterexamples, and we prove the method is sound: if it certifies the model as robust, then the keypoint detection model is guaranteed to be robust. Experiments show that our coupled approach achieves high verified rates and remains effective under strict error thresholds where decoupled methods fail.
Read the full abstract ▾
Machine Learning
On the Value of Tokeniser Pretraining in Physics Foundation Models
Hadi Sotoudeh, Payel Mukhopadhyay, Ruben Ohana et al. (+4 more)
We investigate the impact of tokeniser pretraining on the accuracy and efficiency of physics emulation. Modern high-resolution simulations produce vast volumes of data spanning diverse physical regimes and scales. Training foundation models to learn the dynamics underlying such data enables the modelling of complex multiphysics phenomena, especially in data-limited settings. The emerging class of physics foundation models typically aims to learn two tasks jointly: (i) extracting compact representations of high-resolution spatiotemporal data, and (ii) capturing governing physical dynamics. However, learning both tasks from scratch simultaneously can impede the effectiveness of either process. We show that pretraining the tokeniser with an autoencoding objective prior to training the dynamics model enhances computational efficiency for physics emulation. Notably, the magnitude of this benefit depends on domain
alignment: pretraining on the same physical system as the emulation task yields the largest improvements, while pretraining on other systems provides moderate gains. In-domain pretraining reduces VRMSE by 64% after 10,500 training steps compared to training from scratch. To our knowledge, this is the first systematic investigation of tokeniser pretraining for physics foundation models. We further introduce flexible spatiotemporal compression operations that extend causal convolutions to support runtime-adjustable compression ratios, enabling efficient adaptation to diverse downstream tasks. Our findings provide practical guidance for training efficient physics emulators and highlight the importance of strategic pretraining data selection.
Read the full abstract ▾
All authors Hadi Sotoudeh, Payel Mukhopadhyay, Ruben Ohana, Michael McCabe, Neil D. Lawrence, Shirley Ho, Miles Cranmer
Machine Learning
Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models
Ivan Luiz De Moura Matos, Abdel Djalil Sad Saoud, Ekaterina Iakovleva et al. (+2 more)
The issue of algorithmic biases in deep learning has led to the development of various debiasing techniques, many of which perform complex training procedures or
dataset manipulation. However, an intriguing question arises: is it possible to extract fair and bias-agnostic subnetworks from standard vanilla-trained models without relying on additional data, such as unbiased training set? In this work, we introduce Bias-Invariant Subnetwork Extraction (BISE), a learning strategy that identifies and isolates "bias-free" subnetworks that already exist within conventionally trained models, without retraining or finetuning the original parameters. Our approach demonstrates that such subnetworks can be extracted via
pruning and can operate without modification, effectively relying less on biased features and maintaining robust performance. Our findings contribute towards efficient bias mitigation through structural adaptation of pre-trained neural networks via parameter removal, as opposed to costly strategies that are either data-centric or involve (re)training all model parameters. Extensive experiments on common
benchmarks show the advantages of our approach in terms of the performance and computational efficiency of the resulting debiased model.
Read the full abstract ▾
All authors Ivan Luiz De Moura Matos, Abdel Djalil Sad Saoud, Ekaterina Iakovleva, Vito Paolo Pastore, Enzo Tartaglione