Machine Learning
HYDRA: Hyperbolic Dynamic Representation Architecture for Kolmogorov-Arnold Networks
Zhao Su, Yuxin Xia, Haoran Li et al. (+4 more)
Kolmogorov-Arnold Networks (KANs) enhance nonlinear function approximation by replacing scalar weights with learnable univariate functions. However, assigning an independent function to every connection results in substantial parameter redundancy, limiting their scalability and efficiency. To reduce this redundancy, we introduce
HYperbolic Dynamic Representation Architecture (HYDRA), a parameter-efficient
hyperbolic extension of KAN that combines spline-based functional learning with representations in the Poincaré ball. HYDRA maps vector-valued inputs into a bounded
hyperbolic latent space, performs KAN-style updates in tangent space, and employs a low-rank prototype block to share functional transformations across hidden dimensions. The resulting
hyperbolic representations provide a structured radial coordinate for interpretation, while radius control improves training stability by preventing boundary saturation. Extensive experiments across eight
benchmark datasets demonstrate that HYDRA consistently achieves competitive or superior predictive performance while improving parameter efficiency and representation interpretability.
Read the full abstract ▾
All authors Zhao Su, Yuxin Xia, Haoran Li, Jun Shen, Qi Zhu, Qingguo Zhou, Binbin Yong
Machine Learning · Cross-listed from cs.AI
How to Spend Your Oracle Budget: Practical Guidance for Protein Structure Prediction Models
Aleksandra Kalisz, Jack Simons, Krisztina Sinkovics et al. (+4 more)
Foundation models for
protein structure prediction remain unreliable on certain targets. External oracles can flag and correct these failures, but biological oracles are expensive, making oracle budget a critical constraint. Existing guidance methods, such as FK-steering, DPO, and Best K-of-N sampling, differ in how they spend this budget, yet no systematic comparison exists to guide method selection. To bridge this gap, we
benchmark these methods alongside the recently proposed Optimisation Over Outputs (O3), which applies off-the-shelf optimisers within a generative model's latent subspace. We extend the usage of O3 to
protein structure prediction models. Overall, our work provides the first practical reference for oracle budget-aware guidance. Our evaluation on two
protein targets, calmodulin (1CLL) and E. coli aspartate transcarbamoylase (9EEH), reveals that no single method consistently dominates across all budgets and oracles. Specifically, O3 proves most effective at low oracle budgets, while FK-steering and DPO demonstrate improved performance as the budget increases. We distil these findings into actionable recommendations for practitioners operating under real-world oracle-budget constraints.
Read the full abstract ▾
All authors Aleksandra Kalisz, Jack Simons, Krisztina Sinkovics, Noam Ghenassia, Shikha Surana, Henry Moss, Paul Duckworth
Machine Learning · Cross-listed from cs.DC
RoutePack: Replay-Guided Expert Placement and Rollout Trajectory Scheduling for MoE Reinforcement Learning
Yibo Shen, Xudong Han, Xiaowei Zhu et al. (+2 more)
Reinforcement learning (RL) post-training of Mixture-of-Experts (MoE) models couples two load-balancing problems. Rollout-trajectory composition determines
attention and dense feed-forward work across data-parallel (DP) replicas, while
token routing determines expert work across expert-parallel ranks. Optimizing either side independently can worsen the other. Routing replay exposes trajectory lengths and layer-wise expert demand before training, creating an opportunity to coordinate both controls. We present RoutePack, a replay-guided planner that coordinates expert placement and rollout-trajectory scheduling. Its key insight is that placement determines a batchwise expert-rank load bound that trajectory scheduling alone cannot lower, while scheduling controls the hotspots within individual execution micro-steps. RoutePack incrementally adjusts expert placement while preserving expert residency, then uses parallel population annealing to reschedule complete trajectories into
token-capped microbatches. Its objective jointly balances dense-side work and layer-wise expert peaks under a fixed, capacity-feasible micro-step count and the deployed synchronization groups. The planner preserves logical
token routing and keeps placement fixed throughout forward and backward execution to maintain training-state consistency. We evaluate mixed-task RL post-training of two hybrid-
attention MoE models (7.9B and 124B parameters) across six NVIDIA H20-3e configurations spanning 8K-32K contexts. Within the Megatron training region, RoutePack achieves 4.44-11.85% higher throughput than a DP-local first-fit-decreasing baseline and 1.33-5.28% higher throughput than expert rerouting alone. Additional evaluations of the 7.9B model on H200 and H20 retain positive gains. We separately characterize planning and admission costs.
Read the full abstract ▾
All authors Yibo Shen, Xudong Han, Xiaowei Zhu, Gen Li, Zhenxuan Pan
Machine Learning · Cross-listed from cs.CV
Autonomous Telerehabilitation via Skeletal Motion Prediction and Joint-Level Performance Assessment
Lara Pereira, João Ruivo Paulo, Pedro Santos et al. (+1 more)
Autonomous rehabilitation systems must not only recognize human motion but also provide structured feedback to support users without continuous therapist supervision. This paper presents a telerehabilitation pipeline that integrates skeleton-based exercise quality assessment and short-term motion prediction into a two-module system operating on marker-free RGB
video. A self-attentive Bidirectional LSTM performs exercise quality classification using MMD-NCA metric learning, while a graph-based motion prediction module computes per-joint position errors between predicted and observed poses, generating spatially localized deviation signals. Each module is evaluated independently on established
benchmarks: the classifier achieves 96.45% mean-class accuracy on squat sequences from the PROZIS
dataset, and the adopted
STARS predictor achieves a mean MPJPE of 75.8 mm at 560 ms on Human3.6M, outperforming graph and recurrent baselines across all prediction horizons. The framework is designed for eventual deployment in assistive robotics and home-based rehabilitation contexts; end-to-end integration and clinical validation are important directions for future work. By combining motion recognition and prediction in a single system, this work contributes a step toward autonomous, feedback-driven telerehabilitation, for more accessible and scalable rehabilitation solutions.
Read the full abstract ▾
All authors Lara Pereira, João Ruivo Paulo, Pedro Santos, Paulo Peixoto
Machine Learning · Cross-listed from cs.SE
ADEPT: A Unified Framework for Deep Learning Test Adequacy
Yidi Kao, Shawn Burnham, Tommi Rose Fahy et al. (+1 more)
Over the past decade, many test adequacy metrics have been proposed for deep learning that characterize test
dataset adequacy from different perspectives, e.g.,
neuron activation behavior, latent feature coverage, decision-boundary exploration, etc. However, these metrics are typically released as independent research prototypes with substantially different installation and preprocessing requirements, execution workflows, and configuration mechanisms. These complications make them quite difficult to reproduce, compare, and adopt in research work and practical deployment alike. In this paper, we present the engineering details of ADEPT, a framework that integrates representative adequacy techniques, including
neuron-coverage-based metrics, surprise adequacy, input distribution coverage, boundary coverage, and source- and model-level mutation score, under a consistent execution workflow. ADEPT provides a template-based metric interface with well-defined extension points for integrating new adequacy metrics. Furthermore, it provides YAML-based configuration management, preprocessing-cache reuse, and structured result reporting, making it easy to use in any research and development workflows. ADEPT is designed for researchers and practitioners who wish to reproduce and apply adequacy metrics without spending days or weeks implementing missing tooling or configuring disparate research prototypes. A demo
video is available at https://aub.ie/ADEPT_
video.
Read the full abstract ▾
All authors Yidi Kao, Shawn Burnham, Tommi Rose Fahy, Ali Ghanbari
Machine Learning · Cross-listed from cs.AR
FQTree: Fine-grained Quantization and Hardware Generation of Boosted Decision Trees
Zhiqiang Que, Chang Sun, Haiyang Wang et al. (+6 more)
Boosted decision trees (BDTs) are widely used in latency-critical applications, but efficient hardware deployment remains challenging. Existing designs often rely on uniform or manually tuned fixed-point formats, which can introduce unnecessary hardware cost or accuracy loss. This work presents the FQTree
algorithm{https://github.com/ecs-bristol/FQTree} for fine-grained quantization-aware training of BDTs, together with the QXGB framework for automatic hardware generation. FQTree introduces a hardware-oriented leaf-value quantization scheme that uses a global quantization step together with a tree-wise shift, enabling compact non-negative integer leaf representations, controlled clipping/
pruning, and bias folding to reduce datapath cost. This work further applies this quantization during boosting so that later trees adapt to the errors of the already-quantized ensemble, and then lowers the trained model into low-latency hardware implementations through a compiler-based flow. Results on JSC, MNIST, and NID show that our method reduces LUT usage by 26-57% compared with the state-of-the-art FPGA-based BDT designs while matching or improving accuracy.
Read the full abstract ▾
All authors Zhiqiang Que, Chang Sun, Haiyang Wang, Dinesh Pamunuwa, Roshan Weerasekera, Qijia Tang, Bakhtiar Zadeh, Wayne Luk, Maria Spiropulu
Machine Learning · Cross-listed from cs.CL
A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench
Praveen Reddy, Charuta Mandke, Suvrankar Datta et al. (+4 more)
General-purpose large
language models (LLMs) have recently been reported to match or exceed specialized clinical AI tools on medical
benchmarks, but such comparisons draw on a narrow set of systems and on
benchmarks developed largely in high-income settings. We evaluate VITA, a
retrieval-augmented generation (RAG) system purpose-built for contextual knowledge
retrieval in India and other low- and middle-income (LMIC) settings. VITA retrieves from a curated corpus of disease-specific guidelines, India-specific antimicrobial resistance data, national formulary constraints, and resource-limited care protocols; its architecture and corpus are proprietary, but the
benchmark, the physician-written rubrics, and our full response and scoring outputs are public for independent verification. On 4,023 English-
language HealthBench questions (80.5% of the
benchmark), scored with a GPT-4.1 judge, VITA ranked first with 51.9% of possible rubric points, ahead of GPT-5.4 (46.1%), o4-mini (44.3%), Gemini 3.1 Pro (42.6%), and Claude Sonnet 4.6 (37.3%), and scored highest on 45.4% of questions. To test robustness to newer models and judge lineage, a 500-question subset was re-run against current-generation models (GPT-5.5, Claude Opus 4.8, Gemini 3.5 Pro, Grok 4.3) and graded by a neutral open-weight judge (DeepSeek-V4-Pro) sharing no lineage with any system tested. Here the gap narrowed to parity: VITA and GPT-5.5 were statistically indistinguishable on mean per-question score, while VITA led on points-weighted score and won the most questions. VITA's advantages in accuracy and completeness persisted under the neutral judge; its communication scores were lower. These results indicate that a purpose-built clinical RAG system remains competitive with frontier LLMs on an open
benchmark, consistent with corpus specificity as a design variable that improves grounding at some cost to communication polish.
Read the full abstract ▾
All authors Praveen Reddy, Charuta Mandke, Suvrankar Datta, Sarah Khan, Siddharth Reddy Anthireddy, Shitij Arora, Vishal Singh
Machine Learning
Adversarial Resilience of Poisson-Process Submodular Maximization over Matroids, and Full-Bandit Learning
Vaneet Aggarwal
We study nonnegative submodular maximization on $n$ elements subject to a general matroid of rank $k$, when the offline
algorithm is given an arbitrary controlled value oracle. Our main result is an
adversarial resilience
theorem for the Spiteful Greedy Swap Poisson Process (SGS-Poisson): without modifying its Poisson intensity, single-element exchange rule, or spiteful drop step, the
algorithm retains limiting approximation factors $1/e$ for non-monotone objectives and $1-1/e$ for monotone objectives. More precisely, given an error bound $\xi\ge0$, under every controlled oracle $\widehat f$ satisfying $|\widehat f(S)-f(S)|\le \xi$ for every set $S$, our implementation returns a feasible set with expected value at least $(1/e-\varepsilon)OPT-O(k\xi)$ and $(1-1/e-\varepsilon)OPT-O(k\xi)$, respectively, where $OPT$ is the feasible optimum. The implementation uses a deterministically bounded budget of $O(nk^{2}\varepsilon^{-2}\log n\log^{2}(1/\varepsilon))$ oracle calls. As a consequence, an offline-to-online reduction yields full-bandit combinatorial multi-armed bandit (CMAB)
algorithms for general matroid-constrained submodular rewards with exact limiting approximation-regret factors $1/e$ and $1-1/e$ and $\widetilde O(n^{1/5}k^{4/5}T^{4/5})$ regret over $T$ rounds. These online guarantees allow exploration to play sets that become independent after deleting at most one element; exploitation and the
benchmark remain matroid-feasible. For unit-capacity partition matroids we obtain $\widetilde O(n^{1/5}k^{3/5}T^{4/5})$ under the same exploration relaxation. We establish a deterministic query budget by truncating the Poisson process and identify the enlarged action set needed to answer its offline queries.
Read the full abstract ▾
Machine Learning
Attractor Image-Based Deep Learning of Arterial Pulse Waves for Age Classification
Sara Vardanega, Patrick Segers, Philip Aston et al. (+3 more)
Arterial pulse waveform morphology evolves with age, reflecting structural and functional changes in the cardiovascular system. Thus, vascular age is a valuable surrogate marker of cardiovascular health, and premature vascular ageing can indicate increased disease risk. Pulse wave analysis could support risk stratification in otherwise asymptomatic adults. We transformed pulse wave time-series data from photoplethysmography (PPG) and arterial tonometry into images, using the Symmetric Projection Attractor Reconstruction (SPAR) method. These SPAR images were used to train a convolutional neural network to classify healthy subjects into two closely spaced age groups (35-40 and 50-55 years). The model demonstrated consistent classification performance across internal and external test sets, achieving F1 scores above 70% for both PPG and tonometry signals. These results suggest that SPAR-derived pulse wave images contain discriminative morphological features even among healthy adults close in age. This proof-of-concept lays the groundwork for future research into the use of SPAR for early risk detection using smart wearables.
Read the full abstract ▾
All authors Sara Vardanega, Patrick Segers, Philip Aston, Ernst Rietzschel, Jordi Alastruey, Manasi Nandi
Machine Learning · Cross-listed from math.OC
The Advective Fisher-Rao Geometry of Deterministic Measure Transport
Benjamin Gess and Johannes Müller
A novel advective Fisher-Rao metric is introduced for optimization tasks on paths of probability measures governed by the continuity equation. This metric is shown to lead to optimal descent directions. It is then shown that this metric arises naturally from three different perspectives: As the rescaled zero-noise limit of the Fisher-Rao metric on path measures, as the expected value of the second variation of the Freidlin–Wentzell large deviation rate functional, and as the Hessian of the Benamou–Brenier action functional from dynamic optimal transport. We supplement this
geometric construction with computational experiments. Here, we demonstrate empirically that the advective Fisher-Rao metric yields the desired optimal fitting of probability densities, whereas the Gauss–Newton method yields optimal fitting of velocity fields.
Read the full abstract ▾
Machine Learning
Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning
Mirko Konstantin, Stefan Zachow and Anirban Mukhopadhyay
Federated learning (FL) enables collaborative model training across distributed clients while keeping data local. A central challenge is determining which client updates are beneficial for aggregation with respect to each client's target domain. Existing methods typically address this problem in parameter space by comparing model parameters or gradients. However, parameter-space similarity can be a poor proxy for predictive behavior, especially under heterogeneous, non-IID data. Consequently, updates that are misaligned with a client's target domain, including those caused by heterogeneous data or malfunctioning clients, may degrade local model performance. We propose Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a
federated learning framework that performs update selection in function space. LIGHTYEAR uses an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client. By relating model parameters to local predictive responses, the Neural Tangent Kernel (NTK) provides a more expressive criterion for update selection than parameter-space similarity alone. Because function-space information is not available before aggregation in conventional centralized FL, LIGHTYEAR uses a peer-to-peer (P2P)
topology in which clients exchange updates directly and evaluate incoming models on private validation data. Each client selects only updates that are beneficial for its own target domain and aggregates them using a regularized rule that improves stability under heterogeneity. Across five
datasets and nine baseline methods, LIGHTYEAR consistently outperforms centralized FL baselines and existing P2P approaches.
Read the full abstract ▾
Machine Learning
Confidence Calibration of Deep Learning Systems
Coby Penso
In high-stakes applications, reliable confidence estimates are as important as the predictions themselves. Confidence calibration ensures that predicted probabilities reflect the likelihood of correctness, making it essential for safe deployment of deep learning models. However, existing methods typically assume access to clean validation data, which is often unrealistic due to label noise and domain shifts. This thesis develops methods for improving calibration under these conditions. First, we address calibration under label noise. Standard methods can produce misleading confidence estimates when labels are unreliable. We propose a framework that uses an estimated noise model to reconstruct noise-free confidence estimates by modeling the relationship between noisy and clean label distributions. We extend this approach to Conformal Prediction (CP), which provides set-valued predictions with guaranteed coverage. Our noise-aware CP method estimates clean conformity scores despite label noise, enabling reliable uncertainty quantification. Next, we study calibration in unsupervised domain adaptation, where a model trained on a labeled source domain is adapted to an unlabeled target domain. Since labeled target data are unavailable, we estimate target-domain accuracy from source performance and domain discrepancies, enabling calibration without target labels. We also consider
privacy-preserving settings in which user labels and model outputs must remain protected. We propose a locally differentially private conformal prediction framework that provides valid uncertainty quantification while maintaining
privacy guarantees and balancing
privacy, computational feasibility, and prediction reliability. Our results bridge calibration theory and practical deployment in safety-critical applications, contributing to reliable,
privacy-preserving, and noise-resilient neural network predictions.
Read the full abstract ▾
Machine Learning
Roman Joeres, Ilya Senatorov, Anastasia Kolchina et al. (+2 more)
Protein language models (PLMs) have transferred the latest advances from natural
language processing to computational biology. These models, trained on large corpora of
protein sequence data, are widely used to translate amino acid sequences into latent-space
embeddings, ready for use in diverse downstream tasks (DTs). By consensus,
embeddings from the models' last layers are used, while the models' internal behavior remains poorly understood. We analyzed 13 PLMs across 15 DTs and 9
datasets to assess the value of
embeddings from intermediate PLM layers. We trained probe models on
embeddings from each layer, compared their performance, and showed that the last layers of PLMs rarely produced
embeddings that led to the best results on downstream tasks. Furthermore, we identified a connection between how models learn a certain DT and the similarity between that DT and the pre-training objective. For example, for residue-level downstream tasks, we observed a steady increase in performance across almost all PLM layers, which we attributed to their similarity to most PLMs' pre-training objectives. To allow the community to capitalize on our findings, we provide PLMSommelier, a Python package that automatically identifies the best PLM layer for a given DT with ~98% accuracy and creates a truncated model using only the early layers up to the best-performing layer. This will help users save time and
memory during inference and yield better predictive performance.
Read the full abstract ▾
All authors Roman Joeres, Ilya Senatorov, Anastasia Kolchina, Dietrich Klakow, Olga V. Kalinina
Machine Learning · Cross-listed from cs.CV
Look What the Probes Dragged In! Real-World Chest X-ray Shortcuts in MedCLIP
Nikolette Pedersen, Regitze Sydendal, Veronika Cheplygina et al. (+1 more)
Vision-
language models, such as contrastive
language-image pre-training (CLIP)-based approaches, have reached state-of-the-art (SOTA) results in medical artificial intelligence. However, recent work reveals that CLIP-based models remain vulnerable to shortcuts. We investigate how real-world shortcuts manifest across different layers of the medical CLIP-based model, MedCLIP, and its
vision encoder, a frozen ResNet-50. We attach 17 linear classification probes to the intermediate layers of the ResNet-50 and train them on three different
dataset configurations and targets: NIH-CXR14 (pneumothorax) and PadChest (cardiomegaly and pneumothorax). This setup allows us to observe model behaviour during evaluation using subgroup-based calibration and layer-wise confidence curves. We find that the final linear probes achieve a high AUROC but poor calibration in the models. The layer-wise confidence analyses suggest that shortcuts emerge at different depths. Patterns consistent with localised shortcuts, such as drains, appear at later layers, while patterns consistent with diffuse shortcuts, such as scanner-specific noise patterns, emerge earlier, aligning with previous work. Finally, we conduct a manual analysis of the images, which reveals data quality issues in both NIH-CXR14 and PadChest. Our findings underscore that even SOTA models remain vulnerable to shortcuts, and the need for high-quality and well-annotated
datasets to draw solid conclusions. Code can be found on our GitHub: https://github.com/nikodice4/MedCLIP_shortcuts.
Read the full abstract ▾
All authors Nikolette Pedersen, Regitze Sydendal, Veronika Cheplygina, Théo Sourget
Machine Learning
NAE: Normalizing AutoEncoder
Muhammad Abdur Rafae and Niels Landwehr
We consider the setting of Normalizing flows with approximate inverses, an established paradigm spanning both full-dimensional ($d=D$) and bottleneck ($d<D$) settings, and group these models under the term flow autoencoders. We present a theoretical investigation into their training dynamics and prove that the proposed loss used by existing approaches is suboptimal; specifically, both encoder and decoder surrogates must be optimized in
alignment with reconstruction loss. Guided by these insights, we propose Normalizing Autoencoder (NAE), which employs a novel conditional loss that aligns the surrogate loss gradient with that of reconstruction loss, directly improving upon the current standard. Extensive experiments across molecule generation, tabular data, and image
benchmarks demonstrate that NAE achieves state of the art performance. Our work highlights the importance of loss
alignment in flow autoencoders and establishes NAE as a powerful generative framework.
Read the full abstract ▾
Machine Learning
Faithful, Sufficient and Understandable: Rethinking Graph Counterfactual Explanations via Discrete Diffusion Inversion
David Bechtoldt and Sidney Bender
Graph Neural Networks (GNNs) achieve strong predictive performance on graph-structured data across domains such as chemistry, biology, and network analysis, yet they provide no intrinsic explanation of their predictions. This limits their adoption in high-stakes and safety-critical settings. Counterfactual explanations address this by revealing the minimal structural modifications that would change a model's prediction. On graphs, however, such a modification is hard to produce. The search space is discrete and combinatorial, and a valid answer must respect categorical node and edge types together with domain rules such as chemical valency in the case of molecular graphs. Existing explainers give up one of two things. Either edits are not held on the data
manifold, or the search does not span the full edit space. We propose Graph
Diffusion Counterfactual Explanation via Inversion (GDCE-I), which gives up neither. A discrete denoising
diffusion model with a novel discrete inversion scheme enables distribution-aware edits leveraging the whole domain edit space. We further address the incomplete and inconsistent evaluation of graph counterfactuals by deriving a framework of explanation desiderata and applying it to every method under one shared protocol. Across four
benchmarks, GDCE-I outperforms related work by a large margin on the defined framework. For the molecular domain, we further qualitatively show that GDCE-I attains interpretable in-distribution solutions.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Better Slots, Better Worlds: Representation Quality & Robustness in Object-Centric World Models
Shukrullo Nazirjonov, Sai Prasanna, Anna Manasyan et al. (+1 more)
Learning world models from offline trajectories enables
agents to accomplish different tasks through planning. Object-centric (OC) representations, which decompose a scene into a set of slots that bind to its objects, have been proposed as an inductive bias for world models that are more sample-efficient and generalize better. Yet prior object-centric world models (OCWMs) take the slot encoder as given and evaluate only in-distribution, leaving open whether the object-centric bias actually delivers for planning and what within the OCWM drives it. We conduct a controlled study of OCWMs for visual model-predictive control along two axes: object-centric representation quality and generalization under distribution shift relative to scene-centric models. We find that (i) planning success correlates positively with unsupervised slot-quality metrics (FG-ARI, mBO), though the gains saturate at high slot quality; (ii) with well-bound slots, the auxiliary proprioception inputs and masking inductive bias that prior methods relied on become unnecessary; and (iii) under unseen distribution shifts, the OCWM with well-bound slots is more robust overall than the end-to-end trained scene-centric LeWM, while DINO-WM, built on similar frozen pretrained features, remains competitive – pointing to pretrained features as a key contributor to robustness.
Read the full abstract ▾
All authors Shukrullo Nazirjonov, Sai Prasanna, Anna Manasyan, Georg Martius
Machine Learning · Cross-listed from cs.CV
Draw This First
Dazhi Zhong, Rowan Bradbury and Grant Davis
We invert the typical formulation of sketch generation: instead of drawing strokes in order, we predict a 2D field that defines the order in which strokes are drawn. We use a pretrained latent flow-matching
transformer to supply the image prior to predict an intermediate representation, while training the VAE's decoder to predict the order field, stroke mask, and stroke
segmentation. We vectorize the predicted
segmentation into polylines and sort them by the field, producing an ordered vector sketch. Our model can predict an ordered vector sketch from a text description or derender an image into ordered vectors; for either, it follows text instructions specifying the order of drawing.
Read the full abstract ▾
Machine Learning
Towards Truly Unsupervised Evaluation of Feature Selection – Extended Version
Hafiz Saud Arshad, Muhammad Rajabinasab and Arthur Zimek
Feature selection is one of the most important and fundamental tasks in data mining, tackled by a family of methods with an established set of evaluation techniques to measure the quality of a specific method. Most of the methods commonly used for the unsupervised evaluation of feature selection
algorithms suffer from critical design flaws which question their unsupervised nature. In this paper, we provide a critical discussion on the established allegedly unsupervised evaluation techniques, and shed light on the reasons why they are not truly unsupervised but, at best, supervised evaluation under an unsupervised downstream task. We also propose a novel, truly unsupervised evaluation framework to measure the quality of the feature selection
algorithms without any form of information about the labels. The proposed framework utilizes unsupervised Principal Component Analysis, and optimal transport to measure the quality of the feature selection methods in a truly unsupervised manner.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization
TaeHo Yoon and Nicolas Loizou
Acceleration for deterministic root-finding problems has been extensively studied in recent years; specifically, the anchor-based, or Halpern-type methods achieve optimal convergence rates with respect to the operator norm. However, acceleration via these methods does not directly carry over to
stochastic setting due to accumulation of errors, unless one enforces diminishing variance via increasing batch sizes or variance reduction techniques. In this work, we show that another class of acceleration, namely the dual-anchor mechanism, extends to the
stochastic setting without such error accumulation, in contrast to anchor-based
algorithms. Consequently, we cleanly achieve $O(\epsilon^{-3})$ complexity with iteration-independent batch size, without any variance reduction or double-loop recursive regularization, for
stochastic root-finding (resp. fixed-point) problems with cocoercivity (resp. square-nonexpansivity) in expectation. For strongly monotone operators, the same
algorithm attains a sharper $\widetilde{O} (\epsilon^{-2})$ complexity, nearly matching the lower bound in terms of $\epsilon$-dependence.
Read the full abstract ▾
Machine Learning
Clustered Randomized Smoothing for Stochastic Prediction Functions
Eduardo Figueiredo, Frederik Mathiesen, Julian Schumann et al. (+3 more)
Modern
stochastic predictors can model rich, multi-modal outcome distributions. However, this expressive power comes with challenges in ensuring robust predictions $-$ a critical requirement in safety-critical domains. Randomized smoothing is a leading technique for improving robustness, particularly against
adversarial perturbations. Yet, in
stochastic multi-modal regression settings, randomized smoothing often fails due to mode collapse, yielding averaged predictions that do not reflect the underlying distribution. To address this limitation, we propose clustered $\alpha$-smoothing, a framework that (1) partitions noisy samples using an arbitrary clustering
algorithm, (2) applies $\alpha$-smoothing locally within each cluster, and (3) combines the resulting predictions into a mixture distribution. By interpreting the smoothing distribution as a mixture of $\alpha$-smoothers, we derive a lower bound on the probability that the smoothed prediction lies within a union of compact regions corresponding to distinct modes. We empirically evaluate our framework on two
benchmarks, demonstrating substantial improvements over state-of-the-art methods. In
stochastic trajectory prediction on a driving simulator
dataset, our approach achieves, on average, a $27\%$ lower Wasserstein distance to the ground-truth distribution compared to $\alpha$-smoothing. In quadrotor control, where modes correspond to distinct feasible paths to a target, our method reduces the collision rate by $81\%$ relative to the state-of-the-art randomized smoothing.
Read the full abstract ▾
All authors Eduardo Figueiredo, Frederik Mathiesen, Julian Schumann, Jens Kober, Arkady Zgonnikov, Luca Laurenti
Machine Learning · Cross-listed from cs.AI
Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence
Mengru Wang, Junfeng Fang, Shuofei Qiao et al. (+17 more)
AI models are increasingly used in scientific discovery and human decision-making. Yet how AI models work and what risks they pose remain poorly understood. As AI development becomes faster and more automated, research on the mechanisms underlying AI remains largely manual. To bridge this gap, we introduce Mechanist, an agentic system that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI. To ground novel mechanism hypotheses, we construct a scientific knowledge graph of 13,000 studies on AI mechanisms, alongside a multidisciplinary database of 43 million papers spanning 26 fields. For reliable experiment execution, we curate a library of 32 foundational methods for mechanism analysis. Compared with Claude Code and existing AI-scientist systems, Mechanist generates higher-quality mechanism hypotheses and executes experiments more reliably. Across four case studies, Mechanist autonomously discovers new model behaviors and their underlying mechanisms, and translates these discoveries into mechanism-guided interventions and interdisciplinary design. Specifically, Mechanist first uncovers a counterintuitive safety risk in scientific laboratories, showing that unsafe traits can transfer to fine-tuned student models through apparently safe training data and emerge across modalities. Mechanist then develops a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, infer the beliefs of others, and how these mechanisms emerge during pretraining. Building on this theory, Mechanist develops targeted interventions that improve model performance across diverse scenarios. Finally, Mechanist can also advance interdisciplinary discovery through mechanistic design, providing an alternative to the computationally intensive generate-and-rerank paradigm.
Read the full abstract ▾
All authors Mengru Wang, Junfeng Fang, Shuofei Qiao, Zhenqian Xu, Haoming Xu, Haoxiong Wang, Shumin Deng, Linyi Yang, Xin Xu, Yunzhi Yao, Dan Zhang, Fei Shen, Zhixiang Cui, Buqiang Xu, Haozhe Luo, Yunxiang Wei, Ningyu Zhang, Julian McAuley, Tat Seng Chua, Huajun Chen
Machine Learning
Uncertainty-Aware Probabilistic Constrained Clustering from Entangled Pairwise Supervision
Shaojie Zhang and Ke Chen
Pairwise constrained clustering typically relies on hard must-link/cannot-link labels, whereas realistic pairwise supervision may be real-valued and entangle intrinsic ambiguity, expert judgment, and
stochastic corruption. Existing deep constrained clustering (DCC) methods mainly target hard, expert-agnostic constraints, treating soft labels mostly numerically rather than semantically. We formalize this setting as uncertainty-aware probabilistic constrained clustering (UPCC), defining a canonical aleatoric relation through a heterogeneous observation process and analyzing its conditional identifiability. We introduce ProbPair, an angular pairwise objective for probabilistic relations, and build ECI-PP, an
estimator–corrector–integrator framework that refines imperfect supervision via belief estimation, correction, and reliability-aware integration. Across challenging probabilistic supervision settings, experiments on diverse
benchmarks show that ECI-PP outperforms state-of-the-art DCC methods and remains robust with a shared default configuration.
Read the full abstract ▾
Machine Learning
SoftWater: Class-Aware Rate Allocation for Softmax Quantization
Joao V. Cavalcanti and Ashia C. Wilson
Post-training quantization pipelines routinely leave the softmax output layer in high precision. Yet in small LLMs with modern vocabularies, the head holds 15–30% of all parameters, so a nominal “2-bit” model with an fp16 head can store several times as many bits per weight. We pose softmax-layer quantization as a rate-distortion problem under the KL divergence between the original and quantized output distributions. A second-order analysis reveals a class-aware geometry: quantization error is weighted jointly by feature covariance and class-specific softmax
curvature. A separability approximation replaces the $Kn\times Kn$ Cholesky with one $n\times n$ factorization rescaled per class, making the
lattice encodable by successive interference cancellation, with both statistics from a single forward pass. The resulting method, SoftWater, gives fine grids to frequent, low-variance classes and coarse grids to rare ones, a large gap under Zipfian
token distributions. Across five models from 1B to 32B, SoftWater outperforms the released WaterSIC quantizer (near-optimal under linear-layer WMSE but not output KL) at matched head rates on 59 of 60 test points, using none of that pipeline's refinements and cutting head-induced KL by $6.5\times$–$8.3\times$ at 2 bits. On Llama-3.2-1B-Instruct with quantized bodies, a 2-bit head removes 45–60% of stored bytes for a $2.9$–$3.7\%$ perplexity increase. Because the class-side statistic comes from calibration data, matching calibration to the deployment domain gives the lowest KL on that domain throughout. On a tied model, a 4-bit head is near-lossless and a 2-bit head costs under 4% perplexity, making head quantization of such models practical.
Read the full abstract ▾
Machine Learning
Reducing Symmetry Increase in Equivariant Neural Networks
Ning Lin, Jiacheng Cen, Anyi Li et al. (+2 more)
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing
geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's
symmetries. The mathematical essence of this phenomenon is that a symmetric input, after being processed by an equivariant map, experiences an increase in
symmetry. While prior research has documented
symmetry increase in specific cases, a rigorous understanding of its underlying causes and general reduction strategies remains lacking. In this paper, we provide a detailed and in-depth characterization of
symmetry increase together with a principled framework for its reduction: (i) For any given feature space and input
symmetry group, we prove that the increased
symmetry admits an infimum determined by the structure of the feature space; (ii) Building on this foundation, we develop a computable
algorithm to derive this infimum, and propose practical guidelines for feature design to prevent harmful
symmetry increases. (iii) Under standard regularity assumptions, we demonstrate that for most equivariant maps, our guidelines effectively reduce
symmetry increase. To complement our theoretical findings, we provide visualizations and experiments on both synthetic
datasets and the real-world QM9
dataset. The results validate our theoretical predictions.
Read the full abstract ▾
All authors Ning Lin, Jiacheng Cen, Anyi Li, Wenbing Huang, Hao Sun
Machine Learning · Cross-listed from math.OC
Adaptive Bregman Proximal Stochastic Gradient with a Stabilized Barzilai–Borwein Step Size
Chenhan Jin, Shengze Xu, Binghui Xie et al. (+4 more)
Bregman proximal
stochastic gradient (BPSG) methods bring variance-reduced composite optimization to objectives whose geometry is poorly captured by Euclidean smoothness. Their performance, however, remains sensitive to the step size: raw
stochastic curvature estimates can fluctuate sharply, whereas line searches add repeated proximal evaluations. We introduce Ada-BPSG, a line-search-free BPSG method that couples the SAGA gradient table with a stabilized Barzilai–Borwein (BB) candidate. A mediant aggregates incremental secant information so that nearly singular local ratios receive little weight, and an explicit safeguard translates the resulting
curvature estimate into the bounded step-size sequence required for convergence. This design yields a direct analytical chain from relative smoothness and component-wise variance control to convergence in finite-dimensional normed spaces. We prove an $O(n/K)$ ergodic rate for convex objectives, a restarted linear rate under relative quadratic growth, and an $O(1/K)$ bound for a Bregman proximal residual in the nonconvex setting. On logistic regression and sparse nonnegative
matrix factorization, Ada-BPSG combines low objective values with substantially less sensitivity to the initial step size than standard variance-reduced baselines, while avoiding line search.
Read the full abstract ▾
All authors Chenhan Jin, Shengze Xu, Binghui Xie, Kaiwen Zhou, Fan Jia, James Cheng, Tieyong Zeng
Machine Learning
Dual-Model Sentiment Analysis of Consumer Reviews in the Retail Coffee Sector Using Machine Learning and Deep Learning Approaches
Muntasir Hasan Kanchan, Md. Alamgir Hossain, Md. Samiul Islam et al. (+1 more)
Consumer reviews play an important role in shaping brand perception and business strategies, particularly in service-driven industries such as retail coffee. This study presents a comparative sentiment analysis framework for Starbucks customer reviews using classical machine learning and deep learning approaches. The
dataset, collected from ConsumerAffairs, contains more than 700 reviews and was analyzed through preprocessing and exploratory data analysis to identify temporal and geographic patterns. Sentiment labels were generated by binarizing
star ratings, with ratings of 4 and 5 classified as positive and ratings of 1 to 3 as negative. The resulting
dataset was substantially imbalanced toward negative sentiment. Five machine learning classifiers, including Logistic Regression, Support Vector Machine (SVM), Decision Tree, Random Forest, and Naive Bayes, were evaluated alongside five deep learning models: LSTM, RNN, Bidirectional LSTM, GRU, and CNN. Model performance was assessed using accuracy, precision, recall, and F1-score. SVM achieved the highest accuracy among the machine learning models at 91.0 percent, while Bidirectional LSTM showed the strongest performance among the deep learning models and demonstrated good generalization on unseen data. The findings also show that class imbalance negatively affected positive sentiment recall across several models. Overall, this study provides a comparative evaluation of machine learning and deep learning approaches for real-world consumer sentiment analysis and highlights the importance of appropriate model selection and preprocessing for customer experience analytics in the retail coffee sector.
Read the full abstract ▾
All authors Muntasir Hasan Kanchan, Md. Alamgir Hossain, Md. Samiul Islam, Muhammad Masud Tarek
Machine Learning
Remote Sensing and Machine Learning-Based Analysis of Land Use and Vegetation Change in Dhaka District, Bangladesh
Muhammad Masud Tarek, Md. Alamgir Hossain, Md. Samiul Islam et al. (+1 more)
Rapid urbanization in Dhaka District, Bangladesh has triggered substantial alterations in land use and environmental conditions, necessitating systematic monitoring for informed urban planning and ecological sustainability. This study employs remote sensing data and machine learning techniques to analyze spatiotemporal changes in land cover and vegetation dynamics between 2019 and 2024. High-resolution satellite imagery from Sentinel-2 MSI and Landsat 8 was utilized to classify land cover types and compute spectral indices including the Normalized Difference Vegetation Index (NDVI), Normalized Difference Built-up Index (NDBI), and Normalized Difference Water Index (NDWI). A supervised machine learning approach incorporating Decision Tree, K-Nearest Neighbors (KNN), and Random Forest classifiers was applied using labeled geospatial training points within Google Earth Engine. Accuracy assessments were conducted using confusion
matrices and kappa statistics. Results indicate a 59.5% increase in urban built-up areas and a significant decline in vegetation (-8.46%) and water bodies (-7.77%) over the five-year period. Land conversion from vegetated and aquatic areas to urban infrastructure was identified as a dominant trend. Among the models, Random Forest demonstrated the highest classification accuracy. These findings underscore the growing environmental pressures driven by unregulated urban expansion in Dhaka. The study highlights the potential of remote sensing and machine learning tools in providing timely, actionable data to support sustainable urban development, land-use regulation, and ecosystem conservation policies.
Read the full abstract ▾
All authors Muhammad Masud Tarek, Md. Alamgir Hossain, Md. Samiul Islam, Muntasir Hasan Kanchan
Machine Learning · Cross-listed from q-bio.NC
Beyond Local Power: Functional Connectivity Analysis for Subject-Independent Learning Style Recognition
Wiga Maulana Baihaqi, Indriana Hidayah, Sri Kusrohmaniah et al. (+1 more)
Identifying individual learning styles optimizes pedagogical efficacy. While traditional questionnaires are structured, behavioral tracking methods require prolonged interaction log accumulation. To overcome these temporal constraints, this paper proposes an objective Electroencephalography (
EEG) approach evaluating Phase Locking Value (PLV) connectivity against localized features across the Active-Reflective (AR) and Verbal-Visual (VV) Felder-Silverman dimensions.
EEG signals were recorded from 28 participants during Raven's Advanced Progressive
Matrices tasks. Support Vector Machine classification used Leave-One-Subject-Out Cross-Validation (LOSO-CV) alongside a 70:30 intra-subject split. The VV dimension achieved 70.00% subject-level accuracy driven by distinct fronto-occipital polarization. Conversely, the AR dimension yielded lower cross-subject generalizability (55.56%) due to overlapping executive networks and a "Systematic Neural Inversion" phenomenon, where stable individual connectivity signatures operated diametrically opposed to global boundaries (up to 20-0 voting margins). Ultimately, these outcomes demonstrate that rigid "one-size-fits-all" classifiers are bounded by biological diversity, emphasizing the need for future adaptive feature transformation techniques to bridge the cross-subject generalization gap.
Read the full abstract ▾
All authors Wiga Maulana Baihaqi, Indriana Hidayah, Sri Kusrohmaniah, Noor Akhmad Setiawan
Machine Learning · Cross-listed from cs.CV
A Remote Approach to Cashew Orchard Detection: Leveraging Active Learning with Satellite Imagery in Guinea-Bissau
Miguel Pereira, Sofia C. Pereira, Maria J.P. Vasconcelos et al. (+3 more)
Cashew production is a widespread economic activity in Guinea-Bissau, as well as other countries in West Africa. However, unregulated cashew production can be directly associated with increasing regionwide deforestation rates, biodiversity losses, and a fragile economic structure. There is no nationwide database for listing or georeferencing cashew orchards, so there is a clear need to remotely map their locations. In recent years, multiple methods for detecting orchards have been developed, though they have only been applied on a regional level. This work expands regional analyses to a nationwide scale. It develops a scalable and cost-effective remote approach, based on Sentinel-2 satellite imagery, using Machine Learning techniques to detect cashew orchards automatically. Margin-based Active Learning techniques were employed to develop an optimal training set in terms of the number of points and their informativeness, leading to a cashew map with 94.0% balanced accuracy obtained entirely off-site. We created two
datasets and a 2021 cashew map with 10m spatial resolution that are openly accessible through GitHub. The results demonstrate the possibility of a broader cashew orchard mapping, creating a new stepping stone for this environmental application.
Read the full abstract ▾
All authors Miguel Pereira, Sofia C. Pereira, Maria J.P. Vasconcelos, Patrícia Guedes, Luke L. Powell, João P. Pedroso