Machine Learning · Cross-listed from cs.AI
Mixture-of-Models: Distributed Inference over Heterogeneous LLMs as Model-Level Mixture-of-Experts
Tims Pecerskis and Aivars Smirnovs
Switch
Transformers scale MoE model capacity by routing
tokens in
memory for one jointly trained model, requiring high-bandwidth interconnects, large
memory, centralised ownership, and lack scalable composability. We propose Mixture-of-Models (MoM) architecture as Switch
Transformer scaling to semantic level which composes inference across independently trained distributed LLMs, each run on its own GPU. The models reach joint consensus using N-Way Self-Evaluating Deliberation protocol: a combination of social-choice methods such as Delphi method and Quadratic Voting as well as Inference-Time scaling paradigm. We start with an empirical model that characterizes consensus as an inference-time trade-off between signal extraction and contextual noise accumulation, establishing a mathematical basis for convergence driven inference stopping strategies. We show experimental results where ensemble of smaller (<21B) models matches closely 120B MoE (GPT-OSS-120B) on AIME 2025 while larger ensemble of open-weight models reaches 90.0%, close to proprietary models at time of writing.
Read the full abstract ▾
Machine Learning
The Art of Being Difficult: Combining Human and AI Strengths to Find Adversarial Instances for Heuristics
Henri Nikoleit, Ankit Anand, Anurag Murty Naredla et al. (+1 more)
We demonstrate the power of human-LLM collaboration in tackling open problems in theoretical computer science. Focusing on combinatorial optimization, we refine outputs from the FunSearch
algorithm [Romera-Paredes et al., Nature 2023] to derive state-of-the-art lower bounds for standard heuristics. Specifically, we target the generation of
adversarial instances where these heuristics perform poorly. By iterating on FunSearch's outputs, we identify improved constructions for hierarchical $k$-median clustering, bin packing, the knapsack problem, and a generalization of Lovász's gasoline problem - some of these have not seen much improvement for over a decade, despite intermittent
attention. These results illustrate how expert oversight can effectively extrapolate algorithmic insights from LLM-based evolutionary methods to break long-standing barriers. Our findings demonstrate that while LLMs provide critical initial patterns, human expertise is essential for transforming these patterns into mathematically rigorous and insightful constructions. This work highlights that LLMs are a strong collaborative tool in mathematics and computer science research.
Read the full abstract ▾
All authors Henri Nikoleit, Ankit Anand, Anurag Murty Naredla, Heiko Röglin
Machine Learning
Interpolation of GEDI Biomass Estimates with Calibrated Uncertainty Quantification
Robin Young and Srinivasan Keshav
Wall-to-wall biomass density estimation from NASA's GEDI mission requires interpolating sparse
LIDAR observations across heterogeneous landscapes. While Random Forest and XGBoost are widely used, they predict each location independently from satellite features, without adapting to the varying difficulty of heterogeneous landscapes. We introduce Attentive Neural Processes (ANPs), a probabilistic meta-learning architecture that explicitly conditions predictions on local observation sets as well as geospatial foundation model
embeddings. ANPs learn a flexible spatial covariance approximator that is more uncertain in complex landscapes and less in homogeneous areas. We validate this approach across five distinct biomes ranging from tropical Amazonian forests to boreal, temperate, and alpine ecosystems, demonstrating that ANPs achieve competitive accuracy while maintaining uncertainty calibration. We demonstrate a scalable, empirically calibrated alternative to ensemble architectures for continental-scale biomass mapping.
Read the full abstract ▾
Machine Learning
Uncertainty propagation through trained multi-layer perceptrons: Exact analytical results
Andrew Thompson and Miles McCrory
We give analytical results for propagation of uncertainty through trained multi-layer perceptrons (MLPs) with a single hidden layer and ReLU activation functions. More precisely, we give expressions for the mean and variance of the output when the input is multivariate Gaussian. In contrast to previous results, we obtain exact expressions without resort to a series expansion.
Read the full abstract ▾
Machine Learning
Sample-wise Constrained Learning via a Sequential Penalty Approach with Applications in Image Processing
Francesca Lanzillotta, Chiara Albisani, Davide Pucci et al. (+3 more)
In many learning tasks, certain requirements on the processing of individual data samples can arguably be formalized, in an intelligible way, as strict constraints of the underlying optimization problem, rather than by means of arbitrary penalties that often require heavy trial-and-error tuning. In this paper we show that, in these scenarios, learning can be carried out exploiting a sequential penalty method that allows to explicitly deal with constraints. For the proposed
algorithm we show that, under classical assumptions and in the considered learning scenario, penalty subproblems can provably be solved in finite time with suitable guarantees of approximate stationarity in expectation; we also show that, when the latter condition is guaranteed for the subproblems, the overall sequential scheme possesses almost sure convergence properties to KKT-stationary points of (general) constrained problems. The results of experiments on sandbox and real-world image processing tasks show that the method is indeed viable to be used in practical deep learning scenarios.
Read the full abstract ▾
All authors Francesca Lanzillotta, Chiara Albisani, Davide Pucci, Daniele Baracchi, Alessandro Piva, Matteo Lapucci
Machine Learning · Cross-listed from cs.CR
Building a Robust Risk-Based Access Control System to Combat Ransomware's Capability to Encrypt
Kenan Begovic, Abdulaziz Al-Ali and Qutaibah Malluhi
Ransomware core capability, unauthorized encryption, demands controls that identify and block malicious cryptographic activity without disrupting legitimate use. We present a probabilistic, risk-based access control architecture that couples machine learning inference with mandatory access control to regulate encryption on Linux in real time. The system builds a specialized
dataset from the native ftrace framework using the function_graph tracer, yielding high-resolution kernel-function execution traces augmented with resource and I/O counters. These traces support both a supervised classifier and interpretable rules that drive an SELinux policy via lightweight booleans, enabling context-sensitive permit/deny decisions at the moment encryption begins. Compared to approaches centered on sandboxing, hypervisor introspection, or coarse system-call telemetry, the function-level tracing we adopt provides finer behavioral granularity than syscall-only telemetry while avoiding the virtualization/VMI overhead of sandbox-based approaches. Our current user-space prototype has a non-trivial footprint under burst I/O; we quantify it and recognize that a production kernel-space solution should aim to address this. We detail
dataset construction, model training and rule extraction, and the run-time integration that gates file writes for suspect encryption while preserving benign cryptographic workflows. During evaluation, the two-layer composition retains model-level detection quality while delivering rule-like responsiveness; we also quantify operational footprint and outline engineering steps to reduce CPU and
memory overhead for enterprise deployment. The result is a practical path from behavioral tracing and learning to enforceable, explainable, and risk-proportionate encryption control on production Linux systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Persuasion Tokens for Editing Factual Knowledge in LLMs
Paul Youssef, Christin Seifert and Jörg Schlötterer
In-context knowledge editing (IKE) is a promising technique for updating Large
Language Models (LLMs) with new information. However, IKE relies on lengthy, fact-specific demonstrations which are costly to create and consume significant context window space. In this paper, we introduce persuasion
tokens (P-
Tokens) – special
tokens trained to replicate the effect of IKE demonstrations, enabling efficient knowledge editing without requiring fact-specific demonstrations. We evaluate P-
Tokens across two editing
datasets and three LLMs, demonstrating performance comparable to, and often exceeding, IKE. We further find that editing performance is robust to distractors with small negative effects to neighboring facts, and that increasing the number of P-
Tokens improves performance. Our work addresses key limitations of IKE and provides a more practical and scalable alternative for editing LLMs.
Read the full abstract ▾
Machine Learning · Cross-listed from math.ST
Kernel smoothing on manifolds
Eunseong Bae and Wolfgang Polonik
Under the assumption that data lie on a compact (unknown)
manifold without boundary, we derive finite sample bounds for kernel smoothing and its (first and second) derivatives, and we establish asymptotic normality through Berry-Esseen type bounds. Special cases include kernel density estimation, kernel regression and the heat kernel signature. Connections to the graph
Laplacian are also discussed.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
AutoRegressive Generation with B-rep Holistic Token Sequence Representation
Jiahao Li, Yunpeng Bai, Yongkang Dai et al. (+3 more)
Previous representation and generation approaches for the B-rep relied on graph-based representations that disentangle
geometric and topological features through decoupled computational pipelines, thereby precluding the application of sequence-based generative frameworks, such as
transformer architectures that have demonstrated remarkable performance. In this paper, we propose BrepARG, the first attempt to encode B-rep's geometry and
topology into a holistic
token sequence representation, enabling sequence-based B-rep generation with an autoregressive architecture. Specifically, BrepARG encodes B-rep into 3 types of
tokens: geometry and position
tokens representing
geometric features, and face index
tokens representing
topology. Then the holistic
token sequence is constructed hierarchically, starting with constructing the geometry blocks (i.e., faces and edges) using the above
tokens, followed by geometry block sequencing. Finally, we assemble the holistic sequence representation for the entire B-rep. We also construct a
transformer-based autoregressive model that learns the distribution over holistic
token sequences via next-
token prediction, using a multi-layer decoder-only architecture with causal masking. Experiments demonstrate that BrepARG achieves state-of-the-art (SOTA) performance. BrepARG validates the feasibility of representing B-rep as holistic
token sequences, opening new directions for B-rep generation.
Read the full abstract ▾
All authors Jiahao Li, Yunpeng Bai, Yongkang Dai, Hao Guo, Hongping Gan, Yilei Shi
Machine Learning · Cross-listed from eess.SY
ReLU Networks for Model Predictive Control: Network Complexity and Performance Guarantees
Xingchen Li and Keyou You
Recent years have witnessed a resurgence in using ReLU neural networks (NNs) to represent model predictive control (MPC) policies. However, determining the sufficient network complexity to ensure closed-loop guarantees remains a fundamental open problem. This involves a critical precision-complexity trade-off: undersized networks may fail to capture the MPC policy, while oversized ones may outweigh the benefits of ReLU network approximation. In this work, we project the MPC actions onto a tightened constraint set to satisfy the hard constraints, and establish a state-dependent Lipschitz continuity property for the optimal MPC cost function, which enables sharp convergence analysis of the closed-loop system. For the first time, we derive explicit bounds on ReLU network width and depth for approximating MPC policies with closed-loop guarantees. To reduce network complexity for the convergence to user-specified target sets, we propose a non-uniform error framework with a state-aware scaling function to adaptively adjust both the input and output of the ReLU network. Our contributions provide a foundational step toward certifiable ReLU NN-based MPC.
Read the full abstract ▾
Machine Learning
Dynamic Expert-Guided Model Averaging for Causal Discovery
Adrick Tench and Thomas Demeester
Would-be practitioners of causal discovery face a dizzying array of
algorithms without a clear best choice. This abundance of competitive methods makes ensembling a natural strategy for practical applications. At the same time, real-world use cases frequently violate the assumptions on which common causal discovery
algorithms are based, forcing reliance on expert knowledge. Inspired by recent work on dynamically requested expert knowledge and large
language models (LLMs) as experts, we present a flexible model averaging method that integrates selective expert querying to ensemble a diverse set of causal discovery
algorithms. Crucially, we distinguish between edge existence and orientation, enabling the method to leverage the complementary strengths of data-driven discovery and expert input. We further consider the realistic setting of limited access to an imperfect expert, using disagreement among
algorithms to query the expert in cases of greater uncertainty. Experiments demonstrate that our method consistently outperforms strong baselines on both clean and noisy data. Code and data are available at https://anonymous.4open.science/r/expert-cd-ensemble-3282/.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
A Feature Extraction Pipeline for Enhancing Lightweight Neural Networks in sEMG-based Joint Torque Estimation
Kartik Chari, Raid Dokhan, Anas Homsi et al. (+2 more)
Robot-assisted rehabilitation offers an effective approach, wherein exoskeletons adapt to users' needs and provide personalized assistance. However, to deliver such assistance, accurate prediction of the user's joint torques is essential. In this work, we propose a feature extraction pipeline using 8-channel surface electromyography (sEMG) signals to predict elbow and shoulder joint torques. For preliminary evaluation, this pipeline was integrated into two neural network models: the Multilayer Perceptron (MLP) and the Temporal Convolutional Network (TCN). Data were collected from a single subject performing elbow and shoulder movements under three load conditions (0 kg, 1.10 kg, and 1.85 kg) using three motion-capture cameras. Reference torques were estimated from center-of-mass kinematics under the assumption of static
equilibrium. Our offline analyses showed that, with our feature extraction pipeline, MLP model achieved mean RMSE of 0.963 N m, 1.403 N m, and 1.434 N m (over five seeds) for elbow, front-shoulder, and side-shoulder joints, respectively, which were comparable to the TCN performance. These results demonstrate that the proposed feature extraction pipeline enables a simple MLP to achieve performance comparable to that of a network designed explicitly for temporal dependencies. This finding is particularly relevant for applications with limited training data, a common scenario patient care.
Read the full abstract ▾
All authors Kartik Chari, Raid Dokhan, Anas Homsi, Niklas Kueper, Elsa Andrea Kirchner
Machine Learning · Cross-listed from cs.SD
I Guess That's Why They Call it the Blues: Causal Analysis for Audio Classifiers
David A. Kelly and Hana Chockler
It is well-known that audio classifiers often rely on non-musically relevant features and spurious correlations to classify audio. Hence audio classifiers are easy to manipulate or confuse, resulting in wrong classifications. While inducing a misclassification is not hard, until now the set of features that the classifiers rely on was not well understood. In this paper we introduce a new method that uses causal
reasoning to discover features of the frequency space that are sufficient and necessary for a given classification. We describe an implementation of this
algorithm in the tool FreqReX and provide experimental results on a number of standard
benchmark datasets. Our experiments show that causally sufficient and necessary subsets allow us to manipulate the outputs of the models in a variety of ways by changing the input very slightly. Namely, a change to one out of 240,000 frequencies results in a change in classification 58% of the time, and the change can be so small that it is practically inaudible. These results show that causal analysis is useful for understanding the
reasoning process of audio classifiers and can be used to successfully manipulate their outputs.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
Fast, faithful and photorealistic diffusion-based image super-resolution with enhanced Flow Map models
Maxence Noble, Gonzalo Iñaki Quintana, Benjamin Aubin et al. (+1 more)
Diffusion-based image super-resolution (SR) has recently attracted significant
attention by leveraging the expressive power of large pre-trained text-to-image
diffusion models (DMs). A central practical challenge is resolving the trade-off between reconstruction faithfulness and photorealism. To address inference efficiency, many recent works have explored knowledge
distillation strategies specifically tailored to SR, enabling one-step
diffusion-based approaches. However, these teacher-student formulations are inherently constrained by information compression, which can degrade perceptual cues such as lifelike textures and depth of field, even with high overall perceptual quality. In parallel, self-
distillation DMs, known as Flow Map models, have emerged as a promising alternative for image generation tasks, enabling fast inference while preserving the expressivity and training stability of standard DMs. Building on these developments, we propose FlowMapSR, a novel
diffusion-based framework for image super-resolution explicitly designed for efficient inference. Beyond adapting Flow Map models to SR, we introduce two complementary enhancements: (i) positive-negative prompting guidance, based on a generalization of classifier free-guidance paradigm to Flow Map models, and (ii)
adversarial fine-tuning using Low-Rank Adaptation (LoRA). Among the considered Flow Map formulations (Eulerian, Lagrangian, and Shortcut), we find that the Shortcut variant consistently achieves the best performance when combined with these enhancements. Extensive experiments show that FlowMapSR achieves a better balance between reconstruction faithfulness and photorealism than recent state-of-the-art methods for both x4 and x8 upscaling, while maintaining competitive inference time. Notably, a single model is used for both upscaling factors, without any scale-specific conditioning or degradation-guided mechanisms.
Read the full abstract ▾
All authors Maxence Noble, Gonzalo Iñaki Quintana, Benjamin Aubin, Clément Chadebec
Machine Learning
Provably Robust Bayesian Counterfactual Explanations under Model Changes
Jamie Duell and Xiuyi Fan
Counterfactual explanations (CEs) offer interpretable insights into machine learning predictions by answering “what if?" questions. However, in real-world settings where models are frequently updated, existing counterfactual explanations can quickly become invalid or unreliable. In this paper, we introduce Probabilistically Safe CEs (PSCE), a method for generating counterfactual explanations that are $\delta$-safe, to ensure high predictive confidence, and $\epsilon$-robust to ensure low predictive variance. Based on
Bayesian principles, PSCE provides formal probabilistic guarantees for CEs under model changes which are adhered to in what we refer to as the $\langle \delta, \epsilon \rangle$-set. Uncertainty-aware constraints are integrated into our optimization framework and we validate our method empirically across diverse
datasets. We compare our approach against state-of-the-art
Bayesian CE methods, where PSCE produces counterfactual explanations that are not only more plausible and discriminative, but also provably robust under model change.
Read the full abstract ▾
Machine Learning
Dual-Prototype Disentanglement: A Context-Aware Enhancement Framework for Time Series Forecasting
Haonan Yang, Yulin He, Jianchao Tang et al. (+1 more)
Real-world time series are governed by both recurring structures, such as trends and seasonality, and infrequent yet critical variations, such as abrupt shifts and rare events. However, existing methods often lack an explicit mechanism to organize and utilize these heterogeneous patterns according to their distinct
forecasting roles. Consequently, common and rare patterns can become entangled, preventing models from dynamically distinguishing and selectively leveraging them according to context. To address this issue, we propose Dual-Prototype Adaptive Disentanglement (DPAD), a model-agnostic framework that organizes temporal patterns by their
forecasting roles. Specifically, we construct a Dynamic Dual-Prototype bank (DDP), comprising a common pattern bank initialized with structured temporal priors to represent prevalent dynamics, and a rare bank that adaptively memorizes infrequent deviations. Then a Dual-Path Context-aware routing (DPC) mechanism enhances outputs with selectively retrieved context-specific pattern representations from DDP. A Disentanglement-Guided Loss (DGLoss) is further introduced to ensure that each prototype bank specializes in its designated role while maintaining sufficient coverage. Extensive experiments across diverse real-world
benchmarks demonstrate that DPAD consistently improves the
forecasting performance of a range of time-series models.
Read the full abstract ▾
All authors Haonan Yang, Yulin He, Jianchao Tang, Zhuo Li
Machine Learning
E2Former-V2: On-the-Fly Equivariant Attention with Linear Activation Memory
Lin Huang, Chengxiang Huang, Ziang Wang et al. (+7 more)
Equivariant Graph Neural Networks (EGNNs) have become a widely used approach for modeling 3D atomistic systems. However, mainstream architectures face critical scalability bottlenecks due to the explicit construction of
geometric features or dense tensor products on every edge. To overcome this, we introduce E2Former-V2, a scalable architecture that integrates algebraic sparsity with hardware-aware execution. We first propose Equivariant Axis-Aligned Sparsification (EAAS). EAAS builds on Wigner-$6j$ convolution by exploiting an $\mathrm{SO}(3) \rightarrow \mathrm{SO}(2)$ change of basis to transform computationally expensive dense tensor contractions into efficient, sparse parity re-indexing operations. Building on this representation, we introduce On-the-Fly Equivariant
Attention, a fully node-centric mechanism implemented via a custom fused Triton kernel. By eliminating materialized edge tensors and maximizing SRAM utilization, our kernel achieves a \textbf{20$\times$ improvement in TFLOPS} compared to standard implementations. Extensive experiments on the SPICE and OMol25
datasets demonstrate that E2Former-V2 maintains comparable predictive performance while notably accelerating inference. This work demonstrates that large equivariant
transformers can be trained efficiently using widely accessible GPU platforms. The code is avalible at https://github.com/IQuestLab/UBio-MolFM/tree/e2formerv2.
Read the full abstract ▾
All authors Lin Huang, Chengxiang Huang, Ziang Wang, Yiyue Du, Chu Wang, Haocheng Lu, Yunyang Li, Xiaoli Liu, Arthur Jiang, Jia Zhang
Machine Learning · Cross-listed from cs.CV
A Lightweight Medical Image Classification Framework via Self-Supervised Contrastive Learning and Quantum-Enhanced Feature Modeling
Jingsong Xia and Siqi Wang
Intelligent medical image analysis is essential for clinical decision support but is often limited by scarce annotations, constrained computational resources, and suboptimal model generalization. To address these challenges, we propose a lightweight medical image classification framework that integrates self-supervised contrastive learning with
quantum-enhanced feature modeling. MobileNetV2 is employed as a compact backbone and pretrained using a SimCLR-style self-supervised paradigm on unlabeled images. A lightweight parameterized
quantum circuit (PQC) is embedded as a
quantum feature enhancement module, forming a hybrid classical-
quantum architecture, which is subsequently fine-tuned on limited labeled data. Experimental results demonstrate that, with only approximately 2-3 million parameters and low computational cost, the proposed method consistently outperforms classical baselines without self-supervised learning or
quantum enhancement in terms of Accuracy, AUC, and F1-score. Feature visualization further indicates improved discriminability and representation stability. Overall, this work provides a practical and forward-looking solution for high-performance medical artificial intelligence under resource-constrained settings.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Efficient Learning of Stationary Diffusions with Stein-type Discrepancies
Fabian Bleile, Sarah Lumpp and Mathias Drton
Learning a stationary
diffusion amounts to estimating the parameters of a
stochastic differential equation whose stationary distribution matches a target distribution. We build on the recently introduced kernel deviation from stationarity (KDS), which enforces stationarity by evaluating expectations of the
diffusion's generator in a reproducing kernel
Hilbert space. Leveraging the connection between KDS and Stein discrepancies, we introduce the Stein-type KDS (SKDS) as an alternative formulation. We prove that a vanishing SKDS guarantees
alignment of the learned
diffusion's stationary distribution with the target. Furthermore, under broad parametrizations, SKDS is convex with an empirical version that is $\epsilon$-quasiconvex with high probability. Empirically, learning with SKDS attains comparable accuracy to KDS while substantially reducing computational cost and yields improvements over the majority of competitive baselines.
Read the full abstract ▾
Machine Learning
FI-TW: An Open Train-Weather Dataset for Railway Delay Analysis in Finland
Vinicius Pozzobon Borin, Jean Michel de Souza Sant'Ana, Usama Raheel et al. (+1 more)
Train delays result from complex interactions between operational, technical, and environmental factors. While weather impacts railway reliability, particularly in Nordic regions, existing
datasets rarely integrate meteorological information with operational train data. This study presents the first publicly available
dataset combining Finnish railway operations with synchronized meteorological observations from 2018-2024. The
dataset integrates operational metrics from Finland Digitraffic Railway Traffic Service with weather measurements from 209 environmental monitoring stations, using spatial-temporal
alignment via Haversine distance. It encompasses 28 engineered features across operational variables and meteorological measurements, covering approximately 38.5 million observations from Finland's 5,915-kilometer rail network. Preprocessing includes strategic missing data handling through spatial fallback
algorithms, cyclical encoding of temporal features, and robust scaling of weather data to address sensor outliers. Analysis reveals distinct seasonal patterns, with winter months exhibiting delay rates exceeding 25% and geographic clustering of high-delay corridors in central and northern Finland. Furthermore, the work demonstrates applications of the data set in analysing the reliability of railway traffic in Finland. A baseline experiment using XGBoost regression achieved a Mean Absolute Error of 2.73 minutes for predicting station-specific delays, demonstrating the
dataset's utility for machine learning applications. The
dataset enables diverse applications, including train delay prediction, weather impact assessment, and infrastructure vulnerability mapping, providing researchers with a flexible resource for machine learning applications in railway operations research.
Read the full abstract ▾
All authors Vinicius Pozzobon Borin, Jean Michel de Souza Sant'Ana, Usama Raheel, Nurul Huda Mahmood
Machine Learning · Cross-listed from eess.SP
Learning Successive Interference Cancellation for Low-Complexity Soft-Output MIMO Detection
Benedikt Fesl and Fatih Capar
Low-complexity multiple-input multiple-output (
MIMO) detection remains a key challenge in modern wireless systems, particularly for 5G reduced capability (RedCap) and internet-of-things (IoT) devices. In this context, the growing interest in deploying machine learning on edge devices must be balanced against stringent constraints on computational complexity and
memory while supporting high-order modulation. Beyond accurate hard detection, reliable soft information is equally critical, as modern receivers rely on soft-input channel decoding, imposing additional requirements on the detector design. In this work, we propose recurSIC, a lightweight learning-based
MIMO detection framework that is structurally inspired by successive interference cancellation (SIC) and incorporates learned processing stages. It generates reliable soft information via multi-path hypothesis tracking with a tunable complexity parameter while requiring only a single forward pass and a minimal parameter count. Numerical results in realistic wireless scenarios show that recurSIC achieves strong hard- and soft-detection performance at very low complexity, making it well suited for edge-constrained
MIMO receivers.
Read the full abstract ▾
Machine Learning
Predicting Startup Success Using Large Language Models: A Novel In-Context Learning Approach
Abdurahman Maarouf, Alket Bakiaj and Stefan Feuerriegel
Venture capital (VC) investments in early-stage startups that end up being successful can yield high returns. However, predicting early-stage startup success remains challenging due to data scarcity (e.g., many VC firms have information about only a few dozen of early-stage startups and whether they were successful). This limits the effectiveness of traditional machine learning methods that rely on large labeled
datasets for model training. To address this challenge, we propose an in-context learning framework for startup success prediction using large
language models (LLMs) that requires no model training and leverages only a small set of labeled startups as demonstration examples. Specifically, we propose a novel k-nearest-neighbor-based in-context learning framework, called kNN-ICL, which selects the most relevant past startups as examples based on similarity. Using real-world profiles from Crunchbase, we find that the kNN-ICL approach achieves higher prediction accuracy than supervised machine learning baselines and vanilla in-context learning. Further, we study how performance varies with the number of in-context examples and find that a high balanced accuracy can be achieved with as few as 50 examples. Together, we demonstrate that in-context learning can serve as a decision-making tool for VC firms operating in data-scarce environments.
Read the full abstract ▾
Machine Learning
Process-Tensor Tomography of SGD: Measuring Non-Markovian Memory via Back-Flow of Distinguishability
Vasileios Sevetlidis and George Pavlidis
This work proposes neural training as a process tensor: a multi-time map that takes a sequence of controllable instruments (batch choices, augmentations, optimizer micro-steps) and returns an observable of the trained model. Building on this operational lens, we introduce a simple, model-agnostic witness of training
memory based on back-flow of distinguishability. In a controlled two-step protocol, we compare outcome distributions after one intervention versus two; the increase $\Delta_{\mathrm{BF}} = D_2 - D_1>0$ (with $D\in\{\mathrm{TV}, \mathrm{JS}, \mathrm{H}\}$ measured on softmax predictions over a fixed probe set) certifies non-Markovianity. We observe consistent positive back-flow with tight bootstrap confidence intervals, amplification under higher momentum, larger batch overlap, and more micro-steps, and collapse under a causal break (resetting optimizer state), directly attributing the effect to optimizer/data-state
memory. The witness is robust across TV/JS/Hellinger, inexpensive to compute, and requires no architectural changes. We position this as a measurement contribution: a principled diagnostic and empirical evidence that practical SGD deviates from the Markov idealization. An exploratory case study illustrates how the micro-level signal can inform curriculum orderings. "Data order matters" turns into a testable operator with confidence bounds, our framework offers a common stage to compare optimizers, curricula, and schedules through their induced training
memory.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IR
PRISM: Purified Representation and Integrated Semantic Modeling for Generative Sequential Recommendation
Dengzhao Fang, Jingtong Gao, Yu Li et al. (+2 more)
Generative Sequential Recommendation (GSR) has emerged as a promising paradigm, reframing recommendation as an autoregressive sequence generation task over discrete Semantic IDs (SIDs), typically derived via codebook-based quantization. Despite its great potential in unifying
retrieval and ranking, existing GSR frameworks still face two critical limitations: (1) impure and unstable semantic tokenization, where quantization methods struggle with interaction noise and codebook collapse, resulting in SIDs with ambiguous discrimination; and (2) lossy and weakly structured generation, where reliance solely on coarse-grained discrete
tokens inevitably introduces information loss and neglects items' hierarchical logic. To address these issues, we propose a novel generative recommendation framework, PRISM, with Purified Representation and Integrated Semantic Modeling. Specifically, to ensure high-quality tokenization, we design a Purified Semantic Quantizer that constructs a robust codebook via adaptive collaborative denoising and hierarchical semantic anchoring mechanisms. To compensate for information loss during quantization, we further propose an Integrated Semantic Recommender, which incorporates a dynamic semantic integration mechanism to integrate fine-grained semantics and enforces logical validity through a semantic structure
alignment objective. PRISM consistently outperforms state-of-the-art baselines across four real-world
datasets, demonstrating substantial performance gains, particularly in high-sparsity scenarios.
Read the full abstract ▾
All authors Dengzhao Fang, Jingtong Gao, Yu Li, Xiangyu Zhao, Yi Chang
Machine Learning
Understanding and Improving UMAP with Geometric and Topological Priors: The JORC-UMAP Algorithm
Xiaobin Li and Run Zhang
Nonlinear dimensionality reduction techniques, particularly UMAP, are widely used for visualizing high-dimensional data. However, UMAP's local Euclidean distance assumption often fails to capture intrinsic
manifold geometry, leading to topological tearing and structural collapse. We identify UMAP's sensitivity to the k-nearest neighbor graph as a key cause. To address this, we introduce Ollivier-
Ricci curvature as a
geometric prior, reinforcing edges at
geometric bottlenecks and reducing redundant links. Since
curvature estimation is noise-sensitive, we also incorporate a topological prior using Jaccard similarity to ensure neighborhood consistency. The resulting method, JORC-UMAP, better distinguishes true
manifold structure from spurious connections. Experiments on synthetic and real-world
datasets show that JORC-UMAP reduces tearing and collapse more effectively than standard UMAP and other DR methods, as measured by SVM accuracy and triplet preservation scores, while maintaining computational efficiency. This work offers a geometry-aware enhancement to UMAP for more faithful data visualization.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Semi-Supervised Hierarchical Open-Set Classification
Erik Wallin, Fredrik Kahl and Lars Hammarstrand
Hierarchical open-set classification handles previously unseen classes by assigning them to the most appropriate high-level category in a class taxonomy. We extend this paradigm to the semi-supervised setting, enabling the use of large-scale, uncurated
datasets containing a mixture of known and unknown classes to improve the hierarchical open-set performance. To this end, we propose a teacher-student framework based on pseudo-labeling. Two key components are introduced: 1) subtree pseudo-labels, which provide reliable supervision in the presence of unknown data, and 2) age-gating, a mechanism that mitigates overconfidence in pseudo-labels. Experiments show that our framework outperforms self-supervised pretraining followed by supervised adaptation, and even matches the fully supervised counterpart when using only 20 labeled samples per class on the iNaturalist19
benchmark. Our code is available at https://github.com/walline/semihoc.
Read the full abstract ▾
Machine Learning
A Collision-Free Hot-Tier Extension for Engram-Style Conditional Memory: A Controlled Study of Training Dynamics
Tao Lin
We investigate whether high-frequency key collisions are a primary bottleneck in Engram-style conditional
memory. To isolate the effect of collisions, we introduce Engram-Nine, a collision-free hot-tier extension that maps the most frequent n-grams through a Minimal Perfect Hash Function (MPHF) while retaining the original multi-head hashed lookup as a cold tier. Under a strictly iso-parameter setup, the collision-free design does not consistently improve validation loss. Through route-stratified evaluation (decomposing per-
token loss into hot/cold contributions), we uncover a consistent "hot-to-cold advantage flip" during training: hot (high-frequency) positions initially have lower loss, but cold positions eventually surpass them. Crucially, collision-free configurations flip earlier than collision-prone baselines, suggesting that collisions act as implicit regularization. We also identify a gating mismatch: the gate learns to favor hot positions early in training, but this preference persists even after the flip, assigning higher weights to positions with higher loss. Our findings suggest that improving lookup precision alone does not guarantee better training outcomes. The dominant limitation may lie in gating credit assignment rather than index accuracy, and collision-induced noise may provide beneficial regularization that should not be naively eliminated.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Curate-Train-Refine: A Closed-Loop Agentic Framework for Zero Shot Classification
Gaurav Maheshwari and Kevin El Haddad
Large
language models (LLMs) and high-capacity encoders have advanced zero and few-shot classification, but their inference cost and latency limit practical deployment. We propose training lightweight text classifiers using dynamically generated supervision from an LLM. Our method employs an iterative, agentic loop in which the LLM curates training data, analyzes model successes and failures, and synthesizes targeted examples to address observed errors. This closed-loop generation and evaluation process progressively improves data quality and adapts it to the downstream classifier and task. Across four widely used
benchmarks, our approach consistently outperforms standard zero and few-shot baselines. These results indicate that LLMs can serve effectively as data curators, enabling accurate and efficient classification without the operational cost of large-model deployment.
Read the full abstract ▾
Machine Learning
Beyond Superficial Unlearning: Sharpness-Aware Robust Erasure of Hallucinations in Multimodal LLMs
Xianya Fang, Feiyang Ren, Xiang Chen et al. (+4 more)
Multimodal LLMs are powerful but prone to object
hallucinations, which describe non-existent entities and harm reliability. While recent unlearning methods attempt to mitigate this, we identify a critical flaw: structural fragility. We empirically demonstrate that standard erasure achieves only superficial suppression, trapping the model in sharp minima where
hallucinations catastrophically resurge after lightweight relearning. To ensure
geometric stability, we propose SARE, which casts unlearning as a targeted min-max optimization problem and uses a Targeted-SAM mechanism to explicitly flatten the loss landscape around hallucinated concepts. By suppressing
hallucinations under simulated worst-case parameter perturbations, our framework ensures robust removal stable against weight shifts. Extensive experiments demonstrate that SARE significantly outperforms baselines in erasure efficacy while preserving general generation quality. Crucially, it maintains persistent
hallucination suppression against relearning and parameter updates, validating the effectiveness of
geometric stabilization.
Read the full abstract ▾
All authors Xianya Fang, Feiyang Ren, Xiang Chen, Yu Tian, Zhen Bi, Haiyang Yu, Sheng-Jun Huang
Machine Learning
DANCE: Dynamic, Available, Neighbor-gated Condensation for Federated Text-Attributed Graphs
Zekai Chen, Haodong Lu, Xunkai Li et al. (+5 more)
Federated graph learning (FGL) enables collaborative training on graph data across multiple clients. With the rise of large
language models (LLMs), textual attributes in FGL graphs are gaining
attention. Text-attributed graph
federated learning (TAG-FGL) improves FGL by explicitly leveraging LLMs to process and integrate these textual features. However, current TAG-FGL methods face three main challenges: (1) Overhead. LLMs for processing long texts incur high
token and computation costs. To make TAG-FGL practical, we introduce graph condensation (GC) to reduce computation load, but this choice also brings new issues. (2) Suboptimal. To reduce LLM overhead, we introduce GC into TAG-FGL by compressing multi-hop texts/neighborhoods into a condensed core with fixed LLM surrogates. However, this one-shot condensation is often not client-adaptive, leading to suboptimal performance. (3) Interpretability. LLM-based condensation further introduces a black-box bottleneck: summaries lack faithful attribution and clear grounding to specific source spans, making local inspection and
auditing difficult. To address the above issues, we propose DANCE, a new TAG-FGL paradigm with GC. To improve suboptimal performance, DANCE performs round-wise, model-in-the-loop condensation refresh using the latest global model. To enhance interpretability, DANCE preserves provenance by storing locally inspectable evidence packs that trace predictions to selected neighbors and source text spans. Across 8 TAG
datasets, DANCE improves accuracy by 2.33% at an 8% condensation ratio, with 33.42% fewer
tokens than baselines.
Read the full abstract ▾
All authors Zekai Chen, Haodong Lu, Xunkai Li, Henan Sun, Jia Li, Hongchao Qin, Rong-Hua Li, Guoren Wang