Machine Learning · Cross-listed from cs.CL
Temperature Fragility and the Conditional Benefits of Truncation Sampling
Francesco La Rosa
Large
language models generate text by sampling each
token from a predicted distribution, and a temperature parameter sets how far the draw strays from the most probable
tokens. Truncation samplers such as top-p and min-p discard the least probable
tokens before the draw, so that sampling at high temperature stays coherent. Their reported accuracy gains come from temperatures of 1.5 to 3, while the defaults of deployed systems cluster between 0.6 and 1.0. Whether they change accuracy at those defaults, and for which models, has not been measured. We test thirteen open-weight models on GSM8K and MMLU-Pro at temperatures 0.7, 1.0, and 1.3 in one controlled pipeline, ten of them under eight decoding configurations. Six of the thirteen models lose 17 to 38 accuracy points on MMLU-Pro between 0.7 and 1.3, and the other seven lose at most 10. The lost accuracy comes from generations that run to the
token limit or never state an answer. These results suggest that truncation samplers improve accuracy primarily when higher temperatures substantially degrade model performance. Where accuracy remains stable across temperatures, none of the tested truncation samplers improves on plain temperature sampling.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
HISPO: Hierarchical Importance-Sampling Policy Optimization with Entropy-Derived Segments
Quoc-Vinh Lai-Dang and Hyo-Sang Shin
Reinforcement learning with verifiable rewards (RLVR) has become a central approach for improving mathematical
reasoning in
language models, but long-form completions introduce a difficult credit-assignment problem: different parts of a solution trace may contribute unevenly to final correctness. Existing policyoptimization objectives for RLVR commonly apply importance-sampling correction at either the
token level (GRPO, DAPO) or the sequence level (GSPO), imposing different granularities for assigning credit across a response. We introduce Hierarchical Importance-Sampling Policy Optimization (HISPO), a segment-level policy-optimization method that constructs rollout-time
entropy-derived contiguous segments, assigns soft
entropy-based saliency weights, and applies clipped importance-sampling correction at the segment granularity. This provides an intermediate correction unit between
token-level GRPO/DAPO and sequence-level GSPO. We evaluate HISPO by fine-tuning Qwen3-1.7B-Base on mathematical
reasoning tasks. Across six
benchmarks, HISPO improves Pass@8 over the strongest baseline on all
benchmarks and matches or exceeds the strongest baseline in Acc@8 on five of them. On AIME25, HISPO improves over GRPO by +3.75 Acc@8 and +3.78 Pass@8, and over GSPO by +2.50 Acc@8 and +1.27 Pass@8. These results suggest that segment-level correction is a promising granularity for RLVR in long-form mathematical
reasoning.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.data-an
Online local learning for generative thermodynamic computing
Huilin Wang and Weibing Deng
Generative thermodynamic computers turn thermal noise into structured data through Langevin dynamics. We train these systems with a local update at each integration step. The reverse-path Onsager-Machlup objective yields a coupling gradient that is a symmetric sum of local residual-state correlations. We apply this gradient immediately rather than accumulating it over a full trajectory. In digital simulations using MNIST prototypes, online and trajectory-batch training reach similar validation losses on fixed noising paths. Models trained online release less heat on average in all five independently seeded pairs, with both models' parameters held fixed during sampling. Auxiliary classifier and nearest-prototype measures change modestly, while pairwise diversity decreases. The response to noise depends strongly on where the errors enter: independent zero-mean errors in the formed updates produce little heat change over a finite range of noise amplitudes, whereas residual offset and temporal correlation have much larger effects. Storing trained couplings requires substantially less precision than resolving deterministic updates during training. Together, these results establish a local online training method and show how update timing, noise structure, and precision affect generative thermodynamic computing.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Graph Matching Relaxations and Amortization for Supervised Graph Prediction
Federico Méndez, Paul Krzakala, Gabriel Melo et al. (+3 more)
End-to-end Supervised Graph Prediction (SGP) requires a permutation-invariant loss to compare predicted and target graphs with arbitrary node orderings. Such losses typically involve a costly graph-matching problem. We first study three Optimal Transport relaxations of this problem and show, theoretically and empirically, that the Gromov-Wasserstein (GW) objective is the most suitable for SGP. Then, to avoid solving the resulting inner optimization for every training example, we propose to amortize the graph matching (node
alignment) problem. For each training sample, the loss function leverages a transport plan provided by a parametric matcher based on the differentiable Sinkhorn
algorithm applied on empirical node distributions. The graph prediction module and the matcher are jointly learned. We showcase the efficiency of this approach on toy and real world SGP problems of increasing complexity including a novel Mass-spectra to Scaffold task that we introduce.
Read the full abstract ▾
All authors Federico Méndez, Paul Krzakala, Gabriel Melo, Charlotte Laclau, Rémi Flamary, Florence d'Alché-Buc
Machine Learning
On the role of the tokenizer in ECG transformer models
Jiawei Li, Fabio Bonassi, Johan Sundström et al. (+2 more)
Tokenization determines both the physiological content presented to an ECG
Transformer and the sequence over which
attention operates. We compare eight tokenization strategies across
Transformer, Informer, Reformer, and FEDformer on the nine-label CPSC2018 classification task. The input projection and principal backbone capacity are controlled to isolate the effect of
token construction. Median-beat and HeartLang tokenization achieve mean macro-AUCs of 0.893 and 0.889 across the four backbones, compared with 0.822 and 0.824 for point-wise and patch-wise tokenization. Pooling the two physiology-aware representations yields an 8.2% relative improvement in macro-AUC. They also reduce mean sequence length from 1,250 to 158
tokens and mean peak training
memory from 5.21 to 0.27 GB. The results show that aligning
tokens with ECG morphology can improve both predictive performance and
memory efficiency without increasing backbone capacity. The source code is available on https://github.com/LeeJarvis996/ecg_tokenizer.
Read the full abstract ▾
All authors Jiawei Li, Fabio Bonassi, Johan Sundström, Thomas B. Schön, Antônio H. Ribeiro
Machine Learning
Single-condition neural solvers encode transferable response spaces for parametric differential equations
Wenbo Cao and Weiwei Zhang
Operator learning for parametric partial differential equations (PDEs) typically builds global models over prescribed domains, requiring cross-condition data or costly physics-constrained training. Here we show that the output Jacobian of a neural solution model trained at one condition defines a reusable response space for cross-condition solution variations. We introduce Linearized Subspace Transfer (LST) to exploit this space and recover target solutions by minimizing the target PDE-system residual over response-space coordinates. Because any single response space has finite coverage, Active Transfer Modeling (ATM) uses post-transfer residuals as coverage indicators to selectively acquire response spaces from additional single-condition models. Across six systems, single-condition response spaces supported cross-condition transfer, with enrichment improving accuracy when added spaces expanded representation capacity. Relative to evaluated physics-informed operator baselines, ATM reduced error and offline construction cost, with orders-of-magnitude accuracy gains in representative cases and millisecond-to-second target adaptation. These results establish neural solvers as reusable local parametric models.
Read the full abstract ▾
Machine Learning
CodeTS: Verifiable Text-to-Time Series Generation via Executable Code
Xudong Yuan, Shunyu Liu, Tongya Zheng et al. (+3 more)
Text-to-Time Series Generation (Text-to-TS) provides a promising paradigm for synthesizing time series from natural
language, enabling scenario-specific generation when real observations are scarce or costly to acquire. However, existing methods typically lack an explicit mechanism for deriving generation logic from textual descriptions to guide time series synthesis. In this paper, we propose CodeTS, a verifiable framework that uses code as an intermediate generation interface, reformulating Text-to-TS generation as a Text-to-Code-to-TS process. CodeTS first maps textual temporal descriptions into an explicit code space, where executable code specifies how textual requirements shape target temporal patterns, and then obtains the time series through code execution. To learn this code generation process reliably without real code annotations, CodeTS constructs aligned Text-Code-TS triplets from structured temporal attributes for supervised initialization. More importantly, we further design multi-stage execution-based rewards that verify format validity, code executability, and time series quality, enabling real Text-TS pairs to provide training signals for Reinforcement Learning with Verifiable Rewards (RLVR). Extensive experiments on eight
benchmarks across short, medium, and long generation lengths demonstrate that CodeTS provides a strong zero-shot solution for Text-to-TS generation, outperforming LLM-based baselines and achieving better averaged results than supervised generative baselines trained on the target
datasets.
Read the full abstract ▾
All authors Xudong Yuan, Shunyu Liu, Tongya Zheng, Huiping Zhuang, Mingli Song, Kaixuan Chen
Machine Learning · Cross-listed from cs.ET
A Game-Theoretic Framework for Incentive-Compatible AI training Under Renewable-Energy Constraints
Konstantinos Varsos, Ramin Khalili, Adamantia Stamou et al. (+2 more)
As artificial intelligence systems increasingly rely on distributed and collaborative training, the energy footprint of these processes becomes a shared responsibility. Modern AI training often unfolds across heterogeneous compute nodes-ranging from cloud clusters to edge devices-whose energy availability is spatially and temporally variable. At the same time, renewable energy grids experience growing levels of excess generation, creating opportunities to align computational workloads with low-carbon energy supply. In this work, we develop a game-theoretic model of carbon-aware AI training in which autonomous
agents strategically choose whether to participate and how intensively to train under limited renewable energy availability. Each
agent balances diminishing learning returns, rewards for remaining within green-energy budgets, and penalties for grid consumption. While our framework applies broadly to distributed AI training, we examine
Federated Learning as a representative case study due to its decentralized structure and flexible scheduling. We analyze
equilibrium existence, efficiency, and adaptive dynamics, and provide simulation evidence that appropriately designed incentives can eliminate grid-based energy usage while preserving model performance. Our findings demonstrate how incentive-compatible training mechanisms can enhance energy efficiency and sharply reduce carbon emissions under renewable-energy constraints.
Read the full abstract ▾
All authors Konstantinos Varsos, Ramin Khalili, Adamantia Stamou, George D. Stamoulis, Vasillios A. Siris
Machine Learning
Representing Clinical Conditions on Vital Signs from Healthy Individuals using Latent Modeling
Rafael Pina, Varuna De Silva and Mindula Illeperuma
Machine learning can be crucial to help scale complex signal processing applications in scenarios such as healthcare. However, these machine learning models need rich
datasets to be trained and there are often cases where it is not possible to access representative
datasets. In this paper, we propose a deep generative model based on conditional variational autoencoders with the objective of augmenting the vital signs of healthy individuals in a way that mimics the patterns of a certain clinical condition. More specifically, we use a publicly available ICU (Intensive Care Unit)
dataset to train our model and then evaluate it using the vital data that we have collected from healthy individuals. Our results demonstrate that the proposed model can not only learn the underlying dynamics of the ICU data but, more importantly, can reshape our collected data from healthy individuals in a way that is aligned with the vital signs of a certain clinical condition. We propose a distance metric that shows how our model can generate samples that are more aligned with the intended clinical labels when compared to the tested baselines.
Read the full abstract ▾
Machine Learning
Robust and Efficient Communication for Multi-Agent Learning
Rafael Pina, Varuna De Silva and Corentin Artaud
Effective communication is a cornerstone of distributed intelligence in Multi-
Agent Reinforcement Learning (MARL), yet ensuring that generated messages are both informative and robust to physical constraints remains a significant challenge. This paper introduces Multi-
Agent Regularized Communication (MARC), a novel framework inspired by information-theoretic principles of conditional mutual information. MARC employs an
attention-based architecture coupled with a unique message regularization mechanism designed to minimize uncertainty regarding future system states, thereby inducing the learning of highly representative communication protocols. Crucially, we evaluate MARC under stringent communication bottlenecks and lossy channels, simulating the real-world constraints of autonomous robotic networks and decentralized systems. Our results demonstrate that MARC significantly outperforms state-of-the-art methods in complex cooperative domains. Furthermore, we provide a deep analysis of message characteristics, proving that MARC maintains high operational performance even under significant data compression, offering a scalable path for deploying intelligent
agents in resource-constrained environments.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Approximating Smooth Functionals with ReLU Networks
Shuhao Jiao
We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable
Hilbert space by ReLU neural networks. A key feature in deep learning for functional data is the varying importance of different coordinates/dimensions. Representing the functional input in a basis expansion, we quantify the importance of each coordinate through both the magnitude of its corresponding basis score and the directional sensitivity of the target functional. Our analysis combines coordinate truncation, anisotropic partitioning, local Taylor approximation, and ReLU network realization, while allowing unrestricted interactions among the retained coordinates. We establish a general nonasymptotic upper bound for the uniform approximation error and a complementary pseudo-dimension-based lower bound for the worst-case approximation error. Under generalized exponential coordinate decay $w_ds_d\asymp\exp(-cd^\rho)$, with $\rho>0$, the upper and lower bounds match at the leading order, which is stretched-exponential in the logarithm of the network size budget, and thus yield the nearly optimal approximation rate. This is the first work to characterize neural network approximation error for infinite-dimensional functional inputs explicitly through the joint dimensional decay of coordinate magnitudes and directional sensitivities.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Parameter-Efficient Adaptation of Pretrained Language Models for Time-Series Forecasting
Tamanna Kumavat, Georg Brunner and Kyriakos Flouris
We study the adaptation of pretrained
language models to univariate time-series
forecasting through a parameter-efficient transfer learning framework, with the goal of understanding which design choices drive effective cross-modal transfer. While
language models operate on discrete textual
tokens, time series consist of continuous numerical observations with temporal dependencies. To bridge this modality gap, we project fixed-length time-series patches directly into the
embedding space of a pretrained GPT-2 backbone, bypassing textual tokenization and treating the
Transformer as a generic sequence encoder. Through controlled ablation studies on seven
benchmark datasets spanning energy, weather, traffic, and finance, we analyze the effects of (i)~representation strategy (continuous
embeddings versus textual serialisation), (ii)~adaptation regime (frozen backbone versus partial or full fine-tuning), (iii)~architectural components such as adapters, pooling strategies, and prediction heads, and (iv)~input context length. Continuous patch-based
embeddings consistently outperform textual prompting and randomly initialised backbones. The adapted pipeline attains MASE within the range of specialised
forecasting architectures while updating less than 1% of total model parameters. Results further indicate that freezing the pretrained backbone and training lightweight projection and adapter modules provides a favourable accuracy–efficiency trade-off with stable behaviour across varying context lengths.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Concept-Grounded Reasoning with Prompt-Driven Localization for Interpretable Structured Report Generation
Xinyue Xu, Hongbin Lin, Juangui Xu et al. (+6 more)
Medical imaging modalities such as ultrasound and X-ray are widely used in clinical practice, where diagnosis follows a structured, evidence-driven workflow aligned with standardized criteria. While
multimodal large
language models (MLLMs) show promise for automated medical report generation, most existing systems rely on end-to-end
multimodal fusion without modeling clinically defined intermediate attributes, leading to limited grounding and interpretability. To address this issue, we propose CORAL (COncept-grounded
ReAsoning with Localization), a
multimodal framework that integrates spatial grounding and concept-level supervision into a unified
reasoning process. CORAL employs a
prompt-driven medical
segmentation model to localize lesions and predicts multi-class clinical attributes through a Concept Bottleneck module. The resulting textual concept
tokens are combined with mask-modulated visual features within an MLLM to enable structured report generation and diagnostic prediction. Experiments on BUS-CoT and IU X-ray
datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs, indicating that concept-grounded
reasoning better aligns generation with clinical decision processes.
Read the full abstract ▾
All authors Xinyue Xu, Hongbin Lin, Juangui Xu, Hualiang Wang, Lehan Wang, Lijie Hu, Weiyang Liu, Adrian Weller, Xiaomeng Li
Machine Learning · Cross-listed from cs.AI
Evaluation Metrics for Safe Reinforcement Learning
Lindsay Spoor, Aske Plaat and Thomas Moerland
Safe reinforcement learning (RL) is commonly formalized as a Constrained Markov Decision Process (CMDP), in which an
agent maximizes expected reward while keeping its expected cumulative cost below a specified safety bound. Existing safe RL
benchmarks predominantly report whether an
algorithm is safe on average, following this expectation-based guarantee. We argue that this convention is insufficient to reliably characterize an
algorithm's true safety: it fails to capture how often and how severely the safety bound is violated, whether this holds consistently across tasks and safety bounds, and whether training-time behavior is representative of behavior of the final converged policy. Therefore, we introduce (i) evaluation metrics for safe RL that address each of these concerns and in addition allow for aggregation across tasks and safety bounds. We furthermore define (ii) a safety tier system to systematically categorize and compare
algorithms in terms of safety and reliability at both training and for a final policy. Using this framework, we provide (iii) an empirical safety evaluation across multiple safety
navigation tasks. Our results show that aggregate metrics, distributional reporting, and task- and safety bound-specific results each reveal information the other metrics cannot. We therefore recommend reporting all three jointly, rather than compressing this information into a single value, as is common practice. We provide SafeRLEval, an open-source evaluation suite to support the reliable characterization of safety in future safe RL research.
Read the full abstract ▾
Machine Learning
The Universe of Universes: Benefit Yield Functions, Implosion Thresholds, and Infrastructure-Aware Optimization in Multi-LLM Systems
Danielle Franklin and Vasu Raj Jain
We introduce the Universe of Universes (UoU) framework, which treats the full ecosystem of major large
language models (LLMs) as a structured
retrieval corpus and proposes a compositional Automated
Reasoning (AR) and Machine Learning (ML) architecture for cross-model
retrieval-augmented generation. The central contribution is the formal characterization of the Benefit Yield Function (BYF), the marginal performance gain per additional model added to an ensemble, and the identification of the implosion threshold {\theta}*: the ensemble size at which BYF crosses zero and aggregate performance begins to degrade. Existing LLM ensemble and mixture-of-
agents systems treat models as responders and aggregate outputs, but do not study performance as a function of ensemble size N across the full model universe.
Benchmark research confirms performance plateaus at the individual model level; model collapse literature establishes that iterative training on AI-generated outputs degrades individual model distributions. Neither body of work formalizes the ensemble-level implosion threshold, models Epistemic Hereditary Drift (EHD) at the ecosystem level, or treats AI manufacturing velocity as a co-variable of {\theta}*. The framework has direct implications for DoD multi-model AI acquisition policy and the emerging science of testing AI-enabled systems.
Read the full abstract ▾
Machine Learning · Cross-listed from gr-qc
Deterministic denoising of long-duration gravitational-wave signal candidates with $\alpha$-(de)blending
Przemysław Figura and Michał Bejger
We present a deterministic machine-learning method for denoising candidate signals from all-sky searches for continuous gravitational waves from rotating
neutron stars. Using the time-domain $F$-statistic search pipeline, we post-process the resulting $F(f,\dot{f})$ candidate patterns with iterative $\alpha$-(de)blending, a deterministic
diffusion-type generative model implemented as a U-Net, well suited to the non-Gaussian, correlated noise of the $F$-statistic output. Two models, trained on data based on simulated 6- and 12-day time-domain segments with software-injected signals similar to the LVK hardware injections, are tested by comparing deblended images to the library of expected signal patterns via the Structural Similarity Index Measure. In general the method recovers the correct sky-position-dependent pattern for signal-to-noise ratios $\rho \gtrsim 4$, moderately below typical all-sky detection thresholds, with recovery depending strongly on pattern morphology, demonstrating that deterministic
diffusion-based denoising may serve as a consistency/veto tool ahead of the semi-coherent coincidence stage. We also discuss limitations and possible improvements of the method.
Read the full abstract ▾
Machine Learning
When Correlations Mislead: Confounder-Aware Multi-View Urban Region Representation Learning
Sean Bin Yang, Ying Sun, Zongyi Xu et al. (+6 more)
Urban region representation learning commonly combines heterogeneous data sources, such as mobility flows, points of interest, and land-use information, to support tasks including mobility analysis, public safety
forecasting, and service demand estimation. Existing multi-view methods typically improve region
embeddings by strengthening interactions across views. However, such methods often overlook view-specific regional structures and may propagate correlations induced by shared latent factors, which can reduce the stability of downstream predictions. To overcome this major limitation, we propose CURE, a confounder-aware framework for multi-view urban region representation learning. CURE first encodes each view with its regional graph structure, estimates a shared latent component, and then reduces its projected influence before cross-view interaction. A hierarchical graph-aware fusion module subsequently aggregates the residual view representations using local and global regional contexts Experiments on three real-world cities show that CURE improves predictive performance, remains robust under missing and noisy input views, and provides reliable cross-view integration through shared component separation and context-dependent view weighting.
Read the full abstract ▾
All authors Sean Bin Yang, Ying Sun, Zongyi Xu, Tung Kieu, Jilin Hu, Bin Yang, Kristian Torp, Hua Lu, Torben Bach Pedersen
Machine Learning
Admissable: Training Reinforcement Learning Agents against Adversarial Missingness
Paul Stahlhofen, Luca Hermes, Tim Kochs et al. (+2 more)
In order to make Reinforcement Learning
algorithms applicable in real world scenarios, safety must be ensured even under adverse operating conditions. In this work, we consider the challenge of
adversarial feature missingness: a scenario in which an adversary occludes features from the
agent's observation in order to reduce performance as much as possible. We formally define
adversarial missingness for Reinforcement Learning and compare it to the related concepts of $\ell_\infty$-norm bounded
adversarial perturbations and learning with missing data. We develop an
adversarial training
algorithm and show its effectiveness in increasing robustness against
adversarial missingness on three MuJoCo
benchmark environments. Compared to a baseline trained with random uniform missingness, our method achieves better robustness on all three tasks.
Read the full abstract ▾
All authors Paul Stahlhofen, Luca Hermes, Tim Kochs, Markus Vieth, Barbara Hammer
Machine Learning · Cross-listed from math.HO
Math for AI safety: an invitation for mathematicians
Lionel Levine
Artificial intelligence threatens to outrun human understanding and control. New mathematics is needed to design AI that is legible, steerable, and cooperative with humanity. I organize this invitation by mathematical field, so you can turn straight to your own: logic and game theory for cooperation; probability for agency and world-models;
algebra and representation theory for learned features; analysis and geometry for generalization and training dynamics. Each section ends with an open problem that is accessible to a working mathematician with no prior experience in AI safety.
Read the full abstract ▾
Machine Learning
Impute-EM: Native Mixed-State Diffusion Models for Heterogeneous Data Imputation
Sergei Kholkin, Kirill Sokolov, Dmitry Baranchuk et al. (+2 more)
Missing values are ubiquitous in heterogeneous data mining, where numerical, categorical, and binary variables often coexist. Many imputation methods, especially
diffusion-based ones, treat discrete variables through continuous surrogates such as one-hot relaxations rather than modeling them natively. This creates a mismatch between the model state space and the mixed discrete and continuous structure of the data. We propose Impute-EM, an Expectation Maximization style framework that alternates between imputing missing entries with the current model and refitting a
diffusion backbone on completed data. We instantiate Impute-EM with native mixed-state
diffusion backbones for heterogeneous data, combining Gaussian and masked categorical components without one-hot relaxations. In exact settings, we characterize the update and show that the observed mask-indexed marginals match the targets at the limit, while making explicit that the full data distribution is generally non-identifiable from incomplete observations alone. Empirically, Impute-EM delivers the best distributional fidelity on mixed-type tabular imputation, on which downstream modeling relies, with text imputation serving as a controlled validation of the native discrete backbone.
Read the full abstract ▾
All authors Sergei Kholkin, Kirill Sokolov, Dmitry Baranchuk, Evgeny Burnaev, Alexander Korotin
Machine Learning · Cross-listed from cs.CL
Artificial entrepreneurial cognition: Locating and causally steering an opportunity recognition dial inside large language models (LLMs)
Christian Fisch, Angela Altmeier, Martin Obschonka et al. (+2 more)
Entrepreneurial cognition is a foundation of entrepreneurship research. Yet the growing involvement of large
language models (LLMs) in entrepreneurial work extends the cognition question beyond human actors to systems whose internal representations remain largely unexplored. We introduce artificial entrepreneurial cognition, the functional organisation of entrepreneurship-relevant representations and computations inside artificial intelligence (AI) systems. We bring mechanistic interpretability into entrepreneurship research through representation engineering. Focusing on opportunity recognition (OR), we construct 636 matched OR-present and OR-absent scenario pairs and recover an OR direction in Llama 3.1 8B-Instruct. Rather than infer the construct from outputs, we intervene directly on this direction, steering the model up and down along what we call the opportunity recognition dial, and its opportunity judgments shift with it. To our knowledge, this is the first causal intervention on an internal representation of an entrepreneurship construct inside an LLM. Held-out tests, lexical and topical controls, behavioural ablation, and
geometric comparisons show that the direction is recoverable, consequential, and distinct from the opportunity evaluation and exploitation directions, although steering it also shifts judgments about these neighbouring stages. Recovery, signed steering, and
geometric separation hold across four additional LLMs spanning different scales and families. These results give the contested distinction between opportunity recognition and evaluation a concrete representational form inside AI systems. More broadly, they establish internal representations as a new object of entrepreneurship inquiry and show how entrepreneurship theory can guide their identification, causal manipulation, and interpretation.
Read the full abstract ▾
All authors Christian Fisch, Angela Altmeier, Martin Obschonka, Michal Kosinski, Pin Ni
Machine Learning
Draining Fictitious Knots: Restoring Distance-Awareness Guarantees for High-Dimensional Spline Networks
Masoud Ataei, Mohammad Javad Khojasteh and Vikas Dhiman
Kolmogorov-Arnold Networks (KANs) with spline activations have recently shown promise for interpretable function approximation. Distance-Aware Error for Kolmogorov Networks (DAREK) introduces a computationally efficient bottom-up approach to uncertainty quantification by equipping KANs with distance-aware error bounds; yet, in high-dimensional settings, the theoretical guarantees can be weakened by the emergence of fictitious
knots. Inspired by the Kolmogorov-Arnold representation
theorem, DAREK adopts a componentwise formulation in which each input dimension is treated separately; as a result, induced
knot locations may appear in the combined input space without corresponding to actual training data. These fictitious
knots mislead the DAREK uncertainty
estimator into reporting low uncertainty far from any real observation, violating the distance-awareness guarantee. We identify this failure mode precisely, characterize its
geometric structure, and propose a drainage uncertainty mechanism that restores distance-awareness by constructing a monotonically decreasing uncertainty path from any fictitious
knot region toward the nearest real
knot. The proposed drainage method provides a practical heuristic correction that mitigates the fictitious-
knot failure mode while restoring theoretical distance-awareness in high-dimensional settings. Experiments on a 2D synthetic
benchmark and a 100-dimensional face
dataset show that drainage raises sampled distance-awareness (SDA) from 85% to 98-99%, matching Gaussian processes at lower computational cost.
Read the full abstract ▾
Machine Learning
Learning CNF Formulas from Uniform Random Solutions: Near-Tight Sample Complexity for Valiant's Algorithm
Weiming Feng, Yixiao Yu and Yiyao Zhang
We revisit Valiant's
algorithm (Commun. ACM'84) for learning $n$-variable CNF formulas with clause size $k$ and variable degree $d$ from i.i.d. uniform random solutions in the local lemma regime. For fixed $t\geq1$, under $k\gtrsim(1+1/t)\log d$, Valiant's
algorithm achieves total variation error $\varepsilon$ with $\widetilde{O}(n^{\lceil t \rceil}/\varepsilon)$ sample complexity. For $t>1$, we prove a matching lower bound for Valiant's
algorithm. At $t=1$ (covering $0<t<1$), we show Valiant's
algorithm has optimal sample complexity up to logarithmic factors by an information-theoretic lower bound $\widetilde\Omega(n/\varepsilon)$.
Read the full abstract ▾
Machine Learning · Cross-listed from math.OC
Improving the Last-Iterate Guarantees of Anytime Algorithms for Stochastic Monotone Variational Inequalities
Jun-Hyun Kim and Ahmet Alacaoglu
We analyze a
stochastic algorithm with Halpern-type anchoring for constrained convex-concave problems and monotone variational inequalities. This single-loop and single-call
algorithm uses one unbiased sample of the gradient operator at every iteration, to be applicable to monotone games with noisy feedback. With $t$ denoting the iteration counter, we prove an anytime last-iterate convergence rate of $O(t^{-1/4})$ for both the gradient-mapping norm and restricted gap, bypassing the $O(t^{-1/5})$ constrained-anytime bottleneck in the literature. Specializing then to multi-point oracles, we use variance reduction to achieve the $O(t^{-1/2})$ rate with an anytime single-loop
algorithm using $2$ samples per iteration. Our results allow constrained problems with a potentially unbounded feasible set; as well as a structured class of
stochastic oracles whose variance need not be uniformly bounded.
Read the full abstract ▾
Machine Learning
BioDCASE: Active Learning for Bioacoustics
Ben McEwen, Rupa Kurinchi-Vendhan, Shiqi Zhang et al. (+3 more)
Ecological monitoring increasingly relies on machine learning models, whose performance depends on the quality and quantity of labelled data. However, obtaining these labels is costly, particularly in passive acoustic monitoring, where vast amounts of data are collected but only a small proportion can feasibly be annotated. Active learning addresses this bottleneck by prioritizing which samples should be labelled. However, progress is difficult to measure, because published methods are evaluated under different models, budgets, evaluation metrics and
datasets. To address this challenge, we present the 2026 Active Learning for Bioacoustics BioDCASE challenge: a systematic evaluation of sampling methods designed to identify effective AL strategies. Participant methods were evaluated across four subsets composed of terrestrial and marine data. Across ten proposed sampling methods from seven teams, the top-ranked method achieved an area under the learning curve 26.4 % higher than random sampling at the same annotation budget, averaged over four data subsets. Significant variation in performance was observed across subsets, with the top-performing submission achieving a 67.1 % gain for the HSN subset over random sampling and a gain of 8 % for the ATBFL subset. Top-ranking submissions combined multiple acquisition signals, and diversity-based selection outperformed pure uncertainty sampling. Furthermore, there is evidence that transitioning from diversity-based to uncertainty-based selection and explicitly reducing redundancy within acquisition batches improve model training. There is also initial evidence that larger acquisition batch sizes may be increasingly beneficial later in the labelling process.
Read the full abstract ▾
All authors Ben McEwen, Rupa Kurinchi-Vendhan, Shiqi Zhang, Lukas Rauch, Marek Herde, Sara Beery
Machine Learning · Cross-listed from stat.ML
Conformal Individual Treatment Effect Estimation under Networked Interference
Matteo Zecchin and Osvaldo Simeone
Conformal counterfactual prediction constructs prediction sets with finite-sample coverage guarantees for counterfactual outcomes and individual treatment effects under the no-interference assumption. In this work, we relax this assumption by allowing each unit's potential outcomes to depend on other units' treatments and
covariates. In this setting, propensity-score reweighting does not restore weighted exchangeability, and existing methods may fail to achieve valid coverage. To address this issue, we develop interference-adjusted weighted conformal prediction that accounts for interference by constructing an observable upper bound on the ideal and unobserved conformal $p$-value under the target intervention. The resulting prediction sets provide finite-sample marginal coverage guarantees for counterfactual outcomes and individual treatment effects in both transductive and inductive settings. We also derive a sharper construction when intervention-induced changes in nonconformity scores are bounded. Numerical experiments show that our methods preserve nominal coverage, whereas existing methods may not.
Read the full abstract ▾
Machine Learning
Bandits with Probing: Optimal Regret and the Limits of Winner Feedback
Yongjie Guan
A learner probes at most $k$ of $n$ arms each round, receives the maximum of their rewards in $[0,1]$, and competes with the best fixed arm. When does the probing advantage pay for learning? We determine two minimax laws. Under independent
stochastic rewards with winner feedback (the maximum and a winning label), or on arbitrary fixed sequences given a single signed contrast between block maxima, the minimax regret has order $\Phi_{n,k}(T)=\min\{\frac{n-k}{n}T,\frac{n-k}{k}\}$, $2\le k<n$. Under winner feedback, both arbitrary joint i.i.d. rewards and fixed sequences have minimax regret of order $R_{n,k}(T)=\frac{n-k}{n}\min\{T,\frac{n+T}{k},\sqrt{\frac{nT}{k}}\}$. Both laws have universal constants and anytime upper bounds. The first reduces regret to a pure coverage cost: same-round contrasts absorb the stability cost, and independence permits exact resampling whose gains fund sample advancement. The second adds a learning cost that becomes comparable to coverage at horizon $n$; beyond $nk$, numerical maxima improve over labels alone. The lower bound allows every adaptive action size.
Read the full abstract ▾
Machine Learning
ProtoGuide: Prototype-Driven Guidance for Class-Conditional Graph Generation
Salvatore Romano, Marco Grassia, Pietro Liò et al. (+1 more)
Discrete
diffusion models are a prominent family for graph generation, but standard class-conditional mechanisms embed the class signal in the denoiser during training, tying the conditioning mechanism to the trained model. Classifier guidance avoids this coupling in continuous domains by steering a frozen model with a classifier's gradient, but discrete graph
diffusion samples discrete edge states, so gradients cannot propagate through the sampled graph. We introduce ProtoGuide, a post-hoc, backbone-agnostic framework that recovers an analogous mechanism. At each reverse step the denoiser's per-edge output is relaxed into a differentiable soft adjacency, embedded by a frozen Siamese graph neural network, and scored against a target-class prototype and its nearest competitor; the resulting per-edge gradient, damped by a cosine schedule, is injected back into the denoiser output. All components stay frozen, so guidance is retargeted by supplying a different prototype. On five classes of real-world networks and two architecturally different backbones, EDGE and DiGress, ProtoGuide raises macro classification accuracy from 50.7% to 73.5% and from 73.6% to 83.8%, and outperforms DiGress's built-in conditional training under our configuration. Gains are largest where the unguided models are weakest, and are not uniform across classes. Per-graph coverage remains high in most settings, while distributional effects are class-dependent. A Best-of-N selection baseline matches this accuracy given enough oversampling, but at a substantial cost in graph diversity. An independently initialized classifier, a directionality test, and a few-shot analysis support target-directed steering and robustness to very small support sets.
Read the full abstract ▾
All authors Salvatore Romano, Marco Grassia, Pietro Liò, Giuseppe Mangioni
Machine Learning · Cross-listed from cs.SD
MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training
Tianyi Xu, Daniel Pimentel-Alarcón, Zuzana Buřivalová et al. (+1 more)
Passive acoustic monitoring can measure biodiversity at larger scales, but time–frequency annotation of animal vocalizations is expensive, site-specific, and difficult to sustain at scale. We present a label-efficient sound detection framework that combines masked audio pretraining with a lightweight detector on mel spectrograms, then further improves robustness through iterative self-training on unlabeled audio. We first pretrain a ViT-based encoder on unlabeled recordings via masked reconstruction and transfer the encoder to a detection backbone. To better separate animal sounds from confounding background, we add a box-level contrastive loss that pulls matched event regions together while pushing noisy negatives apart. We then apply a two-stage pseudo-labeling curriculum to exploit large unlabeled pools without additional annotation. We evaluate the performance on two ecologically distinct domains: tropical rainforest soundscapes (Indonesia) and bird vocalizations in Mediterranean habitats (Spain). On both domains, masked audio pretraining and contrastive learning consistently improve time–frequency detection under temporal and cross-site distribution shift, and self-training yields further gains in out-of-distribution performance. On the rainforest domain, MAST with self-training achieves +0.22 mAP and +0.24 F1 over the strongest baseline under cross-site shift. On the bird domain, self-training achieves +0.12 mAP and +0.10 F1 over the strongest baseline under cross-site shift. Overall, our results show that MAST can effectively extend self-supervised audio representations from clip-level tasks to robust box-level localization across diverse bioacoustic settings, providing a practical path for biodiversity monitoring with limited labels.
Read the full abstract ▾
All authors Tianyi Xu, Daniel Pimentel-Alarcón, Zuzana Buřivalová, Claudia Solís-Lemus
Machine Learning
Convergence rates for generative drifting flows: fixed-scale obstructions and multihead acceleration
Arthur Stéphanovitch and Eddie Aamari
Drifting models offer a promising route to faster generative AI: they perform gradual transport during training, while generating new samples in a single step. This paper asks whether the underlying drifting process can converge rapidly to a target distribution under ideal conditions, before finite-data or optimization effects are introduced. We show that its convergence rate depends critically on how it handles spatial scale. With a single fixed resolution, fine-scale features of the target can become nearly invisible, leading to extremely slow convergence. We introduce a multihead approach that combines scale-normalized information across a continuum of resolutions. We prove that this multihead approach restores exponential convergence near standard reference distributions. These results identify fixed resolution as a key bottleneck and provide a simple route to faster one-step generative models.
Read the full abstract ▾