Machine Learning
When Does Context Routing Help? A Systematic Study of Multi-Modal Fusion in Time Series Forecasting
Ruizhe Zhou, Gaoyuan Du, Xiaoyang Liu et al. (+4 more)
Multi-modal time series
forecasting methods integrate auxiliary context into temporal predictions through increasingly sophisticated fusion mechanisms. A growing body of work reports substantial gains, yet it is often unclear whether they reflect genuine use of the context or incidental architectural effects. We ask a narrower, checkable question: when can auxiliary context help a forecaster at all? We identify two
dataset-level conditions that must both hold: (1) the target is not dominated by a last-value shortcut (low autocorrelation rho_h), and (2) the context carries information about the target beyond history (non-zero conditional mutual information delta; when delta=0 no predictor can benefit—a distribution-free result). Through controlled experiments on MoME (a 14.3B-parameter mixture-of-experts model, 6
datasets, 10 seeds) and four additional fusion mechanisms implemented within a single-backbone testbed (5
datasets), we find that when both conditions hold, text-conditioned expert modulation contributes a sizeable MSE reduction; when either fails, the contribution collapses to the capacity floor of the modulation pathway and carries no context-attributable signal. We establish causality through two interventions: adding a shortcut to MoME suppresses routing contribution by 77-93% across 3
datasets; progressively corrupting context quality drives the context-specific benefit from +44% to negative. We validate the autocorrelation component of our diagnostic on 27 Monash Archive
datasets. We provide a calibrated pre-training diagnostic that, on the
datasets we test, yields no false positives in well-powered settings. We are explicit about the asymmetry of our evidence: the negative arm is broadly reliable, while the large positive magnitudes come from a single model family (MoME) and are corroborated only in direction by the testbed.
Read the full abstract ▾
All authors Ruizhe Zhou, Gaoyuan Du, Xiaoyang Liu, Haoqi Yao, Deepayan Chakrabarti, Jiating Lin, Yixuan Shen
Machine Learning
Multimodal Injury Risk Prediction in Tennis
Francisco Erramuspe Alvarez, Shobharani Polasa, Weihao Qu et al. (+2 more)
Machine learning has had a significant positive impact on the prediction of athlete performance and injury risk. Most works in this field rely on subjective observations and expert assessments, which restrict their effectiveness. In sports like soccer, basketball, and wrestling, some studies attempt to address this challenge by integrating data from alternative sources, such as readings from wearable devices, alongside traditional subjective observations and expert assessments to enhance accuracy. However, similar research in tennis remains largely unexplored. In this paper, we propose a
multimodal Predictive Athlete Readiness framework for Tennis (PART) to assess both performance and injury risk in tennis players. By leveraging machine learning and deep learning techniques, PART processes multiple sources of data collected from nine collegiate tennis players, including physiological metrics, training and match data, sleep data from wearable devices, self-reported information via daily questionnaires, jump assessments, and motion analysis from match play
videos. PART captures four characteristics of tennis players: overall wellness, injury risk, physical capability, and playing style. By integrating these four characteristics by supervised learning, it is capable of providing a holistic assessment of the tennis athlete's condition, along with advanced forecasts of specific body areas at risk such as the upper body (e.g., elbows) or lower body (e.g., knees). Our evaluation, conducted with data from nine collegiate tennis players, shows that PART achieves strong performance in predicting both overall wellness and injury risk. Additionally, our framework also shows promise for recreational tennis players, who often suffer from injuries due to incorrect playing techniques.
Read the full abstract ▾
All authors Francisco Erramuspe Alvarez, Shobharani Polasa, Weihao Qu, Jay Wang, Ling Zheng
Machine Learning
Toward Machine Learning with the Unit as a Primitive: Learning from Unit-Linked Events
Heyang Gong
Machine learning is usually formalized through samples, while the persistent individual to which multiple observed or possible events refer often remains implicit. We propose the unit as an explicit primitive at the level of task semantics. A learning task first declares a population of persistent referents and a sameness criterion; the realized value $u$ denotes the selected referent. Supervised learning is the main formal specialization. Its semantic object is a family of unit-conditioned response laws. Homogeneity is the special case in which those laws coincide; a sample-only conditional is silent as to whether the world is homogeneous or the observed law is only the marginal of a heterogeneous family. What is learned from data is a pair $(T_\phi,R_\theta)$: a tokenizer that produces a contextual unit
token and one shared response-law form that reads it. The structured class takes that form to be a simple relation in the
token; a linear predictor is the running instance. The
token is the learner-side representation through which the task-side unit affects prediction, while a learner specification that omits unit information is unit-insensitive; homogeneity remains a property of the world-side response family. When identity is unresolved, the world-side law mixes unit-conditioned targets, while the learner composes its shared form with a
token. A trusted resolver may fix the unit and supply a lookup
token; otherwise unit abduction forms a
token of the same type from factual evidence. Unlinked single-row observations can fail to distinguish a heterogeneous unit world from a homogeneous pooled world; trusted same-unit pairs separate a restricted witness. The formal results concern this supervised specialization.
Read the full abstract ▾
Machine Learning
GRAPE: Gradient Refinement and Progress-Aware Exploitation for Query-Efficient High-Dimensional Bayesian Optimization
Richard Cornelius Suwandi and Feng Yin
Optimizing expensive, high-dimensional black-box functions remains a central challenge in modern machine learning and scientific discovery. While local
Bayesian optimization mitigates the curse of dimensionality, existing techniques often prioritize the probability of descent over the magnitude of progress. This leads to overly conservative steps that yield negligible improvement, wasting queries on directions that are nearly certain to descend but offer little decrease. We introduce Gradient Refinement and Progress-Aware Exploitation (GRAPE), a two-stage framework that first sharpens the local gradient posterior via a closed-form acquisition function, then selects update directions by maximizing the expected decrease conditional on descent. Theoretical analysis proves that this gradient refinement stage monotonically minimizes local uncertainty and that the progress-aware direction converges to true steepest descent as the posterior sharpens. Empirically, GRAPE demonstrates superior query efficiency across high-dimensional tasks: in black-box
adversarial attacks, it achieves an average 5.4$\times$ speedup over baselines, and on large
language model
prompt optimization tasks, it outperforms the second best method by a reduction of 3.8 log-units in the final average regret.
Read the full abstract ▾
Machine Learning
Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment
Yan Gao, Mohammad Naseri, Javier Fernandez-Marques et al. (+19 more)
Federated learning (FL) has emerged as a key approach for training models across decentralized data, yet benchmarking in FL remains difficult to reproduce, compare, and extend. Existing evaluations are often tied to custom infrastructure, released as incomplete research code, and conducted primarily in simulation, which limits portability and practical relevance. We present Flower Hub, a platform for publishing, discovering, and executing decentralized and
federated applications. We show how it enables reproducible benchmarking by packaging
benchmarks as executable, versioned applications with standardized metadata, pinned dependencies, and explicit evaluation workflows. We instantiate this approach with a multi-domain
benchmark suite spanning cross-silo and cross-device settings, and including tasks in medical imaging, financial tabular learning, legal instruction tuning, phishing URL detection, and audio tagging. We further demonstrate that the same benchmarking application can run across both simulation and deployment runtimes without changing the application code, enabling unified evaluation across varying learning environments. Beyond model quality, our
benchmark design supports system-aware reporting, including runtime and communication metrics. This work advances benchmarking in FL settings from ad hoc code artifacts towards portable, executable, and reusable
benchmark applications.
Read the full abstract ▾
All authors Yan Gao, Mohammad Naseri, Javier Fernandez-Marques, Dimitris Stripelis, Lorenzo Sani, Davide Eynard, Fan Zhang, Hong Jia, Ting Dang, D. B. Emerson, Fatemeh Tavakoli, Ole Werger, Lars Wulfert, Petros Demetrakopoulos, Sofia Tsekeridou, InSeo Song, KangYoon Lee, Honghao Li, Lingjuan Lyu, John P Dickerson, Daniel Janes Beutel, Nicholas D. Lane
Machine Learning · Cross-listed from cs.AI
Towards Reliable, Generalizable, and Specific In-Context Knowledge Editing via Multi-Objective Reinforcement Learning
Xuzhong Wang, Maiqi Jiang, Tejal Nair et al. (+3 more)
Large
Language Models (LLMs) are powerful but limited by static parametric knowledge that becomes outdated once pretraining ends. Knowledge editing addresses this problem by updating model behavior on target facts without full retraining. In particular, in-context knowledge editing has gained
attention because it is training-free and readily applicable to black-box LLMs. Recent reinforcement learning (RL)-based approaches improve over fixed
retrieval strategies by adapting
prompt construction to the quantity-quality trade-off. Despite initial success, they fail to model the
prompt as a structured entity under the distinct and often competing objectives of reliability, generality, and specificity. Previous methods largely optimize a single objective and make decisions over only part of the
prompt construction process, thereby overlooking both the balance of different objectives and the global organization of demonstrations. We propose Multi-Objective In-context Knowledge Editing (MO-IKE), a multi-objective RL
algorithm that formulates
prompt construction for in-context knowledge editing as a Constrained Markov Decision Process. MO-IKE trains a dynamic retriever to optimize competing objectives in knowledge editing, enabling more balanced and globally coherent
prompt construction. On Llama-3.2, MO-IKE improves edit success (reliability) from 85.0% to 92.0%, paraphrase consistency (generality) from 77% to 79%, while increasing retention rate (specificity) by 23.0% compared to prior RL-based methods.
Read the full abstract ▾
All authors Xuzhong Wang, Maiqi Jiang, Tejal Nair, Girija Bhusal, Yanfu Zhang, Haipeng Chen
Machine Learning
Understanding the Energy Scaling of Large Language Model Inference Across Context Lengths and Attention Architectures
Molka Chkir, Syed Muhammad Danish, Jos Höll et al. (+1 more)
The growing adoption of large
language models (LLMs) has raised increasing concerns about the energy consumption and environmental impact of inference. This paper presents a systematic empirical study of decode-phase energy consumption across representative open-source LLMs employing Multi-Head
Attention (MHA), Grouped Query
Attention (GQA), and Grouped Query
Attention with Sliding Window
Attention (SWA) to characterize how
attention architecture influences decode-phase energy consumption under varying inference workloads. We evaluate four models across different context lengths, batch sizes, and generation workloads while measuring GPU energy using NVIDIA hardware counters. We examine the effects of context length,
attention mechanism, Key-Value (KV) cache growth, and batching on decode-phase energy consumption. Results show that
attention mechanism is the primary factor governing how decode energy scales with context length. MHA models exhibit substantially steeper energy growth than GQA models, whereas GQA with SWA maintains nearly constant energy consumption. We further show that model size primarily determines absolute energy consumption, while batching reduces both energy per generated
token and request latency by up to 87%. These findings provide practical guidance for selecting energy-efficient LLM architectures and inference configurations.
Read the full abstract ▾
All authors Molka Chkir, Syed Muhammad Danish, Jos Höll, Arghavan Asad
Machine Learning · Cross-listed from eess.SY
Scalable Self-Supervised Learning for Multiphase AC-OPF in Distribution Systems with Topology Reconfiguration
Hoang T. Nguyen, Shaohui Liu, Reetam Sen Biswas et al. (+4 more)
The proliferation of distributed energy resources (DERs) in distribution grids enables the active coordination of these assets to reduce costs and enable cleaner operations. Realizing this potential requires solving multiphase AC optimal power flow (AC-OPF) quickly across varying loads, DER availabilities, and
topology reconfigurations, at much greater speed and scale than conventional nonlinear solvers. Learning-based surrogates can offer millisecond inference, yet existing methods target largely balanced transmission systems and do not scale to the multiphase, unbalanced, and reconfigurable nature of distribution feeders at utility scale. We present the Penalty + Sequential Linearized Feasibility Seeking (SLFS)
algorithm, a self-supervised learning framework for multiphase distribution AC-OPF under switch-induced
topology changes. Penalty+SLFS requires no labeled optimal solutions and trains directly from the AC-OPF objective and constraints through a differentiable fixed-point power flow solver, avoiding expensive label generation and admitting robust training procedures.
Topology changes are handled efficiently using Sherman-Morrison-Woodbury updates of the admittance-
matrix inverse, while an M-step Jacobian approximation accelerates differentiation through the power flow solver. At inference, SLFS repairs any infeasible predictions, providing feasibility guarantees with low computational overhead. On IEEE feeders ranging from 13 to 8,500 nodes, Penalty+SLFS achieves negligible optimality gaps and near-zero constraint violations, delivers up to three orders of magnitude speedups over IPOPT, and remains robust under large distributional shifts, demonstrating a viable path toward real-time,
topology-aware AC-OPF for large-scale distribution grids.
Read the full abstract ▾
All authors Hoang T. Nguyen, Shaohui Liu, Reetam Sen Biswas, Varsha Pendyala, Nurali Virani, Deepjyoti Deka, Priya L. Donti
Machine Learning
The Von-Neumann State-Space Transformer for neural decoding
Morteza Sarafyazd
Cortical computation is strikingly low-dimensional: a handful of latent variables, carried in a neural population's activity, steer the higher-dimensional responses of individual
neurons. Our aim is sample efficiency-models that decode well from limited data and at small parameter budgets. In a standard
Transformer layer, the feed-forward block applies the same operator to every
token. We suggest a von-Neumann inspired hypothesis of efficient computation as an alternative for neural decoding: a controller decodes an instruction and then executes a
token-specific operator; the usual realization-a soft mixture of experts-only blends their outputs, not operators. We introduce a von-Neumann State-Space
Transformer (VN-SST), a
memory-augmented
Transformer whose feed-forward block is a low-rank instruction bank: a shared base operator plus a small set of learned low-rank instructions, from which a per-
token code synthesizes the weight
matrix actually used at that
token. The code is read from a low- dimensional projection of a carried state-space
memory, so a slow latent trajectory acts as an instruction pointer-mirroring how low-dimensional dynamics may route cortical computation. On three motor-cortex neural-decoding
benchmarks, VN-SST is far more data-efficient than a modern
Transformer, each jointly predicting spikes and decoding behavior. This model wins by a wide margin on the scarcest
benchmark, leads on the other two, and turns longer context into rising rather than falling accuracy. We evaluated that the network compresses a large instruction bank to a few bits per
token, so program capacity acts as a control channel, not an accuracy lever. The same model is also more parameter-efficient on two small text
benchmarks used for
language modeling (LLMs), suggesting a generic mechanism.
Read the full abstract ▾
Machine Learning
The Frame Kernel Method for Multiscale Operator Learning
Branden Frieden, Ryan Whitehead, M. Keith Ballard et al. (+2 more)
We present a natively multiscale operator learning method for the surrogate modeling of (numerical solvers for) multiscale partial differential equations (PDEs). The primary novelty of our method lies in a novel multiscale kernel frame function approximation technique. Leveraging this new kernel frame technique, we cast the operator learning problem as one of learning frame coefficients of output functions as a function of frame coefficients of input functions. The generalization step then automatically allows for a multiscale decomposition of the output functions. Our method is applicable to both tensor-product grids and point clouds. We present interpolation proofs, error estimates, and numerical convergence rates for our frame approximation. We the demonstrate the applicability of our method for the surrogate modeling of inherently multiscale PDEs. The new multiscale frame kernel method is significantly more accurate than popular neural operators on challenging problems from the literature, while simultaneously admitting an a posteriori multiscale decomposition upon generalization.
Read the full abstract ▾
All authors Branden Frieden, Ryan Whitehead, M. Keith Ballard, Robert M. Kirby, Varun Shankar
Machine Learning
NVExplain: Explaining Time Series Forecasting with Latent Trajectory Analysis and Structure-Preserving Surrogates
Muyan Anna Li, Manikandan Ravikiran and Aditi Gautam
Time series
forecasting models are widely used in high-stakes settings, yet their predictions remain difficult to interpret because existing post-hoc methods often ignore temporal dependence and fail to provide horizon-specific explanations. We propose a model-agnostic explainability framework that explains
forecasting predictions by attributing each forecast horizon to temporally relevant historical lags. The framework models
forecasting as a latent trajectory and introduces semantic flow to quantify how information evolves across time in the model's internal representations. By aggregating semantic flow, it constructs a lag-horizon attribution
matrix that captures horizon-resolved temporal influence. To improve explainability, we further generate structure-preserving perturbations and fit sparse local surrogate models, producing human-readable and temporally coherent explanations. We evaluate the method using faithfulness and stability diagnostics across multiple
benchmark datasets. Results show that the semantic-flow variant achieves competitive or superior faithfulness compared to standard post-hoc baselines, while being substantially more computationally efficient. Stability analysis further demonstrates that the explanations are robust and identifies regimes where interpretation should be applied with caution.
Read the full abstract ▾
Machine Learning
DeMMO: Longitudinal and Cross-Disease Modelling of Digital Mobility Outcomes via Multi-Task Learning
Menghui Zhou, Zhipeng Yuan, Vitaveska Lanfranchi et al. (+1 more)
Digital mobility outcomes (DMOs) derived from wearable sensors characterise mobility in daily life and offer a promising means of monitoring disease progression. However, existing DMO studies have typically focused on either a single disease or a single visit. To the best of our knowledge, we are the first to define and study the practical problem of cross-disease longitudinal DMO modelling. We argue that this problem should satisfy at least two requirements. First, the temporal progression of DMOs should be modelled within each disease, as mobility-limiting diseases evolve over time. Second, multiple mobility-limiting diseases should be modelled jointly, as different diseases affect different aspects of human mobility. To address this problem, we propose DeMMO, an interpretable framework for longitudinal, multi-disease, and multi-outcome learning. Its central technical contribution is an interpretable cross-disease and cross-outcome relation-learning mechanism that infers signed relations directly from learned longitudinal DMO-outcome mappings, thereby enabling selective information sharing across cohorts without requiring paired participants. We evaluate DeMMO on the recently released, large-scale, multicentre Mobilise-D
dataset, which presents a challenging modelling setting involving longitudinal observations, multiple clinical outcomes, and four participant-disjoint cohorts representing distinct mobility-limiting diseases. Compared with eight strong structural longitudinal and deep-regression baselines, DeMMO achieves the best overall predictive performance and outperforms the baselines for most individual outcomes. Stability selection further identifies reliable longitudinal DMO patterns that can inform subsequent clinical validation and disease monitoring. The implementation code is available at https://github.com/menghui-zhou/DeMMO.
Read the full abstract ▾
All authors Menghui Zhou, Zhipeng Yuan, Vitaveska Lanfranchi, Po Yang
Machine Learning · Cross-listed from cs.CL
DataKernelBench: Can LLMs Optimize Database Queries on GPUs?
Gokul Karthik Kumar, Yotam Perlitz, Corey Lammie et al. (+2 more)
GPUs increasingly accelerate database systems, but query-specific peak performance still often relies on hand-written kernels. Existing LLM kernel
benchmarks focus on machine learning operators, leaving irregular, heterogeneous, data-movement-heavy database-style operators untested. We introduce DataKernelBench, which translates SQL into validated PyTorch TorchPlan programs and evaluates LLMs that optimize either the core tensor-bounded snippet or the full query in CUDA or Triton through execution-guided repair. Across ten proprietary and open-weight models on TPC-H SF10 with an H100 GPU, the strongest full-query CUDA configuration achieves $2.11\times$ speedup over the TorchPlan baseline at full pass rate. We find that higher-performing implementations commonly use kernel fusion and execution-strategy changes, stronger models benefit most from full-query specialization, and workload context matters more than hardware context. To handle data larger than GPU
memory, we extend TorchPlan with Dask-cuDF for on-demand partition loading on TPC-H SF100 with four H100 GPUs, achieving $2.54\times$ speedup. Project page: https://kerneldf.github.io/datakernelbench
Read the full abstract ▾
All authors Gokul Karthik Kumar, Yotam Perlitz, Corey Lammie, Andrea Giovannini, Katja Hose
Machine Learning · Cross-listed from stat.ML
Improved Analysis for Hessian-free High-resolution Monte Carlo Sampling
Wujun Lv, Xiaoyu Wang, Yingli Wang et al. (+1 more)
Hessian-free high-resolution (HFHR) dynamics augments underdamped Langevin dynamics (ULD) with reversible position
diffusion for sampling problems that arise in machine learning. We establish an explicit quantitative contraction rate for HFHR dynamics under a position Poincaré inequality, weighted Hessian and
Laplacian bounds, and a compact Sobolev
embedding, where the potential function is not necessarily convex. An adapted time-augmented Poincaré inequality yields an explicit rate that improves upon the contraction rate of the underdamped Langevin dynamics. We also give a weak-solution construction and a self-contained spectral proof of the divergence lemma underlying the argument. For HFHR Monte Carlo (HFHRMC)
algorithm, which is based on a discretization scheme of HFHR dynamics, we use a path-space Girsanov argument to obtain a non-asymptotic convergence bound and an explicit iteration complexity in total variation distance. The bounds hold for every $\alpha\geq0$ and $\gamma>0$ and remain regular at the ULD endpoint. Optimizing the iteration complexity bound yields a positive, accuracy-dependent position-
diffusion parameter at finite accuracy, while its leading high-accuracy order coincides with that of the optimized ULD endpoint. Our iteration complexity bound improves upon the existing work on HFHR
algorithms. Numerical experiments including
Bayesian learning problems on real data are provided to illustrate the effect of positive $\alpha$ and its benefit.
Read the full abstract ▾
All authors Wujun Lv, Xiaoyu Wang, Yingli Wang, Lingjiong Zhu
Machine Learning
On the Representational Geometry of Dynamic Programs
Richard F. M. Lim and Ruriko Yoshida
Standard neural architectures often fail to generalize to longer inputs for dynamic programming (DP) targets. We investigate what makes this hard geometrically. Every finite min-plus DP is a shortest path on a DAG, which is equivalently a tropical
polynomial whose extended Newton polyhedron encodes the decision boundary of which path wins. We prove these three descriptions (graph,
polynomial, polyhedron) form isomorphic semirings at two levels — formal
polynomials and their computed functions — connected by operations that characterize all structural redundancies. We then address the length-generalization question geometrically: does the decision boundary at length $T$ decide the boundary at $T+1$? We present two structural negatives. The semiring's two native ways to reduce dimension (setting a variable to each identity) are neither injective nor always closed within the DP. Series and parallel composition fail to construct all DAG
topologies from smaller sub-DAGs, and even all terminal-only operations do not capture all DP compositions.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Behind the [MASK]: Disentangling Representation and Faithfulness in DAPF-Based Dementia Detection
Pardis Ranjbar-Noiey and Natalie Parde
Spoken-
language analysis via
prompt-based domain-adaptive models is a promising direction for low-resource, non-invasive dementia screening, but such models remain internally opaque. We study the interpretability of the Domain-Adapted models via
Prompt-based Fine-tuning (DAPF) framework, which casts dementia detection as diagnosis-related masked-
token prediction. We interpret DAPF and strong baselines using a variety of probing and analysis techniques, finding that DAPF achieved the best overall performance (accuracy=0.83 and macro-F1=0.83) with diagnosis most recoverable from its [MASK] representation. However, this representational advantage did not extend to
token-level explanation faithfulness. DAPF attributions primarily reflected
language task vocabulary, discourse markers, and transcription artifacts, with perturbation tests showing weak or negative effects. This suggests that its masked-
token interface determines diagnosis information without producing faithful
token-level explanations.
Read the full abstract ▾
Machine Learning · Cross-listed from astro-ph.SR
EncoTESS: Age-Sensitive Encodings from Raw TESS Light Curves
Phil R. Van-Lane, Joshua S. Speagle, Ryan Cloutier et al. (+3 more)
Main sequence
stars of spectral types late F through M exhibit systematic variability in photometric light curves, particularly when they are young. Rotational modulation of starspots manifests as quasi-sinusoidal variability, which enables the measurement of rotation periods. Variability can also be
stochastic, as in
stellar flaring. However, since measurements of
stochastic processes depend on the time of observation, they are typically noisier. Considering that different manifestations of variability have unique observational nuances, models that naturally unify these are incredibly useful for
stellar characterization. Towards this goal, we have developed EncoTESS: a Time Series Foundation Model (TSFM) trained on a subset of TESS 2-min light curves. EncoTESS is specifically designed to handle the observational noise, heteroskedastic measurements, irregular sampling, and large data gaps common to TESS data. It is also ~1% of the size of a typical literature TSFM, so can be run easily on a modern laptop. EncoTESS encodes light curves into a fixed-size latent parameter space, which can be used to infer physical
stellar properties and recovers light curve summary statistics well. EncoTESS outperforms rotation period and variability amplitude as age indicators for
stars that have not converged onto the slow rotator sequence yet; broadly these include K and M
stars less than ~100 Myr, and M
stars less than ~1 Gyr. We focus on age inference as an application of EncoTESS in this work, but other downstream tasks such as
stellar classification could also be explored. The architecture of EncoTESS enables its future extension to TESS light curves of all cadences, and additional surveys such as Kepler and the upcoming PLATO mission. The core EncoTESS framework and library of encodings produced for the
stars used in this work are publicly available at https://github.com/philvanlane/encotess.
Read the full abstract ▾
All authors Phil R. Van-Lane, Joshua S. Speagle, Ryan Cloutier, Christopher A. Theissen, Gwendolyn M. Eadie, Ilay Kamai
Machine Learning
Rollout-Decoded Reconstruction for Long-Horizon Prediction in Latent World Models
Rishi Shah and Rishav Shrestha
A latent world model trains its decoder on latents anchored to observations, then deploys it on the model's own free-running rollout, hundreds of steps past the last observation. Rollout-Decoded Reconstruction (RDR) closes this gap with a single loss term that free-runs the model during training exactly as evaluation will, decodes every rollout latent, and penalizes reconstruction error against ground truth. The term adds no parameters, costs training-time compute only, and reduces to the standard objective at weight zero, so every comparison in this paper is a one-flag A/B. On the chaotic Kuramoto-Sivashinsky equation, RDR raises valid prediction time (the time to first crossing of normalized error 0.5) from $3.87 \pm 0.23$ to $6.97 \pm 0.42$ time units at an identical 193,568 parameters, a $1.80\times$ improvement confirmed on seeds never used in selection and holding in 10 of 10 preregistered configurations at ratios of 1.71-2.50$\times$. The results come from a single system; a sweep in which the advantage grows with latent width is descriptive, and control experiments on two classic tasks are preliminary.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure
Kaiqiao Han and Yizhou Sun
The imperfective paradox provides a useful test of compositional semantic analysis. Recent work constructs an NLI
benchmark and reports that models frequently infer completed telic events from progressive descriptions, attributing this behavior to a Teleological Bias. It further argues that prompting interventions cause a Calibration Crisis. We reexamine the
benchmark and conclusions and show that it is substantially affected by conceptual and evaluation mis-specifications. We identify three conceptual mis-specifications. In particular, Aspectual Reduction affects the
benchmark construction, analysis, experiments, and conclusions. Under a strict NLI standard, 76% of Group A instances do not explicitly rule out culmination. In our native-speaker annotation, 38% of Group A examples and 29% of the Group C examples were judged to permit an alternative interpretation. To control these issues and lexical variation, we construct Lexically Matched Minimal Pairs. At the evaluation level, we formulate event-semantic NLI as a Multi-step
Reasoning Problem and assess both intermediate semantic decisions and final predictions. Our results show that models often do not affirm culmination but nevertheless accept the corresponding simple-past hypothesis, a pattern we characterize as Sufficiency Bias. We further show that prompting interventions produce a Decision Shift among labels without reliably improving the underlying semantic understanding and
reasoning. Intermediate and oracle-guided analyses identify two additional failure modes: errors in compositional aspectual classification and Surface-form Attraction toward surface-associated answers. Our experiments on Qwen-7B with suitable
prompts, GPT-5.4, and Qwen-72B provide initial evidence for the context sensitivity of aspectual classification and suggest that these models can achieve performance comparable to that of human annotators.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal
Philipp E. Glass, Allan Tucker, Yongmin Li et al. (+1 more)
Activation steering can be embedded directly into a
language model's weights, shaping behaviour without inference-time intervention and offering a way to encode
alignment prior to release. However, models are routinely fine-tuned after deployment, and it is unknown whether embedded interventions survive this. We study the stability of embedded steering for refusal ablation and brevity amplification across five instruction-tuned models (3B-14B) under non-
adversarial SFT and RLHF. We find that steering degrades more under stronger optimisation pressure against the steered behaviour, which depends on both the training data and the training
algorithm. Refusal ablation loses 69% of its effect on average under our SFT setup, but only 2% under our RLHF setup. The weight-edit, by contrast, remains almost unchanged (on average only 0.4% of it is restored), and the fine-tuning update along the steering direction is close to orthogonal to the pre-edit weight pattern (mean cosine similarity 0.071). Fine-tuning that restores the behaviour does so without reversing the edit. As the steered behaviour is not fully durable, embedded steering should be re-validated after downstream training.
Read the full abstract ▾
All authors Philipp E. Glass, Allan Tucker, Yongmin Li, Alina Miron
Machine Learning
D$^3$-MOPD: Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillation
Zechen Sun, Zhiwei Zhang, Fei Zhao et al. (+7 more)
Multi-teacher on-policy
distillation (MOPD) distills several domain-expert teachers into a single student by minimizing per-domain reverse-KL divergence on the student's own rollouts. Existing approaches typically fix the per-domain data mixture before training, overlooking the fact that different domains converge at substantially different rates: some plateau early while others continue to improve throughout the training budget. A fixed mixture therefore wastes compute on fast-converging domains and undertrains slower-converging ones. To address this, we propose D$^3$-MOPD (Dynamic Domain ScheDuling for MOPD), a zero-overhead scheduler that repurposes the per-domain reverse-KL signal already produced during training to adapt the domain mixture online. Running asynchronously outside the training process, an off-process watcher periodically tracks each domain's KL trajectory, estimates remaining headroom and current improvement rate, and accordingly adjusts the domain sampling ratios without altering the core training loop. Our D$^3$-MOPD scales naturally to arbitrary numbers of domains, and the expected benefit grows as more domains introduce more diverse convergence patterns for the scheduler to exploit. On a Qwen3.6-35B-A3B student distilled from four domain-expert teachers, D$^3$-MOPD closes 97% of the average student-to-teacher performance gap, compared with 63% for vanilla MOPD, reaches the same peak performance with an approximately 3$\times$ reduction in rollout steps, and surpasses the specialist teachers on three of seven
benchmarks.
Read the full abstract ▾
All authors Zechen Sun, Zhiwei Zhang, Fei Zhao, Juntao Li, Mu Chuan, Huayu Deng, Guojian Zhan, Wenliang Chen, Yao Hu, Min Zhang
Machine Learning · Cross-listed from cs.CL
Unsupervised Post-Training of Foundation Models: A Survey
Yijie Xu, Qianyi Cai, Huizai Yao et al. (+9 more)
Foundation-model post-training usually relies on human labels, preference data, stronger teachers, or executable verifiers. We study Unsupervised Post-Training (UPT): update-bearing adaptation on unlabeled inputs whose learning signal is derived from same-lineage model artifacts rather than an external oracle. We catalog 80 strict UPT methods and organize them by the object that supplies the update signal: a prediction statistic, a sample relation, a self-generated target, or an internal evaluator. Beyond inventory, we show how the choice of internal signal and task structure determines whether post-training improves the model or recursively amplifies error. An orthogonal Input Visibility $\times$ Update Persistence view maps deployment regimes and defines a unified framework for UPT selection and evaluation.
Read the full abstract ▾
All authors Yijie Xu, Qianyi Cai, Huizai Yao, Yili Wang, Tianfu Wang, Cehao Yang, Xingbo Yao, Zhiyu Guo, Aiwei Liu, Xuming Hu, Weiyu Guo, Hui Xiong
Machine Learning
MSR-IVA: Masked Structural Residual Independent Vector Analysis for State-Aware Fusion of Structural MRI and Dynamic Functional Network Connectivity
Victor Solomon, Zening Fu, Rafal Angryk et al. (+2 more)
Multimodal fusion of structural MRI (sMRI) and dynamic functional network connectivity (dFNC) can reveal how
brain structure relates to changing functional states. When the same structural latent representation is coupled with multiple states, applying independent vector analysis (IVA) separately to each state can produce unrelated structural decompositions, while forcing identical decompositions may suppress state-specific relationships. In addition, not every subject expresses every dynamic state. We propose masked structural residual IVA (MSR-IVA), a state-aware framework that combines a shared structural representation with state-specific residual adaptations and masks for incomplete state expression. On an Alzheimer's Disease Neuroimaging Initiative cohort, MSR-IVA improved matched source coupling by 6.5% and reduced unmatched dependence by 15.7% relative to the independent pairwise IVA baseline. Among subjects expressing both states, mean absolute cross-state structural source correlation was 0.9177 for MSR-IVA versus 0.2978 for no sharing, demonstrating controlled structural sharing that preserves source correspondence while allowing state-specific adaptation.
Read the full abstract ▾
All authors Victor Solomon, Zening Fu, Rafal Angryk, Vince D. Calhoun, Jingyu Liu
Machine Learning · Cross-listed from cs.CR
Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
Minh Tran, Cuong Dang, Tuc Nguyen et al. (+10 more)
Retrieval-Augmented Generation (RAG) enhances large
language models by grounding outputs in external knowledge, improving factuality and reducing
hallucinations. At the same time, the
retrieval-augmented pipeline introduces new robustness and security risks, including corpus poisoning,
backdoor attacks,
privacy leakage, and
fairness violations. Despite rapid progress in this area, existing surveys remain limited in their treatment of attacker objectives, threat models, and stage-specific defenses across the full RAG pipeline. This survey presents a unified and pipeline-aware overview of RAG robustness. We formalize threat models over the corpus, retriever, and generator, and organize attacks into three main objectives: accuracy,
privacy, and
fairness. We further review defenses from a pipeline-aware perspective, covering the
retrieval, rerank, generation, and traceback stages. In addition, we summarize robustness
benchmarks and explainability methods for more deeply evaluating and explaining RAG robustness.
Read the full abstract ▾
All authors Minh Tran, Cuong Dang, Tuc Nguyen, Khanh-Tung Tran, Minh Huynh Nguyen, Trinh Chau, Kien Le, Do Xuan Long, Jiahao Zhang, Fali Wang, Hoang D. Nguyen, Thanh Le, Suhang Wang
Machine Learning
Clearing the Underbrush: AI-Enhanced RF Interference Suppression
Rahul Jain, Pierre Trepagnier, Rick Gentile et al. (+2 more)
AI-based structured interference rejection has grown more popular because deep learning approaches can outperform traditional methods by jointly considering the signal of interest (SOI) and the signal mixture (SOI plus interference). This work builds on a previous AI-enabled approach utilizing autoregressive
transformer-based models by adding a Finite Scalar Quantization (FSQ) tokenizer layer which aims to improve the interference rejection performance while keeping overall latency to a minimum. Additionally, we experiment with other inference optimization techniques with the goal of speeding up inference without much accuracy loss. We explore this space with an experiment where the SOI is a digitally modulated
radio frequency (RF) signal and the structured interference is a digital television signal, an extremely common type of Orthogonal Frequency-Division Multiplexing (OFDM) transmission. Our results achieve low latency and increased interference rejection over traditional techniques and prior work with other AI-enabled methods. We demonstrate the benefits of the AI-enabled approaches via audio metrics such as Perceptual Evaluation of
Speech Quality (PESQ). Additionally, we explore a variety of applications and detail how our interference rejection
algorithm may be used in operationally-relevant scenarios.
Read the full abstract ▾
All authors Rahul Jain, Pierre Trepagnier, Rick Gentile, Joey Botero, Alexia Schulz
Machine Learning
Resource-Efficient Pruning for Transformer via Low-Rank Importance Estimation
Peng Liu, Huibing Zeng, Yiqun Zhang et al. (+2 more)
With the rapid development of large-scale pre-trained
language models based on
Transformer architectures, their high computational and
memory costs have become a major obstacle to deployment, especially in resource-constrained environments. Traditional
pruning methods typically depend on full gradient-based importance estimation, and they necessitate prior finetuning of the model to achieve satisfactory performance. This process often results in intolerable resource consumption. This paper proposes REP-LIE, a new approach to enable resource-efficient
pruning during the process of finetuning. REP-LIE leverages the gradients of LoRA low-rank
matrices to estimate the importance of weights without requiring full gradient computation. To address the inherent randomness in importance estimation, a stability score is introduced, serving as the basis for iterative
pruning of unimportant model parameters. The pruned model is further finetuned through lightweight updates, eliminating the need for full-parameter optimization in the process of finetuning. Extensive experiments on both medium-scale encoder models and large-scale generative models (LLaMA-7B and Mistral-7B) demonstrate that REP-LIE still achieves competitive performance compared to existing approaches.
Read the full abstract ▾
All authors Peng Liu, Huibing Zeng, Yiqun Zhang, Yang Yi, Jigang Wu
Machine Learning · Cross-listed from cs.DS
Liangchen Ge
We study the Lipschitz stability of
attention retrieval in
hyperbolic spaces. Existing methods lack deterministic guarantees on
attention-weight preservation under finite-precision representations. We introduce HCC+, a theoretical framework exploiting three properties of the Poincaré ball: exponential volume growth enabling query-independent boundary truncation; logarithmic covering radius of
hyperbolic 1-centers enabling dimension-independent critical-key identification; and a packing bound with constants independent of the
embedding dimension. We prove two deterministic guarantees: for exact
retrieval, the per-layer
attention deviation is bounded by 10% of its ideal value; for soft
attention, the total variation distance decays as $O(1/\sqrt{n})$, the rate of finite-sample variance. As a consequence of the guarding mechanism, the framework achieves a storage reduction factor of $6.1\times$ relative to FP16. We provide the first deterministic, query-independent
retrieval certificate in non-Euclidean geometry.
Read the full abstract ▾
Machine Learning
Physics-Informed Error Field Learning: A Post-Training Optimization Framework for Physics-Informed Neural Networks
Jiuyun Sun and Yong Zhang
Physics-Informed Neural Networks (PINNs) have emerged as an important class of numerical methods for solving partial differential equations (PDEs). However, during the late-stage optimization process, further parameter updates often yield diminishing accuracy improvements while increasing computational costs. To address this issue, this paper proposes a Physics-Informed Error Field Learning (PIEFL) framework for PINNs. Unlike conventional approaches that continuously approximate the solution field using a single network, PIEFL introduces an auxiliary error network after the primary network achieves satisfactory accuracy and shifts the learning objective from the solution field to the error field. By deriving error control equations under physical constraints, the error network learns the discrepancy between the current approximation and the exact solution, and the learned error correction is combined with the primary prediction to improve solution accuracy. The proposed framework avoids continuous optimization of the entire solution space and focuses computational resources on correcting existing prediction errors. Moreover, PIEFL requires no modification to the primary network architecture, making it compatible with existing PINN models and applicable as a general post-training optimization strategy. Numerical experiments on representative PDEs demonstrate that PIEFL achieves higher solution accuracy under the same computational budget, validating its effectiveness in improving the performance of PINNs.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Common-Center Geometry and Certified Radial Reconstruction for Energy-Form Full Conformal Regions
Yiheng Feng
This note studies the geometry of full conformal prediction (FullCP) regions generated by an empirical energy-form pairwise score. Candidate-score convexity alone does not guarantee connected FullCP regions, even for empirical averages of losses convex in the candidate argument. For the energy-form score, each leave-one-out training comparison reduces exactly to a pairwise-dissimilarity sublevel condition. Under
symmetry, a constant diagonal, a diagonal lower bound, and attainment of the associated Fréchet-type objective, all comparison regions share a minimizer; if they are convex, every nontrivial exact conformal region is
star-shaped about that point. For power distances $\rho_\beta(x,y)=\|x-y\|^\beta$, this geometry holds for $\beta\ge1$, while the conventional energy score is strictly proper for $0<\beta<2$. For $d=1,\beta=1$, every nontrivial empirical-CRPS FullCP region is a nonempty closed interval (possibly $\mathbb R$ when $m=1$). For $1<\beta<2$ and $m\ge2$, explicit data-checkable derivative bounds give Lipschitz control of the radial exits and exact conformal radial function. Combined with directional root search and classical Lipschitz extensions, they yield certified inner and outer radial envelopes of width at most $\delta+2\widetilde Lh_{\mathcal U}$ and same-ray Hausdorff guarantees. An analytic two-dimensional example shows why preserving
star-shaped but nonconvex geometry can matter. A staged two-dimensional study finds modest but systematic tightening of the generic certificate and frequent robust nonconvexity witnesses, with detected normalized radial departures typically small. The method is intended for low-dimensional multivariate outputs rather than high-dimensional scaling or runtime improvement.
Read the full abstract ▾
Machine Learning
Why and When Neural Networks Improve Local Approximation in Optimization
Chengkuo Bian and Pengcheng Xie
Published experience with neural surrogates in derivative-free optimisation is contradictory: the same family of models that cuts the evaluation count of one solver leaves another unchanged, or makes it worse. We show that the contradiction dissolves once three factors are stated, and that these, rather than the fit accuracy a training curve reports, are what delimit when a learned local model pays. Role: a surrogate that proposes candidates the true objective must still approve helps, while one that replaces a gradient the solver depends on hurts. Radius: a model fitted to an optimisation path is reliable only inside a bounded neighbourhood, and its error neither vanishes as that neighbourhood shrinks nor survives its growth. Room: a surrogate can only accelerate progress the base method is still able to make. We formalise radius-aware local generalisation, relate it to the classical fully linear condition, and test each factor with the surrogate class, training pipeline and base method held fixed. Over 117
benchmark instances safeguarded assistance raises the instances solved to high accuracy from 67 to 84 while gradient replacement lowers them to 65; removing the gradient term from the training loss cuts surrogate acceptance from 0.703 to 0.148; and 1000 paired comparisons over ten noise levels show no noise threshold, only a base method that stops early. The same factors bound the gain: a model-based trust-region solver, which leaves little room, drops from 88 to 86 when the identical surrogate is attached, and released interpolation software stays ahead at 103, and on a Monte-Carlo inventory model repairing the acceptance interface is worth 10.40 cost units against 0.00 for the surrogate.
Read the full abstract ▾