Machine Learning · Cross-listed from econ.GN
Forecasting Weather-Driven Price Dynamics Across Sri Lankan Tea Market Catalogues
Hesandi Mallawarachchi, Senilka Madurapperumage, Nadil Kulathunge et al. (+6 more)
The Colombo Tea Auction (CTA) plays a vital role in determining global tea prices, yet the relationship between local weather conditions and price behavior across different tea catalogues has not been thoroughly explored. In this study, we develop a novel, structured
dataset by extracting information from 105 weekly broker reports spanning late 2023 to 2026, and combined with region-specific weather data. Our analysis focuses on four main tea catalogues of Sri Lankan tea: High Grown, Low Grown, Off-Grade, and
Dust. To better understand the factors influencing tea prices, we apply Granger causality analysis alongside tree-based machine learning models: Random Forest, XGBoost, LightGBM, and Gradient Boosting. Our results show that while
market dynamics are primary drivers, weather conditions also have significant effects. Notably, Low Grown tea shows strong sensitivity to precipitation and sunshine duration (p<0.05) across 1-3-week lags. Off-Grade and
Dust catalogues also exhibit significant responses to temperature variations. Catalogue-specific modelling outperformed unified approaches, with LightGBM emerging as the superior model for three out of four catalogues. Overall, this study highlights the importance of considering both localized weather patterns and catalogue-level differences when
forecasting tea prices, offering a more precise and practical framework for the tea industry.
Read the full abstract ▾
All authors Hesandi Mallawarachchi, Senilka Madurapperumage, Nadil Kulathunge, Thilokya Angeesa, Nethsith Gunaweera, Sandeepa Weerasekara, Patalee Narasinghe, Nisansa de Silva, Sandareka Wickramanayake
Machine Learning
Data Predictability Shapes Weibull Weight-Scale Growth in Transformer Training
Tiexin Ding
A trained
transformer's weight magnitudes can be summarized by a two-parameter Weibull distribution whose shape $k \approx 1.2$ is stable across layers and models, so the scale $\lambda$ carries most training-induced movement. What corpus property sets how much $\lambda$ grows? Using the bigram conditional
entropy $D = H(\text{next} \mid \text{prev})$, a training-free statistic computed before training, we find across controlled corruption families a learning-rate-conditioned law, $\lambda^2 - \lambda_0^2 = C_0(\eta) + C_1(\eta)(H_r - D)^{0.59}$, where $H_r$ is a matched-budget shuffle baseline. The convex exponent is inherited from an independently measured data-side saturation relation rather than fitted directly to the growth curve. After removing the two per-$\eta$ coefficients, 23 runs spanning an order of magnitude in learning rate collapse onto $(H_r - D)^{0.59}$ with unit slope ($R^2 = 0.941$; direct per-$\eta$ fits are weaker, $R^2 \approx 0.82$). Because $D$ is computed before training, the law is a forward predictor: an end-to-end self-validation recovers held-out within-family weight growth with 5.7% relative error. The readout holds at model and per-layer resolutions and across two tested architectures, with the functional form preserved and only the coefficients changing. It also marks its boundary: cross-corpus prediction over-predicts code, implicating redundancy as a second axis of a broader $\Phi(D,R,A,H)$ data-to-weight framework.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Modular Cognitive Architecture Emerges in Large Language Models
Pengrui Han, Jacob Andreas, Evelina Fedorenko et al. (+1 more)
The human
brain exhibits a striking degree of functional specialization, with distinct networks supporting
language, formal
reasoning,
reasoning about other minds, and
reasoning about the physical world. Is this modular organization a fundamental principle of how intelligent systems must be built, or an evolutionary accident specific to biological
brains? Here, we test whether a similar organization emerges in Large
Language Models–another class of intelligent systems created through a very different optimization process. Using
circuit analyses across N=46 tasks spanning four cognitive domains (
language, formal
reasoning, social
reasoning, physical
reasoning), we find that LLMs develop a modular architecture that mirrors the human
brain: tasks drawing on the same network in humans recruit overlapping
neurons in LLMs, whereas tasks drawing on different networks recruit distinct
neurons. The convergent emergence of modularity in
brains and neural networks suggests that it may be a fundamental property of intelligent systems.
Read the full abstract ▾
All authors Pengrui Han, Jacob Andreas, Evelina Fedorenko, Andrea Gregor de Varda
Machine Learning · Cross-listed from econ.GN
Monsoon Mayhem to Market Waves: Forecasting Fisheries Resilience in Sri Lanka
Ruzaini Ahmed, Yohan Jayasinghe, Tharumini Gamage et al. (+6 more)
Sri Lanka's fisheries sector is important for jobs and food supply. Between 2019 and 2025, it faced several major problems at the same time, and how these events together affected fish production and prices is still not well understood. This study develops a framework to connect weather changes, major disruption events, fish production, and prices, with the goal of helping policymakers, traders, and supply chain managers make better decisions. Seasonal patterns are studied using STL decomposition. Spearman lag correlation is used to find delayed effects of
climate on production. Interrupted Time Series (ITS) regression measures the impact of major events. SARIMAX models predict monthly production and prices. Hotspot detection identifies unusual patterns. The results show that marine and inland fisheries behave differently in terms of seasons and
climate effects. Major disruptions caused different levels of impact, and in some cases, one sector helped compensate for another. These findings can support better planning, for example, improving infrastructure in high-risk areas, strengthening cold storage systems, and using early warning alerts for unusual events. Price
forecasting tools should be used as decision-support tools, not as direct
market signals.
Read the full abstract ▾
All authors Ruzaini Ahmed, Yohan Jayasinghe, Tharumini Gamage, Ifaz Ikram, Hasini Lawanya, Nirasha Munasinghe, Patalee Narasinghe, Nisansa de Silva, Sandareka Wickramanayake
Machine Learning · Cross-listed from cs.IR
Yuming Liu, Hongye Yang, Harrison Zhao et al. (+5 more)
We present NEXT (Next-interest EXploration
Transformer), a
reasoning-driven
video recommendation framework that reasons over the
video a user has just watched, infers the viewer's next intent, and retrieves concrete follow-up
videos. Explicit continuations such as episodes are linked directly; implicit cases are handled by generating intent queries and searching for matching candidates. This Item-to-Intent-to-Item formulation produces directed recommendations beyond co-engagement correlation or semantic similarity. To make this framework reliable at scale, we train NEXT-8B, a purpose-trained 8B
vision-
language model with a three-stage recipe: Perception-Enhanced Reinforcement Learning for query-agnostic evidence extraction, Distribution-Aligned Supervised Fine-Tuning over real and synthetic visual QA mixtures, and Group Relative Policy Optimization for last-mile
alignment. NEXT-8B achieves the best single-model DocVQA performance, ranking second overall only behind a multi-
agent system while surpassing a substantially larger 200B+ scale model, and improves next-intent logic-wise quality by 3.3% over the base model in a task-specific LLM-as-a-judge evaluation. We deploy NEXT as an additional
retrieval path in a large-scale social media recommendation system and observe statistically significant production gains, including +0.53% watch time and +0.51% distinct
video exposure. Overall, NEXT shows that a carefully trained compact
vision-
language model can serve as a practical
reasoning engine for next-interest exploration at production scale.
Read the full abstract ▾
All authors Yuming Liu, Hongye Yang, Harrison Zhao, Ellie Zhu, Bokai Cao, Lei Huang, Lizhu Zhang, Xiangjun Fan
Machine Learning
Feature Generation Using LLMs: An Evolutionary Algorithm Approach
Aria Nourbakhsh, Benoît Alcaraz and Christoph Schommer
A crucial step in machine learning pipelines is to present each entity with features or attributes that are representative of the characteristics of the processed entities. Feature engineering is an important step in finding a relation among attributes that otherwise may not be processed by the ML
algorithms. Meanwhile, Large
Language Models have shown promising abilities in coding, mathematical
reasoning, and processing world knowledge. In this work, we utilize an LLM for the problem of feature generation from tabular data based on the previously given features. We have created a pipeline that takes a set of attributes and a
prompt to generate new features. Then, our selection
algorithm selects the best-performing sets of attributes. We apply our method to eight
datasets from different domains and data types. Our results show that, in most cases, the
language model can produce new features based on mathematical and logical operators that are useful for the given tasks and can improve classification results.
Read the full abstract ▾
Machine Learning
More Than Memory: Task-Conditioned Signed FFN Writes in Long-Context Retrieval
Zhibo Yang
FFNs are often treated as parametric
memories. In long-context
retrieval, however, the sharper question is not only what they store, but whether their native residual writes push the current
retrieval state toward or away from the correct answer. We test this by scaling the model's own FFN write one layer at a time, without editing weights or injecting external steering vectors. Across controlled literal and semantic
retrieval suites, native FFN response surfaces are signed, layer-specific, and task-conditioned: the final FFN is a suppressor in 7 of 8 model-suite cases, and 60% of layers switch role between
retrieval modes (95% CI [50%, 69%]). A local directional derivative along the native write separates the two monotone roles: suppressors have negative derivative in 34/35 cases, and amplifiers have positive derivative in 18/18 cases, so the roles are not reducible to write size. On a safety-filtered LongBench
retrieval-QA probe, the same diagnostic predicts attenuation damage with raw R^2=0.796 on Qwen2.5-7B and 0.791 on Qwen3.5-9B; a held-out suppressor-attenuation policy improves
retrieval margins over random and norm-matched controls. These results show that native FFN scaling exposes a signed, task-conditioned residual-write structure in
retrieval, and that write-gradient
alignment is a compact diagnostic for the two monotone roles.
Read the full abstract ▾
Machine Learning
Comprehensive Evaluation of Machine Learning for Type 2 Diabetes Risk Prediction: Large-Scale External Validation and Fairness Analysis
Rajveer Singh Pall, Sameer Yadav, Siddharth Bhalerao et al. (+3 more)
Machine learning-based Type 2 diabetes risk prediction models obtain good internal validation results but lose effectiveness in real-world applications due to deficient external testing and
fairness assessment. We developed a multi-dimensional framework evaluating discrimination, calibration, interpretability, and algorithmic
fairness on nationally representative populations. An XGBoost model was trained on NHANES 2015-2020 (n=15,685) using eight non-laboratory predictors: age, sex, race/ethnicity, BMI, smoking status, physical activity, history of heart attack, and history of stroke. External validation was performed on BRFSS 2020-2022 (n=1,285,783) under realistic distribution shift. Internal validation showed good discrimination (AUC=0.794, 95% CI 0.788-0.800), with performance loss on external validation (AUC=0.717, relative decrease: -9.7%, p<0.001).
Fairness analysis revealed severe bias: elderly adults (>=60) showed AUC=0.607 vs 0.742 for young adults (difference=0.135, p<0.001); obese individuals showed AUC=0.698 vs 0.735 for normal weight (difference=0.037, p<0.001). Gender showed comparable performance (male=0.723 vs female=0.712, p=0.142). Calibration revealed risk overestimation (Brier score=0.123). SHAP analysis identified age, BMI, and physical activity as primary risk drivers. Populations with highest diabetes risk receive the worst algorithmic performance, underscoring the need for
fairness-aware, age-stratified deployment strategies before clinical use.
Read the full abstract ▾
All authors Rajveer Singh Pall, Sameer Yadav, Siddharth Bhalerao, Sourabh Sahu, Ritu Ahluwalia, Bhaskar Awadhiya
Machine Learning
SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling
Yupeng Chang, Yuan Wu and Yi Chang
Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning (PEFT) method for large
language models. Under a fixed rank budget, LoRA parameterizes each adapted weight through a single low-dimensional input-side pathway, which may couple heterogeneous behaviors through shared input directions and induce interference during optimization. We propose Static Orthogonal Subspace LoRA (SOS-LoRA), a drop-in extension that reparameterizes a rank-rtot update as a sum of K static (always-on, non-routed) low-rank experts. SOS-LoRA (i) decomposes the total rank across experts, (ii) applies a fixed multi-scale scaling scheme to encourage scale-separated optimization dynamics, and (iii) promotes diverse input-side directions via cross-expert orthogonal initialization and a lightweight regularizer. SOS-LoRA remains fully mergeable, adding no inference-time parameters or latency after merging. Experiments on
reasoning and knowledge-intensive
benchmarks (Llama 2/3), encoder-based NLU (GLUE), and math
reasoning (GSM8K/MATH) show consistent gains over matched-budget LoRA baselines and recent variants. Code is available at https://github.com/llm172/sos-lora.
Read the full abstract ▾
Machine Learning
Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
Yutong Feng, Shiyuan Piao, Yutong Xia et al. (+5 more)
Spatio-Temporal Foundation Models (STFMs) aim to learn generalizable representations of complex dynamical systems across space and time. However, existing approaches suffer from distributional bias in real-world pre-training data, structural bottlenecks of autoregressive or
diffusion-based paradigms, and objectives that overemphasize point-wise reconstruction in noisy observation space.We propose NeoST, the first spatio-temporal foundation model pre-trained solely on procedurally generated synthetic systems. NeoST introduces a scalable synthetic pre-training corpus to mitigate real-world bias, a latent-space
reasoning architecture that generates and iteratively refines multiple future trajectories without sequential error accumulation, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.Extensive experiments across diverse real-world
benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-world spatio-temporal systems, achieves superior long-horizon stability and inference efficiency.
Read the full abstract ▾
All authors Yutong Feng, Shiyuan Piao, Yutong Xia, Xu Liu, Wenqi Fan, Fugee Tsung, See-Kiong Ng, Yuxuan Liang
Machine Learning
Benchmarking Machine Learning Models for Multi-Omics-Based Breast Cancer Prediction
Priyanka Paudel and Madan Baduwal
Estrogen Receptor (ER) status is a critical biomarker in breast cancer diagnosis, prognosis, and treatment selection. Recent advances in high-throughput sequencing technologies have enabled the generation of multi-omics
datasets that provide complementary molecular information for computational prediction tasks. This study presents a systematic benchmarking analysis of classical machine learning models for ER status prediction using transcriptomic (RNA expression), genomic (copy number variation; CNV), and proteomic (RPPA) data from the TCGA-BRCA cohort. A rigorous experimental framework incorporating stratified train-test splitting, stratified five-fold cross-validation, class imbalance handling, and fold-specific feature selection was employed to ensure reliable evaluation and prevent data leakage. Random Forest, XGBoost, LightGBM, CatBoost, Support Vector Machines (SVM), and Logistic Regression were evaluated across single-omic and multi-omic settings. Results demonstrated that RNA expression provided the strongest predictive signal, while multi-omic integration yielded modest but consistent improvements over individual modalities. Among all evaluated approaches, Random Forest achieved the best overall performance in the integrated multi-omic setting, obtaining a balanced accuracy of 90.3% and an ROC-AUC of 97.1%. Furthermore, recurrent selection of biologically relevant genes, including ESR1, PGR, FOXA1, and GATA3, supported the biological validity of the learned models. These findings indicate that carefully regularized classical machine learning methods remain highly effective for small, high-dimensional genomic
datasets and that multi-omic integration provides complementary information for breast cancer ER status prediction.
Read the full abstract ▾
Machine Learning
High-accuracy Low-Bit KV-Cache Quantization via Local Distribution Restoration
Gradwell Dzikanyanga, Yanqi Pan, Weihao Yang et al. (+3 more)
Long-context large
language model inference relies on the KV cache to avoid redundant
attention computation, but incurs high
memory and bandwidth overheads. Low-bit KV-cache quantization reduces this cost, yet it severely degrade quality; particularly, one-bit quantization reduces accuracy from 84.2% to 47.8% on Llama-3.1-8B under RULER. Rather than common beliefs that absolute error of logits, we find that the root cause is structured local misranking, where the distribution of logits in top-K region is drifted. We thereby propose local distribution restoration, a new technique that detects steps with high local distribution risk from quantized-logit features and restores only the selected top-K candidate distribution before
token selection. We implement DGAP to achieve local distribution restoration, with efficient risk detcetors and correctors. Expeirments show that on Llama-3.1-8B, DGAP recovers K1V1 RULER accuracy from 47.8% to 83.2% and reduces distribution drift from 0.38 to 0.14; across Llama, Mistral, and Qwen models, it preserves the persistent low-bit KV-cache footprint with modest decode overhead.
Read the full abstract ▾
All authors Gradwell Dzikanyanga, Yanqi Pan, Weihao Yang, Donglei Wu, Wen Xia, Hao Huang
Machine Learning
Uncertainty-Aware Sequential Decision Rules for Event-Triggered LLM Invocation in Streaming Systems
Zhaohui Wang
Streaming inference pipelines increasingly pair lightweight fast models with Large
Language Models (LLMs) that provide rich semantic understanding at substantial cost. The central question of when to invoke the LLM has received limited formal treatment. We cast this as a risk-based sequential stopping problem, where a trigger policy fires when a risk functional over the observation history exceeds a threshold. Within this framework, we prove six results: a minimum inter-event time bound excluding trigger chattering; optimality of threshold policies via smooth pasting; approximate SPRT guarantees under estimated parameters; O(sqrt(T log T)) regret for stationary streams, extending to O(sqrt((C_T + 1) T log T)) under C_T changepoints; O(1/sqrt(T)) convergence of online gradient descent for adaptive thresholds; and a calibration-to-miss-rate transfer inequality. Several classical trigger families, including event-triggered, optimal stopping, SPRT, CUSUM, and
Bayesian triggers, can be expressed as special cases of this framework. On turbofan degradation data (CMAPSS) with real LLM calls, we empirically verify the theoretical assumptions, ablate the risk function design, compare against six baselines including a RouteLLM-style router and contextual bandits, and analyze cost sensitivity and LLM failure modes. The results confirm sublinear regret, with alpha < 1 for all principled triggers; high diagnostic quality, with 92.9 percent of 1600 LLM diagnoses reaching grounding score >= 0.75 under our rubric; and that anomaly-score-driven risk functions dominate alternatives by roughly an order of magnitude on the Pareto AUC.
Read the full abstract ▾
Machine Learning
A Novel Latent-Class Attack and its Detection by Class Subspace Orthogonalization
Guangmingmei Yang, David J. Miller and George Kesidis
Deep learning, which in general relies on voluminous amounts of training data, is vulnerable to data poisoning attacks, including error-generic attacks and
backdoors (Trojans). In this work, we propose a new data poisoning attack we dub a latent class attack. Here, all poisoned examples are from a class that is novel (unknown) for the given classification domain and are mislabeled to one of the known classes (the target class) of the domain, so that the model learns to recognize the novel class as a sub-class of the target class. Such attacks could be used e.g. to defeat AI-based access control systems, or could cause a "foe" to be classified as a "friend". We also propose a post-training defense to detect this attack, without any access to the training set. This detection approach builds on "class subspace orthogonalization" (CSO), a plug-and-play paradigm demonstrated to improve existing
backdoor detectors. Here, CSO is used to seek an input (a putative unknown class instance) whose internal representation is not aligned with any of the known classes, and yet which is classified with confidence to one of these classes. Finally, specific to image classification domains, we propose a method for visualizing the estimated unknown class instance, providing explainability to our latent class detections.
Read the full abstract ▾
Machine Learning
Few-Step Boltzmann Generators via Scalable Likelihood Flow Maps
RuiKang OuYang, Hanlin Yu, Xinyue Ai et al. (+7 more)
Recent progress in flow-based generative modeling has led to models that output high-quality samples while using only a small number of function evaluations. However, at present, there is a lack of similar advances in estimating the model likelihood. In particular, most existing methods either rely on restrictive architectures that enable exact calculations, or use
stochastic approximations such as Hutchinson's trace
estimator that introduce substantial variance. In this work, we introduce SCAlable LikeLihood
distillation of flOw maPs (SCALLOP). SCALLOP builds on the recently proposed F2D2, a likelihood flow map model that can generate samples and their densities in a small number of function evaluations. While F2D2 uses Hutchinson's
estimator during training, we introduce an alternative and more scalable likelihood
distillation objective that is Hutchinson-free and admits a vectorized formulation. Empirically, we demonstrate the effectiveness of SCALLOP as a Boltzmann generator in molecular science, and further validate its benefit on image
datasets. SCALLOP significantly reduces both training variance and training time while consistently improving performance compared to F2D2, and is competitive with the state-of-the-art while achieving up to 10x inference speedup over the fastest baseline.
Read the full abstract ▾
All authors RuiKang OuYang, Hanlin Yu, Xinyue Ai, Yutong He, Nicholas M. Boffi, Pradeep Ravikumar, Jose Miguel Hernandez-Lobato, Max Simchowitz, Benjamin Kurt Miller, Omar Chehab
Machine Learning · Cross-listed from cs.CV
BTI-Net: Bidirectional Decoder-Level Task Interaction via Uncertainty-Aware Gating for Multi-Task Medical Image Analysis
Abdullah Al Shafi, Md Kawsar Mahmud Khan Zunayed, Safin Ahmmed et al. (+2 more)
Jointly learning to segment and classify medical images demands cross-task synergy, yet encoder-sharing architectures limit decoder reconstruction to task-private representations, permanently discarding the boundary cues and semantic priors each branch could supply to the other. This work introduces BTI-Net, which establishes bidirectional communication at every decoder level through two parallel pathways via Task Interaction Modules (TIM). Spatial boundary context is gated into the classification branch, while global semantic priors multiplicatively modulate the decoder, with refined features propagating progressively from coarse semantics to fine boundary detail across all four decoder resolutions. Since cross-task interaction is not equally reliable for every input, Uncertainty Proxy
Attention (UPA) gates each TIM output per instance and per level using three signals that capture cross-task
alignment, scene complexity, and prediction confidence, without external annotations or additional inference passes. Experiments on three medical
benchmarks spanning ultrasound, dermoscopy, and
brain MRI demonstrate consistent improvements in
segmentation IoU and classification accuracy over both encoder-sharing and decoder-interaction baselines. Ablation confirms adaptive gating contributes +2.36 IoU over fixed bidirectional interaction, and classification accuracy improves by up to +2.26 points over the strongest multi-task baseline. UPA's uncertainty proxies serve as reliable single-pass task-failure signals without the overhead of
stochastic sampling. Code: https://github.com/C-loud-Nine/BTI-Net_MTL
Read the full abstract ▾
All authors Abdullah Al Shafi, Md Kawsar Mahmud Khan Zunayed, Safin Ahmmed, Sk Imran Hossain, Engelbert Mephu Nguifo
Machine Learning · Cross-listed from stat.ML
Connectivity Estimation using Stochastic Graph Heat Modelling
Stephan Goerttler, Min Wu and Fei He
A growing number of techniques leverage the spatial structures that underlie many real-world
datasets. Despite these advances, the complementary task of estimating spatial structures and understanding their role within these techniques has often been overlooked. In neurophysiological data analysis specifically, numerous methods exist to estimate
brain connectivity, but most are not explicitly model-based, dynamic, multivariate, or directed. To address these limitations, we previously introduced noise-driven heat modelling on graphs for neurophysiological connectivity estimation. In this study, we extend this framework by relaxing earlier noise assumptions and adding regularisation to improve robustness. We also develop a simulation procedure to characterise and evaluate our technique in a controlled setting. Finally, we demonstrate that the technique is able to capture meaningful spatial structure across two experiments, each using two real-world
datasets. The explicit model formulation of our connectivity
estimator has the potential to improve the interpretability of graph-based techniques across a wide range of applications. The code implementing our method is available at https://github.com/sgoerttler/Heat_Connectivity.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
HorizonRelight: Relighting Long-horizon Videos Consistently via Diffusion Transformers
Jing Yang, Mayoore Jaiswal, Zian Wang et al. (+4 more)
Diffusion-based
video relighting enables controllable relighting from a single input
video, but modern
video diffusion backbones are trained on short clips and applied to long-horizon
videos through chunked sliding-window inference, often causing temporal discontinuities at chunk boundaries. We address this by reframing long-horizon relighting as temporally conditioned latent domain translation. Our framework enforces cross-chunk continuity by propagating target-domain latents across boundaries and makes this behavior learnable using masked target-domain self-conditioning, training the model to continue from temporally masked propagated context. We further introduce warm-start prompting with a relit
prompt anchor from a controllable generative model, which establishes the initial target-domain state and creates a general interface for
prompt-based relighting. Experiments on in-the-wild long-horizon
videos show markedly improved temporal consistency, with chunk-boundary artifacts largely reduced and unwanted appearance changes across chunks greatly suppressed.
Read the full abstract ▾
All authors Jing Yang, Mayoore Jaiswal, Zian Wang, Steven Zeng, Rochelle Pereira, Yajie Zhao, Jianyuan Min
Machine Learning
TeDiServe: High SLO Attainment Serving for Diffusion Language Models
Tzu-Tao Chang, Benjamin Yuanyang Hong, Kiet Pham et al. (+1 more)
Diffusion language models (DLMs) have recently emerged as a promising alternative to conventional autoregressive
language models. By generating multiple
tokens in parallel during each denoising step, they offer higher inference throughput while maintaining competitive quality. However, realizing these throughput gains while meeting latency SLOs in a serving system requires addressing challenges introduced by DLMs' unique characteristics. These include navigating the speed-quality tradeoff created by confidence-based denoising, choosing appropriate parallelization levels across model instances under fluctuating load, and coordinating approximate KV caching mechanisms that introduce non-uniform per-step costs. To address these challenges, we present TeDiServe, a cluster-level serving system for DLMs. TeDiServe enables deadline-aware scheduling and adaptive load control through confidence-threshold adjustment, and dynamically reconfigures the cluster by solving a quality-aware optimization problem, while explicitly modeling the step-level heterogeneity introduced by approximate KV caching. Across multiple
benchmarks and real-world traces, TeDiServe improves SLO attainment by up to 56.6 percentage points and reduces end-to-end request latency by up to 46% while incurring less than 1% accuracy drop.
Read the full abstract ▾
All authors Tzu-Tao Chang, Benjamin Yuanyang Hong, Kiet Pham, Shivaram Venkataraman
Machine Learning
Priced Motion Through Optimal Faces: A Normal-Fan Geometry for Non-Stationary Adversarial MDPs
Kai Hidajat
In a changing decision problem, standard dynamic-regret analyses have often equated the cost of non-stationarity to how far loss moves. However, it is simultaneously possible for a loss sequence to travel far and retain the same optimal policy, or for a small movement in loss to force the optimal policy to change completely. Thus, the size of the movement through loss variation, transition variation, or comparator path length describe the adversary's motion, but not the cost of that motion to the control problem. For a more faithful analytic interpretation, this paper develops a normal-fan geometry for finite-horizon
adversarial MDPs with fixed transitions. Occupancy measures form a polytope, and each loss vector exposes an optimal face of that polytope. Non-stationarity in rewards is therefore a path through the normal fan, where motion inside one cone leaves the optimal face unchanged, while crossing a wall may carry regret. We pose the notion of a face-crossing price, which is the minimum regret incurred by remaining on the previous optimal face under the new loss. For any learner that tracks the previous face, dynamic regret decomposes exactly into intrinsic priced face motion plus within-face selection error. The resulting theory separates consequential from harmless non-stationarity, where loss variation can be arbitrarily large at zero price, and identical one-coordinate variation can hide horizon-scale differences in regret.
Read the full abstract ▾
Machine Learning
Statistically Indistinguishable, Operationally Distinct: A Formal Barrier for Tabular Foundation Models
Tassilo Klein and Johannes Hoffart
Tabular foundation models cannot reason about data produced by running systems without access to the rules that govern them. We make this statement falsifiable. The Operational Turing Test (OTT) constructs pairs of legal and rule-violating database states whose $1$- and $2$-way column-value marginals match to a total variation of $<0.02$; Le~Cam's lemma then bounds any values-only classifier at $\geq0.49$ Bayes error. Three values-only baselines (XGBoost, TabICL, TabPFN) hit the bound exactly (accuracy $0.50$, pre-registered two one-sided tests (TOST) $p<0.002$), raw row-level access does not help, exposing relational value consistency closes most of the gap, and only a classifier fed by seven executable rule-derived audits reaches $1.00$ classification accuracy. In three matched $100$-state frontier large-
language-model (LLM) runs, models given the schema, trigger source, rule tables, and state files classify at most $2/50$ legal states as LEGAL; GPT-5.5 accepts $0/50$ legal states even with higher
reasoning effort and a Structured Query
Language (SQL) executor. The access-ladder pattern also appears on a second schema with structurally distinct rule families (banking ledger: cross-row balance, cumulative aggregate). The barrier is identifiability, not capacity: scale, data, and richer features cannot cross it without operational grounding.
Read the full abstract ▾
Machine Learning · Cross-listed from eess.IV
Complete virtual unwrapping and reading of a rolled Herculaneum papyrus
Giorgio Angelotti, Stephen Parsons, Federica Nicolardi et al. (+24 more)
The carbonized papyri from Herculaneum preserve the only large-scale library to survive from classical antiquity, but many unopened rolls remain unread because physical opening risks irreversible damage. X-ray computed microtomography ($\mu$CT) and virtual unwrapping offer a non-invasive route to their texts, yet previous work on sealed Herculaneum scrolls has recovered only localized readings or limited surface regions. Here, using high-resolution phase-contrast $\mu$CT acquired on the BM18 beamline at the European Synchrotron Radiation Facility (ESRF), together with improved computational unrolling and machine learning, we achieve the complete virtual unwrapping and reading of PHerc. 1667 under explicit coverage and papyrological-review criteria. This makes PHerc. 1667 the first Herculaneum papyrus to be fully digitally unrolled and read for extended scholarly study without physical opening. In PHerc. Paris 4, the optimized scan protocol makes ink directly visible in the tomographic volume, allowing three-dimensional ink
segmentation and independent validation of surface-conditioned ink recovery. In PHerc. 139, we recover title and author-attribution evidence identifying the scroll as Philodemus, On Gods, Book 8. These results move virtual unwrapping of the Herculaneum scrolls beyond isolated demonstrations towards a scalable framework for systematic recovery of the still-unopened library.
Read the full abstract ▾
All authors Giorgio Angelotti, Stephen Parsons, Federica Nicolardi, Youssef Nader, Sean Johnson, David Josey, Paul Henderson, Hendrik Schilling, Johannes Rudolph, Forrest McDonald, Elian Rafael Dal Prá, Paul Tafforeau, Alessandro Mirone, Clifford Seth Parker, Jan Paul Posma, Benjamin Kyles, Claudio Vergara, Alessia Lavorante, Rossella Villa, Maria Chiara Robustelli, Marzia D'Angelo, Gianluca Del Mastro, Michael McOsker, Kilian Fleischer, Christy Chapman, Nat Friedman, William Brent Seales
Machine Learning · Cross-listed from math.NA
Residual-Guided Dictionary Learning for Spectrally Accurate Koopman Approximation
George Coote and Matthew J. Colbrook
Koopman theory promises linear structure in nonlinear dynamics, but numerical Koopman spectra are easy to compute and hard to trust. A finite EDMD
matrix always has
eigenvalues; the problem is that many of them may have nothing to do with the infinite-dimensional operator. In this paper we make spectral reliability the objective of dictionary learning. We train neural-network dictionaries not merely to predict the next snapshot, but to minimize Residual Dynamic Mode Decomposition residuals: operator-level a posteriori errors that test whether computed
eigenvalues and modes are genuine Koopman spectral objects. To keep the learned observables from collapsing into an unstable coordinate system, the loss also penalizes the condition number of the lifted data
matrix. Thus the method couples two requirements that should not be separated: small Koopman residuals and a well-conditioned representation. The result is a learned dictionary that is expressive, numerically stable, and spectrally disciplined. Across conservative and dissipative
benchmark systems, the method sharply reduces spectral pollution, improves residual pseudospectral inclusion, and lowers forecast error relative to standard fixed dictionaries. On sea-surface temperature data, it gives cleaner Koopman diagnostics and substantially better one-step forecasts from noisy observations with no governing equations. The message is simple: neural Koopman learning should be judged not by prediction alone, but by whether its spectral claims can be certified. Residuals provide the certificate; conditioning makes it computable.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks
Young-Jun Lee, Seungone Kim, Minki Kang et al. (+5 more)
Would experience designing faster GPU kernels also help close in on a long-standing open mathematical
conjecture? Large
Language Models (LLMs) integrated into evolutionary search have recently produced state-of-the-art solutions on optimization tasks, including open mathematical
conjectures, GPU kernel design, scientific law discovery, and combinatorial puzzles. To achieve this, prior work applied search scaffolds to one target task at a time, so every new problem is approached from scratch and the experience accumulated during search is discarded once the model finishes its attempt. This leaves the capability of iteratively evolving a solution (e.g., knowing which part to mutate and how, deciding when to backtrack) entirely in the scaffold rather than in the model itself. Whether the model itself could acquire this capability and reuse it across different tasks has been largely unexamined. To address this, we introduce
Evolution Fine-Tuning (EFT), a mid-training paradigm that teaches LLMs to evolve solutions across tasks by converting evolutionary search trajectories into supervision. We construct Finch Collection, a 156K-trajectory
dataset spanning 10 domains and 371 optimization tasks, and fine-tune open-source LLMs from 2B to 9B parameters. Empirically, EFT confers cross-task generalization: across 22 held-out tasks, our models surpass their base counterparts by 10.22% on average. Furthermore, when paired with test-time RL, our model matches state-of-the-art performance on two circle-packing tasks and outperforms its base-model counterpart on the Erdős minimum-overlap problem. EFT thus serves as a "practice phase" for general-purpose discovery
agents that do not solve new problems from scratch.
Read the full abstract ▾
All authors Young-Jun Lee, Seungone Kim, Minki Kang, Alistair Cheong Liang Chuen, Zerui Chen, Seungho Han, Taehee Jung, Dongyeop Kang
Machine Learning · Cross-listed from cs.CL
A Comparative Study on Affective Cues in Text Embeddings Across Psychological Emotion Theories
Fabio Ciani, Harald Schweiger, Emilia Parada-Cabaleiro et al. (+1 more)
Text encoders are known for their utility in natural
language processing, as they are able to efficiently compress inputs into dense vectors while preserving semantics. These models have been applied to affective computing, in particular to help with solving sentiment analysis and
emotion recognition tasks. Nevertheless, it remains unclear to what extent the latent representations produced by modern text encoders capture well-defined psychological theories of affect. In this work, we investigate the affective capabilities of twelve recently released text encoders by probing their generated
embeddings as input features for solving regression and classification tasks across three established
emotion frameworks, using both word- and sentence-level data. Additionally, we apply a semantic data-leakage prevention technique to improve robustness in word-level evaluations. Our main findings show that the latent
manifolds of the latest instruction-aware open-weight encoders enclose an equal or even a larger amount of affective information in comparison with proprietary counterparts when evaluated at word level. In contrast,
embeddings of task-tuned and proprietary encoders reach the highest scores on sentence-level affective classification. Furthermore, a qualitative analysis of latent representations and their encoded affective cues is provided.
Read the full abstract ▾
All authors Fabio Ciani, Harald Schweiger, Emilia Parada-Cabaleiro, Markus Schedl
Machine Learning
When Can Conformal Risk Control Certify LLM Outputs? Bounds, Impossibility, and Adaptation for Structured Generation
Varun Kotte
Large
language models (LLMs) deployed for structured generation (NER, JSON extraction, QA, and classification) lack formal reliability guarantees, and standard heuristic abstention policies violate user-specified risk targets on 7.5–12.5% of settings, with no per-deployment guarantee. We characterize when conformal risk control (CRC) can certify structured LLM outputs and when it provably cannot. First, we prove a sharpened, attained feasibility frontier: when base risk \mu exceeds target \alpha, any distribution-free method must abstain on at least (\mu-\alpha)/(M-\alpha) of inputs, yielding a closed-form feasibility test that decides whether CRC can work before running it. Second, we establish a proven certification phase diagram across Hoeffding, empirical Bernstein, and a betting-based e-CRC bound, verified over 716 configurations (six open-weight models 3B–72B, eight
datasets, six scores): at strict targets (\alpha <= 0.20) the certified sets are nested (51/72/80 certified; Hoeffding-to-Bernstein the largest upgrade, +41%; e-CRC best under calibration scarcity), while at relaxed targets the Hoeffding-Bernstein ordering reverses at a closed-form frontier. Third, we show a negative shift result with a constructive counterpart: under cross-
dataset shift the target is violated on 14 of 16 transfers by static CRC and every tested adaptive-conformal-inference (ACI) step size, yet a full-feedback anytime-valid monitor certifies 0 of 16 while remaining non-vacuous. Relaxing the target to \alpha = 0.40 unlocks practical certification (28% NER, 13% QA, 19% CLS). The framework gives a three-step deployment recipe: check feasibility, select the bound and score, then re-check under shift.
Read the full abstract ▾
Machine Learning
A Kernel Fisher Discriminant Analysis-Based Tree Ensemble Classifier: KFDA Forest
Donghwan Kim, Seung Hwan Park and Jun-Geol Baek
In general, an ensemble classifier is more accurate than a single classifier. In this study, we propose an ensemble classifier called the kernel Fisher discriminant analysis forest (KFDA Forest), which is a tree-based ensemble method that applies KFDA. To promote diversity, bootstrap is used, and variable sets are randomly divided into K subsets. KFDA is performed on each subset to increase classification accuracy. KFDA maximizes the distance between classes while minimizing the distance within classes. KFDA can also be applied to classification problems in a nonlinear data structure using the kernel trick because it can transform the input space into a kernel feature space, commonly named a rotation, rather than performing a dimensionality reduction. Because new feature axes and KFDA projections are parallel, decision trees are used as a base classifier. To compare the proposed method with existing ensemble methods, we apply these to real
datasets from the UCI and KEEL repositories.
Read the full abstract ▾
Machine Learning
MOSAIC: Orchestrating Collaborative Knowledge Tracing with Hierarchical Semantic Alignment
Xinjin Li, Mengyue Wang, Yuzhen Lin et al. (+4 more)
Knowledge Tracing (KT) is important for personalized education but traditionally suffers from two key limitations: a reliance on shallow ID-based representations that neglect semantic depth and a restriction to single-granularity mastery estimation that overlooks hierarchical knowledge dependencies. To address these challenges, we propose MOSAIC (Multi-granularity Online Semantic AI for Collaborative Knowledge), a novel framework that orchestrates LLM-driven semantic
alignment with sequential modeling. Unlike methods that use LLMs solely as predictors, MOSAIC leverages a frozen LLM to generate dynamic, context-aware
embeddings and hierarchical prediction
prompts, explicitly capturing collaborative signals and peer interactions. Furthermore, we introduce a cross-granularity consistency objective that jointly regularizes mastery estimation across concept, topic-cluster, and global proficiency levels. Extensive experiments on ASSISTments, EdNet, and a newly collected large-scale MOOC
dataset demonstrate that MOSAIC establishes new state-of-the-art results. Specifically, our method achieves AUC improvements of up to 3.4% and Accuracy gains of up to 2.5 % across all
benchmarks. Notably, MOSAIC exhibits superior robustness in collaboration-rich environments and long-sequence scenarios (AUC 0.862 on MOOC), offering both high predictive precision and semantically grounded interpretability.
Read the full abstract ▾
All authors Xinjin Li, Mengyue Wang, Yuzhen Lin, Pengbin Feng, Ziqi Sha, Yeyang Zhou, Yu Ma
Machine Learning · Cross-listed from cs.CE
Weak Dominant Balance for Robust Identification of Dynamically Consistent Fluid Flow Structure
Samuel Ahnert, Esther Lagemann, H. Jane Bae et al. (+4 more)
Extracting interpretable, localized physical mechanisms from complex spatiotemporal data is a foundational challenge across physics, biology, and engineering, but has remained out of reach on real measurements. The central obstacle is obtaining high-quality gradients of data via numerical differentiation, which amplifies noise, diverges for high-order equations, and falters on irregular geometries, limiting the scope of existing approaches to clean simulations of low-order systems. Here, we present weak dominant balance, a derivative-free framework that projects governing equations into a weak (integral) formulation, offloading differentiation onto smooth analytical test functions and leaving the data untouched. The method sustains accurate regime identification under severe noise where existing approaches categorically fail, delivers the first data-driven decomposition of a third-order partial differential equation applied to turbulent duct flow, and produces matching decompositions across direct numerical simulation and particle-image velocimetry measurements of a wavy channel flow, uncovering a previously uncharacterized dynamical regime. Weak dominant balance brings mechanism-level analysis out of simulation and onto measured data, and opens complex physical systems to direct, equation-grounded interpretation.
Read the full abstract ▾
All authors Samuel Ahnert, Esther Lagemann, H. Jane Bae, Kunihiko Taira, Ricardo Vinuesa, Christian Lagemann, Steven L. Brunton