Machine Learning
SPERA: Spherical Prior EEG Foundation Model with Geometry- and Frequency-Aware Latent Prediction
Minsu Kim, Ye-Sung Kim, Hyeseong Jeon et al. (+3 more)
Electroencephalography (
EEG) provides a non-invasive measure of ongoing neural activity, but building general-purpose
EEG models remains challenging due to the heterogeneity of subjects, devices, and electrode montages. Existing
EEG foundation models predominantly rely on reconstruction-based objectives defined on the observed signal, which contains both neural and non-neural components. We introduce SPERA (Spherical Prior
EEG Representation Architecture), an
EEG foundation model that adopts the joint-
embedding predictive architecture (JEPA) to predict in latent space. SPERA introduces a Legendre-
polynomial spatial prior, incorporated into
attention to encode varying scalp electrode geometries. Two further components adapt the model to
EEG: factorized temporal and spatial
attention interleaved with periodic full-
attention blocks, and a relational spectral regularizer aligning latent similarity structure with spectral views. Pretrained on approximately 80,000 hours of
EEG from 29,048 subjects across 106
datasets, SPERA achieves the highest average balanced accuracy across nine downstream tasks spanning clinical, cognitive, and BCI applications. SPERA further exhibits strong parameter efficiency under linear probing and robustness across varying recording conditions, suggesting its potential as a general-purpose backbone for diverse
EEG analyses.
Read the full abstract ▾
All authors Minsu Kim, Ye-Sung Kim, Hyeseong Jeon, Wooseok Hyung, Joshua Lee, Chang-Hwan Im
Machine Learning
The Premise Is the Problem: Exchangeability Failure in Self-Monitored Test-Time Adaptation
Weijia Han, Lisha Qu, Zhenda Li et al. (+1 more)
Modern forecasting models are often updated after deployment so they can respond to changing data. These updates can also make predictions worse, so practical systems need a reliable monitor that can detect harmful changes and trigger protection. A natural design is to monitor the same prediction errors that guide the updates. This paper asks whether the statistical guarantee behind such a monitor remains valid when monitoring and adaptation use the same feedback. We study this question in multi-step time-series forecasting. We show that overlapping targets and dependence in forecast errors can break a key assumption required by the guarantee. The monitor may then raise alarms even when no harmful change has occurred, and its response can further damage prediction quality. We also find that adaptation can hide sustained changes from its own monitor, while the original frozen model retains a clearer signal. These results expose a basic failure mode in self-monitored adaptation. They show why reliable deployment requires checking the monitor's assumptions, comparing adaptation with the frozen model under realistic feedback, and limiting the effect of every protective response.
Read the full abstract ▾
All authors Weijia Han, Lisha Qu, Zhenda Li, Liying Liang
Machine Learning · Cross-listed from cs.AI
Inference-Time Projection for Physically Valid Biomolecular Diffusion Models
Qurat-ul-ain, Yee Whye Teh, Charlotte M. Deane et al. (+1 more)
AlphaFold 3-style cofolding models predict biomolecular complexes with high structural accuracy, yet a large fraction of their outputs are physically invalid: chains overlap at interfaces, ligand bond lengths and angles are distorted, rings are non-planar, and stereocentres are inverted. Current approaches either steer the sampler with physics-informed potentials, which multiplies sampling cost and
memory overhead making inference impossible on large complexes, or finetune the model, costing time and tying the fix to one architecture. We observe that, unlike structural accuracy, physical validity is fully verifiable at inference time from quantities the sampler already holds. We therefore treat physical validity as a constrained inference problem and introduce two closed-form projection operators applied to the
diffusion model's denoised clean-coordinate estimate, $\hat{x}_0$: an inter-chain van der Waals projection that pushes apart the most severely clashing atom pairs, and a ligand distance-geometry projection that restores bond lengths, angles, internal contacts, planarity and chirality. Both operators are local, sparse and displacement-capped, require no network evaluations, gradients or importance sampling, and leave the denoiser and its weights untouched, so they can be dropped into any AF3-style sampler without retraining. Applied to two independently developed models, Boltz-2 and OpenFold-3, across five
benchmarks (CASP15, CASP16, the PoseBusters monomer and complex sets, and the Boltz physical-validity test set), our method recovers perfect physical validity while preserving structural-accuracy and ligand-placement metrics. These gains are achieved with negligible runtime and
memory overhead, providing a practical, model-agnostic route to physically valid all-atom structure prediction.
Read the full abstract ▾
All authors Qurat-ul-ain, Yee Whye Teh, Charlotte M. Deane, Matteo Cagiada
Machine Learning · Cross-listed from cs.AI
JIVEAdapter: A Multi-Task Additive Low-Rank Adapter via Joint and Individual Variation Explained (JIVE)
Sara Abdali and Pashmina Cameron
Parameter-efficient fine-tuning adapts pretrained models at a fraction of the cost of full fine-tuning, yet most low-rank adapters are single-task and represent each weight update multiplicatively, leaving no explicit account of what is shared across tasks and what is task-specific. We introduce JIVEAdapter, a multi-task "additive" low-rank adapter inspired by statistical Joint and Individual Variation Explained (JIVE). JIVEAdapter decomposes every weight update into a Joint structure shared across all tasks plus a per-task Individual structure, penalizes the Individual structures to be near-orthogonal to the Joint so shared and task-specific signal stay "interpretable" and separated, and allocates rank adaptively across a shared Joint pool and a per-task Individual pool. The Joint is learned once, jointly over a task group or incrementally, one task at a time, then frozen and reused as a prior for new tasks without retraining the shared part. On GLUE and SuperGLUE with DeBERTaV3-base, JIVEAdapter is competitive with strong single-task and multi-task low-rank baselines at a matched per-task effective rank, without extra modules such as MoE, and when a related held-in task exists its frozen Joint serves a held-out task by reusing that task's Individual with only a cheap per-direction scale, otherwise training a small new one.
Read the full abstract ▾
Machine Learning
Shaping the Wind: Nested Potentials for Kinematically Admissible Urban Wind Prediction
Yidi Wang, Yunhe Zhang, Jiawei Gu et al. (+2 more)
Predicting transient urban winds is fundamental to understanding urban microclimates and designing
climate-resilient cities. Building-resolving large-eddy simulation produces detailed incompressible urban wind fields at substantial computational cost for each layout. Neural surrogates offer a faster alternative by learning to predict the
evolution of velocity fields. However, minimizing velocity prediction error does not guarantee local mass conservation and wall impermeability, which together define kinematic admissibility. This limitation stems from an unconstrained output representation: geometry conditioning guides predictions but does not restrict them to admissible velocity fields. Correcting boundary violations in these outputs changes the flux balance in adjacent fluid
cells and may consequently compromise local mass conservation. To address the challenge, we propose Sculpt, a nested potential framework that builds the coupled, geometry-dependent constraints directly into its parameterization. This nested parameterization generates divergence-free velocity updates through the discrete curl of a volume vector potential on the native three-dimensional staggered grid. A shared scalar potential constrains the vector potential's boundary values so that the same operator also enforces impermeability, without a per-step pressure projection. Because backpropagation through this curl attenuates large-scale gradient signals, we parameterize the volume potential at multiple resolutions to better capture large-scale flow structures. We introduce UrbanWindFlow, an LES
dataset spanning urban morphologies and inflow conditions, to evaluate accuracy and kinematic admissibility together.
Read the full abstract ▾
All authors Yidi Wang, Yunhe Zhang, Jiawei Gu, Ziyue Qiao, Pengyang Wang
Machine Learning · Cross-listed from cs.CL
Investigating Model Compression for Neural Machine Translation in the Biomedical Domain
Maria Zafar, Souhail Bakkali and Rejwanul Haque
Large-scale pretrained
transformer models have achieved state-of-the-art performance across diverse machine translation tasks, including multilingual settings. Knowledge
distillation has emerged as a sustainable approach for model compression, transferring knowledge from large teacher models to smaller, more efficient student models. Similarly, quantization, which reduces the numerical precision of model weights and activations (e.g., from 32-bit to 8-bit representations) is widely used to accelerate inference, enabling models to run several times faster during deployment. However, both techniques face limitations when applied to specialized domain data, particularly under low-resource conditions. In knowledge
distillation, the effectiveness of transfer is often constrained by the scarcity of domain-specific parallel data, while quantization can lead to performance degradation as bit precision decreases. In this work, we investigate the combined application of knowledge
distillation and quantization for French-to-English biomedical translation, a domain characterized by specialized terminology and limited parallel resources. We develop and compare multiple fine-tuning strategies to adapt compressed student models to this challenging setting. Our experiments demonstrate that a collaboratively distilled and quantized student model achieves a 69% reduction in size, a 98.21% increase in inference speed, and a 98.46% reduction in CO2 emissions compared to the original baseline all without sacrificing translation quality. These results indicate that jointly optimized compression techniques can yield efficient, high-performance models suitable for translation service providers operating under resource constraints.
Read the full abstract ▾
Machine Learning
Identifiable World Models from Pretrained Diffusion Representations
Ruchi Sandilya, Conor Liston and Logan Grosenick
Diffusion-based world models can generate and predict trajectories in high-dimensional dynamical systems, but predictive accuracy does not imply that their latent coordinates recover the underlying state variables or causal interactions. We ask whether a frozen pretrained
diffusion model can be equipped with identifiable coordinates without retraining its generative backbone. We show that auxiliary-variable nonlinear ICA guarantees can be transferred to Contrastive
Diffusion Alignment (ConDA), which learns only a lightweight
alignment map on top of frozen
diffusion latents. Under standard TCL/GCL assumptions, the aligned representation identifies latent dynamical states up to permutation and componentwise invertible transformations, preserves the latent dynamic structural causal model, and reduces lagged graph recovery to transition-Jacobian sparsity. We evaluate TCL-, GCL-, and CEBRA-based ConDA against TDRL, CaRiNG, IDOL, temporal SuaVE, and iVAE across physical and robotic
video systems. TCL and GCL achieve near-perfect blockwise state recovery and competitive lagged graph recovery, including exact recovery in a simulated falling-body system. In a simulated bipedal
robot, learned dynamics recover the sign and temporal structure of responses to held-out control perturbations. These results show that a frozen generative
diffusion model can be equipped with coordinates that are identifiable, structurally interpretable, and useful for analyzing intervention-relevant dynamics.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Calibrated Evidence Inference With a Small, Permissively Licensed Open Model
Johann Emmanuel Li
Systematic reviewers who use a model to read randomized trials need each answer and a measure of how far to trust it. If the model is calibrated, so that its stated probabilities match how often it is right, a reviewer can accept its confident answers and check the rest by hand. This paper fine-tunes Qwen3-4B-Base with low-rank adapters and a classification head for evidence inference (Lehman et al., 2019; DeYoung et al., 2020): given a trial's article and a
prompt that names an intervention, a comparator and an outcome, classify the result as a significant increase, a significant decrease or no significant difference. The articles of Evidence Inference 2.0 come from the open-access subset of PubMed Central, which includes articles under non-commercial and no-derivatives licenses, so the released model was fine-tuned only on the 1,431 of 2,657 training articles under CC BY, CC0 or the public domain; it is released under the Apache License 2.0 and runs on a laptop. On the test split, its expected calibration error is 0.017 and its log loss is lower than that of Gemma 4 31B-it given the same input zero-shot, meeting criteria registered on the Open Science Framework before testing. Its macro-F1, 0.925, is above every published result on that split that I found, though those systems used other inputs, scoring rules and
prompt filters, and the test labels are public. Accepting its 80 percent most confident answers leaves an error rate of 2.6 percent among them. Context ablations show that the answers depend on the article, but the title alone carries some signal, which an informative title or recall of the trial could explain.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Which Image Property Carries the Jailbreak? A Controlled Dissection of Image-to-Text Jailbreaks
Boyuan Chen, Yehia Dawoud, Hailemariam Mersha et al. (+4 more)
Image-to-text jailbreaks place harmful intent in text, image content, or the relationship between them. We examine image-side factors across four published attack families on a 313-
prompt StrongREJECT slice, using five
multimodal models and an additional appendix evaluation of InternVL3.5-8B. The harmful instruction is held constant across conditions; the baseline
matrix uses one draw per
prompt, and paired ablations use three draws with an automated rubric judge. A bare harmful query, with or without a benign unrelated image, produces little attack success on most victims, while attack images substantially increase it. First, per-tile
entropy and JPEG size do not reliably distinguish attack tiles from size-matched benign distractors, limiting density-only screening. Second, earlier tile-count ladders were confounded by payload visibility. A corrected region-count test found no detectable effect, so the role of tile-count structure remains unresolved. Third, on Qwen3-VL-8B, the E4 manipulation that removes query-specific relatedness lowers ASR by about 0.12. This supports a bounded attribution to the relatedness manipulation, although image-text congruence remains unmeasured. A within-category control reproduces the direction on overlapping stimuli. Five tests survive the global statistical correction, but only E4 supports attribution to one measured descriptor; the within-category result is a robustness check, not a separate attribution. These conclusions remain conditional on the rubric judge.
Read the full abstract ▾
All authors Boyuan Chen, Yehia Dawoud, Hailemariam Mersha, Minghao Shao, Siddharth Garg, Ramesh Karri, Muhammad Shafique
Machine Learning
STOCK-JEPA: Prior-Anchored Latent Revision Representation Learning in Equity Markets
Yizhi Luo, Jiahe Yi, Jianhui Zhang et al. (+1 more)
Learning effective representations helps characterize the structure and dynamics of equity
markets from financial data with a low signal-to-noise ratio. Black-box deep models can capture complex patterns but may overfit sample noise and lack explicit economic structure. Meanwhile, classic linear financial models provide interpretable references, but their oversimplified assumptions leave non-linear signals uncaptured. To combine the strengths of these two directions, we propose Stock-JEPA, a joint-
embedding predictive framework that learns predictable incremental revisions relative to a point-in-time financial prior. First, we leverage a low-complexity financial model to produce fixed statistics summarizing multi-horizon return and risk. A prior projector then maps these statistics into the target encoder's latent space as an anchor. Second, we design a context-conditioned revision predictor to estimate the future representation's predictable displacement from the anchor. Separate losses update the two branches: the anchor learns from prior statistics, while the revision captures additional predictable information from historical context. Third, we freeze all representation modules and train a downstream readout, evaluating its forecasts through cross-sectional ranking and portfolio performance. Theoretically, we prove that optimal revision reduces the prior anchor's expected squared error for the same future representation by exactly $\mathbb{E}[\|\boldsymbol{\Delta}\|_2^2]$. This non-negative gain is the expected squared magnitude of the additional signal predictable from historical context. Experimentally, Stock-JEPA outperforms 13 strong baselines across large-scale China and U.S. equity universes on 5 key evaluation metrics. Ablation studies and representation analysis further demonstrate the value of the learned revisions for representation learning in equity
markets.
Read the full abstract ▾
All authors Yizhi Luo, Jiahe Yi, Jianhui Zhang, Shuo Sun
Machine Learning · Cross-listed from cs.SD
Where Does the Audio Jailbreak Live? A Controlled Frequency-Depth Audit of AdvWave-P on Qwen2-Audio
Boyuan Chen, Minseok Kim, Sohaila Abdulsattar et al. (+4 more)
We audit frequency and decoder-depth claims for AdvWave-P, an additive audio jailbreak, on Qwen2-Audio. The protocol masks frequency components of the perturbation in the short-time
Fourier transform (STFT) domain and measures attack success and audio-span representations. On 520 AdvBench
prompts, the primary judge labels 76.7% of
adversarial inputs as jailbreaks. A condition-blind, single-annotator validation yields a Rogan-Gladen sensitivity estimate of 0.87 for this condition (about 0.83-0.95 with validation-rate uncertainty); this correction is not applied to masked conditions. The apparent frequency ranking depends on the partition: energy share alone predicts the standard eight-band ranking (Spearman's rho = 0.95), and equal-Hz and equal-energy partitions show that masking any tested band can sharply reduce attack success. At a finer 16-band equal-energy resolution, however, masking the narrow 7520-7960 Hz band leaves ASR at 0.10, which remains unresolved without a matched control. Matched-energy scattered-removal tests show that contiguous removal is more damaging in lower bands, while both forms approach the floor in upper bands. Global rescaling leaves ASR near baseline but tests amplitude sensitivity rather than frequency location. In a re-optimization pilot (n = 20), tested single- and two-band supports reach ASRs of 0.00, 0.25, and 0.40, while random supports covering about half the STFT bins reach a mean of 0.86. In a
prompt- and energy-adjusted model, audio-span divergence is associated with band necessity, with the coefficient rising from +0.63 at the projector output to +0.93 at layer 30 (contrast +0.293, 95% CI [0.11, 0.51]). This association is not a causal localization, and single-layer patching does not establish a causal layer. The results support partition-aware auditing of frequency claims, leaving the fine-resolution top-band result and broader generality open.
Read the full abstract ▾
All authors Boyuan Chen, Minseok Kim, Sohaila Abdulsattar, Minghao Shao, Siddharth Garg, Ramesh Karri, Muhammad Shafique
Machine Learning
Yiping Ji, James Martens and Simon Lucey
Diffusion models learn semantic representations while generating images. In the Decoupled
Diffusion Transformer (DDT), a condition encoder provides features that guide a velocity decoder in denoising. To enable effective denoising at all noise levels, these features must capture both high-level abstract structures and low-level details. However, skip/residual connections in the encoder allow shallow features to bypass successive transformations, which may limit progressive abstraction, or at least make it difficult to disentangle different levels of abstraction. We propose DDT-RFE, which removes the residual connections around the Self-
Attention and MLP operations in each encoder block while maintaining stable training. To retain the information that abstraction discards but that the decoder still needs, we fuse the input patch
embedding with intermediate and final encoder features to form the encoder output. The decoder thus has access to information from multiple encoder depths, while each encoder block is able to learn more abstract representations. DDT-RFE achieves overall improvements over DDT across visual understanding tasks, including image classification, semantic segmentation, object discovery, and semantic correspondence, while using fewer encoder blocks. It also achieves a lower FID for image generation on ImageNet.
Read the full abstract ▾
Machine Learning
Mask-Guided KV Cache Eviction in Block Diffusion Language Models
Gleb Molodtsov, Ekaterina Alimaskina, Evgeny Uskov et al. (+2 more)
Block
diffusion language models keep a large key-value (KV) cache throughout generation and attend to it at every denoising step, limiting both
memory capacity and generation speed. Reducing these costs requires deciding which past
tokens to use for denoising the current block (selection) and which to keep in
memory for future blocks (eviction). We propose MaskAhead, a training-free method that solves both tasks with a single mask-query-based ranking mechanism. Current-block masks guide selection, while probes of upcoming masked blocks guide eviction. Both rank KV entries by their estimated contribution to the
attention output. Our quantized variant, Q-MaskAhead, computes selection and
attention directly from low-bit KV, largely preserving the selected entries. Experiments on Fast-dLLM-v2, DreamReasoner, and LLaDA2.0-mini cover long-generation
reasoning, long-
prompt question answering, and needle-in-a-haystack
retrieval. On long-
prompt QA, MaskAhead reduces KV
memory by $9.5\times$ on average with a 1.2-point mean F1 loss relative to dense inference. Q-MaskAhead increases the reduction to $20.1\times$ with a 2.3-point mean F1 loss. In a batch-32 systems profile, MaskAhead achieves $1.23\times$ end-to-end and $1.68\times$ decode-stage speedups over dense inference.
Read the full abstract ▾
All authors Gleb Molodtsov, Ekaterina Alimaskina, Evgeny Uskov, Artur Zagitov, Aleksandr Beznosikov
Machine Learning
DART-ES: Difficulty-Aware Reweighting and Targeted Replay for Fine-Tuning LLMs with Evolution Strategies
Zhishen Sun, Hongzhan Wang, Sizhe Dang et al. (+2 more)
Evolution Strategies (ES) enable
memory efficient full parameter fine-tuning of large
language models (LLMs) using only forward computation. However, standard ES uniformly averages rewards across problems and compresses problem level population feedback into a single scalar, making it difficult to capture how the learning value of each problem changes with model capability. To address this limitation, we propose Difficulty-Aware Reweighting and Targeted Replay for
Evolution Strategies (DART-ES). DART-ES estimates the local solvability of each problem from its pass rate across the perturbation population and aggregates historical observations to construct a dynamic difficulty state. This shared state jointly guides continuous difficulty reweighting and rare solvable sample replay, thereby improving perturbation direction evaluation and training data allocation without introducing an additional difficulty model or backpropagation. Extensive experiments show that DART-ES achieves good fine-tuning performance. DART-ES outperforms ES on all five base models and improves the average accuracy from 72.07% to 73.53%, exceeding the 73.26% achieved by GRPO on GSM8K. Across five challenging mathematical
reasoning benchmarks, DART-ES achieves an average accuracy of 49.20%, compared with 48.34% for ES and remains competitive with strong 7B models trained with RL. Further experiments show consistent gains in instruction tuning, code generation and the Countdown task with a 14B model, demonstrating strong generalization across tasks and scalability to larger models. Beyond performance gains, DART-ES also shows clear advantages in system efficiency. It reduces runtime per step by 15.2%–50.2% and peak
memory usage per GPU by 21.1%–51.1% compared with GRPO. Despite performing full parameter fine-tuning, DART-ES also requires less runtime and GPU
memory than GRPO+LoRA.
Read the full abstract ▾
All authors Zhishen Sun, Hongzhan Wang, Sizhe Dang, Guang Dai, Haishan Ye
Machine Learning
Repair Lot Skyline: A Weighted Constraint Satisfaction Approach to Pavement Repair Optimization from Geospatial Hazard Density
Takato Yasuno, Keita Kobayashi, Ryuta Sakaguchi et al. (+1 more)
Pavement agencies must translate a spatially distributed distress inventory into a bounded, actionable repair-lot plan: accident-critical defects (potholes) must always be addressed, lower-risk defects (cracks) should be included only when their benefit justifies the repair cost, and historical patch locations signal re-degradation risk without themselves triggering repair. We formalize this as a Repair Lot Skyline problem: a Weighted Constraint Satisfaction Problem (WCSP) defined over chainage (distance along the road) rather than over time, so that it requires only a single-epoch distress survey and makes no claim about future deterioration. The WCSP identifies 143 candidate hazard clusters (61 hard, 82 soft), of which 106 are merged into a final repair plan totaling 1,997.4 m—83.7% of the 2,385.9 m that would be required if every soft candidate were included regardless of cost. This plan covers 100% of observed potholes (138/138) and 91.6% of observed cracks (404/441), capturing 93.6% (542/579) of the total hazard benefit available in the full candidate set. The skyline frontier shows pronounced diminishing returns beyond this point: the remaining 37 excluded soft candidates would add only 6.8% additional benefit for a 19.4% increase in repair length. We further formalize the minimum-lot-length $L_{\min}$ and historical-context radius $\kappa$ as a joint, four-objective hyperparameter search over this WCSP; on the same case study, the recommended configuration ($L_{\min} = 14.7$ m, $\kappa = 10$ m) reduces repair-crew mobilizations by 7.1% relative to an untuned default, at the cost of a 12.9% larger budget and a 1.1-percentage-point lower crack coverage.
Read the full abstract ▾
All authors Takato Yasuno, Keita Kobayashi, Ryuta Sakaguchi, Takuya Okamoto
Machine Learning
An Information-Theoretic Evaluation Framework for Benchmark and Model Diagnosis in Knowledge Tracing
Houru Jiang, Zixi Wang, Tengteng Cheng et al. (+6 more)
Knowledge tracing (KT) models are predominantly evaluated using aggregate metrics such as area under the curve (AUC) and accuracy. However, these global scores obscure where the remaining errors originate and fail to indicate whether a
benchmark is approaching saturation. While estimating a global theoretical performance limit is challenging in realistic KT settings, it is possible to quantify local predictability. To address this, we propose an information-theoretic evaluation framework for KT
benchmark diagnosis. We use Context Tree Weighting (CTW) on item-response histories and current-item queries as an operational causal uncertainty coordinate, while distinguishing it from the unobserved Local Irreducible Uncertainty (LIU) under the full KT information set. By projecting predictions onto this shared uncertainty coordinate, we evaluate model performance gains across distinct
entropy bands rather than only at the global level. Comprehensive evaluations on NIPS Task 3/4 and
Algebra 2005 reveal that model improvements are highly non-uniform. Modern KT models show substantial gains in high-
entropy regions, and additional item-aware references, log-loss, and equal-frequency analyses support this localization. The framework also flags regions where apparent gains require checks for noise-sensitive behavior. By surfacing these local modeling failures alongside genuine gains, this approach provides a diagnostic tool for studying both residual predictive structure and the limitations of current KT
benchmarks and models.
Read the full abstract ▾
All authors Houru Jiang, Zixi Wang, Tengteng Cheng, Xueyi Li, Mingliang Hou, Jiaqi Zheng, Renqiang Luo, Teng Guo, Zitao Liu
Machine Learning
A Data-Driven Framework for Unsupervised Monitoring of Transmission Systems Using End-of-Line Testing Data: A Case Study at Ford Motor Company
Mohammad N. Bisheh, Mehrdad Moradi, Parinaz Farajiparvar et al. (+6 more)
Sensing technologies have advanced rapidly across industries ranging from energy to automotive manufacturing. These systems generate high-dimensional (HD) data characterized by complex nonlinear patterns and strong temporal dependencies. Traditional statistical monitoring methods are often limited in their ability to capture such nonlinear structure. Likewise, many analytical approaches used in End-of-Line testing rely on predefined thresholds and heuristic rules, which restrict their ability to detect informative anomaly signatures in HD temporal data. In contrast, while modern deep learning and generative AI models offer strong predictive capabilities, they are often unsuitable in applications where data are costly to collect and where the monitoring system must remain interpretable, low-latency, computationally efficient, and usable by non-technical practitioners. To overcome these limitations, we propose an advanced multivariate monitoring framework for HD data. The framework operates in two stages. In the first stage, the data are preprocessed to remove incomplete and non-informative samples and to temporally align time series data. In the second stage, nonlinear dimensionality reduction is performed, followed by anomaly detection through a control chart based phase I monitoring procedure. The framework can be used in both unsupervised and supervised settings, depending on the availability of ground truth labels during training. Moreover, its flexible and modular structure allows practitioners to adapt its components to different domains and operational requirements. We evaluate the proposed framework on real production data from an automotive manufacturing environment at Ford Motor Company. The proposed method achieves higher accuracy, recall, and F1 score than the company's existing model, improving these metrics from 0.50, 0.30, and 0.429 to 0.625, 1.00, and 0.769, respectively.
Read the full abstract ▾
All authors Mohammad N. Bisheh, Mehrdad Moradi, Parinaz Farajiparvar, Colin Brady, Rajesh Gupta, Xueling Li, Javad Navaei, Milad Parvaneh, Kamran Paynabar
Machine Learning · Cross-listed from cs.CV
Hierarchy-GBP: Accelerating Factor Graph Inference via Abstraction and Recovery
Yuzhou Cheng, Tom Yates, Ignacio Alzugaray et al. (+3 more)
Gaussian Belief Propagation (GBP) is a distributed inference
algorithm that passes messages in graphical models, making it attractive for scalable spatial intelligence. However, we find GBP most effective locally: it rapidly smooths message errors that vary sharply between neighbor variables, but corrects global errors across distant graph regions incrementally through long-range message propagations. We propose Hierarchy-GBP (H-GBP), an iterative, two-stage framework that accelerates GBP by first solving these global errors with a coarse graph approximation (abstraction) and projecting the results back to the original graph (recovery), then refining the remaining local errors with GBP. We prove H-GBP convergence to the optimum by deriving the combined
matrix operator of our abstraction and recovery steps and analyzing its spectral radius. Experiments on linear sparse graphs show that H-GBP converges fundamentally faster than standard GBP. Moreover, we validate H-GBP on two important spatial problems: Pose Graph Optimization (PGO) and Bundle Adjustment (BA). H-GBP markedly accelerates large-scale PGO and achieves state-of-the-art runtime across all tested BA scales.
Read the full abstract ▾
All authors Yuzhou Cheng, Tom Yates, Ignacio Alzugaray, Danyal Akarca, Pedro A. M. Mediano, Andrew J. Davison
Machine Learning
Delay-coordinate reconstruction and conditional-moment causal diagnostics in stochastic systems
Jun Ohkubo
Partial observation and delay-coordinate reconstruction are rooted in deterministic dynamical-systems theory, whereas there are many systems with intrinsic stochasticity. We propose a conditional-moment interpretation of delay-coordinate reconstruction for
stochastic systems, in which a delay vector is used to reconstruct conditional moments of a future distribution rather than a unique future sample path. Two complementary arguments motivate this viewpoint. First, the probability density of a
stochastic differential equation obeys a deterministic Fokker-Planck equation and, under certain assumptions, is represented by an infinite deterministic hierarchy of moments. Hence, a finite-moment closure suggests a Takens-like finite-dimensional approximation. Second, a discussion based on the Koopman operator theory clarifies that the time
evolution of an observable in the Mori-Zwanzig formalism yields a conditional expectation in
stochastic systems. Then, the orthogonal "noise" term in the coefficient-space Mori-Zwanzig equation vanishes in the
stochastic cases; this result is consistent with the moment-based argument. As an application of this
stochastic delay-reconstruction viewpoint, we revisit convergent cross mapping (CCM) for diagnosing certain causal relationships. Although CCM based on the
embedding theorem cannot generally be applied to
stochastic systems, it is possible to examine certain types of causal relationships by using conditional moments. Using coupled logistic systems with additive and multiplicative coupling mechanisms, we discuss how causal relationships are embedded in
stochastic systems.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ME
Causal Lag Structure Discovery in Confounded Time Series via Orthogonalized Adaptive Estimation
Hong Kiat Tan, Isaac-Neil Zanoria, James Chen et al. (+2 more)
Finding which variables cause which others in multivariate time series, and at what lags, is central to science and policy, yet existing methods force a choice between flexible confounder adjustment, data-driven lag selection, and inference that controls the false discovery rate (FDR). ORACLE-VARX does all three in one pipeline. First, double/debiased machine learning (DML) removes nonlinear confounder effects from the outcomes and the lagged series. Second, adaptive causal lag estimation (ACLE) picks the lag order at each time step by sequential significance tests, tracking regime changes. Third, entry-wise $z$-tests with Benjamini–Hochberg correction select directed edges at a target FDR. We prove that in each rolling window, the debiased coefficients are asymptotically normal around a window-averaged target, so their $z$-tests are asymptotically valid. On a synthetic
benchmark with time-varying structure and nonlinear confounding, ORACLE-VARX (LightGBM) tracks the true lag order best (RMSE $0.96$ vs $1.1$–$1.5$), has edge FDR $0.047$, close to PCMCI ($0.045$) and below VAR ($0.129$) and VAR-LiNGAM ($0.187$), and forecasts better than all three. On nine U.S. sector ETFs with macroeconomic confounders, it yields interpretable causal graphs whose lag order rises in high-
volatility regimes.
Read the full abstract ▾
All authors Hong Kiat Tan, Isaac-Neil Zanoria, James Chen, Haoyang Lyu, Mihai Cucuringu
Machine Learning · Cross-listed from cs.CV
Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation
Team Kandinsky, Julia Agafonova, Bulat Akhmatov et al. (+85 more)
We present Kandinsky 6.0
Video, a family of foundation
diffusion models for synchronized text-to-audio-
video generation, comprising Kandinsky 6.0
Video Lite (3B parameters) and Kandinsky 6.0
Video Pro (29B parameters). Both models generate 5-second
video clips with synchronized 44 kHz audio, including lip-sync, in text-to-audio-
video (T2AV) and image-to-audio-
video (I2AV) modes; a built-in super-resolution model raises the output resolution to Full-HD (1920$\times$1080). Building on the
video generation capabilities of Kandinsky 5.0, Kandinsky 6.0
Video employs a dual-stream CrossDiT architecture that connects a pretrained
video stream and a newly trained audio stream through bidirectional cross-
attention for temporal and semantic
alignment. Our continuous pretraining strategy first trains the audio stream from scratch on large-scale audio corpora and then trains both streams jointly on paired audio-
video data while preserving unimodal fidelity; pretraining is followed by supervised fine-tuning, reinforcement-learning-based post-training, and
distillation. In side-by-side human evaluation, Kandinsky 6.0
Video Pro clearly outperforms its predecessor, Kandinsky 5.0
Video Pro, and remains competitive with leading audio-
video generation models, particularly in
speech quality. To accelerate open research and deployment in multimedia generation, we release the code, model checkpoints, and diffusers integration under the MIT license.
Read the full abstract ▾
All authors Team Kandinsky, Julia Agafonova, Bulat Akhmatov, Mikhail Aksyutin, Grigorii Alekseenko, Anastasia Aliaskina, Olga Androsova, Vladimir Arkhipkin, Anna Averchenkova, Alexander Belykh, Serafima Bocharova, Sofiya Bogakovskaya, Anton Bukashkin, Mark Bulygin, Kirill Buzygin, Irina Cheremnykh, Kirill Chernyshev, Mikhail Chernyshov, Vladimir Chernyy, David Chikovani, Georgy Daniltsev, Denis Dimitrov, Anna Dmitrienko, Vladimir Dokholyan, Sergey Emelyanov, Dmitry Ermilov, Georgii Fedorov, Polina Gavrilova, Nikolai Gerasimenko, Aleksandr Gordeev, Andrey Inozemtsev, Andrei Ivaniuta, Alexander Ivanov, Mikhail Karaev, Anastasiia Kargapoltseva, Ivan Kirillov, Nikita Kiselev, Valeria Kobenko, Yury Kolabushin, Denis Koposov, Anatoly Korobov, Vladimir Korviakov, Kirill Kozlov, Denis Krzhivokolskiy, Konstantin Kuklev, Alexander Kunitsyn, Sergey Kuzin, Vladislav Lakhtionov, Alexey Letunovskiy, Maxim Litvinov, Alexander Lyulkov, Georgy Makarov, Kirill Malakhov, Egor Malykh, Mikhail Mamaev, Dmitrii Mikhailov, Polina Mikhailova, Ivan Mikheev, Elizaveta Muromtseva, Nikolai Nazarkin, Tatiana Nikulina, Lev Novitskiy, Stanislav Onuchin, Nikita Osterov, Denis Parkhomenko, Anatoliy Parpara, Vladimir Polovnikov, Konstantin Reznikov, Azat Saginbaev, Nikita Samsonov, Alexander Sentsov, Nikita Shaimov, Artem Sherstyuk, Andrey Shutkin, Egor Silvestrov, Bulat Suleimanov, Matvey Suprunov, Sergey Taranov, Irina Tolstykh, Tatiana Trofimuk, Ilya Trushkin, Aleksandra Tsybina, Olga Varlashina, Viacheslav Vasilev, Ilya Vasiliev, Eugeny Vilisov, Sergey Yakubson, Konstantin Zakharov
Machine Learning
Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Models
Kaiyuan Deng, Yuchen Li, Yang Xiao et al. (+3 more)
Existing attacks on unlearned
diffusion models assume that the erased concepts are known in advance and focus on recovering them. In practice, however, model providers may not disclose which concepts have been removed, and even with access to the original base model, an adversary may still lack a clear target to attack. In this paper, we aim to answer the following critical but overlooked questions: which concepts have been erased from the model, and how many have been erased in total? To this end, we present Tracer, a framework that rapidly and accurately identifies erased concepts and estimates their number. Tracer efficiently identifies erased concepts without generating and classifying images. By combining lightweight spectral analysis of weight footprints, it enables efficient search over large candidate vocabularies. To distinguish multiple erased concepts, we introduce a footprint coverage objective that guides sequential discovery. Tracer estimates the number of erased concepts by detecting a sharp decline in candidate confidence as the selected concepts account for the erasure footprint, without requiring labeled examples for calibration. The framework requires only lightweight linear
algebra and limited forward probes, with no prior knowledge of the unlearning
algorithm. Experiments across text-to-image and text-to-
video backbones and diverse unlearning methods demonstrate that Tracer identifies erased concepts and estimates their number in seconds, achieving 150 to 137,000 times and 133 to 20,000 times speedups over MIA and brute-force search on image and
video models, respectively, with substantially higher identification accuracy.
Read the full abstract ▾
All authors Kaiyuan Deng, Yuchen Li, Yang Xiao, Bo Hui, Geng Yuan, Xiaolong Ma
Machine Learning
An LLM-in-the-loop RL Framework for Bioinformatics Feature Selection
Xinyuan Wang, Deepti Agrawal and Yanjie Fu
High-dimensional bioinformatics data, characterized by a large number of features relative to the number of samples, pose major challenges such as the “curse of dimensionality,” leading to overfitting, high computational cost, and poor generalization. Traditional feature selection methods often suffer from limited scalability and adaptability in such domains. We propose an LLM-in-the-loop reinforcement learning (RL) framework for bioinformatics feature selection, where the RL
agent formulates feature selection as a sequential decision-making task, while the large
language model (LLM) enhances the process in two ways: (1) guiding exploration through domain-informed advice, and (2) providing hybrid rewards that integrate data-driven performance with knowledge-driven evaluation. The LLM also produces explanations to improve interpretability for human experts without altering the RL policy update. Experiments on diverse bioinformatics
datasets show that the LLM-in-the-loop framework outperforms baselines, achieves stable performance across downstream models, and converges faster than pure RL.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Gaussian Limits for SGD Without Stationary Moments
Xiaoli Li and Wei Biao Wu
Temporal dependence can separate the Gaussian approximation of
stochastic gradient descent from its stationary moments. For unmodified least-squares SGD, we construct a design with standard Gaussian marginals whose stationary error has every positive moment infinite. Independent observations with the same marginals instead give finite stationary variance. Both regimes retain a Gaussian small-step limit. Our general theory establishes pathwise contraction from a finite second design moment, then uses score cancellation and localization to obtain stationary Gaussian and Ornstein–Uhlenbeck limits. Independent Gaussian regression errors yield an exact conditional Gaussian law and total-variation convergence under the same design integrability. Stronger design conditions identify a positive first-order total-variation constant and a deterministic covariance correction with $o(a)$ error. A scalar coverage expansion translates this correction into its inference consequence. Experiments examine distributional error, coverage, and calibration with dependent scores. Together, these results establish precise probability-law approximation beyond moment-based stationary analysis.
Read the full abstract ▾
Machine Learning
When the Cross-Silo Federation Goes Offline: Continual Learning for Site Onboarding with Limited Unlabeled Data
Ahmadreza Eslaminia, Klara Nahrstedt and Chenhui Shao
An organization often holds too little labeled data to train a model that generalizes, and the records that would supply the rest sit with organizations that cannot release them. Cross-silo
federated learning offers a way through, since participants exchange model parameters rather than records, but it ordinarily settles two aspects of the arrangement in advance, the participating sites and the classes the model can predict, and deployment can breach both. A new site joins after training, once the established sites have finished their engagement and gone offline, and its records arrive unlabeled, mixing conditions the model already recognizes with conditions no participant has observed. We present an autonomous three-stage procedure that expands the model entirely at the joining site: reconstruction experts screen for novelty, clustering separates the flagged records into candidate conditions, and class means describe the old classes, all inside one shared representation. Those classes were learned from records that never leave their owners, so the usual defenses against forgetting are unavailable, and the procedure supplies the evidence they would have carried from either of two dissimilar sources, prototypes held by the federation or records held by the joining site. On a real industrial condition-monitoring
dataset, run end to end with no label consulted, either source holds old-class accuracy at 0.868 or above with forgetting at most 0.063, and the two differ by 0.021, so a configuration can be chosen by the disclosure it permits rather than the accuracy it delivers. Both keep old- and new-class accuracy in balance where every alternative we measure gives up one for the other, and both retain more of the old classes than
distillation- and regularization-based baselines. The balance still holds with only 6 labeled records per arriving condition and 3 retained per old class.
Read the full abstract ▾