Machine Learning
AutoPref: Automatic Discovery of Task-Specific Preference Objectives for Neural Combinatorial Optimization
Shengda Gu, Kai Li, Xinyi Ke et al. (+3 more)
Combinatorial optimization problems (COPs) underpin many real-world decisions, but their exponentially large search spaces make high-quality solutions costly to obtain. Neural combinatorial optimization (NCO) learns fast construction policies, typically with reinforcement learning (RL), while preference-based NCO improves sample efficiency by learning from relative solution quality. However, existing preference objectives combine two distinct design choices in manually specified, one-size-fits-all formulations: what learning signal to extract from each solution pair and how to weight each pair relative to the sampled set. We present AutoPref, the first LLM-guided framework for automated preference-objective discovery in NCO. AutoPref factorizes the objective into a pairwise loss program, which defines the learning signal, and a set-aware weighting program, which determines each pair's relative contribution. Their composition forms a unified programmatic objective space containing existing preference objectives as special cases. To make its search tractable, we introduce a staged conditional search strategy with behavioral gates that filter inadmissible programs before short-horizon training and evaluation. Across TSP, CVRP, FFSP, and JSSP, AutoPref consistently outperforms strong hand-designed baselines across problem scales, demonstrating the benefits and scalability of automated objective discovery for NCO.
Read the full abstract ▾
All authors Shengda Gu, Kai Li, Xinyi Ke, Haobo Fu, Yifan Zhang, Jian Cheng
Machine Learning · Cross-listed from quant-ph
Complementary Matrix-Gated QKAN Fast-Weight Programmers for Quantum Dynamics Forecasting
Kuo-Chung Peng, Samuel Yen-Chi Chen, Jiun-Cheng Jiang et al. (+14 more)
Sequence models must decide what to write into
memory and what to retain. In
quantum and
quantum-inspired sequence learning, nonlinear recurrent updates often require repeated
circuit evaluations and sequential backpropagation through time, making long contexts costly. Gated fast-weight programmers (FWPs) based on
quantum-inspired Kolmogorov-Arnold networks (QKANs) alleviate this bottleneck by storing context in time-varying fast parameters. However, their scalar gate applies one retention-write balance to every fast-state coordinate, forcing all parameters to share a
memory timescale. We introduce Self-Modulating QKAN-based FWPs, which replace this broadcast gate with low-rank-generated element-wise modulation of the new-proposal branch, a bounded old-state branch, or both. We further propose Complementary
Matrix Gating (CMG), which uses one sigmoid
matrix gate to retain the old state and its complement to write the new proposal. CMG provides coordinate-wise
memory control while preserving the bounded convex update and affine prefix-scan structure of scalar gating, at the modulation-head cost of a single-branch rule. We compare four self-modulating rules with scalar gating across four FWP architectures combining classical and QKAN-based slow and fast programmers. Across seven single-step
forecasting benchmarks and five sequence lengths, CMG gives the most consistent improvements for architectures whose fast programmer incorporates a QKAN-based module. In direct multi-step
forecasting of Jaynes-Cummings and transmon-resonator dynamics simulated with CUDA-Q Dynamics, CMG models maintain mean-squared errors on the order of 0.001 or lower across
forecasting horizons of 4, 8, and 16 steps, while improving on their scalar-gated counterparts by at least 91.2%. These results establish coordinate-wise complementary modulation as a stable and effective update for QKAN-based FWPs.
Read the full abstract ▾
All authors Kuo-Chung Peng, Samuel Yen-Chi Chen, Jiun-Cheng Jiang, Chen-Yu Liu, En-Jui Kuo, Yun-Yuan Wang, Tzung-Chi Huang, Prayag Tiwari, Chi-Sheng Chen, Chun-Hua Lin, Yu-Chao Hsu, Tai-Yue Li, Saif Al-Kuwari, Simon See, Kuan-Cheng Chen, Nan-Yow Chen, Hsi-Sheng Goan
Machine Learning
TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement
Cheng Wei
Federated fine-tuning of large
language models (LLMs) enables collaborative training without exposing raw data. However, a recent attack, NeuroImprint, demonstrates that a malicious parameter server can corrupt a PEFT adapter into a
privacy backdoor: by assigning a dedicated memorization
neuron to each training sample and ensuring each
neuron updates at most once, the server can analytically reconstruct 59%–79% of client training data with high semantic fidelity. Existing defenses—including local differential
privacy (LDP) and gradient clipping—either fail against this attack or impose unacceptable utility degradation. We present TriShield, a three-layer deterministic defense that completely prevents NeuroImprint-style reconstruction with zero model utility loss and no additional communication rounds. TriShield consists of: (1) a Parameter Artifact Detector that identifies
memory-
neuron signatures in distributed model parameters before local training begins; (2) a Stateful Virtual Iteration} mechanism that forces Adam/AdamW's momentum state to irreversibly entangle gradients across virtual steps, invalidating NeuroImprint's closed-form inversion; and (3) a Zero-Utility Orthogonal Projection operator that projects all local gradient updates onto the main-task semantic subspace computed via SVD, physically eliminating any gradient components that carry private memorization. We prove theoretically that after Layers 2 and 3, the mutual information between the uploaded gradient and any individual training sample is zero. Experiments on GPT-2 (117M) and Llama-Guard-3-1B verify that TriShield reduces NeuroImprint reconstruction rate to 0% across all tested attack variants, while maintaining or improving training accuracy, with less than 5% additional GPU computation overhead.
Read the full abstract ▾
Machine Learning
Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting
Xiang Yuan, Kaiqing Lei, Zhenyu Jin et al. (+3 more)
The performance of Large
Language Models (LLMs) is fundamentally influenced by the distributional composition of multi-domain pre-training data. While manual heuristics were prevalent in early models, they increasingly fail to capture the intricate synergies between domains as data complexity grows. To overcome the issue, a dominant approach seeks to fit a proxy function mapping between domain weights and their corresponding validation losses, and then find the optimal domain weights to minimize validation losses. These methods rely on strong structural assumptions, such as rank invariance or scaling laws, which are often violated, resulting in non-negligible estimation bias. A promising approach is to directly optimize the weighting scheme from data. However, it suffers from unstable optimization trajectory and prohibitive computational overhead, limiting its potential to search better domain weights configurations. This paper presents a
Bayesian domain weighting method to infer the weights from a Dirichlet distribution via introducing Gamma prior information learned from observations. Experimental results demonstrate that proposed method could achieve stable and efficient domain weights learning, and identifies optimal mixtures while consuming substantially less data than search-based function-fitting methods, revitalizing optimization-based domain weighting for large-scale applications.
Read the full abstract ▾
All authors Xiang Yuan, Kaiqing Lei, Zhenyu Jin, Jun Shu, Deyu Meng, Zongben Xu
Machine Learning
ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow
Dongxiu Liu, Haoyi Niu, Peng Cheng et al. (+5 more)
In the physical world we inhabit, space and time are fundamentally continuous. However, existing machine learning paradigms for world modeling are largely confined to discrete-time prediction, thereby exhibiting significant inefficiency in capturing the dynamics of physical world. We introduce Physical-Time Flow (PT-Flow), a novel approach that learns a continuous latent velocity field operating in physical time. Crucially, the underlying dynamics of sequential data are parameterized by an ordinary differential equation (ODE) embedded in a well-structured representation space. Under this paradigm, the prediction of future can be recast as temporal integration via an ODE solver in the compressed latent space. Building upon PT-Flow, we construct ODEWorld, a continuous-time latent world model that is both efficient and versatile. By extracting time-variant features and enforcing ODE properties on both the dynamical representation space and the latent velocity field, ODEWorld effectively addresses the long-standing representation collapse issue in latent world model literature. This also enables high-quality image reconstruction even after long-horizon prediction. Moreover, its continuous nature allows for arbitrary temporal resolution and even backward prediction, which is impossible for most discrete-time models. Lastly, ODEWorld can provide rich planning-oriented information to facilitate downstream policy learning. Comprehensive experiments demonstrate that ODEWorld successfully reconciles planning-conducive dynamics abstraction with visual realism, excelling in both
video generation and robotic control. Project page: https://dstate.github.io/odeworld_website/.
Read the full abstract ▾
All authors Dongxiu Liu, Haoyi Niu, Peng Cheng, Yuan Gao, Xirui Kang, Sangli Teng, Koushil Sreenath, Xianyuan Zhan
Machine Learning
Exact Action Values Are Not Enough: Rollout-Verified Reinforcement Fine-Tuning of a Reasoning Model for Multi-Zone VAV Control
Takumi Shioda, Kohei Terashima and Tatsuo Nagai
Multi-zone variable-air-volume control must balance thermal comfort, indoor air quality, and electricity use across several continuous actuators. Model predictive control and reinforcement learning are widely studied, but deployment typically requires building-specific modeling or training, limiting scalability. We first test whether a frontier
reasoning model (an LLM trained to use additional inference-time computation) can achieve competitive VAV control from text without building-specific training. With that capability established, we then test whether TD3-guided reinforcement fine-tuning (RFT) can transfer control knowledge into a locally deployable open-weight model. Five controllers are evaluated over three summer days in a physics-based four-zone emulator. Relative to a Guideline 36-based baseline, TD3 reduced HVAC electricity by 4.5% while improving temperature and CO$_2$ compliance. Without building-specific training, GPT-5 achieved the largest reduction (6.2%) but reduced the ventilation margin. For RFT, deterministic rollouts restore a saved state, apply one candidate, and follow TD3 to score each action.
Auditing a learned critic against these rollouts exposed a failure hidden by its near-perfect across-time correlation ($r=0.9998$): within-state ranking was unreliable; the critic selected the rollout-best candidate in only 5 of 10 states. Even with the rollout verifier, 200 RFT steps produced no sustained improvement in sampled-action return; the open-weight controller used more electricity than the baseline before and after training, and its five-minute predictions remained worse than persistence. GPT-5 predicted transitions far better. Exact rollout scores rank sampled actions but reveal neither next-state effects nor an improvement direction. The unchanged transition errors motivate transition-focused supervised fine-tuning before value-based RFT.
Read the full abstract ▾
Machine Learning
S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring
Glenn Anta Bucagu, Thorir Mar Ingolfsson, Yawei Li et al. (+1 more)
Foundation models offer a promising paradigm for Electroencephalography (
EEG) analysis, leveraging generalizable representations from vast unlabeled
datasets. Yet,
Transformer-based architectures face a critical bottleneck: global
attention mechanisms couple the
attention memory state to the signal duration, causing
memory overflow during continuous monitoring. To address this, we introduce S-CEReBrO (Streaming CEReBrO), an
evolution of the CEReBrO architecture designed for continuous monitoring. Our novel Windowed Alternating
Attention mechanism factorizes
attention computation into fixed-size spatiotemporal windows, so that under streaming, only the active window remains resident and the
attention state is bounded independently of signal duration. Empirical scaling analysis shows that windowed alternating
attention can process signals 100X longer than full self-
attention and 3X longer than low-rank linear
attention. Compared to low-rank linear
attention on long contexts, windowed alternating
attention requires 55% of the
memory while increasing inference throughput by 2.1X. Pre-trained on >25,000 hours of recordings from >12,000 subjects, S-CEReBrO achieves state-of-the-art performance on 7 of 11 downstream tasks, with up to 60% fewer parameters. This work represents a significant step toward the realization of efficient, generalizable, and continuous
EEG monitoring. An accompanying code repository is available. An accompanying code repository is available.
Read the full abstract ▾
All authors Glenn Anta Bucagu, Thorir Mar Ingolfsson, Yawei Li, Luca Benini
Machine Learning · Cross-listed from cs.SD
Integrating Contextual Embeddings into Evaluation of Expressive MIDI Piano Performances
Dmitrii Gavrilev, Ilya Borovik and Vladimir Viro
Objective evaluation of expressive MIDI piano performances typically relies on attribute statistics such as timing, velocity, and duration of individual notes. However, these methods often disregard dependencies between notes, which poses a potential limitation in assessing the similarity between two sets of performances. In generative applications, the wide variety of expressive attributes makes it difficult to aggregate them into a single scalar metric for model selection. In this work, we reexamine attribute-scoped metrics and explore the perceptual properties of contextual
embeddings from self-supervised symbolic music models, Aria and CLaMP3. Results from our listening study indicate that these models can be used as perceptual proxies, showing agreement with per-sample human ratings on par with traditional metrics. To measure conditional distributional similarity, we adapt Kernel Audio Distance to the symbolic music domain. Unlike Pearson correlation and reconstruction error, kernel-based methods on contextual
embeddings do not require note
alignment and are sensitive to contextual perturbations. To facilitate reproducibility, we release Pereval, an open-source library that integrates performance evaluation utilities, including both attribute-scoped and deep feature metrics.
Read the full abstract ▾
Machine Learning
Class-Aware Reinforcement Learning for Counterfactual Explanation Generation
Muhammad Adil Saleem, Syed Ali Raza and Mary-Anne Williams
Counterfactual explanations (CFEs) enhance the interpretability of black-box models by generating alternative instances with adjusted feature values that achieve a contrastive outcome. Reinforcement learning (RL) offers a promising approach for CFE generation, enabling efficient exploration of counterfactual instances while ensuring control over key metrics like validity, sparsity, and proximity. Previous studies have formulated RL states exclusively using features derived from the predictors in the supervised
dataset. This study explores the impact of including an instance's predicted class, alongside features derived from the predictors, in the RL state representation for generating CFEs. The hypothesis is that class-awareness enhances exploration efficiency and improves policy optimality. We compare the proposed class-aware RL method with the class-blind RL method, which is similar but excludes the instance's class information from the state representation. The comparison was conducted using seven
datasets from diverse domains, varying in size. The results show that during training, class-aware RL offers benefits in terms of convergence speed, reward optimization, and episode length reduction. Moreover, it generates significantly more valid CFEs compared to class-blind RL. Finally, the instance's class-based feature consistently ranks among the most influential predictors in RL's action-selection, as shown by the SHAP and LIME values, underscoring the significance of class-awareness in RL for CFE generation. The impact is heightened clarity, faster learning, improved validity, and more effective counterfactual generation across diverse
datasets.
Read the full abstract ▾
Machine Learning
Contrastive Concept Importance: Explaining Pairwise Class Decisions Through Automatically Extracted Concept Representations
Roel Visser, Isaac Roberts and Barbara Hammer
Concept-based explanations are a prevalent way to explain the decisions of complex black-box methods through semantically meaningful, human interpretable concepts. To attribute the contribution of such concepts to a model's decisions, feature attribution methods are used to quantify how strongly each concept contributes to a model output. These attributions are typically computed for a single output class and therefore answer a non-contrastive "why P?" question. In many situations, however, such as cases of misclassification, class confusion, and low- margin predictions, the more natural question to ask is "why P rather than Q?". We introduce contrastive concept importance, which attributes the logit margin between a target class and a contrast, or foil, class to concepts in an automatically extracted visual concept basis. The resulting scores are signed, indicating whether a concept supports the target over the foil or the foil over the target, and can be decomposed into target-logit and foil-logit effects. This makes it possible to distinguish globally important concepts from concepts that specifically influence a class-pair distinction, including whether their effect is shared, one-sided, or directly contrastive. We evaluate our method both qualitatively and quantitatively on a range of ImageNet class pairs. Our results show that contrastive concept importance reveals class-pair specific model behavior that is not captured by standard concept importance alone, as well as capturing information on the semantic structure of the underlying ImageNet classes.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Dynamic Spectral Filtering for Temporal Graph Learning: Learning Evolving Propagation Operators
Yan Kong
Temporal graph learning is commonly organized around the
evolution of node states or the encoding of interaction histories. We study an underexplored, operator-centric question: should the graph propagation mechanism itself evolve over time? We introduce Dynamic Spectral Filtering (DSF), which represents propagation at snapshot t by a Chebyshev
polynomial filter with vector-valued, time-dependent coefficients. DSF explicitly treats these compact multi-order coefficients as recurrent temporal states. A recurrent branch proposes updates, while multiplicative global and order-specific gates regulate their magnitude. The temporal state is independent of the number of nodes. On MOOC, Wikipedia, and Reddit temporal link-prediction
benchmarks, converged DSF runs attain AP scores of 0.7851, 0.9088, and 0.9860, respectively, with 93K to 133K trainable parameters, 68 to 182 MB peak GPU
memory, and 1.6 to 2.1 seconds of training per epoch. Against the closely related DEFT baseline, DSF is better on MOOC, within 0.001 AP on Reddit, and modestly lower on Wikipedia, while using 8.3 to 8.6 times fewer parameters, 25 to 33 times less GPU
memory, and 5 to 19 times less time per epoch. Relative to all measured alternatives, it uses 3.3 to 38.6 times less GPU
memory. These results support direct spectral-response
evolution as a useful temporal inductive bias when computational efficiency is a first-class requirement.
Read the full abstract ▾
Machine Learning · Cross-listed from q-fin.GN
ZAPs: A Reward Attribution Framework for DeFi Ecosystems with Adversarial-Robust Scoring via Parallel Anomaly Ensemble Detection
Girish G N, Ashutosh Sahoo, Ajay Bhat et al. (+4 more)
Incentive programs are central to user acquisition in decentralized finance, but many reward systems rely on raw volume, transaction count, and wallet count, making them vulnerable to bots and sybil operations. We present ZAPs, a reward attribution framework that combines economic contribution scoring with
adversarial robustness. A composite activity score uses protocol-specific percentile normalization to limit whale dominance while preserving differentiation among users. A two-layer weighting mechanism combines protocol share within sector and sector share within the ecosystem, which reduces the profitability of farming small protocols. We show that the maximum reward obtainable from any protocol is bounded by that protocol's global volume share. ZAPs also introduces a four-layer defense stack consisting of transaction-level integrity checks, a parallel anomaly ensemble, post-distribution behavioral
memory, and graph-based sybil clustering. The anomaly ensemble combines a one-class reconstruction model with an isolation forest and applies graduated rather than binary penalties. On 1,073 labeled malicious wallets covering 124,638 transactions, the ensemble achieves 0.923 +/- 0.013 ROC-AUC, compared with 0.891 +/- 0.016 for the reconstruction model alone, when the isolation forest is trained on benign wallets. Training it on the pooled population reverses its polarity and removes the ensemble gain. Controlled simulations reduce
adversarial reward capture by 30-90 percent while legitimate-user scenarios change by 1-8 percent. Live campaigns recorded a 56 percent reduction in sybil allocation, a 49 percent increase in quality-wallet participation, and a 50 percent reduction in sell pressure.
Read the full abstract ▾
All authors Girish G N, Ashutosh Sahoo, Ajay Bhat, Akshay SP, Gurukiran S, Parag Paul, Dhanashekar Kandaswamy
Machine Learning · Cross-listed from eess.SY
Safety-Gated Agentic Supervisory Control on a Coupled Distillation Benchmark: Regime Map, Auditable Gate, and Co-Design Findings
Christian Rosenthal
An open-weight LLM can write composition setpoints every five minutes. What a plant still needs is a hard check: named constraints, logged margins, and an admit/block decision before the regulatory layer moves. This paper puts that check in a rule-based forked-twin counterfactual gate (nine pinned constraints) and leaves the regulatory layer unchanged. On Skogestad's Column A the ladder is PID-only (C0), linear MPC (C1), ungated
agent (C2), and gated
agent (C3) under one contract: identical level closure (M_D, M_B), scenarios, and seeds; C2/C3 share the linear-MPC backend. The split is not subtle. Off-nominal target acquisition: the
agent beats Pareto-tuned linear MPC in the strong band (C2/C1 IAE ratio 0.361 at the upper CI). Disturbance rejection on the same 16-point grid inverts by 16.03 at the upper CI (10.18 at the point estimate), where an ungated LLM supervisor does not belong. The gate compresses a specification-abandonment attractor into a bounded offset (d approx. -1.4; P95
cell IAE 11.5 to 0.77). A one-line
prompt fix removes the attractor at source (6/10 to 0/10; sensitivity only, not a new headline). In a 250-
cell statistical pass, 534 of 590 gate interventions are spec-on-bound geometry: the operating specification sits on a safety limit, so a well-behaved OP becomes inoperable while misbehaving ones are only contained; 318 blocks still correct actively harmful proposals. Headlines are single-column and model-conditional on DeepSeek-V4-Flash. A second-family sweep (NVIDIA Nemotron-3-Super) keeps the disturbance-rejection fails band and plant-side failure geography; magnitudes and protocol operability stay model-conditional, and Super target-acquisition strong
cells are survivors only (not confirmation). Transfer means twin, constraint envelope, and setpoint interface, not a second plant class measured here.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.ET
Nanoparticle Networks for Neuromorphic Computing
Jonas Mensing, Wilfred G. van der Wiel and Andreas Heuer
Physical computing leverages complex dynamical systems for energy-efficient data processing. In this work, we present a neuromorphic architecture based on metallic nanoparticles interconnected by molecular junctions on a $\text{SiO}_2$/Si substrate. We demonstrate that surrounding static control electrodes transform this nanoparticle network from a passive reservoir into a tunable nonlinear dynamical system. By analyzing how these electrodes route simple one-dimensional voltage inputs into multidimensional signal responses, we establish three core design rules to maximize computational performance. First, operating near the system's cutoff frequency achieves an optimal balance between nonlinear charge tunneling and linear capacitive
memory. Second, tuning the underlying $\text{SiO}_2$ thickness sets the electrostatic screening length and dictates the
memory type. Thick
oxide layers reduce the screening length, causing networks larger than this length to transition into a persistent, non-volatile-like regime. Conversely, networks smaller than the screening length exhibit only fading
memory. Third, introducing structural disorder via heterogeneous molecular junctions overcomes inherent limits on expressivity. While a network's computational expressivity scales with its physical size, it is ultimately capped by the screening length. Breaking internal spatial
symmetries with localized disorder bypasses this saturation, allowing control voltages to independently manipulate specific signal amplitudes and phases, universally maximizing performance for dynamic neuromorphic applications.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
FeatFix: Reuse What You Verify through Local Exact-Feature Correction for Faster Cached Diffusion Inference
Hanshuai Cui, Zhiqing Tang, Zhi Yao et al. (+3 more)
Diffusion models are widely used to generate high-quality images and
videos, but their iterative denoising process remains computationally intensive. A growing class of training-free accelerators reduces this cost by reusing cached intermediate features or
forecasting future ones. To control draft drift, these methods sometimes compute an exact block feature for verification. Yet the resulting exact feature is typically used only to measure discrepancy or guide a later decision and is then discarded. We find that this previously computed feature can instead be reused for correction. Forwarding it at the verification site resets the local draft residual and reduces downstream feature error. Based on this observation, we introduce FeatFix, a local exact-feature correction method for cached
diffusion inference. FeatFix operates at a fixed sparse set of layer–timestep sites. At each selected site, it replaces the complete draft block output with the exact output computed from the same incoming state, avoiding
token- or channel-level partial replacement and full-timestep recomputation. Experiments across four image and
video backbones show that FeatFix consistently accelerates generation, achieving a speedup of up to $6.70\times$ over Vanilla while maintaining competitive output quality.
Read the full abstract ▾
All authors Hanshuai Cui, Zhiqing Tang, Zhi Yao, Qianli Ma, Fanshuai Meng, Weijia Jia
Machine Learning · Cross-listed from cs.AI
STEREODISCO: Discovering Stereotypicality in LLMs
Farane Jalali Farahani, Corina Dima, Mojtaba Nayyeri et al. (+2 more)
LLMs encode, convey, and perpetuate stereotypes. Prior computational research focuses on a small set of semantic axes investigated in social psychology, and operates on word
embeddings produced by
language models, leaving open which other semantic axes carry stereotypical associations in LLMs and how LLMs internally represent such axes. We introduce STEREODISCO, a framework that adapts the semantic differential method (Osgood et al., 1957) to the systematic study of stereotypes in LLM internal representations. STEREODISCO constructs approx. 2,000 candidate semantic axes from WordNet antonym synsets, recovers each as a
geometric axis in the LLM's activation space via probing, and identifies stereotypical axes via a statistical test over concept projections. As a case study, we apply STEREODISCO to social group stereotypes with LLAMA-3-8B-INSTRUCT and MISTRAL-7B-INSTRUCT. We find that the two LLMs agree with each other on social group ratings more than with humans, suggesting that LLM-encoded stereotype content diverges from that documented in social psychology. We also discover stereotypical axes not investigated in prior work – including humble vs. proud, narrow-minded vs. broad-minded, and cowardly vs. brave, which human annotators independently confirm.
Read the full abstract ▾
All authors Farane Jalali Farahani, Corina Dima, Mojtaba Nayyeri, Raphael H. Heiberger, Steffen Staab
Machine Learning · Cross-listed from stat.ML
Robust Estimation of Sparse Numerical Vectors under Local Differential Privacy
Puning Zhao, Zhikun Zhang, Shaowei Wang et al. (+5 more)
Local differential
privacy (LDP) protocols are vulnerable to poisoning attacks. Existing research have proposed efficient defense strategies for single-item users. However, in practice, a user may possess multiple items. The defense against poisoning attacks for multi-item users is challenging, because due to larger output spaces, the adversary can conduct more powerful attacks without being detected. In this paper, we address the robust sparse vector mean estimation problem, in which each user has a vector with $m$ nonzero coordinates. We propose Randomized Projection with Clipping (RPC). Firstly, the server sends a random binary vector to each user. The user then projects its local data on the vector, and clip the value to restrict the attacker's capability. To handle clipping bias, we propose a correction method based on a careful analysis that gives an exact expression of the bias. As a result, bias-variance tradeoff is no longer needed, thus the clipping threshold can be further reduced to shrink the output space and enhance robustness. We provide a rigorous theoretical guarantee of the estimation error under all possible attacks. Numerical experiments show that under trusted environments, our new method achieves comparable or better performance than existing methods, indicating that our method is already an efficient
estimator in its own right. Under untrusted environments, our method is also significantly more robust to poisoning attacks.
Read the full abstract ▾
All authors Puning Zhao, Zhikun Zhang, Shaowei Wang, Sheng Yue, Bangzhou Xin, Tianhang Zheng, Pengfei Zhang, Xiaochun Cao
Machine Learning
Learning-Augmented and Randomized Algorithms for Line Aggregation with Delays
Tianhang Lu, Runtian Ren, Shengcai Liu et al. (+1 more)
This paper studies learning-augmented and randomized online aggregation with delays on a line metric. We consider advice given as online suggested service lengths, and evaluate the
algorithms in terms of robustness and consistency. For each $\lambda \in (0,1]$, we first propose a deterministic learning-augmented Balance
algorithm that is $(4/\lambda+1/\lambda^2)$-robust and $(4+\lambda)$-consistent. We also propose a randomized
algorithm for the problem in the classical
adversarial model, which is $(e+1)$-competitive against an oblivious adversary, improving over the deterministic $5$-competitive Balance
benchmark~\cite{bienkowski2013chain}. Notably, this competitive ratio is even lower than the lower bound of $4$ for deterministic online
algorithms. Moreover, we establish a lower bound of $e$ on the competitive ratio of randomized online
algorithms, improving the previous lower bound of $e/(e-1)$. Besides, we combine the two ideas and obtain a randomized learning-augmented
algorithm that is $(e/\lambda+1/\lambda^2)$-robust and $(e+\lambda)$-consistent. Finally, we conduct numerical experiments to complement our theoretical analysis and evaluate the empirical performance of our
algorithms.
Read the full abstract ▾
All authors Tianhang Lu, Runtian Ren, Shengcai Liu, Ke Tang
Machine Learning
Revisiting Predictive Process Monitoring in the Age of Foundation Models: A Comparative Study of Sequence, Tabular, and LLM Approaches
Lennart Fertig, Lukas Kirchdorfer and Tobias Sesterhenn
Predictive process monitoring (PPM) leverages event logs to forecast the future of running process instances, for instance, predicting the next activity, the remaining time until case completion, or the time to the next event. While PPM research in recent years has been dominated by deep sequence models trained from scratch, such as Long Short-Term
Memory (LSTM) models, foundation-model approaches—particularly large
language models (LLMs)—are increasingly explored for PPM. At the same time, tabular foundation models with in-context learning capabilities offer a promising alternative but have not yet been systematically benchmarked for PPM. Thus, it remains unclear whether classical sequence-based models remain competitive in this evolving landscape. This paper compares the three modeling paradigms both conceptually and empirically through a controlled
benchmark across multiple
datasets and prediction tasks. The results show that sequence models consistently perform best for next activity prediction, whereas tabular foundation models are competitive on temporal tasks, with LLMs usually lagging behind despite higher cost.
Read the full abstract ▾
Machine Learning
LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts
Ken Ding
Reinforcement learning from verifiable rewards (RLVR) for mathematical
reasoning suffers from a structural blind spot: on "cliff"
prompts-those on which every sampled rollout in a group fails-the group-normalized advantage is identically zero, so GRPO produces no gradient on precisely the
prompts at the frontier of the model's capability. We introduce LoRA Scaffolded Policy Optimization (LSPO), a sampling-time mechanism that recovers this lost gradient. Each RL step, LSPO detects cliff
prompts, fits a small low-rank (LoRA) adapter by a brief supervised step on their ground-truth solutions, re-rolls the cliffs with the base-plus-adapter model, splices the now-successful completions back into the RL batch with an importance-sampling correction, and takes a GRPO step on the base alone; the adapter receives only the supervised gradient and is discarded at checkpoint, yielding a base-only model. On DeepMath-103K with DeepSeek-R1-Distill-Qwen-1.5B, evaluated over n=5 paired seeds per arm at a matched 1000-step reporting horizon, LSPO's 5-seed mean matches or beats a DAPO baseline on all 16 (
benchmark, pass@k)
cells (15 strict wins and one exact tie), with gains of up to +10.7 points on AIME24/pass@4, +6.7 points on AIME24 and AIME26 at pass@16, and +2.4 points on MATH500/pass@1; averaged over the 16
cells the improvement is +3.8 points.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Reasoning Consensus: Structural Ensembling of LLM Reasoning via Weighted DAG Aggregation
Amruta Parulekar, Jinu Lee, Dilek Hakkani-Tür et al. (+1 more)
Large
Language Models (LLMs) explore problems through chain-of-thought, but this exploration is buried in unstructured prose. On high-stakes tasks, users cannot tell which steps are well-supported, which alternatives were seriously considered, or how the final conclusion compares to those the model discarded. We propose a framework that ensembles the
reasoning structure, not just the answers, of multiple LLMs by weighted merging of Directed Acyclic Graphs (DAGs) extracted from
reasoning chains. We weight each step by how many traces independently attest to it, to return "Consensus
Reasoning". Across six
benchmarks spanning statutory interpretation, graduate-level science, narrative multi-hop
reasoning, and first-order logic, our ensemble outperforms a matched-budget majority-vote baseline, with a maximum accuracy gain of 3.1% on MuSR-MM (narrative multi-hop
reasoning). On a single model, the framework matches or exceeds self-consistency at the same trace budget while additionally exposing an inspectable consensus
reasoning graph. Ensemble weights correlate with LLM-judge rankings of
reasoning quality at Spearman $\rho = 0.30$-$0.51$, and consensus subgraphs are preferred over alternatives leading to the majority-vote answer in 54.4-65.4% of head-to-head comparisons across five of six
datasets. We observe that our framework can also be used to analyze diverse
reasoning perspectives for a problem.
Read the full abstract ▾
All authors Amruta Parulekar, Jinu Lee, Dilek Hakkani-Tür, Hari Sundaram
Machine Learning · Cross-listed from math.AP
Fractional Parabolic Partial Differential Equations in Anisotropic Spectral Barron Spaces: Regularity and Neural Approximation
Jae-Hwan Choi, Hyojae Lim, Jinsol Seo et al. (+2 more)
We study fractional
parabolic initial-value problems with lower-order drift and potential terms in anisotropic spectral Barron spaces, defined by weighted space–time
Fourier $L^1$ norms adapted to
parabolic scaling. We prove existence, uniqueness, and maximal regularity with a gain of one derivative in time and $\gamma$ derivatives in space, where $\gamma>0$ is the order of the fractional
Laplacian. The
evolution is defined only for $t\geq0$, whereas the finite-time norm requires a global extension with sufficient temporal
Fourier decay. We construct a finite reflected
semigroup extension using a Vandermonde system to match derivatives at $t=0$, obtaining temporal
Fourier estimates uniform in the
semigroup parameter. Combined with
Fourier multiplier estimates for the damped principal operator, it yields maximal regularity. Dimension-independent multiplication estimates support a finite regularity bootstrap, while interpolation and sufficient damping absorb the lower-order terms in the base estimate. The a priori estimate and the method of continuity yield maximal regularity without smallness assumptions on the lower-order coefficients. A frequency-localized counterexample shows that a uniform-in-time spatial Barron bound on the forcing does not imply the corresponding two-derivative solution bound, even for the one-dimensional heat equation. Using this regularity,
Fourier sampling yields $n^{-1/2}$ approximation rates for the solution in mixed space–time Sobolev norms using shallow networks with suitable activations. Sampling in a product
Hilbert space yields a population-level PINN consistency estimate for shallow cosine networks on a bounded cylinder. There exists a single width-$n$ network for which the sum of the squared mixed-Sobolev solution error, the squared $L^2$-norm of the residual for the whole-space fractional equation, and the squared initial-data error is $O(n^{-1})$.
Read the full abstract ▾
All authors Jae-Hwan Choi, Hyojae Lim, Jinsol Seo, Young-Jin Sim, Changhoon Song
Machine Learning
RIPPLE: Generating Multi-Channel Phase, Not Recovering It
Jaehyuk Lee, Yeajin Lee, Dayeon Shin et al. (+1 more)
Generative models synthesize magnitude spectra with high fidelity, while phase is delegated to a recovery module—Griffin–Lim, a vocoder, or a latent decoder—applied independently to each channel. For multi-channel waveforms this delegation is costly: the physical content of spatial audio and three-component seismograms lives in the phase relationships between channels, precisely what channel-independent recovery cannot produce. The cost is also invisible, since the magnitude-based metrics common to both fields barely move when inter-channel phase coherence collapses—so a pipeline can discard the physical information in its output while still scoring well. We argue that phase should be generated, not recovered, and present RIPPLE (Rectified Inter-channel Phase with Prior-based LEarning), which reinterprets Griffin–Lim as a phase **prior** rather than a final
estimator: initialized from the source phase, this prior carries the inter-channel structure to be preserved, and a rectified flow refines it toward the target under an explicit inter-channel phase loss. Tested on first-order ambisonics environment transfer and seismic cross-station translation—two physically unrelated domains—RIPPLE outperforms recovery-based pipelines on the coherence metrics that downstream analyses consume. The seismic case is decisive: across architecturally distinct generators, per-channel recovery leaves S-wave polarization error near the $57.3^\circ$ random expectation, whereas learned phase reduces it to $33.8^\circ$.
Read the full abstract ▾
All authors Jaehyuk Lee, Yeajin Lee, Dayeon Shin, Donghun Lee
Machine Learning
Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold
Songshuo Lu, Zhi Chen and Yaohua Tang
A single reinforcement-learning run can produce a strong reasoner yet an incomplete teacher: it often amplifies only a subset of the valid solution modes. We argue that reinforcement learning (RL)-trained policies should therefore be viewed as local probes of a multi-basin
reasoning solution
manifold, rather than as globally reliable supervisors. Based on this view, we propose an expand-then-compress framework that couples teacher construction with multi-teacher policy
distillation. In the expansion stage, Residual Group Relative Policy Optimization (RGRPO) trains a sequence of teachers from a common initialization and redirects each later round toward examples not yet covered by the accumulated teacher union. In the compression stage, reliability-gated Teacher-Union On-policy
Distillation (TU-OPD) lets the student learn from its own response prefixes. For each example, only reliable teachers contribute, and their sampled-
token OPD losses are weighted by their per-example quality. We further introduce Consensus-Residual Decomposition, which preserves a winner teacher's excess
token preferences over its reliable peers, preventing specialist behavior from being suppressed during teacher aggregation. Experiments on mathematical
reasoning, code generation, and instruction following show that the resulting Qwen3-1.7B student consistently outperforms the strongest individual teacher across all three domains, yielding relative improvements of 2.0%, 8.3%, and 6.9%, respectively, while retaining single-model inference. These results establish a simple but powerful principle: stronger students can be obtained not by selecting a single better teacher, but by deliberately constructing and compressing a complementary teacher union.
Read the full abstract ▾
Machine Learning
Train Small, Deploy Large: Zero-Shot GNN Transfer Through Geometric Renormalization
Robert Jankowski, Pedro Almagro-Blanco, Marián Boguñá et al. (+2 more)
Graph neural networks (GNNs) can operate on large graphs but become infrastructure-sensitive at the scale of millions of nodes and typically require scalable training techniques for even larger graphs. This raises a central question: when can a model trained on a smaller, scaled-down replica of a graph be deployed on the full-resolution graph without retraining? We introduce a zero-shot transfer protocol in which a GNN is trained on a graph coarse-grained by
geometric renormalization (GR), and the resulting weights are transferred directly to the original network. Across synthetic and real-world networks, training on GR scaled-down replicas preserves much of the original-scale predictive performance while significantly reducing training cost. We further find that learned representations and predictive trajectories remain aligned across scales. These findings suggest that structural similarity may be more important than network size in determining GNN transferability, opening a path toward scale-equivariant graph architectures.
Read the full abstract ▾
All authors Robert Jankowski, Pedro Almagro-Blanco, Marián Boguñá, Melanie Weber, M. Ángeles Serrano
Machine Learning · Cross-listed from cs.CL
Gradient-free Task-Conditioned Retrieval for On-Device In-Context Learning
Xinyu Luo, Hui Liu, Yihua Shao et al. (+3 more)
On-device in-context learning (ICL) relies on pre-inference
retrieval to select demonstrations for useful context before downstream model inference. This
retrieval must exploit task-specific information while operating over local
memories under limited computation,
memory, and data-exposure budgets. We propose Conditional
Retrieval Alignment (CoRA), a gradient-free framework that converts a frozen encoder into a task-conditioned retriever using paired candidate inputs and outputs. CoRA selects complementary encoder layers, constructs an output-derived conditioning space from candidate
memory, and aligns candidate input representations to this space through closed-form ridge regression. Low-rank factorization then produces a compact
retrieval basis where candidate outputs are used only during offline index construction, whereas query-time
retrieval requires only the query input and precomputed index. We show that CoRA's rank-constrained basis is the optimal low-rank compression of the output-conditioned fitted representation, and derive an exact two-pass streaming construction that avoids materializing the full fitted
matrix. We further extend the framework to
multimodal exemplar
retrieval by incorporating visual representations into the conditioning and
retrieval spaces. Experiments across ten textual
datasets and four
multimodal benchmarks with Llama-3.2-1B, MobileLLM-Pro, OpenFlamingo-3B, and Qwen3.5-2B, as well as end-to-end Raspberry Pi~5 deployment demonstrate that CoRA supports effective task-conditioned
retrieval without retriever fine-tuning, backpropagation, or target-model calls.
Read the full abstract ▾
All authors Xinyu Luo, Hui Liu, Yihua Shao, Junyi Yang, Arindam Basu, Haoliang Li
Machine Learning · Cross-listed from cs.CV
DS@GT ARC at ImageCLEFmedical 2026: Architectural Diversity for Concept Detection and Foundation-Model Scaling for Caption Prediction in Medical Image Analysis
Bowen Wang, Youwen Zhang and Ritesh Mehta
We describe the DS@GT submissions to the ImageCLEFmedical Caption 2026 challenge, which continues a long-running
benchmark on the ROCOv2
dataset with two tracks: Concept Detection (Task 1), assigning UMLS Concept Unique Identifiers (CUIs) to radiology images, and Caption Prediction (Task 2), generating natural-
language captions. For Task 1, our primary submission was a three-way late-fusion ensemble of ConvNeXt-V2, BiomedCLIP ViT-B/16, and DenseNet-169 with a regularized ”Honest Threshold Tuning” procedure designed to avoid validation overfitting on rare concepts; this submission ranked first on the official submission with a primary $F_1$ of $0.5790$ and a secondary $F_1$ of $0.9657$. In parallel, we submitted a training-free KNN
retrieval pipeline over frozen BiomedCLIP
embeddings, which reached a primary $F_1$ of $0.5780$ and a secondary $F_1$ of $0.9599$-essentially matching the fine-tuned ensemble on the primary track at a fraction of the cost. For Task 2, our submissions included a fine-tuned Gemma-3 27B model (overall $0.3571$, ranking third in the official submission), a fully fine-tuned BLIP pipeline with custom Vizwins merging ($0.3564$), and a zero-shot MedGemma-4B run with a PubMed-style
prompt ($0.3186$), spanning a wide range of model scales and training costs. Code: https://github.com/dsgt-arc/imageclef-caption-2026.
Read the full abstract ▾
Machine Learning
DAS-PMVC: A Framework for Partial Multi-View Clustering via Dual Alignment and Structure Enhancement
Shubin Ma, Liang Zhao, Chuanye He et al. (+4 more)
In recent years, multi-view clustering has attracted widespread research interest. However, due to limitations in data collection devices, data across different views often suffer from misalignment, leading to the partial view
alignment problem (PVAP). To mitigate the impact of view asymmetry and irrelevant samples, this paper proposes a framework for partial multi-view clustering via dual
alignment and structure enhancement (DAS-PMVC), which leverages view structure consistency and semantic relevance. Specifically, DAS-PMVC includes three parts: anchor graph structure
alignment, where sample joint
embedding representations with consistent latent space are derived from anchor point relationships for initial view
alignment; structure-enhanced feature learning, where the model learns view structure information through pretraining and combines multi-view graph convolutional networks to further extract deep latent features from the aligned graph structure to improve the discriminative power of representations; and a dual
alignment strategy, where initial
alignment is performed through the anchor graph in the pretraining phase, and contrastive learning loss and the Hungarian
algorithm are introduced in the training phase to further optimize the
alignment of latent features. Experimental results on various
datasets demonstrate that the DAS-PMVC framework outperforms existing state-of-the-art methods in clustering performance, showcasing its effectiveness and superiority.
Read the full abstract ▾
All authors Shubin Ma, Liang Zhao, Chuanye He, Zhenjiao Liu, Liang Zou, Lin Yuanbo Wu, Yu Shao
Machine Learning
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
Yuxin Chen, Liang Luo, Buyun Zhang et al. (+44 more)
Modern recommendation models gain prediction quality by scaling feature-interaction and sequence modules, but production cost constraints cap how far systems can scale. In this work, we propose Request-Oriented Compute Sharing (ROCS), a modeling and inference paradigm that exploits a unique property of recommendation inference: each user request is evaluated against many candidates, while request-side features are shared across candidates. ROCS defers request-candidate interactions as late as possible, isolates candidate-dependent representations, and evaluates substantial portions of the model once per request rather than once per candidate, significantly improving inference efficiency while maintaining or improving prediction quality. To realize this paradigm, we develop Generalized Layer Masking (GLM) to enforce candidate isolation in feature-interaction architectures, and Deep Cross
Attention (DCA) to extend request-oriented sharing to sequence architectures. To support efficient GPU deployment, we co-design In-Kernel Broadcast Optimization (IKBO) that significantly accelerates ROCS model execution. Experiments on public
benchmarks show that ROCS consistently improves the quality-efficiency tradeoff across recommendation backbones. On production-scale workloads, ROCS achieves up to a 3x QPS improvement on
retrieval models without quality degradation and a 0.5% relative LogLoss improvement with a 50% QPS gain on a short-form
video ranking model. ROCS has been deployed across large-scale recommendation systems spanning ads and organic surfaces,
retrieval and ranking stages, and more than two orders of magnitude in inference complexity, delivering significant online gains at reduced infrastructure cost.
Read the full abstract ▾
All authors Yuxin Chen, Liang Luo, Buyun Zhang, Jian Jiao, Boda Li, Haoyu Wang, Tongyi Tang, Ao Cai, Zijian Shen, Zhengkai Zhang, Wenyi Xie, Ryan Dick, Han Liu, Neng Shi, Bin Yu, Jianbo Xiao, Shuyao Bi, Hongtao Yu, Yuanwei Fang, Zhuoran Zhao, Sijia Chen, Yang Chen, Shuqi Yang, Qianru Li, Zikun Liu, Wei Ling, Sihan Zeng, Longhao Jin, Jiaxin Lu, Yinbin Ma, Jiawei Li, Yichen Ruan, Yong Ler Lee, Birmingham Guan, Zijian Li, Jianbo Sun, Zhengyu Zhang, Zeliang Chen, Xiaohan Wei, Yuchen Hao, GP Musumeci, Venkatesh Ranganathan, Yantao Yao, Chunqiang Tang, Wenlin Chen, Santanu Kolay, Ellie Dingqiao Wen
Machine Learning
Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teachers
Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi et al. (+1 more)
Deep neural networks (DNNs) have achieved remarkable success in classical machine learning problems. However, they are known to be vulnerable to
adversarial attacks. Countermeasures proposed in the literature, notably Information Bottleneck
Distillation (IBD) introduced by Kuang et al., degrade the classification accuracy on clean inputs while improving the robustness to
adversarial inputs. In this work, we extend the IBD framework by introducing an extra teacher model (clean teacher) trained with only clean inputs, into the
distillation process from a robust teacher model trained by
adversarial training. The features of both clean and robust teachers are transferred to the student through a cross-layer
attention matrix. Experimental results on the CIFAR-10 and CIFAR-100
datasets show that the proposed method improves classification accuracy on clean samples compared to the original IBD, while maintaining similar accuracy on
adversarial samples. Furthermore, our methods are competitive with state-of-the-art approaches, including the recent dual-teacher
distillation framework B-MTARD, particularly in terms of the harmonic mean between clean and robust accuracy. We also analyze the impact of different training settings that have different influences on the
attention module.
Read the full abstract ▾
All authors Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi, Kave Salamatian