Machine Learning · Cross-listed from cs.AI
RewardHarness: Self-Evolving Agentic Post-Training
Yuxuan Zhang, Cong Wei, Penghui Du et al. (+11 more)
Aligning generative models with human preferences remains a central challenge. While RLHF is widely adopted, it relies on large-scale preference data. This limitation is particularly evident in evaluating instruction-guided image edits, even though humans can often learn such preferences from only a few examples. We present RewardHarness, a training-free, self-evolving agentic reward framework that reframes reward modeling as context
evolution rather than weight optimization. Instead of learning from large-scale annotations, RewardHarness aligns with human preferences by iteratively evolving a library of tools and skills from as few as 100 preference demonstrations. Given a source image, an edited image, and an editing instruction, an orchestrator
agent selects the most relevant subset of tools and skills from the maintained library, and sub-
agents invoke them to construct a
reasoning chain that produces a reward score. By comparing predicted scores with ground-truth preferences and analyzing successes and failures in the
reasoning process, the orchestrator automatically refines its library of tools and skills without additional human annotation. Using only 0.05% of the EditReward preference data, RewardHarness achieves 47.4% average accuracy on EditReward-Bench K=2/3/4 and GenAI-Bench, surpassing GPT-5 by 5.3 points. When used as a reward signal in FlowGRPO, RL-tuned models achieve 3.52 on ImgEdit-Bench. Code and model are available at the project page: https://rewardharness.com/
Read the full abstract ▾
All authors Yuxuan Zhang, Cong Wei, Penghui Du, Bo Li, Junwen Miao, Huaisong Zhang, Songcheng Cai, Yubo Wang, Dongfu Jiang, Yuyu Zhang, Changqian Yu, Ping Nie, Wenhu Chen, Kelsey R. Allen
Machine Learning
METBRA25Y: Brazil Surface Meteorology Archive with Harmonized Variables and Quality Control
Matheus Lima Castro, William Dantas Vichete and Leopoldo Lusquino Filho
This data paper describes METBRA25Y, a harmonized archive of hourly surface meteorological observations from Brazil derived from public historical records of the Instituto Nacional de Meteorologia (INMET). The
dataset was designed to support reproducible environmental, climatological, hydrological, agricultural, urban-risk, and machine-learning studies that require station-level meteorological time series with standardized variable names and explicit quality-control metadata. The processing workflow ingests annual INMET archives, parses station metadata from raw file headers, normalizes heterogeneous Portuguese column names into a canonical schema, constructs hourly timestamps, consolidates observations by city and station, and exports compressed CSV files together with station manifests, per-station quality flags, daily precipitation aggregates, variable-level failure summaries, and missing-data audits. The quality-control protocol follows a two-stage strategy: first, physically implausible values are converted to missing values and flagged; second, temporal and cross-variable consistency checks generate diagnostic flags without necessarily overwriting the original measurements. The resulting package covers observations between 2000 and 2025, with stationspecific temporal coverage, and includes key meteorological variables such as precipitation, air temperature, dew point, relative humidity, atmospheric pressure, wind speed, wind gust, wind direction, and global
solar radiation. Based on the summary files included in the current release snapshot, the archive contains 616 unique station codes across variable summaries, of which 605 have coordinates within a broad Brazil plausibility envelope. This paper documents the
dataset provenance, file organization, harmonized schema, quality-control rules, technical validation outputs, limitations, and recommended usage practices.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Supersampling Stable Diffusion and Beyond: Training-Free Neural Network Scaling via Signal Interpolation
Md Abu Obaida Zishan, Jannatun Noor and Annajiat Alim Rasel
This work presents a training-free approach for interpolating neural networks using signal interpolation methods (i.e., linear, bilinear, bicubic). We discovered this phenomenon while attempting to interpolate the convolution layers of Stable-
Diffusion-1.5 for beyond-training-resolution (BTR) image generation. As we know, for generating BTR images using DDPMs (Denoising
Diffusion Probabilistic Models), there are mainly two approaches, namely, 1) patch-wise
diffusion and 2) convolution-kernel dilation. Patch-wise
diffusion cannot solve the object-duplication problem efficiently, while the kernel dilation approach may be harder to fine-tune later. Thus, we proposed convolution-kernel interpolation of the Stable
Diffusion 1.5 model for $4\times$ BTR image generation. We discover and show mathematically that signal interpolation methods can correctly scale convolution kernels when multiplied by a flat-attenuation factor, while achieving competitive performance against the state of the art. We then applied the same methodology to non-DDPM deep neural network architectures, adapting them for BTR image classification, achieving a worst-case performance drop of $\sim 7\%$ on the ImageNet-1k validation set. Again, we subsample the penultimate fully connected layer of VGG-16 using our method, reducing up to $\sim 9\%$ parameters with $\sim 5\%$ loss in performance. Finally, we also discuss how we can reduce the
memory footprint of training neural networks to $4\times$ or greater.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Structured Recurrent Mixers for Massively Parallelized Sequence Generation
Benjamin L. Badger
Over the last two decades,
language modeling has experienced a shift from the use of predominantly recurrent architectures that process
tokens sequentially during training and inference to non-recurrent models that process sequence elements in parallel during training, which results in greater training efficiency and stability at the expense of lower inference throughput. Here we introduce the Structured Recurrent Mixer, an architecture that allows for algebraic conversion between a sequence parallel representation at train time and a recurrent representation at inference, notably without the need for specialized kernels or device-specific
memory management. We show experimentally that this dual representation allows for greater training efficiency, higher input information capacity, and larger inference throughput and concurrency when compared to other linear complexity models. We postulate that recurrent models are poorly suited to extended sequence length scaling for information-rich inputs typical of
language, but are well suited to scaling in the sample (batch) dimension due to their constant
memory per sample. We provide Mojo/MAX inference implementations of SRMs exhibiting 12x the throughput and 170x the concurrency of similarly powerful
Transformers inferenced on vLLM, increases characteristic of Pytorch implementations resulting in a 30% increase in compute-constant GSM8k Pass@k. We conclude by demonstrating that SRMs are effective reinforcement learning training candidates.
Read the full abstract ▾
Machine Learning
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
Beshr IslamBouli and David Jin
Post-training weight-only quantization to 4 bits is widely used to reduce the
memory and compute costs of large
language model inference. Existing PTQ methods, such as AWQ and GPTQ, improve how weights are mapped onto a fixed 4-bit grid through scaling, clipping, or error compensation. To further improve accuracy, methods such as OmniQuant and QuIP\# uses gradient-assisted
algorithms at the cost of hours of quantization time. In this work, we propose AAAC (Activation-Aware Adaptive Codebooks), a lightweight method for 4-bit LLM weight quantization. AAAC replaces the fixed scalar codebook used in standard quantization with two small learned scalar codebooks (64 bytes) per layer. Each group of weights selects the codebook that minimizes activation-weighted reconstruction error, encoding the choice in the unused sign bit of the group's positive scale and adding zero storage overhead. AAAC completes in 3–30 minutes on a single GPU, and adds no
memory beyond the model itself. We evaluate against AWQ, GPTQ, IF4, GPTVQ, OmniQuant, SqueezeLLM, and QuIP\# across model families. AAAC outperforms baselines at orders-of-magnitude less quantization time.
Read the full abstract ▾
Machine Learning
Structure-Centric Graph Foundation Model via Geometric Bases
Xiaodong He, Haolan He, Ruiyi Fang et al. (+2 more)
Graph foundation models (GFMs) seek transferable representations across graph domains but are limited by structural heterogeneity and incompatible node feature spaces. We propose Structure-Centric Graph Foundation Models (SCGFM), which treat graph
topology as the primary source of transferable knowledge. Modeling graphs as metric measure spaces, SCGFM introduces learnable
geometric bases that define a shared structural coordinate system. Graphs are aligned to these bases via Gromov-Wasserstein distances, yielding structure-aligned latent representations that accommodate heterogeneous graph
topologies. To address feature incompatibility, SCGFM employs a structure-aware feature re-encoding mechanism that unifies node representations without assuming a fixed feature dimensionality or requiring
dataset-specific preprocessing. Experiments on graph- and node-level tasks demonstrate strong in-domain and cross-domain generalization, outperforming existing GFM approaches.
Read the full abstract ▾
All authors Xiaodong He, Haolan He, Ruiyi Fang, Ming Sun, Zhao Kang
Machine Learning
Event Fields: Learning Latent Event Structure for Waveform Foundation Models
Li Na, Yuanyun Zhang and Shi Li
We propose a new class of waveform foundation models that departs from conventional sequence based representations by modeling physiological time series as realizations of latent event processes. Rather than treating signals as collections of local
tokens or patches, our approach assumes that clinically meaningful structure arises from temporally extended, interacting events whose boundaries and dynamics are not directly observed. To capture this structure, we introduce a self supervised learning framework that enforces consistency across
stochastic segmentations and time frequency projections of the same waveform, encouraging representations that are invariant to signal level perturbations while preserving event level organization. The resulting model combines a
segmentation aware encoder with a latent interaction operator that captures dependencies among inferred events, and naturally extends to
multimodal settings by aligning modalities through shared event representations. Across a range of physiological
benchmarks, including arrhythmia classification, hemodynamic prediction, and waveform
retrieval, the proposed method improves performance, robustness, and label efficiency relative to strong sequence based baselines. These results suggest that shifting from signal centric to event centric representations provides a more appropriate inductive bias for modeling physiological dynamics and offers a complementary path to scaling foundation models in healthcare.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Core-Halo Decomposition: Decentralizing Large-Scale Fixed-Point Problems
Haixiang, Yang Xu, Jiefu Zhang et al. (+4 more)
We study solving large-scale fixed-point equation \(x^\star=\bar F(x^\star)\) with decomposition. Standard strict decomposition assigns each
agent a disjoint block and evaluates updates using only owned coordinates. For most operators, however, a block update may depend on variables outside the block. Truncating these dependencies by strict decomposition changes the mean operator and creates structural bias that cannot be removed by more samples, smaller stepsizes, or additional consensus. We therefore propose Core-
Halo decomposition, which separates write ownership from read-only evaluation context: each
agent updates its own core and reads from an overlapping
halo. By aligning the Core-
Halo decomposition with the block-dependence structure of $\bar F$, the original fixed-point problem can be implemented faithfully in a decentralized multi-
agent system. We further characterize the fundamental obstruction faced by strict decomposition through a Bellman closure condition and a blockwise bias lower bound, showing that local-only updates can alter the original fixed-point operator. Finally, we conduct extensive experiments across a range of application settings, and demonstrate that Core-
Halo achieves near-centralized performance while retaining the parallelism benefits of decentralization.
Read the full abstract ▾
All authors Haixiang, Yang Xu, Jiefu Zhang, Xudong Wu, Zihan Zhou, Jun He, Jiayu Chen
Machine Learning · Cross-listed from cs.SE
Semantic Voting: Execution-Grounded Consensus for LLM Code Generation
Shan Jiang, Zijian Yi and Chenguang Zhu
LLM code-generation pipelines often sample multiple candidates and select one final answer without access to a complete oracle. Existing pipelines mix textual voting, ranking, and execution-based agreement, but the relative contribution of each component remains unclear. We study 18 configurations across different models, thinking levels, and
benchmarks, comparing output-pattern majority voting, weighted voting, MBR-Exec, and SemanticVote - a method that clusters candidates by execution fingerprints on LLM-generated inputs. Three findings emerge. (1) The best execution-based selector exceeds output-pattern majority voting by 19-52 percentage points on every configuration, with every execution-based selector exceeding it by at least 18 points. (2) Once candidates are executed on diverse inputs, aggregation rule has limited effect: SemanticVote, weighted voting, and MBR-Exec are statistically indistinguishable across all 18 configurations. The largest factor is input quality: sketch-based input generation consistently outperforms direct LLM generation by 0.6-2.1 pp and random fuzzing by up to 11.3 pp. (3) Thinking level interacts differently with selection families: deeper thinking improves majority voting by 12 pp but execution-based methods stay flat or degrade as candidate diversity falls. These results frame inference-time code selection as a signal-quality problem rather than an aggregation-rule problem: when oracles are unavailable, the behavioral evidence matters more than the aggregation rule.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.SI
Attention-based graph neural networks: a survey
Chengcheng Sun, Chenhao Li, Xiang Lin et al. (+4 more)
Graph neural networks (GNNs) aim to learn well-trained representations in a lower-dimension space for downstream tasks while preserving the topological structures. In recent years,
attention mechanism, which is brilliant in the fields of natural
language processing and computer
vision, is introduced to GNNs to adaptively select the discriminative features and automatically filter the noisy information. To the best of our knowledge, due to the fast-paced advances in this domain, a systematic overview of
attention-based GNNs is still missing. To fill this gap, this paper aims to provide a comprehensive survey on recent advances in
attention-based GNNs. Firstly, we propose a novel two-level taxonomy for
attention-based GNNs from the perspective of development history and architectural perspectives. Specifically, the upper level reveals the three developmental stages of
attention-based GNNs, including graph recurrent
attention networks, graph
attention networks, and graph
transformers. The lower level focuses on various typical architectures of each stage. Secondly, we review these
attention-based methods following the proposed taxonomy in detail and summarize the advantages and disadvantages of various models. A model characteristics table is also provided for a more comprehensive comparison. Thirdly, we share our thoughts on some open issues and future directions of
attention-based GNNs. We hope this survey will provide researchers with an up-to-date reference regarding applications of
attention-based GNNs. In addition, to cope with the rapid development in this field, we intend to share the relevant latest papers as an open resource at https://github.com/sunxiaobei/awesome-
attention-based-gnns.
Read the full abstract ▾
All authors Chengcheng Sun, Chenhao Li, Xiang Lin, Tianji Zheng, Fanrong Meng, Xiaobin Rui, Zhixiao Wang
Machine Learning
MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI
Bohan Lyu, Yucheng Yang, Siqiao Huang et al. (+25 more)
Modern AI progress has been driven by ML methods that are generalizable across settings and scalable to larger regimes. As large
language models demonstrate advanced capabilities in
reasoning, coding, and engineering tasks, it is increasingly important to understand whether they can discover such methods rather than only apply existing ones. We introduce MLS-Bench, a
benchmark for evaluating whether AI systems can invent generalizable and scalable ML methods. MLS-Bench contains 140 tasks across 12 domains, each requiring an
agent to improve one targeted component of an ML system or
algorithm and demonstrate that the improvement generalizes across controlled settings and scales. We find that current
agents remain far from reliably surpassing human-designed methods, and that engineering-style tuning is easier for them than genuine method invention. We further study the effects of test-time scaling, adaptive compute allocation, and context provision on
agents' discovery performance, together with case studies of their behavior. Our analyses suggest that the bottleneck is not only in proposing new methods, but also in the scientific insight needed to plan, validate, and scale claims about them. More search, compute, or context alone does not remove this bottleneck. We build and maintain a community platform for cumulative and comparable iteration, and release the data and code at https://mls-bench.com.
Read the full abstract ▾
All authors Bohan Lyu, Yucheng Yang, Siqiao Huang, Jiaru Zhang, Qixin Xu, Xinghan Li, Xinyang Han, Yicheng Zhang, Huaqing Zhang, Runhan Huang, Kaicheng Yang, Zitao Chen, Wentao Guo, Junlin Yang, Xinyue Ai, Wenhao Chai, Yadi Cao, Ziran Yang, Kun Wang, Dapeng Jiang, Huan-ang Gao, Shange Tang, Chengshuai Shi, Simon S. Du, Max Simchowitz, Jiantao Jiao, Dawn Song, Chi Jin
Machine Learning
PHIDA: Persistence-Guided Node-to-Cluster Mapping for Online Clustering
Naoki Masuyama, Yusuke Nojima, Stefan Wermter et al. (+3 more)
Online clustering methods that adaptively create and update nodes as data arrive often make node learning explicit, whereas the mapping from the learned node state to output clusters often remains implicit or simplified. Implicit mappings make output clusters sensitive to weak graph bridges or local relations based on distance in the graph over learned nodes, leaving no explicit constraint on which node groups remain intact during mapping. This paper addresses this gap by proposing PHIDA, a persistence-guided node-to-cluster mapping method for online clustering with learned nodes. PHIDA implements this mapping within Adaptive Resonance Theory (ART)-based online clustering by combining Inverse-Distance ART (IDA) node learning with node-to-cluster mapping constrained by Persistent Homology (PH). Experiments on 24
benchmark datasets show that PHIDA achieves the best average ranks in stationary comparisons that include the recent stationary-only clustering methods, while also improving aggregate performance in the nonstationary setting over the evaluated online methods that adaptively create and update nodes. Ablations and comparisons with conventional node-to-cluster mappings indicate that the observed gains are associated with PH-constrained mapping that preserves raw PH components, together with the use of the PH component view during node learning. Source code is available at https://github.com/Masuyama-lab/PHIDA
Read the full abstract ▾
All authors Naoki Masuyama, Yusuke Nojima, Stefan Wermter, Yuichiro Toda, Hisao Ishibuchi, Chu Kiong Loo
Machine Learning
The Cancellation Hypothesis in Critic-Free RL: From Outcome Rewards to Token Credits
Tianhao Cheng, Zeyu Huang, Zihan Qiu et al. (+5 more)
A commonly accepted explanation of critic-free RL for LLMs, based on sequence-level rewards, is that it reinforces successful rollouts with a positive advantage while penalizing failed ones. In contrast, we study critic-free RL from a
token-level perspective, revealing the
token-flipping phenomenon: positive and negative rollouts exhibit remarkably similar proportions of
tokens whose probabilities are boosted or suppressed during RL training. To explain this phenomenon, we further show that a
token's change in probability is not fully determined by its own advantage; coupled gradient interactions with other
tokens also play a non-negligible role. Specifically, these
token coupling effects occur primarily between identical
tokens that are both predicted with low confidence. Building upon this analysis, we propose the cancellation hypothesis: as a result of coupling, opposing signals cancel out for
tokens shared by positive and negative rollouts, while
tokens more specific to successful rollouts receive stronger reinforcement, thereby inducing hidden
token-level credit assignment from rollout-level rewards. We support this hypothesis with complementary empirical evidence. (1) Compared with training on only positive rollouts, critic-free RL shifts updates from template and formatting
tokens toward
reasoning tokens; (2)
Tokens boosted by critic-free RL consistently demonstrate higher value than suppressed
tokens, regardless of whether they originate from positive or negative rollouts. Guided by this view, we implement two batching interventions to encourage or preserve cancellation in critic-free RL training: query-preserved mini-batching and reward-balanced batching. Despite their simplicity, these interventions improve RLVR training across multiple model scales, supporting cancellation as both an explanatory principle and a practical design criterion for critic-free RL training.
Read the full abstract ▾
All authors Tianhao Cheng, Zeyu Huang, Zihan Qiu, Yu Cheng, Edoardo Ponti, Yinghui Xu, Ivan Titov, Zenglin Xu
Machine Learning
Optimised Support Vector Regression for California Housing Price Prediction: The Critical Role of Feature Engineering and Hyperparameter Tuning
Emmanuel Adutwum
In the recent literature, Support Vector Regression (SVR) has been cited as one of the weakest performers on the California Housing
benchmark dataset, with Preethi et al. (2025)specifically ranking it last among the
algorithms they tested, reporting an R2 of only 0.60. This paper examines whether the previously reported performance reflects experimental configuration choices rather than an inherent algorithmic limitation. A structured experimental workflow is applied: ten domain-motivated derived features are constructed from the eight raw inputs, an exploratory ensemble feature importance analysis identifies the most predictive candidates, and a randomised search over hyperparameter combinations with three-fold cross-validation selects the optimal SVR configuration within a leakage-safe scikit-learn Pipeline. A formal four-stage ablation study isolates the contribution of each component: scaling alone accounts for +0.744 in R2 (from -0.054 to 0.690), feature engineering adds +0.026 (to 0.716), and hyperparameter tuning contributes +0.008 (to 0.723). The resulting tuned SVR achieves a test R2 of 0.723, a 0.123-point absolute improvement over the previously reported SVR result (from 0.60 to 0.723, approximately 20% relative gain). In the ten-model comparison, the tuned SVR ranks fourth with R2 = 0.723, below XGBoost (0.832), Random Forest (0.814) and Gradient Boosting (0.783), while substantially outperforming simpler baselines. Ten-fold cross-validation yields a mean R2 of 0.703 (95% CI: [0.630, 0.775]), confirming robust generalisation. The observed improvement from R2 = 0.60 to R2 = 0.723 is associated primarily with proper feature scaling within a unified preprocessing pipeline, with domain-motivated feature engineering and systematic hyperparameter tuning, providing further incremental gains.
Read the full abstract ▾
Machine Learning
Sketch-and-Verify: Structured Inference-Time Scaling via Program Sketching
Shan Jiang, Zijian Yi and Chenguang Zhu
SKETCHVERIFY is a within-tier cost-performance policy, not a universal accuracy improvement. The operational question: a practitioner stuck with a small, cheap code model (here, Gemini 3.1 Flash Lite) for latency, deployment, or budget reasons – how should they spend a small amount of extra test-time compute? SKETCHVERIFY factorizes the search space: the LLM enumerates K distinct algorithmic strategies, writes a program sketch for each (a partial program with ?? holes), and fills each sketch M times, producing K x M structurally diverse candidates that are verified by execution and selected by fingerprint clustering. Each extra sketch is guaranteed to explore a different
algorithm; each extra flat sample likely duplicates an existing one. Our central evidence is a cost-quality Pareto plot on HumanEval+ across three Gemini tiers (Lite, Flash, Pro), and a reanalysis of the 19 problems where Lite greedy fails. Two findings: (1) Within-tier, sketching dominates flat sampling at matched candidate count. On the hard subset, Lite Sketch K=2, M=5 recovers 11/19 (58%) vs. flat N=10 at 5/19 (26%, +32pp); Lite Sketch K=10, M=10 recovers 15/19 (79%) vs. flat N=100 at 10/19 (53%, +26pp). Flat cannot close the gap even at ~3x the budget: flat N=50 still loses to Sketch K=2, M=5 by +11pp. (2) Cross-tier, sketching does not replace upgrading. Pro greedy (89%) dominates Lite Sketch K=10, M=10 (79%) on both pass@1 and dollar cost. Practitioner rule: if a stronger tier is available, use greedy on it; otherwise sketching is the cost-effective way to spend extra compute. We characterize the K-vs-M trade-off via a Flash Lite scaling sweep, report HumanEval+ saturation on Flash and Pro, and show the method composes cleanly with execution-based selection from the concurrent Semantic Voting line of work.
Read the full abstract ▾
Machine Learning
Fitting Multilinear Polynomials for Logic Gate Networks
Youngsung Kim
We study learnable logic gate networks that stack layers of 2-input Boolean gates to build combinational
circuits. Every 2-input gate has a unique multilinear
polynomial with 4 coefficients, so the 16 Boolean gates form a codebook of prototypes in a 4-dimensional space, reducing training to a vector-quantization problem. The baseline method, Soft-Mix, learns a 16-dimensional softmax over gate identities, but the codebook has rank~4: 11 of 15 simplex directions carry nullspace gradient, and at uniform initialization the backward signal vanishes exactly. We prove that no affine product reparameterization fixes the resulting interaction-coefficient starvation under STE, and show that the covariance Jacobian of soft-VQ selection bypasses it by coupling the starved coefficient to the always-active constant channel. Working in the 4-dimensional
polynomial space reduces each
neuron from 16 to 4 parameters. On seven
datasets, at least one 4-parameter method matches or exceeds Soft-Mix on every
dataset; the CovJac advantage over STE grows monotonically with interaction demand across all seven
datasets. At depth, Soft-Mix collapses ($-37.3$pp on CIFAR-10 at 12 layers) while CovJac holds ($-0.5$pp on CIFAR-10, stable on MNIST).
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
Privacy-Aware Video Anomaly Detection through Orthogonal Subspace Projection
Lei Wang, Wenxiang Diao, Andrew Busch et al. (+2 more)
Video anomaly detection (VAD) systems often prioritize accuracy while overlooking
privacy concerns, limiting their suitability for real-world deployment. We propose the Orthogonal Projection Layer (OPL), a lightweight module that removes task-irrelevant variations to produce representations focused on anomaly-relevant cues. To address
privacy risks in human-centered scenarios, we introduce Guided OPL (G-OPL), which suppresses facial attributes using weak supervision from face-presence signals while preserving non-identifying features such as pose and motion. A cosine
alignment objective enforces consistent capture and removal of facial information without identity labels or
adversarial training. We further present a
privacy-aware evaluation framework that jointly assesses detection performance and
privacy preservation, and enables analysis of how sensitive information is filtered. Experiments show that
embedding privacy constraints into model design reduces sensitive information while maintaining or improving detection accuracy, supporting projection-based architectures as a principled approach for
privacy-aware VAD.
Read the full abstract ▾
All authors Lei Wang, Wenxiang Diao, Andrew Busch, Jun Zhou, Yongsheng Gao
Machine Learning
FLUX: Geometry-Aware Longitudinal Flow Matching with Mixture of Experts
Josue Ortega Caro, Yongxu Zhang, Hannah M Batchelor et al. (+3 more)
Many biological systems evolve through continuous local dynamics while switching between latent regimes defined by learning, stimulus context, internal state, or developmental stage. These processes are often observed only as unpaired longitudinal snapshots: the same
cells,
neurons, or animals are not tracked as matched trajectories, even though population states are sampled across successive stages. This creates two coupled challenges. First, trajectories must respect curved low-dimensional
manifolds embedded in high-dimensional biological measurements. Second, the model must identify when the transport mechanism itself changes. We introduce FLUX (FLow matching for Unpaired longitudinal data with miXture-of-experts), a geometry-aware longitudinal flow-matching framework for joint transport modeling and unsupervised regime discovery. FLUX learns a data-dependent metric from pooled labeled and unlabeled observations, uses that metric to construct geometry-aware conditional paths between adjacent marginals, and decomposes the resulting velocity field into sparse expert vector fields selected by a Straight-Through Gumbel-Softmax router. Across
manifold controls, a regime-switching Lorenz system, widefield cortical calcium imaging during associative learning, and embryoid body single-
cell differentiation, FLUX reconstructs longitudinal transport while recovering interpretable regime structure. Ablations show that mixture-of-experts routing alone is insufficient: FLUX without
geometric learning can fit local transport but fails or weakens regime discovery when regimes are encoded in local dynamics. These results suggest that geometry-aware velocity decomposition provides a general strategy for discovering latent biological state transitions from unpaired longitudinal snapshots.
Read the full abstract ▾
All authors Josue Ortega Caro, Yongxu Zhang, Hannah M Batchelor, Sizhuang He, Jessica Cardin, Shreya Saxena
Machine Learning · Cross-listed from cs.CL
AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
Aritra Mazumder, Shubhashis Roy Dipta, Nusrat Jahan Lia et al. (+10 more)
Multi-
agent systems achieve state-of-the-art outcomes through peer collaboration. However, when an
agent in the pipeline silently drops a constraint, the system's final output may look correct even though the
reasoning chain was quietly corrupted, and existing outcome-based evaluations are blind to such multi-hop process failures. To make these vulnerabilities measurable before deployment, we introduce AgentCollabBench, a diagnostic
benchmark of 900 human-validated tasks spanning software engineering, DevOps, and data engineering. Each task isolates one of four behavioral risks: instruction decay (does a constraint survive peer pressure?), false-belief contagion (does a falsehood spread through consensus?), context leakage (does information bleed between tasks?), and tracer durability (does marked data reach the final
agent?). Evaluating four modern LLMs (GPT 4.1 mini, Gemini 2.5 Flash Lite, Qwen-3.5-35B-A3B, and Llama 3.1 8B Instruct), we expose model-specific vulnerability profiles invisible to outcome-only evaluation; Qwen-3.5-35B-A3B, for example, leads on tracer durability and instruction stability, while GPT 4.1 mini leads on leakage containment and false-belief resistance. Beyond per-model differences, communication
topology emerges as a primary risk factor that explains 7-40% of the variance in multi-hop information survival. The effect traces to a synthesis bottleneck specific to converging-DAG nodes: an
agent weighing competing parent inputs discards constraints carried by a minority branch, a bottleneck structurally absent from linear chains. AgentCollabBench demonstrates that suboptimal
topology can silently erase the safeguards of highly capable models, arguing that multi-
agent reliability is fundamentally a structural problem and that scaling model intelligence alone is no substitute for architecture.
Read the full abstract ▾
All authors Aritra Mazumder, Shubhashis Roy Dipta, Nusrat Jahan Lia, Tanzila Khan, Kainat Raisa Hossain, Nehaa Shri, Shubhrangshu Debsarkar, Humayra Tasnim, Gour Gupal Talukder Shawon, Debjoty Mitra, Sumaiya Ahmed Rani, Al Jami Islam Anik, Al Nafeu Khan
Machine Learning
PAAC: Privacy-Aware Agentic Device-Cloud Collaboration
Liangqi Yuan, Wenzhi Fang, Shiqiang Wang et al. (+1 more)
Large
language model (LLM)
agents face a structural tension: cloud
agents provide strong
reasoning but expose user data, while on-device
agents preserve
privacy at the cost of overall capability. Existing device-cloud designs treat this boundary as a compute split rather than a trust boundary suited to agentic workloads, and existing sanitizers force a choice between policy flexibility and the structural fidelity tool calls require. In this work, we develop PAAC, a
privacy-aware agentic framework that aligns planner–executor decomposition with the device-cloud boundary so that role specialization itself becomes the
privacy mechanism. The cloud
agent reasons over typed placeholder
tokens that preserve each sensitive value's
reasoning role while discarding its content, while the on-device
agent identifies sensitive spans and distills each step's execution outcome into compact key findings. Sanitization confines the on-device LLM to proposing which spans to mask, while a deterministic registry performs all substitution and reversal, keeping actions directly executable on device. On three agentic
benchmarks under strict
privacy settings, PAAC dominates the Pareto frontier of
privacy and accuracy, improving average accuracy by 15-36% and reducing average leakage by 2-6$\times$ over state-of-the-art device-cloud baselines, with the largest margins on
privacy targets outside fixed entity taxonomies. We find consistent improvements on 17 additional
benchmarks spanning 10 domains, including math, science, and finance.
Read the full abstract ▾
All authors Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, Christopher G. Brinton
Machine Learning · Cross-listed from physics.plasm-ph
Energy-based models for diagnostic reconstruction and analysis in a laboratory plasma device
Phil Travis and Troy Carter
Energy-based models (EBMs) provide a powerful and flexible way of learning a joint probability distribution over data by constructing an energy surface. This energy surface enables insight extraction and conditional sampling. We apply EBMs to laboratory
plasma physics, a domain characterized by highly nonlinear phenomena. These phenomena are studied using
plasma diagnostics, which are often difficult to analyze and subject to hardware degradation. In addition, the possible configuration space of a
plasma device is sufficiently large that it cannot be efficiently searched using conventional analysis techniques. EBMs address these issues. At the Large
Plasma Device (LAPD), a CNN- and
attention-based EBM is trained on a set of randomly generated machine conditions and their corresponding diagnostic time series. We demonstrate diagnostic reconstruction using this EBM on real data and show that additional diagnostics improves reconstruction error and generation quality. The energy surface is directly evaluated for an ill-posed inverse problem: inferring probe position from a time-series measurement. This inference illuminates
symmetries in the data, potentially leading to a method of inquiry to supplement conventional data analysis. Trends in diagnostic signals are inferred via conditional sampling over machine inputs. In addition, this
multimodal EBM is able to unconditionally reproduce all distributional modes, suggesting future potential in anomaly detection on the LAPD. Fundamentally, this work demonstrates the flexibility and efficacy of EBM-based generative modeling of laboratory
plasma data, and showcases multiple practical uses of just a single trained EBM in the physical sciences.
Read the full abstract ▾
Machine Learning
ReLibra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning
Chao Jin, Xinming Wei, Yinmin Zhong et al. (+6 more)
Load imbalance is a long-standing challenge in Mixture-of-Experts (MoE) training and is exacerbated in reinforcement learning (RL) for LLMs, where hot experts can shift frequently across micro-batches. Existing MoE training systems rely on historical loads to predict future expert demand, making them less effective under sharp fluctuations. We propose ReLibra, an MoE RL training system that exploits a unique opportunity in RL's rollout-training workflow, routing replay, to enable fine-grained load balancing at micro-batch granularity. Because rollout and training process the same
tokens with the same MoE parameters, the
token-to-expert routing decisions are known before training starts. Leveraging this information, ReLibra places two MoE load-balancing mechanisms at inter- and intra-batch timescales, matching their communication patterns to hierarchical network bandwidths. At the inter-batch timescale, ReLibra performs expert reordering to redistribute experts for batch-level cross-node balancing; at the intra-batch timescale, it dynamically performs expert replication within a node to absorb micro-batch-level load fluctuations. Experiments on diverse MoE LLMs and RL workloads show that ReLibra improves training throughput by up to 1.6$\times$ over Megatron-LM and by up to 1.2$\times$ over EPLB, even when EPLB is given oracle loads. Moreover, ReLibra remains within 6%-10% of the throughput of an idealized balanced baseline.
Read the full abstract ▾
All authors Chao Jin, Xinming Wei, Yinmin Zhong, Chengxu Yang, Bingyang Wu, Ruidong Zhu, Zili Zhang, Yuliang Liu, Xin Jin
Machine Learning · Cross-listed from eess.SP
Large Language Models over Networks: Collaborative Intelligence under Resource Constraints
Liangqi Yuan, Wenzhi Fang, Shiqiang Wang et al. (+2 more)
Large
language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving. Yet cloud-based LLM services alone cannot serve a growing class of applications, including those operating under intermittent connectivity, sub-second latency budgets, data-residency constraints, or sustained high-volume inference. On-device deployment is in turn constrained by limited computation and
memory. No single endpoint can deliver high-quality service across this spectrum. This article focuses on collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural
language or structured messages. Such collaboration strives for superior response quality under heterogeneous resource constraints spanning computation,
memory, communication, and cost across network tiers. We present collaborative inference along two complementary and composable dimensions: vertical device-cloud collaboration and horizontal multi-
agent collaboration, which can be combined into hybrid
topologies in practice. We then examine learning to collaborate, addressing the training of routing policies and the development of cooperative capabilities among LLMs. Finally, we identify open research challenges including scaling under resource heterogeneity and trustworthy collaborative intelligence.
Read the full abstract ▾
All authors Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, H. Vincent Poor, Christopher G. Brinton
Machine Learning
Md Atik Ahamed, Mihir Parmar, Palash Goyal et al. (+4 more)
Time Series Foundation Models (TSFMs) excel at numerical
forecasting but operate as black boxes lacking qualitative
reasoning. Conversely, applying LLMs directly to temporal data introduces a modality gap: text tokenizers fragment continuous numerical values, degrading mathematical relationships and exploding sequence lengths, leading to computational overhead. To resolve this, we introduce STRIDE (Strategic Time-series
Reasoning Injected via Distilled
Embeddings), a novel framework natively integrating LLM
reasoning into the continuous
embedding space of TSFMs. Instead of discrete
tokens, STRIDE distills
reasoning traces into a lightweight LLM and dynamically projects its pooled hidden states as a continuous cross-modal prior into the numerical encoder. Jointly optimized via cross-
entropy and
quantile losses, STRIDE readily attaches to diverse architectures (e.g., Chronos-2, Timer-S1). STRIDE ranks #1 on the GIFT-Eval leaderboard (0.625 MASE, 0.423 CRPS) when ensembled with Synapse. On TFRBench, STRIDE consistently outperforms baselines in both
forecasting accuracy and
reasoning quality, with ablations confirming gains stem from strategic
reasoning. Thus, injecting semantic
reasoning as a continuous prior equips TSFMs with human-interpretable
reasoning while improving predictive accuracy.
Read the full abstract ▾
All authors Md Atik Ahamed, Mihir Parmar, Palash Goyal, Chun-Liang Li, Qiang Cheng, Tomas Pfister, Jinsung Yoon
Machine Learning · Cross-listed from cs.CV
Beyond Toy Benchmarks: A Systematic Evaluation of OOD Detection Methods For Plant Pathology Classification
Devesh Shah
Out-of-distribution (OOD) detection is essential for reliable deployment of deep learning systems, yet the majority of existing methods are evaluated on small, visually homogeneous
benchmarks. In this work, we study six OOD detection methods spanning post-hoc scoring, auxiliary objectives, energy-based models, and constrained optimization on the Plant Pathology 2021
dataset, a fine-grained task with natural distribution shifts. Energy-based fine-tuning performs best across OOD settings, improving detection over the softmax baseline while preserving in-distribution accuracy. Analysis shows these gains stem from both a restructuring of the
embedding space alongside calibration of the scoring function. We further document practical training instabilities that arise when scaling constrained optimization methods to moderate-sized
datasets, findings that are largely absent from existing literature. Our results demonstrate that principled OOD detection is achievable on real-world domain-specific data and that
benchmark evaluations alone may not capture the challenges that emerge in practice.
Read the full abstract ▾
Machine Learning
Robust Server Defense Against Unreliable Clients in One-Shot Fair Collaborative Machine Learning
Chia-Yuan Wu, Frank E. Curtis and Daniel P. Robinson
Collaborative machine learning (CML) enables multiple clients to train a global model jointly in a data-distributed setting. To address data
privacy and communication efficiency, one-shot CML has been increasingly adopted, where clients communicate with the server only once by sharing synthetic or processed proxy data. This single-round communication, however, eliminates the possibility of iterative correction at the server, making the learning process particularly vulnerable to client unreliability. In this setting, unreliable clients, whether malicious or non-malicious, may provide biased proxy data that favors certain groups, thereby degrading the
fairness of the global model and harming minority or unprivileged groups. In this work, we propose a server-side defense framework based on a bilevel optimization formulation. The proposed approach learns client-level weights to mitigate the influence of biased client proxy data while enforcing
fairness constraints by using a very small trusted root
dataset available at the server. Experimental results on
benchmark datasets show that our method improves
fairness with little accuracy loss under biased proxy data contributions from unreliable clients. Moreover, the proposed approach remains effective even when unreliable clients make up a majority of the system, consistently outperforming other existing methods.
Read the full abstract ▾
Machine Learning
Liam Davis, Leopold Haller, Alberto Alfarano et al. (+1 more)
We introduce the
Lattice Deduction
Transformer (LDT), a recurrent
transformer that approximates logically sound deduction by projecting its latent state through a
lattice between forward passes. We train on-policy in a process that mirrors deduction in a search-based constraint solver and supervise training via a domain-agnostic approximation of the set of solution candidates based on abstract interpretation. An $800$K-parameter LDT outperforms prior small recurrent reasoners on Sudoku-Extreme with under $2\%$ of their estimated training FLOPs, solving $99.999\%$ of the $422{,}786$ test puzzles. The model is nearly always sound, in that it gives a wrong answer for $4$ of $1.27$M puzzle evaluations, and otherwise returns a correct answer or abstains. A $1.8$M-parameter LDT variant reaches $99.8\%$ accuracy on Maze-Hard. Our code is available at github.com/lcrh/
lattice-deduction-
transformers.
Read the full abstract ▾
All authors Liam Davis, Leopold Haller, Alberto Alfarano, Mark Santolucito
Machine Learning · Cross-listed from cs.AR
FLARE: One-Shot PE-Level Fault Localization in Systolic Arrays via Algebraic Test Vectors
Logashree Venkatasubramanian, Zishen Wan and Viveck Cadambe
Systolic arrays are the dominant compute fabric for neural network inference. Prior work has addressed column-level fault detection efficiently with uniform test patterns, but row-level (PE-level) fault localization within a faulty column remains open without resorting to hardware redundancy. The fundamental obstacle is that uniform test inputs destroy per-row signatures: any test that activates every row equally cannot distinguish which row is the source of an observed deviation. In this paper, we propose a lightweight, purely algorithmic remedy based on coprime test vectors. By assigning pairwise coprime integers as test-input entries, a permanent weight-register fault produces a deviation whose divisibility signature uniquely identifies the faulty row. Under a general bounded error model, a single test pass localizes the faulty row with high probability. This error model covers a broader class of faults than what prior dataflow-aware testing work has primarily emphasized. When one round is insufficient, a second pass using a ratio computation achieves exact localization; for the special case of single-bit errors, odd coprime entries guarantee exact localization in one round. For INT16 arithmetic, a single test pass covers array sizes up to $256{\times}256$ with localization probability above $0.98$, at a test cost under $1\%$ of one inference GEMM tile.
Read the full abstract ▾
Machine Learning
Jiaxuan Zou, Ruifeng Ren and Yong Liu
Long-context
language modeling remains central to modern sequence modeling, but the quadratic cost of
Transformer attention makes scaling computationally prohibitive. Linear recurrent models address this bottleneck by compressing the context into a fixed-size state, making the rule that forgets, writes, and edits information a central design problem. To address state maintenance, Gated DeltaNet (GDN) combines gated state decay with delta-rule residual writes, using a learnable coefficient to balance forgetting and update magnitude. However, this coefficient is learned empirically rather than derived from the underlying objective, which can lead to suboptimal update magnitudes. We revisit the online-regression objective underlying GDN and, inspired by the Kaczmarz projection method, derive the key-norm-normalized dynamic step size $\beta_t = \eta_t / (\|k_t\|_2^2 + \epsilon)$ for residual updates. We propose Kaczmarz Linear
Attention (KLA), a one-scalar modification of GDN that preserves the state shape, gates, linear recurrence, and chunkwise parallel
algorithm. At the 0.4B scale with a 1B-
token budget, KLA achieves the lowest validation perplexity among evaluated linear-time baselines, 8.09 versus 8.50 for GDN, and remains stable up to 65K
tokens. On controlled tasks, KLA reaches 100% on single-needle-in-a-haystack
retrieval, improves 8x multi-query associative recall by 7.03 points over GDN, and delivers 2.1x higher decode throughput at 32K context. These results suggest that the key-norm-normalized Kaczmarz coefficient is a first-order design axis for delta-rule sequence models: it improves accuracy, extrapolation, and decoding efficiency without changing the recurrent state or hardware kernel.
Read the full abstract ▾
Machine Learning
HAServe: Horizon-Aware Scheduling and Memory Co-Design for Efficient LLM Serving
Xingyu Qu, Tianhao Lin, Yiqi Li et al. (+2 more)
Online large
language model (LLM) services process requests that share context but differ in prefix structure and arrival timing. Finite capacity couples request admission and key-value (KV) retention. This paper addresses one online serving problem: using a unified view of visible demand to choose the feasible joint admission and KV-retention candidate with the lowest predicted latency cost as prefix structures and arrival rates vary. To model this problem, we introduce HorizonSchema, a workload abstraction that connects exact prefix paths, permitted serializations, and released consumers as shared inputs to both decisions. Our analysis shows that useful reuse depends on demand and prefix survival, while service order changes latency even at equal computation. Furthermore, we present HAServe, which turns this abstraction into a scheduling–
memory co-design method of systematic optimization for online LLM serving. We implement HAServe on SGLang and evaluate Qwen3-4B-Instruct-2507 and Llama2-13B on five workloads using eight H100 GPUs. Across the evaluated workloads, HAServe achieves up to $1.14$–$5.90\times$ P99 TTFT speedups and up to $1.08$–$1.44\times$ the throughput, with KV-cache hit rate up to 20.59 percentage points higher than the strongest baseline for each metric and setting.
Read the full abstract ▾
All authors Xingyu Qu, Tianhao Lin, Yiqi Li, Zhiyu Chen, Sheng Wang