Machine Learning
PQ-HSA: Reusing Product-Quantized Scores for Hybrid Sparse-Approximate Attention
Kunming Shao, Jierun Chen, Yanli Wang et al. (+4 more)
At each decoding step a
language model attends over the key-value (KV) cache of every earlier
token, so at long context the
attention call is bounded by
memory bandwidth. Sparse
attention reads only a subset of keys chosen by a cheap score estimate, and most methods give the unread
tokens zero weight. The output then draws on only a small fraction of the KV cache, and accuracy drops at small budgets, most on tasks that aggregate information across the context. An inverted-file product-quantization (IVF-PQ) index over the cached keys computes an approximate score for every indexed
token in order to rank them; after ranking, those scores approximate the
attention logits of the
tokens left out. PQ-HSA (hybrid sparse-approximate
attention) attends the selected
tokens with their original keys and values, and the unselected
tokens, the background, enter the same softmax through those scores, summed per inverted list and multiplied by the list's mean value. At 128K and a 1-2%
retrieval budget, PQ-HSA is more accurate than Quest and SnapKV on Llama-3.1-8B and Qwen3-30B-A3B and stays close to full
attention in macro accuracy; with the same selector, the background term raises macro accuracy on the 8B model from 0.71 to 0.83. In the same 128K setting, inside vLLM on one NVIDIA H20, the decode
attention call runs 1.6x faster than the FlashAttention-3 kernel; the speedup grows with context length, and a cost model fitted on 8B to 30B models gives the context length at which it begins. A vLLM plugin runs PQ-HSA on two engine versions without changes to the engine source; code is available at https://github.com/KunmingSHAO/pqhsa_release.
Read the full abstract ▾
All authors Kunming Shao, Jierun Chen, Yanli Wang, Ruoyu Wang, Haoli Bai, Kwang-Ting Cheng, Chi Ying Tsui
Information Retrieval
Beyond the Beam: Constructive Repair and Candidate Completion for Generative Recommendation
Zijun Zhao, Peng Zhang, Gang Zhang et al. (+3 more)
Generative recommenders retrieve items by generating identifiers, but a valid identifier can remain outside the beam after catalog expansion. This raises two connected questions: which failures can identifier assignment repair, and how should
retrieval proceed beyond the initial beam? We characterize assignment repair with a fixed generator and retained old identifiers. Output-invariance certificates identify failures shared by all admissible assignments. Under a common effective prefix, coupled support and ranking constraints give the exact feasible interval of new-item counts for target recovery. Building on this characterization, Beyond the Beam (BB) obtains minimum-replacement repairs through an integral flow formulation, selects a shared map and adapts the generator. At inference, generative likelihood and collaborative evidence define one score for ranking, candidate priority and stopping. Retained prefix bounds guide candidate completion and certify its global Top-$K$ when the stopping condition is met. Exhaustive finite-catalog evaluation confirms construction in every feasible case. Across three Amazon Reviews categories and three random seeds, the full T5 procedure improves mean Recall@10 by 15.5–46.3% and NDCG@10 by 15.2–44.4% over the best-performing evaluated generative baseline for each
dataset and metric. Matched controls show that shared construction and adaptation improve new-target ranking and certification efficiency on Beauty and Toys. Combined scoring and candidate completion improve NDCG@10 across all three
datasets with both T5 and decoder-only LC-Rec.
Read the full abstract ▾
All authors Zijun Zhao, Peng Zhang, Gang Zhang, Yuanchi Ma, Hui He, Zhendong Niu
Combinatorics
The Fibonacci numbers are not 3-accessible
William J. Wesley
A $D$-diffsequence is a sequence of integers $x_1 < \dots < x_k$ such that $x_{i+1} -x_i \in D$ for $1 \le i \le k-1$. The set $D$ is called $r$-accessible if every $r$-coloring of the positive integers contains arbitrarily long monochromatic $D$-diffsequences. This note proves that the set of Fibonacci numbers is not 3-accessible, resolving an open problem of Landman and Robertson from 2007.
Read the full abstract ▾
Human-Computer Interaction
Accounting for Stochasticity in Studies of Large Language Model Refusal
Emma Lurie, Stephanie T. Wang, Sorelle A. Friedler et al. (+1 more)
We present preliminary empirical evidence that single-observation queries are insufficient for evaluations of LLM refusal behaviors. Using a longitudinal
auditing system, we issued identical
prompts 100 times each across four dates to GPT-4.1 for two socially salient topics across 20 Wikipedia sources. Refusal outcomes were consistent with a stable Bernoulli process, yet 20% of sources fell within a decision-boundary region where a single query is largely uninformative. Reliable quantification of refusals required between 15 and 25 repeated queries, well above the single-observation standard common in existing evaluations.
Read the full abstract ▾
All authors Emma Lurie, Stephanie T. Wang, Sorelle A. Friedler, Danaé Metaxa
Sound
DuraS2ST: Chain-of-Thought and Reinforcement Learning for Duration-Aligned Speech-to-Speech Translation
Yayue Deng, Dingdong Wang, Yuxuan Hu et al. (+7 more)
Speech-to-
speech translation (S2ST) in time-sensitive applications such as
video dubbing requires not only semantic fidelity and speaker preservation, but also strict duration consistency to avoid audio-visual misalignment. However, existing S2ST systems largely generate target
speech without explicit temporal planning, making duration control an unresolved challenge. We introduce DuraS2ST, a duration-aligned
reasoning framework that enables a single
speech language model to first generate an explicit chain-of-thought (CoT) for planning target wording and phonetic length, and then synthesize the corresponding
speech tokens. To support this paradigm, we construct DuraSet-440K, a high-quality duration-aligned CoT corpus for supervised initialization. We further optimize the model with multi-modal multi-dimensional reinforcement learning, using a Duration Margin Reward to balance translation quality and duration consistency, and Modality-Aware Reward Attribution to assign rewards to appropriate
token spans. Experiments on CVSS-T show that DuraS2ST achieves a strong balance between translation quality and duration consistency, outperforming competitive open-source and commercial baselines. Project page: https://github.com/Mia11939/DuraS2ST.
Read the full abstract ▾
All authors Yayue Deng, Dingdong Wang, Yuxuan Hu, Jinyu Li, Yanqing Liu, Yuanyuan Wang, Weidong Chen, Helen M. Meng, Shujie Liu, Xixin Wu
Machine Learning (Statistics)
Multi-Marginal Inverse Optimal Transport for Contrastive Learning Via Explicit Anchor-Positive-Negative Coupling
Ngoc-Hai Nguyen, Thuan Nguyen, Prakash Ishwar et al. (+1 more)
Inverse Optimal Transport (OT) based methods for representation learning learn representations such that the global OT coupling between a pair of data marginals in the representation space, concentrates on the positive pairs. This is in contrast to previous methods that primarily focused on pairwise matching. However, these methods $\textit{DO NOT}$ utilize negative pairs and hence are not truly contrastive in their approach. We show that this leads to issues of dimensional collapse and hence degraded downstream performance. To alleviate this, we develop a novel multi-marginal (MM) inverse OT (IOT) contrastive learning (CL) approach called Neg-MMIOT-CL, which learns representations such that the global multi-marginal OT (MMOT) coupling between a triple of data marginals, with respect to a carefully designed ground-cost between triplets of data points in the representation space, concentrates on the anchor-positive-negative $\textit{triplets}$. For a latent class model, we empirically show that Neg-MMIOT-CL alleviates dimensional collapse. Furthermore, for a specific choice of ground cost for all triplets in representation space, we prove that the optimal representation configuration for Neg-MMIOT-CL exhibits equiangular property for within-class and across-class representations, which translates to Neural-Collapse when the representation dimension is larger than the number of classes minus one – a result that is $\textit{previously established only}$ for pairwise contrastive learning methods. Finally, we propose Neg-IOT-CL-PushPull, that is a computationally efficient alternative to Neg-MMIOT-CL, alleviating the high cost of computing MMOT plans needed during implementation. We apply these methods on both synthetic and real-world
datasets and show significant improvements over existing OT-based contrastive learning methods.
Read the full abstract ▾
All authors Ngoc-Hai Nguyen, Thuan Nguyen, Prakash Ishwar, Shuchin Aeron
Machine Learning (Statistics)
Weighted Spline-Expanded Networks with Distributional Balancing for Continuous Treatment Effects
Shucheng Liu, Chan Park and Guanhua Chen
Estimating causal effects with continuous treatments in observational studies is challenging due to confounding, model misspecification, and high-dimensional
covariates. We propose the Weighted Spline-Expanded Network (WSENet), an end-to-end neural framework that addresses these challenges by combining
covariate balancing, structured treatment
embedding, and bias-corrected outcome estimation. WSENet first applies Distance
Covariate Optimal Weights to induce distributional independence between
covariates and treatment without relying on parametric models. It then learns the conditional outcome via a structured network that fuses outcome-relevant representations of
covariates with a spline-expanded treatment input, enabling smooth and flexible modeling of the dose-response relationship. To mitigate residual bias, we introduce Weighted Targeted Regularization, a correction technique based on efficient influence functions that yields a doubly robust
estimator. Extensive evaluations on semi-synthetic and real-world
datasets, including high-dimensional genomic and environmental health data, demonstrate that WSENet consistently outperforms existing baselines in both accuracy and stability.
Read the full abstract ▾
Signal Processing
Error Performance Analysis of CFO-Impaired OFDM-Based Rate-Splitting SCMA
Minerva Priyadarsini, Kuntal Deka, Zilong Liu et al. (+1 more)
Rate-splitting sparse code multiple access (RS-SCMA) superimposes QAM-modulated common messages and SCMA-encoded private messages over the same resource elements. This paper analyzes the bit error rate (BER) performance of orthogonal frequency division multiplexing (OFDM)-based RS-SCMA in the presence of carrier frequency offset (CFO) over frequency-selective Rayleigh fading channels. A finite-alphabet BER framework is developed for common-layer detection, successive interference cancellation (SIC), and private-layer SCMA detection. The proposed analysis explicitly retains the common and private-layer signal components and models the unresolved intra- and inter-block CFO leakage as conditionally Gaussian interference using its conditional second-order statistics. Residual interference caused by erroneous common-layer cancellation and the effect of a tunable message-splitting factor are also incorporated. The results show that CFO produces a high-SNR BER floor that increases with the normalized frequency offset. Under the considered conditions, the overloaded SCMA private layer is more severely affected and dominates the overall BER. Simulations with perfect SIC further show that eliminating common-layer decision errors has only a marginal impact on the private-layer BER floor, indicating that the dominant degradation arises from CFO-impaired private-layer detection rather than SIC error propagation. Monte Carlo simulations validate the derived BER expressions and demonstrate their accuracy in capturing the layer-wise and overall BER performance over the considered CFO range.
Read the full abstract ▾
All authors Minerva Priyadarsini, Kuntal Deka, Zilong Liu, Sanjeev Sharma
Computation and Language
The Effects of Incremental Instruction Delivery on Language-Model Creative Writing
Anshuman Singh, Abrar Eyasir, Haseeb Yaqoob et al. (+1 more)
Large
language models are increasingly used as interactive writing tools, where users develop stories, revise ideas, and introduce new requirements across multiple turns rather than specifying a complete brief upfront. Yet most evidence on multi-turn instruction degradation comes from tasks with objectively verifiable outcomes, leaving unclear whether incremental interaction harms creative artifacts in ways that explicit requirement checks cannot capture. We study this question using 160 human-authored creative-writing tasks across six genres, presenting each intended specification either upfront or progressively over 5-9 turns to six distinct open-weight model families, yielding 960 matched pairs. Progressive delivery reduces explicit constraint adherence and produces its largest writing-quality degradation in structure/coherence. The structural gap persists among outputs with equal observed adherence, suggesting that measured requirement loss alone does not explain the observed structural difference. We define Creative Integrity as a compact measure of joint adherence and narrative structure; under incremental delivery, models retain 71.2% of FULL Creative Integrity (95% CI [68.2%, 74.3%]). A three-rater human study over 50 matched pairs independently recovers FULL advantages in structure/coherence, craft, and genre effectiveness, while automated scores remain positively associated with aggregated human ratings. These findings show that interactive creative-writing systems should be evaluated not only on whether requirements survive conversation, but also on whether evolving requirements remain coherently integrated into the final artifact. Our
dataset,
benchmarks, and source code are available at: https://github.com/solusops/SISTER-2026-Team19
Read the full abstract ▾
All authors Anshuman Singh, Abrar Eyasir, Haseeb Yaqoob, John Manavalan
Robotics
MomWorld: Momentum-Aware Latent World Model for Long-Horizon Autonomous Driving
Ziying Song, Shengkai Zhang, Lei Yang et al. (+6 more)
Long-horizon planning enables autonomous vehicles to anticipate scene
evolution and potential risks, supporting safe and stable decisions in complex interactions. However, existing methods struggle to propagate motion trends from observed history into the future. Long rollouts based on a single latent state may further attenuate useful dynamics, retain stale motion patterns, and disrupt reliable near-term plans. We introduce MomWorld, a momentum-aware latent world model for long-horizon planning. MomWorld extracts scene motion trends from historical-to-current observations and propagates latent momentum into future horizons, jointly predicting future configuration and momentum states. A learnable momentum persistence mechanism preserves stable trends, scene-conditioned momentum updates adapt future dynamics, and a scene-adaptive reset gate suppresses stale momentum under abrupt changes. We further propose MoFlow, a momentum-conditioned flow-matching module that refines a base trajectory to align with the predicted future scene
evolution in only a few integration steps, with a horizon-aware residual fusion that preserves near-term planning stability while permitting stronger long-range corrections. Extensive experiments on NAVSIM, nuScenes and Bench2Drive demonstrate that MomWorld improves long-horizon planning consistency and reduces the average collision rate by 12.2% relative to MomAD over a 6-second planning horizon.
Read the full abstract ▾
All authors Ziying Song, Shengkai Zhang, Lei Yang, Haozhuang Chi, Yuchen Liu, Jiangtao Su, Lin Liu, Ziyang Liu, Chen Lv
Physics and Society
Rethinking Group Differences in Psychopathology Networks: A Slow-Fast Perspective on Context and Symptom Activation
Kyuri Park, Denny Borsboom, Mike Lees et al. (+6 more)
Groups differing in social and economic circumstances often differ markedly in depressive symptom levels, yet their estimated symptom networks show few clear differences. Such null results are often read as evidence that the groups' symptom systems are the same, but they only show that no difference was detected in how symptoms interact. How readily symptoms become present, and how this depends on the contextual conditions under which they are observed, is rarely compared. Building on a slow-fast perspective, we treat depressive symptoms as a relatively fast-changing system embedded in more persistent social, economic, psychosocial, health, and lifestyle conditions that shape symptom activation. Using a cohort of 23,689 adults from Amsterdam, we illustrate this with PHQ-9 depressive symptoms and a composite index of persistent socioeconomic, psychosocial, and health-related conditions, which we call Slow Risk Load (SRL). Participants with high SRL had substantially higher symptom levels than those with low SRL. A standard Network Comparison Test detected no overall difference in network structure, although global strength was modestly higher in the high-SRL group. Ising models, which separate how symptoms co-occur from how readily each becomes present, indicate higher activation under high SRL for eight of nine symptoms, and model comparison consistently favored group-specific activation over group-specific interactions. The slow-fast perspective reframes group network comparisons as a multiple-timescale problem in which network structure, symptom activation, and context should be considered jointly. It places symptom-focused and context-focused interventions at different layers of the same coupled system rather than in competition.
Read the full abstract ▾
All authors Kyuri Park, Denny Borsboom, Mike Lees, Leonie Elsenburg, Gaby Lunansky, Karien Stronks, Johan Bollen, Mary Nicolaou, Vítor V. Vasconcelos
Computer Vision
Constrained Edit Fields for Training-Free Flow Editing
Jingxuan Kang, Yinsong Wang, Che Liu et al. (+1 more)
Text-guided image editing aims to perform a desired edit while preserving source content unrelated to it. Pretrained rectified-flow models enable training-free editing of real images through modifications to their sampling trajectories. However, responses at locations unrelated to the desired edit can still accumulate along the editing trajectory and become visible in the final result. To overcome this, we propose Constrained Edit Fields (CEF), which assigns each spatial location a continuous edit responsibility that quantifies its relevance to the desired edit. CEF estimates edit responsibility directly from the source image when the relevant content is present. For edits whose target content is absent from the source, CEF first generates an unconstrained proposal to reveal its realized spatial support and then estimates responsibility from that proposal. At each editing step, CEF decomposes the base edit field into
prompt-induced and trajectory-induced components, enabling edit responsibility to preserve instruction-relevant updates while suppressing unintended trajectory-induced changes. Evaluated on all 700 PIE-Bench examples, CEF achieves state-of-the-art Structure Distance, background LPIPS, and background MSE with both Stable
Diffusion 3.5 Medium and FLUX, while retaining competitive instruction
alignment. On Stable
Diffusion 3.5 Medium, it reduces these metrics over the previous best results by 10.2%, 21.2%, and 48.0%, respectively.
Read the full abstract ▾
All authors Jingxuan Kang, Yinsong Wang, Che Liu, Chen Qin
Computation and Language
Yorùbá in Unicode: An Overview of a Problem
Kólá Túbòsún
There is a recurrent problem in the writing of Yorùbá on the internet and on the computer that has proven intractable over the years. The
language, along with other African
languages that depend on diacritics for disambiguation, requires a small set of precomposed characters that Unicode does not encode. This has forced writers and digital systems to rely on combining character sequences that behave inconsistently across platforms, corrupt under font substitution, and fail in search. This paper documents that failure across a range of real world contexts, from published books to web platforms to mobile keyboards, using personal and empirical evidence. It identifies Unicode's NFC normalization stability policy as the structural constraint that prevents a straightforward fix, arguing for direct intervention of the Consortium in solving the active problem, proposing a formal encoding request for the four core Yorùbá characters as the most durable path to resolution.
Read the full abstract ▾
Combinatorics
A quasipolynomial inverse theorem for the $\mathsf{U}^k(\mathbb{F}_p^n)$ norm in the high characteristic
Luka Milićević
We prove an inverse
theorem for the Gowers uniformity norm $\mathsf{U}^k(\mathbb{F}_p^n)$ with quasipolynomial bounds in the case when $p \geq k$. The inverse
theorem follows from a quasipolynomial structure
theorem for Freiman multihomomorphisms, which are a natural generalization of Freiman homomorphisms to maps of several variables. The proof of the structure
theorem for Freiman multihomomorphisms is the central result of the paper and rests on three main ingredients: algebraic regularity method, abstract Balog-Szemerédi-Gowers
theorem and the theory of multilinear maps defined on multilinear varieties. The last ingredient originates from an earlier work of Gowers and the author, and is significantly expanded in this paper. In particular, once the theory of such maps is in place, the proof of the structure
theorem for Freiman multihomomorphisms is relatively short, especially compared to the previous quantitative results in the inverse theory of Gowers norms.
Read the full abstract ▾
Combinatorics
Strong $q$-log-convexity of the $d$-Hoggatt polynomials
Shane Chern and Wenle Shi
In this paper, three ($q$-)log-convexity or concavity properties related to the $d$-Hoggatt numbers are investigated. First, we show that the $d$-Hoggatt transformation preserves the log-convexity of a sequence. We then establish the strong $q$-log-concavity of a $q$-analog of $d$-Hoggatt numbers. Finally, we prove the core result of this work, namely, the sequence of $d$-Hoggatt
polynomials is strongly $q$-log-convex, based on the theory of Schur functions.
Read the full abstract ▾
Artificial Intelligence
HTN Planning as a Coordination Layer for Multi-Server MCP Tool Orchestration
Eliott Jacopin, Éric Jacopin and Koichi Takahashi
The Model Context Protocol (MCP) isolates servers by design: only the host can orchestrate cross-server workflows. When the host is a large
language model, the resulting orchestrations are non-deterministic, non-reproducible, and pay one inference round-trip per tool call. We present a coordination architecture in which a Hierarchical Task Network (HTN) planner generates a verifiable cross-server plan once, and a runtime middleware executes it deterministically across multiple MCP servers, binding cross-action data dependencies via a template mechanism (\verb|${context.X}|) substituted at execution time. The architecture mirrors MCP's isolation constraint: each compound task decomposes into server-local primitive actions, and inter-server data flow is bound at execution time via JSON-path output extractors. We instantiate the architecture on five HTN domains spanning laboratory robotics, bioinformatics and multiscale modelling, and demonstrate end-to-end execution from a browser-based plan controller against eight live third-party MCP servers querying real biological databases.
Read the full abstract ▾
Statistical Methodology
Rerandomization under Interference
Ziang Yuan, Xinran Li and Shuangning Li
Covariates are widely used in randomized experiments to improve precision. However, in the presence of interference, where outcomes may depend on the treatment assignments of other units, standard
covariate adjustment methods may fail to preserve desirable properties such as the no-harm property, meaning that incorporating
covariates does not worsen
estimator performance. Existing approaches that incorporate
covariates under interference typically rely on specifying a particular interference structure, and their guarantees can be sensitive to misspecification. In this paper, we study how to incorporate
covariate information in a way that preserves the no-harm property while remaining largely agnostic to the underlying interference structure. We focus on the estimation of the expected average treatment effect (EATE) using the Hájek
estimator and study rerandomization under interference, a design-stage procedure that restricts the assignment space to allocations with sufficiently small
covariate imbalance. Our framework also allows the
covariates used for rerandomization to depend on the treatment assignment itself, such as the proportion of treated neighbors, which naturally arises in settings with interference. We show that, even under interference, rerandomization can improve estimation precision asymptotically relative to unrestricted Bernoulli randomization, relying only on mild conditions on the dependence structure across units. When a conservative dependence graph is available, we further develop an optimization-based conservative variance
estimator for inference under rerandomization.
Read the full abstract ▾
Signal Processing
Wave-Domain Semantic Equalization Using a Practical Dynamic Metasurface Antenna with Strong Mutual Coupling
Yassine Sghaier, Emilio Calvanese Strinati and Philipp del Hougne
Semantic mismatch between independently trained AI-native
agents in heterogeneous networks can impair semantic communications. Hybrid analog-digital semantic equalization can align the incompatible latent representations without retraining the semantic transceivers. We study a practical realization of this approach based on a fabricated dynamic metasurface
antenna (DMA), an emerging low-cost, low-power, ultracompact technology for hybrid analog-digital beamforming. We model the DMA-assisted channel using multiport-network theory (MNT), accounting for mutual coupling (MC), structural scattering, and binary lossy tuning states. We use the experimentally estimated MNT parameters of our fabricated 19-GHz DMA prototype with strong MC. We jointly optimize digital pre- and post-equalizers and the DMA at a reference receiver geometry for latent-space
alignment, then freeze the digital stages and adapt only the DMA after receiver motion. In our CIFAR-10 image-classification task at the receiver, the jointly optimized system achieves 94.5% accuracy, while DMA-only adaptation restores a median accuracy of 90.3% after receiver motion. Our semantic-aware wave-domain adaptation substantially outperforms semantic-unaware
benchmarks. We further observe that varying the DMA configuration across channel uses provides little additional benefit for either semantic-aware or semantic-unaware optimization.
Read the full abstract ▾
Machine Learning
ALDER: Discovering the Laws of a World by Acting in It
Teng Cao, Yu Deng, Quentin Delfosse et al. (+1 more)
Reliable world models should not only predict future states but express how actions change the world in an explicit, transparent and testable form, such as equations. Yet methods that rely on a fixed set of trajectories cannot distinguish equally good competing hypotheses, while searches over a fixed set of predefined candidates cannot discover equations outside the initial hypothesis space. We introduce ALDER (Action-guided Law Discovery, Evaluation, and Revision), a method that actively proposes novel experiments to test and revise models. Specifically, ALDER proposes parametric equations; a numerical optimizer fits their coefficients; an independent verifier tests these candidates on held-out data. To distinguish between competing valid hypotheses, a cost- and safety-aware selector queries interventions, in the form of novel experiments. The resulting counterexamples update the evidence ledger and guide the next structural revision, while incompatible laws are discarded. Across an in-house
benchmark, ODE equation discovery tasks, and robotic experiments, ALDER discovers laws beyond its initial formula set, repairs failed model proposals, distinguishes fixed candidate models with fewer interactions, and improves out-of-distribution prediction. Furthermore, given a current state and a target, ALDER selects control actions by solving the inverse problem defined by its validated world model. Together, these results show that explicit equation-based world models can be tested and revised through interaction, then naturally used to guide goal-directed control.
Read the full abstract ▾
All authors Teng Cao, Yu Deng, Quentin Delfosse, Kristian Kersting
Machine Learning (Statistics)
A Statistical Perspective on Knowledge Distillation: Foundations, Classical Methods, and Large Language Model Extensions
Luyang Fang, Haoran Lu, Jiazhang Cai et al. (+4 more)
Knowledge
Distillation (KD) has emerged as a vital paradigm for transferring the capabilities of high-capacity models to efficient “student” counterparts, addressing critical challenges in computational cost, deployment constraints, and
privacy-sensitive settings. Although KD is widely used in practice, it is often viewed primarily as an engineering technique, with a unified statistical perspective remaining less developed. This review bridges that gap by presenting a unified
Bayesian formulation of KD that formulates teacher predictions as prior information. This provides a principled interpretation of how teacher information is incorporated into student learning and establishes a rigorous connection to uncertainty quantification. We demonstrate how this foundational lens reconciles classical
distillation with modern extensions in generative and foundation-model systems, showing that contemporary developments remain rooted in these same statistical principles. By synthesizing theory with emerging methodologies and diverse applications, this review provides a conceptual roadmap and identifies critical open problems for the future of the field.
Read the full abstract ▾
All authors Luyang Fang, Haoran Lu, Jiazhang Cai, Tao Wang, Huimin Cheng, Wenxuan Zhong, Ping Ma
Artificial Intelligence
Robust Biomolecular Complex Design Across Protein Conformational Landscapes
Qingyuan Zeng, Zongqi Xu, Anglin Liu et al. (+7 more)
Proteins populate conformational ensembles, yet structure-based biomolecular design typically optimizes candidates against a single target conformation. Consequently, a candidate that fits one state can lose favorable interactions or develop steric clashes when the target adopts another. We introduce FlexEvo, a model-agnostic evolutionary framework that adapts candidates once at inference time from a single target conformation to improve compatibility with alternative natural conformations unseen during adaptation, without retraining the source model or requiring a conformational ensemble. FlexEvo casts cross-state adaptation as geometry-constrained bi-objective optimization, balancing preservation of input-state interactions against robustness to plausible conformational perturbations. To limit the search space and reduce invalid structural edits, geometry-derived FlexBoxes define protected anchor regions, adaptable regions for local exploration, and forbidden regions for clash avoidance. A unified all-atom representation supports
topology-preserving adaptation across diverse binder categories, while Pareto selection preserves nondominated candidates across the two objectives. We evaluate FlexEvo across multiple generation baselines and nine representative binder categories spanning diverse molecular sizes and structural
topologies. FlexEvo reduces the category-balanced mean relative performance degradation from 47.8% to 4.4%, while adding only 1.4–3.1 minutes of adaptation per sample. These results establish single-state inference-time adaptation as a practical route toward robust biomolecular complex design across
protein conformational landscapes.
Read the full abstract ▾
All authors Qingyuan Zeng, Zongqi Xu, Anglin Liu, Ziqi Gong, Pengxiang Cai, Zixin Guan, Yunan Chen, Sen Gao, Min Zhou, Jintai Chen
Machine Learning (Statistics)
Reliable Replay through Spatial Coherence in Online Continual Learning
Haixiang Sun, Jiefu Zhang, Yinghao He et al. (+4 more)
Continually adapting models to new tasks requires retaining earlier knowledge under limited
memory and computation. Experience replay addresses this challenge, but priorities based on individual loss increases overlook how related
memories respond to the same update and can overemphasize isolated responses. We introduce SPatial coHErent risk control for REplay (SPHERE), a general replay-allocation method applicable across a broad range of learning settings. SPHERE uses a representation kernel to aggregate signed prospective loss changes, attenuating unsupported spikes while retaining coherent increases. It then formulates allocation as
entropy-regularized transport, redistributing uniform source mass toward supported high-risk regions while penalizing long-distance transfers. We derive replay coefficients from the transport objective's sensitivity to the original loss changes and blend them with uniform replay to maintain baseline rehearsal. Our analysis establishes conditions under which kernel aggregation improves risk estimation and bounds transport-value inflation due to residual noise and smoothing bias. Experiments demonstrate that SPHERE improves accuracy and reduces forgetting across noisy-label
vision tasks, continual
language-model instruction tuning, and code-generation reinforcement learning with incomplete test rewards.
Read the full abstract ▾
All authors Haixiang Sun, Jiefu Zhang, Yinghao He, Yang Xu, Vaneet Aggarwal, Bharat Bhargava, Andrew L. Liu
Statistical Mechanics
Quantum Dynamics of Full Counting Statistics in Fermionic Lattices with Localized Gain
Bijay Kumar Agarwalla and Manas Kulkarni
We investigate the dynamics of full counting statistics (FCS) of the growth of the total number of
fermions in a one-dimensional non-interacting
lattice subjected to a localized particle gain (source) at one edge. The dynamics of the setup is modeled by the Gorini–Kossakowski–Sudarshan–Lindblad (GKSL)
quantum master equation. We recast the counting problem that involves counting particle number at every
lattice site to a problem where only the local injected site is involved. We employ the Schwinger-Keldysh path integral formalism within the GKSL framework and derive an analytical expression for the cumulant generating function at arbitrary times and obtain a Levitov-Lesovik type formula in the long-time limit, earlier derived for boundary driven setups in the steady-state. For clean
lattices with either short- or long-range hopping, supporting single-particle delocalized eigenstates, we show that all cumulants grow linearly with time in the asymptotic regime. Our analytical results are in excellent agreement with direct numerical simulations. These findings establish a general framework for characterizing FCS and
quantum fluctuations in driven open fermionic systems with localized particle injection.
Read the full abstract ▾
Computer Vision
GeoShrink: Accelerating Diffusion Transformers with Two Lines of Code
Haosen Li, Wenshuo Chen, Shaofeng Liang et al. (+3 more)
Diffusion transformers incur substantial inference cost through repeated model evaluations along a sampling trajectory. We introduce GeoShrink, a training-free acceleration method that retains the original solver grid while evaluating the model only at a prescribed set of anchors. At skipped stages, GeoShrink predicts the solver-facing output by adding a geometrically retained fraction of the latest observed innovation to the most recent exact output. We derive this rule from chordal tangent transport and round-trip line projection, and establish a
geometric anchor-spacing principle that minimizes the largest adjacent gap expansion under fixed coverage and first span. The analysis characterizes the
geometric closure and propagation of prediction errors without assuming access to future model outputs. Experiments cover image,
video, motion, and audio generation, together with adapted 3D backends. At approximately $5\times$ acceleration, GeoShrink improves FLUX PSNR by 3.10 dB over the strongest listed baseline. On HunyuanVideo, it achieves a reported $4.99\times$ speedup and improves ChronoMagic-Bench-150 PSNR by 5.44 dB over the strongest listed fidelity baseline. Comparisons at fixed evaluation budgets further show substantial gains on motion, audio, music, and 3D generation.
Read the full abstract ▾
All authors Haosen Li, Wenshuo Chen, Shaofeng Liang, Lei Wang, Bowen Tian, Yutao Yue
Machine Learning
BOReFT: Manifold Steering of Language Models for Black-box Optimization
Dhruv Agarwal, Rico Angell, Kavitha Srinivas et al. (+4 more)
Language models are increasingly used as proposal models for black-box search, from program optimization to molecular design. Existing approaches typically improve proposals through iterative prompting or parameter updates, offering limited control over how completely and efficiently the model's search space is explored. Continuous optimization methods, such as
Bayesian optimization, provide a principled way to search but require a suitable domain to operate over. To address this, we introduce BOReFT, which learns a compact, low-dimensional space of hidden-state interventions in a frozen
language model, and uses this space as the search domain for
Bayesian optimization with an external scoring function. Empirically, we find that the learned domain spans semantic regions and exhibits smoothness properties that support search. Theoretically, we show that semantic coverage and interpolation control the best score available in the learned space, and that decoding from this space yields a standard
stochastic-bandit observation model for adaptive search. We evaluate BOReFT on the interpretable word search task "Semantle" and on three more real-world discovery tasks in de novo molecule property optimization. Compared to strong LLM baselines, BOReFT finds in Semantle a higher number of hidden targets and, on two out of three molecular objectives, achieves higher property scores. Consequently, our method provides a principled new bridge between discrete proposal spaces of LLM-based search and continuous black-box optimization.
Read the full abstract ▾
All authors Dhruv Agarwal, Rico Angell, Kavitha Srinivas, Tahira Naseem, Horst Samulowitz, Willie Neiswanger, Andrew McCallum
Computation and Language
Shared Experience, Separate Learning: Companion Confidence Calibration for LLMs
Shiyu Ni, Keping Bi, Jiafeng Guo et al. (+4 more)
Reliable self-assessment is essential for large
language models (LLMs), yet they often remain highly confident when their answers are wrong. We study concurrent confidence calibration, where confidence is learned alongside capability improvement rather than calibrated only after training. Reinforcement learning from verifiable rewards (RLVR) provides a natural setting for this paradigm, as it continuously produces responses paired with verifiable correctness feedback. Existing concurrent methods, however, learn both capability and confidence through reinforcement learning within shared policy parameters, potentially coupling two fundamentally different learning problems. We instead propose shared experience but separate learning: capability and confidence learn from the same trajectories, but through separate optimization mechanisms and parameters. Based on this principle, we introduce CoCal (Companion Confidence Calibration), which trains a lightweight companion from rollout hidden states and verifier-derived correctness supervision while leaving task optimization unchanged. Experiments on Qwen3-8B and Qwen3-14B show that CoCal improves confidence estimation without sacrificing task performance, outperforming both RL-based concurrent methods and matched post-hoc calibration. The learned companion further generalizes across domains and policy shifts, while the benefits of CoCal persist at both scales.
Read the full abstract ▾
All authors Shiyu Ni, Keping Bi, Jiafeng Guo, Yilong Xu, Jingtong Wu, Zengxin Han, Xueqi Cheng
Computation and Language
From Granular Revision Operations to Meaningful Revision Units: Evaluating LLMs for Revision Boundary Detection
Yu Tian, Andrew Potter, Katerina Christhilf et al. (+4 more)
Revision traces provide valuable evidence about students' writing processes, but their usefulness for learning analytics depends on how individual revisions are represented. Automated draft-comparison methods often produce granular edit operations that can fragment a single purposeful revision into multiple analytic units. This study evaluates whether LLMs can identify meaningful revision unit boundaries in structured revision operation data and whether they provide value beyond simple non-LLM baselines. Using 113 matched draft–revision pairs from undergraduate writing, expert annotation yielded 4,344 candidate boundaries. We compared zero- and few-shot GPT-5.5 and base Qwen3-32B, parameter-efficient fine-tuning of Qwen3-32B, and majority and proximity-based baselines. Despite receiving revision context and task instructions, no prompted LLM condition outperformed the proximity heuristic (macro-F1 = .825). In contrast, fine-tuned Qwen3-32B using the two context representation achieved the highest macro-F1 (.859), identifying more same-unit relationships while maintaining precision comparable to the heuristic. Deterministic post-processing substantially improved the prompted models but added little benefit to the strongest fine-tuned model. These findings suggest that LLMs can support revision boundary judgment when task-adapted, but general purpose prompting alone may not outperform transparent structural heuristics.
Read the full abstract ▾
All authors Yu Tian, Andrew Potter, Katerina Christhilf, Motahareh Darvishpour Ahandani, Jessica Early, Steve Graham, Danielle S. McNamara
Information Theory
A proof of the Braun-Etzion-Vardy bound for binary subspace codes
Srikanth B. Pai
The notion of a linear subspace code in a projective space was introduced by Braun, Etzion and Vardy (2013), who conjectured that a linear subspace code in the projective space has at most 2^n codewords. We resolve this
conjecture. A novel character-theoretic argument is used to prove the
conjecture and the proof is self-contained.
Read the full abstract ▾
Combinatorics
A note on three-quarters circulant digraphs
C. Dalfó, M. A. Fiol and M. A. Reyes
We introduce and study a new family of circulant digraphs associated with the cyclic group $\Z_N$, obtained by restricting admissible combinations of two generators $a$ and $b$ to three coordinate sectors. The resulting distance-like function differs from the standard directed distance in circulant digraphs and gives rise to new
geometric and combinatorial phenomena. Using planar
lattice representations and periodic tessellations, we analyze the growth of reachable sets and derive Moore-type upper bounds for the corresponding order/diameter problem. We construct explicit infinite families of three-quarters circulant structures with the prescribed diameter and provide
lattice-based methods for determining admissible generator pairs. Separate constructions are obtained for even and odd diameters. In addition, computational experiments for small and moderate orders suggest improved families for even diameters and motivate a conjectural asymptotic formula for the maximum attainable order. The paper highlights the interplay between constrained
lattice representations, periodic tilings, and extremal problems for circulant networks.
Read the full abstract ▾
Artificial Intelligence
Self-Designed Evaluators and Warm Memory for Long-Horizon Agents
Saeid Asgari, Emre Kiciman, Leonardo de Oliveira Nunes et al. (+1 more)
A tool-using
language-model
agent deployed over a long stream of tasks receives no reward, so it cannot tell whether it succeeded, cannot safely retry, and cannot label the experience it needs to improve. We present SelfSuite, in which the
agent's own base model, given only the world's public materials, designs a small evaluation suite of weighted judges and grounded per-task briefs, freezes it, and uses it to gate a keep-best retry and to label a typed, outcome-tracked
memory. On matched five-repeat
benchmarks over tau2-bench and AppWorld, SelfSuite scores above the plain
agent without any labels, matches methods given ten expert labels on tau2-bench, and trails Agentic Context Engineering (ACE) on AppWorld, where code execution gives a direct success signal. In an ablation campaign run on the same tasks, it is above label-free ACE in every repeat, and the gated second attempt is the only component whose removal hurts in every repeat. We also simulate a subject-matter expert who grades ten onboarding tasks per world. Using those labels to calibrate SelfSuite's evaluator gives a small, consistent gain, and using them to warm up ACE's
memory lifts ACE to tie calibrated SelfSuite. A single-run study on a second model family shows the same ordering.
Read the full abstract ▾
All authors Saeid Asgari, Emre Kiciman, Leonardo de Oliveira Nunes, Ranveer Chandra