Information Theory · Cross-listed from cs.LG
Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting
Xinwei Qiang, Xiang Fang, Chang Chen et al. (+3 more)
Block drafters propose several
tokens in one forward pass, before earlier target
tokens are realised. Their rejection mixes two losses: missing within-block path information and imperfect modelling of observable information. Accepted length cannot distinguish them. We separate the two with an information floor, the minimum expected rejection at a specified conditioning order; rejection above this floor is the model gap. Measurements across four domains and four open-weight targets, plus floor estimates on a frontier API target, yield three findings. First, the all-parallel floor reaches $0.286$ at the final slot on Qwen3-4B, limiting even the best proposal to $71\%$ per-slot acceptance. Second, one realised
token removes $86$–$100\%$ of this floor, a locality also recovered by an independent mutual-information analysis. Third, current drafters remain far above their floors: the final-slot model gap accounts for $43$–$64\%$ of DFlash rejection and $85$–$92\%$ of DSpark's oracle-conditioned rejection. Guided by this separation of short-range conditioning from proposal quality, we replace a context-independent predecessor correction with a prefix-
attention head that reads committed context conditional on the predecessor. With supporting backbone components, the resulting Qwen3-4B drafter improves mean serving accepted length by $3.93\%$ over the released DSpark checkpoint across nine tasks, without increasing the conditioning order. Code and data: https://github.com/tie-pilot-qxw/specfloor; drafter checkpoints: https://huggingface.co/TIE-Pilot/dspark-attnconv-block7-qwen3-4b.
Read the full abstract ▾
All authors Xinwei Qiang, Xiang Fang, Chang Chen, Zaifeng Pan, Yue Guan, Yufei Ding
Information Theory
Deterministic Identification over Additive Gaussian Channels
Jonathan E. W. Huffmann and Holger Boche
Modern communication systems impose strict demands on data rate, reliability, and power efficiency. In this context, emerging communication paradigms such as identification via channels have become an important topic in post-Shannon information theory, offering the potential for substantially higher identification rates than in conventional channel coding.Deterministic identification is particularly interesting for specialized communication scenarios because it provides a balance between implementation complexity and the communication gains due to higher identification rates. It is therefore a promising communication scheme for future communication systems, including molecular communication systems. Additive Gaussian channels, particularly the additive white Gaussian channel, are among the most important channel models for analyzing the performance of communication systems in information and communication theory. This importance stems from both their mathematical tractability and their ubiquitous appearance in practical applications. To date the deterministic identification capacity for additive Gaussian channels remains unknown even for the simplest case of the additive white Gaussian channel. In this paper, we establish tight bounds on the deterministic identification capacity of additive Gaussian channels by introducing a new perspective on deterministic identification. To this end, we apply results from
lattice theory to obtain new capacity results.
Read the full abstract ▾
Information Theory · Cross-listed from quant-ph
Strong Converse Exponent of Quantum State Merging
Mario Berta, Hao-Chung Cheng, Roberto Rubboli et al. (+1 more)
We determine the strong converse exponent for the
entanglement cost of
quantum state merging, showing that it is characterized by the optimized $\alpha$-$z$ conditional Rényi
entropies with $z=\alpha/2\in[1/2,1]$. This contrasts with the sandwiched conditional Rényi
entropies that typically govern strong converse exponents in
quantum information theory. As a consequence, we derive the strong converse exponent of the partially smoothed conditional min-
entropy in purified distance. This exponent is governed by club-sandwiched conditional
entropies, whereas global smoothing leads to a sandwiched expression.
Read the full abstract ▾
All authors Mario Berta, Hao-Chung Cheng, Roberto Rubboli, Marco Tomamichel
Information Theory
Knowledge Distillation Driven Semantic NOMA with GAN Refinement for 6G Robotic Vehicle Networks
Qifei Wang, Zhen Gao, Li Qiao et al. (+4 more)
To achieve sustainable intelligent mobility, 6G-empowered robotic vehicles (RVs) require high-fidelity visual perception under stringent bandwidth and energy constraints. Semantic communication offers a spectral-efficient solution but suffers from severe interference in uplink non-orthogonal multiple access (NOMA) RV networks. To address this, we propose a knowledge
distillation-driven and generative models-enhanced NOMA framework for robust and green RV communications, named KDG-SemNOMA. First, we develop a ConvNeXt-based deep joint source-channel coding (DeepJSCC) architecture with an enhanced
attention feature (AF) module for dynamic channel adaptation. Second, to mitigate interference without inference overhead, an orthogonal transmission teacher model guides the NOMA student model via a two-stage knowledge
distillation strategy. Finally, to address the over-smoothing artifacts of pixel-wise optimization, we introduce a channel-conditional GAN (cGAN). By explicitly taking the Stage-I initial reconstruction and channel states as conditional inputs, this module refines coarse outputs into high-fidelity images with realistic textures. Experiments on FFHQ-256 demonstrate that KDG-SemNOMA significantly outperforms state-of-the-art methods in both pixel-level accuracy and perceptual fidelity.
Read the full abstract ▾
All authors Qifei Wang, Zhen Gao, Li Qiao, Ziwei Wan, De Mi, Dapeng Li, Ying Sun
Information Theory · Cross-listed from quant-ph
Conditional contraction coefficients and their applications to quantum networks
Christoph Hirche, Ian George, Theshani Nuradha et al. (+1 more)
Contraction coefficients quantify the loss of distinguishability induced by a channel and provide a strong form of the data-processing inequality. While standard contraction coefficients ignore auxiliary
quantum systems, existing extensions based on complete contraction coefficients require the compared states to have identical reference marginals. In this work, we introduce conditional contraction coefficients, a novel family that incorporates arbitrary
quantum reference systems by subtracting the distinguishability already present in the reference system. We develop a general framework for contraction coefficients with such
quantum side information, including the corresponding strong-data-processing-inequality (SDPI) constants, expansion coefficients, and relative contraction coefficients. For the trace distance, we show that the optimization can be restricted to orthogonal input states. For the
quantum relative
entropy, we prove that its conditional contraction coefficient is exactly equal to the contraction coefficient of the conditional mutual information, extending the classical correspondence between relative-
entropy contraction and mutual-information contraction to the setting with
quantum side information. More generally, we identify structural properties of divergences required for these results and discuss extensions beyond the relative
entropy. These results establish a unified framework for analyzing information contraction in
quantum network settings, where
quantum side information and distributed correlations are intrinsic features of the information-processing task. Applications include an extension of the Polyanskiy-Wu bounds on mutual information contraction, new perspectives on mixing times, and fundamental limits on
quantum memories.
Read the full abstract ▾
All authors Christoph Hirche, Ian George, Theshani Nuradha, Mark M. Wilde
Information Theory
Real-Time Reconstruction of Markov Sources over MPR Channels
Pansee S. Elessawy and Nikolaos Pappas
This paper studies the real-time reconstruction and remote actuation of two binary Markov sources over a shared wireless channel with multi-packet reception (MPR). Unlike many existing collision-based formulations that discard simulta- neous transmissions, we exploit MPR and evaluate communica- tion through reconstruction and actuation errors rather than raw delivery rates. We consider two sensors observing the sources and aim to find sampling policies that minimize the weighted real- time reconstruction error (RTE), or equivalently the weighted cost of actuation error (CAE) for the considered binary sources, under per-sensor sampling constraints. We first obtain closed- form expressions for the RTE and CAE in terms of the effective update probabilities. When the sensors randomize independently, the MPR-induced update-rate map becomes bilinear, and the constrained optimization is nonconvex. We exploit the geometry of the achievable update-rate region to show that the search over Pareto-efficient independently randomized policies reduces to a finite set of one-dimensional boundary-branch searches with closed-form candidates. As a
benchmark, we allow time sharing among joint sensor actions, and we prove that at most two Pareto- extreme modes are sufficient, and use this
benchmark to quantify the loss caused by independent randomization. Numerical results reveal that MPR capability alone is not sufficient; simultaneous decoding improves the task-oriented objective only when con- current reception is reliable for both sources; otherwise, policies that avoid simultaneous transmissions can be equally effective.
Read the full abstract ▾
Information Theory · Cross-listed from math.PR
A note on shifting distributions via Poisson races
Amir Yehudayoff
This expository note is about simulating a target distribution $Q$ from observations of a proposal distribution $P$. In the model suggested by Harsha, Jain, McAllester and Radhakrishnan, we observe an infinite sequence of i.i.d. samples $X_1,X_2,\ldots$ from $P$. The goal is to find some index $I \in \{1,2,\ldots\}$ such that $X_I$ is distributed like $Q$ while minimizing $\mathbb{E} \log I$. Following the Poisson-race approach developed by Maddison, Li and El Gamal, and others, this note shows that if $D(Q||P) < \infty$ then there is a $P$ to $Q$ simulator $I$ such that $\mathbb{E} [\log I] \leq D(Q||P) + 1.45 \|Q-P\|_1$. In the other direction, for every $P$ to $Q$ simulator $I$, the cost is at least $\mathbb{E}[\log I] \geq \frac{1}{2} \max \{ D(Q||P), \|Q-P\|_1 \}$.
Read the full abstract ▾
Information Theory · Cross-listed from eess.SY
Energy-Neutral Coverage Optimization by Joint Deployment and Scheduling in Ambient IoT Devices with Directional Sensing
David E. Ruíz-Guirola, Samuel Montejo-Sánchez, Richard Demo Souza et al. (+1 more)
Ambient IoT (A-IoT) devices rely on energy harvesting and duty cycling to sustain operation, thereby fundamentally changing collaborative sensing compared with traditional always-ON sensor networks. In this paper, we study the joint deployment and sensing scheduling of A-IoT devices equipped with directional sensing. We explore four solution strategies: (i) a grid deployment with static duty cycling, (ii) a centralized policy-gradient reinforcement learning (RL) approach that begins with a grid deployment and learns energy-aware device relocation and duty-cycling policies, (iii) a mixed-integer linear programming (LP) approach that couples static deployment design with duty-cycle allocation, and (iv) a hybrid LP+RL that combines optimization-based initialization with learning-based refinement. Using representative A-IoT use cases, we evaluate coverage as a function of device density, field-of-view, and maximum feasible duty cycle, determined by harvested energy and device consumption. Numerical results indicate that the proposed LP+RL and RL policies consistently outperform both the grid baseline and the LP-based method, achieving up to 2x higher mean effective coverage in low and medium energy harvesting (EH) regimes. In contrast, the standalone LP method remains limited by its conservative static duty cycle allocation under tight EH constraints. Moreover, the structured initialization of the LP+RL method substantially accelerates convergence, reducing the total offline optimization time by up to 10x compared to the standalone RL.
Read the full abstract ▾
All authors David E. Ruíz-Guirola, Samuel Montejo-Sánchez, Richard Demo Souza, Onel L. A. López
Information Theory
Minimum Rate For Partially Observable Linear System with Side Information: LQG Plant and Gaussian-Markov Source
Sijie Li and Hyeji Kim
This paper studies the minimum rate required for a partially observable linear system with side information. The Linear Quadratic Gaussian(LQG) plant and the Gaussian-Markov source are considered. We show that a class of linear policies is sufficient for optimizing the conditional directed information lower bound. We also show that the resulting optimization problem is convex for the scalar case in both time-varying and time-invariant systems. Our results generalize the past works that consider the case with full or partial observation only, and the case with full observation and side information. Numerical simulations are presented to illustrate the effect of side information for partially observable systems.
Read the full abstract ▾
Information Theory · Cross-listed from cs.PF
TOPIQ: Statistical Error Propagation for Quantity-of-Interest Prediction under Lossy Compression
Youyuan Liu, Bo Jiang, Taolue Yang et al. (+3 more)
Lossy compression is essential for managing massive scientific data, but per-element error bounds do not translate into bounds on downstream quantities of interest (QoIs) such as regional averages, neural network predictions, or multi-field derived quantities. We present TOPIQ, a statistical error-propagation framework that predicts QoI-level bias and uncertainty from compact compression metadata (less than 0.1% of original data). TOPIQ decomposes QoIs into primitive operators with closed-form propagation rules accounting for spatial error correlation and data-error coupling; new QoIs are supported by composition at runtime with no per-QoI derivation or retraining. Across 552 evaluations spanning 4
datasets, 3 compressors, 4 QoI families, and 8 error bounds, 93.1% of configurations achieve well-calibrated predictions. Pre-computed metadata enables post-hoc uncertainty quantification for arbitrary query regions at 56x-402x speedup over direct computation. A case study demonstrates integration into an AI-driven analysis pipeline with end-to-end confidence intervals for dynamically composed queries.
Read the full abstract ▾
All authors Youyuan Liu, Bo Jiang, Taolue Yang, Sheng Di, Robert Underwood, Sian Jin
Information Theory
Spectral Approximation and Ergodic-Capacity Convergence of HMIMO Channels under Spatial-Wavenumber Domain Mismatch
Hangsong Yan, Hong Yang and Shu Sun
We establish quantitative results on finite-dimensional spectral approximation and ergodic-capacity convergence for continuous Holographic Multiple-Input Multiple-Output (HMIMO) channels with square apertures and physically prescribed circular wavenumber support. The resulting spatial-wavenumber domain mismatch leads to a non-separable square-disk concentration problem for which the classical separable construction based on prolate spheroidal wave functions (PSWFs) cannot be directly applied. We project the continuous operator onto a tensor-product subspace of one-dimensional (1D) PSWFs while preserving the circular wavenumber support, yielding a generally non-diagonal but highly sparse finite-dimensional
matrix. We show that the whole-spectrum approximation error, accounting for retained-
eigenvalue perturbations and the residual spectral tail, remains controlled by a 1D PSWF
eigenvalue-tail envelope despite the loss of separability and induced off-diagonal coupling. Beyond an explicit 1D truncation threshold, this error decays super-exponentially. This analysis further yields an asymptotic upper envelope for the eigenspectrum under the flattened two-dimensional
eigenvalue ordering. We further establish a non-asymptotic upper bound on the gap between the actual ergodic capacities of the continuous and tensor-PSWF-truncated channels under their respective transmit-covariance optimizations. Combined with the spectral result, this capacity-gap bound inherits the same super-exponential dependence on the truncation order. Finally, quadrature rules with explicit radial and angular node thresholds are developed for evaluating the projected
matrix. Numerical results show that conventional truncation based on spatial degrees of freedom can omit performance-relevant modes, particularly for compact apertures.
Read the full abstract ▾
Information Theory · Cross-listed from cs.DB
Size Bounds for CQs Under Acyclic Constraints
Stefan Mengel and Andrei Romashchenko
We study size bounds for conjunctive query (CQ) results which in recent years have played a crucial role in database theory. In particular, we compare the so-called entropic bound which is known to be asymptotically tight but not known to be computable, and its computable relaxation called the polymatroid bound, which is generally not tight. We focus here on conjunctive queries under acyclic functional dependencies. These queries are known to be well-behaved in the sense that, in the case without projections, both bounds coincide. We show that this picture changes when projections are allowed: in this case, even for acyclic functional dependencies, there is in general a
polynomial gap between the two bounds. We complement this negative result by showing a special case for which the polymatroid bound is tight under acyclic functional dependencies and projections that is characterized by the position of the output variables in a topological order of the query variables.
Read the full abstract ▾
Information Theory
Sharp Minimax Regret for Infinite-Memory Logistic Prediction
Vaneet Aggarwal
We determine the minimax cumulative log-loss regret of a finite-alphabet, exogenously driven source with genuinely infinite input
memory: independent Rademacher inputs $(U_t)$ are observed sequentially and the next binary mark has logit $\sum_{j\ge1}\theta_jU_{t+1-j}$, the unknown coefficients obeying a summable envelope $|\theta_j|\le r_j$, $\sum_jr_j\le B$. At horizon $T$, lag $j$ can move the logit by at most $r_j$ and is exercised in only $n_{T,j}=(T-j+1)_+$ rounds, and the two limitations combine into the sum $\Gamma_T(r)=\sum_{j\le T}\log(1+n_{T,j}r_j^{2})$. One coordinate-localised
Bayesian mixture achieves $R_T(r)\le C\Gamma_T(r)$ for every summable envelope with $C$ universal. Our main result is a matching nonasymptotic converse for the canonical exponential and
polynomial envelopes; its new ingredients are a modular finite-sample information bound for logistic experiments with an exogenous random design, and a conditioning estimate for the overlapping Toeplitz lag
matrix obtained by exhibiting each off-diagonal Gram sum as a sum of independent Rademacher variables indexed by the edges of a forest, needing neither local asymptotic normality nor any spectral
theorem for random Toeplitz
matrices. So $\Gamma_T(r)$ is the minimax regret scale here, giving $\Theta(\alpha^{-1}\log^{2}T)$ for $r_j=Ae^{-\alpha j}$ and $\Theta(T^{1/(2s)})$ for $r_j=Aj^{-s}$, $s>1$ — the latter without the extra $(\log T)^{1-1/(2s)}$ factor any window-truncation analysis pays. We also show
memory decay cannot determine regret, and that a profile-scaled online Newton predictor attains $O_B(\Gamma_T(r))$ in
polynomial time per round.
Read the full abstract ▾
Information Theory
Decoding Algorithms for MDS Array Codes
Sara D. Cardell, Gustavo Terra Bastos and Cintya Wink de Oliveira Benedito
We study decoding procedures for a family of MDS array codes previously constructed from the Kronecker product of a superregular
matrix and a non-singular
matrix over a finite field. By exploiting the particular structure of their parity-check
matrices, we develop decoding
algorithms for different channel models. For the erasure channel, we provide an
algorithm capable of recovering any pattern of up to $n-k$ symbol erasures. For the $q$-ary symmetric channel, we investigate the decoding of one and two symbol errors and give explicit procedures for determining their locations and values. We also consider the particular case in which the superregular
matrix is a Vandermonde
matrix, showing how its additional algebraic structure can be exploited in the decoding process. Explicit examples over different finite fields are provided to illustrate the proposed procedures.
Read the full abstract ▾