Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Information Theory

cs.IT Watch Information Theory (sign in to use)

Monday 12 October

Information Theory · Cross-listed from quant-ph

Pathfinder-BP: A Low-Latency Recursive Decoder for Topological Codes

We propose Pathfinder-BP, a recursive decoder for topological quantum codes that combines belief propagation with controlled bit-flipping. An initial BP pass identifies and permanently fixes high-confidence errors. Upon convergence failure, the decoder recursively explores alternative error patterns by tentatively flipping low-confidence variables. On surface codes, Pathfinder-BP achieves a noise threshold and logical error rate comparable to minimum-weight perfect matching (MWPM). A crucial advantage of Pathfinder-BP is its highly parallelizable structure, which yields a worst-case decoding latency quasi-linear in the code distance, making it a suitable candidate for low-latency hardware implementation.
Read the full abstract

Information Theory

Log Density Spectrum Based Decoding of Polar Codes for Channels with Strong Phase Noise

A decoding method is proposed for polar codes under channels with strong phase noise. In contrast to traditional SC decoding, where recursions are performed on scalar soft information such as probabilities, the proposed decoding recurses over probability density functions defined on the phase-noise variable. To make the density-level recursion computationally tractable, we represent each log-density by its spectrum, and derive approximate formulas for the spectrum recursion, therefore, the proposed method is termed log-density spectrum recursion (LDSR). Numerical results are given under strong phase noise and AWGN, showing that a prominent superiority of error performance is achieved by LDSR in comparison with traditional SC and Kalman-based methods.
Read the full abstract

Information Theory

More Interactions Make Communications More Efficient: A Directed Information Bottleneck Approach to Task-Oriented Communication

Task-oriented communication improves communication efficiency by transmitting semantic information required for downstream tasks, but may incur redundancy with previously transmitted information. To address this issue, we propose a variational directed information bottleneck joint source-channel coding (VDIB-JSCC) framework that transmits only previously unacquired information. Specifically, we use directed information to characterize the causal flow of task-relevant incremental information and theoretically demonstrate its advantages over mutual information and conditional mutual information. Based on this metric, we formulate a directed information bottleneck (DIB) objective that balances task-relevant information preservation and redundant information compression. We further derive a variational formulation, termed VDIB, whose tractable upper bound serves as the training loss. The encoder integrates a Vision Transformer for frame-wise semantic extraction and a Mamba-based causal state-space model for capturing temporal dependencies and extracting incremental information, while the decoder uses a Mamba-based state-space model for task inference. Experiments on CIFAR-10 and CIFAR-100 video streams demonstrate that VDIB-JSCC achieves higher classification accuracy with less transmitted information than state-of-the-art semantic communication methods. These results show that leveraging historical knowledge enables task completion with less information, improving communication efficiency.
Read the full abstract

All authors Zhirun Zheng, Yiqin Deng, Phone Lin, Yuguang Fang

Information Theory

Generalized Hamming Weights and Subcode Support Weight Distributions of Twisted Reed-Solomon Codes

In this paper, we determine the generalized Hamming weight hierarchies of two classes of twisted Reed–Solomon (TRS) codes. For TRS codes with one twist, we first determine all possible values of the generalized Hamming weights. We then give a complete characterization of the weight hierarchy in terms of the minimum distance of the dual code. Explicit conditions are derived for the minimum distances of the code and its dual, which make the whole weight hierarchy explicitly computable. As special cases, the obtained results recover several known characterizations for TRS codes to be MDS, NMDS codes, and $s$-gap AMDS codes. For TRS codes with two twists, we similarly determine the complete generalized Hamming weight hierarchy. In this case, the higher generalized Hamming weights are determined by the first two generalized Hamming weights of the dual code, for which explicit rank conditions are also obtained. In addition, we further determine the subcode support weight distributions of this class of single-twist TRS codes. More precisely, for each $1 \le r \le k$, we derive an explicit formula for the number of $r$-dimensional subcodes with a prescribed support size, thereby giving a complete description of the support distribution of subcodes.
Read the full abstract

All authors Chao Liu, Hao Chen, Dabin Zheng, Yaqi Chen

Information Theory · Cross-listed from quant-ph

Classical and Quantum Information Measures Characterizing Convertibility: An Axiomatic Approach

In this PhD thesis, we take an axiomatic approach to studying multivariate divergences, which are measures of the information content of a tuple of classical or quantum states. These divergences are studied in the context of large-sample and catalytic convertibility of tuples of states. This type of convertibility is characterized by the ordering of an axiomatically defined family of divergences. In this thesis, we identify this family of divergences explicitly for two scenarios, one classical and one quantum. In the classical setting, we show that the relevant divergences are certain generalizations of the bivariate Rényi divergences. Unlike previous studies, we do not require the distributions to have the same support, yielding interesting new divergences. In the quantum setting, we consider pairs of classical-quantum states with pure component states, and find that majorization is characterized by all $\alpha$-$z$ relative entropies for $\alpha<1$. This demonstrates the first operational interpretation of these relative entropies.
Read the full abstract

Information Theory

RPA Decoding with Few Subspace Projections for Correcting Structured Error Patterns in RM Codes

The Recursive Projection Aggregation (RPA) decoder for Reed-Muller codes uses, in its default implementation, all one-dimensional subspaces at each level of recursion. We restrict our attention to the case where the number of errors is $t<d_{\mathrm{min}}/2$. It is known that if the number of errors is $t<d_{\mathrm{min}}/2$, then $2t+1$ subspaces at each level of recursion are sufficient to correct these errors. We show that, for structured error patterns, the number of subspaces required for correction can be reduced further by appropriately selecting the subspaces used for projection. In particular, when errors occur as a single burst of length $t < d_{\mathrm{min}}/2$, three subspaces at each level of recursion suffice for correction. We obtain similar results for multiple bursts and other structured error patterns as well.
Read the full abstract

Information Theory

Learning Rules, Memorizing Facts: An Information-Theoretic View of Hallucination

Understanding the relationship between generalization and hallucination requires clarifying the role of memorization. We study this relationship from an information-theoretic perspective through a Selector-Density Conditional Mutual Information (CMI) framework. In the sparse random-fact limit, a uniformly bounded information budget per fact forces a nonvanishing false-positive rate at recall bounded away from zero, even when all facts are observed during training. In a rules-and-facts setting, we decompose model-data mutual information into rule learning, atomic-fact memory, and excess memorization, revealing competition between rule learning and factual memory even without excess memorization. A joint budget bound quantifies the resulting constraint on atomic hallucination under perfect atomic recall. Moreover, an additive Gaussian model shows that maintaining high recall for atomic and composite facts incurs false positives even under optimal data mixing. Finally, controlled relational experiments hint how training exposure, rule complexity, and model capacity shape these trade-offs.
Read the full abstract

All authors Gengze Xu, Anxin Guo, Yifei Hu, Xiangyu Wu, Yong Liu, Ziqiao Wang

Information Theory

Double circulant LCD codes over $\mathbb{F}_2+u\mathbb{F}_2$ and constructions for binary LCD codes

Linear complementary dual (LCD) codes are linear codes that intersect their dual trivially. We study double circulant (Euclidean) LCD codes over the finite chain ring $R=\mathbb F_2+u\mathbb F_2$, where $u^2=0$. Using the Chinese Remainder Theorem and the factorization of $x^n-1$, we derive an explicit enumeration formula, with a separate treatment of the constituent corresponding to $x-1$, which is not obtained by a formal specialization of the previously known counting formula over $\mathbb F_q[u]/(u^k)$. Assuming Artin's primitive root conjecture for $2$, we show that the binary Gray images contain an asymptotically good family of rate $1/2$ whose relative distance can approach $H_2^{-1}(1/4)$ from below. More importantly from the constructive viewpoint, by combining Gray images with a known extension construction, we obtain a binary LCD code with parameters $[29,15,7]$ and construct two explicit permutation-inequivalent binary LCD codes with parameters $[45,23,9]$.
Read the full abstract

Information Theory

Fourier entropy of threshold functions and their compositions

We prove the Fourier entropy–influence inequality for every linear threshold function, with a constant below $8.937$ and no restrictions on its weights or threshold. The argument applies to regular Boolean functions and gives logarithmic bounds for their Boolean combinations and for unions with bounded variable overlap. Character multiplication preserves the estimates and yields bounds by influence minus variance. Under arbitrary product measures, symmetric thresholds satisfy the strengthened inequality needed for read-once composition, uniformly in depth and fan-in. A joint description of each Fourier set refines the overlap estimate and recovers constant bounds for addressing. Exact selector identities extend the bound to orthogonal multiplexers with shared data. Rare activation preserves the optimal constant of unrestricted FEI. The proofs combine pivotal antichains, spectral reference measures, and product-measure composition.
Read the full abstract

Information Theory

Robust Quickest Change Detection with Cost-Efficient Experiment Design and Unknown Post-Change Model

A quickest change detection problem is studied where the decision-maker has access to multiple experiments, but can choose only one experiment at a time to detect the change. The true post-change distribution under each experiment is unknown, but known to belong to a family of distributions. It is assumed that each of these families of distributions has an element that is least favorable in a well-defined sense. An algorithm is proposed that is designed using the least favorable distributions. It is shown that the algorithm is first-order asymptotically optimal, as the rate of false alarms goes to zero. This algorithm minimizes the worst-case average detection delay over all post-change distributions within the assumed families, subject to a constraint on the average run length to false alarm and constraints on the average fraction of time each experiment is performed before change.
Read the full abstract

Information Theory

Delay-Doppler Duality: Simplifying Exact Equalization of Zak-OTFS in Time or in Frequency

The Zak-OTFS carrier waveform is a pulse in the delay-Doppler (DD) domain, and given the input-output (I/O) relation at one DD point in a frame, we can predict the I/O relation at all other points. While it is simple to acquire the effective wireless channel with Zak-OTFS, the cost of embracing inter-carrier interference is equalization complexity. The cost of linear minimum mean-square error (LMMSE) equalization grows cubically with the frame size. We describe how to use channel knowledge to reduce equalization complexity. For any pulse shaping filter, and for fractional delays and Dopplers, we show that the DD channel matrix is unitarily similar, through the discrete Zak transform, to a time-domain (TD) matrix that is a weighted sum of cyclic delay and Doppler operators, where the weights are the taps of the effective channel. We also obtain a dual frequency-domain (FD) form through an additional Fourier transform. The TD matrix is banded with a width set by the delay spread that is independent of the Doppler spread, whereas the FD matrix is banded with a width set only by the Doppler spread. We design a receiver that reads off the two widths from the channel estimate, selects the cheaper domain, and solves the banded system exactly, using a bordered Cholesky factorization that treats the quasi-periodic wrap-around without sacrificing degrees of freedom. Filters that decay slowly need a few preconditioned conjugate gradient steps. We show that the complexity of equalization becomes linear in the frame size. Simulations match direct DD-LMMSE error rates with three to four orders of magnitude fewer multiplications. A single receiver handles channels with very different delay and Doppler characteristics.
Read the full abstract

Information Theory · Cross-listed from cs.LG

When Does Compensating Inherited Quantization Error Help? Mean-Square Guarantees for Layer-Wise Quantization and Their Limits

Layer-wise post-training quantization methods let each layer compensate for the quantization error inherited from earlier layers. We ask under what assumptions such compensation guarantees a network output error no larger than without it. For linear networks of any depth, with the required calibration Gram matrices invertible, if the rounding error of each layer has zero conditional mean and a deterministic second-moment map common to both schemes, as under subtractive dither on an unbounded grid, then compensation with strengths in $[0,1]$ fixed in advance gives a mean-square output error no larger than no compensation, each scheme being followed along its own trajectory. The exact gain separates error removed directly from reduced excitation of later rounding noise. Outside this setting the comparison can fail. With rectified linear activations it fails for two independent reasons: noise changes activation states, and, even when no state changes, the correction moves error from samples on which a unit is inactive to samples on which it is active. With stable activations, commutators between the calibration projection and the activation patterns bound the excess. With a bounded codebook, even unique exact layer-wise minimizers with no overload can give a larger output error. For fixed rounding errors it already fails for two scalar linear layers, refuting a published universal guarantee. An exact residual identity gives the necessary and sufficient condition on a common layer input. All these comparisons hold the calibration input fixed. On two 8B models at 4 bits, fixed half-strength compensation beat uncompensated rounding in every model, text and draw tested, yet no rule for choosing the strength beat a layer-wise rule consistently, and lowering the full-network loss on the data used to choose did not reliably lower it on new text.
Read the full abstract

Sunday 11 October

Information Theory

Stabilizer entropy cones as self-dual slices of linear rank cones: a five-party gap and dependence on the local dimension

Entanglement entropies of multipartite pure states satisfy linear inequalities; for stabilizer states, the backbone of quantum error correction, the achievable entropy vectors generate a convex cone. We show that, after a linear lift, the closure of this cone is exactly the self-dual slice of a classical object on one more element than there are parties: the closed cone generated by the rank functions of arrangements of subspaces over $\mathrm{GF}(d)$ for a prime local dimension $d$, and of subgroups of abelian groups of exponent dividing $d$ in general. This correspondence brings two features of rank inequalities into play. First, the purifier acts as an extra variable: already for five parties the stabilizer cone is strictly smaller than the quantum linear rank cone, with which earlier evidence suggested it might coincide; an explicit six-variable inequality witnesses the gap. Second, the characteristic of the field enters, so the cone depends on the local dimension. It does not for up to four parties. With enough parties, the cone of $\mathbb{Z}_d$ qudits is contained in that of $\mathbb{Z}_{d'}$ qudits if and only if $d$ divides $d'$, and already for six parties the logical zero state of Steane's code lies outside the stabilizer cone of every odd local dimension. For this last result we show that the seven-variable characteristic-dependent inequalities we checked all follow from Shannon inequalities on the self-dual slice, and instead transfer one of them through a family of subgroups derived from the Fano plane.
Read the full abstract

Information Theory

Embedding Information in Policy Randomness: Distribution-Preserving Implicit Communication in MDPs

This paper investigates a new role of randomness in stochastic policies–as a medium for embedding and communicating information. Specifically, we consider an agent interacting with an environment according to a prescribed stochastic policy. By carefully modulating the random action realizations of the policy, information can be embedded into the agent's behavior without altering the prescribed action distribution. We cast this problem as a channel coding problem, in which the controlled system implicitly serves as a communication channel. We then propose a coding scheme, termed posterior-matching randomness modulation (PRAM), that selects actions according to the message to be transmitted. With the aid of common randomness independent of the control system, PRAM preserves the prescribed action distribution for every state and every message. We further prove that PRAM is asymptotically reliable: the message can be decoded with arbitrarily small error probability as the observed state trajectory becomes sufficiently long. Experiments demonstrate the effectiveness of the proposed framework in applications including multi-bit policy watermarking and trajectory provenance.
Read the full abstract

Information Theory

Exact Privacy–Distortion Design of Support-Constrained Randomized Response

Full-support local randomizers can incur unnecessary reporting distortion on ordered alphabets. We study randomized response whose output is restricted to a radius-$t$ neighborhood of the private input. Input-dependent supports create privacy leakage at their boundaries, so ordinary likelihood-ratio calibration is no longer sufficient. Under approximate local privacy restricted to input pairs at distance at most $H$, we derive the exact hockey-stick defect of the mechanism. We then solve the joint design of the truthful probability, off-truth probability, and support radius. The fixed-radius optimum is available in closed form, while the global optimum requires only a one-dimensional integer search. We characterize both the privacy floor and its inverse, namely the smallest defect attainable under a reporting-distortion budget. We further prove that stringent approximate privacy forces the support size to scale as $\Omega(H/\delta)$. Deterministic evaluations illustrate the resulting tradeoff.
Read the full abstract

Information Theory · Cross-listed from quant-ph

Quantum simultaneous decoding via pretty-good measurements of tilted states

A multiple-access channel transmits messages from several senders to one receiver. For quantum outputs, the receiver must distinguish states depending on all messages, while tests for different decoding errors may not commute. In this paper, we give a self-contained account of simultaneous decoding, in which one joint measurement recovers the messages at the established achievable rates using independent codebooks. The proof combines Sen's tilted-subspace method with Cheng's analysis of the pretty-good measurement, a standard measurement for distinguishing quantum states. Auxiliary labels close to a common reference state separate the geometric directions associated with different errors. A Gram-matrix bound then allows separately chosen binary tests to certify the performance of one measurement. The labels are removed from the completed decoder, leading to a deterministic packing theorem that applies to finite state ensembles, including unequal priors. We develop the two-sender case first, with worked examples and figures, before treating any fixed number of senders. Sharper finite-error guarantees for the same decoder are developed separately, with sharp bounds for two specified steps of the analysis rather than a claim of globally optimal decoding. Applications recover known achievable rates for unassisted and entanglement-assisted classical communication, distributed classical compression with quantum side information, and coherent-state communication through a beamsplitter or passive multiport device. The common proof makes the geometric mechanism and its quantitative consequences available without repeating the construction for each application.
Read the full abstract

Information Theory

Fine asymptotics for approximate simulation of symmetric channels

We study the approximate channel simulation problem in the normal approximation regime. Our starting point is the fact established by Cao et al.~\cite{Cao_channel_simulation}, that the minimal codebook size $M_\epsilon^*(W)$ for a fixed error tolerance $\epsilon>0$ is closely related to the $\epsilon$-smooth channel max-information, $I_{\max}^\epsilon(W)$. Using this, we refine earlier large-blocklength approximations to $M_\epsilon^*(W^{\otimes n})$ for discrete memoryless channels. Specifically, we provide tight third-order expansions for Gallager-symmetric channels and demonstrate that the third-order term is channel-dependent. As in corresponding channel coding expansions, we identify a substantial rate gap between singular and nonsingular channels. Notably, the direction of this gap is reversed relative to channel coding: The third-order term is $-\frac{1}{2}\log n$ for singular channels but $O(\log\log n)$ for nonsingular channels.
Read the full abstract

Information Theory

Higher-order nonhomomorphicity under affine restrictions

We study the behavior of the $k$-th-order nonhomomorphicity of Boolean functions under restriction to affine subspaces. For odd $k\ge3$ and $0\le t<n$, we introduce the $t$-restriction $k$-th-order nonhomomorphicity $P_{k,t}(f)$, defined as the minimum of $P_k(f_{\mid A})$ over affine spaces of codimension at most $t$. For a direct sum of $m$ monomials of degrees at least two, we determine $P_{k,t}$ exactly. When $t<m$, a minimum is obtained by setting one variable equal to zero in each of the $t$ monomials of lowest degree. When $m\le t<n$, the minimum is zero. The proof combines multiplicativity of even Walsh moments with a contraction argument for affine restrictions. We establish a general formula for the Walsh transform of a Boolean function restricted to an affine subspace of arbitrary codimension, yielding an exact spectral characterization of the $t$-restriction normalized nonlinearity $\rnnl_t(f)$. As odd $k$ tends to infinity, $(1-2P_{k,t}(f))^{1/(k+1)}$ converges to $1-2\rnnl_t(f)$. We further prove that every non-affine function has a hyperplane restriction with strictly smaller odd-order nonhomomorphicity. Finally, we determine $\rnnl_1(T_{d,n})$ exactly for all threshold functions with $n\ge3$. This proves the two threshold-function conjectures left open in~\cite{SergeAnaPante2025}, including the previously unresolved odd-majority case, where a minimizing hyperplane is $x_1+x_2=0$.
Read the full abstract

Information Theory

Low-Latency Belief-Propagation Decoding of Quantum LDPC Codes with Reinforcement-Learned Scheduling

Belief propagation (BP) is attractive for quantum low-density parity-check (QLDPC) decoding because of its low complexity and low latency, but short cycles and degeneracy can limit its error-correction performance. Reinforcement-learned sequential scheduling (RL-S) substantially improves this performance by adapting the variable node (VN) update order, although Q-table storage and heap-based selection introduce additional overhead. We propose bucket RL-S (B-RL-S), which quantizes learned Q-values into integer priorities and schedules VN updates using dynamically maintained first-in, first-out buckets. Without retraining or changing the BP message updates, B-RL-S reduces policy storage and refreshes only priorities affected by local residual syndrome changes. A fixed number of priority levels enables constant-time VN selection and queue relocation. For the $[[288,12,18]]$ bivariate bicycle code over the depolarizing channel at $p=0.05$, B-RL-S with eight priority levels achieves approximately one fifth the block error rate (BLER) of flooding BP followed by order-10 ordered-statistics decoding. The respective caps are $1000$ sequential iterations for B-RL-S and $1000$ flooding iterations for the BP stage of the reference decoder. These results highlight strong error correction with inexpensive scheduling and no additional post-processing stage.
Read the full abstract

All authors Mohsen Moradi, Ghilas Aissou, Taejoon Kim, Remi A. Chou, David G. M. Mitchell

Information Theory

Residual-Information Generalization Bounds Under Selective Privacy: Extended Version

Information-theoretic generalization bounds usually control dependence on the task sample, whereas selective privacy may constrain only a different, private coordinate. We study i.i.d. pairs $(X,Y)$, where a learner observes $(X^n,Y^n)$, leaks at most $\epsilon_n$ nats about $X^n$, and incurs loss on $Y$. A chain-rule transfer converts this mismatched constraint into the closed-form universal certificate $I(W;Y^n)\leq\min\{nH(Y),nH(Y\mid X)+\epsilon_n\}$. Combining the certificate with mutual-information generalization yields a bound governed by residual task uncertainty and normalized privacy leakage. We derive complementary task-compression and individual-sample refinements, identify a sharp obstruction showing why selective privacy alone need not give a vanishing certificate, and invert the bound to obtain the exact leakage allowance for a prescribed generalization tolerance. We prove exact information-level achievability for deterministic-task and independent-residual source families and show that the resulting $\sqrt{\epsilon_n/n}$ generalization rate is tight up to a universal constant. We also give a fixed-horizon adaptive application in which conditional leakage costs add.
Read the full abstract

Information Theory

Superpolynomial Lists for Reed-Solomon Codes over Binary Extension Fields

Recent work shows that ordinary Reed–Solomon codes over prime fields admit polynomial-size lists up to list-decoding capacity on every evaluation set. We show a contrasting phenomenon over binary extension fields. For every fixed rate $0<R<1$, there exists $\varepsilon_R>0$ such that, for all sufficiently large $h$, the full-field Reed–Solomon code over $\mathbb{F}_{2^h}$ has a received word with $n^{\Omega_R(\log n)}$ codewords within relative distance $1-R-\varepsilon_R$. To our knowledge, these are the first superpolynomial ordinary Reed–Solomon list-size lower bounds at fixed positive rate and a fixed positive agreement gap above the rate. Thus capacity-approaching polynomial list size for Reed–Solomon codes is not uniform over the underlying field, and the obstruction occurs at every fixed rate. As a puncturing consequence, the same holds for primitive Reed–Solomon codes; the obstruction is also robust under controlled dense puncturing. The proof begins with an exact correspondence that turns certain binary Bose–Chaudhuri–Hocquenghem weight-enumerator coefficients into sizes of single Reed–Solomon lists. A classical binary weight family gives the initial quarter-rate construction, while a quadratic-form enumeration supplies its superpolynomial multiplicity. A common-kernel quotient/descent argument and common-root padding then extend the construction to all rates. We also determine the optimized agreement envelope achieved by the construction.
Read the full abstract

Information Theory

On the Average List Size of VT Codes under Two Insertions or Two Deletions

Levenshtein's classical theorem shows that any code that can uniquely decode from \(t\) deletions can also uniquely decode from \(t\) insertions. This symmetry, however, does not generally hold in the list-decoding regime. Several prior results on the capacity and radii of list-decoding from insertions and deletions suggest that insertions are easier to handle than deletions. In an effort to better understand this asymmetry between insertions and deletions under list-decoding, we study an analogous version of McEliece's average list size problem. More precisely, given a binary code \(\mathcal{C}\) of length \(n\) that uniquely corrects \(t'\) insertions (or deletions), we are interested in the average number of codewords returned by a \(t\)-insertion (or deletion) list decoder, where \(t' < t\). Focusing on the case of \(t'=1\) and \(t=2\), we find that under two insertions, this quantity is at most \((n+12)/4\) for any VT code of length \(n\), whereas under two deletions it is at least \(n/2 - 10\) when averaged over all length-\(n\) VT codes. The nearly factor-of-two gap between these upper and lower bounds serves to confirm the relative ease of list-decoding of insertions compared to deletions from an average-case perspective.
Read the full abstract

All authors Anisha Banerjee, Roni Con, Antonia Wachter-Zeh, Eitan Yaakobi

Information Theory · Cross-listed from math.PR

Sharp Rogers–Shephard Inequalities for Rényi Entropy

We establish sharp weighted Rogers–Shephard inequalities for Rényi differential entropy. For independent copies $X,X'$ of a log-concave random vector in $\R^n$, we prove $h_p(aX-bX')\le h_p(X)+n\log(a+b)$ for all $a,b>0$ and $1\le p\le\infty$, where orders $1$ and $\infty$ correspond to Shannon entropy and min-entropy, respectively. The Shannon case settles the entropic Rogers–Shephard conjecture of Madiman and Kontoyiannis and its higher-dimensional formulation. For $1\le p<\infty$, equality holds precisely for invertible affine images of random vectors with independent exponential coordinates. At $p=\infty$, the larger equality class is characterized by possibly extended-valued convex potentials that are positively homogeneous of degree one. For every $0<p<1$, uniform laws on sufficiently high-dimensional simplices violate the unweighted bound. Thus $p=1$ is the critical order for this bound to hold in every dimension. The proof combines weighted convex intersections with an exponential lift to a log-affine probability density. At the Shannon endpoint, this lift yields an exact deficit decomposition with a conditional mutual information term for an additive channel with asymmetric Laplace noise.
Read the full abstract

All authors Bang-Xian Han, Guoqing Zhan, Chiyu Zhou, Zhuo-Nan Zhu

Information Theory

Correcting Burst Errors with Applications to DNA-Based Data Storage

We introduce the problem of permutation burst error-correction, motivated by applications in molecular data storage with nanopore sequencing read platforms. A permutation burst of length $k$ is an error that affects a substring of length $k$ of a string of length $n$ by permuting its entries. Permutation burst errors can be corrected using classical burst error-correcting codes, such as Fire and Reed-Solomon codes. Here, we study how to reduce the redundancy of these classical codes by taking into account that the burst error preserves the weight of the substring. In addition, we further restrict our model to handle cyclic permutation bursts only, which are of independent practical and mathematical interest. Our main results include proving/establishing the following: a) for $k=o(\sqrt{n}),$ the redundancy of any permutation burst error correcting code is at least $\log n-k+\log \binom{2k-2}{k-2}+o(1).$ This result follows from McDiarmid's concentration bound; b) for $k \geq 9$ and $k=o\left(\sqrt{\frac{n}{\log n}}\right)$, the redundancy of any cyclic permutation burst error correcting code is at least $\log n +\log (k(k-1)-56)-\log\,80+o(1).$ c) the asymptotic redundancy upper bounds for the two types of substring errors that are twice the lower bound; d) constructions for both permutation burst and cyclic permutation burst errors that involve Varshamov-Tenegoltz type constraints or represent generalizations and extensions of Fire and other codes. In particular, we exhibit polynomially encodable and decodable codes that can correct cyclic bursts of length $\leq k$ using $\log n + k + 49\log k + O(\log \log k)$ redundant bits.
Read the full abstract

Information Theory

Exact Sampling Under Campbell Cost via Rényi Channel Simulation Divergence

We study the communication cost of exact sampling under Campbell's exponential coding criterion. Given a target distribution $P$ and a proposal distribution $Q$, the sender and receiver share i.i.d. $Q$-distributed samples, and the sender communicates an index so that the selected sample has distribution exactly $P$. We introduce a new information measure Rényi channel simulation divergence $R_\alpha(P\Vert Q)$, a Rényi generalization of the channel simulation divergence. We then show that it lower bounds the Campbell cost of every exact sampler and is achievable, via a new layered sampling scheme, within an additive constant depending only on the Campbell parameter. This yields a tighter one-shot characterization than previous bounds based on Rényi divergences of different orders. For i.i.d. product distributions, under suitable non-degeneracy and moment conditions, we further show that the gap beyond the leading Rényi-divergence term is $\frac{1}{2}\log n+O(1)$.
Read the full abstract

Information Theory · Cross-listed from eess.SP

Bistatic Delay-Doppler Sensing for 3D Target Localization in Distributed Wi-Fi Systems

This paper investigates how bistatic delay information improves the observability and localization performance of 3D moving targets in distributed Wi-Fi sensing. Motivated by emerging IEEE 802.11bf WLAN sensing, we consider one transmitter, multiple distributed receivers, and a single target with unknown position and velocity. We compare Doppler-only sensing with delay-Doppler sensing, in which delay-capable receivers provide additional bistatic path-length information for target localization. We derive the Fisher information matrix (FIM) and the position equivalent FIM (EFIM) for both sensing formulations. The resulting observability analysis shows that Doppler-only localization generically requires six receivers, whereas delay-Doppler localization can become locally identifiable with three receivers under non-degenerate geometry. We further study mixed receiver architectures in which only a subset of receivers provides delay estimates. Simulations with Wi-Fi-like bandwidths confirm the transition and quantify the roles of bandwidth, Doppler noise, receiver geometry, and delay-capable receiver availability.
Read the full abstract

All authors Kyungmin Hong, Moein Ahmadi, Thomas Stifter, Mounim A. El Yacoubi, Bhavani Shankar Mysore R

Information Theory

Binary Codes Correcting Multiple Edits via Polynomial Hashing and Entropy Recycling

Codes in the Levenshtein metric that can correct multiple edits (including deletions, insertions, and substitutions) have attracted a lot of attention recently owing to their application in DNA-based storage and racetrack memories. In this work, we present new techniques for designing explicit binary codes of length $n$ that correct $t$ edits with at most $(2t+1)\log n+O_t(\log\log n)$ bits of redundancy. We first propose to use a polynomial hash to construct a systematic code that corrects $t$ edits with $(4t+2)\log n+O_t(\log\log n)$ bits of redundancy. Then we observe that there are many polynomial hashes like this, some are good, some are bad. So, we propose the second technique, called entropy recycling, to use the indices of good hashes to carry information. As a consequence, we obtain a partial-systematic code of length $n$ that corrects $t$ edits with at most $(2t+1)\log n+O_t(\log\log n)$ bits of redundancy. Finally, we provide deterministic polynomial-time encoding and decoding algorithms for our code.
Read the full abstract

Information Theory

Rate-Distortion Theory of Gaussian Mixtures via Posterior Label Sampling

We study the mean-squared error (MSE) rate-distortion function (RDF) of finite Gaussian mixture models (GMMs) in $\mathbb{R}^d$, for which neither the RDF nor an optimal test channel is known in closed form. It can be sandwiched between the genie-aided conditional RDF $R_{\mathbf{X} | T}(D)$, which can be characterized in closed-form for a GMM, and $R_{\mathbf{X} | T}(D)+I(\mathbf{X};T)$, where $T$ denotes the latent component label. We show that the upper bound can be achieved operationally by the proposed Gibbs-relabelling method (GRM), which conveys a label from the posterior $P_{T|\mathbf{X}}$ through channel simulation at rate $I(\mathbf{X};T)$, followed by vector quantization of the resulting exactly Gaussian conditional sources by reverse water-filling. Using the Shannon lower bound (SLB), we prove that GRM attains the SLB, and hence the RDF, of a general finite mixture if and only if every component attains its own SLB. For a GMM this holds throughout the fully active interval $D/d<\min_{k,m}\lambda_{k,m}$, where $\lambda_{k,m}$ are the eigenvalues of the component covariances, and for scalar GMMs the condition is also necessary. Since GRM cannot operate below the label cost $I(\mathbf{X};T)$, we propose a complementary stochastic scalar quantization (SSQ) scheme for low rates, which simulates an $M$-ary reconstruction index. We show that at low rates SSQ with $M\geq 2$ matches the Shannon RDF to first order: $R(D_{\max}-\delta)=\delta/(2\tau^2)+o(\delta)$, where $\tau^2$ is a directional sub-Gaussian proxy of the mixture, and quantify the price of a finite codebook through a strict rate gap at high rates and a distortion floor whose joint scaling in $M$ and number of mixture components is characterized. Numerical results illustrate both regimes and the finite-blocklength cost of channel simulation.
Read the full abstract

Information Theory · Cross-listed from math.CO

Hull Variation in Symplectic Linear Codes

We study the variation of hull dimensions of symplectic linear codes under equivalence transformations preserving the symplectic distance. For a symplectic linear code $C$, we introduce an even-valued invariant $\Delta(C)$, which depends only on the equivalence class of $C$. This invariant can be expressed as the generic rank of an associated space of alternating matrices and, equivalently, as twice a corresponding matroid matching number. If $C$ has dimension $k$, we prove that the minimum symplectic hull dimension among all codes equivalent to $C$ is $k-\Delta(C)$. Consequently, $C$ is equivalent to a symplectic LCD code if and only if $\Delta(C)=k$. We further show that every symplectic MDS code of even dimension is monomially equivalent to a symplectic LCD code. For a symplectic self-orthogonal code $C$, we determine all attainable hull dimensions: for every integer $r$ with $0\le r\le \Delta(C)/2$, there exists a monomially equivalent code whose symplectic hull has dimension $k-2r$. We also refine a standard form for the generator matrix of a symplectic self-orthogonal code and derive the corresponding parity-check matrix and the Gram matrix of the dual code. In the self-dual case, we characterize $\Delta(C)$ through minors of an associated symmetric matrix. Furthermore, we relate hull variation to graph matchings, establish a bound for $\Delta(C)$, and give graph constructions producing symplectic self-dual MDS, optimal, and best-known codes.
Read the full abstract

Information Theory

SiteEvo: An LLM Agent for Autonomous Discovery of Site-Adaptive Massive MIMO Detectors

Designing a massive multiple-input multiple-output (MIMO) detector that is tailored to each specific deployment site can improve detection performance, yet most existing detectors, including deep-unfolding methods, adapt mainly within human-prescribed algorithmic structures. This letter proposes SiteEvo, a site-adaptive agentic discovery framework that extends adaptation from parameter tuning to detector-algorithm discovery. Rather than selecting or recombining predefined architectures, the large language model (LLM) agent synthesizes complete executable detector programs under a common design contract by combining its pretrained communication knowledge, site-specific wireless evidence, and experience accumulated during the search. Each proposal defines its own processing structure and trainable parameterization, and the parameters are optimized by a communication-system backend. The resulting high-performance detector is further structurally refined to reduce block-level computational cost under a bit error rate (BER)-preservation constraint and is finally frozen for deployment without online LLM inference. Simulations across three sites demonstrate BER reductions by factors of 2.6 to 6.0 over the best-performing competing detector at each site at 6 dB, while structural refinement reduces block-level computational cost by 16.7-34.1% under a validation-BER constraint.
Read the full abstract

All authors Jiajia Guo, Chao-Kai Wen, Shi Jin, Jun Zhang