Peer Revenue Fully funded

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Information Theory

cs.IT Watch Information Theory (sign in to use)

Monday 20 July

Information Theory · Cross-listed from eess.SP

Radio Map Updating from Streaming Spectrum Measurements via Memory-Based Online Gaussian Processes

Radio maps, which estimate spatial radio-frequency characteristics from spectrum measurements, are essential for applications such as spectrum management and network planning. With the continuous arrival of spectrum measurements, conventional batch processing methods for radio map reconstruction become computationally prohibitive, as they require reprocessing all accumulated measurements for each radio map update. To address this, we propose a memory-based online sparse variational Gaussian process (M-OSVGP) method that efficiently updates radio maps from streaming spectrum measurements. Our method employs sparse variational inference and updates the posterior online by minimizing a hybrid objective that integrates newly received measurements and a memory subset of previous ones to mitigate catastrophic forgetting. To further improve posterior approximation as measurements accumulate over spatially diverse regions, we extend M-OSVGP with a grid-assisted online inducing point selection (GOIPS) algorithm. GOIPS dynamically adapts the number and locations of inducing points based on measurement density and spatial correlation, providing a more informative inducing set while maintaining computational efficiency. Extensive simulations demonstrate the effectiveness of our proposed methods in reconstruction accuracy, computational efficiency, and uncertainty quantification, compared to existing batch and online baselines across various scenarios.
Read the full abstract

All authors Yuanyuan Deng, Bo Zhou, Tian Chen, Shijian Gao, Jia Yan, Lantu Guo, Qiuming Zhu, Qihui Wu

Information Theory · Cross-listed from cond-mat.stat-mech

A Unified Discrete and Continuous Theory of Core-Halo Complexity Maximizers

The maximization of statistical complexity has long been associated with the emergence of probability distributions lying between perfect order and complete disorder. While previous studies have shown that complexity-maximizing distributions exhibit a two-level structure in finite discrete systems, an analogous unified treatment for both discrete and continuous probability spaces has remained unavailable. In this work, we develop a general variational framework for a generalized statistical complexity constructed from Shannon and Renyi entropies. We derive a common stationary equation governing both discrete probability masses and continuous probability densities and prove that every stationary solution necessarily possesses exactly two probability levels, establishing a universal core-halo structure. We further demonstrate that the optimization problem reduces to a single multiplicity parameter and prove that the global complexity maximum is attained by the smallest admissible core, corresponding to a single dominant state in the discrete case and an infinitesimal core in the continuous limit. These results provide a complete analytical characterization of the complexity-maximizing distributions and reveal a common mathematical structure underlying complexity optimization in both discrete and continuous settings. The framework establishes a unified foundation for generalized statistical complexity with potential applications in statistical mechanics, information theory, and the analysis of complex systems.
Read the full abstract

Information Theory

Task-Oriented Precoding for Edge Inference over Large-Scale MIMO Systems

Future wireless networks are expected to support networked artificial intelligence (AI) services, where multiple devices transmit learned features to an edge server for distributed inference. This setting calls for task-oriented physical-layer optimization, where wireless transmission should preserve useful information for inference rather than only maximize the rate or reconstruct the transmitted signals. A key physical-layer control variable is the multiple-input multiple-output precoder, which determines how device features are shaped and combined over wireless channels. Existing task-oriented precoding methods typically adapt the precoder to instantaneous channel state information at the transmitter (CSIT). However, in multi-device MIMO systems, acquiring the aggregate channel, feeding back CSI or optimized precoders, and reoptimizing across coherence blocks introduce substantial overhead. This paper develops a random-matrix-theoretic framework based on statistical CSIT that designs a slow-timescale precoder from channel covariance statistics and training-set feature statistics, without requiring instantaneous CSIT. We adopt maximal coding rate reduction (MCR${^2}$) to measure the class separability of the received features, yielding a task-aware utility for MIMO precoder design. Since this utility still depends on random small-scale fading, we derive a deterministic approximation that converts it into a fixed-point objective depending only on long-term statistics and large-system dimension ratios via random matrix theory. A projected block-coordinate ascent and successive convex approximation algorithm is developed to optimize this deterministic objective under per-device power constraints. Experiments on ModelNet10 verify the approximation and show that the proposed statistical precoder improves task-aware mode allocation and inference performance over competitive benchmarks.
Read the full abstract

All authors Hongru Li, Zeyan Zhuang, Zixin Wang, Hengtao He, Shenghui Song, Jun Zhang, Khaled B. Letaief

Information Theory

Autonomous Discovery of Wireless Communications Algorithms

Large language model (LLM)-driven evolutionary search is an emerging algorithm-discovery paradigm that has already produced novel results in several scientific fields. Yet its application to wireless communications remains largely unexplored. To bridge this gap, we introduce The AI Telco Engineer (AITE), a framework to autonomously design algorithms for complex communication problems, while navigating performance-complexity tradeoffs. We showcase AITE on two challenging physical-layer problems: designing an equalizer for an orthogonal time-frequency space (OTFS) system, and constructing a receiver algorithm for an orthogonal frequency-division multiplexing (OFDM) system using a custom constellation and operating without pilots. For the first task, AITE develops algorithms that outperform the best-known solutions while reducing computational latency by a factor of 3.6 compared to the strongest baseline. For the second task, it discovers the first explicit, explainable algorithms that achieve performance parity with state-of-the-art neural receivers. These results demonstrate the strong potential of LLM-driven evolutionary search for the autonomous discovery of next-generation wireless communications algorithms.
Read the full abstract

All authors Fayçal Aït Aoudia, Jakob Hoydis, Sebastian Cammerer, Gian Marti, Merlin Nimier-David, Nicolas Roussel, Alexander Keller

Information Theory

Skew CRT codes and their decoding in poly skew metric

In this paper, we introduce skew CRT codes and the poly-skew metric with associated error model, and we provide a decoding algorithm for this family of codes under the poly-skew metric.

Information Theory

Enhanced ODMA for Massive Sparse Access: Hybrid Data-Frozen Bit Transmission and Fixed-Point Analysis over Block Fading Channels

This paper proposes a novel on-off division multiple access (ODMA) transmission scheme that enables efficient joint multi-user channel estimation and iterative decoding by inserting a small number of frozen symbols into the codewords. Functionally analogous to pilots, these symbols are sparsely distributed within the codeword. Unlike conventional pilot-based methods, our approach requires only a minimal number of frozen symbols (e.g., $5\sim20$ symbols per user in a 300-user system), which serve dual purposes as both estimation references and decoding aids. By employing low-complexity single-user channel estimation and decoding, combined with simple iterative interference cancellation, the scheme achieves performance equivalent to that with perfectly known user channels, even when accounting for the additional energy and bandwidth costs of the frozen symbols. Furthermore, for the large-scale ODMA sparse multiple access system, this paper proposes a fixed-point analysis method, which can accurately estimate the iterative convergence performance over multi-user block fading channels by only leveraging the decoding functions under the single-user AWGN channel. This method is applicable to performance analysis for arbitrary code lengths, code rates, and decoding algorithms. It eliminates the need for extensive Monte Carlo simulation time, and provides an efficient tool for the design of multi-user codes.
Read the full abstract

All authors Guanghui Song, Jianxiang Yan, Ying Li, Zhaoji Zhang, Jun Cheng

Information Theory · Cross-listed from cs.LG

One-step lowest-variance selection in a Gaussian random-field model motivated by masked diffusion: Total correlation and a square root collision threshold

Motivated by confidence-guided parallel unmasking in masked discrete diffusion, we study a single selection step in a stylized Gaussian random-field model. A locally dependent nonnegative score field represents position wise uncertainty, and the scheduler selects the K positions with the smallest scores. Dependence among the selected positions is measured through a distance-dependent Gaussian correlation model. This separation provides a tractable framework for quantifying how the geometry of low-score locations affects the dependence cost of factorized parallel decoding. We establish two complementary results. In a conservative sub-square-root regime, the conditional Gaussian total correlation of the selected block vanishes in probability. At the square-root scale, it remains non-negligible with positive asymptotic probability and admits a strictly positive expectation lower bound. Synthetic experiments support the predicted finite-size behavior. These results provide a rigorous stochastic-geometry baseline for understanding how budget size, score dependence, and spatial correlation jointly shape one-step confidence-based selection in masked discrete diffusion.
Read the full abstract

Information Theory

Generalized BCH Codes and Twisted Goppa Codes Attaining Their Designed Distances

Determining the true minimum distance of an alternant code remains a notoriously difficult problem in coding theory. In this paper, we study the minimum distances of generalized BCH codes and twisted Goppa codes through their parity-check matrices. We first give a necessary and sufficient condition for an alternant code to attain its designed distance and apply it to generalized BCH codes. As applications, we prove that broad classes of generalized BCH codes have minimum distances equal to their designed distances. These classes provide explicit infinite families rather than isolated examples. We characterize when a twisted Goppa code $\Gamma(L,g,\eta)$ with $\operatorname{deg} g=t$ satisfies $d(\Gamma(L,g,\eta))=t+1$, and derive structured classes and infinite families attaining this distance.
Read the full abstract

All authors Yaqi Chen, Hao Chen, Cunsheng Ding, Huimin Lao, Chao Liu, Conghui Xie

Information Theory

Critical-Set-Aided Simplified Blind SCL Recognition of Polar Codes

Blind recognition of polar codes from noisy observations is a key problem in non-cooperative signal processing. Although existing blind successive cancellation list (BSCL) recognition exploits channel soft information, it performs two-hypothesis path expansion at every source-bit position, resulting in high complexity. In this paper, we first analyze the first recognition-error positions in the blind successive cancellation (BSC) recognition and observe that they are closely related to the corresponding contribution terms in the existing Bhattacharyya-parameter-based upper bounds. Based on this observation, a critical-set-aided simplified blind successive cancellation list (SBSCL) recognition method is proposed. SBSCL performs two-hypothesis path expansion only at the selected critical-set positions and keeps BSC recognition at the remaining positions, thereby reducing complexity. To improve the reliability of critical-set selection and refine the performance analysis, density-evolution (DE)-based bounds are further developed. Under the ideal SC-consistent condition, the synthetic log-likelihood-ratio (LLR) distributions obtained from density evolution are used to compute the optimized Chernoff coefficient for the upper bound and the overlap coefficient for the lower bound. Simulation results show that the DE-based bounds are tighter than the Bhattacharyya-parameter-based bounds. In the considered settings, the gap between the DE upper and lower bounds is within $1$ dB around a recognition-error probability of $10^{-2}$. Furthermore, SBSCL achieves nearly the same recognition success rate as BSCL, and the size of critical set decreases rapidly as the signal-to-noise ratio (SNR) increases.
Read the full abstract

All authors Changwei Tu, Cheng Yang, Xianzhao Feng, Kai Niu

Sunday 19 July

Information Theory · Cross-listed from cs.LG

Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones

Linear attention promises constant-time recurrent inference but degrades sharply on associative recall. We formulate attention recall as a spherical-packing problem and introduce Kernelized Linear Attention Activations (KATA), a framework whose feature maps are derived from first principles by certifying nonnegative attention weights through a self-dual homogeneous cone. Building on this observation, we show that rank-one positive semi-definite (PSD) features offer a favorable capacity–interference tradeoff. KATA recovers a parameter-free convex output gate and characterizes associative capacity through the Welch interference floor. For tolerances above this floor, KATA enlarges the state without adding parameters and admits spherical codes with exponentially many keys in the projection dimension. We implement KATA as fused Triton kernels at two operating points: a flash-attention-style forward up to ${\sim}1.6\times$ FlashAttention-2 throughput, and an exact $O(T)$ chunked-state form that reaches ${\sim}11\times$ FlashAttention-2 forward throughput at $131$k tokens. An associative scan of the first-order feature lowers the inter-chunk recurrence depth to $O(\log(T/C))$ for chunk size $C$ and averages ${\sim}2.4\times$ the throughput of a matched sequential linear-attention baseline. On long-range MQAR and repeated-key overwrite, several KATA variants outperform Gated DeltaNet, with parameter counts and state sizes reported alongside accuracy. Induction preserves near-perfect recall, while kernel benchmarks show that the maps can be implemented efficiently. KATA retains $0.985$ MQAR at a $16\times$ out-of-distribution length, approaching the softmax with roughly one quarter of the KV-cache entries. Experiments on 340M-parameter LLMs reveal a feature-dependent fluency trade-off and clarify how positional embeddings, delta rules, and decay gates interact with feature geometry.
Read the full abstract

Information Theory · Cross-listed from quant-ph

Constraints on recovering quantum information after erasure

Suppose that we encode quantum information into $ n $ physical carriers. With what probabilities can we perfectly recover the quantum information after sets of carriers have been erased? This question arises in the study of quantum erasure channels, for which M.B. Hastings conjectured that if each carrier is independently erased with probability $ t \geq 1/2 $, then it is impossible to recover the quantum information with asymptotically vanishing error and probability $ > 1-t $. We prove this conjecture by establishing new no-cloning constraints on realizable recovery probabilities.
Read the full abstract

All authors Mohammad A. Alhejji, Noah Lordi, Omkar Baraskar, Ariel Shlosberg, Emanuel Knill

Information Theory

Rate-Distortion-Perception Theory: Redefining the Fundamental Limits of Information Representation

Classical rate-distortion (RD) theory has long established the fundamental limits of lossy compression by quantifying the minimum number of bits required to represent a source under a prescribed distortion constraint. However, widely used distortion measures such as mean-squared error often fail to capture perceptual quality or semantic validity, which are increasingly central in modern learning-driven applications. Rate-distortion-perception (RDP) theory extends the RD framework by introducing perception as a third fundamental axis, quantified via distributional similarity between the source and reconstructed signals, leading to the rate-distortion-perception function (RDPF). This tutorial provides a structured overview of the coding principles underlying perception-aware lossy compression and surveys recent achievability results under different randomness assumptions. It then presents a unifying optimization viewpoint for computing the RDPF as defined by Blau and Michaeli, for both discrete and continuous sources under broad families of perceptual constraints, including f-divergences, alpha-divergences, and Wasserstein-based metrics. Special attention is given to computational tools such as alternating minimization schemes, Newton-based methods, and convex optimization formulations, as well as to analytically tractable cases such as Gaussian sources and the perfect-realism regime. Unlike recent broad surveys that emphasize generative architectures and AI-empowered communication systems, this tutorial focuses on the coding-theoretic and computational machinery needed to characterize, compute, and interpret the RDP limits. Finally, the tutorial outlines promising research directions at the intersection of information theory, neural compression, robust source coding, and perception-aware networked control systems.
Read the full abstract

Information Theory

Joint Channel Estimation and Dynamics-Aware Grouping for Time-Varying RIS-Assisted OTA Federated Learning

Reconfigurable intelligent surface (RIS)-assisted over-the-air federated learning (OTA-FL) enables efficient distributed intelligence but suffers from time-varying channels, imperfect channel state information (CSI), and strong user heterogeneity, which jointly degrade aggregation accuracy and cause severe model update cancellation. To address these issues, we propose a unified framework for joint channel estimation and dynamics-aware user grouping in RIS-assisted OTA-FL systems, enabling reliable learning under imperfect CSI and heterogeneous dynamics. The framework integrates gated recurrent unit (GRU) for temporal modeling to capture time-varying CSI evolution, OTA-based federated aggregation with personalization, and RIS-aware physical-layer optimization in a closed loop. In addition, we design a dynamics-aware grouping strategy based on long-term path-loss and short-term channel dynamics to reduce inter-user conflicts under heterogeneous conditions. Simulation results show that the proposed method achieves substantial gains in CSI estimation accuracy and OTA aggregation performance in low-pilot and high-mobility regimes, while improving convergence speed and robustness under strong user heterogeneity.
Read the full abstract

All authors Ziqi Li, Shuangzhi Li, Uchechukwu Awada, Jiankang Zhang

Information Theory

Dynamic Channel Knowledge Map: Fundamentals, Construction, and Applications

Wireless communication networks are evolving toward extremely large antenna arrays, millimeter-wave and terahertz bands, and dense heterogeneous deployments, all of which increase channel dimensionality and make channel acquisition increasingly costly. Channel knowledge map (CKM) establishes a mapping from geographical locations to channel characteristics, providing location-specific prior information to reduce the overhead of channel acquisition. Most existing CKM research, however, has focused on quasi-static propagation features shaped by quasi-static environmental structures such as buildings and terrain, leaving unaddressed the time-varying channel component introduced by dynamic scatterers, terminal attitude changes, and radio-frequency (RF) impairments. This article presents a new concept of dynamic CKM as a middle layer that links quasi-static environmental priors to physical-layer signal processing by providing time-evolving channel representations. We first introduce the fundamentals of dynamic CKM, clarifying its relationship with the quasi-static CKM and the physical layer. We then survey representative construction methods and discuss how dynamic CKM can support pilot design, interference suppression, and integrated sensing and communications. Finally, we outline key open research directions in the co-design of dynamic CKM construction and physical-layer signal processing. These discussions offer an architectural perspective on the role of dynamic CKM in emerging 6G systems.
Read the full abstract

Information Theory · Cross-listed from math.ST

Fundamental Limits of Query-Based Subgraph Detection

The planted subgraph detection problem asks whether a random graph contains a hidden structured subgraph. In the classical formulation, the entire adjacency matrix is observed and one distinguishes between an Erdős–Rényi random graph and one obtained by planting a copy of a prescribed graph inside an Erdős–Rényi random graph. The statistical and computational limits of this problem under full observation are now well understood, even for arbitrary planted subgraphs. In this paper, we investigate an information-limited version of the problem in which the planted structure is an arbitrary sequence of graphs $\Gamma=(\Gamma_n)_{n\geq1}$, where $\Gamma_n$ is embedded in an ambient graph on $n$ vertices, but the observer does not have access to the full adjacency matrix. Instead, information is acquired through a limited number of non-adaptive edge queries. We study the minimum query complexity required for reliable detection. We derive general information-theoretic lower bounds and complementary algorithmic upper bounds on the query complexity as functions of the query budget and structural properties of the planted graph. The proposed algorithms exploit three distinct structural mechanisms: dense local motifs, high-degree vertices, and global edge density. We establish matching bounds, up to polylogarithmic factors, for several broad families of planted graphs, including clique-like, bounded-cover, and hub-dominated graph classes. Our framework substantially generalizes existing query-complexity results for planted clique and planted dense subgraph models, providing a unified treatment of arbitrary planted subgraphs under restricted graph access.
Read the full abstract

Information Theory

Cofilling Shattering: A Syndrome-Support Hierarchy for Check Erasures

Let $A:\mathbb{F}_2^n\to\mathbb{F}_2^m$ be a binary linear map with fixed coordinate bases, let $C_A=\ker A$, and let $\lambda_A(y)$ be the minimum Hamming weight of a preimage of the syndrome $y$. We define $\operatorname{Shat}_{q,s}(A)$ as the least common check support of a $q$-dimensional syndrome subspace whose every nonzero element has coset-leader weight at least $s$. It therefore distinguishes release of $q$ independent syndromes from release of a subspace with no easy linear combination. Deleting check coordinates $F$ releases $\ker A_{\bar{F}}/\ker A$, canonically isomorphic to $(\operatorname{im} A)[F]$. Finiteness implies $R_q(C_A)\ge \mathsf{N}_2(q,s)$, where $\mathsf{N}_2(q,s)$ is the shortest length of a binary code of dimension $q$ and distance at least $s$; profile-Griesmer bounds independently control common check support. The hierarchy is coordinate-relabeling invariant but can change under a change of check basis. For the pair-repetition code $C_n=\{(x,x):x\in\mathbb{F}_2^n\}$, the standard realization $H_0=[I_n\ I_n]$ has $\operatorname{Shat}_{q,s}(H_0)=\mathsf{N}_2(q,s)$ whenever feasible. For every $q\ge 1$ and $s\ge 2$, with $n=\mathsf{N}_2(q,s)$, a row-equivalent realization of the same code has value $q$. For a simplicial coboundary map $A=\delta_k$, check erasure is top-face erasure and the released quotient is emergent cohomology. At $s=1$ the hierarchy reduces to generalized Hamming weights and is Tutte-determined; for $s\ge 2$, even identical labeled cut codes can have different values.
Read the full abstract

Saturday 18 July

Information Theory · Cross-listed from cs.RO

Information-Based Trajectory Planning for Spacecraft-to-Spacecraft Tracking and Navigation in Cislunar Space

We present a trajectory planning method that balances information collection and control effort to improve cislunar spacecraft-to-spacecraft absolute tracking. Expanding use of cislunar space requires alternative navigation and tracking procedures that minimize reliance on ground-based support. Among efforts to address this need, spacecraft-to-spacecraft tracking exploits nonlinear dynamical tracers encoded in a series of relative measurements to infer absolute states of both an observer and a target. The geometry between spacecraft operating under this mode can significantly influence tracking performance. This work considers this geometrical impact by designing observer trajectories that jointly balance control effort and an information-theoretic quantification of the expected spacecraft-to-spacecraft tracking performance. Our methods are designed for multiple low-thrust observation platforms of various sensing modalities and can incorporate multiple space object targets in planning. By leveraging information gain in the optimal control problem, we report almost an order of magnitude improvement in the expected navigation and tracking errors for an optical observer operating in a Distant Retrograde Orbit (DRO). This work demonstrates the feasibility and value of information-optimal low-thrust spacecraft trajectory design for current and upcoming cislunar missions.
Read the full abstract

Information Theory

Transmit Coefficients and Receive Combining Vector Design for OTA-FL with Imperfect CSI

Over-the-air (OTA) computation has recently gained significant attentions as an effective approach to enhance the communication efficiency of wireless federated learning (FL). By enabling simultaneous transmission and aggregation of local model updates, OTA-FL can substantially reduce both latency and bandwidth consumption. However, a key challenge lies in the imperfect aggregation of global models caused by channel state information (CSI) uncertainty, which introduces distortion to the final learning performance. To address this issue, we study the long-term mean squared error (MSE) minimization problem for OTA-FL under imperfect CSI conditions. Through convergence analysis, we establish an upper bound for the time-averaged MSE, thereby revealing the effect of aggregation errors accumulated throughout multiple communication rounds on the overall training performances. Based on this analysis, an optimization framework is developed to minimize the long-term MSE via the joint design of (i) transmit coefficients at the local devices and (ii) receive combining vectors at the parameter server (PS). Since this alternating optimization approach requires non-causal CSI, a Lyapunov-based optimization method is further introduced to handle causal CSI scenarios. By incorporating virtual queues to characterize long-term energy consumption, the proposed method effectively decouples temporal dependencies and allows transmit coefficients to be optimized based on the causal CSI of each aggregation round. Comprehensive evaluations on Fashion-MNIST, CIFAR-10 and CIFAR-100 datasets have demonstrated that the proposed algorithms can significantly reduce the degradation of test accuracy caused by imperfect CSI. Comparisons with other benchmark schemes further verify the superiority of our proposed algorithms.
Read the full abstract

All authors Xiaoyan Ma, Shahryar Zehtabi, Yinan Zou, Taejoon Kim, Christopher G. Brinton

Information Theory · Cross-listed from math.OC

Relative Entropy-Bounded Ambiguous Chance Constraints for Robust Planning in Nonlinear Systems

We consider defining risk probability in stochastic control problems under distribution ambiguity. Current approaches for chance-constrained control typically assume that the true state distribution is known and Gaussian distributed. These assumptions are not amenable to many real-world engineering applications where system dynamics are nonlinear and only approximately modeled. In this work, we define a distribution ambiguity set and, with a variational expression for exponential integrals, bound the expected risk value under an unknown distribution that resides within a relative entropy distance of a nominal Gaussian reference distribution. Our bound recovers the reference risk value in the zero-divergence limit. A method is presented to determine the relative entropy distance defining the ambiguity set that is a function of the reference covariance evolution and second-order dynamical truncation errors. The resulting contributions provide a framework for handling distributional ambiguity in nonlinear covariance steering problems. A stochastic spacecraft guidance example is presented to demonstrate our contributions.
Read the full abstract

Information Theory

Tight Sample Bounds for Renyi and Min-Entropy Estimation

Estimating entropy from samples is fundamental in information theory and property testing. Shannon entropy measures average uncertainty and can be estimated to constant additive accuracy over a $k$-symbol alphabet using $\Theta(k/\log k)$ samples. Min-entropy depends only on the most likely symbol. Both are special cases of order-$\alpha$ R'{e}nyi entropy, $H_\alpha$. We characterize the sample complexity of estimating min-entropy and R'{e}nyi entropy for $k$ and integer $\alpha>1$; our lower bounds also hold for noninteger $\alpha\ge1.001$. We prove that min-entropy estimation to constant additive accuracy has sample complexity $\Theta(k\log k)$. The upper bound uses the largest empirical frequency and concentration via dyadic grouping. The matching lower bound hides a slightly heavier symbol at a uniformly random location. Thus, min-entropy requires $\Theta(\log^2 k)$ more samples than Shannon entropy and corrects a previously stated $\Theta(k/\log k)$ characterization. For every integer $2\le\alpha\le c_0\log k$, we prove the matching fixed-accuracy bound $\Theta_{c_0}(\alpha k^{1-1/\alpha})$. Previous results gave $\Omega_\alpha(k^{1-1/\alpha})$ for fixed integer $\alpha>1$ and $O_{c_0}(\alpha^2k^{1-1/\alpha})$ for all integer $\alpha>1$. Our upper bound analyzes an unbiased falling-factorial estimator based on $\alpha$-way collisions, while a hidden-heavy-coordinate construction gives the matching lower bound and shows that the factor $\alpha$ is unavoidable. For every real $1.001\le\alpha\le c_0\log k$, we prove the uniform lower bound $\Omega_{c_0}(\alpha k^{1-1/\alpha})$. Finally, since $0\le H_\alpha(p)-H_\infty(p)\le\log k/(\alpha-1)$, min-entropy uniformly approximates $H_\alpha$ when $\alpha$ is a sufficiently large multiple of $\log k$. Combining this reduction with our min-entropy bounds gives $\Theta_\varepsilon(k\log k)$ sample complexity in the high-order regime.
Read the full abstract

Information Theory · Cross-listed from math.AG

Explicit LCP of MDS Codes and LCD Codes on Hyperelliptic Curves via Mumford Representation

We study algebraic geometry codes on hyperelliptic curves of genus $g \geq 2$ equipped with a rational Weierstrass point, with a focus on complementarity properties. Our first contribution is a characterization of non-special divisors of degree $g$ and $g-1$ via the polynomial degrees of their reduced Mumford representation, reducing a classical geometric problem to a degree test on univariate polynomials. Using this, we construct Linear Complementary Pairs (LCP) of codes via polynomial arithmetic on the Jacobian and provide a criterion in terms of Mumford degrees for the resulting codes to be Maximum Distance Separable (MDS). Under a $2$-torsion condition in the Jacobian, equivalently a divisibility condition on the Mumford polynomials, we obtain explicit multipliers that turn these pairs into Linear Complementary Dual (LCD) codes. We give an MDS LCP with parameters $[8,4,5]_{16}$ in characteristic two, an MDS LCD code with parameters $[10,5,6]_{25}$ on the maximal hyperelliptic curve $y^2=x^5+x$, and an LCD code with parameters $[28,14]_{49}$ on $y^2=x^7+x$. Motivated by the explicit $q=5$ computation, we conjecture for odd prime powers $q\ge5$ the existence of MDS LCD codes with parameters $[2q,q,q+1]_{q^2}$ arising from this construction.
Read the full abstract

Information Theory · Cross-listed from cs.RO

Approximate Relative Entropy Constraints for Nonlinear Covariance Steering Under Distribution Ambiguity

Covariance steering provides an efficient framework for designing linear stochastic feedback policies, but its extension to nonlinear systems relies on a Gaussian surrogate obtained through local linearization. Because this surrogate may differ substantially from the true nonlinear state distribution, risk-sensitive quantities such as collision probability and mean-squared error may be inaccurately estimated. This work develops a distributionally robust covariance-steering framework based on the relative entropy, also known as the Kullback-Leibler divergence (KLD), to account for ambiguity in the propagated probability density function. Using a variational representation of exponential integrals, we derive computable upper bounds on risk-sensitive quantities over a KLD ambiguity set. We then formulate an upper bound on the time rate of change of the KLD between the true nonlinear distribution and a Gaussian reference surrogate. Under some assumptions, this bound is controlled by decision variables within a covariance-steering formulation. The resulting constraints are incorporated into a sequential convex programming algorithm to design stochastic guidance policies that keep the true distribution close to its Gaussian surrogate while enforcing bounds on risk-sensitive performance measures. The proposed approach is demonstrated on a challenging nonlinear spacecraft transfer between two near-rectilinear halo orbits.
Read the full abstract

Information Theory

Decoding Desarguesian spread codes beyond half minimum distance

Spread codes are a well-known family of constant-dimension subspace-metric codes. For constant dimension $k$ and ambient space dimension $n$ being a multiple of $k$, these codes have minimum distance $2k$ and a rich geometric structure. In this paper, we study the decoding capabilities of the Nearest Neighbor Decoder for Desarguesian spread codes, establishing that unique decoding is still achievable beyond half the minimum distance. Motivated by this, we develop a new decoding algorithm to uniquely decode Desarguesian spread codes in the presence of both insertions and deletions, which increase and decrease, respectively, the dimension of the transmitted codeword. Even when the sum of the dimensions of insertions and deletions exceeds half the minimum distance, provided that deletions are of dimension at most $k-2$, the algorithm succeeds with a small decoding failure. We also propose two refinements to this algorithm that, empirically, can handle nearly as many insertions as the Nearest Neighbor Decoder.
Read the full abstract

Information Theory

Sustainable Air-Ground Integrated Coverage Networks: ISCC Architecture, Technologies, and Testbed

The rapid emergence of sixth-generation (6G) networks and the low-altitude economy has accelerated the evolution of wireless infrastructures toward air-ground integrated coverage networks (AGICNs), which seamlessly fuse terrestrial and aerial communication resources. However, existing AGICN studies primarily focus on coverage enhancement, while ignoring sustainability. Pursuing sustainable AGICNs introduces new challenges due to the multidimensional resource coupling across heterogeneous air-ground segments. In view of this, this paper presents a comprehensive survey and tutorial on sustainable AGICNs, aiming to balance coverage capacity with carbon efficiency in low-altitude economies. An integrated sensing, communication, and computation (ISCC)-driven architecture, which enables dynamic resource orchestration through closed-loop control, is proposed. We thus introduce a multi-dimensional sustainability metric system, which covers operational efficiency, task-oriented performance, and full lifecycle carbon emissions, to quantify energy and carbon footprints. We review enabling technologies, including artificial intelligence, hybrid precoding, integrated sensing and communication, and simultaneous wireless information and power transfer, and discuss their integration into the ISCC framework to minimize energy consumption while maintaining robust coverage. Experimental results on a real-world testbed demonstrate a 20% reduction in power consumption while achieving over 90% coverage probability, highlighting the feasibility of sustainable AGICNs for future green networks.
Read the full abstract

All authors J. Liu, X. Zhang, M. Sheng, R. Zhang, N. Zhao, J. Wang, J. Li

Information Theory

Revisiting the Stability of the Ingleton Inequality: A Tropicalization-Free Approach

The classical Ingleton inequality is known to hold for entropic points under specific exact conditional independence constraints. Recently, Matveev and Romashchenko (2026) investigated the stability of these implications, quantifying the extent to which the Ingleton inequality can be violated when a group of conditional mutual information terms is small but non-zero. While their proofs relied fundamentally on the complex framework of tropical probability spaces, we revisit these stability results using a completely tropicalization-free approach. By developing an alternative framework, we significantly streamline the underlying concepts and proofs, derive explicit error terms, and improve some estimates. Furthermore, we resolve an open problem posed in prior work by exhibiting a new infinite family of entropy inequalities that establishes the stability of the sum of two Ingleton expressions.
Read the full abstract

Information Theory

Leveraging von Mises Message-Passing for Massive MIMO Detection

Detection in massive multiple input multiple output (mMIMO) systems suffers from exponential complexity while using optimal decoders like maximum a posteriori (MAP). We propose a belief propagation-based detector based on directional statistics, applicable to mMIMO systems relying on PSK modulations. Thanks to a continuous relaxation of the PSK modulation to the unit circle, and to the use of von Mises parametric representations of the messages to obtain sparse representations of the (generally infinite dimensional) messages, the proposed method allows for approximate detection with a low complexity which does not depend on the PSK modulation order. Extensions of the algorithm to imperfect channel realizations are also present. We quantify the performance and complexity of the proposed approach and compare it with detection algorithms based on Gaussian approximation.
Read the full abstract

All authors Sweta Suresh, Arthur Michon, Charly Poulliat, Maxime Guillaud, Claire Goursaud

Information Theory · Cross-listed from cs.CR

Synchronization-Free Algebraic Fingerprints for Large Language Models: From Autoregressive to Diffusion Models

Large Language Models (LLMs) have created an urgent need for reliable watermarking methods that enable attribution of generated text while remaining robust to editing and paraphrasing. We propose a novel synchronization-free watermarking scheme in which every watermark consists of a single binary congruence generated from a pair of neighbouring tokens. For each token pair, a cryptographic hash determines an evaluation point of a Reed–Solomon polynomial representing the secret identity, while the parity of the polynomial evaluation determines the watermark bit embedded into the second token of the pair. Since each congruence is self-contained and depends only on the local token pair, the proposed construction is naturally resistant to insertions, deletions, and token reordering. We analyse the recovery problem from an algebraic perspective, discuss several decoding algorithms suitable for different identity sizes, and model watermark corruption as a Binary Symmetric Channel. The analysis shows that reliable recovery requires only a small redundancy even for relatively high token corruption rates. Unlike existing block-based watermarking schemes, the proposed method avoids synchronization problems while providing a flexible framework for embedding both short and long secret identities.
Read the full abstract

Information Theory

Variable Rate Lossy Source-Channel Coding over Channels with Feedback

A variable-rate lossy joint source-channel coding scheme for burst-noise communication channels with noiseless feedback is introduced. The scheme comprises a multi-stage channel optimized vector quantization system that dynamically allocates bits via a greedy algorithm among the variable-rate residual quantizers at each stage, based on the channel output sequence received at the encoder through the feedback link, thereby generalizing a prior fixed-rate scheme. Simulations over an $M$-th order Markov noise Polya contagion channel demonstrate that the proposed variable-rate scheme consistently outperforms the fixed-rate scheme, regardless of the channel bit error rate and noise correlation, achieving signal-to-noise ratio gains of up to about 4.5 dB.
Read the full abstract

Friday 17 July

Information Theory · Cross-listed from cs.LG

Capacity and Redundancy Trade-offs in Multi-Task Learning

In multi-task learning (MTL) negative transfer is often considered as an optimization artifact, but it can also be viewed as a consequence of limited shared capacity and weak task redundancy. We investigate this effect through a Capacity–Redundancy (CR) identity that decomposes the sum of per-task predictive informations into joint predictive information that includes label redundancy defined via total correlation (TC), and a residual coupling term that quantifies interference left unresolved by the shared representation. Additionally, we show two key results: (i) a clustering-gap decomposition that gives a necessary and sufficient condition for clustered sharing to outperform global sharing, and (ii) a gradient–TC bridge in a Gaussian multi-task model that formally justifies gradient cosine similarity as a proxy for redundancy ordering. Empirically, we estimate the residual coupling $\Delta$ from validation residual correlations, showing that clustered LoRA substantially reduces $\widehat{\Delta}$, outperforms size-matched random partitions, and results in statistically significant gains with multi-seed confidence intervals.
Read the full abstract

Information Theory · Cross-listed from quant-ph

Efficient Exact Quantum Sampling from the Sun-Wootters Distribution for Optimal Polynomial Intersection

Optimal Polynomial Intersection (OPI) is a structured optimization problem for which Decoded Quantum Interferometry (DQI) attains a satisfaction guarantee governed by the semicircle law. Sun and Wootters recently showed that, for balanced OPI over prime fields, a Fourier-defined distribution $P_u$ gives a strict worst-case improvement from limiting rate $0.6225$ onward and asymptotically perfect solutions from rate $0.7496$ onward, and asked whether $P_u$ can be sampled efficiently. We answer this question for Reed–Solomon OPI parameters satisfying their exponent condition strictly below the dual Johnson radius. Under coherent membership-oracle access, we give a bounded-error polynomial-time quantum sampler for $P_u$. The ideal circuit samples $P_u$ exactly conditioned on success, while a finite-precision implementation achieves any prescribed inverse-polynomial total-variation error. Consequently, every fixed limiting rate $0.6225\le r<1$ admits a strict worst-case improvement over the DQI semicircle value, and every limiting rate $r\ge 3/4$ admits solutions of satisfaction $1-o(1)$ with high probability. The algorithm coherently sums the amplitudes of all low-weight errors in each syndrome class using deterministic complete list decoding. Complete Reed–Solomon list decoding and the Sun–Wootters denominator estimate make the list size and postselection overhead polynomial. In concurrent and independent work, Horinaga and Yamakawa obtain worst-case OPI algorithms over prime-power fields and exact satisfaction at every fixed rate strictly above $3/4$.
Read the full abstract