Information Theory
Estimation with Quantized Parameter Side-Information
Mathis Wetterwald, Ofer Shayevitz and Michèle Wigger
This paper presents upper and lower bounds on the minimax risk of a parameter estimation problem where the
estimator not only observes independent and identically distributed observations but also a quantized version of the parameter, where the quantization has a limitation on the number of levels, but can be optimized otherwise. Our upper and lower bounds have similar decay rates for large number of observation samples and quantization levels.
Read the full abstract ▾
Information Theory · Cross-listed from math.CO
Covering radius of rank-metric codes via covering lifts and clubs
Giuseppe Marino, Alessandro Neri and Alessia Vallefuoco
We introduce a generator-
matrix-based
geometric approach to the covering radius of $\mathbb F_{q^m}$-linear rank-metric codes. Starting from the $q$-system associated with the code, we define the notion of covering lift and show that the covering radius can be recovered from the hyperplane weights of such lifts. This yields an exact criterion, in terms of the length and effective length of the code, for the covering radius to attain its largest possible value, together with the upper bound $\rho(\mathcal{C})\le \min\{m,n\}-1$ in all remaining cases. We then specialize our approach to $1$-dimensional codes, for which the effective length coincides with the minimum rank distance. In this setting, codes attaining $\rho(\mathcal{C})= \min\{m,n\}-1$ are related to covering lifts that are clubs. More generally, we characterize this case through linear maps into suitable quotient spaces, obtaining equivalent interpretations in terms of generalised evasive subspaces and auxiliary
matrix rank-metric codes. These results determine the covering radius for several boundary values of the minimum distance and for all $1$-dimensional codes with extension degree $m\in\{3,4,5\}$. For $m=6$ and every $q$ we provide constructions and, for $q\in\{2,3\}$, computational results showing that $1$-dimensional rank-metric codes with the same length and minimum distance can have different covering radii.
Read the full abstract ▾
Information Theory
Breaking linearity in the sum-rank metric: MSRD codes from switched $\sigma$-rational normal curves
Daniele Bartoli, Giovanni Giuseppe Grimaldi and Giovanni Longobardi
We construct a family of scalar-closed, non-additive maximum sum-rank distance codes (MSRD) by switching norm components of a $\sigma$-rational normal curve and lifting the resulting point set through a linearized Reed–Solomon syndrome map. We characterize the admissible switches by a multiplicative stability condition on the norm classes. In particular, for $1 \leq n_i \leq m$, $N=n_1+\ldots+n_{\ell}$ and $2\leq\delta\leq N-1$, every partition of $\mathbb{F}_q^*$ satisfying a specific condition, referred to as Condition $(\diamond)$, yields a code in $\mathbb{F}_{q^m}^{N}$ of size $q^{m(N-\delta+1)}$ and minimum sum-rank distance $\delta$; the code is non-additive whenever a curve component is retained. For $\ell\geq2$ these appear to be the first non-additive MSRD codes in the literature, all previously known families with more than one block being $\mathbb{F}_q$-linear. Moreover, every code of the family has the same sum-rank weight distribution as an $\mathbb{F}_{q^m}$-linear MSRD code with the same parameters, although the geometry of the switched set distinguishes it from linearized Reed–Solomon codes. In the single-block case, an explicit rank isometry identifies the construction with the cone codes of Durante, Grimaldi and Longobardi.
Read the full abstract ▾
Information Theory · Cross-listed from cs.LG
Recovery Guarantees for Posterior Sampling of One-Bit Compressed Sensing
Jing Ma, Yujia Wu and Zhaoqiang Liu
We study the sample complexity of noisy one-bit compressed sensing for signals drawn from a prior distribution. By characterizing the effective distributional complexity of the prior via its approximate covering number, we prove that posterior sampling achieves accurate recovery with high probability when the number of measurements scales with the logarithm of the approximate covering number, up to a one-bit separation gap factor. This upper bound is robust to learned prior mismatch. Specifically, we show that posterior sampling with an approximate prior remains reliable, provided that the learned prior distribution is sufficiently close to the true signal distribution in Wasserstein distance. In addition, we establish a sample complexity lower bound for any reliable method of noisy one-bit compressed sensing, showing that our upper bound is nearly matched in its main prior dependent term. To approximate the ideal posterior sampling process for real world scenarios, we instantiate posterior sampling through a plug-and-play
algorithm with
diffusion priors. Experiments on the FFHQ and ImageNet
datasets demonstrate the effectiveness of our proposed approach.
Read the full abstract ▾
Information Theory
Distributed Constrained Resource Management in 6G Networks: A Scalable Hybrid Model-Learning Framework
Thang X. Vu, Cuong Le, Symeon Chatzinotas et al. (+1 more)
Radio Resource Management (RRM) is a fundamental challenge in 6G wireless networks, particularly under dynamic user demands, inter-
cell interference, and heterogeneous QoS constraints. Centralized optimization solutions are often infeasible in practice due to the lack of system-wide state information, dynamic conditions, and signaling delays, making distributed learning-based approaches attractive. However, conventional multi-
agent reinforcement learning (MARL) struggles with scalability and constraint satisfaction in such highly dynamic environments. We introduce a scalable two-phase hybrid learning framework to address the RRM challenges where orthogonal-frequency division multiplexing (OFDM) domain knowledge is explicitly incorporated into the MARL pipeline. In our proposed two-phase learning framework, the first phase allocates a minimum resource to satisfy users' QoS requirements based on channel statistics, avoiding the inefficiencies of training policies under hard QoS constraints. Subsequently, a multiagent system is employed in the second phase to optimally allocate the remaining resources for system throughput maximization. By exploiting the structural interference information, we propose a scalable MARL
algorithm which decomposes the original learning problem into smaller subproblems that can be handled independently, thereby avoiding exponential growth of the action space without compromising performance. Extensive simulations in realistic scenarios with 50MHz bandwidth and different numerologies show that our method significantly outperforms existing optimization and learning baselines, offering up to 40% throughput improvement, and 100% constraint satisfaction with minimal resource usage.
Read the full abstract ▾
All authors Thang X. Vu, Cuong Le, Symeon Chatzinotas, Bjorn Ottersten
Information Theory · Cross-listed from eess.SP
Outage-Aware Robust Sensing for UAV ISAC Systems
Qiming Li, Lei Zhang, Haoran Xu et al. (+1 more)
Integrated sensing and communications (
ISAC) infrastructure can provide an external perception layer for aerial
robots, but dynamic blockage makes the set of informative sensing links at the next update uncertain. A single predicted availability pattern can miss low-information outcomes, whereas protecting all patterns wastes power and may become infeasible when all links are blocked. We propose an outage-aware conformal robust power-allocation framework for multi-BS
UAV tracking. A history-aware joint predictor maps radar and tracking histories to next-slot availability probabilities, and adaptive prediction sets retain plausible correlated patterns. Tracking constraints are enforced only for retained non-outage patterns, while the all-blocked pattern is treated separately as a physical sensing outage. With per-slot beam directions fixed, the resulting scalar-power allocation is a semidefinite program. A reliability bound relates serviceable-slot tracking failure to deployed-policy miscoverage and optimization infeasibility. Across paired closed-loop simulations over five random seeds, the method attains a 0.01% serviceable-slot constraint-violation rate and uses 47.0% less power than protection over all non-outage patterns.
Read the full abstract ▾
All authors Qiming Li, Lei Zhang, Haoran Xu, Lina Mohjazi
Information Theory
From an entropic inequality for sums in abelian groups to the theorems of Ruzsa and Plünnecke
Chandra Nair, Aref Namayandeh and Zhaobang Zhu
We give a direct information-theoretic proof of the Plünnecke–Ruzsa inequalities for finite subsets of abelian groups. The argument rests on an elementary coupling inequality proved by maximizing joint
entropy, without tensor powers or asymptotic constructions. It also yields Ruzsa's sumset triangle inequality and an extension to non-abelian groups.
Read the full abstract ▾
Information Theory
Three Improved Lower Bounds for Binary Error-Correcting Codes
Kazuki Ota, Takayuki Osa and Tatsuya Harada
Error-correcting codes are a foundational technology for representing information robustly against noise in communication and data storage. The binary code packing problem, which seeks to maximize the number of codewords at a fixed code length and minimum distance, is a fundamental problem in coding theory that remains open for many parameter choices. We give explicit binary codes with parameters $(n,M,d)=(22,84,9)$, $(24,196,9)$, and $(25,65,11)$, where $n$, $M$, and $d$ denote the code length, number of codewords, and minimum distance, respectively. These establish the lower bounds $A_2(22,9)\ge84$, $A_2(24,9)\ge196$, and $A_2(25,11)\ge65$, where $A_2(n,d)$ denotes the maximum possible number of codewords, improving the respective lower bounds 80, 192, and 64 in the public table of general binary codes. Most notably, the third result improves the lower bound supplied by a lexicographic construction for the first time in 66 years, while the first two improve bounds that have stood for 21 years. The new codes represent more messages without changing the code length or error-correction guarantees, with gains compounding multiplicatively across blocks. We release all codewords and an exhaustive verifier to enable independent verification.
Read the full abstract ▾
Information Theory
DRL-Based AoI Minimization for RSMA in Finite-Blocklength MU-MISO
Enes Kaya, Mehmet Alp Demircioğlu, Elif Tugce Ceran et al. (+1 more)
This letter investigates age-of-information (AoI) minimization in multi-user wireless networks operating in the finite-blocklength (FBL) regime, which is critical for low-latency transmission of short state-update packets. While rate-splitting multiple access (RSMA) provides a powerful and flexible framework for interference management in multi-user FBL systems, the joint optimization of its parameters, such as precoding vectors, power allocation, and rate-splitting ratios, to guarantee information freshness results in analytically intractable complexity. To address this challenge, we propose an actor–critic deep reinforcement learning (DRL) framework to learn dynamic resource-allocation policies in multi-user multiple-input single-output (MU-MISO) broadcast channels. Simulation results show that the proposed RSMA-RL framework achieves consistently lower AoI than the state-of-the-art
benchmarks, with substantial gains observed at low signal-to-noise ratio (SNR) and short blocklengths, while matching
benchmark performance at high SNR with significantly lower online complexity via a single neural-network forward pass at execution.
Read the full abstract ▾
All authors Enes Kaya, Mehmet Alp Demircioğlu, Elif Tugce Ceran, Melda Yuksel
Information Theory · Cross-listed from quant-ph
Minimal Scenarios Witnessing Genuine Multipartite Nonlocality Possibilistically
Soumyadip Patra and Peter Bierhorst
We study a strong form of genuine multipartite nonlocality (GMNL), where an $n$-partite scenario admits a
quantum strategy generating outcomes belonging to a restricted set with certainty, whereas every strategy built from nonclassical resources shared among at most $k<n$ parties fails this criterion with positive probability. This generalizes what is referred to as strong contextuality/nonlocality, or pseudotelepathy, beyond ruling out local models. For the simplest five-party Bell scenario of two binary-outcome measurement settings per party, we use the inflation technique to demonstrate that a game with a perfect
quantum strategy witnesses a robust separation from models built from bipartite resources under local operations and shared randomness (LOSR), establishing the behavior is LOSR-GMNL. We provide broader context for this result by investigating its simulability with resources shared among three or four parties under different resource models and alternative definitions of GMNL. A central open question is whether $n$-party
quantum resources can achieve such a separation from models built from $(n-1)$-party resources, for $n\ge 3$. We make progress here by constructing a family of robustly LOSR-GMNL games with $\epsilon$-optimal
quantum strategies experimentally indistinguishable in the limit from true probability-one-winning strategies for any desired level of precision.
Read the full abstract ▾
Information Theory · Cross-listed from cs.DS
Tight Bounds for Equivalence Testing with Non-Adaptive Conditional Samples
Gautam Kamath
We study distribution testing with access to non-adaptive conditional samples. Specifically, we give tight bounds for equivalence testing, determining whether two unknown distributions are equal to or $\varepsilon$-far from each other in total variation distance. Our
algorithm and lower bound show that $\tilde \Theta\left(\frac{\log n}{\varepsilon^2}\right)$ queries are necessary and sufficient for this problem. These results demonstrate that the complexity of uniformity, identity, and equivalence testing with non-adaptive conditional samples are all $\tilde \Theta(\log n)$.
Read the full abstract ▾
Information Theory · Cross-listed from q-bio.NC
If the brain were so simple: Information-based drivers of intelligence and their interaction through an n-body-inspired framework
Brett J. Kagan, Johnson Zhou, Forough Habibollahi et al. (+1 more)
Theories of how the
brain processes information and returns intelligent outputs are numerous and often difficult to conclusively test. Here, we consider an approach grounded in informatics and algorithmic thermodynamics for how neural systems respond to different patterns of information in different ways, with a focus on the relationship between the information
entropy of a signal and a new proposed quantity we term information enthalpy, which represents the internal resource of structured information within a system available to be used for predictions and other forms of work. To evaluate the capacity of the external signals to increase information enthalpy in the system, we introduce the information enthalpy potential (IEP) as a defined metric. We propose and implement a method for quantifying the amount of potential information enthalpy - the IEP - in a given signal across a range of example signals. We offer a
conjecture of how information enthalpy may be treated by neural systems within a biological framework before showing how it may integrate and offer falsifiability to existing approaches such as the Free Energy Principle. This framework is also positioned in light of the role of neural criticality. Finally, we postulate a testable and falsifiable framework where the distinct mechanisms within each driver interact through a described n-body-inspired model to govern the "motion" of a neural system through a high dimensional state-space. Through these processes, it is proposed that these features form the most fundamental basis of the neural drivers that give rise to the complexity of adaptive behaviors that are commonly called: intelligence.
Read the full abstract ▾
All authors Brett J. Kagan, Johnson Zhou, Forough Habibollahi, Valentina Baccetti
Information Theory · Cross-listed from stat.ML
A General $\widetilde{\Omega}(\sqrt{T \gamma_T})$ Lower Bound for Kernel Bandits
Chenkai Ma and Jonathan Scarlett
The kernel bandit problem consists of sequentially optimizing an unknown function with noisy feedback, where the function has bounded norm in a given Reproducing Kernel
Hilbert Space (RKHS). A central quantity in the regret analysis of kernel bandits is the maximum information gain $\gamma_T$. In particular, the best existing upper bounds scale as $\sqrt{T\gamma_T}$ up to log factors, and nearly-matching lower bounds have been derived for specific kernels such as squared exponential and Matérn. However, lower bounds for general kernels are lacking, thus making it unclear in what generality the upper bounds are near-optimal. In this paper, we establish a general $\Omega(\sqrt{T\gamma_T/\log T})$ minimax regret lower bound for non-constant continuous kernels on compact domains, establishing near-optimality (within log factors) in a very general sense. We show that the log factor appearing in this bound is unavoidable in general, but that it can be removed under certain conditions. Among other things, our findings imply that the minimax-optimal scaling is exactly $\Theta(\sqrt{T\gamma_T})$ (i.e., within constant factors) for the Matérn-$\nu$ kernel with $\nu \in (0,2)$, $\gamma$-exponential kernel with $\gamma \in (0,2)$, and certain piecewise-
polynomial kernels.
Read the full abstract ▾
Information Theory · Cross-listed from cond-mat.stat-mech
A general framework for crystallization in maximal hard-core models
Alexander Barg, Qidong He and Geyang Wang
We study general maximal hard-core models on periodic
lattice-type graphs. While for standard models, phase coexistence may arise for high activity, for maximal models this behavior may occur also for sufficiently low activity values. Relying on the concept of volume allocation, we develop a unified set of assumptions that imply the Peierls condition, a convergent cluster expansion for the partition function, and hence the conclusions of Pirogov-Sinai theory for this class of models. We further check these assumptions for a number of examples including standard
lattice-type periodic graphs, proving crystallization for both high and low activity. We also derive estimates for the values of activity that bound the phase coexistence regions in the phase diagram, rewriting for this purpose the proof of a technical result in the derivation of Pirogov-Sinai theory.
Read the full abstract ▾
Information Theory
Language Modeling is Monotone Compression
Noam Mazor, Andrew Morgan and Rafael Pass
A long-standing hypothesis in artificial intelligence and neuroscience posits that intelligence is closely related to compression: the ability to compress information efficiently intuitively reflects capacities associated with intelligence and learning. Indeed, recent experimental works verify this intuition by showing connections between the capabilities of large
language models (LLMs) and their ability as compressors: for instance, Deletang et al. (ICLR'24) demonstrate that LLMs can be used as powerful compressors, and Huang et al. (COLM'24) show that the compression ability of LLMs is highly correlated with their performance on
benchmarks for knowledge and
reasoning. In this work, we initiate a theoretical study of this connection. Our main result is that LLMs (formally modeled as next-
token predictors) are equivalent to monotone (a.k.a. order-preserving) compression
algorithms—namely, compression
algorithms where the encoding process preserves the ordering of the inputs—in the sense that the one can be constructed from the other while preserving the same error up to an additive gap of 2. We next show that the monotonicity is required for this equivalence to hold if and only if cryptographic (infinitely-often) one-way functions exist. As a direct corollary, we get a cryptographic result of independent interest: the notion of next-bit pseudoentropy (a computational analogue of
entropy) of a distribution is equivalent to monotone incompressibility of the distribution. (Previously, it was only known (Haitner et al., ITCS'23) that incompressibility implies next-bit pseudoentropy.)
Read the full abstract ▾