Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Information Theory

cs.IT Watch Information Theory (sign in to use)

Thursday 8 October

Information Theory

Estimation with Quantized Parameter Side-Information

This paper presents upper and lower bounds on the minimax risk of a parameter estimation problem where the estimator not only observes independent and identically distributed observations but also a quantized version of the parameter, where the quantization has a limitation on the number of levels, but can be optimized otherwise. Our upper and lower bounds have similar decay rates for large number of observation samples and quantization levels.
Read the full abstract

Information Theory · Cross-listed from math.CO

Covering radius of rank-metric codes via covering lifts and clubs

We introduce a generator-matrix-based geometric approach to the covering radius of $\mathbb F_{q^m}$-linear rank-metric codes. Starting from the $q$-system associated with the code, we define the notion of covering lift and show that the covering radius can be recovered from the hyperplane weights of such lifts. This yields an exact criterion, in terms of the length and effective length of the code, for the covering radius to attain its largest possible value, together with the upper bound $\rho(\mathcal{C})\le \min\{m,n\}-1$ in all remaining cases. We then specialize our approach to $1$-dimensional codes, for which the effective length coincides with the minimum rank distance. In this setting, codes attaining $\rho(\mathcal{C})= \min\{m,n\}-1$ are related to covering lifts that are clubs. More generally, we characterize this case through linear maps into suitable quotient spaces, obtaining equivalent interpretations in terms of generalised evasive subspaces and auxiliary matrix rank-metric codes. These results determine the covering radius for several boundary values of the minimum distance and for all $1$-dimensional codes with extension degree $m\in\{3,4,5\}$. For $m=6$ and every $q$ we provide constructions and, for $q\in\{2,3\}$, computational results showing that $1$-dimensional rank-metric codes with the same length and minimum distance can have different covering radii.
Read the full abstract

Information Theory

Breaking linearity in the sum-rank metric: MSRD codes from switched $\sigma$-rational normal curves

We construct a family of scalar-closed, non-additive maximum sum-rank distance codes (MSRD) by switching norm components of a $\sigma$-rational normal curve and lifting the resulting point set through a linearized Reed–Solomon syndrome map. We characterize the admissible switches by a multiplicative stability condition on the norm classes. In particular, for $1 \leq n_i \leq m$, $N=n_1+\ldots+n_{\ell}$ and $2\leq\delta\leq N-1$, every partition of $\mathbb{F}_q^*$ satisfying a specific condition, referred to as Condition $(\diamond)$, yields a code in $\mathbb{F}_{q^m}^{N}$ of size $q^{m(N-\delta+1)}$ and minimum sum-rank distance $\delta$; the code is non-additive whenever a curve component is retained. For $\ell\geq2$ these appear to be the first non-additive MSRD codes in the literature, all previously known families with more than one block being $\mathbb{F}_q$-linear. Moreover, every code of the family has the same sum-rank weight distribution as an $\mathbb{F}_{q^m}$-linear MSRD code with the same parameters, although the geometry of the switched set distinguishes it from linearized Reed–Solomon codes. In the single-block case, an explicit rank isometry identifies the construction with the cone codes of Durante, Grimaldi and Longobardi.
Read the full abstract

Information Theory · Cross-listed from cs.LG

Recovery Guarantees for Posterior Sampling of One-Bit Compressed Sensing

We study the sample complexity of noisy one-bit compressed sensing for signals drawn from a prior distribution. By characterizing the effective distributional complexity of the prior via its approximate covering number, we prove that posterior sampling achieves accurate recovery with high probability when the number of measurements scales with the logarithm of the approximate covering number, up to a one-bit separation gap factor. This upper bound is robust to learned prior mismatch. Specifically, we show that posterior sampling with an approximate prior remains reliable, provided that the learned prior distribution is sufficiently close to the true signal distribution in Wasserstein distance. In addition, we establish a sample complexity lower bound for any reliable method of noisy one-bit compressed sensing, showing that our upper bound is nearly matched in its main prior dependent term. To approximate the ideal posterior sampling process for real world scenarios, we instantiate posterior sampling through a plug-and-play algorithm with diffusion priors. Experiments on the FFHQ and ImageNet datasets demonstrate the effectiveness of our proposed approach.
Read the full abstract

Information Theory

Distributed Constrained Resource Management in 6G Networks: A Scalable Hybrid Model-Learning Framework

Radio Resource Management (RRM) is a fundamental challenge in 6G wireless networks, particularly under dynamic user demands, inter-cell interference, and heterogeneous QoS constraints. Centralized optimization solutions are often infeasible in practice due to the lack of system-wide state information, dynamic conditions, and signaling delays, making distributed learning-based approaches attractive. However, conventional multi-agent reinforcement learning (MARL) struggles with scalability and constraint satisfaction in such highly dynamic environments. We introduce a scalable two-phase hybrid learning framework to address the RRM challenges where orthogonal-frequency division multiplexing (OFDM) domain knowledge is explicitly incorporated into the MARL pipeline. In our proposed two-phase learning framework, the first phase allocates a minimum resource to satisfy users' QoS requirements based on channel statistics, avoiding the inefficiencies of training policies under hard QoS constraints. Subsequently, a multiagent system is employed in the second phase to optimally allocate the remaining resources for system throughput maximization. By exploiting the structural interference information, we propose a scalable MARL algorithm which decomposes the original learning problem into smaller subproblems that can be handled independently, thereby avoiding exponential growth of the action space without compromising performance. Extensive simulations in realistic scenarios with 50MHz bandwidth and different numerologies show that our method significantly outperforms existing optimization and learning baselines, offering up to 40% throughput improvement, and 100% constraint satisfaction with minimal resource usage.
Read the full abstract

All authors Thang X. Vu, Cuong Le, Symeon Chatzinotas, Bjorn Ottersten

Information Theory · Cross-listed from eess.SP

Outage-Aware Robust Sensing for UAV ISAC Systems

Integrated sensing and communications (ISAC) infrastructure can provide an external perception layer for aerial robots, but dynamic blockage makes the set of informative sensing links at the next update uncertain. A single predicted availability pattern can miss low-information outcomes, whereas protecting all patterns wastes power and may become infeasible when all links are blocked. We propose an outage-aware conformal robust power-allocation framework for multi-BS UAV tracking. A history-aware joint predictor maps radar and tracking histories to next-slot availability probabilities, and adaptive prediction sets retain plausible correlated patterns. Tracking constraints are enforced only for retained non-outage patterns, while the all-blocked pattern is treated separately as a physical sensing outage. With per-slot beam directions fixed, the resulting scalar-power allocation is a semidefinite program. A reliability bound relates serviceable-slot tracking failure to deployed-policy miscoverage and optimization infeasibility. Across paired closed-loop simulations over five random seeds, the method attains a 0.01% serviceable-slot constraint-violation rate and uses 47.0% less power than protection over all non-outage patterns.
Read the full abstract

All authors Qiming Li, Lei Zhang, Haoran Xu, Lina Mohjazi

Information Theory

From an entropic inequality for sums in abelian groups to the theorems of Ruzsa and Plünnecke

We give a direct information-theoretic proof of the Plünnecke–Ruzsa inequalities for finite subsets of abelian groups. The argument rests on an elementary coupling inequality proved by maximizing joint entropy, without tensor powers or asymptotic constructions. It also yields Ruzsa's sumset triangle inequality and an extension to non-abelian groups.
Read the full abstract

Information Theory

Three Improved Lower Bounds for Binary Error-Correcting Codes

Error-correcting codes are a foundational technology for representing information robustly against noise in communication and data storage. The binary code packing problem, which seeks to maximize the number of codewords at a fixed code length and minimum distance, is a fundamental problem in coding theory that remains open for many parameter choices. We give explicit binary codes with parameters $(n,M,d)=(22,84,9)$, $(24,196,9)$, and $(25,65,11)$, where $n$, $M$, and $d$ denote the code length, number of codewords, and minimum distance, respectively. These establish the lower bounds $A_2(22,9)\ge84$, $A_2(24,9)\ge196$, and $A_2(25,11)\ge65$, where $A_2(n,d)$ denotes the maximum possible number of codewords, improving the respective lower bounds 80, 192, and 64 in the public table of general binary codes. Most notably, the third result improves the lower bound supplied by a lexicographic construction for the first time in 66 years, while the first two improve bounds that have stood for 21 years. The new codes represent more messages without changing the code length or error-correction guarantees, with gains compounding multiplicatively across blocks. We release all codewords and an exhaustive verifier to enable independent verification.
Read the full abstract

Information Theory

DRL-Based AoI Minimization for RSMA in Finite-Blocklength MU-MISO

This letter investigates age-of-information (AoI) minimization in multi-user wireless networks operating in the finite-blocklength (FBL) regime, which is critical for low-latency transmission of short state-update packets. While rate-splitting multiple access (RSMA) provides a powerful and flexible framework for interference management in multi-user FBL systems, the joint optimization of its parameters, such as precoding vectors, power allocation, and rate-splitting ratios, to guarantee information freshness results in analytically intractable complexity. To address this challenge, we propose an actor–critic deep reinforcement learning (DRL) framework to learn dynamic resource-allocation policies in multi-user multiple-input single-output (MU-MISO) broadcast channels. Simulation results show that the proposed RSMA-RL framework achieves consistently lower AoI than the state-of-the-art benchmarks, with substantial gains observed at low signal-to-noise ratio (SNR) and short blocklengths, while matching benchmark performance at high SNR with significantly lower online complexity via a single neural-network forward pass at execution.
Read the full abstract

All authors Enes Kaya, Mehmet Alp Demircioğlu, Elif Tugce Ceran, Melda Yuksel

Information Theory · Cross-listed from quant-ph

Minimal Scenarios Witnessing Genuine Multipartite Nonlocality Possibilistically

We study a strong form of genuine multipartite nonlocality (GMNL), where an $n$-partite scenario admits a quantum strategy generating outcomes belonging to a restricted set with certainty, whereas every strategy built from nonclassical resources shared among at most $k<n$ parties fails this criterion with positive probability. This generalizes what is referred to as strong contextuality/nonlocality, or pseudotelepathy, beyond ruling out local models. For the simplest five-party Bell scenario of two binary-outcome measurement settings per party, we use the inflation technique to demonstrate that a game with a perfect quantum strategy witnesses a robust separation from models built from bipartite resources under local operations and shared randomness (LOSR), establishing the behavior is LOSR-GMNL. We provide broader context for this result by investigating its simulability with resources shared among three or four parties under different resource models and alternative definitions of GMNL. A central open question is whether $n$-party quantum resources can achieve such a separation from models built from $(n-1)$-party resources, for $n\ge 3$. We make progress here by constructing a family of robustly LOSR-GMNL games with $\epsilon$-optimal quantum strategies experimentally indistinguishable in the limit from true probability-one-winning strategies for any desired level of precision.
Read the full abstract

Information Theory · Cross-listed from cs.DS

Tight Bounds for Equivalence Testing with Non-Adaptive Conditional Samples

We study distribution testing with access to non-adaptive conditional samples. Specifically, we give tight bounds for equivalence testing, determining whether two unknown distributions are equal to or $\varepsilon$-far from each other in total variation distance. Our algorithm and lower bound show that $\tilde \Theta\left(\frac{\log n}{\varepsilon^2}\right)$ queries are necessary and sufficient for this problem. These results demonstrate that the complexity of uniformity, identity, and equivalence testing with non-adaptive conditional samples are all $\tilde \Theta(\log n)$.
Read the full abstract

Information Theory · Cross-listed from q-bio.NC

If the brain were so simple: Information-based drivers of intelligence and their interaction through an n-body-inspired framework

Theories of how the brain processes information and returns intelligent outputs are numerous and often difficult to conclusively test. Here, we consider an approach grounded in informatics and algorithmic thermodynamics for how neural systems respond to different patterns of information in different ways, with a focus on the relationship between the information entropy of a signal and a new proposed quantity we term information enthalpy, which represents the internal resource of structured information within a system available to be used for predictions and other forms of work. To evaluate the capacity of the external signals to increase information enthalpy in the system, we introduce the information enthalpy potential (IEP) as a defined metric. We propose and implement a method for quantifying the amount of potential information enthalpy - the IEP - in a given signal across a range of example signals. We offer a conjecture of how information enthalpy may be treated by neural systems within a biological framework before showing how it may integrate and offer falsifiability to existing approaches such as the Free Energy Principle. This framework is also positioned in light of the role of neural criticality. Finally, we postulate a testable and falsifiable framework where the distinct mechanisms within each driver interact through a described n-body-inspired model to govern the "motion" of a neural system through a high dimensional state-space. Through these processes, it is proposed that these features form the most fundamental basis of the neural drivers that give rise to the complexity of adaptive behaviors that are commonly called: intelligence.
Read the full abstract

All authors Brett J. Kagan, Johnson Zhou, Forough Habibollahi, Valentina Baccetti

Information Theory · Cross-listed from stat.ML

A General $\widetilde{\Omega}(\sqrt{T \gamma_T})$ Lower Bound for Kernel Bandits

The kernel bandit problem consists of sequentially optimizing an unknown function with noisy feedback, where the function has bounded norm in a given Reproducing Kernel Hilbert Space (RKHS). A central quantity in the regret analysis of kernel bandits is the maximum information gain $\gamma_T$. In particular, the best existing upper bounds scale as $\sqrt{T\gamma_T}$ up to log factors, and nearly-matching lower bounds have been derived for specific kernels such as squared exponential and Matérn. However, lower bounds for general kernels are lacking, thus making it unclear in what generality the upper bounds are near-optimal. In this paper, we establish a general $\Omega(\sqrt{T\gamma_T/\log T})$ minimax regret lower bound for non-constant continuous kernels on compact domains, establishing near-optimality (within log factors) in a very general sense. We show that the log factor appearing in this bound is unavoidable in general, but that it can be removed under certain conditions. Among other things, our findings imply that the minimax-optimal scaling is exactly $\Theta(\sqrt{T\gamma_T})$ (i.e., within constant factors) for the Matérn-$\nu$ kernel with $\nu \in (0,2)$, $\gamma$-exponential kernel with $\gamma \in (0,2)$, and certain piecewise-polynomial kernels.
Read the full abstract

Information Theory · Cross-listed from cond-mat.stat-mech

A general framework for crystallization in maximal hard-core models

We study general maximal hard-core models on periodic lattice-type graphs. While for standard models, phase coexistence may arise for high activity, for maximal models this behavior may occur also for sufficiently low activity values. Relying on the concept of volume allocation, we develop a unified set of assumptions that imply the Peierls condition, a convergent cluster expansion for the partition function, and hence the conclusions of Pirogov-Sinai theory for this class of models. We further check these assumptions for a number of examples including standard lattice-type periodic graphs, proving crystallization for both high and low activity. We also derive estimates for the values of activity that bound the phase coexistence regions in the phase diagram, rewriting for this purpose the proof of a technical result in the derivation of Pirogov-Sinai theory.
Read the full abstract

Information Theory

Language Modeling is Monotone Compression

A long-standing hypothesis in artificial intelligence and neuroscience posits that intelligence is closely related to compression: the ability to compress information efficiently intuitively reflects capacities associated with intelligence and learning. Indeed, recent experimental works verify this intuition by showing connections between the capabilities of large language models (LLMs) and their ability as compressors: for instance, Deletang et al. (ICLR'24) demonstrate that LLMs can be used as powerful compressors, and Huang et al. (COLM'24) show that the compression ability of LLMs is highly correlated with their performance on benchmarks for knowledge and reasoning. In this work, we initiate a theoretical study of this connection. Our main result is that LLMs (formally modeled as next-token predictors) are equivalent to monotone (a.k.a. order-preserving) compression algorithms—namely, compression algorithms where the encoding process preserves the ordering of the inputs—in the sense that the one can be constructed from the other while preserving the same error up to an additive gap of 2. We next show that the monotonicity is required for this equivalence to hold if and only if cryptographic (infinitely-often) one-way functions exist. As a direct corollary, we get a cryptographic result of independent interest: the notion of next-bit pseudoentropy (a computational analogue of entropy) of a distribution is equivalent to monotone incompressibility of the distribution. (Previously, it was only known (Haitner et al., ITCS'23) that incompressibility implies next-bit pseudoentropy.)
Read the full abstract

Wednesday 7 October

Information Theory

QoS-Aware Joint Concurrency and HARQ-Limit Design for HARQ-CC-Aided Slow Fluid Antenna Multiple Access

Hybrid automatic repeat request with Chase combining (HARQ-CC) improves the reliability of slow fluid antenna multiple access (sFAMA), but its transmission limit must be jointly optimized with user concurrency. This paper proposes a quality-of-service (QoS)-aware joint design of concurrency level U and HARQ limit C for downlink HARQ-CC-aided sFAMA. Based on the selected-port signal-to-interference ratio (SIR) distribution, we derive multi-round decoding failure probabilities and develop a two-stage optimization that maximizes feasible concurrency and then selects the minimum C satisfying outage and mean-service constraints. We prove that the first feasible C minimizes service duration for a given U, and that feasible U values form an initial segment, enabling optimal bisection search. Numerical results agree with Monte Carlo simulations under full Jakes spatial covariance. The proposed adaptive FAS design supports six concurrent users, compared with three to five for fixed-HARQ FAS and two for HARQ-adaptive fixed-position-antenna (FPA), demonstrating that joint concurrency and HARQ adaptation expands the QoS-feasible region of FAS.
Read the full abstract

All authors Sixu Han, Kai-Kit Wong, Hanjiang Hong, Haoyu Liang

Information Theory · Cross-listed from math.CO

A $G$-Invariant Erdős–Hanani Theorem with Applications to Optical Orthogonal and Constant-Weight Codes

Let $k>t\ge 2$ be fixed integers, and let $G$ be a permutation group on a set of $v$ points. A $G$-invariant packing is free if every block has $|G|$ distinct images under $G$. We give two conditions on $G$ under which, as $v\to\infty$, there is a free $G$-invariant $t$-$(v,k,1)$ packing with $(1-o(1))\binom{v}{t}/\binom{k}{t}$ blocks. This is a $G$-invariant form of the Erdős–Hanani theorem on asymptotically optimal packings. For semiregular groups $G$, packings with these properties exist if and only if all but $o(v^t)$ of the $t$-subsets of points have trivial setwise stabiliser. This property is satisfied by every semiregular group for $t\ge 3$, and by every cyclic semiregular group for $t=2$. As a consequence, the Johnson bound is asymptotically attained at all lengths by optical orthogonal codes of constant weight $w$ and constant correlation $\lambda$ with $w\ge\lambda+2$, by their multidimensional and signature pattern versions, and by cyclic and quasi-cyclic constant-weight codes. The same method gives a form of the theorem for group divisible packings that are invariant under a cyclic group. This determines the asymptotic maximum size of two-dimensional optical orthogonal codes with at most one pulse per wavelength.
Read the full abstract

Information Theory

Modeling and Performance Evaluation of Distributed Satellite Antennas: Space Is Much Closer Than One Might Think

Distributed satellites can deliver broadband connectivity directly to commercial handsets. In the formation of arrays (FoA) architecture, cooperating satellites synthesize a large aperture whoseextent depends on their number, arrangement, and spacing, with intersatellite distances ranging from tens of meters to hundreds of kilometers. For representative S-band geostationary and low-Earth-orbit configurations, the Fraunhofer distance may exceed the nadir link range at spacings on the order of a hundred meters. Therefore, accurate assessment of FoA performance across satellite populations and spacings requires spherical-wave modeling for users in the radiative near field, below the formation's Fraunhofer distance. Planar models of satellite positions, array orientations, and user locations must also be assessed against spherical geometries accounting for orbital and Earth curvature. This paper combines these models to quantify how simplifying assumptions affect throughput and outage probability. Numerical results show that multi-antenna precoding based on a local plane-wave approximation within each satellite array closely matches the spherical-wave reference. A global plane-wave approximation can instead cause severe losses, including full outage. In the investigated scenarios, formation curvature has little impact, whereas neglecting Earth curvature overestimates low-Earth-orbit throughput. These findings clarify when geometric and propagation approximations remain reliable and demonstrate that, for large distributed antenna apertures, space is much closer than one might think.
Read the full abstract

All authors Giacomo Bacci, Riccardo De Gaudenzi, Marco Luise, Luca Sanguinetti

Information Theory · Cross-listed from eess.SP

Low-complexity Equalization of Zak-OTFS Via Neumann Series

We describe a general method for selecting an orthonormal basis of carrier waveforms that aligns the basis with delay / Doppler characteristics of a wireless channel. We show that our method enables low-complexity equalization for two channels of practical interest. The first is satellite communication and the second is communication from a ground station to an unmanned aerial vehicle (UAV). After Doppler compensation both scenarios are characterized by a first line of sight (LOS) path with zero delay and zero Doppler shift, and a second weaker path. We show that our method is robust to fractional delay and Doppler shifts. We consider orthonormal bases of carrier waveforms that are obtained from the pulsone basis of Zak-OTFS carrier waveforms by applying a generalization of the discrete affine Fourier transform (GDAFT). This family of waveforms includes AFDM and other modulations proposed for 6G. What distinguishes these bases is that the carrier waveforms are common eigenvectors of some maximal commutative subgroup S of a Heisenberg-Weyl group of discrete delay and Doppler shifts. We describe how to choose S to mitigate the damaging effects of interference between carriers. We represent the wireless channel by delay-Doppler taps and observe that a channel tap located within S multiplies every waveform by a complex phase. If all channel taps are located within S, then the channel multiplies every waveform by a complex phase, and a single tap equalizer supports reliable communication. This is the case for a linear time-invariant (LTI) channel, where S is the group of discrete time shifts and the carrier waveforms are discrete tones (OFDM). In general, we choose the subgroup S to maximize the diagonal component of the channel energy.
Read the full abstract

All authors Vineetha Yogesh, Saif Khan Mohammed, Sandesh Rao Mattu, Ronny Hadani, Robert Calderbank

Information Theory · Cross-listed from math.ST

On the Impossibility of Estimating the Average Mixing Time

We establish nearly-matching converse bounds on the sample complexity of estimating the average mixing time of a Markov chain from a single stationary trajectory. Unlike the worst-case mixing time, which maximises the total-variation distance to stationarity over all initial states, the average mixing time averages this distance under the stationary distribution and can therefore be substantially smaller. Our converse bounds establish when estimation is impossible, and match the key dependencies in existing achievability bounds, up to logarithmic factors. Together, these bounds provide nearly-tight PAC guarantees for estimating the average mixing time.
Read the full abstract

Information Theory · Cross-listed from cs.LG

How Many Repeated Pairwise Comparisons Are Needed for Ranking under Heterogeneity?

We study ranking models by population-average utility from pairwise comparisons when preferences vary across users and tasks. Prior work shows that a single comparison per user can be insufficient to identify the alternative with the highest average utility, even with arbitrarily many users (Golz et al., 2025). We investigate how many repeated comparisons within each user-task context are necessary and sufficient for ranking recovery. Under a heterogeneous Bradley-Terry model with fixed inverse temperature, we start with a naive MLE-based algorithm that requires $\Omega(1/\Delta^2)$ repeated comparisons per context to ensure ranking recovery. We then present two MLE-based variants and a randomized Russian Roulette-style algorithm that recover the ranking using $O(\log(1/\Delta))$ repeated comparisons per context, and we prove that this logarithmic dependence is optimal. Despite this worst-case requirement, our Russian Roulette algorithm uses only $O(1)$ comparisons per context in expectation. Synthetic experiments and semi-synthetic experiments based on Arena data compare the four algorithms in settings with varying levels of preference heterogeneity and under varying context distributions.
Read the full abstract

Information Theory

The Hellinger Conjecture for Antipodal Boolean Functions

The balanced Hellinger conjecture compares the overlap of two output laws induced by a Boolean decision under binary symmetric noise. We prove that a single input coordinate minimizes this overlap among antipodal decisions, whose values reverse when all input signs are reversed, throughout the noise interval. The proof uses a normalized nuclear norm of the cross-class block of the Gram matrix of square-root channel densities. Multiplication by an antipodal decision exchanges the even and odd Fourier spaces preserved by the Gram matrix. A coordinate exchange attains the minimum cost of matching their spectra. A matrix contraction extends the sharp bound to arbitrary independent binary-input channels. Fixed coordinate flips and randomized decisions follow from the same argument and concavity. These extensions give bounds for information and testing, including linear Gaussian observations, and a weighted edge-boundary inequality.
Read the full abstract

Information Theory · Cross-listed from cs.LG

Why Forget-Only Unlearning Needs Memorization

Machine unlearning asks for a deletion algorithm whose output is close to retraining from scratch without the selected forget examples. In this work, we study forget-only unlearning, where the deletion algorithm receives only the trained model and the examples to forget, with no retained data or extra training information. We ask whether forget-only unlearning is always possible. We first show that this depends on the learning method: different datasets can produce the same trained model but require very different outputs after the same examples are removed. Using this observation, we derive lower bounds on how accurately unlearning can match retraining and instantiate them for several standard learning algorithms. We then ask what must be true when forget-only unlearning succeeds. To this end, we derive lower bounds on what an algorithm must memorize about the training data to handle arbitrary deletion requests. For simple threshold learners, the required information can be as large as the entire dataset, even though ordinary training keeps only one boundary point. Overall, our results show that information discarded during ordinary learning may be needed later for deletion, so models designed for forget-only unlearning may need to retain more information than standard training does.
Read the full abstract

Information Theory

Polarforming for MIMO Covert Communications

Covert communication conceals wireless transmission activity, but conventional multi-antenna designs rely mainly on spatial beamforming and can be constrained by limited spatial degrees of freedom. This paper investigates polarization-reconfigurable antenna (PRA)-aided multiple-input multiple-output (MIMO) covert communication, where Alice and Bob perform transmit and receive polarforming, respectively, and Willie uses a vertically polarized antenna for radiometric detection. We jointly optimize the transmit covariance matrix and the transmit and receive phase shift vectors to maximize Bob's achievable rate under the covertness and transmit power constraints. Under Gaussian signaling, perfect polarized channel state information, and known noise power at Willie, we derive his optimal radiometric threshold and exact minimum detection error probability (DEP) for a finite observation length, and convert the DEP requirement into a deterministic constraint on his received signal power. We further prove that, for a fixed transmission design, positive signal leakage becomes detectable as the observation length tends to infinity. To solve the design problem for a finite observation length, we develop an alternating optimization (AO) algorithm with a generalized water-filling covariance update and closed-form phase updates. Numerical results validate the exact DEP analysis and the rapid convergence of the proposed algorithm. Compared with the sufficient divergence-based condition, the exact covertness constraint permits a $0.99$–$1.12$~dB higher received signal-to-noise ratio (SNR) at Willie, while joint transmit and receive polarforming provides a rate gain of up to $53.8\%$ over the considered benchmark schemes.
Read the full abstract

All authors Xin Xie, Jinpeng Xu, Yihang Huang, Li Zhou, Zhaolong Ning

Information Theory

Settling the Sample Complexity of Rényi Entropy Estimation

Rényi entropy estimation has been comprehensively investigated by Acharya, Orlitsky, Suresh and Tyagi (SODA 2015; IEEE Trans. Inf. Theory 2017) and consequent works, whereas only the sample complexity of Rényi entropy estimation of integer order has been settled. In this paper, we settle the sample complexity of Rényi entropy estimation of noninteger order, thereby completing the complexity picture of Rényi entropy estimation. Specifically, we show that for any noninteger $\alpha > 0$, it is sufficient and necessary to use \[ \Theta\!\left(\frac{d^{\max\{1/\alpha,1\}}}{\varepsilon^{1/\alpha}\log(d)} + \frac{d^{|1-1/\alpha|}}{\varepsilon^2}\right) \] samples to estimate the Rényi entropy of order $\alpha$ of an unknown discrete distribution over an alphabet of size $d$ to within additive error $\varepsilon$. For the upper bound, we reduce the bias using a refined polynomial approximation estimator for large probabilities. For the lower bound, we employ a different hard instance equipped with a new moment matching construction. The constructive moment matching has constant bounded high-order moments, while attaining a fixed ratio between the $\alpha$-th moments, which is of independent interest.
Read the full abstract

Information Theory

Pathwise Information Certificates for Decentralized Adaptive Sensing

We study decentralized adaptive sensing, where multiple agents choose measurements from evolving local beliefs while exchanging information over a communication graph. We ask whether the measurements actually selected by an adaptive policy have collected enough evidence to distinguish the true target from every plausible alternative. We develop a pathwise certificate based on the Rényi–Chernoff information accumulated along the realized sensing trajectory. It yields nonasymptotic MAP-error bounds and an anytime, network-wide stopping rule for arbitrary history-dependent sensing policies, while separating accumulated statistical information from a bounded network-mixing transient. Linear growth of the information against the least-resolved competitor implies exponential decay of MAP and squared-localization error. A classical pairwise KL converse, specialized to the adaptive decentralized transcript, shows that insufficient information on any pair prevents a positive uniform error exponent, confirming the hardest competitor as a fundamental bottleneck. Across policies, graph topologies, sensor profiles, and seeds, the worst-competitor score correlates more strongly with localization speed than an average-pair proxy in both 1D ($r=0.89$ versus $0.40$) and structured 2D sensing ($r=0.77$ versus $0.48$). Our results provide a practical way to certify and diagnose adaptive multi-agent sensing systems using the evidence they actually collect.
Read the full abstract

Information Theory · Cross-listed from eess.SP

Antenna Coding by Pixel Antennas: Unlocking A New Dimension for Wireless Communications

Conventional wireless communication systems employ antennas with fixed radiation patterns. Enhancing system performance by increasing the number of antennas would push the limits of cost and size for communication devices. To overcome this issue, novel antenna technologies are urgently desired to break through the fundamental performance bottleneck and engineering limits. Pixel antennas provide a general framework for highly reconfigurable antennas, which can flexibly adjust radiation patterns via controlling variable lumped elements placed across adjacent pixels, unlocking novel degrees of freedom (DoF) in the beamspace domain. Recently, an emerging antenna coding technique links wireless system performance to the antenna configurations. This enables the use of pixel antennas to enhance system performance by leveraging the DoF from the antenna side. In this article, we provide a comprehensive overview on antenna coding by pixel antennas. Specifically, based on the pixel antenna model, the concept of antenna coding is firstly introduced, followed by a discussion on the classifications of antenna coding. Two cases for channel gain and multiplexing gain enhancement are highlighted to show the effectiveness of antenna coding in wireless communications. Besides, various optimization approaches for antenna coding design are investigated. Finally, the challenges and opportunities are outlined, providing guidance for future works.
Read the full abstract

All authors Zixiang Han, Shanpu Shen, Yifei Yuan, Jiangzhou Wang, Ross Murch, Chih-Lin I

Information Theory

MacKay-Neal Codes Achieve Capacity under MAP Decoding

Statistical-mechanical analyses predict that MacKay-Neal codes can achieve channel capacity at fixed degrees. We prove this prediction for the uncoupled $(\ell,3,3)$ MN socket ensemble for every fixed integer $\ell\ge4$. For each binary-input memoryless symmetric channel of capacity strictly greater than $3/\ell$, the ensemble-average block error probability under maximum a posteriori (MAP) decoding is $O(\log N/N)$, where $N$ is the transmitted blocklength. The actual transmitted rate converges to $3/\ell$. The result includes both punctured and transmitted variables and does not condition the sparse square matrix on invertibility. The proof establishes an entropy inequality for six bits subject to even parity by an analytic argument that reduces the domain by symmetry, restricts its interior stationary points, where both partial derivatives vanish, to the diagonal, and controls the resulting scalar functions by explicit polynomial bounds. Exact configuration counts then bound the conditional entropy on the binary symmetric channel. A standard comparison by mutual information extends the entropy bound to general symmetric channels; independent output erasures and a minimum-distance estimate for the transmitted code yield the bit and block error bounds.
Read the full abstract

Information Theory

q-ary GRAND

We develop q-ary Guessing Random Additive Noise Decoding (GRAND) for linear codes over a q-ary alphabet. The decoder works on a sorted symbol-likelihood array and uses three local child generation rules to generate symbol deviation patterns. We then prove that these rules induce a monotone rooted spanning tree of the full row-index space, so best-first traversal gives maximum-likelihood (ML) decoding under unlimited search. Reed-Solomon simulations verify ML agreement and show the finite-budget performance-complexity tradeoff.
Read the full abstract

Information Theory

Beyond LLM-GA: Secure Fluid Antenna Systems with ReEvo-Designed Memetic Algorithm

Fluid antenna systems (FASs) offer significant spatial flexibility, yet securing them against eavesdropping is critical for practical FAS deployment in military, satellite, and internet-of-things networks. Although large language model (LLM)-assisted genetic algorithms (LLM-GAs) can address this secure FAS port selection problem, whether further algorithmic improvement is possible warrants deeper investigation. To this end, we propose a memetic algorithm based on reflective evolution (ReEvo). Unlike the state-of-the-art LLM-GAs, which design only crossover or mutation operators with an LLM, our algorithm leverages an LLM to evolve dedicated crossover, mutation, and local-search operators offline. These operators are then embedded into a memetic search framework, thereby obviating any online LLM queries during execution. Simulation results at equal generation counts demonstrate that our proposed algorithm achieves a higher secure sum-rate than the conventional GA and the state-of-the-art LLM-GAs.
Read the full abstract