Information Theory
Papers
Fresh from arXiv, typeset properly. Highlighted words trade on the market.
Friday 3 July
Adaptive token processing has emerged as a promising approach for improving the efficiency of semantic communication systems. However, existing semantic communication frameworks largely overlook token-level multiple access and the impact of semantic interference among simultaneously transmitted semantic tokens. In this paper, we propose Adaptive Token Selection and Token-Domain Multiple Access (ATS-ToDMA), a novel cross-modal semantic communication framework that jointly performs semantic token selection, interference-aware scheduling, and semantic-aware power allocation. The proposed framework introduces a Semantic Signal-to-Interference-plus-Noise Ratio (SSINR) metric that captures the combined effects of channel impairments and semantic interference arising from token similarity. A transformer-based scheduler is developed to allocate selected semantic tokens across token-domain transmission slots while mitigating both intra-modal and cross-modal semantic interference. To characterize the behavior of the proposed system, analytical bounds on semantic interference and feasible token occupancy are derived, together with a closed-form approximation for semantic-aware power allocation. Simulation results demonstrate significant gains in semantic throughput and semantic decoding accuracy while reducing aggregate semantic interference and transmit power compared with OMA, Semantic NOMA, Random-TS, and Greedy ATS benchmarks.
Information Theory · Cross-listed from cs.DC
Scale-Free Beamforming using Swarm Arrays for Remote Sensing under Interference
We consider a swarm array of autonomous relays that seek to cooperatively forward a desired signal to a fusion center with the maximum possible fidelity while canceling out a number of interferers. We present a distributed algorithm for computing the optimal zero-forcing beamforming weights at the relays without requiring prior channel knowledge. Crucially, our algorithm is scale-free in the sense that the computational and bandwidth overheads are completely independent of the size of the array. We build on recent work that introduced the concept of a Collective Array that enables such scale-free computation by imposing a constraint that the array must always function as a swarm i.e. array elements can only ever communicate with external nodes collectively and never individually. While this is a very severe restriction, we show that it allows useful computations such as zero-forcing beamforming while being robust to noise and channel time-variations.
Read the full abstract
Information Theory · Cross-listed from cs.LG
Towards Diverse and Comprehensive Benchmarks for Mutual Information Estimation
Mutual information (MI) estimation is a central problem in machine learning and statistics; however, existing benchmarks typically evaluate estimators on simplified, low-dimensional distributions, leaving their performance on complex, realistic data largely unexplored. We address this gap with a comprehensive benchmarking framework grounded in a unified copula-theoretic perspective that subsumes existing benchmarks as special cases. Within this framework, we propose two complementary families of tests: a copula-first family that systematically varies ground-truth MI, dimensionality, and marginal complexity using synthetic and flow-based transformations; and a marginals-first family that couples real-world image data with controlled dependency structures, extending the classic same-class-pairing paradigm. We use this suite to extensively evaluate three classes of estimators: non-parametric, discriminative, and generative. Contrary to prevailing assumptions, our results indicate that there is no universal winner: each category can systematically outperform all other estimators under specific setups. By analyzing these cases, we identify fundamental estimation barriers and propose new tests that more effectively stress these specific limitations. We share the open source code at https://github.com/VanessB/mutinfo.
Read the full abstract
Information Theory · Cross-listed from math.CO
The multilinear forms Cayley graph and the eigenvalue method for tensor codes
The connections between graph theory, and more generally association schemes, and coding theory were established by Delsarte for the Hamming metric and rank-metric codes. The ambient metric space of Hamming-metric codes and rank-metric codes can be seen as Cayley graphs generated by words of weight one. The metrics considered then coincide with the geodesic distances of these distance-regular graphs. We focus on a generalisation of this framework to the space of tensors over a finite field, endowed with the tensor-rank as a metric. This space corresponds to the Cayley graph generated by rank-one tensors, which is not distance-regular for tensors of order at least 3. We show that the spectrum of this graph has a recursive expression and depends on the possible intersections between tensor subspaces of large enough dimension and the Segre variety. The spectrum of this graph for 3-order tensors can be expressed with the rank distribution of the rank-metric codes generated by these tensors. In particular, we obtain the complete spectrum of the graph for 2x3x3 tensors over any finite field. We apply this result to derive bounds on the dimension of tensor codes in the tensor-rank metric using the eigenvalue method, and in particular the ratio-type bound.
Read the full abstract
Information Theory
Deep-Unfolded Wideband ISAC Beamforming for DMA Under Frequency-Selective Lorentzian Model
Integrated sensing and communications (ISAC), empowered by dynamic metasurface antennas (DMAs), has emerged as a promising paradigm for next-generation wireless networks. However, existing DMA-based designs commonly rely on the frequency-flat response model for DMA elements, which is accurate only in narrowband scenarios and can cause significant phase and magnitude mismatches in wideband and ultra-wideband systems. This paper investigates a DMA-based wideband ISAC system under a frequency-selective Lorentzian response model, which accurately captures the frequency-dependent behavior of DMA elements. We aim to jointly balance the aggregate signal-to-interference-plus-noise ratio (SINR) of communication users and the signal-to-noise ratio (SNR) of the radar target. To this end, we first develop an alternating optimization framework based on projected gradient ascent (PGA), deriving closed-form gradients of the objective function with respect to the digital beamforming vectors, resonance frequencies, and damping factors under the frequency-selective Lorentzian DMA model. We then propose an unfolded PGA architecture that preserves the interpretability of model-based optimization while learning key hyperparameters to accelerate convergence. Simulation results show that the frequency-selective Lorentzian model improves performance by approximately 20% over its frequency-flat approximation. Moreover, deep-unfolded PGA achieves up to 20-fold faster convergence and improves the objective value by up to 7% compared with PGA-based benchmarks.
Read the full abstract
Information Theory
Diffusion-Based Noise-Adaptive Null-Space Channel Estimation for OFDM Systems
Accurate channel estimation in orthogonal frequency division multiplexing (OFDM) systems remains challenging when demodulation reference signal (DMRS) observations are sparse and noisy, and when DMRS configurations vary across deployment scenarios. This paper proposes DANCE (Diffusion-based Noise-Adaptive Null-space Channel Estimation), a diffusion-based channel estimator for OFDM systems. We formulate DMRS-aided channel estimation as a sparse linear inverse problem whose measurement operator is induced by the pilot pattern. The resulting range-null space decomposition separates the measurement-constrained range-space component from the unobserved null-space component, which is reconstructed through a learned diffusion prior. To avoid directly imposing noisy pilot samples as exact constraints, DANCE introduces a noise-adaptive posterior correction into the reverse diffusion process. The correction coefficient and the residual sampling variance are jointly calibrated according to the observation noise level, thereby reducing pilot-noise injection while retaining useful measurement information. We further design a conditional U-Net denoiser for complex-valued OFDM channel grids, where the real and imaginary components are represented as separate feature channels and downsampling is performed only along the subcarrier dimension. Simulations based on 5G NR tapped delay line (TDL) and clustered delay line (CDL) channel models show that DANCE achieves consistently lower normalized mean squared error (NMSE) than conventional estimators and diffusion-based posterior sampling methods under different signal-to-noise ratios, DMRS configurations, Doppler frequency shifts, and train-test distribution mismatches.
Read the full abstract
Information Theory · Cross-listed from cs.LG
CuBAS: Information Geometric Curvature-Based Adaptive Sampling for Supervised Classification
The informativeness of a training set is as consequential as its size, yet most sampling strategies remain agnostic to the intrinsic geometry of the data distribution. We introduce CuBAS (Curvature-Based Adaptive Sampling), an information-geometric framework for adaptive data selection in supervised classification, grounded in the q-state Potts Markov random field (MRF) model. The central insight is that a labeled dataset can be viewed as a statistical manifold, on which local curvature, estimated via the ratio of second to first-order observed Fisher information, faithfully encodes the geometric complexity of the data distribution. We construct a k-nearest-neighbor graph over the labeled data and derive a closed-form curvature score at each vertex from the Potts sufficient statistics. This curvature signal partitions the graph into two complementary regimes: low-curvature regions, corresponding to smooth, homogeneous clusters, and high-curvature regions, concentrated around decision boundaries that are disproportionately informative for classification. By selecting nodes from both regimes, CuBAS constructs compact yet maximally informative training subsets. Empirical evaluation across more than 60 benchmark datasets demonstrates consistent and statistically significant improvements over random sampling and uncertainty-based baselines, across a wide range of labeling budgets and classifier architectures. CuBAS is computationally efficient (linear in the number of k-NN graph edges), theoretically grounded in the differential geometry of statistical manifolds, and interpretable in terms of the local shape operator of the data manifold.
Read the full abstract
Information Theory
Open Problem: Is Interaction Necessary for Order-Optimal 1-bit Mean Estimation?
We ask whether interaction is necessary for order-optimal 1-bit mean estimation over nonparametric finite-moment classes. Adaptive threshold-query protocols achieve the order-optimal 1-bit minimax rate, and the same rate is attainable with general 1-bit queries using only one adaptive transition (i.e., two stages of querying). In the non-adaptive setting, threshold and interval queries are known to be highly suboptimal, but the case of arbitrary non-adaptive quantizers remains unresolved. Can such quantizers match the adaptive rate, yielding an optimal one-shot protocol? Or is the known two-stage estimator stage-optimal, with a single adaptive transition being necessary and sufficient?
Read the full abstract
Information Theory
Cramér-Rao Bound Optimization for Massive MIMO DFRC Systems with 1-Bit DACs and ADCs
In this paper, we investigate the dual-function radar-communication (DFRC) design for massive multiple-input multiple-output (MIMO) systems equipped with 1-bit digital-to-analog converters (DACs) at the transmitter and 1-bit analog-to-digital converters (ADCs) at the receiver, motivated by the need for low-cost and power-efficient implementations of massive MIMO systems. We consider a downlink scenario where the transmit signal matrix is optimized to enhance sensing performance while satisfying communication quality of service (QoS) requirements. Specifically, the objective is to minimize the 1-bit Cramér-Rao bound (CRB) for estimating the azimuth angle of a point-like target under symbol-level constructive interference (CI) constraints. We conduct an asymptotic analysis of the 1-bit Fisher information, revealing its nonmonotonicity with the signal-to-noise ratio (SNR), and introduce amplitude constraints to exclude regions where the objective function value is clearly suboptimal and facilitate convergence to high-quality solutions. The resulting problem is a nonconvex optimization challenge with coupled binary and linear constraints. We transform the discrete problem into a continuous constrained one, characterize its global and local minima, and tackle it via the augmented Lagrangian method (ALM) and a spectral projected gradient (SPG) method combined with nonmonotone line search. The solution is further refined via local search and cutting-plane techniques. Extensive numerical experiments verify our analysis, showing that the proposed approach exhibits promising DFRC performance compared to benchmark schemes.