Information Theory · Cross-listed from cs.LG
External Clustering Validation by the Homogeneity-Parsimony Trade-off
Andreas Tiffeau-Mayer
Scalar metrics are often used to evaluate clusterings against known classes, but they can obscure a fundamental trade-off: clusterings should be informative about class labels while avoiding unnecessary fragmentation. Here we describe normalized scores of cluster homogeneity and parsimony that quantify this trade-off. These scores build on the information bottleneck principle, modified to not reward lossy compression. We show by example and mathematical proof that our definitions of these scores have the intuitive property of varying monotonically under cluster refinement in contrast to related proposals. Extending the information-theoretic framework beyond Shannon
entropies, we furthermore derive set-matching and pair-based counterparts of the homogeneity and parsimony scores. These unify commonly used evaluation criteria and show that, in the pair-based setting, the homogeneity-parsimony trade-off recovers the receiver operating characteristic of binary classifiers. We demonstrate the framework's utility for feature selection and
algorithm comparison, illustrating how considering scores jointly can clarify clustering operating points and identify Pareto-optimal solutions.
Read the full abstract ▾
Information Theory
Pipelined Gradient Coding
Xian Su and Jun Li
In large-scale machine learning, distributed training commonly involves multiple workers evaluating the gradients of the model on different
dataset partitions. A common challenge is the presence of straggling workers, which may significantly slow down training. Traditional gradient coding (GC) addresses this by duplicating
dataset partitions across workers, allowing for the replacement of missing gradients from stragglers. However, GC requires workers to evaluate gradients on multiple
dataset partitions in each step, potentially increasing overall training time. In this paper, we propose to pipeline GC, such that gradient evaluation is segmented across multiple steps and each worker evaluates gradients on just a single
dataset partition per step. We develop the pipelined version for fractional repetition (FR) and cyclic repetition (CR), two representative
dataset placement schemes in GC, and prove convergence guarantees for both. Through extensive simulations and experiments on cloud infrastructure, our schemes not only significantly reduce training time but also accelerate convergence compared to GC and other baselines.
Read the full abstract ▾
Information Theory · Cross-listed from math.PR
Exact Scale–Shape Factorization of the Typical Poisson–Voronoi Cell Volume in Arbitrary Dimension
Tian Shi and Minghua Xia
Despite more than six decades of research, a tractable closed-form distribution for the typical Poisson–Voronoi
cell volume remains unknown beyond one dimension. Building on the classical complementary-
theorem structure for the Poisson–Voronoi fundamental region, we develop an explicit configuration-space factorization of the Palm-typical
cell volume for a tessellation generated by a stationary Poisson point process of intensity \(\lambda>0\) in \(\mathbb R^d\), \(d\geq1\). Conditional on the number \(k\) of effective facets, the Voronoi flower volume is a \(\operatorname{Gamma}(k,\operatorname{rate}=\lambda)\) scale variable independent of the effective-neighbour configuration normalized to have unit flower volume. Mapping this normalized configuration to its
cell-to-flower volume ratio \(A_k\) gives the conditional
cell-volume representation as the product of the classical Gamma scale and a bounded
geometric shape factor. From this representation, we derive exact mixture formulae, transform and moment identities, and criteria characterizing when the conditional
cell-volume laws are Gamma. We also distinguish shape-factor variability within facet-number strata from mixing across strata as two sources of departure from a single Gamma law and recover the universal bound \(0<A_k\leq2^{-d}\). For the lower tail, we reduce critical negative moments of the shape factor to inverse-volume integrals over normalized configuration spaces. Under explicit critical-integrability and higher-facet summability assumptions, the density of the intensity-normalized
cell volume has leading order \(y^d\) as \(y\downarrow0\). The framework recovers the one-dimensional distribution, admits explicit planar coordinates, and supports numerical evaluation of the mixture representation in dimensions two through four.
Read the full abstract ▾
Information Theory
Mapped ADMM: A Robust Algorithm for 1-Bit mMIMO Detection
Mohammad Amin Keshmiri and Masoud Ardakani
Recently, it has been reported that one-bit massive
MIMO (mMIMO) detection is equivalent to a binary classification problem that can be solved efficiently using support vector machine (SVM). Inspired by this result, we first reformulate SVM in a decentralized form consisting of multiple classifiers. This enables the use of the consensus alternating direction method of multipliers (CADMM), a technique that can improve robustness and performance through its inherent consensus making. We further update CADMM to output only valid constellation points and achieve significantly improved detection performance. In our method, by changing the size of the grouped classifiers, we balance the number of classifiers for consensus accuracy with sufficient data per group to ensure classifier robustness. Ultimately, we demonstrate that our proposed method significantly outperforms existing practically feasible methods for one-bit mMIMO detection.
Read the full abstract ▾
Information Theory · Cross-listed from cs.CR
Constant-time decoding of Gabidulin codes and their generalizations with application to RQC
Nicolas Aragon, Chloé Baïsse, Anthony Fraga et al. (+2 more)
Gabidulin codes are a rank metric analog of Reed-Solomon codes. They are of great importance in rank-metric cryptography, as they are employed in highly efficient rank-based schemes, such as RQC and the Loidreau cryptosystem. However, until now, there has been no constant-time implementation of these codes, even though such an implementation is crucial for real-life development of cryptosystems. In this paper, we propose the first constant-time decoding
algorithm of augmented Gabidulin (AG) codes, a simple variation of Gabidulin codes, in which columns of zeros are added to the end of a Gabidulin code. AG codes are used in the most efficient variants of the RQC cryptosystem. We prove that AG codes decoding can be achieved with quadratic complexity by reducing the decoding of an AG code to the decoding of a shorter standard Gabidulin code. In addition to the description of the complete AG codes decoding procedure, we present a constant-time
algorithm for the left division of $q$-
polynomials that can be used in every constant-time implementation of Gabidulin code variants. These
algorithms are integrated into the RQC-Block-MS-AG scheme, and we evaluate the performance of our implementation through
benchmarks. Our results show that our constant-time implementation outperforms the original RQC, though it remains approximately four to five times slower than HQC. However, it achieves ciphertext and key sizes about four times smaller, highlighting an appealing trade-off between performance and compactness.
Read the full abstract ▾
All authors Nicolas Aragon, Chloé Baïsse, Anthony Fraga, Philippe Gaborit, Ilaria Zappatore
Information Theory
Fundamental Limits of MIMO ISAC: An Antenna Array Architecture Perspective
Po-Chih Chen, Ming-Chun Lee and Yu-Chih Huang
This paper investigates the fundamental limits of
MIMO integrated sensing and communications (
ISAC) systems, specifically comparing sparse arrays (SAs) against conventional uniform linear arrays (ULAs). A unified theoretical analysis of ergodic channel capacity and the Cramér$\unicode{x2013}$Rao bound (CRB) for angle estimation is developed while accounting for the array geometry. Utilizing the framework of
stochastic majorization, the study reveals that SAs consistently outperform ULAs by creating a more $\unicode{x201C}$uniform$\unicode{x201D}$ spatial
eigenvalue distribution, which decorrelates the multipath environment and increases communication capacity. For sensing, the paper proves that the angle CRB is inversely proportional to the array's second-order central moment of
antenna positions asymptotically, demonstrating that SAs achieve superior accuracy$\unicode{x2014}$improving by up to the square of the number of
antennas$\unicode{x2014}$due to their increased physical aperture. These analyses and conclusions are demonstrated to be also valid for
MIMO ISAC systems employed with the modern waveforms OTFS and OFDM, suggesting that spatial geometry, rather than waveform, is the primary driver of fundamental performance gains in the spatial dimension.
Read the full abstract ▾
Information Theory
Two-Way Wiretap Channel under Mixed Secrecy Constraint
Yanling Chen and Masahito Hayashi
This paper studies the two-way wiretap channel (TW-WC) with an external eavesdropper under strong one-sided (mixed) secrecy: only User~1's message is required to be secure from the eavesdropper, while no secrecy constraint is imposed on User~2's message. Secrecy is measured by the information leakage of User~1's message to the eavesdropper. Applying a non-adaptive construction and a one-sided reduction of the key-exchange construction, we obtain exponential error and leakage bounds for every fixed number of adaptive rounds. A quantitative one-time-pad argument propagates the leakage of the key used in the next round, and an explicit padding construction removes the initialization-rate loss for all sufficiently large blocklengths. We derive the corresponding strong mixed-secrecy achievable regions. The non-adaptive construction recovers the previously reported weak one-sided single-letter inner bound under the stronger secrecy criterion. The overall adaptive achievable region includes the non-adaptive construction as a fallback and also contains a key-exchange subregion that can be strictly larger.
Read the full abstract ▾
Information Theory
Learning to Decode Quantum LDPC Codes via Cluster-Based Sequential Belief Propagation
Mohsen Moradi, Taejoon Kim, Rémi A. Chou et al. (+1 more)
Belief-propagation (BP) decoding for
quantum low-density parity-check (QLDPC) codes is attractive due to its low complexity, but its performance is often limited by short cycles, degeneracy, and convergence failures. Recently, reinforcement-learning-based sequential variable-node (VN) scheduling (RL-S) was shown to improve BP decoding by learning state-dependent update orders. However, the VN-by-VN nature of that approach offers limited within-iteration parallelism, since only one VN is updated at a time. In this paper, we propose a cluster-based extension of RL-S for QLDPC codes. The VNs are partitioned into fixed clusters, and at each scheduling step the RL
agent selects one cluster to update, after which all VNs in that cluster are updated in parallel using the same pre-update incoming messages. To keep the tabular state space practical for large cluster sizes, we introduce a permutation-invariant cluster state based on a normalized histogram of local mismatch weights, followed by quantization. This representation makes the number of cluster states depend on the quantization resolution rather than the cluster size. We also develop the corresponding cluster-level Markov decision process, reward function, and Q-learning update. Numerical results on representative QLDPC codes show that the proposed clustered learned scheduling preserves most of the error-rate benefit of VN-level learned sequential scheduling while substantially reducing the number of scheduling decisions per BP iteration, thereby providing an attractive latency-parallelism tradeoff.
Read the full abstract ▾
All authors Mohsen Moradi, Taejoon Kim, Rémi A. Chou, David G. M. Mitchell
Information Theory
On Runlength Limited Codes for BICM Systems
Stephan Zeitz, Florian Roth, Meik Dörpinghaus et al. (+1 more)
We study the use of runlength-limited (RLL) block codes in bit-interleaved coded modulation (BICM) systems. In this setting, the RLL code acts as the symbol mapper, whose assignment between input bits and RLL symbols is critical for performance. In this work, we aim at optimizing the assignment scheme of RLL codes. One of the main applications of RLL codes is the mitigation of intersymbol interference (ISI) in systems with coarse quantization. However, channel
memory and quantization nonlinearity complicate information theoretic analysis. To enable analytical treatment, we consider a block channel with 1-bit analog-to-digital conversion, modeling the transmission of a single RLL code block. For this channel, we investigate the relationship between the achievable rate in BICM systems-termed BICM capacity-and the RLL code's assignment scheme. Focusing on low signal-to-noise ratios (SNRs), we derive the optimization problem yielding the optimal assignment scheme. By looking at asymptotically large block-lengths, we infer a practical optimization strategy for RLL codes with finite block-length and channels with inter-block interference. Further, we extend this optimization to two-state RLL (TS-RLL) codes, which offer higher code rates than state independent RLL codes. We demonstrate that optimized TS-RLL codes exhibit significant performance improvements over literature counterparts.
Read the full abstract ▾
All authors Stephan Zeitz, Florian Roth, Meik Dörpinghaus, Gerhard Fettweis
Information Theory · Cross-listed from math.FA
On the sample complexity of Fourier compressed sensing: wavelets versus shearlets
Giovanni S. Alberti, Alessandro Felisi and Iş{\i}l Güleken
This paper explores the measurement requirements for signal recovery in compressed sensing, comparing the performance of shearlet frames with traditional wavelet systems. Directional representation systems such as shearlets are known for their ability to sparsely represent images with anisotropic features, which allows for efficient nonlinear approximations. The central question we address is whether this difference in sparsity allows for a proportional reduction in the number of
Fourier measurements needed for successful reconstruction. On the theoretical front, we study the obstacles encountered when trying to apply standard frame-based recovery results to shearlet systems. First, we show that the (optimal) local coherence between
Fourier measurements and cone-adapted shearlets decays more slowly than the corresponding local coherence for wavelets. Second, managing the sparsifying system's redundancy relies on evaluating a localization factor, which requires lower frame bound estimates that can become exponentially small. Thus, even under optimal theoretical conditions, the number of samples required for shearlets scales quadratically with sparsity, which offers no substantial theoretical reduction over the standard wavelet
benchmark. These theoretical limitations are assessed through a series of numerical experiments on a
dataset of piecewise smooth images. While empirical observations confirm that shearlets can accurately represent these images using fewer coefficients than wavelets, phase diagrams indicate that this advantage in sparsity does not yield a proportional reduction in required
Fourier samples. Ultimately, we conclude that despite the superior nonlinear approximation rates of shearlets, their practical sample complexity in compressed sensing scenarios with subsampled
Fourier measurements remains comparable to that of traditional wavelets.
Read the full abstract ▾
Information Theory
Spatial Semantic Communication: When Semantic Transmission Meets Index Modulation
Xinghao Guo, Yin Xu, Dazhi He et al. (+5 more)
Current digital semantic communication systems have primarily focused on maintaining compatibility with conventional constellation-based modulation. In contrast, index modulation (IM) represents a more spectrally and energy-efficient alternative by exploiting additional dimensions for information conveyance. Recognizing this potential, this paper bridges the gap between IM and semantic communications by proposing a novel spatial semantic communication (SSC) system leveraging cutting-edge fluid
antenna-IM (FA-IM) technology. Compatible with existing joint source-channel coding (JSCC) architectures, the proposed SSC system employs the residual quantization (RQ) approach to discretize analog semantic features for subsequent digital IM transmission. Notably, the proposed SSC system synergizes RQ and IM via a semantic-aware stream splitting scheme, which ensures that critical semantic information undergoes less severe channel fading, thereby further optimizing semantic transmission performance. Simulation results validate that the proposed SSC system effectively integrates the high fidelity of RQ, the reliability of semantic-aware splitting, and the spatial efficiency of FA-IM, thereby providing a robust solution for future digital semantic transmission. The open source code is available at: https://github.com/gxh1106/SSC.
Read the full abstract ▾
All authors Xinghao Guo, Yin Xu, Dazhi He, Hanjiang Hong, Zhiyong Chen, Cixiao Zhang, Yiyan Wu, Wenjun Zhang
Information Theory
Duality and Reverse Self-Dual Constructions for Hyperderivative Reed-Solomon Codes
Hongchang Li, Zhihao Zhu, Weijun Fang et al. (+2 more)
Hyperderivative Reed-Solomon (HRS) codes form a class of maximum-distance-separable codes under the Niederreiter-Rosenbloom-Tsfasman metric and may be viewed as a derivative-evaluation extension of classical Reed-Solomon codes. For generalized Reed-Solomon codes, the Euclidean dual is again a generalized Reed-Solomon code. In this paper, we investigate the corresponding duality problem for HRS codes. Using a residue-theoretic argument, we derive an explicit component-wise representation for the Euclidean dual of an HRS code. The formula shows that, in general, the Euclidean dual is not an HRS code. Instead, it is blockwise upper-triangularly equivalent to a reverse-order HRS evaluation code, where the reversal occurs in the hyperderivative orders within each evaluation block. In particular, for full-domain HRS codes with low multiplicity, the triangular transformations reduce to diagonal scalings, and the Euclidean dual is obtained as the row reversal of an HRS code. Based on this reverse-order dual structure, we further study reverse self-dual HRS codes. We establish explicit criteria for reverse self-duality and construct several families from additive and multiplicative coset structures.
Read the full abstract ▾
All authors Hongchang Li, Zhihao Zhu, Weijun Fang, Jun Zhang, Sihuang Hu
Information Theory · Cross-listed from quant-ph
Latency-Constrained Encoded Quantum Teleportation with Punctured Codes
Mahmoud Saad Abouamer, Jakob Kaltoft Søndergaard and Petar Popovski
Quantum teleportation is a key protocol for transmitting
quantum information using
entanglement and classical communication. Its reliability is constrained by both the availability and fidelity of shared entangled pairs, which are affected by
stochastic generation and
memory decoherence. In this work, we focus on encoded teleportation, in which
quantum information is encoded using a
quantum error-correcting code and transmitted as a codeword. We evaluate reliability in terms of logical error probability, considering latency-constrained settings where
entanglement is accumulated over time and degrades while in
memory. We develop a unified framework that captures the interaction between
entanglement availability, decoherence, and coding decisions. Our results show that the benefits of longer codes depend on the availability and fidelity of entangled pairs, as acquiring additional resources introduces delays that can reduce their quality. To address this latency-reliability tradeoff, we leverage code puncturing to enable flexible encoded teleportation, allowing the effective code length to adapt across different latency regimes while preserving a common stabilizer structure. Numerical results show that encoded teleportation can provide substantial reliability gains over uncoded transmission under a common
entanglement-acquisition latency constraint, and that selecting appropriate punctured codes improves performance across varying latency budgets. Overall, our results highlight the importance of resource-aware adaptation for reliable
quantum networking.
Read the full abstract ▾
Information Theory · Cross-listed from cs.LG
Convergence-Latency-Aware Adaptive Modulation and Resource Allocation in RIS-Assisted Wireless Federated Learning
Liwei Wang, Wen Chen, Jun Li et al. (+4 more)
Federated learning (FL) over wireless networks suffers from significant training latency and degraded convergence due to unreliable wireless transmission, especially under blocked propagation environments. Although reconfigurable intelligent surfaces (RISs) can improve communication reliability, existing wireless FL studies rarely characterize the trade-off between learning convergence and communication delay under modulation-dependent transmission errors. In this paper, we consider a wireless FL system operating under RIS-assisted blocked-link propagation scenarios, and focus on adaptive modulation and sub-channel allocation for convergence-latency aware communication design. By characterizing the effect of symbol errors on uploaded local gradients, we derive a convergence-related upper bound that reveals the impact of symbol error rate (SER) on FL loss decay. Based on this result, we formulate a joint convergence-latency optimization problem, which is cast as a mixed-integer nonlinear programming (MINLP) problem, and solve it using a low-complexity hybrid alternating optimization framework. Extensive experiments on MNIST, CIFAR-10, and
Speech Commands show that the proposed scheme consistently achieves faster convergence and higher test accuracy than existing adaptive communication schemes, especially in complex tasks and challenging wireless scenarios.
Read the full abstract ▾
All authors Liwei Wang, Wen Chen, Jun Li, Qingqing Wu, Ming Ding, Xusheng Zhu, Qiong Wu
Information Theory
Optimization of MIMO STEEP for Secure Communications over MIMOME Channels
Dinglin Yu and Yingbo Hua
A transmission scheme for secure communications, called secret-message transmission by echoing encrypted probes (STEEP), can deliver a positive secrecy rate, even when eavesdropping channels are much stronger than that between users, subject to sufficient asymmetric power allocations from users. This paper considers an optimization problem for STEEP applied to
MIMO (or MIMOME) channels between multi-
antenna users and multi-
antenna Eve. We reveal for the first time a significant increase of the achievable secrecy rate of
MIMO STEEP using optimized precoders.
Read the full abstract ▾
Information Theory
Investigation of STEEP for Secure Communications Over SIMO and MISO Channels Subject to Full-Duplex Jamming and Eavesdropping
Md Saydur Rahman and Yingbo Hua
Secure communications over SIMO and MISO channels between wireless nodes are commonly encountered in applications and widely considered in the literature. This paper investigates such a problem by considering a newly proposed scheme called secrecy-message transmission by echoing encrypted probes (STEEP). We focus on a type of heavily eavesdropped channels where an aggressive eavesdropper (Eve) has multiple
antennas and is capable of jamming and eavesdropping in the full-duplex mode. Assuming optimal jamming and eavesdropping by Eve, we present the secrecy rates achievable by STEEP in two forms: using SIMO channel in phase 1 and using MISO channel in phase 2, and vice versa. With any given channel condition between every pair of nodes and any finite jamming power by Eve, the secrecy rate of STEEP in either form is shown to be positive subject to a positive power in phase 1 and a sufficiently large power in phase 2. The connections between STEEP in either form and the conventional SIMO and MISO schemes are provided, and their secrecy rates are compared comprehensively subject to random fading channels and finite transmit powers.
Read the full abstract ▾
Information Theory
Optimal Break-Resilient Codes
Canran Wang
Break-resilient codes enable reliable communication in the presence of an omniscient adversary that may split a transmitted message at arbitrary boundaries between consecutive symbols, while the receiver observes only an unordered multiset of the resulting fragments. For binary codewords of length~$n$ subject to at most~$t$ breaks, the best known explicit construction has redundancy~$O(t\log_2 n\log_2\log_2\log_2 n)$, whereas the information-theoretic lower bound is~$\Omega(t\log_2(n/t))$. In this paper, we extend the binary break model to any fixed finite field~$\bbF_q$ and establish a redundancy lower bound of~$\Omega(t\log_q(n/t))$. We then give an explicit construction of~$q$-ary break-resilient codes with redundancy~$O(t\log_q n)$ when~$t\leq n^{1-\varepsilon}$ for a fixed constant~$\varepsilon\in(0,1)$, matching the information-theoretic lower bound up to a constant factor. The key idea is to compute a short algebraic fingerprint of the message, which enables the decoder to reject incorrect assemblies of the received fragments.
Read the full abstract ▾