Information Theory · Cross-listed from cs.RO
RoboShape: Information-Theoretic Point Cloud Representations for Privacy-Aware Robot Perception
Oguzhan Baser, Mirac Sozen, Kaan Kale et al. (+2 more)
With the increased adoption of robotic
agents operating in human environments by scanning and sharing 3D representations (e.g., for fleet learning, cloud-based planning, or collaborative mapping), collected point clouds reveal not just the objects in a scene but also sensitive spatial context, such as room function or information that occupants never consented to disclose. Traditional point cloud encoders offer no principled control over this: either all is preserved, or none. Hence, we introduce RoboShape, an information theory guided compression head following the frozen Sonata encoder. We project voxel-level
embeddings using the Donsker-Varadhan formulation of mutual information (MI). Specifically, we maximize the MI between
embeddings and object-level understanding while minimizing it for private attributes. RoboShape leads to 87.5% smaller
embeddings that retain 98.7% of object classification utility while collapsing sensitive attribute predictions by 39.3% across the three real-world indoor
LiDAR datasets. Its
privacy-preserving
embeddings are cheaper to transmit over the network or to train a model for any downstream tasks. We release the RoboShape codebase to give the robotics community a practical, encoder-agnostic tool for building perception pipelines that are compact,
privacy-aware, and deployment-ready.
Read the full abstract ▾
All authors Oguzhan Baser, Mirac Sozen, Kaan Kale, Sandeep Chinchali, Sriram Vishwanath
Information Theory · Cross-listed from math.CO
Sharp Bounds for Discrete Cube Skeleta
Dean Menezes
Fix integers $0\leq k<n$. Let $F_{n,k}(N)$ be the least size of a finite set $B\subset\Z^n$ that contains a filled axis-parallel cube $k$-skeleton centered at each point of some $N$-point set. We prove that $F_{n,k}(N)$ has order $N^{1-(n-k)/(2n^2)}$, with constants depending only on $n$ and $k$. Thornton proved the upper bound and lower bounds with every smaller exponent; the endpoint lower bound was open for $k\geq1$. For square boundaries in $\Z^2$, the exponent is $7/8$. A midpoint count and Shearer's inequality handle large radii; induction in
lattice cells handles small radii.
Read the full abstract ▾
Information Theory
Closed-Loop Bayesian Bandit Encoder with GRAND Receiver for a Bursty Interference Channel
Bhaskar Krishnamachari
Interleaving mitigates burst errors but introduces decoding delay and removes temporal error structure that a channel-aware decoder could exploit. We consider packet-level selection between a random linear code and the same code used with cross-codeword interleaving, over a channel with an unknown number of on/off interferers. The receiver uses Guessing Random Additive Noise Decoding (GRAND) with a replaceable noise model and feeds aggregate channel statistics back to a
Bayesian estimator at the transmitter. Once the interference amplitudes and timing parameters are estimated, the receiver's noise model is replaced: it computes hidden-Markov-model posterior bit-flip probabilities and uses them to order GRAND queries. A discounted Thompson sampler selects between the two transmission modes using a goodput-minus-latency reward whose distribution is endogenously nonstationary: receiver adaptation, rather than channel change, alters the value of each mode. Across five simulation seeds, the interleaved mode is preferred before channel estimation converges. After the learned decoder is activated, the non-interleaved mode becomes preferable because it achieves lower block error rate without interleaving delay. In the reference configuration, the learned noise model reduces block error rate by approximately one order of magnitude relative to ORBGRAND. Using partial channel estimates before full convergence reduces pre-convergence block error rate by up to $4.5\times$. Adding model-predicted utilities as confidence-weighted pseudo-observations reduces post-transition selection of the inferior arm by approximately $65\%$. Under an idealized airtime conversion at a 100~MHz 5G~NR-like symbol rate, the learning transient corresponds to a few milliseconds of occupied symbol time.
Read the full abstract ▾
Information Theory · Cross-listed from quant-ph
LDGM-Based Quantum Codes for Fault-Tolerant Quantum Computation
Yumin Li, Kejing Liu, Hanqing Lou et al. (+1 more)
We construct a new family of Calderbank-Shor-Steane (CSS) codes using the generator and parity-check
matrices of Low-Density Generator
Matrix (LDGM) codes, with row operations applied to both
matrices in order to achieve the desired
quantum rate. Decoding is performed in an iterative manner, by applying message passing over the associated graph, and discrete Density
Evolution (DDE) is used to optimize performance in the depolarizing channel. The proposed construction offers high flexibility and easiness in the design, producing
quantum codes that possess excellent error correction capabilities. By properly designing the structure of the code, we are able to control and bound the weight of the stabilizer generators to a small value, which results in codes particularly well suited for fault-tolerant
quantum computation. At the same time, these codes achieve very good performance in terms of error correction capability.
Read the full abstract ▾
All authors Yumin Li, Kejing Liu, Hanqing Lou, Javier Garcia-Frias
Information Theory · Cross-listed from quant-ph
Superadditivity for Entanglement-Assisted Communication
Hao-Chung Cheng and Mario Berta
The
entanglement-assisted capacity of a
quantum channel admits an additive single-letter characterization, implying that joint encodings across channel uses cannot increase the ultimate communication rate. Here, we show that this additive picture does not extend to communication reliability. Specifically, we prove that the Petz-Rényi channel information can be strictly superadditive for every $\alpha\in[1/2,1)$, yielding a genuine multi-copy enhancement of the
entanglement-assisted random-coding error exponent, even though the
entanglement-assisted capacity remains additive. We establish this phenomenon analytically already for measurement channels, which are
entanglement-breaking and have additive unassisted capacity. Remarkably, this strict superadditivity is witnessed by a separable, classically correlated two-copy channel-input marginal, demonstrating that no
entanglement between the transmitted systems is required. Our results show that, although correlations across channel uses cannot increase the ultimate rate of
entanglement-assisted communication, they can enhance its reliability.
Read the full abstract ▾
Information Theory · Cross-listed from eess.SP
Modular-CAPA-Based Communication Systems: Joint Activation and Beamforming Design
Mengyu Qian, Xidong Mu, Li You et al. (+1 more)
A modular continuous aperture array (CAPA)-based multi-user communication system is investigated, where only a portion of the aperture, namely sub-CAPAs, is activated to serve users. The signal model for the proposed modular CAPA is first introduced. Based on this model, a spectral efficiency (SE) maximization problem is formulated to jointly optimize the sub-CAPA activation and beamforming, subject to constraints on the limited number of active sub-CAPAs and the total transmit power. To address the resulting mixed-integer optimization problem, a branch-and-bound (B&B)-based
algorithm is first proposed for optimal sub-CAPA activation and beamforming design. After that, the spatial bandwidth of the modular CAPA under partial activation is analyzed. The analysis reveals that a modular CAPA with partial sub-CAPAs activated could achieve a maximum spatial bandwidth comparable to that of a conventional CAPA. Motivated by this insight, a low-complexity spatial bandwidth-aware sub-CAPA activation scheme is further proposed. Finally, numerical results demonstrate that i) modular CAPA architectures with partial activation can consistently achieve greater performance gains than adjacent CAPA activations; ii) the proposed B&B scheme outperforms all
benchmark schemes in terms of SE; and iii) the proposed spatial bandwidth-aware scheme provides an attractive performance-complexity tradeoff compared with the proposed B&B-based
algorithm.
Read the full abstract ▾
All authors Mengyu Qian, Xidong Mu, Li You, Michail Matthaiou
Information Theory · Cross-listed from cs.DS
A Correlation-Gap Bound for Nonlinear Gaussian PCA
Minbo Gao, Zhengfeng Ji and Chenghua Liu
Principal component analysis (PCA) is optimal for the linear reconstruction of Gaussian data, a foundational property underlying its central role in
algorithms and signal processing. Its nonlinear analogue, however, is notoriously subtle: in 2011, Mallat and Zeitouni conjectured that the Karhunen–Loève (KL) basis remains optimal even when the retained coordinates are chosen adaptively per sample, a property that would theoretically justify the ubiquitous pipeline of PCA followed by sparse thresholding. In this paper, we establish a $1+O(1/\sqrt{d})$-approximate version of the retained-energy form of the Mallat–Zeitouni
conjecture, showing that the KL basis is within this factor of the optimal basis. This dimension-free comparison depends only on the number of retained coordinates and shows that the possible advantage of optimizing over all orthonormal bases vanishes as $d$ grows. It complements the universal-constant reconstruction-error comparison of Litvak and Tikhomirov (Ann. Appl. Probab., 2018), while providing a comparison naturally suited for algorithmic analysis. Our proof rests on a clean, conceptual reduction: we relax arbitrary rotations to a deterministic threshold bound via Schur–Horn majorization, and identify the remaining loss with the correlation gap of the rank-$d$ uniform matroid over Gaussian level sets.
Read the full abstract ▾
Information Theory
Quantum XYZ Stabilizer Codes
Alessio Baldelli, Davide Orsucci, Francisco Lázaro et al. (+1 more)
Stabilizer codes are often constructed within the Calderbank–Shor–Steane (CSS) framework, where two mutually orthogonal binary classical codes define $X$ and $Z$-type stabilizer generators. While this structure is algebraically convenient, additional non-CSS constraints may help suppress low-weight logical operators and improve decoding performance in the finite-length regime. We thus introduce
quantum XYZ stabilizer codes, whose parity-check
matrix (PCM) is built from three pairwise orthogonal binary PCMs associated with $X$-, $Y$-, and $Z$-type stabilizer generators. A nontrivial point is that an XYZ code instance is not automatically genuinely non-CSS: the same stabilizer group may admit a CSS generating set. We characterize this collapse, obtaining algebraic and rank conditions for deciding when the $Y$-type checks are redundant and when they define genuinely non-CSS stabilizer constraints. We also derive upper and lower bounds on the
quantum minimum distance, including bounds for mixed Pauli logical operators. The novel framework includes a known non-CSS topological code, namely the XYZ$^2$ hexagonal code, and yields also sparse finite-length
quantum low-density parity-check (qLDPC) constructions from intersecting-subset and quasi-dyadic code families. Simulations under depolarizing code-capacity noise and quaternary belief propagation decoding show that the proposed XYZ qLDPC instances can outperform representative CSS qLDPC instances with similar finite-length parameters.
Read the full abstract ▾
All authors Alessio Baldelli, Davide Orsucci, Francisco Lázaro, Massimo Battaglioni
Information Theory · Cross-listed from stat.AP
Optimal Design for Generalized Progressive Hybrid Censored Data via Constrained, Unconstrained, Compound, and Minimax Optimization
Rathin Das, Tanmay Sen and Deepak Prajapati
This paper studies the optimal design of Type-I generalized progressive hybrid censoring schemes for life-testing experiments. The design problem involves simultaneously determining the inspection time, the guaranteed number of failures, and the progressive censoring scheme. First we develop a cost-constrained optimization framework for determining the optimal censoring scheme. Structural properties of the A-optimality criterion and the experimental cost with respect to the inspection time and the guaranteed number of failures are established. It reveals that they are conflicting behaviors which enables to develop an efficient search
algorithm that substantially reduces the computational burden. Building on these theoretical results, a multi-objective optimization model is proposed to simultaneously minimize A-optimality criterion and the experimental cost. A Variable Neighborhood Search (VNS)
algorithm is proposed to efficiently determine the optimal progressive removal vector by exploring the feasible design space while avoiding exhaustive enumeration. The resulting compromise designs simultaneously improve estimation precision and reduce experimental cost. In addition, the Shannon differential
entropy of the observed lifetime distribution is derived and employed as a complementary information-theoretic measure for evaluating the selected censoring schemes. Numerical studies show that
entropy-optimal designs generally differ from A-optimal designs, indicating that Shannon
entropy characterizes uncertainty in the observed data rather than estimation precision. The proposed methodology provides an efficient computational framework for optimal life-test design and offers a foundation for future multi-objective optimization incorporating statistical efficiency, experimental cost, and information-theoretic uncertainty.
Read the full abstract ▾
Information Theory
Covering Sequences and Covering-Sequences Codes
Tuvi Etzion
An $(n,R)_q$-covering sequence is a cyclic sequence, over the finite field $\F_q$, whose consecutive $n$-tuples form a code of length $n$ and covering radius $R$. An $(n,m,R)_q$-covering-sequences code is a set of cyclic sequences of length $m$, over $\F_q$, whose consecutive $n$-tuples form a code of length $n$ and covering radius $R$. These codes are the best building blocks for $(n,R)_q$-covering sequences. We show, for small radii, how cyclic codes and constacyclic codes with small covering radius, can be used to construct such sequences of short length and such codes with a relatively small number of sequences and a total number of codewords in the associated covering code. Sequences with small radius whose length approaches asymptotically to optimality are constructed, especially for an alphabet of
prime power size large enough. With the same construction, interesting codes are also constructed for larger radii.
Read the full abstract ▾
Information Theory
Lossy compression of weighted graph adjacency matrices by transform coding
Kenta Yanagiya, Junya Hara, Hiroshi Higashi et al. (+2 more)
In this paper, we propose a compression framework for weighted graphs in which the graph
topology is transmitted losslessly and edge weights are compressed lossily. A challenge in the lossy compression of edge weights is that the underlying relationships between edges are ambiguous. To address this issue, we first transform the unweighted graph into the corresponding line graph, whose nodes represent the edges of the original graph and whose edges encode the relationships between them. The line graph transform allows us to regard edge weights as a graph signal defined on the line graph. Instead of transmitting the edge-weight vector, we first transform it with a graph filter bank on the line graph. Then, quantization and
entropy coding are performed on the transformed coefficients of the edge weight vector. In addition to the lossy compression method, we formalize edge smoothness on the line graph and show that it serves as a measure of the difficulty of compression. The proposed smoothness measure can be easily calculated without converting to a line graph. This provides insight into the expected compression performance of a given weighted graph. Experiments on synthetic and real-world data validate the effectiveness of the proposed method by comparing it with existing
matrix preprocessing methods.
Read the full abstract ▾
All authors Kenta Yanagiya, Junya Hara, Hiroshi Higashi, Yuichi Tanaka, Antonio Ortega
Information Theory
Random Spherical Codes at High SNR: Error Transitions, Fixed-Error Data Rates, and Converse Gaps
Nikola Zlatanov
This paper characterizes random spherical codebooks over the real additive white Gaussian noise channel in the high signal-to-noise ratio (SNR) regime in which the blocklength is fixed, the SNR per real channel use tends to infinity, and the codebook size grows with SNR. The ensemble exhibits a sharp error-probability transition governed by the intrinsic dimension of the sphere and the codebook-growth order. Below the critical codebook-growth order, the ensemble-average error probability vanishes; at the critical order, it converges to a nontrivial limit; and above that order, it approaches one. For a fixed target average error probability, this transition yields the high-SNR expansion of the ensemble-achievable data rate. The achievable rate and the corresponding converse rate bound have the same high-SNR prelog, establishing first-order optimality within the deterministic equal-energy, average-error class. Their ratio tends to one for every fixed blocklength and target error probability, but their additive difference approaches a strictly positive limit that depends on both. We characterize this rate-bound gap jointly in blocklength and error probability. At fixed error probability, it vanishes with increasing blocklength, with a universal leading $1/n$ behavior and reliability dependence first appearing at the next order. For error probabilities that decrease exponentially with blocklength, we identify the threshold between vanishing and nonvanishing limiting gaps. We also derive blocklength laws for meeting a prescribed gap as the target error probability becomes more stringent.
Read the full abstract ▾
Information Theory
Statistics of the Compression Ratio of a Variable-to-Variable Code: Exact Moments and Asymptotic Behavior
Neri Merhav
A variable-to-variable (V2V) length code parses a source sequence into phrases of variable length and maps each phrase to a binary codeword of, generally, a different random length. After encoding $n$ phrases, the realized compression ratio $R_n=\Lambda_n/\Sigma_n$ – total codeword length over total source-symbol count – is the finite-sample counterpart of the code's asymptotic rate $\rho$, to which it converges only as $n\to\infty$. This paper first derives exact formulas for all integer moments of $R_n$ for a given discrete memoryless source (DMS). Specifically, we obtain a closed-form formula for every moment $\E\{R_n^k\}$ as a one-dimensional integral involving only single-phrase moment generating functions of the pair $(L,\ell)$ – the phrase length, in source symbols, and codeword length, in bits. From these moments we derive an Edgeworth approximation to the cumulative distribution function (CDF) of $R_n$ that is substantially more accurate than the central limit
theorem (CLT) approximation. Using the Laplace method of integration, we also derive explicit closed-form formulas for the bias constant $C=\lim_{n\to\infty}n(\E\{R_n\}-\rho)$ and for the variance constant $\lim_{n\to\infty}n\cdot\Var\{R_n\}$. The analysis extends to Markov sources via state-indexed
matrices with a redundancy formula obtained in closed form. On the coding-theoretic side, we cast V2V length codes as finite-state encoders and apply a generalized Kraft inequality for a compression-rate lower bound, and give a structural decomposition of the bias coefficient that separates cleanly across variable-to-fixed (V2F) length codes, fixed-to-variable (F2V) length codes, and V2V length codes. Applied to the Khodak code of Bugeaud, Drmota, and Szpankowski, this decomposition shows that its improved performance is reflected in its smaller bias constant.
Read the full abstract ▾
Information Theory
Identification Codes and Post-Shannon Communication: Theory, Architectures, and Emerging Applications
Wafa Labidi, Kumar Nilesh, Johannes Rosenberger et al. (+5 more)
Identification (ID) coding, introduced by Ahlswede and Dueck, extends Shannon's classical communication paradigm by replacing message reconstruction with hypothesis testing. Instead of decoding the transmitted message, the receiver only decides whether a particular message was sent. A fundamental result of ID theory is the double-exponential growth in the number of identifiable messages with respect to (w.r.t.) the blocklength. This scaling behavior enables fundamentally new communication architectures for large-scale distributed systems and forms a key building block of post-Shannon communication. While ID cannot replace classical communication in general, it is particularly well-suited for scenarios in which full message reconstruction is unnecessary, such as monitoring, alarming, and control systems. In this survey, we review the theoretical foundations of ID coding and discuss emerging communication architectures and application domains based on this paradigm. Particular emphasis is placed on practical use cases, including monitoring systems, special-purpose data storage, joint identification and sensing (JIDAS), semantic communications, mobile-network control systems and networked consensus testing systems. We further highlight recent system concepts, industrial perspectives, and implementation examples that illustrate how ID-based principles can be realized in practical communication systems.
Read the full abstract ▾
All authors Wafa Labidi, Kumar Nilesh, Johannes Rosenberger, Juan Cabrera, Holger Boche, Christian Deppe, Frank H.P. Fitzek, Marc Geitz
Information Theory
Quasi-Belief Propagation and Neural-Network Check Node Processing for BCH Codes
Guangwen Li
This paper proposes a quasi-BP decoding scheme for BCH codes that preserves the parallelizable structure of belief propagation while exploiting code automorphisms and optimized redundant parity-check
matrices. To eliminate the computationally expensive $\tanh$ and $\tanh^{-1}$ functions in check node updates, we further introduce a neural-network-based variant that replaces them with a lightweight convolutional neural network trained under a triple-constraint loss function enforcing non-negativity and order consistency. Simulation results for three BCH codes demonstrate that quasi-BP decoding achieves competitive frame error rate performance, with a gap within 0.25 decibels compared with belief propagation decoding of an LDPC code of similar blocklength. The neural-network-based variant incurs negligible performance loss while enabling stable deployment with arithmetic operations on hardware accelerators. Concatenation with an ordered statistics decoding variant further bridges the gap to the maximum-likelihood bound. Hence, the proposed schemes offer a viable path toward high-throughput, low-latency decoding of BCH codes in next-generation communication systems.
Read the full abstract ▾
Information Theory · Cross-listed from cs.CC
Space-Entropy Lower Bounds for Random Sampling
Thomas L. Draper and Feras A. Saad
We prove fundamental space lower bounds for exact random sampling using an
entropy source of i.i.d. uniform bits. A classic result from information theory shows that generating $n$ discrete random variables $X_1, \dots, X_n$ requires at least $H(X_1, \dots, X_n)$ input random bits on average, where $H$ is the Shannon
entropy function. How much space must a random sampling
algorithm use in order to approach this information-theoretically optimal
entropy bound? We prove that any random sampling
algorithm that is exact for arbitrary discrete target distributions and consumes at most $H(X_1,\ldots,X_n)+\varepsilon n+o(n)$ input bits in expectation for every output process must use $\Omega(\log(1/\varepsilon))$ bits of space. In fact, i.i.d. sampling from the single distribution $\mathrm{Bernoulli}(1/3)$ already forces at least $(1/{5.116201}-o(1))\log(1/\varepsilon)$ bits of space. If the sampler handles a family of infinitely many Bernoulli distributions, we show a sharper bound of at least $\log(1/\varepsilon)$ bits of space. We also prove lower bounds for general i.i.d. sampling: for almost every distribution on $k$ outcomes, the space is at least $(1/(k+1)-o(1))\log(1/\varepsilon)$ bits. The proof technique is based on a graph-theoretic analysis of the amount of information that any
algorithm can store in its state. Finite state spaces force short cycles around the state-transition graph, and the loss around such cycles reduces to Diophantine lower bounds on fractional parts of integer combinations of log-probabilities. To the best of our knowledge, these results comprise the first known space lower bounds for
entropy-efficient random sampling.
Read the full abstract ▾
Information Theory · Cross-listed from quant-ph
Perturbation Analysis of Maximal Quantum Leakage
Zijia Zhao, Shuixin Xiao and Farhad Farokhi
Maximal
quantum leakage (MQL) is a worst-case information leakage measure that quantifies an adversary's inference advantage gained from accessing
quantum encoding of classical data with arbitrary measurements. While MQL admits an exact characterization for a given ensemble of
quantum states, its robustness to implementation imperfections has not been systematically studied. In this paper, we analyze the sensitivity of maximal
quantum leakage under perturbations of the
quantum encoding. We establish a continuity bound in terms of the trace distance between ideal and perturbed
quantum states, and show, via an example, that this bound is attainable. We further derive fidelity-based and relative-
entropy-based sufficient conditions for bounding the variation of maximal
quantum leakage, and illustrate numerically that these conditions can be loose.
Read the full abstract ▾