Search arXiv⌕ Search

arXiv · 2611.09842

Asymptotic Analysis of Data Deduplication Under a Constant Edit Model

Abstract

Data deduplication has become increasingly important in large-scale storage systems, driven by the rapid growth of digital data. By identifying and removing repeats in the data stream, deduplication saves storage space more efficiently than traditional compression methods. In this paper, we study the performance of variable-length deduplication and multi-chunk deduplication algorithms from an information-theoretic perspective, focusing on data streams where repeats are approximate rather than exact. We consider two probabilistic source models in which source strings consist of repeated blocks that are modified by either a constant number of substitution edits or burst-insertion edits per block, the latter capturing the localized edit patterns commonly observed in real-world data. For both models, we show that variable-length deduplication achieves asymptotically arbitrarily large compression ratios, and that multi-chunk deduplication achieves compression within a constant factor of the source entropy under mild conditions.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Sarvin Motamen, Hao Lou, Farzad Farnoud. 2026-11-06. Asymptotic Analysis of Data Deduplication Under a Constant Edit Model. https://arxiv.org/abs/2611.09842

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

A Two-Term Ziv-Zakai Bound for Multitarget DoA Estimation in MIMO Radar

Multitarget direction of arrival (DoA) estimation is challenging when fluctuating targets are closely spaced: local Cramér--Rao bounds (CRBs) miss global errors, while direct vector Ziv--Zakai bound (ZZB) evaluation is computationally demanding. For colocated multiple-input multiple-output (MIMO) radar with Swerling-II fluctuations and an ordered angular prior, we derive a rigorous two-term Bayesian mean squared error lower bound. A finite separation contribution addresses nonlocal discrimination, while a correction based on the prior averaged Fisher information matrix (FIM) retains target coupling and virtual array geometry. We show that the prior averaged pointwise CRB diverges under unrestricted target spacing when the FIM is nonsingular almost everywhere, although the proposed bound remains finite. Once the averaged FIM is obtained, evaluation avoids displacement optimization and multidimensional binary error integration. The bound supports analysis of snapshot diversity, target spacing, and transmit and receive allocation. Numerical comparison with the vector ZZB shows close agreement at low SNR but greater conservatism at high SNR, quantifying the tradeoff in tightness.

cs.IT↗

Graphical Analysis of Lifted Product Code Constructions

Lifted product codes are an important family of quantum low-density parity-check (QLDPC) codes, as they were the first QLDPC code family shown to be asymptotically good. Understanding the structure of their parity-check matrices $H_{\mathsf{X}}$ and $H_{\mathsf{Z}}$, as well as the associated Tanner graphs, is essential for analyzing their decoding behavior and error-floor performance. In this work, we show that the Tanner graphs of $H_{\mathsf{X}}$ and $H_{\mathsf{Z}}$ are indeed isomorphic, and investigate their graph-theoretical structure. We establish conditions ensuring the connectivity of these graphs and provide bounds on their minimal absorbing sets, providing new insight into the combinatorial structures influencing decoding performance.

cs.IT↗

The Gaussian Multiuser Channel Capacity Region

This work characterizes a Gaussian multiuser-channel capacity-region using receiver-subset message atoms, Gaussian single-user equivalents, and successive decoding. A memoryless multiuser-channel model specifies both the channel conditional distribution and the independent-use input distribution. For all linear Gaussian channels, independent Gaussian user signaling and receiver-local worst-case-noise/MMSE-GDFE transformations provide a single-user-equivalent basis for the complete capacity-region construction. Each user distributes its input autocorrelation among a finite set of receiver-subset atoms, whose rate sums satisfy the applicable receiver constraints. Minimum-energy and weighted-rate-sum designs optimize these allocations, while fixed-allocation linear programs determine feasible atomic rates and synchronized time-shared successive-decoding implementations. The resulting framework includes Gaussian multiple-access, broadcast, and interference channels, with numerical examples illustrating their rate regions and receiver designs.

cs.IT↗