Capacity of Random Channels with Large Alphabets

Abstract

We consider discrete memoryless channels with input alphabet size $n$ and output alphabet size $m$, where $m=$ceil$(γn)$ for some constant $γ>0$. The channel transition matrix consists of entries that, before being normalised, are independent and identically distributed nonnegative random variables $V$ and such that $E[(V \log V)^2]<\infty$. We prove that in the limit as $n\to \infty$ the capacity of such a channel converges to $Ent(V) / E[V]$ almost surely and in $L^2$, where $Ent(V):= E[V\log V]-E[V] \log E[V]$ denotes the entropy of $V$. We further show that, under slightly different model assumptions, the capacity of these random channels converges to this asymptotic value exponentially in $n$. Finally, we present an application in the context of Bayesian optimal experiment design.