A class of generalized Shannon-McMillan theorems for arbitrary discrete information source

In this study, a class of strong limit theorems for the relative entropy densities of random sum of arbitrary information source are discussed by constructing the joint distribution and nonnegative super martingales. As corollaries, some Shannon-McMillan theorems for arbitrary information source, mth-order Markov information source and non-memory information source are obtained and some results for the discrete information source which have been obtained by authors are extended.

Anahtar Kelimeler:

Shannon-McMillan theorem, the consistent distribution, arbitrary information source, relative entropy density, m-order Markov information source, non-memory information source

A class of generalized Shannon-McMillan theorems for arbitrary discrete information source

Keywords:

Shannon-McMillan theorem, the consistent distribution, arbitrary information source, relative entropy density, m-order Markov information source, non-memory information source,

PDF

___

[1] Algoet, P.H. and Cover, T.M.: A sandwich proof of Shannon-Mcmillan theorem. Ann.Probab. 16, 899-909 (1988).
[2] Barron, A.R.: The strong ergodic theorem for densities:Generalized Shannon-McMillan-Breiman theorem. Ann.Probab. 13,1292-1303 (1985)
[3] Breiman, L.: the individual ergodic theorem of information theory. Ann.Math.Statist. 28, 809-811 (1957)
[4] Chung K.L.: A Course in Probability Theory. Academic Press, New York. (1974)
[5] Liu, W. and Yang, W.G.: An extension of Shannon-Mcmillan theorem and some limit properties for nonhomogeneous Markov chains. Stochastic. Process. Appl. 61,129-145 (1996)
[6] Mcmillan, B.: The Basic Theorem of information theory. Ann.Math.Statist. 24,196-216 (1953)
[7] Shannon, C.: A mathematical theory of communication. Bell Syst. Tech. J. 27,379-423 (1948)
[8] Yang W.G. and Liu W.: Strong law of large numbers and Shannon-McMillan theorem for Markov fields on trees. IEEE Trans.Inform.Theory. 48 ,313-318. (2002)
[9] Yang W.G., Ye Z.X. and Liu W.: A local convergence theorem for partial sums of stochastic adapt sequences. Czechoslovak Mathematical Journal, 2, 86-91 (2006)