Entropy coding trade-offs limit how close code rates approach information-theoretic limits

Evidence

In order to have longer symbol lengths, which allow greater coding efficiencies, other approaches are used. Golomb-Rice codes use a predetermined statistical distribution to remove the need for tables, thus allowing longer code words; they are used in some lossless encoding schemes. Another approach is arithmetic coding, which does not need predetermined distributions, but, by using the statistics of the symbols occurrence, can encode very long symbols and so approach the source entropy much more closely. — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2644 (#exemplifies, via linked claim)

Although quantizers very often have a number of levels, which are powers of 2, such as 16, 256 or 65,536, because this makes best use of a binary word, there is no reason that other numbers of levels cannot be used, especially if entropy coding is going to be employed. — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2656 (#corroborates, via linked claim)

Categorization

• XREF: connects to: A data source’s entropy gives the minimum bits needed to encode it — David Howard, Jamie Angus, coustics and Psychoacoustics, 2625; Quantizer levels need not be powers of two when entropy coding is used — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2656; Arithmetic coding approaches source entropy better than table-based Huffman methods — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2644 (+1 more)

Broader Connections

• Arithmetic coding approaches source entropy better than table-based Huffman methods #exemplifies (0.80) — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2644
• Codec effectiveness trades off against bits spent on side information #expands (0.76)
• Quantizer levels need not be powers of two when entropy coding is used #corroborates (0.75) — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2656