Huffman coding’s efficiency collapses for small symbol sets because bits are integral

Evidence

Huffman coding is optimal in that it gets the code rate within one bit of the source entropy. However, because bits only come in integer multiples the efficiency for small symbol sets is quite low. In MPEG_1 layer 3 cod­ ers, some symbols are Huffman coded as pairs to allow a greater coding efficiency. However, there is a practical limit to the size of a Huffman code because the computation and the tables blow up exponentially in size and so become unfeasible. A more subtle limitation is that, for large symbol sets, it becomes very difficult to gather enough symbol statistics data with sufficient accuracy to generate a Huffman code. — David Howard, Jamie Angus, coustics and Psychoacoustics, 2640

Categorization

• XREF: Connects to the earlier chapter’s treatment of perceptual coding and bit-rate limits in MPEG layers.