A source repeating one symbol conveys zero total information

Evidence

As an extreme case consider our person asleep on the keyboard who is sending the same symbol all the time. In this case the probability of the symbol being sent is one, and all other symbols have zero probability of being sent. The total information being sent by this source is zero bits. How can this be? Surely, if the probability is zero, then the selfinformation of these symbols is infinite? In theory this might be true but, as these symbols are never sent, the total information is zero. — David Howard, Jamie Angus, coustics and Psychoacoustics, 2620

In the previous section, we had the case of one symbol in the whole set of possibilities being very small, and as a consequence, the other seven symbols had to have a higher probability, because the probability of all symbols being used has to be 1. As an extreme case, consider our person asleep on the keyboard who is sending the same symbol all the time. In this case, the probability of the symbol being sent is 1, and all other symbols have 0 probability of being sent. The total information being sent by this source is 0 bits. How can this be? Surely, if the probability is 0, then the self-information of these symbols is infinite? In theory, this might be true, but as these symbols are never sent, the total information is 0. — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2623

Categorization

• DEFINE: Clarifies the distinction between per-symbol self-information and total source information — a core information theory concept.
• XREF: Connects to Shannon information theory and entropy concepts; the ‘sleeping typist’ illustration parallels probability-mass pictures in other signal-processing texts. — connects to: Status is zero-sum and cannot rise for everyone at once — W. David Marx, Status and Culture: How Our Desire for…, 73
• corroborated: 2 independent passages support this claim — David Howard, Jamie Angus, coustics and Psychoacoustics, 2620; David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2623