A source sending one symbol repeatedly conveys zero information despite infinite potential self-information.

Evidence

As an extreme case, consider our person asleep on the keyboard who is sending the same symbol all the time. In this case, the probability of the symbol being sent is 1, and all other symbols have 0 probability of being sent. The total information being sent by this source is 0 bits. How can this be? Surely, if the probability is 0, then the self-information of these symbols is infinite? In theory, this might be true, but as these symbols are never sent, the total information is 0. — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2623

Categorization

Category: Contrarian or Surprising Claims

• CANDY: The sleeping keyboardist sending one key forever transmitting exactly zero bits is a vivid, elevator-friendly paradox.
• XREF: Connects to the broader entropy/information theme in Shannon information theory and its physical analog in thermodynamics.

Related

• A source repeating one symbol conveys zero total information (similarity 0.93)