Information content relates to the number of bits required to transmit it

Evidence

In order to measure the information content of a signal, we need to know how likely it is to occur. However, we also want to be able to relate the information content to something real, such as the number of bits necessary to transmit that information. So how could we measure information content in such a way that it is related to the number of bits needed to transmit it? — David M. Howard, Jamie Angus, Acoustics and Psychoacoustics, 2611

Categorization

• QUESTION: The setup of Shannon information theory — worth chasing the full derivation of the entropy equation in the following paragraphs.