Statistical significance marks deviations expected by chance less than once in twenty.

Evidence

If two groups of test subjects do not differ systematically, the examined characteristics of the individual subjects (for example, the duration of an illness after the start of treatment) will nevertheless differ to some extent due to chance. As a result, the calculated mean values in the two groups may also differ somewhat. However, larger deviations of the mean values will be less likely than small ones. These probabilities can be calculated. A deviation in the collected data that is so large that, if there is no true effect, it should occur, purely because of chance, less frequently than in every 20th case, is called significant. In this case, one often assumes that the variation is probably not merely random, that is, that the treatment is actually better in one group than in the other. — Holm Gero Hümmler, Ulrike Schiesser, Fact and Prejudice, 789

Categorization

• DEFINE: A clear plain-language definition of statistical significance useful for explaining it to lay audiences.