Information is surprisal

Surprises inform. Non-surprises do not. Predictable events do not yield much information via their realization. Highly likely events do not yield much information via their realization, as little uncertainty is resolved via their realization.


References

eonline