Unlike most other areas in the natural sciences, the roots of
Information Theory are truly multidisciplinary, originating in physics,
psychology, statistics and communications engineering. What, then, is
information theory and why has it influenced such a wide range of
discip1ines? To answer this question, we need a discussion of the
concept of "information," which is the subject matter of the theory
Information is the reduction of uncertainty. Notice that knowledge has
not exactly been removed from the picture, but it has been given a
highly restrictive meaning: reduction of uncertainty. Certainly
knowledge can make you more sure of things, but we would ordinarily
believe that knowledge is more than simply being less uncertain. We must
just accept the fact that all those nuances of knowledge that go beyond
mere assurance have not been quantified.
With the use of highly imaginative examples, Professor Tavel provides an
easily understood explanation of Information Theory.
The discussion in this text relies on basic probability as well as an
understanding of logarithms and binary numbers. It includes an application
of information theory to DNA sequencing.