This book is intended to be accessible to undergraduate students with two
years of typical mathematics experience, most likely meaning calculus with a little
linear algebra and differential equations. Thus, specifically, there is no assumption
oLa background inabstr&;t algebra or number theory, nor of probability, nor of
linear algebra. All these things are introduced and developed to a degree sufficient
to address the issues at hand.
We will address the fundamental problem of transmitting information effectively
and accurately. The specific mode of transmission does not really play
a role in our discussion. On the other hand, we should mention that the importance
of the iSsues of efficiency and accuracy has increased largely due to the advent of
the internet and, even more so, due to the rapid development of wireless communications.
For this reason it makes sense to think of networked computers or wireless
devices as archetypical fundamental practical. examples.
The underlying concepts of information and information content of data
make sense independently of computers, and are relevant in looking at the operation
of natural languages such as English, and of other modes of operation by which
people acquire and process data.
The issue of efficiency is the obvious one: transmitting information costs time,
money, and bandwidth. It is important to use as little as possible of each of these
resources. Data compression is one way to pursue this efficiency. Some well
known examples of compression schemes are commonly used for graphics: GIFs,
JPEGs, and more recently PNGs. These clever file format schemes are enormously
more efficient in terms of filesize than straightforward bitmap descriptions of graphics
files; There are also general-purpose compression schemes, such as gzip, bzip2,
ZIP, etc.
The issue of accuracy is addressed by detection and correction of errors
that occur during transmission or storage of data. The single most important
practical example is the TCP lIP protocol, widely used on the internet: one basic
aspect of this is that if any of the packets composing a message is discovered to be
mangled or lost, the packet is simply retransmitted. The detection of lost packets
is based on numbering the collection making up a given message: The detection
of mangled packets is by use of 16-bit checksums in the headers of IP and TCP
packets. We will not worry about the technical details of TCP lIP here,· but only
note that email and many other types of internet traffic depend upon this protocol,
which makes essential use of rudimentary error-detection devices.