Information

From NECSIWiki

Information, in some sense, is what we know or communicate.

One of the major advances in the understanding of information occurred in the 1950s with the development of a mathematical theory of communication by Claude Shannon. It is a natural, simple and useful theory. The original article may still be the best description and it is also available as a book.

The 1950s was a time of rapid growth of the telephone and telegraph communication networks. These systems communicate messages through electric wires. Shannon was motivated by the problem of evaluating and comparing the efficiency of various ways of sending signals.

The basis of Shannon's treatment is a distinction between information and meaning. Shannon assumed that for the purpose of designing a communication system, it is not necessary to know the meaning of a message, or what is the message that will be sent. It is only necessary to know what is the set of possible messages. Each message that is sent is one of these possible messages. The designer of the system must design it to allow any of the messages that might be sent. This is precisely because he/she does not know which message will be sent.

For this reason the central issue is simply a count of the number of possible messages. The number of messages that can be sent in a certain amount of time is the main characteristic of the system. Two systems that can send the same number of possible messages can carry the same amount of information. If one of them takes longer to send this same number of messages, then that is how much longer it will have to work to send the same amount of information. The proof results from mapping each of the messages of one system onto each of the messages of the other system.

This definition of information, obtained from counting the number of possible messages, can be related to the length of a text of English, or the length of a string of morse code, or the length of a binary string of characters. The amount of information grows with the length of text, or the number of morse code characters, or the length of a binary string. As long as the sender and recipient know how to translate from one to the other in a unique and mutually agreed upon way, then each of these can carry the same messages, the same information.

[edit] Additional topics

• Analyzing a simple case
• Analyzing the case of English
• Complexity and information
• Complexity in common things

==Related concepts== meaning, complexity