Course Description:

Information Theory is a field that has been central to the development of modern communications and computing technologies. The goal of this course is to provide the student with a thorough understanding of the concepts of entropy and information, and how to apply these to real world problems such as speech recognition, language engineering, signal compression, and financial modeling. A secondary goal is to develop a mathematically rigorous understanding of methods for measuring and manipulating various measures of information in signals and systems.

Course Topics:

• Introduction and Preview
• Entropy, Relative Entropy and Mutual Information
• Entropy Rates of a Stochastic Process
• Data Compression
• Gambling and Data Compression
• Channel Capacity
• Differential Entropy
• Gaussian Channel
• Rate Distortion Theory
• Information Theory and Statistics
• Maximum Entropy
• Universal Source Coding
• Network Information Theory
• Information Theory and Portfolio Theory

Questions or comments about the material presented here can be directed to picone@temple.edu.