ECE 8526: Information Theory
Syllabus
Contact Information:

Lecture	ENGR 820 (Small BioE Conference Room)     W: 1:00 - 3:00 PM
Lecturer	Joseph Picone, Professor     Office: ENGR 703A     Office Hours: (MWF) 8 AM - 11 AM     Phone: 215-204-4841     Email: picone@temple.edu     Skype: joseph.picone
Social Media	https://www.facebook.com/groups/temple.engineering.ece8527/     temple.engineering.ece8527@groups.facebook.com
Email	http://groups.google.com/group/temple-engineering-ece8526     temple-engineering-ece8526@googlegroups.com
Website	http://www.isip.piconepress.edu/courses/temple/ece_8526
Required Textbook	T.M. Cover and J.A. Thomas     Elements of Information Theory     Wiley Interscience, ISBN-13: 978-0-471-24195-9, 2006.   (Available online through the Temple Library here.)
Reference Textbooks	D.J.C. MacKay     Information Theory, Inference and Learning Algorithms     Cambridge University Press, ISBN: 978-0521642989, 2004.
Prerequisites	ENGR 5022 (minimum grade: B-)   ENGR 5033 (minimum grade: B-)

Grading Policies:

Item	Weight
Exam No. 1	15%
Exam No. 2	15%
Exam No. 3	15%
Final Exam	15%
Computer Homework	40%
TOTAL:	100%

Information Theory is a field that has been central to the development of modern communications and computing technologies. The goal of this course is to provide the student with a thorough understanding of the concepts of entropy and information, and how to apply these to real world problems such as speech recognition, language engineering, signal compression, and financial modeling. A secondary goal is to develop a mathematically rigorous understanding of methods for measuring and manipulating various measures of information in signals and systems.

The course requirements include three in-class exams and a final exam. In addition, students will be expected to complete a course project that involves operating on real non-stationary data source, and estimating the short-term and long-term statistical properties of the data. The specifics of this assignment will be negotiated in writing with the course instructor, and fully defined by the fifth week of the course.

Lecture Schedule:

The lecture component will cover the following topics:

Class	Date	Topic(s)
01	01/18	Chapter 1: Introduction and Preview
02	01/25	Chapter 2: Entropy, Relative Entropy and Mutual Information
3	02/01	Chapter 4: Entropy Rates of a Stochastic Process
4	02/08	Chapter 5: Data Compression
5	02/15	Chapter 6: Gambling and Data Compression     Exam No. 1 (Chapters 1-4)
6	02/22	Chapter 7: Channel Capacity
7	03/01	Chapter 8: Differential Entropy
8	03/08	Chapter 9: Gaussian Channel
--	03/15	Spring Break
9	03/22	Chapter 10: Rate Distortion Theory
10	03/29	Chapter 10: Rate Distortion Theory (Problems)     Exam No. 2 (Chapters 5-9)
11	04/05	Chapter 11: Information Theory and Statistics
12	04/12	Chapter 12: Maximum Entropy     Chapter 13: Universal Source Coding
13	04/19	Chapter 14: Kolmogorov Complexity
14	04/26	Chapter 15: Network Information Theory     Exam No. 3 (Chapters 10-14)
15	05/10	Final Exam

Please note that the dates above are fixed since they have been arranged to optimize a number of constraints. You need to adjust your schedules, including job interviews and site visits, accordingly.

The homework schedule is as follows:

Number	Date	Chapter: Problem(s)
01	01/25	Chapter 2: 1, 4, 8, 11, 12, 18
02	02/01	Chapter 4: 3, 7, 12, 23, 25
03	02/08	Chapter 5: 4, 10, 12, 17, 30
04	02/15	Chapter 6: 1, 2, 3, 5, 15
05	02/22	Chapter 7: 1, 2, 7, 12, 18
06	03/01	Chapter 8: 1, 3, 5, 8, 9
07	03/08	Chapter 9: 1, 3, 5, 7, 11
08	03/22	Chapter 10: 1, 5, 8, 13, 17
09	03/29	Chapter 11: 1, 2, 6
10	04/05	Chapter 12: 1, 6, 7
11	04/12	Chapter 13: 2, 3, 5
12	04/19	Chapter 14: 1, 2, 3
13	04/26	Chapter 15: 1, 2, 12