SYLLABUS
Contact Information:

Lecture Time	MWF: 9 - 9:50 AM
Lecture Place	106 Simrall
Lecturer	Joseph Picone, Professor
Lecturer Office	228 Simrall
Lecturer Office Hours	available after class or any other times by appointment
Lecturer Phone	662-312-4209
Lecturer Email	joseph.picone@gmail.com
Lecturer Instant Messaging	Google Talk: joseph.picone@gmail.com
Class Alias	ece8443@ece.msstate.edu
URL	http://www.isip.piconepress.edu/publications/courses/ece_8443
Textbook (Reference)	R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, Second Edition, Wiley Interscience, ISBN: 0-471-05669-3, 2000 (supporting material available at http://rii.ricoh.com/~stork/DHS.html)
Suggested Reference	D. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003 (also available at http://www.inference.phy.cam.ac.uk/mackay/itprnn/book.html)
Prerequisite	Statistics, Signal Processing, many years of research, and attendance at a lecture where the speaker proudly exclaimed "PCA Fails!"
Other Reference Materials	Pattern Recognition in Java: an applet designed to demonstrate fundamental concepts of pattern recognition. Internet Links: Godfried T. Toussaint's online resources that include lots of valuable links and course notes. Human Computer Interface Design: a multi-university course sponsored by the National Science Foundation.

Grading Policies:

Item:	Date:	Weight:
Mid-Term Exam	02/25/2009	50%
Final Exam	04/27/2009	50%
Special Projects	Unspecified	10% per project

Exams:

We will have two in-class exams in this course (a mid-term and a final). Each will be closed books and notes. You will be allowed one page (double-sided) of notes. Calculators or other computing devices will not be allowed (or needed). The exams will require critical thinking. They will consist of one to two problems for which you will supply extended written solutions. The quality and clarity of your solution will be as important as whether you obtained the correct answer.

In addition, students can select and execute special projects on topics that are of interest to the class or two their research. These will be negotiated on an individual basis with the instructor, and must be agreed to in writing before the work can proceed. Typically in this class interesting issues arise that merit extended analysis. Students can earn extra credit by exploring these topics. Projects commonly involve a theoretical derivation, a computer simulation and a four-page paper summarizing the findings.

Attendance Policy:

Attendance is encouraged of course, but not formally count towards your grade. Most students tend not to do well in this class without frequent interaction with the instructor and the other students on the material.

Schedule:

Class	Date	Sections	Slides	Audio	Topic
01	Wednesday, January 07, 2009	01.01 - 01.04	01.01 - 01.11	01	Course Introduction
02	Friday, January 09, 2009	01.05 - 02.02	01.12 - 02.05	02	Introduction, Probability Decision Theory
03	Monday, January 12, 2009	02.03	02.06 - 02.22	02b	Bayes Decision Theory
04	Wednesday, January 14, 2009	02.04	03.01 - 03.15	03	Multivariate Gaussian Densities
05	Friday, January 16, 2009	02.05, 02.06	03.16 - 04.07	04	Transformations, Threshold Decoding
N/A	Monday, January 19, 2009	N/A	N/A	N/A	Holiday: Martin Luther King Day
06	Wednesday, January 21, 2009	02.07 - 02.12	04.08 - 05.02	04b	Error Bounds
07	Friday, January 23, 2009	03.01 - 03.02	05.03 - 05.11	05	Maximum Likelihoood Estimation
08	Monday, January 26, 2009	03.02	05.12 - 06.10	06	Variance Relationships
09	Wednesday, January 28, 2009	03.03	06.11 - 06.19	06b	Bayesian Estimation
10	Friday, January 30, 2009	03.04	07.01 - 07.11	07	Multivariate Gaussian Case
11	Monday, Feburary 02, 2009	03.05 - 03.07	07.13 - 08.04	08	Sufficient Statistics, Dimensionality
12	Wednesday, February 04, 2009	03.08	08.05 - 08.10	08b	Component Analysis
13	Friday, February 06, 2009	03.08	09.01 - 09.09	09	Discriminant Analysis
14	Monday, February 09, 2009	10.13.3	10.01 - 10.08	10	Heteroscedastic LDA and ICA
15	Wednesday, Feburary 11, 2009	03.09	11.01 - 11.12	11	The Expectation Maximization Algorithm
16	Friday, February 13, 2009	03.10	12.01 - 12.09	12	Hidden Markov Models - The Basics
17	Monday, February 16, 2009	03.10	12.10 - 13.04	13	Parameter Estimation in Hidden Markov Models
18	Wednesday, Feburary 18, 2009	03.10	13.04 - 13.08	13b	Continuous Distribution Hidden Markov Models
19	Friday, February 20, 2009	03.10	13.08 - 13.08	13c	Application of HMMs
20	Monday, February 23, 2009	04.01 - 04.03	14.01 - 14.08	14	Parzen Window
21	Wednesday, February 25, 2009	01.01 - 03.10	Mid-Term	N/A	Mid-Term
22	Friday, Feburary 27, 2009	04.04 - 04.07	14.09 - 16.02	16	Nearest-Neighbor Classification, Risk Minimization
23	Monday, March 02, 2009	05.01 - 05.03	16.03 - 16.11	16b	Support Vector Machines
24	Wednesday, March 04, 2009	05.04	17.01 - 17.06	17	Neural Networks
25	Friday, March 06, 2009	06.01 - 07.06	17.07 - 17.14	17b	Backpropagation
26	Monday, March 09, 2009	08.01 - 08.07	18.01 - 18.13	18	Decision Trees
27	Wednesday, March 11, 2009	09.01 - 09.02	19.01 - 19.06	19	Foundations of Machine Learning
28	Friday, March 13, 2009	09.03 - 09.04	19.07 - 19.13	19b	Bias, Variance and Bootstrapping
N/A	Monday, March 16, 2009	N/A	N/A	N/A	Spring Break
N/A	Wednesday, March 18, 2009	N/A	N/A	N/A	Spring Break
N/A	Friday, March 20, 2009	N/A	N/A	N/A	Spring Break
29	Monday, March 23, 2009	09.05	20.01 - 20.08	20	Estimating and Comparing Classifiers
30	Wednesday, March 25, 2009	09.06	20.09 - 21.07	21	Combining Classifiers, Reinforcement Learning
31	Friday, March 27, 2009	09.07	21.08 - 21.18	21b	Markov Decision Processes
32	Monday, March 30, 2009	10.01, 10.02	22.01 - 22.06	22	Mixture Densities
33	Wednesday, April 01, 2009	10.04, 10.05	22.07 - 23.04	23	Unsupervised Bayesian Learning
34	Friday, April 03, 2009	10.05, 10.06	23.05 - 23.14	23b	Hierarchical Clustering
35	Monday, April 06, 2009	10.07 - 10.12	24.01 - 24.06	24	On-Line Clustering
36	Wednesday, April 08, 2009	10.13, 10.14	24.07 - 25.05	24b	Discriminative Training
N/A	Friday, April 10, 2009	N/A	N/A	N/A	Holiday: Good Friday
37	Monday, April 13, 2009	N/A	25.06 - 26.05	25	Particle Filters
38	Wednesday, April 15, 2009	N/A	26.06 - 26.16	26	Monte Carlo Methods, Posterior Estimation
39	Friday, April 17, 2009	N/A	27.01 - 27.05	27	PAC-Bayes Bound
40	Monday, April 20, 2009	N/A	27.06 - 28.02	27b	Applications of the PAC-Bayes Bound
41	Wednesday, April 22, 2009	N/A	28.02 - 28.15	28	Statistical Significance
42	Friday, April 24, 2009	N/A	29.01 - 29.19	29	Pattern Recognition Applications
43	Monday, April 27, 2009	Comprehensive	Final	N/A	Final Exam (8 AM - 11 AM)

Homework:

Homework will be assigned but not graded. Solutions are available online.

Number	Chapter: Problem(s)
1	Chapter 2: 2, 3, 8, 10
2	Chapter 2: 15, 21, 23, 24
3	Chapter 2: 37, 40, 43, 44
4	Chapter 3: 1, 4, 11, 13, 16
5	Chapter 3: 17, 24, 38, 41, 50
6	Chapter 4: 2, 3, 10, 13, 14
7	Chapter 5: 9, 32, 33
8	Chapter 6: 2, 10, 21
9	Chapter 8: 5, 8, 19
10	Chapter 9: 18, 20, 27
11	Chapter 10: 4, 5, 14