SYLLABUS

Contact Information:

Time TTH: 11 - 12:15 PM
Place 250 Simrall
Instructor Joseph Picone, Professor
Office 228 Simrall
Office Hours Flexible - send email for an appt.
Email picone@ece.msstate.edu
Class Alias ece_8443@ece.msstate.edu
URL http://www.ece.msstate.edu/research/isip/publications/courses/ece_8443
Textbook (Reference) R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, Second Edition, Wiley Interscience, ISBN: 0-471-05669-3, 2000 (supporting material available at http://rii.ricoh.com/~stork/DHS.html)
Suggested Reference D. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003 (also available at http://www.inference.phy.cam.ac.uk/mackay/itprnn/book.html)
Prerequisite Statistics, Signal Processing, many years of research, and attendance at a lecture where the speaker proudly exclaimed "PCA Fails!"
Other Reference Materials Pattern Recognition in Java: an applet designed to demonstrate fundamental concepts of pattern recognition.
Internet Links: Godfried T. Toussaint's online resources that include lots of valuable links and course notes.
Human Computer Interface Design: a multi-university course sponsored by the National Science Foundation.


Grading Policies:

Item: Date: Weight:
  Mid-Term Exam   TBD   50%
  Final Exam   TBD   50%
  Special Projects   Unspecified   10% per project


Exams:

We will have two in-class exams in this course (a mid-term and a final). Each will be closed books and notes. You will be allowed one page (double-sided) of notes. Calculators or other computing devices will not be allowed (or needed). The exams will require critical thinking. They will consist of one to two problems for which you will supply extended written solutions. The quality and clarity of your solution will be as important as whether you obtained the correct answer.

In addition, students can select and execute special projects on topics that are of interest to the class or two their research. These will be negotiated on an individual basis with the instructor, and must be agreed to in writing before the work can proceed. Typically in this class interesting issues arise that merit extended analysis. Students can earn extra credit by exploring these topics. Projects commonly involve a theoretical derivation, a computer simulation and a four-page paper summarizing the findings.

Attendance Policy:

Attendance is encouraged of course, but not formally count towards your grade. Most students tend not to do well in this class without frequent interaction with the instructor and the other students on the material.

Schedule:

Class Date Sections Topic
1 01/10 Chapter 1 Course Introduction
2 01/15 2.1 - 2.3 Bayes Decision Theory
3 01/17 2.4 - 2.5 Multivariate Gaussian Densities
4 01/22 2.6 - 2.8 Discriminant Functions For Gaussians
5 01/24 2.9, 3.1 - 3.2.3 Discrete Features, Maximum Likelihood
6 01/29 3.2.3 - 3.4 Bayesian Estimation
7 01/31 3.5, 3.6 Generalized Bayesian Parameter Estimation,
Sufficient Statistics
8 02/05 3.7, 3.8.1 Problems of Dimensionality,
Principal Components Analysis
9 02/07 3.8.2, 3.8.3 Linear Discriminant Analysis
10 02/12 10.13 Heteroscedastic Linear Discriminant Analysis
Independent Component Analysis
11 02/14 3.9 The Expectation Maximization Algorithm
12 02/19 3.10.1 - 3.10.5 Hidden Markov Models
13 02/21 3.10.6 The Baum-Welch Algorithm
14 02/26 4.1 - 4.9 Nonparametric Techniques
15 02/28 1.1 - 3.9 Mid-Term
16 03/04 5.11 Support Vector Machines
17 03/06 6.1 - 6.4 Neural Netwoks
18 03/18 8.1 - 8.8 Decision Trees
19 03/20 9.1 - 9.4 Resampling for Estimating Statistics
20 03/25 9.5 - 9.7 Resampling for Classifier Design
21 03/27 N/A Reinforcement Learning
22 04/01 10.1 - 10.4 Clustering
23 04/03 10.5 - 10.9 Hierarchical Clustering
24 04/08 10.10 - 10.14 Networks, Maps and Clustering
25 04/10 N/A Discriminative Training
26 04/15 N/A Particle Filters
27 04/17 N/A PAC-Bayes Bounds
28 04/22 N/A Statistical Significance and Confidence
29 04/24 N/A Examples of Pattern Recognition in Speech Recognition
30 05/01 Comprehensive 3 PM to 6 PM


Homework:

Homework will be assigned but not graded. Solutions are available online.

Number Chapter: Problem(s)
1 Chapter 2: 2, 3, 8, 10
2 Chapter 2: 15, 21, 23, 24
3 Chapter 2: 37, 40, 43, 44
4 Chapter 3: 1, 4, 11, 13, 16
5 Chapter 3: 17, 24, 38, 41, 50
6 Chapter 4: 2, 3, 10, 13, 14
7 Chapter 5: 9, 32, 33
8 Chapter 6: 2, 10, 21
9 Chapter 8: 5, 8, 19
10 Chapter 9: 18, 20, 27
11 Chapter 10: 4, 5, 14