Pattern Recognition Applet: Class Independent Linear Discrimination Analysis

In class independent LDA the within-class scatter is first computed which gives us a measure of how the points in each data set is distributed about their means. The between-class scatter is then computed taking the distances, from the global mean, of each data set into account. This in turn maximizes the between-class scatter for each set proportionately with respect to the other data sets. The transform is then determined by deriving the eigen vectors from the resulting ratio of the between-class scatter to the within-class scatter. Points from the current space are mapped to the new feature space my multiplying it with the transpose of the transform.

The within-class scatter is computed which defines the scatter of samples around their respective means Our goal is to minimize the within-class scatter in order for the points in each data set to be as close together as possible.
The between-class scatter is computed which defines scatter of expected vectors around global mean. Our goal is to maximize the between-class scatter so that each data set is a far a possible from each other.
Where is the overall mean.
The transform is defined by the eigen vectors and eigen values of the covariance. The eigen vectors describe the coordinate system of the new feature space and the eigen values describe the variance of the data set in the new feature space. Where are the eigenvalues and are the eigenvectors of the ratio of the between-class scatter to the within-class scatter. This is in keeping with out goal of minimizing the within-class scatter and maximizing the between-class scatter.
Here is a brief example of how the class independent LDA scheme works:

First select the Two Gaussian data set from the Patterns menu. Following that select the Class Independent LDA option under the Algorithms menu. Initialize this algorithm by selecting Initialize from the Go menu.In order to compute the line of discrimination select the Next option under the Go menu. This will display the first step of the process, data sets in both the input plot (top left) and the output plot (bottom left) of the applet. Also, the process description box indicates which step you are currently on and the algorithm that is currently being used to compute the line of discrimination.
The second step of the process computes the mean of the each data set. The mean of each data sets is displayed on the output plot as black dots near the corresponding data sets. The value of the mean for each data set, which corresponds to the current scale, is displayed on the process description box. The covariance and transformation matrices used to compute the line of discrimination for the data set are also displayed in the process description box.
The third step of the process displays the line of discrimination of the given data sets as determined by the class independent LDA algorithm. Also, the classification error for each data set along with the total classification error is displayed on the process description box.

Click here to go back to the main tutorial page.