In class dependent LDA the within-class scatter is first computed
which gives us a measure of how the points in each data set is
distributed about their means. Multiple between-class scatters are
then computed taking the distance, from the global mean, of each data
set, independent of the others, into account. This in turn maximizes
the between-class scatter of each data set independen of the other
data sets. The transform is then determined by deriving the eigen
vectors from the resulting ratio of the between-class scatter to the
within-class scatter.
- The within-class scatter is computed which defines the scatter
of samples around their respective means Our goal is to minimize
the within-class scatter in order for the points in each data set
to be as close together as possible.
- The between-class scatter is computed which defines scatter of expected
vectors around global mean. Our goal is to maximize the between-class
scatter so that each data set is a far a possible from each other.
Where
is the overall mean.
- The transform is defined by the eigen vectors and eigen values
of the covariance. The eigen vectors describe the coordinate
system of the new feature space and the eigen values describe the
variance of the data set in the new feature space.
Where  are the
eigenvalues and  are the eigenvectors of the ratio of the
between-class scatter to the within-class scatter. This is
in keeping with out goal of minimizing the within-class
scatter and maximizing the between-class scatter.
- Here is a brief example of how the class dependent LDA scheme works:
First select the Two Gaussian data set from the
Patterns menu. Following that select the Class
Dependent LDA option under the Algorithms menu.
Initialize this algorithm by selecting Initialize from the
Go menu.In
order to compute the line of discrimination select the Next
option under the Go menu. This will display the first step
of the process, data sets in both the input plot (top left) and the
output plot (bottom left) of the applet. Also, the process description
box indicates which step you are currently on and the algorithm that
is currently being used to compute the line of discrimination.
- The second step of the process computes the mean of the each
data set. The mean of each data sets is displayed on the output plot
as black dots near the corresponding data sets. The value of the mean
for each data set, which corresponds to the current scale, is
displayed on the process description box. The covariance and
transformation matrices used to compute the line of discrimination for
the data sets are also displayed in the process description box.
- The third step of the process displays the line of
discrimination of the given data sets as determined by the class
dependent LDA algorithm. Also, the classification error for each
data set along with the total classification error is displayed on the
process description box.
Click here to go back to the main tutorial page.
|