Hidden Markov Models (HMM) have been the most widely used and
successful modeling units for speech recognition. Existence of
efficient ways to estimate the model parameters has been a driving
force for its popularity. Traditionally Maximum Likelihood (ML) has
been the chosen of parameter estimation paradigm. The goal of this
estimation process is to maximize the likelihood of the data given the
model. Therefore it is a representative technique where every model
uses only the positive examples to update its parameters.
Though ML is useful, it does not guarantee improved recognition
performance. Several techniques have been developed over the years
that try to increase the discrimination power of the models by
exposing the models to negative examples. Maximum Mutual Information
(MMI) is one such estimation paradigm where the goal is to maximize
the mutual information between the models and the training data. In
one form this boils down to a maximum a posteriori (MAP)
estimator. Another discriminative technique that has gained prominence
in the recent past is the Minimum Classification Error (MCE) where the
goal is to explicitly improve classification performance of the models
on the training data. In general discriminative techniques take longer
to optimize parameters compared to ML. Their success on LVCSR tasks
has been marginal at best. This talk will provide a mathematical
description of some of these discriminative techniques and discuss
their pros and cons.
Additional items of interest: