KATZ SMOOTHING BASED ON GOOD-TURING ESTIMATES
- Katz smoothing applies Good-Turing estimates to the problem
of backoff language models.
- Katz smoothing uses a form of discounting in which
the amount of discounting is proportional to that predicted
by the Good-Turing estimate.
- The total number of counts discounted in the global distribution
is equal to the total number of counts that should be assigned
to N-grams with zero counts according to the Good-Turing estimate
(preserving the unit area constraint for the pdf).
- Katz Smoothing:
