KNESER-NEY BIGRAM SMOOTHING

• Absolute discounting involves subtracting a fixed discount, D, from each nonzero count, an redistributing this probability mass to N-grams with zero counts.
  
  
• We implement absolute discounting using an interpolated model:
  
  
  
• Kneser-Ney smoothing combines notions of discounting with a backoff model. Here is an algorithm for bigram smoothing:
  
  
  
• Knesser-Ney smoothing constructs a lower order distribution that is consistent with the smoothed higher order distribution.