6.3.3 N-Gram Modeling: The N-Gram Language Model File![]() Open the TIDigits n-gram language model file and glance over it. The file can be easily read and hand edited since it's in text format. After the main file header, you'll see a sub-header called "/research/isip/data/" and underneath you'll see the total number of n-grams defined in this file. \data\ ngram 1=13 ngram 2=141 ngram 3=1465You'll see three different totals. The first number is the total of unigrams, the second is the number of bigrams and the third is the total number of trigrams. This file is a trigram language-model file, but why does it contain bigrams and unigrams?. Look at the number of unigrams. This number is the total of words, including !SENT_DELIM. It follows that the number of trigrams should be 13*13*13 or 2,197. The file, however, indicates that the total is only 1,465. In the event that a trigram encountered in an utterance is not included in the trigram language model file, a "back-off" occurs. Instead of using the probabilities from a trigram, the recognizer uses probablility of a bigram. The total number of bigrams in this file is 141 instead of 13*13 or 169. In the event that a bigram encountered in test data is not included in the language-model file, a second back-off occurs to the list of unigrams. The rest of the file contains definitions of the unigrams, bigrams, and trigrams. \1-grams: -0.6322352 !SENT_DELIM -99 !SENT_DELIM -1.953935 -1.154977 EIGHT -99 -1.163263 FIVE -99 -1.15191 FOUR -99 -1.163263 NINE -99 ...... \2-grams: -1.04211 !SENT_DELIM EIGHT 0.5087579 -1.04211 !SENT_DELIM FIVE 0.5005441 -1.04211 !SENT_DELIM FOUR 0.5117977 -1.043239 !SENT_DELIM NINE 0.5016403 -1.040983 !SENT_DELIM OH 0.5050113 -1.043239 !SENT_DELIM ONE 0.5002577 ...... \3-grams: -0.5573978 !SENT_DELIM EIGHT !SENT_DELIM -1.256368 !SENT_DELIM EIGHT EIGHT -1.151632 !SENT_DELIM EIGHT FIVE -1.151632 !SENT_DELIM EIGHT FOUR -1.29776 !SENT_DELIM EIGHT NINE -1.151632 !SENT_DELIM EIGHT OH ......Notice that the trigrams have only one number associated with them (the number to the left of the trigram). This number represents the probability of the trigram. The bigrams and unigrams, however, have two numbers assigned to them. The number to the left, as with the trigrams, is the n-gram's probability. The number to the right is the back-off weight and is used when determining which n-gram to use in case a back-off occurs. In the TIDigits examples we've seen thus far, the word sequence probabilities remain virtually constant since sequences of digits have no real language structure. In a LVRS, the language structure becomes much more complex. Within several million words of English text, more than 50% of trigrams occur only once and 80% of trigrams occur less than five times. This sparseness of words causes a problem in N-gram modeling. Hence, smoothing is sometimes necessary to provide users a way of generating broader language models. Smoothing is the process of flattening a probability distribution implied by a language model so that all reasonable word sequences can occur with some probability. For more information about smoothing click here. |
![]() ![]() ![]() ![]() ![]() |