• introduction to entropy and its properties
  
  
• relative entropy, mutual information and cross entropy
  
  
• why cross entropy is important
  
  
• application of cross entropy on estimation of bigram language models
  
  
• briefly introduce other applications of entropies