RECURRENT NETWORKS: TOWARDS TIME SYNCHRONOUS DECODING
To incorporate time synchronous behavior into a neural network, we need some sort of feedback looop. The architecture below is known as a recurrent network:


A more popular version of this is the time delay neural network (TDNN):


These recurrent networks have been extremely important to allowing the integration of neural networks into the Markov model statistical framework we use in speech recognition. Such systems are known as hybrid systems.