RECURRENT NETWORKS: TOWARDS TIME SYNCHRONOUS DECODING
To incorporate time synchronous behavior into a neural network,
we need some sort of feedback looop. The architecture
below is known as a recurrent network:
A more popular version of this is the time delay neural network (TDNN):
These recurrent networks have been extremely important to allowing
the integration of neural networks into the Markov model statistical
framework we use in speech recognition. Such systems are known
as hybrid systems.