Deep recurrent neural networks Artificial intelligence

early on, deep learning applied sequence learning recurrent neural networks (rnns) general computers , can run arbitrary programs process arbitrary sequences of inputs. depth of rnn unlimited , depends on length of input sequence. rnns can trained gradient descent suffer vanishing gradient problem. in 1992, shown unsupervised pre-training of stack of recurrent neural networks can speed subsequent supervised learning of deep sequential problems.

numerous researchers use variants of deep learning recurrent nn called long short-term memory (lstm) network published hochreiter & schmidhuber in 1997. lstm trained connectionist temporal classification (ctc). @ google, microsoft , baidu approach has revolutionised speech recognition. example, in 2015, google s speech recognition experienced dramatic performance jump of 49% through ctc-trained lstm, available through google voice billions of smartphone users. google used lstm improve machine translation, language modeling , multilingual language processing. lstm combined cnns improved automatic image captioning , plethora of other applications.