top of page

Training an RNN with teacher forcing



Teacher forcing is a (really simple) way of #training an #rnn (a recurrent neural network) .


RNNs have a variable length input and this is by design. Actually, this is why they are mainly used (to convert a sequence - like text - into a single, fixed length encoding - #embedding).


When dealing with RNNs, because of their inherent sequential nature, how you do the training greatly influences the outcome / performance of the model. In this instance, a naive training procedure will usually underperform, so you need to employ a more subtle of training.


One such training procedure which yields great results is called teacher forcing, a strategy that we will (try) to explain in this article.


189 views0 comments
bottom of page