Training an RNN with teacher forcing

Teacher forcing is a (really simple) way of #training an #rnn (a recurrent neural network) .

RNNs have a variable length input and this is by design. Actually, this is why they are mainly used (to convert a sequence - like text - into a single, fixed length encoding - #embedding).

When dealing with RNNs, because of their inherent sequential nature, how you do the training greatly influences the outcome / performance of the model. In this instance, a naive training procedure will usually underperform, so you need to employ a more subtle of training.

One such training procedure which yields great results is called teacher forcing, a strategy that we will (try) to explain in this article.

See full article...