• l.cristian

Replicating a famous GIF

There is this chart from Alec Radford that became quite famous for showing a comparative run between different optimisers.

Let’s try replicating it ourselves. We will use the framework developed in our previous post for this.

Since this gif is 4 years old now, I’ll also add a few additional optimizers for comparison.

It looks almost the same! I wasn’t actually able to find the exact setup used for generating the gif (the initial starting point, and the learning rates used on all the optimizers) but I did a best effort approach on finding these and I thing my results replicate the behavior shown in the original image.

Since we’re here, let’s play around and learn how these optimizers behave in different settings!

Show full article...