top of page

Replicating a famous GIF

There is this chart from Alec Radford that became quite famous for showing a comparative run between different optimisers.

Let’s try replicating it ourselves. We will use the framework developed in our previous post for this.

Since this gif is 4 years old now, I’ll also add a few additional optimizers for comparison.

It looks almost the same! I wasn’t actually able to find the exact setup used for generating the gif (the initial starting point, and the learning rates used on all the optimizers) but I did a best effort approach on finding these and I thing my results replicate the behavior shown in the original image.

Since we’re here, let’s play around and learn how these optimizers behave in different settings!

31 views0 comments

Bình luận

bottom of page