Hi /r/MachineLearning.

​

As part of my recent side project to learn about reinforcement learning I’ve created a clone of the classic Snake game as a reinforcement learning environment and solved it with advantage actor-critic. This is one of the warm-ups from OpenAI’s requests for research 2 (https://openai.com/blog/requests-for-research-2/).

​

You might be thinking this sounds like a very run of the mill introductory RL project. Well here are a few things that I think make it more interesting than just that.

1. I went completely overboard on the environment. Its implemented in pure PyTorch in a vectorized fashion such that I can run 1000s of environment in parallel on a single machine.
2. I compare performance of a few architectures, including a model copied from Deepmind’s recent Relational RL paper (spoilers, it doesn’t outcompete the other agents on this very simple task).
3. I evaluate the performance of an agent trained on a small environment in a larger environment – a limited form of RL transfer learning.

​

Medium article: https://towardsdatascience.com/learning-to-play-snake-at-1-million-fps-4aae8d36d2f1

Code: https://github.com/oscarknagg/wurm/tree/medium-article-1

​

Here’s a GIF of one of the final policies:

​

Processing gif tdsja08fssz21…

I’m currently working on the “Slitherin'” suggestion on OpenAI’s request for research 2.0. Here’s a preliminary GIF.

​

Processing gif qkerdp6kusz21…