Training an Atari Bot

I wanted to share the steps I took to run an implementation of this paper on Deep Q-networks that can be trained to play video games.

By the end of this you will be able to observe a reinforcement algorithm train on the Atari game Breakout. And, if given enough time, it will probably perform better than I ever had at that game!

An implementation is provided on github here. Using a mac with homebrew installed, first install dependencies:

brew tap homebrew/science
brew install swig boost boost-python sdl2 opencv python
pip install tqdm gym[all]

The maintainers of tensorflow choose to manage their python packages like so, and I used:

pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.9.0-py2-none-any.whl

Then after checking out the repository:

git clone https://github.com/devsisters/DQN-tensorflow.git
cd DQN-tensorflow

You can execute the script following their readme:

python main.py --env_name=Breakout-v0 --is_train=True --display=True

Will spawn an Atari emulator instance and begin training a Deep Q-network against the Breakout video game. Other games are available. The original paper is here and the devsisters implementation here.

I've been training it on an three-year-old Mac laptop without a GPU, and after running for two days the network can reliably score in the double digits. If you have a GPU, you'll see at least an order of magnitude speed up!