While trying out reinforcement learning I built some custom ball & beam environments since I was already familiar with it from control theory labs. I built it as a first order system where the angle of the beam is under full control (did not want to spend time simulating a motor). So it can be baselined by using a simple PID controller.

https://github.com/simon-larsson/ballbeam-gym

There are currently environments for three objectives:

• Balancing – just keeping the ball on beam
• Setpoint – keeping the ball as close as possible to a setpoint
• Throw – throwing the ball as far as possible to the right

The environments have two different state spaces. The agent can either use key-variables (position, velocity, angle) or the images from the visualization as state space.

Hope someone else wants to try it! 🙂