How to Set Up OpenAI Gymnasium

What is Gymnasium?

Gymnasium is the maintained successor to OpenAI Gym. It provides a standard API for reinforcement learning environments, along with a collection of environments you can use for experiments and training. citeturn0search3turn0search2

If you’re following an older tutorial that tells you to install gym, be aware that Gymnasium has a newer API. The old done return value was split into terminated and truncated, among other changes. citeturn0search1

Install it

Start with a virtual environment:

python -m venv .venv
source .venv/bin/activate

On Windows, activate it with:

.venv\Scripts\Activate.ps1

Then install Gymnasium:

pip install gymnasium

That’s enough for the basic environments.

Try an environment

Create a Python file such as test_gym.py:

import gymnasium as gym

env = gym.make("CartPole-v1")
observation, info = env.reset(seed=42)

for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        observation, info = env.reset()

env.close()

The important part is the new step() return value. Gymnasium gives you terminated and truncated separately instead of the old single done value. citeturn0search3turn0search1

Rendering an environment

If you want to see an environment instead of just stepping through it, create it with a rendering mode:

env = gym.make("CartPole-v1", render_mode="human")

Not every environment uses the same rendering dependencies, so the extra packages you need depend on which environment you choose.

For example, the Box2D environments have their own dependencies. Current Gymnasium documentation recommends installing SWIG and the Box2D extra for those environments. citeturn0search13

pip install swig
pip install "gymnasium[box2d]"

Installing other environments

Gymnasium has different environment families with different optional dependencies. You don’t need to install every extra just to use the basic API.

Install the extra for the environment family you actually want to use, then create it with gym.make().

The basic Gymnasium loop

Most Gymnasium code eventually comes down to this pattern:

import gymnasium as gym

env = gym.make("CartPole-v1")
observation, info = env.reset()

while True:
    action = env.action_space.sample()
    observation, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        break

env.close()

Your reinforcement learning algorithm replaces action_space.sample() with the policy it is training or evaluating.

What to do next

Once Gymnasium is running, the next step is usually connecting an RL algorithm to the environment. You can also start creating custom environments using Gymnasium’s environment API. The official documentation has a custom-environment workflow for doing that. citeturn0search8turn0search9

If your installation doesn’t work, see How to Fix Common Gymnasium Installation Problems.