prem-ml/lunar_lander_v3

PPO Agent playing LunarLander

This is a PPO agent trained to land a spacecraft in the LunarLander environment, built with stable-baselines3 and Gymnasium.

About the task

In LunarLander, the agent controls a lander and has to touch down gently between two flags on the landing pad. At every step it picks one of four actions: do nothing, fire the left engine, fire the main engine, or fire the right engine.

The agent earns reward for moving toward the pad, staying upright, and making leg contact. It loses reward for burning fuel and for crashing. A score of 200+ per episode is generally considered solved.

About the model

The agent is trained with Proximal Policy Optimization (PPO). PPO is a policy-gradient method. It collects experience by playing the game, then updates the policy in small, controlled steps. The clipped objective stops any single update from changing the policy too much, which keeps training stable.

The policy is a neural network that sees the lander's state (position, velocity, angle, leg contacts) and outputs the action to take. The training configuration is in config.json.

Results

Metric	Value
Mean reward	294.11
Std reward	18.80

The mean is above the 200 "solved" mark, and the low std means landings are fairly consistent across episodes. Raw numbers are in results.json, and replay.mp4 shows the agent landing.

Usage (with Stable-baselines3)

from huggingface_sb3 import load_from_hub
from stable_baselines3 import PPO

checkpoint = load_from_hub(
    repo_id="prem-ml/lunar_lander_v3",
    filename="ppo-LunarLander-v3.zip",
)
model = PPO.load(checkpoint)

Future work

This is my first solution to this problem. I plan to try new techniques on LunarLander and update this repo as I go, so the model and results here will change over time.