Which RL environments can a model actually learn from?

The shortlist

Each number is how close a player got to the best possible score, out of 100. Read a row from left to right: a big climb means a model can learn there.

“Cheap trick” is the best answer that needs no skill: the same answer every time, or the question’s own input played back. We only tried it on Reasoning Gym, and added it after the training run below. Scores are on each environment’s own scale, rescaled so 0 is the worst score seen and 100 the best possible.

Does the shortlist predict training?

What Beam caught

Three official environments were broken as shipped. The ladder flagged each one, and Beam read the code.

How Beam is used

1. Plays the top of the ladderFrom low to max effort, after reading each environment’s README.
2. Fixes broken environmentsReads the source when a check fails, explains why, writes the patch.
3. Sets up environments it has never seenWrote the setup for community environments from their README.
4. A ladder inside one modelIts effort settings give several skill levels without other models.
5. Audits trainingJudged blind whether the trained model learned the skill, the format, or a cheat.

Also measured

Limits

• Most environments judged with Beam only have two to four tasks: a first pass, not a final call.
• One small model, Qwen3-0.6B, trained for 100 GRPO steps per family, once each.
• The cheap-trick check covers Reasoning Gym only, was added after training, and tries three tricks. The maze run found a fourth.
• Every number here comes from the files in the dataset, which also hold each check’s full statistics.