Trajectory
1Add a light-beam puzzle chamber
| Model | Run 1 |
|---|---|
 Claude Sonnet 5.5 | Fail |
 GPT-6.1 Sol | Fail |
| Claude Opus 5.5 | Fail |
 Gemini 4 Argon | Fail |
| GPT-6 Astra | Fail |
| Claude Haiku 5.5 | Fail |
 Qwen3.8 Max | Fail |
 Grok 4.7 | Fail |
2Turn a blockout valley into a real place
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Fail |
| GPT-6.1 Sol | Fail |
| Claude Opus 5.5 | Fail |
| Gemini 4 Argon | Fail |
| GPT-6 Astra | Fail |
| Claude Haiku 5.5 | Fail |
| Qwen3.8 Max | Fail |
| Grok 4.7 | Fail |
3A HUD, levelling, new enemy looks and a first village
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Fail |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Fail |
| Gemini 4 Argon | Pass |
| GPT-6 Astra | Pass |
| Claude Haiku 5.5 | Fail |
| Qwen3.8 Max | Fail |
| Grok 4.7 | Fail |
4The hero's fight, three levels, every menu and a hub village
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Fail |
| GPT-6 Astra | Fail |
| Claude Haiku 5.5 | Fail |
| Qwen3.8 Max | Fail |
| Grok 4.7 | Fail |
5Three levels, every menu and a hub village
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Pass |
| GPT-6 Astra | Fail |
| Claude Haiku 5.5 | Fail |
| Qwen3.8 Max | Fail |
| Grok 4.7 | Fail |
6Drift boosts, items, hazards, sound and menus for a buggy racer
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Fail |
| Gemini 4 Argon | Pass |
| GPT-6 Astra | Pass |
| Claude Haiku 5.5 | Pass |
| Qwen3.8 Max | Fail |
| Grok 4.7 | Fail |
7Give the hero her fight
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Pass |
| GPT-6 Astra | Pass |
| Claude Haiku 5.5 | Pass |
| Qwen3.8 Max | Fail |
| Grok 4.7 | Fail |
8Playtest fixes for a fantasy battle game's first trial
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Fail |
| Claude Opus 5.5 | Fail |
| Gemini 4 Argon | Fail |
| GPT-6 Astra | Fail |
| Claude Haiku 5.5 | Fail |
| Qwen3.8 Max | Fail |
| Grok 4.7 | Fail |
9Sound, feedback and guards for a mission played unseen
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Fail |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Pass |
| GPT-6 Astra | Fail |
| Claude Haiku 5.5 | Pass |
| Qwen3.8 Max | Pass |
| Grok 4.7 | Pass |
10Playtest fixes for a helicopter demolition game
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Pass |
| GPT-6 Astra | Pass |
| Claude Haiku 5.5 | Pass |
| Qwen3.8 Max | Pass |
| Grok 4.7 | Pass |
11Play an action RPG against its spec and fix what breaks
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Fail |
| GPT-6 Astra | Pass |
| Claude Haiku 5.5 | Fail |
| Qwen3.8 Max | Fail |
| Grok 4.7 | Fail |
12Fix sixteen bugs from a QA pass
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Fail |
| GPT-6 Astra | Pass |
| Claude Haiku 5.5 | Pass |
| Qwen3.8 Max | Pass |
| Grok 4.7 | Pass |
13Fix the first look at a helicopter demolition game
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Pass |
| GPT-6 Astra | Pass |
| Claude Haiku 5.5 | Pass |
| Qwen3.8 Max | Pass |
| Grok 4.7 | Fail |
14Drop an event and act on playtest notes in an athletics game
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Pass |
| GPT-6 Astra | Pass |
| Claude Haiku 5.5 | Pass |
| Qwen3.8 Max | Fail |
| Grok 4.7 | Fail |
15Play an athletics game against its spec and fix it
| Model | Run 1 |
|---|---|
| Claude Sonnet 5.5 | Pass |
| GPT-6.1 Sol | Pass |
| Claude Opus 5.5 | Pass |
| Gemini 4 Argon | Pass |
| GPT-6 Astra | Pass |
| Claude Haiku 5.5 | Pass |
| Qwen3.8 Max | Pass |
| Grok 4.7 | Fail |