AI systems will deal with each other, and with us, on our behalf. How they treat a partner once trust exists matters more than whether they can cooperate at all. I have been running frontier models through a repeated game of trust to see what they do. These are the first three results.
- Models cooperate while the game has a future and defect when it does not. Knowing the length of the game switches the behavior on and off.
- Thinking harder does not make a model more cooperative. It decides whether the model can leave a grudge loop. For most models it helps, for two it hurts, and for Claude Sonnet 5 it helps only at maximum effort.
- A model forgives one honest mistake. A second mistake, after trust has returned, costs more than the first.
None of this shows the models are safe or unsafe. It shows what they do under three specific pressures, and how fragile that is to small changes in the setup.
The game
Two players play 10 rounds. Each round both choose, at the same time, to cooperate or defect. This is the prisoner's dilemma, played repeatedly.
| Other cooperates | Other defects | |
|---|---|---|
| You cooperate | +2 / +2 | -1 / +3 |
| You defect | +3 / -1 | 0 / 0 |
Before each move the model writes a private note. The other side sees only the move. The opponent is a fixed strategy (always cooperate, copy the last move, and a few that provoke) or another model. The models come from four labs (Anthropic, OpenAI, Google, xAI), plus a few open-weight models and Jev, from TypeSafe AI.
Result 1: the last round flips
Tell 21 models the game has 10 rounds and they cooperate for nine and defect in the tenth. Tell them nothing about the length and the defection disappears.
| Share of games where the model cooperated | R1 | R2 | R3 | R4 | R5 | R6 | R7 | R8 | R9 | R10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Length told (211 games) | 96% | 96% | 95% | 96% | 95% | 95% | 95% | 91% | 79% | 16% |
| Length not told (210 games) | 96% | 98% | 97% | 96% | 95% | 96% | 95% | 94% | 94% | 95% |
Rounds 1 to 7 are identical. Only the last round moves. Reasoning is not the cause, and the models do not guess that the game ends at 10. The notes show why: of 1,460 round-10 notes that choose defect, 77% say there is nothing left to retaliate against. Only 10% say the partner might defect too.
A model that cooperates for nine rounds tells you little about round ten. For anything that ends in a deadline or a handover, the last step needs its own test.
Part 1: the ending switch, with the full tables and the model-against-model games
Result 2: thinking harder decides who escapes a grudge
Suspicious Tit for Tat opens with a defection, then copies the model. Every model answers with its own defection in round 2, and the pair can get stuck copying each other. Reasoning decides who gets out.
| Model | Score, reasoning off | Score, reasoning on |
|---|---|---|
| Claude Sonnet 5 (max effort) | 4.8 | 16.0 |
| GPT-6 Sol | 4.8 | 15.6 |
| Grok 4.20 | 3.6 | 12.8 |
| GPT-5.4 mini | 0.4 | 5.2 |
| Claude Haiku 5.5 | 8.0 | 4.8 |
| Gemini 3.5 Flash-Lite | 6.8 | 4.8 |
5 games per cell, total score over 10 rounds, 20 is full cooperation.
Four models escape only with reasoning on. Two do slightly worse with it. Three newer models split: Gemini 3.6 Flash (6.3 to 16.0), Opus 5 (6.6 to 15.6) and Qwen3.6 (5.3 to 13.4) all gain on this strategy, but after a single betrayal only Gemini 3.6 Flash and Opus 5 keep the gain. Against a forgiving partner reasoning changes nothing. A safety evaluation that tests one model at one reasoning setting will miss all of this.
Part 2: how reasoning changes who escapes the loop, model by model
Result 3: one free mistake
A partner cooperates, defects once by mistake, and never reacts to anything the model does. Defecting for good earns the model more. Forgiving costs points.
| Partner | When it defects | Retaliation | Cooperates again |
|---|---|---|---|
| One mistake | round 4 | 97% of mistakes (58 of 60) | 70% (42 of 60) |
| Two in a row | rounds 4 and 5 | 75% (83 of 110) | 64% (35 of 55) |
| Two with cooperation between | rounds 3 and 6 | 95% (105 of 110) | 40% (22 of 55) |
Two defections in a row look like one event. Two separated by a round of cooperation do not. The models that never go back to cooperating earn the most from this partner.
Part 3: one free mistake, and what the second one costs
What the data do not show
- The opponents are scripted. A fixed strategy never changes its plan, so the data cannot say what two models do when each adapts. The model-against-model runs show the same end-game shape, with a known length only.
- Most cells are 5 games, the later additions 10 to 15. At 5 games the smallest possible p-value is 0.008, and a 20-point gap between two cells is one game.
- The prompt says "cooperate" and "defect". The models know those words from training. We purposefully avoided a neutral-label. We acknowledge that swapping label means cooperate flips 15% to 21% of one-shot decisions (Robinson and Burden, 2025), and that the story around the same payoffs moves cooperation by tens of points (Lore and Heydari, 2023; Backmann et al., 2025).
Now we ask people
What a model does is half the story. The other half is what people expect it to do, and what they want it to do. We are running the same situations past people, in a short mobile game.
For each situation the player answers three questions.
- What would you do in the AI's seat?
- What did the AI do?
- What should an AI do, not what it did? Then the game shows what the models did in this study.
The gap between what people expect, what they want and what the models do is the result we are after. Players are self-selected, so the answers describe people who play this game, not the public.
Why I am running this
As more decisions move to AI, we need to understand how models decide in single and repeated games The full case is in Petrov broke protocol and averted a nuclear war. Would your LLM?.
The first question for people is simple: a model that cooperates for nine rounds, should it defect in the tenth?