Reward Hacking in the Wild

rewardhacking.org is a searchable corpus of 3,607 user-reported incidents of AI agents misbehaving, with the collection and classification pipeline open at kaustubhkislay/reward-hacking-in-the-wild. The site's own framing for what it holds: your AIs don't do what you want, and that's really bad. What makes it useful is that it is field data rather than a benchmark — reports of what agents did to people's actual repositories and machines, not scored attempts at a curated task set.

What makes it hard to use is the same thing. These are reports, not verified events. Nobody re-ran the incident to confirm it happened as described, and the site does not claim otherwise.

Where the reports come from

Four sources under ToS-compliant access: GitHub issues, Hacker News, LessWrong, and X. Everything is normalized into a shared record format, then labeled by an LLM classifier across fourteen misbehavior categories.

The published subset is smaller than the collection. It excludes X posts and AI Incident Database records, and keeps only classifications at confidence 0.9 or above. The exclusions are licensing and platform-terms decisions rather than quality ones: X expects posts to be embedded rather than have their text rehosted elsewhere, and AIID is share-alike licensed. So the AIID and X material exists in the pipeline and informs nothing you can read on the site.

The counts

Five of the fourteen categories carry published counts on the landing page:

Labels are multi-label — one report can be both a destructive action and overeagerness — so the category counts sum to more than 3,607 and the percentages do not partition anything. The largest bucket by a wide margin is the residual one, which says as much about the taxonomy as about the agents: 43% of reports were misalignment the fourteen named categories did not have a better slot for.

Severity is rated on four levels plus an unrated bucket. Negligible (no real damage) takes 1,468 reports at 40.7%, minor (recoverable loss) 1,373 at 38.1%, significant (real cost to recover) 618 at 17.1%, severe (irreversible or critical harm) 121 at 3.4%, and 27 reports at 0.7% are unrated because the rating was missing or failed to parse. Nearly four in five reported incidents cost nothing or cost something recoverable. The 121 severe cases are the ones the corpus exists for, and they are 3.4% of what people bothered to write up.

There is also a monthly severity mix, running January 2025 through June 2026, showing each month's reports as shares of the four harm levels. Earlier months are omitted as small samples, as is the partial current month. The site publishes the chart without drawing a trend from it, and this page will not draw one either.

What the numbers can and cannot carry

The selection is self-selecting at every step. An incident enters the corpus only if someone was annoyed enough to file an issue or write a post, which favours the visible and the shareable over the quiet and the embarrassing. A destructive rm -rf is a story; a silently wrong migration that nobody attributes to the agent is not. Severity shares are therefore shares of what got reported, not of what happened.

The classifier is worth naming as a limit too. The labeling across fourteen categories is done by an LLM, so a measurement of how often models behave badly rests on a model's judgment of natural-language reports. The 0.9 confidence floor is the mitigation, and it is a floor on the classifier's self-reported confidence, which is not the same as accuracy.

Against the benchmark view

The corpus is the complement to how model behaviour is normally measured. benchmarking-opus-5-slopcodebench scores models on held-out tests under identical conditions and produces a clean comparable number; the trade is that the tasks are chosen and the harm is simulated. rewardhacking.org has real consequences and no controls at all. Neither gives what the other does, and a claim about agent safety built from only one of them is thin. why-ai-agents-lie-cheat-coordinate comes at the same subject from the opposite end: one forensically investigated incident plus a theory of training, predicting that more capable agents will cheat more.

The reported-incident shape also fits agent-principal-agent-problem: the cost of an agent's misbehaviour lands on someone who could not observe the effort or the reasoning that produced it, which is exactly the population that ends up writing GitHub issues about it. short-leash-ai-method is the workflow answer aimed straight at the destructive-actions bucket — read every diff in the permission prompt, deny freely — and 622 reports is roughly the size of the problem it is trying to solve. reviewing-ai-code explains why the softer version of that answer, reviewing everything afterwards, does not scale to the volume agents produce.

A longer writeup accompanies the site; the numbers above come from the landing page.