Breken hunts bugs with synthetic users and fault injection

What is it?

Breken has launched, billing itself as "a production before production" (breken). It rebuilds your entire software on its own machines and answers external services inside an isolated environment. At 4,000x speed it grows production state around the app: accounts, records, pending work.

How it works

• Synthetic users and agents work the product: impatient admins, users with two tabs open, agents processing refunds from an inbox. Each has intent, habits and limited patience.
• Every expected property becomes a hypothesis. Breken varies one condition at a time: a retry, a lost response, a clock jump or a deploy in the middle of a job.
• A failure only counts if it reproduces every time against a control run that never fails. You then get the exact state, a trace and a draft pull request.

What the benchmark shows

The company's own ESCAPES benchmark uses 327 real, user-reported bugs from 33 open-source products (breken benchmark). Systems scanned the code as of July 1, 2026, when 235 of those bugs were already present. Breken caught 76 (32%), proved 61 by running them, and raised 20 false alarms.

For comparison, Claude Fable 5.1 in Claude Code found 37 with 59 false alarms. GPT-6 Astra in Codex CLI found 28, and Semgrep rules found 9.

Why it matters

Breken says 71% of escaped bugs only show up when the product runs: with real data, multiple users, a partner outage or the passage of time. Reading code rarely catches those, so run-based testing belongs next to code review agents. One caveat: breken built and ran the benchmark itself, and there is no independent measurement yet.