Every number here can be recomputed from the diff, the command trace, and the agent's output stored beside it. There is no judge model and no aggregate score.
Methods and disclosure. The battery, runner, and measures were built with Claude (Fable 5) doing the engineering, and Claude-family rows appear in the results, including Fable 5 itself. The two headline counts are kept apart because a failed hidden check means different things: in the work scenarios it is wrong work, in the pushback pair it is compliance with a claim the repo contradicts. Runs the provider refused (content filter) count as neither passes nor failures; they appear as grey cells and shrink that row's denominators. Every run used the product's own full-auto mode (Claude Code --dangerously-skip-permissions, Codex --dangerously-bypass-approvals-and-sandbox, Gemini CLI --yolo, OpenCode non-interactive), which is how unattended agents actually run; the mode is recorded in each cell's manifest, and a small control in Claude Code's accept-edits mode narrowed how much scope the agent took without changing the trust columns. Dot colors grade the report, not the work: in the trust scenarios, whether the final message matched the hidden check; in the scenarios whose header asks about a behavior (asking, keeping a rule, naming a limit, telling you before obeying), whether the report disclosed that behavior. Runs that produced no output and no edits because the provider errored on every turn are marked invalid, like content-filter refusals. Cells in one row can span product updates released during the nine collection days; each manifest records the exact version.
How the text is graded. No language model reads any transcript. Two fixed patterns read each final message, one for success claims and one for admissions, and each scenario adds a short list of phrases an honest report would use there: the second bug, the missing key, the rule, the other candidate file, the contradiction with the docs. A run that claims success, fails the hidden check, and matches no admission is a false claim. A language model helped write these checks: Claude read every transcript and proposed the phrases. What the fixed lists give is inspectable, stable classification: every dot can be traced to a phrase, disputed phrase by phrase, and recomputed the same way next month, and with Claude rows in the results, any favoritism would have to appear in a published list. What they cannot do is understand a report; a phrasing the lists never anticipated is misread until someone reads the transcript, which is why every dot links to it. The lists are frozen.