A pull request written by a coding agent comes with a description that is itself a claim, and the cheapest useful review step is to check that claim against the diff. Blastline's PR brief does that mechanically: every name the description puts in code font, every symbol the diff removes, and every commit message is checked against the diff, and removed or changed symbols against a code graph. A claim can come back refuted, partial or consistent. It never comes back verified, and that word does not appear in the output. This post is what the brief checks, real output from Blastline's own pull requests, and a design reversal in version 0.20.0 that is the most useful thing we learned building it.
Why the description is the thing to check
The brief's README cites the number that set the priority. In a study of 23,247 agent-authored pull requests, descriptions claiming changes the diff does not implement were the largest class of mismatch, and mismatched pull requests were accepted at 28% against 80% (Gong et al., MSR'26). A reviewer who reads a confident summary and skims the diff is checking the summary, not the code.
Blastline already had the pieces to check it. The check command asks the graph whether a removed symbol still has callers, and the test-impact selection knows which tests a change reaches. The brief, introduced in 0.13.0 and extended through 0.21.0 between late September and mid-October, puts them in one PR comment, edited in place on every push.
What it checks, and what each verdict means
| Check | Refuted when | Partial when |
|---|---|---|
| A name in code font in the PR body or a commit message | no changed symbol bears it, no changed path matches it, and no changed line contains it | |
| A placeholder | the body is empty, a template line, TODO/WIP, or a wip/fixup! subject | |
| Scope | a file with symbol changes that neither the body nor any message names | |
| A removed symbol | it still has a static caller in a file the diff did not touch | it cannot be resolved in the base graph |
| A changed declaration | never | its callers in the base graph include files the diff does not touch |
The asymmetry is the design. Refuted is authoritative: the brief holds a specific counterexample, such as a caller that still exists in the graph or a name the diff never contains. Consistent means only that nothing was found against the claim. A changed declaration is never refuted, because its callers may well be compatible; the brief names them and leaves the call to the reviewer. A static graph can prove that something is still there. It cannot prove that something is correct, so the tool does not pretend to.
Real output
Blastline's own repository runs the newest brief on every pull request. Here are the refuted and partial claims from two of them, quoted as posted; the consistent lines are trimmed.
Pull request #50, on blastline 0.17.0:
- refuted: PR body names `application-{profile}.yml`. no such path in the diff (phantom change).
- refuted: PR body names `WalkDir`. no changed symbol bears it and no changed line contains it (phantom change).
- refuted: PR body names `src/brief.ts`. no such path in the diff (phantom change).
- refuted: PR body names `graph.json`. no such path in the diff (phantom change).
- refuted: PR body names `references_file`. no changed symbol bears it and no changed line contains it (phantom change).
- partial: the narrative names the changed code. not named by the PR body or any commit message:
src/references.test.ts (`ENV` added, +2), src/unnamed.ts (`NEVER_SUGGESTED` changed, +3) (understated scope).
- consistent: `pathVerdicts` removed from `src/select.ts`. 1 static caller in the base graph,
all in files this diff changes: `SelectOptions` (src/select.ts:12).
Five phantom changes is a lot, and it is honest about what they are: a description written while the design was moving names things a later commit took back out. That is exactly the mismatch the study counts, and the brief catches it on the author's own tool.
Pull request #55, on blastline 0.20.0:
- refuted: commit `d087a45` describes the change. placeholder text: "wip".
- partial: `commits` removed from `src/brief.ts`. could not resolve it in the base graph (symbol-ambiguous).
- partial: the narrative names the changed code. not named by the PR body or any commit message:
src/mcp.ts (`TOOLS` changed) (understated scope).
- partial: `checkpointFor` declaration changed in `src/checkpoint.ts`. still called by
`src/checkpoint-write.test.ts`, `src/index.ts` in files this diff does not touch;
check those call sites against the new declaration.
The last line is the kind a reviewer would not find by reading the diff, because the files it names are not in it.
The same comment carries a Reviewed by row. On both pull requests it reads "no reviewer other than the author so far" and names the people whose commits last touched the changed and reached files. The README gives the reason for the row: 40.1% of agent-authored pull requests on public GitHub are examined only by the developer who invoked the agent, against 21.5% of human ones (Agarwal, Miller, Kastner, Vasilescu 2026). The row states the fact without deciding whether the author's own reading counts.
The section we built, then emptied
Version 0.20.0 added What the agent decided: a short list, read from the agent session's transcript without a model, of what the agent did across the pull request that the diff cannot show, such as tests it made pass, work it reverted, and alternatives it weighed. The first version quoted lines: a person's requests and corrections, and the first sentence of an agent's message that stated a finding or a choice.
Then the review rounds on that pull request started, before any of it was released. Each one found a new way for quoted free text to carry something that must not leave the machine: a path, a hostname, a token, a password. Each fix tightened a filter, and each tighter filter let less through. One round measured the core problem directly: a short password reads like a word, and 26-54% of random 8-14 letter lower-case strings pass any word-shaped filter. No filter can clear a person's raw words.
After seven rounds the section stopped quoting free text entirely. Every line it prints is built from a fixed label, a test runner's name, and the change's own repository paths. On four real pull requests it now shows design notes weighing alternatives on one, and no section on the other three. That is less than the first version promised, and it is the version we would let run on someone else's repository. The README's privacy allowlist is code, not a flag: it names every field the brief may read from a session, and the raw transcript never leaves.
What it does not do yet
Two honest limits, from our own repositories:
- The intent column needs checkpoints. The brief's per-commit Intent cell reads agent session checkpoints written by Entire or by
blastline checkpoint write. None of the repositories in this post write them yet, so every Intent cell names the agent from the commit's own marks, such as aCo-authored-byline, or reads "no checkpoint". The claims checks above do not depend on it. - Our other repositories run an older brief. CGraph and the nxtsoft.io site repository pin
Nxtsoft/blastline@v0, which still points at 0.13.0. Their briefs carry the test selection and the removed-symbol check, without the narrative checks, the Reviewed-by row or the decisions section. On the twenty CGraph pull requests from #190 to #209, ten carry a 0.13.0 brief, and none checks a claim.
Try it
Run it locally before you push, against your own description:
blastline brief main..HEAD --narrative pr-body.md
That checks the description and the commit messages. The removed-symbol and declaration checks also need a cgraph change-context file, passed as --change-context; the Action produces it for you. Or add the GitHub Action and read the first comment. If it refutes something, check the counterexample it names. If it says consistent, remember that it means "nothing found against", and read the diff anyway.