verify-subagent-reports
verifiedf93656e8-4096-40f9-bd86-8f92bf120e45
Trust but verify a subagent/agent's "done" claim — run the tests, read the diff, check the artifacts actually exist. Use before accepting any delegated or automated result.
Metadata
Skill file
# Verify Subagent Reports
Use before accepting any delegated or automated "done" claim. A subagent's prose
summary is a claim, not evidence. Verify it the way you would verify a stranger's
work: run the tests, read the diff, and confirm the artifacts exist.
## 1. The verification checklist
For every "done" report, work through all of these:
```bash
# 1. Tests actually pass — run them yourself
pytest -q # or go test ./... / npm test / cargo test
# 2. Claimed files exist and are non-empty
ls -la src/auth/password.py
test -s src/auth/password.py && echo "non-empty" || echo "EMPTY OR MISSING"
# 3. The diff matches the stated scope
git diff --stat main...HEAD # or the subagent's branch
git diff main...HEAD # read the actual changes
# 4. No secrets / debug junk left behind
git diff main...HEAD | rg -n 'API_KEY|SECRET|password\s*=|print\(|console\.log|TODO'
```
Each check is independent — a "done" claim fails if *any* of them fails.
## 2. The "show me the output" rule
Require the actual command output or artifact, not a prose summary:
| Claim | Required evidence |
|---|---|
| "All tests pass" | The `pytest -q` output (or you run it yourself) |
| "The feature works" | The artifact exists + a run with real output |
| "No security issues" | The diff read by you, not the subagent's reassurance |
| "I created file X" | `ls -la X` shows it, and it is non-empty |
A subagent that reports "done" but cannot produce the output has not finished —
it has narrated finishing.
## 3. Spot-check vs full verification
| Change type | Verification depth |
|---|---|
| Small, low-risk (docs, formatting) | Spot-check: skim the diff, confirm files exist |
| Logic change (bug fix, new function) | Full: run tests + read the diff |
| Security / auth / schema change | Full + adversarial: also check for injection, missing authz, migration rollback |
Do not spot-check a schema migration. Do not full-verify a README typo. Match the
depth to the blast radius.
## 4. What to do when verification fails
1. **Do not integrate.** Reject the report, not the person.
2. **State the exact failing check** — "`pytest tests/test_password.py` fails with
2 errors; `src/auth/password.py` is missing" — not "this isn't done."
3. **Return it with a precise next action** (the same bar as a handoff note).
4. Re-verify after the fix — do not trust the second "done" any more than the first.
## Guardrails
- Do **not** accept "all tests pass" without running them (or seeing the output).
- Do **not** trust a claim about a file you have never opened or confirmed exists.
- Do **not** integrate a result that fails any verification check — no matter how
confident the subagent sounds.
- Do match verification depth to risk — full adversarial review for security/schema,
spot-check for cosmetic changes.
- Do re-verify fixes; the second report deserves the same skepticism as the first.
## Pitfalls
- **Accepting "all tests pass" on faith** — the subagent may have run the wrong
suite, or not run anything, or the tests are broken.
- **Trusting a claim about a file you never opened** — "the migration is in
migrations/004" but the file is empty or was never created.
- **Not reading the diff** — the change does more (or less) than the summary says.
- **Spot-checking a high-risk change** — a security fix deserves full review, not
a skim.
- **Re-trusting after a failed first report** — "it's fixed now" is a new claim
requiring fresh verification, not a free pass.
## Verify / Checklist
- [ ] Tests were run (or their real output inspected) — not just claimed to pass.
- [ ] Every claimed file exists and is non-empty.
- [ ] The diff matches the stated scope — no surprise or unrelated changes.
- [ ] No secrets, debug prints, or TODOs left in the diff.
- [ ] Verification depth matched the change's risk (full review for security/schema).
- [ ] A failed verification produced a precise, actionable rejection — not just "no."
- [ ] Any fix was re-verified before integration.Attached files
No attached files.