Plan: findings get a recorded disposition, and a review’s outcome is one of five, not one line (#96)

On this page

Status: Done (2026-09-13) — four units on feat/review-disposition; the gate, the strip and the five outcomes pinned by unit and by the real handlers; pi-workflow 0.18.0.

Branch: feat/review-disposition · Issue: #96 · Close step: the MR’s Closes #96; after merge one line on the issue with the merge SHA and what was deferred.

Design

What is wrong

The reflection pause is advisory by doctrine (modes.adoc, "It informs rather than gates"): a completed review claims the ledger key whatever its verdict (reflection-cycle.ts claim/conclude; git-guard.ts claimReflection), and the identical retry — same diff, same message, same declared context — proceeds. What the author did with a FLAGS finding lives in the conversation and nowhere else: the commit that lands carries no trace that J3 was disputed, J5 accepted, or either simply re-run past. The external review of c398a905 named this, and every day since has shown the shape — today’s session took several findings, declined one with a reason, and the record of both is this transcript.

The record is also flat. ReviewOutcome.kind is reviewed | unreviewed | interrupted; settle says CLEAN or FLAGS; a review that answered N/A on a question — "cannot be settled here" — is reported CLEAN like one that settled every question, and the guard’s log (git_guard.reflection_done) carries outcome: reviewed for both.

What changes

Two things, both on the commit path only. The plan review’s cadence is one pause and then the human, who is the disposition; nothing there changes.

1. A disposition per flagged question, in the commit message, before the identical retry lands.

FLAGS (2): J3 FAIL … J5 FAIL …
  │
  ├─ author changes the diff        ─► new key ─► fresh review (as today)
  │
  └─ author re-runs the SAME commit ─► ledger says "reviewed, flagged J3 J5"
        message carries                     │
          Review-Response: J3 disputed - …  ├─► one line per flagged J? ─► lands
          Review-Response: J5 accepted - …  │
        message lacks one ─────────────────►┴─► deterministic refusal naming the J

The artefact is a trailer, Review-Response: J<n> <disposition> - <reason>, one line per flagged question, disposition ∈ disputed | accepted, reason non-empty. It lands in git history with the commit — durable, per commit, readable in git log, no sidecar. fixed is not a disposition on the identical retry: a fix changes the diff, the key changes, and the fresh review says whether it is fixed; a Review-Response: J3 fixed on an unchanged diff is refused as a contradiction with the words "the diff is unchanged; a fix is reviewed afresh".

The response is not part of the review’s identity, and the reviewer never sees it. The ledger key hashes the diff, the message and the declared context (index.ts ~498, stagedDiffKey). Adding a response line to the message must not make a new key — that would re-review, produce possibly different findings, and loop. So the key hashes the message with Review-Response: lines removed, and buildReviewPrompt hands the reviewer the message with them removed too: a cold reader that read the author’s rebuttal would be reviewing the argument, not the diff. The lines reach only git. They are read from the message text the guard can see — -m, or -F on a readable file — and nowhere else: git’s own --trailer 'Review-Response: …' route is refused by name ("write the response in the message body"), because the guard reads the message, not git’s trailer assembly, and a second place to look is a second place to miss (git-guard.ts collectCommitFlags already reads --trailer only for Co-Authored-By). An editor-composed message is not visible to the guard at all; today such a commit is reviewed with an empty message. After a FLAGS review, an identical retry with no visible message is refused by name ("the cold review flagged J3, J5; this retry’s message is not visible to the guard, so its responses cannot be read - commit with -m or -F") - fail-closed with the exit stated, never a pass the gate could not check.

What the ledger remembers. ReflectionCycle gains findings: Map<key, { flagged: string[]; unsettled: string[]; verdict }>, filled by conclude on a reviewed outcome from the review text. The flagged questions are read the way the battery’s scorer reads them (tools/model-battery/scorers/anchors.mjs reviewBlocks: a line opening with J<n> and a verdict word within reach, or a short title with the verdict on the next line, or a verdict-first FAIL on J3) — the same rules, in pi-workflow’s own review-findings.ts, pinned by the same shape tests, because the scorer’s shapes are the shapes the models actually write. If the verdict is FLAGS (n) with n > 0 and no FAIL block parses, the requirement degrades to "at least one Review-Response: line" so the record exists; a review whose verdict is CLEAN requires nothing.

Where the guard checks. index.ts at the commit site, on the branch where claim() returns false — the identical retry: requireDisposition(cycle, key, message) returns a deterministic refusal (commit guard (deterministic policy, not a model judgement): the cold review flagged J3, J5; each needs a Review-Response: line … the reviewer’s text is above/was: …) or nothing. The UNKNOWN_STAGED case (a diff the guard could not read) has no key to remember findings under and keeps today’s one-shot behaviour; stated as a residual.

2. Five outcomes, named where the author and the log see them.

Outcome When Where it shows

clean

reviewed, verdict CLEAN, every J answered PASS/FAIL

notice, log

partial

reviewed, verdict CLEAN, one or more J answered N/A

notice "CLEAN (J5 not settled)", log outcome: partial, unsettled: [J5], one line at the head of the reflection text

flags

reviewed, verdict FLAGS (n)

notice, log outcome: flags, flagged: [J3, J5]

unreviewed

the reviewer could not be reached, or no verdict line

as today, plus outcome: unreviewed on the log

interrupted

the human’s escape

as today

classifyOutcome(outcome) in reflection-cycle.ts is the one reader; settle and the git_guard.reflection_done / plan_review.done log lines use it. The plan review gets the classification for free (its outcomes are the same type); its cadence does not change.

What does not change

Who reviews, the bounded attempts and the self-review fallback, the identical-diff retry passing after a CLEAN review, the plan review’s cadence, the J-questions, the Review-Context trailer, the snapshot tools (#97), the judge. No new config key: the gate is the doctrine, not a knob (pre-1.0, no compatibility scaffolding).

Risks, by failure shape

Fail-open. The findings parser misses a FAIL block the model wrote in a new shape → the retry is asked for fewer responses than there were findings; the floor is today’s behaviour (advisory, proceeds), and the FLAGS (n) count cross-check keeps at least one line required. Stated in the log: git_guard.disposition carries flagged, responded, verdict.

Fail-closed. The parser sees a FAIL that is not one (a quoted "J3 FAIL" in prose) → the author is asked to respond to a non-finding. The same reach/length rules that hold the battery’s calibration at 148 cells / 1 disagreement bound this; a response line costs the author one sentence, and the refusal quotes the reviewer’s line so a phantom is visible as one.

Silent. An author who writes junk reasons is not stopped — the pause is advisory by design and stays so; what changes is that the junk is in git, signed, where a human reads it. A Review-Response: line on a commit that was never flagged is allowed and inert (a developer may pre-empt).

Loop. Response lines in the key or the prompt would re-review on every edit. Both are stripped; a test pins that adding a response line yields the same key and the same reviewer prompt.

Residual. UNKNOWN_STAGED commits (unreadable diff) carry no findings and are not gated; -a commits are keyed and gated like any other. The findings live beside the ledger, per session: a FLAGS review followed by a pi restart forgets the requirement, exactly as the ledger forgets the review — the retry then reviews afresh, which is the safer of the two outcomes. Both stated in modes.adoc.

Scope

Each unit is one commit on the branch, reviewed by the pause it changes; the test category is named per unit.

  1. U1 — findings and responses (packages/pi-workflow/review-findings.ts). flaggedRubrics(review), unsettledRubrics(review) (J answered N/A), parseReviewResponses(message){ rubric, disposition, reason }[] with malformed lines named, stripReviewResponses(message). Contract tests: the four header shapes the battery scorer pins (one-line, bold em-dash, title-then-verdict, verdict-first) each yield their J ids; a quoted "J3 FAIL" deep in a long prose line does not; N/A collected the same way; responses parsed with disposition and reason, a missing reason and an unknown disposition named; strip removes exactly the response lines and nothing else (roundtrip on a message without them is identity).

  2. U2 — the gate (reflection-cycle.ts, git-guard.ts, index.ts, reflection-review.ts). findings on the cycle, filled in conclude; requireDisposition; the key and buildReviewPrompt over the stripped message. Unit tests: FLAGS then identical retry without responses is refused naming J3 and J5; with both lines it passes; fixed on the identical diff refused with the words; CLEAN then retry passes with no line; FLAGS (2) with no parseable block requires one line; a response line does not change the key or the prompt; a --trailer 'Review-Response: …' on the command is refused by name. Integration in `review-context-pause.test.ts’s harness (real handlers, scripted reviewer): the refusal text, then the pass with responses.

  3. U3 — outcomes (reflection-cycle.ts classifyOutcome, settle, the two done log lines, the reflection header for partial). Unit tests: a CLEAN with a J5 N/A classifies partial and the notice says which; every-PASS is clean; FLAGS is flags with the ids; unreviewed and interrupted unchanged. Wiring test: git_guard.reflection_done carries outcome and unsettled.

  4. U4 — docs and release. modes.adoc reflection-pause entry (the disposition rule, the trailer grammar, the five outcomes, the residual); tuning.adoc gets the trailer beside Review-Context: in the cookbook; CHANGELOG; pi-workflow minor. The durable copy of this plan under docs/modules/ROOT/pages/plans/ with nav (first commit on the branch).

Out of scope, filed or dropped: a disposition for plan-review findings (the human’s approval is the disposition); recording a CLEAN review’s verdict in the commit itself (no finding, no trailer); a machine-readable dispositions index across commits (git log --grep Review-Response is the index); reviewer memory of prior dispositions (#70’s territory).

Edit this page · latest