External Review — Quality & Risk Unit

Date: 2026-06-01 Commissioned by: COO / Principal (per OI-5 / SR-1 in reports/2026-06-01-program-assurance-signoff.md) Reviewer: External Review Partners (third line of defense — independent) Scope: Q&R charter, SOP-085/086/087, JA-085, the program-level assurance sign-off, and the operational risk register — Q&R’s own instrument and the artifacts it produced for the Documentation & Knowledge Program G3 gate.

Executive Summary

The Quality & Risk unit is fit for purpose as a second line of defense with two structural conditions that must be addressed: the unit functions as the org’s sole independent assurance mechanism and its SOPs are technically sound, but a permanent constraint (no shell access) means every gate verdict it issues rests on the audited party providing machine-attestation — a control-design dependency that inverts the auditor/auditee relationship on the most critical proof. Additionally, the routing-without-escalation model has produced zero closures on Wave-1 control gaps that have been open since the program’s first sample, revealing a gap between the unit’s stated follow-up mechanism and its actual enforcement posture.

Overall rating: Fit with conditions.

Method

This review applies the unit’s own four-test bar (executable, non-redundant, falsifiable, interlocking) to Q&R’s own artifacts — the same standard the unit holds all other units to. Evidence read line-by-line:

• docs/units/quality-risk/charter.md
• docs/units/quality-risk/job-aids/JA-085-four-test-review-checklist.md
• docs/sop/SOP-085-internal-audit.md
• docs/sop/SOP-086-risk-register-maintenance.md
• docs/sop/SOP-087-four-test-stratified-sample.md
• reports/2026-06-01-program-assurance-signoff.md
• reports/risk-register.md
• reports/audit-schedule.md
• reports/2026-06-01-wave1-qr-sample.md (§1 excerpt)
• reports/2026-06-01-waves3-4-qr-sample.md (§0–§4 excerpt)
• reports/2026-06-01-program-evaluation-and-recommendations.md (full)

The unit’s SOPs exist at the cited paths in docs/sop/. This review did not assess artifacts in other units’ lanes except where they bear on Q&R’s independence or coverage.

Findings

F-1 — The no-shell constraint is a structural control-design weakness, not a disclosed limitation (Medium)

Evidence: SOP-085 step 6 instructs “Confirm both scripted gates. Run node scripts/handoff-reconcile.mjs and node scripts/km-index-check.mjs if a shell is available; otherwise logic-trace them against source + spec, label the result a trace not an execution, and request Operations/PMO paste literal stdout.” The risk register logs this as R-006, rated Low/Mitigating.

The assurance sign-off (§7) explicitly states Q&R “logic-traced” both gates and that “Operations/PMO paste literal stdout into the wave checkpoint record to machine-attest the headline.”

The structural problem: Operations is Q&R’s counterparty on the reconciler — the unit that owns the mechanism being audited (charter.md “Hard boundaries” table). Requiring the auditee to provide the machine-attestation proof that the audit relies on is not a temporary workaround; it is a permanent structural dependency. A Low/Mitigating rating under-weights this. The compensating control (disclose + ask) is correct practice for a disclosed gap, but the underlying design — the second line cannot independently execute the primary gate it assures — is a Medium structural weakness, not a Low one.

Severity: Medium. The disclosure regime is honest and the trace methodology is rigorous. This is not an active defect or a cover-up. But the rating in the risk register should reflect the control-design constraint, not just the compensating-control posture.

Recommendation: Re-rate R-006 to Medium. Commission the machine-attestation tooling in CI (roadmap A4) as a P0, not P1 — this is the only path to Q&R having an independently verifiable gate, which is the structural requirement for a credible second line on the reconciler.

F-2 — Wave-1 control gaps have no escalation mechanism and none have closed (Medium)

Evidence: The control-gap registry in reports/risk-register.md shows six open gaps: G-W1-1 (Documentarian, open since Wave 1), G-W1-2 (Process Engineer, open since Wave 1), G-W2-1 (CPTO), G-W2-2 (Design Research Director), G-W2-3 (HR), G-W2-4 (Product Manager). At G3 sign-off — the program’s completion gate — zero have been formally closed.

The charter Operating rule 8 states “New trigger → request, don’t invent.” The Troubleshooting section states “A Critical finding sits unaddressed by the owning Lead → escalate to the Principal/COO after a reasonable window.” But no escalation mechanism is defined for Medium/Low gaps, and no “reasonable window” is specified in SOP-085 or SOP-086.

The assurance sign-off acknowledges the staleness class but frames it as “not a blocker” and “don’t let it become standing.” It has, in effect, already become standing: G-W1-1 and G-W1-2 are open across the full span of a five-wave program.

The structural gap: Q&R’s model is route-and-monitor. The SOPs do not define what happens when a routed Medium finding remains open across multiple waves with no closure. The audit-schedule.md carries “confirm closure at the next cycle” but does not specify what Q&R does when an owner does not close — other than for Critical findings (escalate to Principal/COO). This creates a category of findings that are routed, not closed, and not escalated, and the unit has no formalized response to that state.

Severity: Medium. This is not theater — the gaps are real, documented, and tracked. But routing without an escalation path for persistent-open Medium findings is a process gap in Q&R’s own SOP, producing a result (zero closures at program completion) that Q&R would flag as a conformance defect in any other unit’s standing controls.

Recommendation: SOP-086 should define a maximum age for an open Medium control gap (suggested: two full wave cycles, or a calendar quarter) after which Q&R escalates to the COO regardless of severity. The escalation path for Critical already exists — extend it to aged Medium. Assign: Q&R Director (propose) → Process Engineer (SOP revision) → COO (approve threshold).

F-3 — Solo-unit structure has no formalized independent cadence for third-line review (Medium)

Evidence: Q&R is a solo unit — one Director, no internal four-eyes, no peer reviewer within the unit. The charter states this explicitly and correctly: “solo unit, one member.” SOP-085 “Roles/approvers” reads “Q&R Director runs the audit and issues the verdict (sole reviewer; independent).”

The self-review disclosure (SR-1) in the assurance sign-off recommends commissioning External Review Partners but does not establish a recurring cadence. OI-5 treats this as a one-off commission, and the program evaluation (F2) lists it as a P1 “Principal decision.” The audit-schedule.md has no entry for periodic third-line review of Q&R.

The gap: SR-1 is appropriately self-disclosed and demonstrates the unit’s integrity. But treating third-line review as a one-off program event rather than a standing cadence leaves the unit without structural independence controls between commissioned reviews. The charter identifies External Review Partners as the third line, but no trigger, frequency, or conditions exist for recurring engagement.

Severity: Medium. A solo second-line unit that is reviewed externally only when it self-flags a conflict is not the same control as one that is reviewed on a defined recurring schedule. The unit’s value depends entirely on independence; that independence has no standing control other than self-disclosure.

Recommendation: Add an annual (or per-program-cycle) Q&R external review entry to reports/audit-schedule.md, with External Review Partners as the named reviewer. Make SR-1 a standing cadence entry, not a one-shot commission. Owner: COO (policy); Q&R Director (schedule entry). Effort: trivial; structural value: high.

F-4 — The assurance sign-off correctly discloses SR-1 but conflates two distinct independence breaks (Observation)

Evidence: The sign-off (§8) correctly identifies that Q&R reviewed its own Wave-3 artifacts and marks those cells as “self-review (not independent).” However, the framing presents this as one independence break. There are structurally two:

1. Content self-review: Q&R applied the four-test bar to the Q&R charter, SOP-085/086/087, the risk register, and the audit schedule. The verdicts on those artifacts (“✓ self-review”) cannot be independent.

2. Method self-review: Q&R assessed the quality of its own sampling method (SOP-087) using that same method. This is the deeper circularity — the anti-theater gate cannot independently validate the gate design itself.

The sign-off does not distinguish these. The first is inherent to any second-line unit and is correctly handled by routing to External Review. The second is a design-level question: who evaluates whether the four-test bar itself is set at the right level, and whether SOP-087’s stratification rules are adequate?

Severity: Observation. This review closes the first break. The second (gate-design adequacy) is outside Q&R’s own lane and would require the Principal or COO to commission a periodic methodology review. It is not a defect in the current program; it is a gap in the ongoing governance model.

Recommendation: When commissioning periodic External Review (per F-3), explicitly scope the methodology review: “Is the four-test bar set at the right level? Is SOP-087 stratification adequate for the org’s current size and complexity?” This is a one-paragraph addition to the external review brief.

F-5 — SOPs 085/086/087 pass the four-test bar on independent read (Observation, favorable)

Applied the four-test bar to Q&R’s own SOPs:

Executable: SOP-085 steps 1–8 are numbered, specific, and sequenced. SOP-087 includes a worked example (Wave-1 precedent cite). JA-085 is a checklist a fresh instance can follow without incumbent context. Pass.

Non-redundant: All three SOPs link to docs/canon.md, interface-contract-spec.md, master plan sections, and docs/handoff-triggers.md rather than restating them. No copy-pasted policy or rule text found. Pass.

Falsifiable: SOP-087 names the exact size rule (≥20% of frame, min cell-coverage floor), names the three stratification dimensions, cites specific wave reports (2026-06-01-wave1-qr-sample.md) as worked examples. SOP-086 names the exact register file path. Pass.

Interlocking: SOP-085 Interface Contract declares review requested as input and review passed / artifact approved as outputs, with named counterparties. SOP-087 mirrors these. The charter’s collaborators table agrees on triggers and direction. Pass.

Severity: Observation (favorable). Q&R’s SOPs are at or above the bar it applies to others. This is the expected standard for a second-line unit; noting it explicitly because the self-review could not certify it.

F-6 — Risk register posture and scoping are credible (Observation, favorable)

The register correctly scopes to operational/process risk only, routes adjacent risks (security, strategic, delivery, archive) to named owners, and explicitly lists those routed categories under “Risks owned elsewhere” so a reader cannot mistake omission for absence. The six open risks are each named as events (not vague consequences), rated with likelihood x impact, and assigned named owner Leads. R-003 (fan-in blind spot) is rated High despite the gate reading GREEN — demonstrating that Q&R did not allow a GREEN gate to suppress a known structural gap. This is the correct posture for a second-line risk register.

Severity: Observation (favorable).

Concurrence with previously disclosed open items

The following OI-1 through OI-5 items from the assurance sign-off are examined here only to confirm agreement or surface a divergence:

• OI-1 (machine-attest the gate): Concur. Per F-1 above, re-rate to Medium and promote to P0.
• OI-2 (staleness class): Concur. These are real but non-blocking. No new finding added.
• OI-3 (archive batch): Outside Q&R’s lane (Archivist). No finding added.
• OI-4 (fan-in CI complement): Concur; R-003 is correctly rated High and correctly kept Open.
• OI-5 / SR-1 (external review commission): This review closes SR-1 for this program cycle. Per F-3, it should be converted to a standing cadence, not treated as satisfied by this report alone.

Ranked risks (Q&R-specific)

Prioritized recommendations

Overall rating

Fit with conditions.

The Q&R unit has done real work: its SOPs are technically sound and pass their own four-test bar, its risk register is credible and correctly scoped, it independently discovered a Critical defect (C-1) that the program’s own gate missed, and it self-disclosed the independence conflict before being asked. These are the behaviors of a functioning second line.

The conditions are structural, not cosmetic. A second-line unit that cannot independently execute its primary gate proof, has no formalized response to persistently-open routed findings, and has no recurring independent oversight of itself is a unit whose independence is asserted rather than enforced. None of these conditions is hard to remedy; none requires significant effort. All three are known and deferrable — which is exactly the pattern the unit itself hunts in others.

The unit should address R1 and R2 before the next audit cycle. R3 and R4 convert a one-shot commission into a standing control.

External Review Partners — independent third-line reviewer, retained by the Principal. This report is read-only with respect to all org documents. Findings are routed as proposals to named owners. Routes to: Principal (primary) · COO (R1/R2/R3 ownership) · Q&R Director (for acknowledgment and response) · Archivist (for preservation).