Skip to content

SOP-049 — Gate evidence

Guide

<!-- last-reviewed: 2026-06-01 · owner: Design Research (Design Research Director) -->

SOP-049 — Gate evidence

SOP · owner: Design Research (Design Research Director). Layer: SOP/Runbook (docs/canon.md). The phases and their gates live in docs/methodology.md; the phase-gate quorum/decision rules live in docs/sop/gate-quorum.md (SOP-005) — not restated here. State note (2026-06-01): gate review sits with CPTO + Principal (the Mentor role was offboarded 2026-05-31); evidence built on simulated studies carries a stated confidence penalty.

Interface Contract

  • Inputs: raw synthesis ← from UX Research Associate · trigger: raw synthesis delivered
  • Inputs: telemetry signal summary ← from Analytics Engineer · trigger: usage data ready
  • Inputs: seam resolution (pursue / research-pass / drop) ← from CPTO · trigger: review passed
  • Outputs: research finding / evidence (seam) → to CPTO · trigger: review requested
  • Outputs: gate-evidence report → to CPTO · trigger: gate evidence ready
  • Outputs: gate-evidence report → to Principal · trigger: gate evidence ready
  • Outputs: concept-de-risk evidence → to Innovation Director · trigger: gate evidence ready
  • Outputs: gate-evidence report → to Quality & Risk · trigger: review requested
  • Outputs: gate-evidence report → to Archivist · trigger: artifact approved
  • Handoff trigger (in): raw synthesis (or a telemetry summary) is delivered and interpretation can begin
  • Handoff trigger (out): the gate-evidence report is interpreted, the pursue/pivot/drop call is made, and it is delivered to CPTO + Principal before the gate
  • Definition of Done before handoff:
    • [ ] Findings synthesized by theme (affinity logic): observation / interpretation / implication, kept distinct
    • [ ] Confidence level stated on every finding; simulated studies say so with the penalty named
    • [ ] A clear pursue / pivot / drop recommendation against the gate question
    • [ ] Open questions and evidence gaps surfaced — gate blockers flagged immediately, not buried
    • [ ] Report written to reports/research-<YYYY-MM-DD>-<slug>.md
    • [ ] Learnings captured to docs/knowledge/ (and to docs/knowledge/research/ once the library exists)
    • [ ] Report handed to the Archivist (0 drift)

Registered triggers (raw synthesis delivered, gate evidence ready) — added to docs/handoff-triggers.md (Wave-2 block, Operations-approved 2026-06-01). usage data ready, review requested, and artifact approved already existed. The CPTO / Innovation / Quality & Risk / UX-Research / Analytics counterparties are not Interface-Contract identities here, so those edges reconcile as pending (§3.1.1 R3); the Principal edge is exempt (Principal) (§3.1 R2).

Purpose

Turn raw synthesis (or a telemetry summary) into the interpreted evidence a phase gate needs — a falsifiable report with a stated confidence level and a clear pursue/pivot/drop recommendation — delivered to CPTO + Principal before the gate.

When to use

Whenever raw synthesis is delivered by the UXR (or a telemetry summary by the Analytics Engineer) and a phase gate is approaching, or whenever Innovation needs interpreted evidence to de-risk a concept.

Flow

flowchart LR
  A["Receive raw synthesis<br/>/ telemetry summary"] --> B["Synthesize by theme<br/>(observation→interpretation→implication)"]
  B --> C["State confidence<br/>per finding"]
  C --> D["Make the call:<br/>pursue / pivot / drop"]
  D --> E["Write report +<br/>flag gate blockers"]
  E --> F["Deliver to CPTO + Principal<br/>before the gate"]
  classDef s fill:#1d4ed8,color:#fff,stroke:#1e3a8a
  class A,B,C,D,E,F s

Text fallback: receive the raw synthesis (or telemetry summary) → synthesize by theme using affinity logic (observation → interpretation → implication, kept distinct) → state a confidence level on each finding → make the pursue/pivot/drop call against the gate question → write reports/research-<YYYY-MM-DD>-<slug>.md and flag any gate blockers → deliver to CPTO + Principal before the gate review.

Steps

  1. Receive the synthesis. Take the UXR’s raw coding/affinity/tallies (or the telemetry summary). Do not re-do the legwork; interpret it.
  2. Synthesize by theme. Group by theme, not by participant. For each theme: observation (what was seen), interpretation (what it means), implication (what to do). Keep the three distinct — never present an interpretation as an observation.
  3. State confidence. Assign a confidence level to each finding grounded in the data (sample size, simulated vs. live, signal strength). A simulated study says so and names the penalty. Never report above what the data supports.
  4. Make the call. Against the gate question: pursue (evidence supports advancing), pivot (evidence redirects), or drop (evidence does not support).
  5. Surface gaps. Name open questions and evidence gaps. If evidence is insufficient to clear the gate, say so now — a gate blocker is flagged immediately, never buried.
  6. Write the report. reports/research-<YYYY-MM-DD>-<slug>.md: research question, method and participants, findings (by theme), confidence level, open questions, recommendation.
  7. Deliver and hand off. Brief CPTO + Principal (and Innovation, where relevant) before the gate; hand to Q&R for independent assurance (review requested); hand to the Archivist; capture learnings (into docs/knowledge/research/ once stood up).

Roles / approvers

Run by the Design Research Director — the sole authority on what the evidence means and whether it is sufficient for the gate (the UXR explicitly does not gate evidence). The CPTO + Principal review the evidence and the deliverable and make the advance/hold gate decision (SOP-005). Quality & Risk provides independent assurance on the report.

Guardrails

  • Evidence over opinion — report what the data says, including negative results; never inflate confidence.
  • Raw and interpreted stay distinct — observation, interpretation, implication are separable in the report.
  • Gate blockers are surfaced, not buried — an evidence gap that would let a gate clear on no evidence is flagged the moment it is known.
  • Personal data never enters a report — PA-only; the Archivist preserves no personal data.
  • Stay in your lane — you provide gate evidence; the gate decision is CPTO + Principal’s.
  • Do not restate docs/methodology.md phases or gate-quorum.md rules — link to them.

See also: SOP-048 (research plan) · SOP-005 (docs/sop/gate-quorum.md) · docs/methodology.md · docs/units/design-research/charter.md · docs/handoff-triggers.md.