Skip to content

SOP-053 — Telemetry & feedback synthesis

Guide

<!-- last-reviewed: 2026-06-01 · owner: Design Research (Design Research Director) -->

SOP-053 — Telemetry & feedback synthesis

SOP · owner: Design Research (Design Research Director). Layer: SOP/Runbook (docs/canon.md). The Grow/Operate phases and the loop back to Ideate live in docs/methodology.md — not restated here. State note (2026-06-01): the product data layer is deferred (ADR-0006) and the Analytics Engineer is unhired, so no telemetry source exists today; this procedure is stub-first and runs once a live signal source is wired. It is the Grow/Operate half of the evidence spine — distinct from the gate-evidence procedure (SOP-049), which is the per-gate report.

Interface Contract

  • Inputs: telemetry signal summary ← from UX Research Associate · trigger: raw synthesis delivered
  • Inputs: in-market signal feed ← from Analytics Engineer · trigger: usage data ready
  • Inputs: research-grade field signal (a disengagement pattern needing a formal study) ← from <Customer Success> · trigger: field-feedback routed
  • Outputs: feedback synthesis → to CPTO · trigger: gate evidence ready
  • Outputs: next-cycle ideation input → to Innovation Director · trigger: gate evidence ready
  • Outputs: interpreted research findings → to <Product / Creative / Customer Success> · trigger: research findings issued
  • Outputs: feedback synthesis → to Archivist · trigger: artifact approved
  • Handoff trigger (in): a telemetry/feedback summary is delivered (Grow or Operate), or a retrospective is due
  • Handoff trigger (out): the synthesis is interpreted, the next-cycle implications are stated, and it is delivered to CPTO + Innovation
  • Definition of Done before handoff:
    • [ ] In-market / telemetry signals synthesized by theme (observation / interpretation / implication, kept distinct)
    • [ ] Confidence level stated; thin or simulated signals say so with the penalty named
    • [ ] Next-ideation-cycle implications stated explicitly (the loop back to Ideate)
    • [ ] No personal user data in the synthesis (aggregate/anonymised only)
    • [ ] Learnings captured to docs/knowledge/ (and docs/knowledge/research/ once the library exists)
    • [ ] Synthesis handed to the Archivist (0 drift)

Registered triggers (raw synthesis delivered, gate evidence ready) — added to docs/handoff-triggers.md (Wave-2 block, Operations-approved 2026-06-01). usage data ready and artifact approved already existed. The CPTO / Innovation / UX-Research / Analytics-Engineer counterparties are not Interface-Contract identities here, so those edges reconcile as pending (§3.1.1 R3) — clean for this wave.

Purpose

Turn in-market and in-operation signals (telemetry, user feedback, retrospectives) into interpreted evidence that improves the running product and seeds the next ideation cycle — closing the Standard Flow’s loop from Operate back to Ideate.

When to use

At the Grow and Operate phases: when a telemetry/feedback summary is delivered, when a retrospective is due, or when an in-market signal needs a structured read to inform the next cycle. Not used pre-launch (no telemetry source exists yet).

Flow

flowchart LR
  A["Receive telemetry /<br/>feedback summary"] --> B["Synthesize by theme<br/>(observation→interpretation→implication)"]
  B --> C["State confidence;<br/>label thin/simulated signals"]
  C --> D["State next-cycle<br/>implications"]
  D --> E["Deliver to CPTO +<br/>Innovation (next Ideate)"]
  classDef s fill:#1d4ed8,color:#fff,stroke:#1e3a8a
  class A,B,C,D,E s

Text fallback: receive the telemetry/feedback summary (from the UXR or the Analytics Engineer) → synthesize by theme (observation → interpretation → implication, kept distinct) → state a confidence level and label thin or simulated signals → state the implications for the next ideation cycle → deliver to CPTO + Innovation Director to seed the next Ideate.

Steps

  1. Receive the summary. Take the UXR’s raw telemetry signal summary (event, frequency, sequence context, anomalies) or the Analytics Engineer’s in-market feed. Do not redo the pull; interpret it.
  2. Synthesize by theme. Group signals by theme. For each: observation (the raw signal), interpretation (what it indicates), implication (what to change or build). Keep distinct.
  3. State confidence. Ground confidence in signal strength and source quality; a thin or simulated signal says so with the penalty named. Never over-claim.
  4. State next-cycle implications. Name what the signal means for the next ideation cycle — the loop from Operate back to Ideate is this procedure’s whole point.
  5. Deliver and hand off. Brief CPTO and the Innovation Director; hand to the Archivist; capture learnings (into docs/knowledge/research/ once the library exists).

Roles / approvers

Run by the Design Research Director — interpretation and the next-cycle call are the Director’s. The UXR and Analytics Engineer supply raw signals. CPTO and the Innovation Director consume the synthesis for the next cycle.

Guardrails

  • Evidence over opinion — report what the signals say, including negative results.
  • Confidence honesty — thin/simulated signals are labeled; no over-claiming.
  • Raw and interpreted stay distinct.
  • No personal user data — aggregate/anonymised only; PA-only for anything personal.
  • Stay in your lane — you interpret signals into evidence; you do not own the telemetry pipeline (Data & Analytics) or generate the next concept (Innovation).
  • Do not restate docs/methodology.md’s Grow/Operate phases — link to it.

See also: SOP-048 (research plan) · SOP-049 (gate evidence) · SOP-051 (usability protocol) · docs/methodology.md · docs/units/design-research/charter.md · docs/handoff-triggers.md.