Skip to content

SOP-060 — Telemetry & event-schema procedure

Guide

<!-- last-reviewed: 2026-06-01 · owner: Data & Analytics (Data & Analytics Director) -->

SOP-060 — Telemetry & event-schema procedure

SOP · owner: Data & Analytics (Data & Analytics Director). Layer: SOP/Runbook (docs/canon.md). The flywheel context lives in docs/concept.md; analytics terms in docs/glossary.md — not restated here. Plan-phase note: no product data layer exists yet (ADR-0006); until the Plan→Build gate this authors the taxonomy stub-first against the planned platform.

Interface Contract

  • Inputs: measurement question / new feature or content type ← from self (DA Director) or Product / Innovation · trigger: briefing posted
  • Inputs: efficacy criteria / calibration / spec requirements ← from <issuing unit> · trigger: requirements issued
  • Inputs: learner health-score requirements (signals · thresholds · interventions) ← from <Customer Success> · trigger: health-score requirements issued
  • Outputs: versioned event taxonomy + telemetry spec → to Archivist · trigger: artifact approved
  • Outputs: analytics-term / schema-change notice → to Documentarian · trigger: document changed
  • Handoff trigger (in): a measurement question or new content type needs events defined
  • Handoff trigger (out): the taxonomy version is approved and ready for preservation; Engineering can implement the spec
  • Definition of Done before handoff:
    • [ ] Each event named verb-object (e.g. exercise_completed); names are case-exact and stable
    • [ ] Required + optional properties listed with types, enum constraints, and cardinality notes
    • [ ] The business question each event answers is named (no question → not authored)
    • [ ] Privacy annotation on every property; no Principal personal data (PA-only); retention deferred to Legal
    • [ ] Schema version explicit (v1, v2…); any change to a published schema flags the breaking surface and is coordinated with Engineering
    • [ ] Chunk-tag enum ([HOOK] [MAP] [SCOREBOARD] [RADIO] [DELTA] [PIT] [CTA]) controlled, not free text
    • [ ] No dollars (token counts only; usd_equiv is Finance’s)
    • [ ] Learnings captured to docs/knowledge/ if any
    • [ ] Approved against the frozen exemplar; handed to the Archivist (0 drift)

Purpose

Produce an authoritative, versioned event taxonomy and a telemetry spec Engineering can implement without ambiguity — so the platform measures the learning signals the flywheel needs (completion, quiz, rating, recall, time-on-task) and the schema is a durable contract rather than a permanent liability.

When to use

A new feature, content type, or measurement question needs events defined; or a published schema must change (a versioned, breaking-change procedure).

Flow

flowchart LR
  A["Name the<br/>business question"] --> B["Audit existing<br/>events / specs"]
  B --> C["Draft taxonomy:<br/>name · props · types · enums · cardinality"]
  C --> D["Privacy-annotate<br/>every property"]
  D --> E{"Changes a published<br/>schema?"}
  E -->|yes| F["Version + flag breaking<br/>surface; coordinate Engineering"]
  E -->|no| G["Version v1"]
  F --> H["Write telemetry spec<br/>→ Engineering"]
  G --> H
  H --> I["Hand to Archivist"]
  classDef s fill:#1d4ed8,color:#fff,stroke:#1e3a8a
  classDef d fill:#0f766e,color:#fff,stroke:#134e4a
  class A,B,C,D,F,G,H,I s
  class E d

Text fallback: name the business question → audit existing events/specs → draft the taxonomy (verb-object name, required/optional properties, types, enum constraints, cardinality) → privacy-annotate every property → does this change a published schema? (yes → version it, flag the breaking surface, coordinate with Engineering; no → version v1) → write the telemetry spec for Engineering → hand to the Archivist.

Steps

  1. Name the business question. What decision changes based on this event? No question → do not author the event (anti-vanity).
  2. Audit what exists. Grep/Glob/Read current event definitions and specs. Understand what is already tracked before proposing anything new.
  3. Draft the taxonomy. For each event: verb-object name, required properties, optional properties, types, enum constraints (use the controlled chunk-tag enum where relevant), and cardinality notes.
  4. Privacy-annotate. Tag every property’s privacy class; learner data is operational and never the Principal’s personal data (PA-only). Retention rules wait on Legal.
  5. Version explicitly. New schema → v1. Change to a published schema → bump the version, call out the breaking surface in plain words, and coordinate with Engineering before it ships. Never silently alter.
  6. Write the telemetry spec. Events, trigger conditions, payload fields, destinations (PostHog / warehouse), privacy constraints — precise enough that the Analytics Engineer / Platform Engineer can implement with no improvisation.
  7. Hand off. Approved taxonomy + spec to the Archivist; schema-change notice to the Documentarian; capture any learning.

Roles / approvers

Run by the Data & Analytics Director. Charter/manual approval routes to the CPTO; SOP approval is the Director’s own (owner’s Director per canon RACI — for a Lead, the cluster head). Engineering must agree to any schema change before it ships; schema conflicts escalate to the Principal.

Guardrails

  • The schema is a contract — versioned, breaking-change-aware, never silently changed.
  • Name the question or do not author the metric — anti-vanity.
  • No model/prompt tuning — that is AI Systems; DA defines what to measure.
  • No dollars — token counts only; usd_equiv is Finance’s.
  • No Principal personal data — PA-only; privacy-annotate everything; retention waits on Legal.
  • Do not restate docs/concept.md or docs/glossary.md — link to them.

Job aid: JA-060-event-schema-checklist.md. See also: SOP-061 (scoring & ranking) · SOP-063 (instrumentation verification) · docs/units/data-analytics/charter.md.