SOP-063 — Instrumentation verification (completeness matrix)

SOP · owner: Data & Analytics (Analytics Engineer). Layer: SOP/Runbook (docs/canon.md). The taxonomy being verified against is the DA Director’s (SOP-060) — not restated. Plan-phase / phase-gated note: no instrumented application code exists yet (ADR-0006); this is authored stub-first. Until a platform stands up, the DA Director runs it directly.

Interface Contract

Inputs: approved event taxonomy ← from Data & Analytics Director · trigger: briefing posted
Outputs: instrumentation completeness matrix → to Archivist · trigger: artifact approved
Handoff trigger (in): a taxonomy version is approved, or instrumentation is shipped and needs verifying
Handoff trigger (out): the completeness matrix is produced, gaps are ranked, and the report is ready for preservation
Definition of Done before handoff:
- [ ] Every taxonomy event recorded as present / missing / mismatched in application code
- [ ] Every code-side capture call recorded as matched / unmatched (an unmatched call = potential schema drift)
- [ ] Gaps ranked by severity (missing required event > wrong property name > wrong type > unknown event)
- [ ] Remediation actions written for the Platform Engineer (who owns the capture code)
- [ ] Report surfaced to the DA Director; learnings captured if any
- [ ] Approved against the frozen exemplar; handed to the Archivist (0 drift)

Purpose

Verify that the events firing in application code match the DA Director’s versioned taxonomy — correct names, required properties, types, cardinality, trigger context — so schema drift is caught before it corrupts historical comparisons, and the Platform Engineer gets a precise remediation list.

When to use

A new taxonomy version is approved, or the Platform Engineer ships instrumentation that needs checking against the taxonomy.

Flow

flowchart LR
  A["Obtain canonical<br/>taxonomy (SOP-060)"] --> B["Enumerate expected<br/>events + properties"]
  B --> C["Grep/Glob capture<br/>calls in app code"]
  C --> D["Build completeness<br/>matrix"]
  D --> E["Rank gaps by<br/>severity"]
  E --> F["Remediation →<br/>Platform Engineer"]
  F --> G["Report → DA Director<br/>+ Archivist"]
  classDef s fill:#1d4ed8,color:#fff,stroke:#1e3a8a
  class A,B,C,D,E,F,G s

Text fallback: obtain the canonical taxonomy (SOP-060) → enumerate expected events and properties → Grep/Glob every capture call in the application code → build the completeness matrix (each taxonomy event present/missing/mismatched; each code call matched/unmatched) → rank gaps by severity → write remediation for the Platform Engineer → report to the DA Director and hand to the Archivist.

Steps

Obtain the canonical taxonomy — the DA Director’s current versioned event list.
Enumerate expected events — name, required/optional properties, types, trigger context.
Check the application code (Grep/Glob) — for each capture call verify the event name matches exactly (case-sensitive, verb-object), required properties are present and correctly named, no extra properties, trigger context matches.
Build the completeness matrix — every taxonomy event: present / missing / mismatched; every code call: matched / unmatched (unmatched = potential drift).
Rank gaps — missing required event > wrong property name > wrong type > unknown event.
Write remediation for the Platform Engineer (precise field-level fixes).
Report to the DA Director; hand to the Archivist; capture any learning.

Roles / approvers

Run by the Analytics Engineer (or the DA Director while unhired). The Engineer verifies; the DA Director owns the taxonomy and any schema decision; the Platform Engineer owns and fixes the application capture code.

Guardrails

Verify, do not design — this is QA against the taxonomy; schema decisions are the DA Director’s.
No application code edits — write remediation; the Platform Engineer fixes.
Exact-match discipline — a one-character event-name drift is a reported defect, not a rounding.
No Principal personal data — PA-only.
Do not restate the taxonomy — reference the SOP-060 version.

See also: SOP-060 (event schema — the source of truth) · SOP-062 (pipeline upkeep) · docs/units/data-analytics/charter.md.