Skip to content

SOP-051 — Usability study protocol

Guide

<!-- last-reviewed: 2026-06-01 · owner: Design Research (UX Research Associate) -->

SOP-051 — Usability study protocol

SOP · owner: Design Research (UX Research Associate). Layer: SOP/Runbook (docs/canon.md). The study design (question, method, instruments, exit criteria) is the Director’s (SOP-048) — this SOP is execution within a Director protocol, not its authorship. State note (2026-06-01): no Director protocol is filed and no participant seam exists; absent a live seam, sessions are simulated at a confidence penalty the Director states. The procedure is stranger-executable; its trigger is a real (or simulated) study.

Interface Contract

  • Inputs: research brief + protocol ← from Design Research Director · trigger: research brief issued
  • Inputs: participant scheduling (personal data) ← from <Personal Assistant> · trigger: personal-data scoped
  • Inputs: telemetry event data ← from Analytics Engineer · trigger: usage data ready
  • Outputs: raw synthesis (coding / affinity / tallies) → to Design Research Director · trigger: raw synthesis delivered
  • Outputs: study artifacts → to Archivist · trigger: artifact approved
  • Handoff trigger (in): the Director issues a study brief and protocol
  • Handoff trigger (out): clean, coded raw synthesis is delivered to the Director for interpretation
  • Definition of Done before handoff:
    • [ ] Logistics confirmed in advance (recruitment per criteria, consent collected, recording set up) — no session-day surprises
    • [ ] Session facilitated faithfully to the Director’s guide (verbatim prompts, no leading, silence held)
    • [ ] Out-of-scope signals noted as observations, not pursued
    • [ ] Transcript coded to the Director’s taxonomy; provisional tags flagged, not improvised silently
    • [ ] Affinity groups labeled observation-level (not conclusions); frequency tallies as code / count / % / verbatim
    • [ ] No interpretation, conclusion, or recommendation attached (the Director’s job)
    • [ ] Personal data kept PA-routed; none enters the synthesis or the archive
    • [ ] Learnings captured to docs/knowledge/ if execution surfaced one
    • [ ] Artifacts handed to the Archivist (0 drift)

Registered triggers (research brief issued, raw synthesis delivered) — added to docs/handoff-triggers.md (Wave-2 block, Operations-approved 2026-06-01). personal-data scoped, usage data ready, and artifact approved already existed. The participant-scheduling input is consumed via a placeholder-producer sink (the PA routes personal data, §3.1 R1); the Design-Research-Director and Analytics-Engineer edges reconcile as pending (not Interface-Contract identities here, §3.1.1 R3).

Purpose

Execute a moderated usability study faithfully within the Director’s protocol — logistics, facilitation, and raw synthesis — and deliver clean, coded raw output the Director can interpret without redoing the legwork. Expands the unit’s throughput so the Director is not a serial bottleneck.

When to use

Whenever the Director issues a study brief and protocol for a usability test (or a telemetry compilation task). Not used to design a study — only to run one already designed.

Flow

flowchart LR
  A["Receive brief<br/>+ protocol"] --> B["Logistics:<br/>recruit · consent · recording"]
  B --> C["Facilitate:<br/>verbatim · no leading · hold silence"]
  C --> D["Raw synthesis:<br/>code · affinity · tally"]
  D --> E["Deliver to Director<br/>(+ cover note)"]
  classDef s fill:#1d4ed8,color:#fff,stroke:#1e3a8a
  class A,B,C,D,E s

Text fallback: receive the brief + protocol → manage logistics (recruit against the Director’s criteria — personal data via the PA; collect consent; set up recording) → facilitate the session faithfully (read prompts verbatim, hold silence, never lead, note out-of-scope signals without pursuing) → produce raw synthesis (code to the Director’s taxonomy, observation-level affinity maps, frequency tallies) → deliver to the Director with a cover note. For the session-day checklist use JA-051 run-a-usability-session.

Steps

  1. Receive the brief. Read the Director’s protocol, research question, participant criteria, coding taxonomy, and expected output format. Confirm scope, timeline, and format; surface ambiguity to the Director — do not fill gaps by assumption.
  2. Manage logistics. Recruit against the criteria; route any personal contact/calendar need through the PA (personal-data scoped). Schedule. Collect consent. Configure recording (screen capture, notes template). Confirm everything in advance — session-day surprises are logistics failures.
  3. Facilitate. Run the Director’s guide exactly: verbatim prompts, hold silence, no leading. Capture verbatim quotes and observed behaviors in real time. Note out-of-scope signals as observations; do not pursue them. Never improvise the guide.
  4. Code the transcript. Tag every statement/observation with the Director’s taxonomy. A statement that does not fit gets a provisional tag, flagged to the Director — never an improvised canonical code.
  5. Affinity-map. Group coded observations by behavioral/topical similarity. Label groups observation-level (“paused before submitting form”), not conclusions. Preserve granularity.
  6. Tally. Count participants per coded behavior/theme. Present as a table: code / count / % of participants / illustrative verbatim quote.
  7. Deliver and file. Hand the raw synthesis to the Director with a cover note (what, which sessions, provisional tags, anomalies, logistics issues). File all artifacts to the repository per the Director’s naming/tag convention; hand to the Archivist; capture learnings.

Roles / approvers

Run by the UX Research Associate. The Design Research Director owns the protocol, the taxonomy, and all interpretation; the UXR executes and produces raw output only. The PA owns any personal-data recruitment step. The Analytics Engineer owns telemetry data infrastructure (UXR frames the pull).

Guardrails

  • Faithful, not directive — create conditions for authentic behavior; never steer. Confirmation bias at facilitation is contamination.
  • Raw, never interpreted — surface “7 of 9 paused here,” never “users are confused.” Interpretation and the gate call belong to the Director.
  • Never gate evidence — sufficiency for a gate is the Director’s call, not the UXR’s.
  • Never modify the protocol — no added questions, no extended sessions, no unauthorized probing.
  • Personal data is PA-only — recruitment routes through the PA; never accessed directly, never archived.
  • Do not restate the Director’s protocol — link to the brief.

Job aid: JA-051 run-a-usability-session. See also: SOP-048 (research plan) · SOP-049 (gate evidence) · docs/units/design-research/charter.md · docs/handoff-triggers.md.