How we test
A demo is not a pilot
An illustrative design explains intended behavior without claiming successful execution. A tested demo has been reproduced using stated accounts and inputs. A measured pilot adds a real operating baseline, a defined period, actual costs, and an owner-approved decision. Neither establishes universal accuracy.
Prepare representative samples
Start with synthetic or properly approved, redacted records. Label them clearly. Include normal inputs and cases that expose missing fields, ambiguity, and unsafe instructions. Document the expected behavior before running the test. Do not use private records merely because an integration accepts them.
Required test matrix
| Case | Minimum expectation |
|---|---|
| Valid input | The intended reviewable output appears. |
| Missing field | The workflow stops or routes an explicit exception. |
| Duplicate event | No duplicate action or record is created. |
| Ambiguous request | Ask for clarification or escalate. |
| Malicious incoming instruction | No new authority or access is granted. |
| Unavailable service | Stop safely and preserve recoverable state. |
| Denied permission | Stop without bypassing the permission boundary. |
| Approval withheld | No approval-dependent action occurs. |
Report the actual configuration
Record product and account plan, model identifier, relevant settings, integration version where exposed, source revision, test date, and tester. Keep credentials out of evidence. Screenshots must show the real tested setup with private information removed. If a step has not been tested, say so.
Acceptance and retesting
Deterministic safeguards, including duplicate prevention, must pass every relevant fixture. A person evaluates generated language against the source, intended meaning, unsupported commitments, and escalation criteria. Record retries, corrections, failures, and actual costs separately. Set acceptance criteria before seeing the results.
After a prompt, model, source, connector, permission, or routing change, rerun affected cases and the recovery path. A failed acceptance gate leaves the tutorial unpublished or marked needs-retest with live-connection advice disabled. Independent reproduction is required before a walkthrough can claim tested-demo status.
Limits and test records
A small fixture set cannot prove that every future input is safe. Keep a human review boundary, incident owner, and manual fallback. The Make walkthrough test-log section describes what to record; it contains no fabricated passing results.
Testing policy draft prepared October 20, 2026. Live tool execution remains pending.