Launch Readiness Checklist
Verify each requirement before launching your experiment. Every item represents an industry-standard practice that helps ensure valid, reliable results.
0 of 12 completed
Question & metrics
Hypothesis is falsifiable and documented
Exactly one primary metric defined
1–3 secondary metrics documented
At least 2 guardrail metrics with degradation thresholds set
Sizing & duration
Target population and exclusions defined
Minimum Detectable Effect (MDE) justified by business impact
Sample size calculated and reviewed
Duration covers at least 2 full weeks
Decision & launch
Decision criteria pre-registered (ship, iterate, or revert)
A/A test passed this quarter
Rollout ramp-up plan documented
Experiment design peer-reviewed before launch
Recommended Statistical Defaults
| Parameter | Default | Range |
|---|---|---|
| Significance level (α): false positive tolerance | 0.05 | 0.01 to 0.10 |
| Statistical power (1 - β): detection probability | 0.80 | 0.80 to 0.95 |
| Minimum Detectable Effect (MDE) | Varies by metric and business context | Determined per experiment |
| Minimum experiment duration | 14 days | Non-negotiable minimum |
| Maximum experiment duration | 56 days (8 weeks) | Extend only with documented justification |
| Test directionality | Two-tailed | One-tailed requires pre-registered justification |
Beyond the theory
If you've got the theory down, see how it plays out in the simulator.
See the simulator