Core Concept
Crafting a Testable Hypothesis
A well-formed hypothesis is the foundation of every experiment. It defines what you expect, why, and how you'll measure it.
A strong experiment hypothesis follows this structured template. Each part serves a specific purpose: the change you're testing, the outcome you're measuring, the direction you expect, and the reasoning behind it.
"If we [change], then [measurable outcome] will [direction], because [rationale]."
Real-world example
"If we replace the recommendation carousel with a personalized 'Continue Reading' shelf, then the 7-day reading session rate will increase by at least 3%, because users will find it easier to resume books they've already started."
What makes a hypothesis valid
- Falsifiable: the experiment must be able to prove it wrong. If no result could disprove it, it's not testable.
- Directional: state whether you expect the metric to increase or decrease, not just "change."
- Evidence-based rationale: ground your reasoning in user research, data analysis, or domain expertise, not assumptions or gut feeling.
- Pre-registered: write it before the experiment launches. Crafting a hypothesis after seeing results is called HARKing (Hypothesizing After Results are Known) and leads to false conclusions.
Common pitfall
Weak hypotheses to avoid: "We want to see what happens if we change X" is exploratory analysis, not a hypothesis. "This will improve engagement" is too vague. Which engagement metric? By how much? Without specifics, you can't determine success or failure.
Beyond the theory
If you've got the theory down, see how it plays out in the simulator.
See the simulator