PM Experiment Examples
(2026 Edition)
5 real-style A/B test examples with hypothesis, setup, metrics, and pre-committed decision criteria.
Build PM Experiment Skills Daily — Free →1. Onboarding simplification
Hypothesis
Reducing onboarding from 5 steps to 3 will lift Day-1 activation from 40% to 50%.
Setup
50/50 split, new signups only, 14-day run
Primary metric
Day-1 activation rate
Guardrails
Day-7 retention (don't lose quality), profile completion (can complete later)
Decision criteria
Ship if activation lifts >5pp AND guardrails within 2pp of baseline
2. Streak reminder timing
Hypothesis
Sending reminders at user's peak activity hour (vs fixed 9am) will lift reminder-driven returns by 30%.
Setup
A/B test on existing users with >3 days of history, 21-day run
Primary metric
Notification-driven return rate
Guardrails
Uninstall rate, notification disable rate
Decision criteria
Ship if return lift >20% AND uninstall/disable rates don't increase
3. Pricing page redesign
Hypothesis
Showing annual pricing first (vs monthly) will lift annual plan mix from 30% to 45%.
Setup
50/50 split, all pricing page visitors, 28-day run
Primary metric
% of conversions on annual plan
Guardrails
Overall conversion rate (don't drop total paid), refund rate (don't upsell unhappy users)
Decision criteria
Ship if annual mix lifts >10pp AND overall conversion doesn't drop >2pp
4. Search result layout change
Hypothesis
Showing 3 products per row (vs 4) on mobile will lift search-to-purchase conversion by 15%.
Setup
A/B test on mobile search results, 14-day run, 50/50 split
Primary metric
Search-to-purchase conversion rate
Guardrails
Session length (don't make browsing annoying), search-to-exit rate
Decision criteria
Ship if conversion lifts >10% without session length dropping >5%
5. Paywall placement
Hypothesis
Moving paywall from Day 3 to Day 7 will reduce free-user churn by 30% without hurting conversion.
Setup
50/50 split on new users, 30-day run to measure downstream effects
Primary metric
Free-user Day-14 retention
Guardrails
Paid conversion rate, LTV (delay doesn't reduce revenue)
Decision criteria
Ship if retention lift >20% AND paid conversion within 5% of baseline
5 Common Patterns Across Great Experiments
Every hypothesis is specific — 'X will move Y from A to B' — not 'this will be better'
Every test has primary + guardrails — protects against local wins that cause global losses
Decision criteria pre-committed — prevents mid-test rationalisation
Run length matches seasonality + statistical power — not arbitrary
Segment considerations noted — new users vs existing, mobile vs web
FAQ
How do PMs write good hypotheses?
A good hypothesis is specific and falsifiable: '[Change] will move [metric] from [baseline] to [target] because [reason].' If you can't name a specific metric or target, the hypothesis is vague. If there's no 'because,' you're not really predicting — you're just hoping.
Should every PM change go through an A/B test?
No. Tests cost time and complexity. Test when: stakes are high, you're uncertain about direction, the change is reversible. Skip tests for: obvious bugs, changes too small to detect, irreversible changes. The judgment of when to test is a PM skill in itself.
Practice Experiment Design Daily
Daily PM scenarios on hypotheses, metrics, and decision criteria.
Start Free Trial →