A/B Test Designer

You are a data scientist specializing in experimentation and causal inference. Help me design a rigorous A/B test.

What I want to test: [HYPOTHESIS]
Primary metric: [PRIMARY_METRIC]
Current baseline: [BASELINE_VALUE]
Minimum detectable effect: [MDE]
Traffic/sample available: [TRAFFIC]
Platform/context: [PLATFORM]

Design the experiment:

1. **Hypothesis**:
   - Null hypothesis (H₀)
   - Alternative hypothesis (H₁)
   - One-tailed or two-tailed test and why

2. **Sample Size Calculation**:
   - Statistical power (typically 80%)
   - Significance level (typically 5%)
   - Calculated sample per variant
   - Expected test duration

3. **Test Design**:
   - Control vs. treatment definition
   - Randomization unit (user, session, page view)
   - Segmentation strategy
   - Guardrail metrics (what should NOT change)
   - Secondary metrics

4. **Potential Pitfalls**:
   - Simpson's paradox risk
   - Novelty/primacy effects
   - Sample ratio mismatch
   - Multiple testing correction (Bonferroni if multiple variants)
   - Seasonality effects

5. **Analysis Plan**:
   - Statistical test to use (t-test, chi-squared, Mann-Whitney)
   - Confidence interval reporting
   - Practical significance vs. statistical significance
   - Segmented analysis plan

6. **Decision Framework**: When to stop, when to extend, when to call it

7. **Documentation Template**: How to write up results