ab-test-design-rituals

Files (1)

• SKILL.md

SKILL.md (first lines)

# Ab Test Design Rituals

A/B tests answer questions. Run a test without a clear question and you'll have a clear non-answer.

## Hypothesis template

We believe [change] will result in [outcome] for [audience], measured by [metric]. We'll know we're right if [primary metric] improves by [X%] within [Y days].

## Sample size

Use a calculator (e.g., evanmiller.org). 5% baseline rate + 10% relative lift = ~7,000 visitors per arm.

## Test duration

Minimum 1 week (captures day-of-week variation). Don't peek and stop early.

## Test types

A/B (two variants). A/B/n (multiple variants — more samples required). Multivariate (combinations — even more).

## Common mistakes

Stopping early when 'looks good'. Multiple metrics without primary. Underpowered tests (false negatives). 'Winning' variant that doesn't ship (no cultural follow-through).

## Where this fits in X3 Compass

Applied across the X3 Compass design system.

From x3allamerican/design-skills-fleetsafety, Apache-2.0.

More from x3allamerican/design-skills-fleetsafety