How to use it
- Enter visitors and conversions for variant A.
- Enter visitors and conversions for variant B.
- Calculate and interpret the p-value alongside practical impact.
Method and formula
Conversion rate = conversions ÷ visitors. Relative uplift = (rate B − rate A) ÷ rate A × 100. The p-value comes from a two-sided two-proportion z-test using the pooled conversion rate.
Worked example
A: 100 of 1,000 (10%). B: 130 of 1,000 (13%). Relative uplift is 30% and the two-sided p-value is about 0.0355, so the difference is significant at 95%.
Limitations
- The test assumes independent assignment and a stable conversion definition.
- It does not correct for multiple variants, sequential testing or peeking.
- Statistical significance is not the same as practical or revenue significance.