Experimentation

Free A/B Test Significance Calculator: Two-Proportion Test

Compare two variants with a two-proportion z-test. Enter visitors and conversions for A and B to see conversion rates, relative uplift, p-value and whether the difference is significant at 95%.

✓ Private by default: calculations run in your browser and inputs are not uploaded.

Enter independent visitors and conversions for each variant. Conversions cannot exceed visitors.

Results

How to use it

  1. Enter visitors and conversions for variant A.
  2. Enter visitors and conversions for variant B.
  3. Calculate and interpret the p-value alongside practical impact.

Method and formula

Conversion rate = conversions ÷ visitors. Relative uplift = (rate B − rate A) ÷ rate A × 100. The p-value comes from a two-sided two-proportion z-test using the pooled conversion rate.

Worked example

A: 100 of 1,000 (10%). B: 130 of 1,000 (13%). Relative uplift is 30% and the two-sided p-value is about 0.0355, so the difference is significant at 95%.

Limitations

  • The test assumes independent assignment and a stable conversion definition.
  • It does not correct for multiple variants, sequential testing or peeking.
  • Statistical significance is not the same as practical or revenue significance.

Inputs and assumptions

More scenarios

A visitorsA conversionsB visitorsB conversionsRate ARate BUpliftp-valueSignificant
1,0001001,00013010%13%+30%0.0355Yes
100101001210%12%+20%0.6513No
1000100100%10%—0.0012Yes

Common mistakes

Sources and review

The result is a two-proportion z-test with a pooled standard error and a two-sided p-value. It is a statistical estimate, not a guarantee that B will outperform A in the future.

Reviewed October 22, 2026. This tool does not replace a pre-registered experiment design or a sample-size calculation.

FAQ

Common questions

What does the p-value mean?

It is the probability of seeing a difference at least this large if there were no real difference. It is not the probability that B is better or that the result will repeat.

Is 95% confidence always enough?

No. The acceptable risk depends on the decision. High-stakes changes may need stronger evidence and a guardrail review.

Can I test three or more variants?

This tool compares two variants. Testing several variants increases the chance of a false positive; use a multiple-comparison correction.

If the result is significant, should I ship B?

Not automatically. Check practical significance, revenue impact, guardrail metrics and whether the test was run as planned.

Should I use visitors or sessions?

Use the unit that was randomly assigned. If users were split, analyze users; sessions can be correlated within a user.

Does the calculator tell me how long to run the test?

No. It analyzes the data you have. Decide the sample size and duration before launching, and avoid peeking.

Methods and sources reviewed October 22, 2026

Found an error or have a suggestion? Tell us.

How we review formulas and sources