A/B Test Significance

Bayesian A/B test calculator: probability B beats A, credible intervals, expected uplift and risk.

Control (A)

Variant (B)

Credible level

Probability B beats A

92.4%

B is likely better, but not yet conclusive.

Posterior conversion-rate distributions

2.31%9.72%
Control (A) Variant (B)
Conversion rate — A (control)
5.09%
95% credible interval — A
3.82% – 6.53%
Conversion rate — B (variant)
6.59%
95% credible interval — B
5.14% – 8.20%
P(B > A)
92.4%
P(A > B)
7.6%
Expected uplift (B vs A)
+29.41% relative · +1.50% absolute
Expected loss if you ship B
0.04%
Edge Tools by Islam Answers
01:56 PM