Statistics you can defend

Probability variant is best

Read it directly. Unlike a p-value, it is the answer to the question you actually asked.

Directly readableAcross all armsFeeds the recommendation

A p-value tells you how surprising your data would be under an assumption you do not believe. Almost everybody reads it as the probability the variant is better, which is not what it says. This figure is the number people think they are reading, computed properly.

It is the share of simulated outcomes in which this variant comes out ahead of every other arm, including the control. Because it is computed across all arms at once, the probabilities over an experiment sum to one, and a three-way test does not need a separate correction to be read.

It is what the console recommendation is built on. A variant at or above ninety-five per cent, whose frequentist result also clears its corrected threshold, is reported as a winner with high confidence. Above eighty per cent it is reported as trending, with the honest advice to keep running.

It pairs with expected loss rather than replacing it. A high probability with a large expected loss is a different decision from a high probability with a negligible one, and the second is usually what makes shipping easy.

Better experiments, better conversions

Test on all visitors with the world’s lightest script and make confident decisions powered by real-time reporting.