Experiment statistics you can defend

That is what the second half of this page is for. A result arrives with the size of the difference, the interval around it, and how likely it is to be chance, computed two independent ways and shown side by side. Revenue is measured per variant rather than inferred from conversion rate, so a variant that converts more often and sells cheaper does not read as a win. And the platform states its own uncertainty rather than hiding it, which is the part that survives scrutiny: a result that is not yet conclusive says so, in words, instead of being presented as one that is.

No, and the design assumes you do not have one. Every figure carries its own explanation in place, written to say what that number is telling you rather than to define the term, and several of them exist specifically to head off a misreading: the difference column states outright that it says nothing on its own about whether the gap is real. Alongside the figures there is a recommendation in plain words. What it will not do is simplify by hiding, so somebody who does want the p-value and the interval will find both.

Three defences, and they are worth comparing against whatever you use now. Testing several variants at once raises the chance one looks like a winner by luck, so the bar is tightened to compensate, and the console states the threshold it is applying rather than leaving you to work it out. Looking at a result every day inflates the false-positive rate, so the test that is valid to read continuously is separate from the one that is not, and the platform will not stop an experiment early unless both agree. And before any of that, it checks that traffic actually split the way you configured it, because if it did not then nothing computed from it holds.

Sometimes yes and sometimes genuinely no, and this is where the platform is most useful before you have spent three months finding out. Every experiment reports the smallest difference the traffic you have can reliably detect. If the effect you are chasing sits below that floor, no amount of patience will resolve it, and the console says so rather than leaving a test at "not significant" indefinitely. Knowing which of your pages can answer a question and which cannot is worth more on a modest site than on a large one.

No. Both are computed for every experiment and shown together, so this is a reading preference rather than a configuration decision you can get wrong. The frequentist half reports how unlikely the result would be if the change did nothing. The Bayesian half reports the probability each variant is best, and what it would cost you to choose wrongly. Teams tend to find the second easier to act on and the first easier to defend, which is exactly why neither was dropped.