Learnings

Same-test patterns merged

When two patterns rest on the same tests, they are one piece of evidence seen twice, and the page treats them as one.

One per set of testsOverlap namedNo double counting

A test that changed the layout of a sign-up page counts as a layout test, a sign-up page test and perhaps a trust test. That is correct, and it is also a trap: three patterns appear, each looking like separate support for the others, when all three rest on the same result.

Learnings features one pattern per distinct set of tests, strongest first, so the same evidence is not presented as three findings at the top of the page. Where another pattern is built from exactly the same tests, the page says so in plain words: the same two tests also make up layout on the sign-up page.

The same rule runs through the statistics. The adjustment for multiple comparisons counts distinct sets of tests rather than patterns, and Ask about your learnings is told which patterns share their tests, so neither the page nor the assistant can present one result seen from two angles as two pieces of proof.

Better experiments, better conversions

Test on all visitors with the world’s lightest script and make confident decisions powered by real-time reporting.Start trialRequest a demo