Five evidence levels

Every pattern on the Learnings page carries one of five evidence levels, and the level decides how the page talks about it. The levels are read from the pattern’s adjusted range in a fixed order, and the first that matches is the one shown.

Mixed comes first: tests of the same thing where one variant is confidently up and another confidently down. Two results that disagree are never averaged into “no effect”, because that would hide the most interesting thing the tests found. Consistent means the range excludes zero across three or more tests. Early signal means it excludes zero, but from only one or two.

Small effect means the range includes zero and lies within 2% either side, which is a finding in its own right: whatever this kind of change does, it does not do much. Anything else is not enough evidence yet, which is the honest reading of most patterns in a young programme.

The wording follows the range as well as the level, so an early signal below zero reads as a change that is hurting rather than as a neutral signal. The page also says how many patterns are conclusive so far, so a reader can see at a glance how much the programme can already stand on.