Survey Results by Experiment Variant

Every time a survey is shown or answered, Pertento records which variant of each experiment the visitor was in at that moment. The By variant section of a survey's Results tab uses that record to split the survey's results by variant.

Surveys tied to an experiment

If the survey is tied to an experiment on the Targeting step, By variant opens straight away, split by that experiment's variants. See Choosing Who Sees a Survey.

Any other survey

For a survey that is not tied to an experiment, By variant starts closed. Use Compare by variant to open it, then split by any experiment the survey's visitors were in. If none of them was in an experiment, the section says so.

Choosing the experiment

The Experiment to split by list shows the linked experiment first, then every other experiment the survey's visitors were in, each with its status.

What the table shows

• Variant: the variant's name. The control is marked control.
• Shown: how many times the survey was shown to visitors in that variant.
• Answered: how many visitors in that variant answered.
• Response rate: answered divided by shown.
• One column for each score question, with its net score or average for that variant.

Each choice question gets a table of its own, with every option against every variant, showing the share and the count.

A variant that has since been removed from the experiment is shown as removed. Only visitors who were in the chosen experiment are counted; a visitor who never entered it is not placed in any variant.

Few answers

A variant with fewer than 30 respondents is marked few answers. Differences there are more likely noise than a real effect: a handful of answers is not a verdict.

What it does not do

The split is descriptive. There is no significance test and no winner. Use the experiment's own results to decide the outcome, and the survey to understand why it happened.

An experiment can tell you that variant B converted better, but not why. If the variant that converted better also scores much higher on ease, you have both the result and the reason for it. And a variant that wins on conversion while scoring worse on satisfaction is a warning rather than a win.