How do I analyze the results of an A/B test?

Bring the best out of your results

Madeleine White avatar
Écrit par Madeleine White
Mis à jour il y a plus d’une semaine

You have used an A/B test and you would now like to see the first results? This article will explain how you can consult the results of an A/B test and how these can be exported, allowing you to analyse and validate your tests.

Have a look at our other articles in this section if you wish to understand how to create and launch an A/B test!

First, a visual approach to the Dashboard

You can find the visuals of your A/B test results anytime you want in the 'Analysis' section of the Dashboard. To do this, you just need to select a segment from the filter options.

Once the segment has been selected, all graphs on the page will be filtered to the chosen segment. For example, if you chose the segment 'Volatiles', all the statistics will concern this type of reader.

Now that you have selected a segment, you will be able to see that a new contour line is visible on all of the graphs. This represents the two scenarios that you have tested: A and B.

This method is very useful for quick and regular consultations of results or for observing a trend in a certain time period. You can now see whether one of the scenarios has clearly out-performed another.

This can also help you to work out if a test is nearing completion or not, an important date to be aware of.

Exporting data

In order to analyze your data more easily and accurately you can also export the statistics recieved from your A/B tests. In order to do this, choose the time period that you wish to analyze, as well as the segment chosen for this particular A/B test, and then click on 'Download'. You will be given a choice of format for the export. 

You can find your exports in the section 'Exports' on the Dashboard.

The exports show all statistics from the global performance and the action performance, but additionally the evolution of the segments. This allows you to see if the observed sample is enough to validate the test or not. 

Analze the statistics from the exports

When you created an A/B test, you were asked to determin the distribution of the scenarios by percentage of readers:

It is important to take this percentage into account when analyzing the results of an A/B test, especially if you haven't used an equal balance of the two scenarios (I.e. 50%/50%). To do this, use cross-multiplication.

For example, an result looking a bit too high or low may be due to an inbalance in the scenarios - That there is an under-representation of one of the scenarios in comparison to another. 

Analyzing your data will allow you to draw conclusions about the best scenarios to use for your website. These will give you the statistics you need to create new tests.

If you have any further questions, feel free to contact us via the Intercom chat!

Avez-vous trouvé la réponse à votre question ?