You've setup an A/B test and now would like to see the first results? This article will explain how you can consult the results of an A/B test, allowing you to analyze and validate your tests outside of the Dashboard.
A/B tests analysis
You can find the visuals of your A/B test results anytime you want in the 'Analysis' section of the Dashboard. To do this, you just need to select a context, then a segment and finally the scenario where your A/B test is taking place (you can select this from the filter options).
The option to “Display A/B test results” will appear just below your filters.
Now that you have selected a second scenario, you'll be able to see that a new contour line is now visible on all graphs. This represents the two scenarios that you have tested: A and B.
This method is very useful for quick check-ins on your results or for observing a trend within a given time period, discovering whether one of the scenarios has out-performed another or not.
This can also help you to work out if a test is nearing completion or not, an important date to be aware of.
Analyze statistics from your Dashboard
To properly analyze your A/B test data, it is important to recall the points you defined beforehand:
Your initial hypothesis
A reminder of the two variants, the duration, and the scope of the test
Your analysis KPIs
Once your test results are compared to your KPIs, you will be able to make the right decision, either choosing the most effective version or extending the test if necessary.
It's important to take the percentage of audience distribution into account when analyzing the results of an A/B test, especially if you haven't distributed your audience equally (i.e. 50%:50%). You'll have to properly weigh the data before analyzing them.
💡Ideally, we recommend creating a test log to keep a record of the tested hypotheses (duration, scope, user journey, versions), as well as the recorded results.
And it doesn't stop there. Once a hypothesis is validated, let's test a new one!
Statistics in Your Own Analytics Tool
To complete your analysis, if you have connected Poool with your own Analytics tool (GA4, Piano, or others), you can find the performance attributed to your different versions.
Example of a Piano view:
💡To clearly analyze your data, we recommend labeling "Version A" and "Version B" in the title of your scenarios. This way, you can easily find this information in the performance reports in Piano or GA4.
Analyzing your data will allow you to draw conclusions about the best scenarios to use for your website as well as to create new tests and continuously optimize your strategy.
If you have any further questions, feel free to contact us via the Intercom chat!