Skip to main content
All CollectionsStatistics
What methodology should be followed to create A/B tests with Poool Access?
What methodology should be followed to create A/B tests with Poool Access?

Some best practices to help you design your A/B tests

Flore Bayle avatar
Written by Flore Bayle
Updated over 3 months ago

Want to set up an A/B test but don’t know where to start?

In this article, we describe the main steps to follow in order to build a solid testing plan.

Methodology for building and conducting an A/B Test

To optimize your A/B test, it is important to document your process at each stage of your test.

The main points are the following:

  • Identifying a problem and formulating a hypothesis to address it

  • Defining the two variants, the duration, and the scope of the test

  • Choosing the KPIs for analysis

  • At the end of the period, comparing the performance of both versions

  • Selecting the option with the best results

  • Documenting the test (tested versions, duration, scope) to iterate on new hypotheses

Once your first A/B test is completed, don’t stop there. Develop new hypotheses and test again!

Some tips for successful testing

To obtain meaningful results, it is important to be mindful of various criteria.

Sample size

It is ideal to test variations on a large sample in order to obtain reliable results. To calculate the optimal sample size based on a given factor, several websites exist, including this one.

Identify your priorities

Often, when testing a component of a page in the hope of having a positive impact on the conversion rate, one can overlook a more obvious, bigger problem elsewhere.

For example, changing the wording of a button in a widget is a good test idea... but it’s clearly not enough, and could even be counterproductive if, behind it, the conversion funnel is not effective at all.

Think about identifying the elements to test/optimize as a priority for your audience (e.g., paywall placement on the page, length of subscription forms, number of steps in the funnel, attractiveness of offers, etc.).

Once these elements are under control, you can conduct more advanced tests.

Test duration

While you should let A/B tests run long enough to obtain reliable results, it’s important not to let them run for too long either. Cookie expiration or deletion, or the risk of Google deprecation: the negative impact of a test running too long can be real in terms of result reliability or SEO.

Consider external events in your analysis

Several external factors beyond the current A/B test may impact the use of your website (weekday vs weekend, trending news, type of content, etc.). Make sure to put the statistics in context when analyzing the results.

Once your A/B test is launched, monitor the results in the statistics tab. Check out our dedicated article on this topic.

Do not hesitate to contact us if you need any advice on conducting your A/B tests. Our teams will be happy to help 🙂

Did this answer your question?