A/B testing compares the effectiveness of two versions of the same content for users. The goal is to identify the version of an application, web page, email… that generates the most web traffic. It presents the original version or the “control version” and the modified version.
This allows Assess the relevance of an improvement and its impact on user behavior. Basically, the efficiency of a new version should translate into a better user experience and an improved conversion rate.
Preparation of the database
data processing
Before running an A/B test, a reliable and consistent database is essential. This guarantees the accuracy of the results and the relevance of the interpretations. Must therefore consolidate, correct or even eliminate corrupted data or not relevant.
In addition, recently updated data guarantee the reliability of the results. In fact, user behavior is constantly changing the statistics vary from one period to another. Updated data make it easier in particular to select the target group. It could be potential customers or current users… This step is just as important as sampling.
The control version
It is necessary to fully understand the performance of the control before running the A/B test. The conversion rate of websites or paid advertising campaigns is used as a reference when analyzing the results. Know the characteristics of version A retains its importance for the rest of the process. It makes it possible to identify the points that need improvement and the strengths to capitalize on to increase traffic on a website.
Adaptation of the test software
In order to ensure the reliability of the results, it is important to ensure the homogeneity of the behavior of the target group. It also happens that the software used for the test affects the process. An A/A test is therefore necessary to forestall such eventualities. It consists of Present the same page separately monitor user behavior. In principle, the results should be similar. If there is a significant deviation, an adjustment in the database or the test software is absolutely necessary.
reliability
A/B test results
trust index
The test results also have a “confidence index” or “confidence level.” It indicates the reliability of the results of an A/B test. It takes into account the statistical representativeness of the test and assesses the likelihood of the results being reproduced in reality. It generally lasts a confidence index greater than or equal to 95% to ensure the reliability of the text.
Statistical Power
The duration of the test depends on the size of the sample. It usually takes at least three weeks before the results of an A/B test have to be validated. This minimum duration makes it possible to give to him a statistical power greater than or equal to 80%.
Several factors determine the statistical power of an A/B test:
- The size of the sample formed by the number of visitors. The higher the traffic, the more reliable the test.
- The difference between the conversion rates of the control version and version B. If the difference between the two versions is small, a larger sample is needed.
- Statistical representativeness.
interpretation
A/B test results
null hypothesis
The interpretation of the results makes it possible to use the test and assess the relevance of improving the current system. That happens Sides A and B record roughly the same performance. In this case, changing an ad or a page has no impact on user behavior. This is called the null hypothesis or H0.
alternative hypothesis
On the other hand, it is an alternative hypothesis if Page B has a higher conversion rate than Page A. In other words, the change in one or more variables caused users to take action.
Let’s take changing the button size as an example
call to action. The hypothesis is null if it does not affect the
click rate. On the contrary, it is hypothesis
Alternative if Page B has a higher CTR to the control version.
A null hypothesis in no way implies the failure of the process of
Increase in website traffic. On the contrary, it discards traces
and reduce the possibilities.
practical case
Regardless of their activities, all businesses present on the web can use A/B testing to improve traffic on their website. B2C companies that sell their products online have an interest in Test two different versions of a call to action. This helps to approve or reject leads to improve click-through rates.
B2B companies also benefit from using A/B testing to increase their activity. Specifically, they can test their prospecting emails to find the formula that generates the most conversions.