An A/B Split Test is the process of testing multiple options of a variable. It can be the copy, the price, or a creative like the banner.
For example, let us assume that you have developed a mobile game that has in-app purchases for additional levels.
You can test different banners for these levels to find out which designs are clicked the most.
Perhaps, you want to create video advertisements for 16-20 year males in the US to encourage them to install a new mobile game.
Can you be sure that your video will work? A better idea will be to test an advertisement on a small group and analyze the result.
You can A/B test with two videos and see which one performs best. The result will tell you which advertisement is likely to convert better.
Initially, A/B Split Testing was used for web pages to find the best variables to boost conversions – sales or leads. In a mobile-first market, A/B Split Testing is increasingly being used also for mobile marketing.
Testing is the best way to find out whether something is going to work. It is the best scientific way to decide.
Step-By-Step Guide For A/B Split Testing
This guide will show you how to split test for your mobile marketing –
1. Form Your Hypothesis
Hypothesis is a prediction or idea that you want to test. It must be clear and measurable. It is like an assumption.
Your subsequent test will either confirm or deny it. For example, you may assume that a blue background for your advertisement will work better.
2. Decide The Elements You Want To Test
There are so many elements that can be tested. They will all have varying effects on the performance of your advertisement. You can test –
- The ad copy
- Caption
- Headlines
- The brightness and color level
- Backgrounds
- Soundtrack and music
- Boxes and buttons
- Speed of the video
- Length of the video
Most of them refer to video advertisements because this is huge on mobiles.
3. Create Two Advertisements for Any One Element
You have to create two versions of the same element (headline, background, the ad copy, speed or length of the video).
For example, you can have two versions of the soundtrack if you want – a rock and a pop. Don’t make the mistake of making multiple changes.
Just work on one element at a time. Test, find the winner, implement the winner, and then you can test another element in a brand new A/B test.
4. Split Your Audience
Once you have decided what to test, it is now time to finalize the audience. Group A can be your control group.
The audience here will receive your default element that is the variation you think is most likely to succeed. Group B can be your test group.
Those here will receive the experimental variation. Test and control group testing is not a random experiment. A lot of thought and planning goes into making it a success.
Split your audience down the middle. In other words, 50% of the audience should see Version 1 and the other 50% should see Version 2.
But do make sure that the group sizes are large enough. Otherwise, you may not get accurate results. The result of you’re A/B test may not be accurate if the audience size is small.
5. Run Your Test
Make sure that you release both the versions simultaneously. Remember, you can only get dependable results if they both run together.
In other words, you cannot run Version 1 this month and Version 2 the next because you will never know if the results were different because of the two variations of the two months being different.
Not just the timing, the duration of the test will also matter. So, run both the variations for the same time – for a week or two weeks, depending on how many people are seeing them.
Facebook Business Center guidelines recommend that an A/B Split Test should run for a minimum of 4 days. For some businesses, it may have to run for more than a month. It all depends on the traffic.
You need to give your test enough time to ensure that the result will be statistically significant. A statistical significance of 95% is good. You don’t want to declare a winner with inadequate statistics.
6. Declare Your Winner
Your test is over once you have enough data for both the versions. It is now time to analyze the result and find out which version did better.
Sometimes, there will be a huge difference because one version has defeated the other by a massive margin. At other times, the difference will be small.
But whatever it is, you will have a winner. And if your test has run for a decent time, if both the versions have received enough traffic, and if the result is statistically significant, then you can depend on the finding.
7. Implement The Changes
Your A/B Split Test will tell you which variation performed better. For example, the test may reveal that “Install For Free” is more effective than “Install Now”.
So, you will want to implement the winning variation, the one that more people loved.
A/B Testing is a huge development in technology that is allowing developers and marketers to find out what will work best. It will surely help in mobile marketing.
But remember, you have tested only one of many possible elements (perhaps the background, headline, ad copy, speed or the length of the video).
That’s because an A/B Split Test will allow you to test only two variations of a single element at one time. There is so much more to test.
If you have tested the length of the video and found a winner, implement it, and then move on to the second element (perhaps the headline).
In this way, you have to keep testing different variables. In some instances, the control group may win and in other cases, it can be the challenger.
Remember to start with the element that seems the most important to you. Select one that may significantly improve the end result.
