Split testing, also known as A/B Testing, is a powerful way to really understand what works for your audience and get better results and ROI from your digital marketing campaigns.
It works off the methodology of sending different types of messaging to different people and seeing which performs better. So you choose a portion of your segment to include in the testing, and then send the better performing version to everyone else.
Split testing gets you better results from individual campaigns, and also helps you build up data that ensures better effectiveness long term.
As with nearly everything you do there’s ways to get better results and effectiveness from your split or A/B testing. This includes but is not limited to deciding which segments of your audience to include in the tests, how many people in those segments you should use, what to test and how long to run the test for.
We unpack some of those intricacies below.
Split Testing: An Overview
By now, you should be aware that segmentation matters. Very few of your sends are suitable to be sent to your entire database every time. The easiest way to describe segments is to use a car dealer example – the messaging you’d send to people who had test driven a car is very different to what you’d say to people who had bought one.
Split testing helps you take that segmentation even further to really get to know the behaviours of people within that demographic. You can test things like subject lines, content, call to action, scheduling, layouts, use of images and other design features and more. You shouldn’t test them all at the same time though. You need to be able to clearly see what worked and that gets blurred if you’re doing too much at once. Even if you see definite results, you won’t be sure what created them.
Most contemporary software applications like ActiveCampaign and Klaviyo make it really easy to conduct split testing. There are inbuilt features that virtually automate parts of the process and the details you take care of are easy. You just need to decide who to include in the test, and what you’re testing.
So What Percentage of Your Database Makes A Good Test?
Spoiler alert: it’s not as simple as just doing a 50/50 split. It’s also not too complicated either.
It’s not that you can’t literally split your segment or list down the middle, it’s that there’s more effective ways to do it, especially if one of your tests performs dramatically better than the rest.
That’s why the best split tests are for a significant but smaller portion of your database first – then you can still use the better version for the bulk of your database to maximise returns.
We recommend using 30% of your database in your testing. So you send 30% of recipients your two versions, and then the remaining 70% receive the better performing one once the test is complete. These figures can be adjusted, but we don’t recommend going more than 10% either way. 20% is the minimum amount of recipients needed for an effective Split Test, and 40% is the maximum you should aim for.
We work with 30% as a general rule because it’s a big enough sample size to be able to trust the results and see discernable differences in audience behaviour, while allowing a lot of room to reap the rewards if your test winner has the potential to bring big boosts in revenue or engagement to your campaign results.
Initially, you don’t need to be too concerned about how engaged your audience is to be included in your test. You never know – it could be that change you’re testing that re-engages someone who rarely opens your emails and that’s valuable data too. Though this is also a great time to think about your overall CRM hygiene and whether it’s time for a tidy up.
How Long Should A Test Run For?
This is one of those “how long is a piece of string?” style questions because it does depend on a few factors, and this is where your own data can come in handy.
Have a look at your previous campaign activity and explore when people interact with your content. Also consider what you want people to do – and when – once they’ve interacted with your send.
Unless you’re specifically testing a shorter time frame – and this can be beneficial at times. For example, to help you understand how to time your promotional activity for a flash sale. We wouldn’t advise jumping into the deep end with that kind of test for your first one though, or at least not without engaging assistance to make sure it’s done effectively.
Generally if you’re starting out with split testing, you want to give your test at least five to seven days to run.
To truly understand your database’s behaviour, you want to be able to see which days and times are most effective. You might learn that your database is most active on a Thursday, but that intel is lost if you only run the test for 12 hours on a Monday.
Don’t forget that the very point of testing is to get real data that you can use to make informed decisions for future campaign work. So if you want to get started with split testing straight away, here’s an exercise you can implement (or modify to suit your database):
- Choose a less time sensitive send – a message that still has relevance and value to recipients whether they get it immediately or in a week. For example, your monthly newsletter is just as meaningful if your audience gets it in the last week of the month prior as it is if they get it in the first week of the month it’s intended for. Letting people know that your new season stock has arrived is also less time sensitive than a restock that’s likely to sell out within days (or hours as some items do!).
2. Start small. Remember even if there’s multiple things you want to test, you don’t want to do it at once. A great way to get a feel for how split testing works and improve results include things like using two different subject lines (bonus points if you can add personalisation like a name to one of them).
3. Let your system do the heavy lifting for you. Many of the contemporary platforms like Klaviyo, ActiveCampaign and more have inbuilt split testing features that virtually automate the process for you.
4. Don’t forget to go back and analyse the results once the test and send has been completed.
Let’s Talk Split Testing Strategy
Do you want to have a chat about your current digital marketing campaigns and how Split Testing could make them even more effective? (Read: make you even more money). Book a complimentary strategy call with the Greenfish Marketing Team today. Reserve your time here.
