Split test email campaigns
- Michael Kalnicki
Split testing your emails allows you to determine what type of emails or content work best for your audience. How people respond to your emails might differ based on what type of communication you send out and might not be consistent with responses that other organizations get. It is important therefore to test your emails to optimize your communication and make it work for your specific audience.
You can split test basing on a variety of content, for example:
Different subject lines
Length of emails
Different messaging
Different styling
Use of images
Personalization level
The idea behind a split test is to send two or more versions of the same email to a small sample of your intended audience to determine which email performs better and achieves better open, click through, and completion rates.
For example, you can send two different emails to 5% of the total list each and check the results before sending out the most effective email to the entire recipients list.
Creating a split test email
A Split Test campaign is created by going to Email > Create Email, and choosing Split Test Campaign. The email is built in the same way as a standard campaign (see Creating email campaigns), except you will end up with two or more messages within your single email campaign. You will see that you are on the first tab of the campaign (labelled “A”) representing that version of the email. Also the Name of your Email is appended with “A”
Once you create and save your first version “A”, you will see “+ Add another” next to the tab. If you click this, a new email message “B” is created, copying the settings from the previous version. You could then load in the content from version A via the “Duplicate from an existing email” drop down.
You can then edit version B to make any changes you want to test, e.g. the subject line.
You can add further versions if you wish, and you can have up to 8 versions.
Before you send out a split test
Once you create and save your split test, you should test them independently to make sure the formatting and content are correct. You can do this in the normal way via the test button (see this page for more information).
Sending a split test
After your ‘Split Test’ emails have been created and saved, you can proceed to the query builder to create your send group. Once the query has run, you will need to assign a percentage of the query audience to be tested.
The number entered will be divided among the emails you have created evenly.
For example, entering 10 means that both versions will be sent to 10% of the entire group you are sending to. If you have two split test messages, then that means 5% of the selected supporters will receive one message and 5% the other message.
To do a 50/50 test, enter 100 in the percentage box.
You will be able to send a test immediately or schedule the test for a specific date and time.
Reviewing the results
Once you run a split test, you will need to wait at least one hour for the software to collect the results and then you can decide which email template to send out to the remaining supporters on your recipient’s list.
To view the results of your test, go to Email > Manage emails and click on the name of the specific test on the list displayed on the page. Click the yellow icon to view the statistics.
You will be able to see the list of emails you sent out as part of your test and the individual reporting (open, click through rates) for each email.
Once you decide which email performed better, simply click the ‘Send as Winner’ button to the right of that email .
The emails will be sent to the initially selected audience, except for users who received the test. For example, if 20% of the group received the split test messages, then the remaining 80% will receive the winner.
When you send the winner, the selection list you send a split test to will not re-query – i.e. the original selection list chosen at the send, or scheduled send, is used for the entire process.
FAQs
Are split tests random? How do they work?
A split test sends to a randomised selection of supporters from the send group. It then sends in alternating order one at a time – i.e. Email A is sent, then Email B, then A, then B, and so on.
Can I automate the winner?
Yes, you can automate the winner selection based on criteria you define. For more information on enabling and setting this up, check the page about the email settings.
How do I use split tests results with the query builder?
The “Name of Email” field for each email message within a campaign is what is listed in the query builder under the Email criteria – i.e. the name that is appended with A or B.
If the winner was version B, then the original split test and remaining supporters would be included under that email name.
How many split tests can I make in one campaign?
Up to 8.