How to A/B Test an Email

Through A/B testing, you can develop a stronger marketing strategy to reach your customers at the right time with the right message. Here's how to tackle a test in three simple steps.

Katie Caron
09/14/2018
Printer Friendly, PDF & Email
Adobestock A/B testing graphic
(Photo: aa_amie - stock.adobe.com)

According to research from eMarketer, the median email marketing ROI is 122 percent — four times higher than any other digital marketing channel. But, no business will ever reach that return unless they know what their email subscribers’ habits and preferences are. 

That’s where A/B testing comes in. This strategy involves sending two variations of the same email campaign to see which one gets the best response from subscribers. When testing over time, you’ll develop a better marketing strategy to reach your customers at the right time with the right message, which will likely lead to better sales. We spoke with digital marketing experts for tips on how to tackle a test in three simple steps. Here’s what they said.

1. Choose a variable to test and set your goal

First, brainstorm ideas for variables to test —  subject lines, send times, content layout, calls-to-action, etc. Kevin Tash, CEO and founder of digital marketing agency Tack Media, says it’s important to test just one variable at a time. 

“Some people might be testing color schemes, other people might be testing pieces of graphics and other people might be testing subject lines, which are the main drivers of higher email open rates,” Tash says. 

Chris Donald, President of Operations and Managing Partner at email marketing agency InboxArmy, says for small businesses with under 10,000 subscribers, it’s useful to focus more on email content and timing rather than subject lines. Donald explains that if you have a small subscriber list, it’s hard to get enough data on open rates to draw any valid conclusions. 

Next, define what a successful test will look like. For a subject line, your goal might be to improve open rates, while changes to your email’s contents might bring the goal of higher click-through rates and conversions to sales. Setting a clear goal and defining your metrics will help you run the most effective test possible. 

2. Split sample groups and send

If your email list is small, Tash says it’s best to split it in half and send one email variation to each so you can get enough relevant data. If your list is relatively large — over 10,000 or so — you can split a sample percentage of your email list — say 10 percent — to run the test and then send the more successful email to the rest of your subscribers.

Run the tests at the exact same time to make sure you aren’t introducing timing as a factor in the experiment. The exception to this is if you’re A/B testing the timing of the email itself.

Donald recommends giving emails 5-7 days — or at least 72 hours — to see proper results. 

3. Review results and take action

Finally, look at your results and identify the better-performing email. Take notes on what made that email successful and incorporate those findings into future campaigns. Take your test results with a grain of salt as well. 

Donald emphasizes that one successful test isn’t necessarily conclusive. 

“If you see that something works, you have to make sure you can repeat that success before you say ‘OK, this is the way we’re going to do it moving forward,’” he says. “If you have a win, then you need to repeat that win.” 

Ultimately, the more you test, the more you’ll know what a win looks like and what steps will get you there. If you can get in the habit of regularly testing and incorporating email marketing tactics, you’ll see higher open rates, click-through rates and conversions. The ROI is there when it comes to email marketing — it’s up to you to go and get it. 

leaderboard2 - default