A/B Testing in Emails: Know Which Campaign Would Work The Best
September 20th 2021
Remote Sales
AB Testing
Today, emails are more powerful than they have ever been. Most businesses across the world believe email has been the channel generating the highest ROI for them. Customers too are in love with emails. According to Bluecore, 74% of Baby Boomers think email is the most personal channel to receive communications from brands, followed by 72% of Gen X, 64% of Millennials, and 60% of Gen Z.

But, the truth is - Everyone loves emails, but not everyone opens them! Here are the email stats across all industries:

email statistics for different industries

Source: Constant contact

In this article, we will focus on how the sales/marketing team can unlock the full potential of email marketing with an optimized strategy - A/B Testing. Let’s get started!

Table of contents:

  • What is A/B Testing?
  • Why Email Marketing?
  • How to perform A/B Testing?
  • Which elements to test during A/B Testing?
  • A/B Testing Best Practices
  • Start A/B Testing today

What is A/B Testing?

A/B testing in email marketing is the process of sending out two different versions of an email to two different segments of your list, with an ultimate goal of finding the winner email. This technique is also known as split email testing.

With A/B testing campaigns, you'll get access to useful insights that'll help you:

  • Understand your audience's behavior and interest
  • Increase your email open & click-through rates
  • Save money, time & resources

Here’s a quick overview of the working of A/B testing in email marketing.

AB testing in email marketing

Why Email Marketing?

Email marketing is the most sought-after communication channel used by businesses to reach customers. Even today, emails are considered highly relevant and effective as they help you reach large audiences at an affordable price. According to the year in review by Campaign Monitor, email marketing ROI is 4200% ($42 for every $1 spent). Unbelievable, isn’t it?

Email marketing also helps you sense the pulse of your audience. Your email open rate, engagement rate, and conversion rate are the key indicators of your brand perception. Here are 5 reasons why businesses love emails:

  • Allows targeted messaging
  • Strengthen relationship with customers
  • Improves brand awareness
  • Generate leads & boost sales/revenue
  • Affordable

In simple terms, email plays an important role in steering the business growth. A/B testing helps you focus on maximizing the impact of your email campaigns.

How to Perform A/B Testing?

Irrespective of the email automation tool you are using, A/B testing campaigns is super easy. Here's a 3 step mantra that will ensure you start-off on the right note when a/b testing emails.

steps for ab testing

a) Pick one Variable to Test

Every email contains several variables that could be tested - subject line, content, visuals, style, call to action and so on. But, to evaluate the effectiveness of your email campaign, it is always ideal to choose one independent variable and test it thoroughly. If two or more variables are tested simultaneously, then you cannot be sure of each variable's contribution to your email's performance.

b) Determine your Sample Size

Your sample size i.e your sample email list size is important to achieve statistically significant results. Smaller sample sizes often make it difficult to confirm your hypothesis.

If you have an email list with over 1000 subscribers, we recommend you stick to the 80/20 rule or the 50/50 rule. In the case of 80/20 rule, send out one variant of your email to 10% of your list and the other variant to another 10%. Depending on which variant performs best, send out that to the rest of the 80%. Alternatively, you could test your email variants on two sample lists having 25% of emails each and send out the winner variant to the rest of the 50%.

There is no hard or fast rule on the sample size. You can choose your sample size as long as it gives statistically significant results to determine the winner email variant.

c) Define your Success Metrics

Before you roll out your campaign, pick a metric that you want to improve. While you can measure more than one metric in every campaign, it is best to focus on one metric at a time. This will help you set clear goals on how much improvement you want to see.

For instance, here's a way to define your success metric - "I want to a/b test to improve my email open rate from 10% to 16%".

Some of the most commonly measured email metrics are open rate, click-through rate, unsubscribe rate, bounce rate, engagement rate, list growth rate, conversions.

Which Elements to Test During A/B Testing?

You can A/B test almost every element of your email to understand what interests or catches the attention of your audience. Here are a few ideas to help you get started:

1) Subject Lines

Your email subject line is the most crucial sentence of your email. In fact, OptinMonster says 47% of email recipients will open your email based solely on the subject line.

As a sales/marketing rep, you send out emails day in and day out. Imagine, what it is like to send out an email that your customers do not open? That is why identifying a subject line that strikes a cord with your audience can be very valuable for your email campaign.

Here are a few things to experiment with email subject lines:

  • Length: Keep your subject line short and catchy (around 60-70 characters)
  • Symbols & Numbers: Characters/statistics always capture the attention of readers
  • Personalisation: Adding the subscriber's name in the subject line helps you stand out from the spam emails
  • Urgency & exclusivity: Fear of missing out or the joy of exclusivity often drives users to act immediately

Facts/Best practices that come in handy while you are working on your email subject line:

best practices for email subject line

2) Content

A catchy subject line can only get your audience to open your email, the rest lies on the email body or content. It is your content that drives your audience to act on the Call-to-action (CTA), thus improving your conversion rate.

According to research by Microsoft, the average human attention span is 8 seconds. So, only a great email copy will help you get them hooked. Here are a few things you can test while you work on the copy:

  • Length: While it is good to keep your email copy short, sometimes it might backfire. Certain audiences prefer long-form content, while certain others prefer short-form content. Usually, short-form content is loved by audiences with smaller attention spans. They prefer straight-forward messages that are easy to read and digest. On the other hand, audiences who love story-telling and personalisation are impressed with long forms of content. It is always best to test and conclude what works for your audience.
  • Tone: The tone/voice of your email represents the personality of your brand. It is always good to adapt a consistent tone throughout your message as this creates a good perception of your brand. Further, it is said that a positive conversational tone engages the reader's brain in a very powerful way. In your next email, incorporate a positive tone and see its influence on engagement & conversion metrics.
  • Visuals: Visuals undoubtedly make emails super engaging. But having said that, it is not necessary to include visuals in every email. Remember that too many images or heavy sized images can hamper your user’s email experience and increase the bounce rate. So, making sure you strike the right balance of text:image ratio is a must.
  • CTAs and buttons: When it comes to CTAs & buttons, color, positioning, size and copy make all the difference. These elements directly influence the click-through rate of your buttons.
  • Let’s dive deep into understanding each of these elements

    • Color: You might think a red colored button is most ideal as it creates a sense of urgency and excitement. However, on the contrary, it might stimulate a negative vibe and prevent your audience from clicking on it. It is therefore suggested to choose neutral colors like blue, yellow, green or orange which are known to be happy & warm colors.
    • Position & size: You can place your button or CTA anywhere in the email as long as you give enough context for the audience to act upon it. Also, make sure that your CTA button text is legible and big enough to draw attention and is not obnoxious or threateningly large.
    • Copy: Combine action-oriented CTA copy with a sense of urgency to drive results. For example, ‘Get 25% discount today’ is more actionable than ‘Avail discount’. Similarly, ‘Try it out for free’ is more attractive than ‘Sign up’.

3) Design

First impression may not be the last impression, but it does matter a lot. This applies to emails as well. Email design is the first visual aspect that grabs the attention of your audience and hooks them on to reading further.

Although there are no exact guidelines to designing emails, here are a few things that'll give your email a structured look.

  • Adding a header with logo
  • Following your brand color scheme
  • Using relevant images, GIFs or animations in the body
  • Optimising for mobile first design
  • Appropriate CTA
  • Footer
  • Links to website/social media

4) Timing

Have you ever opened an email when you are in the middle of work or when you are off for a vacation?

Very likely you wouldn't have. Or even if you did, you would have opened it much later. That's exactly why timing is everything when it comes to sending out email campaigns. Emails that land up in the inbox on the right day at the right time have greater chances of being opened, read and acted upon.

According to Sendinblue, the best day for B2B businesses to send out their emails is either a Wednesday/Thursday and the best time is 8:00 - 10:00 AM or 2:00 - 3:00 PM. Also, here’s a table to give you specific insights on the optimal day and time for each type of business.

best time to send email


Also, when you are A/B testing email timings, keep in mind to test different time zones if your business caters to a global audience.

A/B Testing Best Practices

By now you must have realised that A/B testing is a continuous process. Meaning, it is hard to determine how many tests will help you understand your audience. As a marketing/sales rep, it is a must for you to test from time to time and stay on top of things.

Here are a few A/B testing best practices that will help you along the journey:

  • Test one variable at a time: This is a golden rule that one must follow. Most times when you are deciding which element to test, you end up picking more than one element. However, this choice might not give you the optimal results. It makes it difficult to understand the individual impact of each element. So from your first a/b test, remember to prioritize and test one element at a time.
  • Pay attention to sample size: It is important to get your sample size right. Testing your email variants on a relatively smaller audience might not give you statistically significant results. This makes it hard for you to measure the success and take data-driven decisions.
  • Here’s a simple tool to help you work out the ideal sample size - ABTasty

  • Reach out at the right time: Timing plays a critical role in determining the success of your email. You could get the content, design and everything else right, but if your timing is bad, then your email never gets read. It is best to plan your emails well-ahead and schedule them accordingly. Click here to read Coschedule’s research on the best time to send emails.
  • Build on your learning: Not every A/B test will give you great results. Some of them might show an increase in a particular metric, while some might show a decrease or even negligible effect. But the key is to use that knowledge while prioritising and testing emails in the future.

Start A/B Testing Today

A/B Testing is a powerful tool to get instant insights about your target audience. Quickly pick a variable, start formulating a hypothesis and test its potential impact on your email campaign’s performance.

That's all it takes for every business to leverage this technique and impact their success metrics. So, get started with A/B testing your email campaigns today!

Swathi Chintala
Contributing Writer
Swathi Chintala
Contributing Writer
Outbound Sales
How to A/B Test your Outbound Sales Campaigns?
Analytics and reporting are progressively becoming a central part of sales. Tracking sales metrics within the activities - such as ...
Upscale Team