How do you A/B test email subject lines?
TL;DR, the essentials
- Subject line A/B testing sends two subject lines to two random samples, then rolls the winner out to the rest of the list.
- The golden rule: one variable at a time, or you will never know what actually moved the needle.
- Aim for at least 1,000 recipients per variant and let the test run 3 to 7 days for a trustworthy result.
- Do not judge on the open rate alone: a clickbait subject inflates opens but sinks the click-to-open rate (CTOR).
The subject line is the first thing your recipient sees, often the only thing they read before deciding to open or ignore. It is the highest-leverage lever in all of email marketing: a handful of words that decide the fate of the whole campaign. Rather than guessing, subject line A/B testing lets your subscribers settle it for you. Here is what to test, how to run it, on what sample size, and how to read the results without falling into the classic traps.
What is subject line A/B testing?
A subject line A/B test (or split test) means writing two different subject lines for the same campaign, sending each to a small, randomly drawn slice of your list, and measuring which one earns the better open rate. The winning version is then sent automatically to all the remaining contacts who have not yet received the campaign.
You do not test to be right, you test to stop guessing. The winning subject line is not the one you prefer, it is the one your subscribers actually open.
The whole thing rests on a random draw: the two samples must be representative and comparable, otherwise the comparison means nothing. It is the same methodological base as A/B testing in email marketing applied to other elements, but focused on the single trigger of the open.
A/B or A/B/n?
Nothing forces you to test only two subject lines. Most platforms let you run three, four versions or more (this is called an A/B/n test). The catch: the more variants you add, the more volume each needs before the gap becomes reliable.
Why test the subject line first?
Because it has the best effort-to-impact ratio in your whole campaign. The subject line, the sender name and the preheader are the only things visible before the open. If the subject fails, all the work on the design, the message and the offer is never seen. A few extra points of open rate mechanically flow through to clicks and conversions.
A quick example sizes up the stakes. On a 20,000-email send, moving from 22% to 26% open rate, just four points, means 800 extra opens for a few minutes of copywriting. No other campaign setting offers that ratio. That is why teams who test almost always start with the subject line before tackling the content or the call-to-action.
What can you test in a subject line?
A subject line looks short, but it packs in about a dozen variables. Here are the ones that move the needle most, to be tested one at a time:
- Length. A study of 5.5 million emails found that 2 to 4 word subject lines hit roughly 46% open rates, with performance dropping beyond 7 words (indicative data, July 2026). Keep in mind that 33 characters or fewer is the only length that displays in full across mobile clients, and more than 60% of emails are now opened on mobile. Test a telegraphic version against a detailed one.
- Personalization. Inserting the first name or a contact data point (“Julie, your picks”) against a generic subject. Personalized subject lines are often credited with open rates around 46% versus 35% without, but not always, so verify it on your own list.
- Question versus statement. “Ready to save 30%?” against “Save 30% this week”. The question creates tension, the statement reassures.
- The number. A subject with a figure (“5 tips”, “-40%”) tends to catch the eye more, especially placed at the start where it stays visible. Some analyses report numbers driving noticeably more opens.
- The emoji. A well-chosen emoji can make the subject stand out in the inbox, but it can also look promotional. Test it, never generalize it blindly.
- Urgency. “Last hours” or “ends tonight” against a neutral tone. Effective now and then, wearing if you overdo it.
- Case. All-lowercase subject lines sometimes beat Title Case, while ALL CAPS drives people away and trips spam filters (avoid it).
One variable at a time, really
If you change the length, the emoji and the first name all at once between your two subject lines, a gap in open rate teaches you nothing: you cannot tell which of the three edits did the work. Always isolate a single parameter.
What method should you follow for a reliable test?
A clean subject line A/B test always runs in the same order. Here are the six steps to respect:
A clear hypothesis
Frame what you want to verify, for example “personalizing the first name raises the open rate”. Without a hypothesis, a test is just a bet.
One variable only
Create two subject lines identical in everything except the tested parameter. Keep the sender name and preheader strictly the same.
A random draw
The platform splits your samples at random so they are comparable. Never send one version in the morning and the other at night.
The same send window
Both variants go out in the same time slot, to neutralize the effect of the moment of day on the open.
A defined decision window
Set in advance how long the test runs before naming a winner, rather than calling it after two hours.
Sending the winner
The winning subject goes out automatically to the rest of the list. Most tools handle that final send for you.
Not every platform is equal at A/B testing
Our comparison ranks the tools that automate subject line split testing, from the random draw to the winner send.
What sample size and duration do you need?
This is where most tests go off the rails. A gap in open rate across 200 people proves nothing: it can be pure chance. For a usable result, aim for at least 1,000 recipients per variant (so 2,000 contacts in total for a standard A/B test). Below that, hold your conclusions with caution.
The most common split is the 20 / 20 / 60 scheme: 20% of the list gets variant A, 20% variant B, and the remaining 60% receive the winning subject once the test settles. On a small list where 20% is not enough to reach 1,000 contacts, raise the tested share, even if it shrinks the segment held for the winner.
On duration, let the test run 3 to 7 days rather than a few hours. People do not open their email at the same pace depending on the hour and the day: too short a window captures partial behavior and skews the verdict. Many tools compute a statistical significance and alert you when the gap reaches a 95% confidence level, the moment you can decide with confidence.
Indicative benchmarks, July 2026, aggregated from provider guides and email marketing platforms. Exact thresholds depend on your baseline open rate and the size of the gap you are chasing.
Small list, different strategy
Under 2,000 contacts, a “sample then winner” A/B test lacks volume. Instead, alternate subject line styles across several successive campaigns and watch the trend over a month. It is less rigorous, but more honest than over-reading 150 opens.
How do you read the results without fooling yourself?
The classic trap is to crown the subject with the highest open rate and stop there. But the subject line does not work alone: it sets an expectation the content has to meet. A clickbait subject can push opens up while disappointing, which drags down the click-to-open rate (CTOR).
A high open rate paired with a weak click-to-open rate is a warning sign: the subject over-promised, the content did not deliver.
So watch at least two metrics side by side: the open rate, which measures the subject’s pulling power, and the click-to-open rate, which measures whether the promise held. The truly winning subject is the one that attracts and delivers. Also check that the gap is statistically significant before concluding: two points of difference on a small volume can be nothing but noise.
Quick quiz
A subject line gets a very high open rate but a weak click-to-open rate. What does that tell you?
What mistakes should you avoid?
Subject line A/B tests almost always fail for the same reasons. Run through this list before you launch yours:
- Testing several variables at once. Length, emoji and first name changed together: unreadable result. One variable, one test.
- Stopping too early. Calling it after two hours on a fragile lead, before the gap stabilizes or reaches significance.
- A sample that is too small. Below 1,000 contacts per variant, chance weighs more than the subject line itself.
- Judging on the open rate alone. Ignoring the click-to-open rate risks rewarding a misleading subject.
- Not building on it. An isolated test teaches little. Log every lesson and build, over the months, a library of what works on your audience.
Mind the Apple Mail effect
Since Apple Mail Privacy Protection, part of the opens are triggered automatically with no human action, which artificially inflates the open rate. Apple Mail typically accounts for 40 to 55% of consumer opens in 2026. All the more reason to cross-check the winning subject with the click rate, which is far less sensitive to this bias.
What role does the email tool play?
Subject line A/B testing depends heavily on your platform. A good email marketing tool handles the random draw of samples, keeps the same send window for both variants, computes statistical significance and automatically sends the winning subject to the rest of the list. Some even offer an A/B/n test on several subjects and let you pick the winner on the criterion of your choice, opens or clicks. It is a criterion often overlooked when choosing a platform, even though it shapes how easily you will optimize every campaign.
The next step
Ready to take action? Compare the platforms that best automate subject line split testing in our guide to the best email marketing software 2026. To go further, see our article on writing a strong subject line, on A/B testing in email marketing more broadly. Find all our content on the email marketing hub.
Frequently asked questions
How many contacts do you need for a subject line A/B test?
Aim for at least 1,000 recipients per variant, so around 2,000 contacts for a standard A/B test. Below that, the gap in open rate is more likely to be chance than the subject line. For a smaller list, it is better to watch a trend across several successive campaigns than to conclude on a low volume.
How long should a subject line test run?
Count on 3 to 7 days rather than a few hours. People do not open their email at the same moment depending on the hour and the day, and too short a window skews the verdict. Many tools alert you as soon as a 95% confidence level is reached, the ideal moment to name the winner.
Should you test the subject or the content first?
The subject first. It is the only element visible before the open, alongside the sender name and preheader, and it offers the best effort-to-impact ratio. If the subject fails, all the work on the content is never seen. Once the subject is optimized, test the content and the call-to-action next.
Does an emoji in the subject line improve the open rate?
Sometimes, but not always. A well-chosen emoji can make the subject stand out in the inbox, but it can also look promotional and wear thin. There is no universal rule: this is exactly the kind of parameter to settle with an A/B test on your own audience.
Why not judge on the open rate alone?
Because a clickbait subject can inflate opens while disappointing, which drags down the click-to-open rate. The truly winning subject attracts and keeps its promise. So cross the open rate with the click-to-open rate, all the more since Apple Mail Privacy Protection has artificially inflated opens since 2021.