Your subject line decides whether an email gets opened. An A/B test sends two subject lines to a small sample of your audience and measures which performs better. SmartrMail then sends the winner to everyone else. You learn what your subscribers respond to before your whole list sees the campaign.
What to test
Change only one thing between A and B. If the offer and the tone both change, you won’t know which one won. Keep both lines honest. The winning subject line still has to match what’s in the email. Don’t run the same test once and call it a rule. A line that wins for a flash sale may not win for a product launch.
A: Save up to 30% on select products this weekend
B: This weekend’s sale: see what made the list
Other variables worth testing:
Discount format
A: 20% off everything, today only
B: Take $15 off your order of $75 or more
Urgency vs. no urgency
A: Last chance: our spring sale ends tonight
B: Our spring sale is on now
Personalization
A: Your weekend picks are here
B: {{first_name}}, your weekend picks are here
Question vs. statement
A: Ready to refresh your summer wardrobe?
B: Our summer collection just landed
Length
A: New arrivals
B: New arrivals: 12 pieces that just landed and are already selling fast
Emoji vs. none
A: Our best-selling sandals are back in stock
B: 👡 Our best-selling sandals are back in stock
Social proof
A: Our new water bottle is here
B: Our new water bottle is here, and 2,000 people already love it
Tone: direct vs. playful
A: Free shipping on orders over $50
B: Your cart called. It wants free shipping
Preparing your A/B Test
Start a new newsletter.
In the left-hand menu, turn on A/B testing.
Enter the two subject lines you want to test.
Set your test size. This is the share of your audience that gets the test, split evenly between both versions. A 20% test size sends subject A to 10% and subject B to 10%.
Choose when the test ends. The test window must be at least 3 hours after the test send.
Send or schedule the campaign. When the test window closes, SmartrMail sends the winning subject line to the rest of your audience.(Note: the Schedule A/B test option you see in the scheduling window is for testing sending times.)
Results
The losing subject line shows results for its test group only.
The winning subject line and the Total view show the full campaign, including the send to the rest of your list.
Get more from your tests
Give the test time to settle. A 3-hour window catches early openers. If your subscribers tend to open in the evening, a longer window gives a truer read.
Size the test to your list. On a smaller list, a handful of opens can swing the result. A larger test size gives each group enough recipients for the difference to mean something.
Read open rates with care. Apple Mail Privacy Protection can record opens that never happened, which inflates open rates. Check clicks and revenue too before you apply a test’s lesson to future campaigns.
Keep a log. Note each test and its winner. After five or six campaigns, you’ll see which framing your subscribers prefer.
Test your send time too
The subject line decides whether an email gets opened. The send time decides whether it gets seen at all. SmartrMail can also A/B test the best sending time for your campaigns. Run a send time test for at least two weeks to get a reliable result.