How to Run A/B Tests on Email Subject Lines in Letterhead

This article explains how to set up and run an A/B test on email subject lines in your newsletters. A/B testing helps you automatically send the best-performing subject line to most of your audience, boosting open rates and engagement.

Before You Start

• You’ll need at least 400 audience members for A/B testing to work.
• A/B testing is currently available only for email subject lines (not preview text or newsletter content).
• The feature automatically splits your audience:
  • 25% receives the test (split between two subject lines).
  • 75% receives the winning subject line based on open rates.

Step-by-Step Instructions

1. Go to your Newsletter draft.

   • From the left-hand menu, select Newsletters.
   • Choose the draft you’d like to edit.

2. Proceed to the Preview & Schedule step.

   • Click Continue editing until you reach Step 2: Preview and Schedule.

3. Enable A/B Testing.

   • Toggle the switch next to Create A/B Test.

4. Enter your subject lines.

   • In the Email Subject fields, type two different subject line variations you want to test.

5. Set your send details.

   • Choose your Audience.
   • Pick your Date and Time to send.
   • (Optional) Send a test email before scheduling.

6. Schedule your newsletter.

   • Click Schedule.
   • Letterhead will send your subject line variations to 25% of your audience, track their open rates, and automatically deliver the winning subject line to the remaining 75% of recipients one hour after the A/B test begins.

Reviewing Results

After your newsletter is sent, you can view the results in the Newsletter Metrics dashboard:

• See both subject lines with full text.
• A “winner” badge will mark the best performer.
• Metrics include:
  • Opens (count)
  • Open Rate (%)
  • Recipients reached

Troubleshooting / FAQs

• What happens if my audience is smaller than 400 members?

A/B testing requires at least 400 recipients. If your audience is smaller, the toggle won’t be available.

• Can I test more than two subject lines?

Not at this time. Each test supports two subject line variations.

• Where can I see past results?

Go to your Published Newsletter metrics. Each newsletter will display the A/B Test Results section if it includes a test.

• When will the winning version be sent?

The winning subject line will be sent to the remaining 75% of your audience one hour after the test begins. Letterhead uses the first hour to measure open rates from the initial 25% test group before selecting and sending the winning version.

Need Help?

If you have questions or need assistance, please contact our support team at support@tryletterhead.com or log a support ticket through the Help Center!