Email is easy to overlook in a test suite. Registration flows, password resets, OTP verification, and magic links are critical application paths, but email is genuinely harder to automate than most other parts of an application.

Unlike a database query or API response, email crosses multiple system boundaries asynchronously. There's no DOM to query, no response body to parse, and no native support in most testing frameworks.

This guide covers how to automate email testing, from choosing the right infrastructure to writing meaningful assertions, so your email workflows get the same level of test coverage as the rest of your application. If you're new to the topic, see our email testing overview first.

Why automating email tests is uniquely difficult

Email breaks the assumptions most testing tools are built on. Here are the three specific problems you'll hit.

Async delivery

Every other assertion in your test suite is synchronous: trigger an action, then check the result. Email doesn't work that way. After your application sends an email, the message travels through your ESP, reaches the receiving mail server, and arrives sometime later, usually within seconds, but occasionally longer.

A test that checks for the email immediately after triggering it will fail. A test that waits a fixed five seconds will be unnecessarily slow when the email arrives in one second, and flaky when it takes six.

Inbox isolation

If you use a fixed test address, such as [email protected], across your test suite, parallel test runs can interfere with each other. Test A triggers a password reset, test B triggers a signup confirmation, and both emails end up in the same inbox. Your assertions may read the wrong email, producing failures that are difficult to diagnose.

No interface to query

UI tests have a DOM. API tests have a response body. Email has neither. To assert on an email programmatically, you need an inbox that exposes its contents through an API, allowing your test code to retrieve messages, parse the response, and extract what it needs. Most email services weren't designed for this use case.

Each of these challenges has a straightforward solution, which the rest of this guide covers.

5 approaches to automated email testing

There are five ways teams typically approach automated email testing. They vary significantly in how much of the actual email pipeline they cover.

Real email providers

Using a real Gmail or Outlook account may seem like the most realistic option. Your test sends to an actual inbox and reads from it. In practice, these services aren't designed for automation. Email providers often treat automated access as suspicious activity, leading to CAPTCHA challenges, rate limiting, multi-factor authentication, or even account suspension.

One variation worth knowing about is email aliasing. Many providers support a + suffix, so [email protected] is delivered to the same inbox as [email protected]. Gmail supports this reliably, while support varies across other providers and is not standardized. It gives you unique addresses for manual spot checks, but it doesn't provide the API access needed for automated testing.

SMTP capture tools

Tools like Mailpit run a local SMTP server that captures all outbound email from your application. They expose captured messages through an API, making them easy to integrate into a test suite. This is a practical option for local development and CI because you can assert on email content without relying on an external service.

What it doesn't cover is actual delivery. Your application never connects to a real ESP, so transport-layer issues won't surface in your tests.

ESP sandboxes

Most major ESPs, including SendGrid, Mailgun, and others, offer a sandbox or test mode that suppresses delivery while logging what would have been sent. These messages are accessible through the provider's API.

The tradeoff is similar to SMTP capture. It's a good way to assert on email content, but it doesn't test delivery through the real sending pipeline.

Backend workarounds

Some teams bypass the email layer entirely by exposing a test-only API endpoint that returns the OTP or confirmation link directly, or by configuring a static OTP that always passes in test environments. This is a practical shortcut and better than having no coverage at all.

What it doesn't verify is whether the email was actually sent, delivered, and formatted correctly. A working internal endpoint and a broken email template can easily coexist without either exposing the problem.

Programmable test inbox

A programmable test inbox is the approach that covers the full pipeline. It accepts real emails and exposes them through an API, allowing your test to trigger a send, retrieve the message, and assert on its contents, all in code and without manual steps.

The key to making this work at scale is generating unique inboxes. Instead of managing a fixed set of test addresses, you create a new inbox address for each test run. With testmail.app, every inbox follows the format {namespace}.{tag}@inbox.testmail.app. Your namespace is fixed to your account, while the tag can be anything, such as a test ID, timestamp, or UUID.

testmail.app exposes two ways to query that inbox: a simple JSON API (used throughout this guide) and a more capable GraphQL API. GraphQL adds server-side filtering by fields like subject or sender, custom sorting, and the ability to request only the fields you need instead of the full email object. Start with the JSON API; reach for GraphQL once a test needs that extra filtering power.

Each test gets its own isolated inbox with no setup required, so parallel test runs never interfere with one another.

Approach	Tests real delivery	API access	Parallel-safe	CI-ready
Real email provider	Yes	No	No	No
SMTP capture	No	Yes	Yes	Yes
ESP sandbox	No	Yes	Yes	Yes
Backend workaround	No	Yes	Yes	Yes
Programmable test inbox	Yes	Yes	Yes	Yes

The anatomy of an automated email test

Regardless of which framework you use, every automated email test follows the same five steps. Understanding this structure before looking at the code makes the implementation straightforward.

1. Generate a unique inbox address

Before anything else, create the address that will receive the email. Generate it at the start of the test so you have it ready before triggering any action. With the namespace and tag system, this is a single line:

const tag = `signup-${Date.now()}`;
const testEmail = `${NAMESPACE}.${tag}@inbox.testmail.app`;

Tags don't need to be registered in advance. You can generate them on the fly. Your namespace acts as a wildcard that accepts any email sent to {namespace}.*@inbox.testmail.app, so any tag you create is immediately valid.

2. Trigger the action that sends the email

Submit the generated address through your application, either via the UI or directly through the API. This is the same step you'd perform in any other end-to-end test: fill out a form, click a button, or send an HTTP request.

3. Fetch the email via API

Query the inbox for the message. The key is that you don't check once and move on. Instead, you wait for the email to arrive. How you handle that waiting is what separates reliable email tests from flaky ones. The next section covers this in more detail.

4. Assert the email contents

Once the email arrives, verify everything that matters: delivery metadata, subject line, sender address, body content, links, and authentication results. What to assert, and how to do it effectively, is covered in the following section.

5. Use the extracted data to continue the test

For verification flows and OTP tests, the email isn't the end of the test. It's a step in the middle. Extract the link or code from the email body and use it to continue the flow by following the verification link, submitting the OTP, and confirming the expected outcome.

That's the complete workflow. The rest of this guide covers each step in more detail.

Email test assertions: what to check and why

The email/API assertions in this section use Chai's expect syntax for readability. In a Playwright test, import Chai separately for these assertions and keep Playwright's built-in expect for page assertions:

import { test, expect as playwrightExpect } from '@playwright/test';
import { expect } from 'chai';

Delivery

Start with the basics: did the right email arrive, from the right sender, within a reasonable time window?

expect(email.to_parsed[0].address).to.equal(testEmail);
expect(email.from).to.equal('Acme Inc <[email protected]>');
expect(email.subject).to.equal('Verify your email address');

The field names above, including to, from, subject, and the others used throughout this guide, reflect testmail.app's JSON API response. If you're using a different programmable inbox provider, the schema will differ, but the underlying assertions remain the same.

The From assertion catches a surprisingly common class of bugs: sender address regressions after ESP migrations or configuration changes. These issues often go unnoticed until they trigger spam filters or break DMARC alignment.

Note that to and from are parsed from the message headers, which the sender controls and can, in principle, spoof. If a test needs the harder-to-spoof address, use envelope_to/envelope_from, which reflect the actual SMTP envelope instead.

Content and dynamic fields

Check that the expected static copy is present and that dynamic fields have been resolved correctly. Unresolved personalization tokens, such as Hello {{first_name}} appearing literally in a sent email, are a common failure mode and are easy to catch with automated assertions.

expect(email.text).to.include('Hi Jane');
expect(email.html).to.not.include('{{');
expect(email.html).to.include('your order #84729');

For OTP and verification flows, extract the token or link from the body:

const link = email.html.match(/href="(https:\/\/yourdomain\.com\/verify\?token=[^"]+)"/)[1];
expect(link).to.include('yourdomain.com/verify');

Links

Extract every URL from the email and verify two things: that it points to the correct environment and that it resolves successfully. A staging URL in a production email is a common, high-impact bug that is easy to ship and difficult to spot through manual testing.

const urls = [...email.html.matchAll(/href="(https?:\/\/[^"]+)"/g)].map(m => m[1]);
for (const url of urls) {
  expect(url).to.include('yourapp.com'); // not staging.yourapp.com
  const res = await fetch(url, { method: 'GET', redirect: 'follow' });
  expect(res.ok).to.equal(true);
}

Note: Some ESPs and redirect services block HEAD requests. Use GET requests with a redirect following to verify that a URL resolves reliably. It's slightly slower, but it produces more accurate results.

Authentication

SPF and DKIM failures are among the most common reasons emails end up in spam, and they often fail silently. DNS changes, ESP migrations, or updates to sending domains can all introduce problems without obvious warning.

testmail.app returns both results directly on the email object, so adding these assertions to your CI pipeline takes very little effort:

expect(email.dkim).to.equal('pass');
expect(email.SPF).to.equal('pass');

If either of these starts failing in CI, you want to know immediately, not after deliverability drops in production.

For headers not exposed as dedicated fields, such as a custom List-Unsubscribe or Reply-To, request the full header list instead. In the JSON API, add &headers=true to your query; in GraphQL, just include the headers field:

const replyTo = email.headers.find(h => h.key === 'reply-to');
expect(replyTo.line).to.include('[email protected]');

Header keys are returned lowercased; line holds the original, unmodified header text.

Spam score

The spam_score field gives you a numerical score generated by testmail.app, which runs each received message through the SpamAssassin ruleset and exposes the result on the email object. It's not a field that's available from every email testing provider.

The default spam threshold is 5; scores below that are generally considered safe, above 5 indicate a high probability of being filtered. Asserting on this during development catches content and template issues before they affect real recipients.

When you include spam_report=true in your API request, the response includes both the score and a detailed breakdown of which rules were triggered:

{
  "spam_score": 1.158,
  "spam_report": "Spam detection software, running on the system \"inbox.testmail.app\", has\nidentified this incoming email as possible spam.  The original message\nhas been attached to this so you can view it (if it isn't spam) or label\nsimilar future email.  If you have any questions, see\n@@CONTACT_ADDRESS@@ for details.\n\nContent preview:  Hello there! Here is a test html message. hello world! [...]\n   \n\nContent analysis details:   (1.2 points, 5.0 required)\n\n pts rule name              description\n---- ---------------------- --------------------------------------------------\n 0.8 HTML_IMAGE_RATIO_02    BODY: HTML has a low ratio of text to image area\n 0.0 HTML_MESSAGE           BODY: HTML included in message\n 0.3 HTML_IMAGE_ONLY_04     BODY: HTML: images with 0-400 bytes of words\n 0.0 T_MIME_NO_TEXT         No text body parts\n\n"
  // ...rest of the email
}

The spam_report field is where the real value is — when a score is unexpectedly high, it tells you exactly which rules fired and by how much, so you know what to fix.

// spam_score requires &spam_report=true in your API request
const res = await axios.get('https://api.testmail.app/api/json', {
  params: {
    apikey: API_KEY,
    namespace: NAMESPACE,
    tag: tag,
    timestamp_from: startTimestamp,
    livequery: true,
    spam_report: true,
  },
});

const email = res.data.emails[0];

expect(email.spam_score).to.be.below(5);

Note that spam scores are not universal — each mail provider configures their own thresholds and rule sets. Use these scores as a signal rather than a guarantee of inbox placement.

Attachments

Not every email test needs to check attachments, but for flows like invoices, receipts, exported reports, or signed documents, the attachment itself is often the actual thing being verified, not just the email around it.

At a minimum, confirm that the expected attachment is present and named correctly. Filter out inline assets first, since embedded images (like a logo in an HTML template) show up in the same attachments array and will throw off a naive count:

// Filter out inline assets (e.g. embedded logos) before counting
const realAttachments = email.attachments.filter(a => !a.related);

expect(realAttachments.length).to.equal(1);
expect(realAttachments[0].filename).to.equal('invoice-84729.pdf');
expect(realAttachments[0].contentType).to.equal('application/pdf');

For a basic sanity check, you don't need to download the file at all. size (in bytes) and checksum are already on the attachment object:

const attachment = realAttachments[0];
expect(attachment.size).to.be.above(0);

When the attachment's actual content matters, not just its presence, download it and compare against the reported size to catch a truncated or corrupted file:

const fileBuffer = await downloadAttachment(attachment.downloadUrl);
expect(fileBuffer.length).to.equal(attachment.size);

One quirk worth knowing: if an email embeds images using cid: references (common for logos in HTML templates), testmail.app automatically inlines them into the html field as base64 data: URIs rather than leaving them as separate attachments. So checking email.html.includes('data:image') is often the right way to confirm a logo rendered, rather than looking for it in the attachments list.

A few things worth checking, depending on your flow:

• File type mismatches. A PDF generator that silently falls back to a broken or empty file, or an incorrect contentType on an otherwise correct file.
• Size sanity checks. An attachment that's suspiciously small (a few bytes) often indicates a generation failure even if the filename and content type look correct.
• Multiple attachments. If a flow can send more than one file, assert on the full set (count and each filename) after filtering out inline assets, not just the first item in the array.

The field names above (filename, contentType, size, checksum, downloadUrl, related) reflect testmail.app's schema and will differ for other providers. One testmail.app-specific caution: each API response is capped at 10MB, so if a test queries several emails at once with a high limit, and those emails carry large attachments, keep the limit low to avoid a truncated response.

Waiting for email delivery in CI

Async delivery is where most automated email tests fall apart. The fix is straightforward once you understand why the naive approaches fail.

Fixed sleep

The most common first attempt:

await page.fill('#email', testEmail);
await page.click('#signup');
await sleep(5000); // wait for email
const email = await fetchEmail(tag);

This fails in both directions. When the email arrives in two seconds, your test waits an unnecessary three. When the pipeline is under load and delivery takes six seconds, your test times out. Fixed sleeps make your suite slower on average and flakier under pressure.

Polling with a fixed interval

An improvement: check for the email repeatedly until it arrives, or a timeout is reached.

const poll = async (tag, timeout = 30000, interval = 2000) => {
  const start = Date.now();
  while (Date.now() - start < timeout) {
    const res = await fetchEmail(tag);
    if (res.count > 0) return res.emails[0];
    await sleep(interval);
  }
  throw new Error('Email did not arrive within timeout');
};

Better than a fixed sleep, but you're still making repeated API calls, and the test continues later than it needs to; you'll always wait at least one full interval after the email arrives. Tight polling intervals also risk testmail.app's API rate limits — free plans are capped at 1,000 requests/hour, and on paid plans, an API key sustaining more than 5 requests/second for an hour is temporarily blacklisted. This is one more reason livequery, covered next, is the better default.

Livequery

The recommended approach. This is a testmail.app-specific API parameter, not a standard email testing feature. If you're using a different programmable inbox provider, check whether it offers an equivalent blocking query or whether you'll need to implement your own polling mechanism. Add livequery=true to your API request and the call blocks until a matching email arrives, returning immediately once it does. No polling in your test code, no unnecessary waiting.

Under the hood, testmail.app implements this long-poll using HTTP 307 redirects. One important detail worth knowing: after one minute of waiting with no match, the API redirects to itself, and the request restarts.

Each redirect resets the HTTP client's timeout, so the timeout must be set at the test suite level, not the HTTP client level. Most testing frameworks have a per-test timeout you can configure. Five minutes is a reasonable starting point — enough to cover the vast majority of delivery scenarios. Setting it too low causes intermittent build failures on edge cases; setting it too high means slow feedback when something is genuinely broken.

Complete example: testing a signup flow with Playwright

The five steps map directly onto any testing framework. Here's a complete end-to-end implementation in Playwright: a signup flow that verifies the confirmation email and completes the verification step:

// 1. Generate a unique inbox address
const startTimestamp = Date.now();
const tag = `${process.env.GITHUB_RUN_ID || 'local'}-signup-${Date.now()}`;
const testEmail = `${NAMESPACE}.${tag}@inbox.testmail.app`;

// 2. Trigger the action
await page.fill('#email', testEmail);
await page.fill('#password', 'testpassword123');
await page.click('#signup');

// 3. Fetch the email via livequery
const res = await axios.get('https://api.testmail.app/api/json', {
  params: {
    apikey: API_KEY,
    namespace: NAMESPACE,
    tag: tag,
    timestamp_from: startTimestamp,
    livequery: true,
    spam_report: true,
  },
});

// 4. Assert email contents
expect(res.data.count).to.equal(1);
const email = res.data.emails[0];

expect(email.from_parsed[0].address).to.equal('[email protected]');
expect(email.from_parsed[0].name).to.equal('Acme Inc');
expect(email.subject).to.equal('Verify your email address');
expect(email.html).to.not.include('{{');
expect(email.dkim).to.equal('pass');
expect(email.SPF).to.equal('pass');
expect(email.spam_score).to.be.below(5);

// 5. Extract the verification link and complete the flow
const verificationLink = email.html.match(
  /href="(https:\/\/acme\.com\/verify\?token=[^"]+)"/
)[1];
expect(verificationLink).to.include('acme.com/verify');

await page.goto(verificationLink);
await playwrightExpect(page.locator('.success-message')).toContainText('Email verified');

For complete implementations in other frameworks, see the JSON API examples and GraphQL API examples in the testmail.app docs, or the testmail.app blog for more framework-specific guides.

Writing reliable email tests

Set up your inbox address and timestamp_from together

testmail.app creates inboxes on the fly — any email sent to {namespace}.{tag}@inbox.testmail.app is automatically captured without any registration or setup step. The two things that matter are generating the tag and recording timestamp_from at the same point, before triggering the action:

const tag = `signup-${Date.now()}`;
const testEmail = `${NAMESPACE}.${tag}@inbox.testmail.app`;
const startTimestamp = Date.now();

await page.fill('#email', testEmail);
await page.click('#submit');

Without timestamp_from, a stale email from a previous run that shares the same tag will satisfy your count assertion and cause content assertions to fail in ways that are hard to trace. Without a unique tag, parallel runs will collide. Both lines are required, and both need to come before the triggering action.

Handle resend flows explicitly

When a user requests a resend, such as a second verification email, a new OTP, or another password reset, your test needs to distinguish the new email from the previous one. Record a new timestamp before triggering the resend, then pass it to the livequery:

// First send
const startTimestamp = Date.now();
await page.click('#send-verification');
const firstEmail = await waitForEmail(tag, startTimestamp);

// User requests resend
const resendTimestamp = Date.now();
await page.click('#resend-verification');
const resendEmail = await waitForEmail(tag, resendTimestamp);

expect(resendEmail.subject).to.equal('Verify your email address');

Test at least one failure path per flow

Most email test suites only cover the happy path. What happens when a verification link is clicked twice? When an OTP is submitted after expiry? When the same email is used to sign up again? These are the cases users actually hit, and they're straightforward to cover once the happy path test exists.

Keep email test helpers in a shared utility

As your application's email surface grows, you'll write the same inbox polling, token extraction, and cleanup logic repeatedly. Extract it into a shared helper early. Individual tests then stay focused on what's specific to that flow rather than repeating infrastructure boilerplate.

CI/CD configuration

Email tests belong in your CI pipeline alongside your other end-to-end tests. The async nature of email delivery introduces a few CI-specific considerations worth getting right from the start.

Store credentials as secrets

Your API key and namespace should never be hardcoded in test files. Store them as encrypted secrets in your CI platform and expose them as environment variables at runtime:

# GitHub Actions
env:
  TESTMAIL_APIKEY: ${{ secrets.TESTMAIL_APIKEY }}
  TESTMAIL_NAMESPACE: ${{ secrets.TESTMAIL_NAMESPACE }}

Generate tags from CI run identifiers

In local development, a timestamp-based tag is sufficient. In CI, tie the tag to the run ID and test name so you can trace exactly which test generated which email during debugging:

const tag = `${process.env.GITHUB_RUN_ID || 'local'}-signup-${Date.now()}`;

To retrieve every email generated by one CI run together, e.g. when debugging a failed build, filter with tag_prefix instead of an exact tag match. A query with tag_prefix=${process.env.GITHUB_RUN_ID} matches every tag created during that run, regardless of which flow generated it.

Set timeouts at the test suite level

See Livequery above: the same 307-redirect behavior means a client-level timeout won't reliably terminate a stuck request. The timeout must be set on your test runner directly. Five minutes is a reasonable starting point; the right value depends on your ESP and pipeline.

Parallelize freely

Because every test generates a unique inbox, email tests are safe to run in parallel without any additional configuration. There's no shared state to manage and no risk of inbox collision between concurrent runs.

Separate email tests from fast unit tests

Email tests are inherently slower than unit or integration tests. Keep them in a dedicated test job or suite so a slow email test doesn't block fast feedback on unrelated code changes. A common pattern is to run unit and integration tests on every commit, and email tests only on changes that touch authentication, notification, or email-related code paths.

Common email test failures (and how to fix them)

Email tests fail in predictable ways. These are the patterns you'll encounter most often and how to resolve them.

Tests pass locally but fail in CI

Almost always a delivery timing issue. Local environments typically have faster, more direct routing to your ESP. In CI, delivery can take longer depending on network conditions and ESP load. The fix is straightforward: switch to livequery if you haven't already, and increase your timeout. Set it at the test suite level and use five minutes as a starting point.

Tests pass individually but fail in parallel

Inbox collision. Two tests are sharing the same tag, so emails from one run land in another run's inbox. Check that your tag generation happens inside the test, not in a shared global variable set once at module load.

// Wrong — same tag shared across every test in this file
const tag = `test-${Date.now()}`;

// Right — tag is generated fresh per test
test('signup flow', async () => {
  const tag = `signup-${Date.now()}-${Math.random().toString(36).slice(2)}`;
});

Assertions are reading the wrong email

Missing the timestamp_from filter. If a previous test run sent an email to the same tag, or if your tag generation isn't sufficiently unique, your fetch may return a stale email. Fix: always record the timestamp before triggering the action and pass it as timestamp_from (see "Set up your inbox address and timestamp_from together" above).

If a flow sends multiple emails in quick succession and timestamp_from alone isn't specific enough, the GraphQL API's advanced_filters lets you filter by subject server-side instead of fetching everything and filtering in your test code:

inbox(
  namespace: "YOUR_NAMESPACE"
  tag: "${tag}"
  timestamp_from: ${startTimestamp}
  advanced_filters: [{ field: subject, match: exact, action: include, value: "Verify your email address" }]
  livequery: true
) {
  emails { subject html from }
}

OTP tests are intermittently failing

This is usually a test isolation issue rather than a race condition caused by OTP expiry. First, check that your tag is unique for each test run. A shared or predictable tag can cause your inbox query to return a previous run's OTP email, resulting in an expired code.

Next, check your livequery timeout. It should be comfortably within the OTP expiry window, not close to it. For example, if your OTP expires after five minutes, a 30-second timeout provides plenty of headroom.

Once the test is reliable, add explicit coverage for failure scenarios. Expired codes and reused codes are both important cases to verify.

Link assertions are failing in production but not in staging

Environment bleed in email templates. A template hardcoded to a staging URL, or a misconfigured environment variable in your ESP, will pass tests run against staging and fail in production. Assert the domain explicitly in your link checks so the test fails in either environment when the URL is wrong:

const urls = [...email.html.matchAll(/href="(https?:\/\/[^"]+)"/g)].map(m => m[1]);
for (const url of urls) {
  expect(url).to.not.include('staging.yourapp.com');
  expect(url).to.include('yourapp.com');
}

SPF or DKIM assertions failing after an infrastructure change

First, verify your DNS records directly using a tool like MXToolbox or by running a dig command against your domain. If the records look correct, DNS propagation may be the cause. Changes can take up to 48 hours to fully propagate, during which authentication results may be inconsistent.

If the records are missing or incorrect, that's the issue to fix, not the test. In that case, the assertion is doing exactly what it's supposed to do.

Regex extraction throws instead of failing cleanly

The link extraction pattern used throughout the guide, email.html.match(/href="..."/)[1], returns null if the pattern doesn't match. Calling [1] on null throws a raw error instead of a clear test failure. When a template changes slightly, this sends whoever's debugging down the wrong path. Fix: check the match result before indexing, or throw a clear message like "Verification link not found in email body" when it's missing.

One action triggers multiple emails

Some flows send more than one email per action, like a signup that fires both a welcome email and a verification email. Tests that assume count equals 1, or grab emails[0] without checking which email it is, can silently read the wrong message. Fix: filter by subject or another distinguishing field, not array position, and be explicit about how many emails you expect.

Next steps

The patterns in this guide cover the core aspects of automated email testing, from choosing the right infrastructure to writing assertions and integrating tests into your CI pipeline. If you're not already using testmail.app, you can sign up and start testing your email workflows in just a few minutes.