Get Started
Welcome aboard! Here you’ll find all the documentation you’ll need. We’re going to assume that you have already signed up and you have access to your console (to retrieve your namespaces and API keys).
Essential concepts
testmail.app receives emails at {namespace}.{tag}@inbox.testmail.app
where {namespace}
is a unique namespace assigned to you (see your console) and {tag}
can be anything you choose.
Namespaces
Think of your namespace as a collection of mailboxes - each namespace supports an unlimited number of email addresses.
Here’s an example: suppose your namespace is acmeinc
; if you send a test email to [email protected]
and another test email to [email protected]
, both emails will be received in the acmeinc
namespace - the first will have the tag hello
and the second will have the tag hey
.
Tags
The tag can be anything you choose. By using different tags, you can create new email addresses and mailboxes on the fly!
Here’s an example: suppose you want to simulate new user signups for testing your app; you can create new users named John with the email [email protected]
and Albert with the email [email protected]
, and then retrieve emails sent to John and Albert separately by using the tag filter or together by querying the whole acmeinc
namespace.
API keys
API keys (retrieve yours from the console) are required to authenticate your API requests. API keys are authorized to access one or more namespaces - you can configure API key permissions in your console.
Retention
Each namespace has a default retention period (1, 3, or 30 days depending on your plan); after this period, old emails are automatically deleted. You don’t need to manually delete emails.
If your use case requires a longer (or shorter) retention period, please reach out - we can customize your namespace retention.
Choosing an API
testmail.app offers two APIs: a simple JSON API and a full-featured GraphQL API. The simple JSON API is easy to get started but has some limitations (see the comparison table).
If you’re already familiar with GraphQL, you can get started right away by taking a look at the schema and docs popovers on the right in the GraphQL playground - it’s fully self-documenting! You can build and test queries in the GraphQL playground or in any API testing tool of your choice (like Postman or Insomnia). Remember to include auth headers when using the GraphQL API.
Comparing the APIs
Feature (Description) |
Simple JSON API | GraphQL API |
---|---|---|
Basic filters Namespace, tag, tag_prefix, and timestamp range |
Yes | Yes |
Pagination Using the limit and offset parameters |
Yes | Yes |
Spam reports Query spam_score and spam_report |
Yes | Yes |
“Live” queries Waiting for new emails |
Yes | Yes |
Return select fields (Instead of all fields) |
No | Yes |
Advanced filters Include/exclude by from, subject, text/html body, etc. |
No | Yes |
Custom sort Default sort is descending by timestamp |
No | Yes |
JSON API: Guide
Quickstart
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.URI;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
class Main {
public static void main(String[] args) throws IOException, InterruptedException, ExecutionException {
// create a client
var client = HttpClient.newHttpClient();
String APIKEY = "YOUR_APIKEY", NAMESPACE = "YOUR_NAMESPACE";
var url = "https://api.testmail.app/api/json?apikey=" + APIKEY + "&namespace=" + NAMESPACE + "&pretty=true";
// create a request
var request = HttpRequest.newBuilder(URI.create(url))
.header("accept", "application/json")
.build();
// use the client to send the request
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
// the response:
System.out.println(response.body());
}
}
using System;
using System.Net.Http;
using System.Threading.Tasks;
class Program {
public static async Task Main(string[] args) {
String APIKEY = "YOUR_API_KEY", NAMESPACE = "YOUR_NAMESPACE";
var url = "https://api.testmail.app/api/json?apikey=" + APIKEY + "&namespace=" + NAMESPACE + "&pretty=true";
using
var client = new HttpClient();
var content = await client.GetStringAsync(url);
Console.WriteLine(content);
}
}
# Get your apikey and namespace from the console after signup/signin
APIKEY="YOUR_APIKEY"
NAMESPACE="YOUR_NAMESPACE"
# Use any HTTP client (or your browser)
curl "https://api.testmail.app/api/json?apikey=${APIKEY}&namespace=${NAMESPACE}&pretty=true"
// Use any HTTP client
const res = await axios.get('https://api.testmail.app/api/json', {
params: {
apikey: 'YOUR_APIKEY',
namespace: 'YOUR_NAMESPACE'
}
});
console.log(res.data);
import os
import json
import requests
API_KEY = 'API_KEY_HERE'
# api-endpoint
URL = "https://api.testmail.app/api/json"
# defining a params dict for the parameters to be sent to the API
PARAMS = {'apikey':API_KEY, 'namespace':'YOUR_NAMESPACE'}
# sending get request and saving the response as response object
r = requests.get(url = URL, params = PARAMS)
# extracting data in json format
data = r.json()
print(json.dumps(data, indent=4))
package main
import (
"os"
"io/ioutil"
"log"
"net/http"
"net/url"
)
func main() {
key := 'API_KEY_HERE'
// Define the parameters
params := url.Values{}
params.Add("apikey", key)
params.Add("namespace", "YOUR_NAMESPACE")
// Create the URL with the parameters
url := "https://api.testmail.app/api/json?" + params.Encode()
resp, err := http.Get(url)
if err != nil {
log.Fatalln(err)
}
//We Read the response body on the line below.
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
//Convert the body to type string
sb := string(body)
log.Printf(sb)
}
require 'net/http'
uri = URI('https://api.testmail.app/api/json')
params = { :apikey => 'YOUR_API_KEY', :namespace => 'YOUR_NAMESPACE' }
uri.query = URI.encode_www_form(params)
res = Net::HTTP.get_response(uri)
puts res.body if res.is_a?(Net::HTTPSuccess)
<?php
$APIKEY = 'YOUR_APIKEY';
$url = 'https://api.testmail.app/api/json';
$query = http_build_query(array('apikey' => $APIKEY, 'namespace' => 'YOUR_NAMESPACE'));
$data = file_get_contents($url . '?' . $query);
echo $data;
Expect something like this (example):
{
"result": "success",
"message": null,
"count": 0,
"limit": 10,
"offset": 0,
"emails": []
}
The JSON API is designed to be simple and work everywhere (including your browser). You can use any HTTP client in any language or environment of choice - just make HTTP GET requests.
The API endpoint: https://api.testmail.app/api/json
Just two parameters are mandatory: your API key (&apikey=YOUR_APIKEY
) and your namespace (&namespace=YOUR_NAMESPACE
). If you’re viewing the output in your browser, add &pretty=true
to prettify the output (makes it more readable).
The API returns a result object (see the example on the right) with the following structure:
"result": "success"
or"result": "fail"
: tells you whether the query was successful or not"message": null
or"message": <string>
: tells you why the query failed (if it failed) or provides helpful warnings"count": <int>
: tells you the number of emails that matched your query"limit": <int>
: tells you the number of emails returned in this request (see limit)"offset": <int>
: tells you the number of emails skipped (see pagination)"emails": [ <EmailObject> ]
: array of emails
To explore all API features and options, check out the JSON API reference.
Waiting for new email
using System;
using System.Net.Http;
using System.Threading.Tasks;
class Program {
public static async Task Main(string[] args) {
String APIKEY = "YOUR_API_KEY", NAMESPACE = "YOUR_NAMESPACE";
var url = "https://api.testmail.app/api/json?apikey=" + APIKEY + "&namespace=" + NAMESPACE + "&pretty=true&livequery=true";
using
var client = new HttpClient();
var content = await client.GetStringAsync(url);
Console.WriteLine(content);
}
}
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.URI;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
class Main {
public static void main(String[] args) throws IOException, InterruptedException, ExecutionException {
// create a client
var client = HttpClient.newHttpClient();
String APIKEY = "YOUR_APIKEY", NAMESPACE = "YOUR_NAMESPACE";
var url = "https://api.testmail.app/api/json?apikey=" + APIKEY + "&namespace=" + NAMESPACE + "&pretty=true&livequery=true";
// create a request
var request = HttpRequest.newBuilder(URI.create(url))
.header("accept", "application/json")
.build();
// use the client to send the request
var response = client.send(request, HttpResponse.BodyHandlers.ofString());
// the response:
System.out.println(response.body());
}
}
# Use of "Live" queries.
curl "https://api.testmail.app/api/json?apikey=${APIKEY}&namespace=${NAMESPACE}&pretty=true&livequery=true"
import os
import json
import requests
API_KEY = 'API_KEY_HERE'
# api-endpoint
URL = "https://api.testmail.app/api/json"
# defining a params dict for the parameters to be sent to the API
PARAMS = {'apikey':API_KEY, 'namespace':'YOUR_NAMESPACE', 'livequery': 'true'}
# sending get request and saving the response as response object
r = requests.get(url = URL, params = PARAMS)
# extracting data in json format
data = r.json()
print(json.dumps(data, indent=4))
// Use any HTTP client
const res = await axios.get('https://api.testmail.app/api/json', {
params: {
apikey: 'YOUR_APIKEY',
namespace: 'YOUR_NAMESPACE',
livequery: 'true'
}
});
console.log(res.data);
package main
import (
"os"
"io/ioutil"
"log"
"net/http"
"net/url"
)
func main() {
key := 'API_KEY_HERE'
// Define the parameters
params := url.Values{}
params.Add("apikey", key)
params.Add("namespace", "YOUR_NAMESPACE")
params.Add("livequery", "true")
// Create the URL with the parameters
url := "https://api.testmail.app/api/json?" + params.Encode()
resp, err := http.Get(url)
if err != nil {
log.Fatalln(err)
}
//We Read the response body on the line below.
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
//Convert the body to type string
sb := string(body)
log.Printf(sb)
}
require 'net/http'
uri = URI('https://api.testmail.app/api/json')
params = { :apikey => 'YOUR_API_KEY', :namespace => 'YOUR_NAMESPACE', :livequery => 'true' }
uri.query = URI.encode_www_form(params)
res = Net::HTTP.get_response(uri)
puts res.body if res.is_a?(Net::HTTPSuccess)
<?php
$APIKEY = 'YOUR_APIKEY';
$url = 'https://api.testmail.app/api/json';
$query = http_build_query(array('apikey' => $APIKEY, 'namespace' => 'YOUR_NAMESPACE', 'livequery' => 'true'));
$data = file_get_contents($url . '?' . $query);
echo $data;
When you’re expecting to receive new emails, instead of repeatedly querying the inbox, add the &livequery=true
parameter to your API call. “Live” queries wait until at least one email is matched before returning. Here’s how it works:
- The API receives your live query and begins waiting for new emails that match your query.
- As soon as at least one matching email is found, the API returns the result immediately.
- After one minute of waiting, the API returns an HTTP 307 redirect to itself - so your HTTP client resends the same query again.
- The above process repeats itself indefinitely, so make sure you set a timeout in your testing suite!
Deleting emails
You might feel the need to delete emails in your namespace after each test - this is anti-pattern!
Instead, when you need to filter out emails from previous tests, we recommend recording the start time at the beginning of your test and feeding this to the timestamp_from filter. Or, when you need to run multiple tests simultaneously, we recommend prefixing your tags with a unique test ID and feeding this to the tag_prefix filter.
Emails are automatically deleted after the default retention period for your namespace (1, 3, or 30 days depending on your plan). We don’t offer a delete function in our APIs or console. This is because we index and store emails using immutable data structures that prioritize query performance (so that you can retrieve emails quickly) with the tradeoff that deleting emails is a somewhat expensive process (computationally) that we do via scheduled cron jobs every night.
JSON API: Reference
?apikey={string}
Your API key - mandatory for all API requests.
Syntax: | &apikey=1c1568a3-db0a-4189-bf28-7987f14f19d0 (example only - not a real apikey) |
Required: | yes |
&namespace={string}
The namespace you wish to query - mandatory for all API requests.
Syntax: | &namespace=abc123 (example only - use your namespace) |
Required: | yes |
&pretty=true
Whether to prettify JSON output (makes it more readable).
Use this when you’re testing the API in your browser; keep it off when using the API programmatically (in production).
Syntax: | “true” or “false” (without the quotes) |
Required: | no - it’s optional |
Default: | false |
&headers=true
Headers example in the email object:
{
"headers": [
{
"line": "Message-Id: <...>",
"key": "message-id"
},
{
"line": "Mime-Version: 1.0",
"key": "mime-version"
}
// ...etc.
]
// ...rest of the email
}
Whether to include email headers in each email object.
Syntax: | “true” or “false” (without the quotes) |
Required: | no - it’s optional |
Default: | false |
&spam_report=true
Spam report example in the email object:
{
"spam_score": 1.158,
"spam_report": "Spam detection software, running on the system \"inbox.testmail.app\", has\nidentified this incoming email as possible spam. The original message\nhas been attached to this so you can view it (if it isn't spam) or label\nsimilar future email. If you have any questions, see\n@@CONTACT_ADDRESS@@ for details.\n\nContent preview: Hello there! Here is a test html message. hello world! [...]\n \n\nContent analysis details: (1.2 points, 5.0 required)\n\n pts rule name description\n---- ---------------------- --------------------------------------------------\n 0.8 HTML_IMAGE_RATIO_02 BODY: HTML has a low ratio of text to image area\n 0.0 HTML_MESSAGE BODY: HTML included in message\n 0.3 HTML_IMAGE_ONLY_04 BODY: HTML: images with 0-400 bytes of words\n 0.0 T_MIME_NO_TEXT No text body parts\n\n"
// ...rest of the email
}
Whether to include spam reports in each email object. This will add a spam_score
field (floating point number that can be negative) and a spam_report
field. See the spam testing guide for details.
Syntax: | “true” or “false” (without the quotes) |
Required: | no - it’s optional |
Default: | false |
&tag={string}
Filter emails by tag (exact match).
Example: an email sent to [email protected]
will be in the test
namespace and have the john.smith
tag.
Syntax: | &tag=test (example only) |
Required: | no - it’s optional |
&tag_prefix={string}
Filter emails by tag prefix.
Example: a query filtered by &tag_prefix=john
will match [email protected]
and also match [email protected]
because both tags (john.smith and john.tory) start with “john”.
Syntax: | &tagprefix=test (example only)_ |
Required: | no - it’s optional |
×tamp_from={int}
Filter emails by starting unix timestamp in milliseconds (number of milliseconds since 1st January 1970, UTC)
Syntax: | integer (e.g. ×tamp_from=1579916290055) |
Required: | no - it’s optional |
×tamp_to={int}
Filter emails by ending unix timestamp in milliseconds (number of milliseconds since 1st January 1970, UTC)
Syntax: | integer (e.g. ×tamp_to=1579916298055) |
Required: | no - it’s optional |
&limit={int}
Maximum number of emails to return (useful for pagination when used with the offset parameter).
Syntax: | integer (e.g. &limit=5) |
Required: | no - it’s optional |
Default: | 10 |
Minimum: | 0 (but no emails will be returned if limit = 0) |
Maximum: | 100 |
&offset={int}
Number of emails to skip/ignore (useful for pagination when used with the limit parameter).
Syntax: | integer (e.g. &offset=5) |
Required: | no - it’s optional |
Default: | 0 |
Minimum: | 0 |
Maximum: | 9899 (contact us if you need higher offset limits) |
&livequery=true
Whether to wait for new emails before responding. See waiting for new email for an explanation of how this works.
Syntax: | “true” or “false” (without the quotes) |
Required: | no - it’s optional |
Default: | false |
GraphQL API: Guide
Setup and auth
package com.example.example;
import java.lang.System;
import com.apollographql.apollo.ApolloCall;
import com.apollographql.apollo.ApolloClient;
import com.apollographql.apollo.api.Response;
import com.apollographql.apollo.exception.ApolloException;
import org.jetbrains.annotations.NotNull;
import com.example.FetchQuery;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.logging._;
import okhttp3.Interceptor;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import java.util._;
public class Main {
private final static Logger LOGGER =
Logger.getLogger(Logger.GLOBAL_LOGGER_NAME);
public static void main(String[] args) {
Map < String, String > headers = new HashMap < String, String > ();
String key = 'API_KEY_HERE';
headers.put("Authorization", "Bearer " + key);
OkHttpClient httpClient = new OkHttpClient.Builder()
.addInterceptor(chain -> {
Request original = chain.request();
Request.Builder builder = original.newBuilder().method(original.method(), original.body());
headers.forEach(builder::header);
return chain.proceed(builder.build());
})
.build();
// First, create an `ApolloClient`
// Replace the serverUrl with your GraphQL endpoint
ApolloClient apolloClient = ApolloClient.builder()
.serverUrl("https://api.testmail.app/api/graphql").okHttpClient(httpClient).build();
// Then enqueue your query
}
}
using GraphQL;
using GraphQL.Client.Http;
using GraphQL.Client.Serializer.Newtonsoft;
using InboxResponse;
using System.Text.Json;
var graphQLHttpClientOptions = new GraphQLHttpClientOptions
{
EndPoint = new Uri("https://api.testmail.app/api/graphql")
};
var httpClient = new HttpClient();
string mySecret = 'API_KEY_HERE';
httpClient.DefaultRequestHeaders.Add("Authorization", "Bearer " + mySecret);
var graphQLClient = new GraphQLHttpClient(graphQLHttpClientOptions, new NewtonsoftJsonSerializer(), httpClient);
package main
import (
"context"
"fmt"
"net/http"
"os"
"github.com/Khan/genqlient/graphql"
)
type authedTransport struct {
wrapped http.RoundTripper
}
func (t *authedTransport) RoundTrip(req *http.Request) (*http.Response, error) {
key := 'API_KEY_HERE'
req.Header.Set("Authorization", "Bearer "+key)
return t.wrapped.RoundTrip(req)
}
func main() {
// create a client (safe to share across requests) by passing in an HTTP client that adds the auth header
client := graphql.NewClient("https://api.testmail.app/api/graphql",
&http.Client{Transport: &authedTransport{wrapped: http.DefaultTransport}})
}
from gql import gql, Client
import json, time
from gql.transport.aiohttp import AIOHTTPTransport
# Select your transport with testmail grahql api endpoint
transport = AIOHTTPTransport(url='https://api.testmail.app/api/graphql', headers={'Authorization': 'Bearer API_KEY'})
# Create a GraphQL client using the defined transport
client = Client(transport=transport, fetch_schema_from_transport=True)
const GraphQLClient = require('@testmail.app/graphql-request').GraphQLClient;
const testmailClient = new GraphQLClient(
// API endpoint:
'https://api.testmail.app/api/graphql',
// Use your API key:
{ headers: { Authorization: 'Bearer API_KEY' } }
);
<?php
require __DIR__ . '/vendor/autoload.php';
use Softonic\GraphQL\ClientBuilder;
$APIKEY = 'YOUR_APIKEY';
$client = ClientBuilder::build(
'https://api.testmail.app/api/graphql',
[
'headers' => ['Authorization' => "Bearer $APIKEY"]
]
);
require 'graphql/client'
require 'graphql/client/http'
# Configure GraphQL endpoint using the basic HTTP network adapter.
HTTP = GraphQL::Client::HTTP.new('https://api.testmail.app/api/graphql') do
def headers(_context)
# Optionally set any HTTP headers
{ "Authorization": 'Bearer ' + 'YOUR_API_KEY' }
end
end
# Fetch latest schema on init, this will make a network request
Schema = GraphQL::Client.load_schema(HTTP)
Client = GraphQL::Client.new(schema: Schema, execute: HTTP)
To use the GraphQL API, you need a GraphQL (recommended) or HTTP client. We’ve included a set of examples on the right in different languages. We have used the following GraphQL clients for the examples:
- For
javascript
we are using a clone of the popular graphql-request client with added features like built-in retries: @testmail.app/graphql-request. - For
php
we are using Softonic/graphql-client. You can easily install it using composer:composer require softonic/graphql-client
- For
ruby
we are using github.com/github/graphql-client. You can easily install it adding the following line to your Gemfile:gem 'graphql-client'
and runningbundle install
. - For
python
we are using github.com/graphql-python/gql which is a GraphQL client for Python. You can easily install it using pip:pip install gql[all]
. - For
golang
we are using github.com/Khan/genqlient which is a strongly-typed GraphQL client for Go. You will have to setup a few things to get it working. You can find the instructions here . You can download the schema for GraphQL queries from the GraphQL Playground. - For
C#
we are using github.com/graphql-dotnet/graphql-client. You can easily install it usingdotnet add package GraphQL.Client
. Newtonsoft.Json is also required. You can easily install it usingdotnet add package Newtonsoft.Json
. To make your project more maintainable, we recommend creating classes for GraphQl responses. You can use json2csharp or quicktype to generate classes from JSON responses. You can also refer to the example provided by the library. - For
java
we are usingapollographql/apollo-android
. You can view the instructions of how to setup the client here.
You can also use any other GraphQL client of your choice. You can find a list of alternative GraphQL clients here.
The API endpoint: https://api.testmail.app/api/graphql
Access to the API is secured using API keys. You can get your API key or register a new API key in the developer console.
The API key must be included in all API requests to the server in an authorization header that looks like this:
Authorization: Bearer API_KEY
Querying the inbox
/**
* The following GraphQL query should be present in a graphql file along with the schema.
query FetchQuery($namespace: String!, $tag: String!) {
inbox(namespace: $namespace, tag_prefix: $tag) {
result
message
count
}
}
*
*/
apolloClient.query(new FetchQuery("NAMESPACE_HERE", "TAG_HERE"))
.enqueue(new ApolloCall.Callback < FetchQuery.Data > () {
@Override
public void onResponse(@NotNull Response < FetchQuery.Data > response) {
LOGGER.log(Level.INFO, "Output: " + response.getData());
}
@Override
public void onFailure(@NotNull ApolloException e) {
LOGGER.log(Level.INFO, "Error", e);
}
});
var inboxRequest = new GraphQLRequest
{
Query = @"
query getInbox($namespace : String!,$tag : String!){
inbox (
namespace:$namespace
tag_prefix:$tag
) {
result
message
count
}
}
",
Variables = new {
Tag = "YOUR_TAG",
Namespace = "YOUR_NAMESPACE",
}
};
var graphQLResponse = await graphQLClient.SendQueryAsync<InboxType>(inboxRequest);
Console.WriteLine(System.Text.Json.JsonSerializer.Serialize(graphQLResponse.Data, new JsonSerializerOptions { WriteIndented = true }));
/*
* This example uses the genqlient library to generate a strongly-typed Go client
* from the GraphQL schema.
* The genqlient.schema file is contains the following schema:
query getInbox($namespace : String!){
inbox (
namespace:$namespace
) {
result
message
count
}
}
*/
func main() {
// create a client (safe to share across requests) by passing in an HTTP client that adds the auth header
client := graphql.NewClient("https://api.testmail.app/api/graphql",
&http.Client{Transport: &authedTransport{wrapped: http.DefaultTransport}})
// getInbox is a function generated by genqlient from the GraphQL schema.
resp, err := getInbox(context.Background(), client, "YOUR_NAMESPACE")
if err != nil {
return
}
fmt.Println(resp.Inbox)
}
# Provide the query to fetch inbox details.
query = gql(
"""
{
inbox (
namespace:"YOUR_NAMESPACE"
) {
result
message
count
}
}
"""
)
# Execute the query on the transport
result = client.execute(query)
print(json.dumps(result, indent=4))
curl -X POST \
https://api-stage.testmail.app/api/graphql \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer '$YOUR_API_KEY'' \
-d '{
"query": "query getInbox($namespace : String!){ inbox ( namespace:$namespace ) { result message count } }",
"variables":{
"namespace": "YOUR_NAMESPACE"
}
}'
testmailClient
.request(
`{
inbox (
namespace:"YOUR_NAMESPACE"
) {
result
message
count
}
}`
)
.then(data => {
console.log(data.inbox);
});
$query = <<<'QUERY'
{
inbox (
namespace:"YOUR_NAMESPACE"
) {
result
message
count
emails {
from
from_parsed {
address
name
}
subject
}
}
}
QUERY;
$variables = [];
$response = $client->query($query, $variables);
$r = $response->getData();
echo "<pre>".json_encode($r, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES)."</pre>";
EmailQuery = Client.parse <<-'GRAPHQL'
query {
inbox (
namespace:"YOUR_NAMESPACE"
) {
result
message
count
emails {
from
}
}
}
GRAPHQL
result = Client.query(EmailQuery)
p result.data
Expect something like this (example):
{
"result": "success",
"message": null,
"count": 2
}
There’s just one query you need: the inbox query.
In the example on the right (a very simple query), testmail will return a count for the total number of emails in the specified namespace. It specifies one argument - the namespace (always mandatory) - and asks for the result, message, and count.
Successful queries will return result: 'success'
; unsuccessful queries will return result: 'fail'
. In case of failure, the message
will usually provide an explanation.
The count
will return an integer. Of course: you can query for a lot more than the count. Please see the docs tab in the GraphQL playground for a full API reference.
If you’re expecting the query to return many emails and you need pagination, you can use the limit
(default is 10
) and offset
arguments.
“Live” queries
curl -X POST \
https://api-stage.testmail.app/api/graphql \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer '$YOUR_API_KEY'' \
-d '{
"query": "query getInbox($namespace : String!){ inbox ( namespace:$namespace livequery:true ) { result message count } }",
"variables":{
"namespace": "YOUR_NAMESPACE"
}
}'
/*
* This example uses the genqlient library to generate a strongly-typed Go client
* from the GraphQL schema.
* The genqlient.schema file is contains the following schema:
query getInbox($namespace : String!, $timestamp : Float!){
inbox (
namespace:$namespace
tag:"john.smith"
timestamp_from:$timestamp
livequery:true
) {
result
message
emails {
from
from_parsed {
address
name
}
subject
}
}
}
*/
func main() {
// create a client (safe to share across requests) by passing in an HTTP client that adds the auth header
client := graphql.NewClient("https://api.testmail.app/api/graphql",
&http.Client{Transport: &authedTransport{wrapped: http.DefaultTransport}})
now := time.Now() // current local time
sec := now.UnixMicro()
// getInbox is a function generated by genqlient from the GraphQL schema.
resp, err := getInbox(context.Background(), client, "YOUR_NAMESPACE", float64(sec))
if err != nil {
return
}
fmt.Println(resp.Inbox)
}
/**
* The following GraphQL query should be present in a graphql file along with the schema.
query FetchQuery($namespace: String!, $tag: String!, $timestamp: Float!) {
inbox(
namespace: $namespace
tag_prefix: $tag
timestamp_from: $timestamp
livequery: true
) {
result
message
emails {
from
from_parsed {
address
name
}
subject
}
}
}
*
*/
// Get the current time in milliseconds
long unixTime = System.currentTimeMillis();
apolloClient.query(new FetchQuery("NAMESPACE_HERE", "TAG_HERE", unixTime))
.enqueue(new ApolloCall.Callback < FetchQuery.Data > () {
@Override
public void onResponse(@NotNull Response < FetchQuery.Data > response) {
LOGGER.log(Level.INFO, "Output: " + response.getData());
}
@Override
public void onFailure(@NotNull ApolloException e) {
LOGGER.log(Level.INFO, "Error", e);
}
});
long unixTimestamp = 1000 * (long)(DateTime.Now.Subtract(new
DateTime(1970, 1, 1))).TotalSeconds;
var inboxRequest = new GraphQLRequest
{
Query = @"
query getInbox($namespace : String!,$tag : String!, $timestamp : Float!){
inbox (
namespace:$namespace
tag_prefix:$tag
timestamp_from:$timestamp
livequery:true
) {
result
message
emails {
from
from_parsed {
address
name
}
subject
}
}
}
",
Variables = new {
Tag = "john.smith",
Namespace = "YOUR_NAMESPACE",
Timestamp = unixTimestamp
}
};
var graphQLResponse = await graphQLClient.SendQueryAsync<InboxType>(inboxRequest);
Console.WriteLine(System.Text.Json.JsonSerializer.Serialize(graphQLResponse.Data, new JsonSerializerOptions { WriteIndented = true }));
$query = <<<'QUERY'
query GetMails($timestamp: Float) {
inbox (
namespace:"YOUR_NAMESPACE"
tag_prefix:"test"
timestamp_from:$timestamp
livequery:true
) {
result
message
emails {
from
from_parsed {
address
name
}
subject
}
}
}
QUERY;
$variables = [
"timestamp" => time() * 1000
];
$response = $client->query($query, $variables);
$r = $response->getData();
echo "<pre>".json_encode($r, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES)."</pre>";
?>
timestamp = int(time.time() * 1000)
# Provide the query to fetch inbox details with livequery as true.
query = gql(
f"""
{{
inbox (
namespace:"YOUR_NAMESPACE"
tag_prefix:"test"
timestamp_from:{timestamp}
livequery:true
) {{
result
message
emails {{
from
from_parsed {{
address
name
}}
subject
}}
}}
}}
"""
)
# Execute the query on the transport
result = client.execute(query)
print(json.dumps(result, indent=4))
const timestamp = Date.now();
testmailClient
.request(
`{
inbox (
namespace:"YOUR_NAMESPACE"
tag:"john.smith"
timestamp_from:${timestamp}
livequery:true
) {
result
message
emails {
from
from_parsed {
address
name
}
subject
}
}
}`
)
.then(data => {
console.log(data.inbox);
});
EmailQuery = Client.parse <<-'GRAPHQL'
query($timestamp: Float) {
inbox (
namespace:"YOUR_NAMESPACE"
tag_prefix:"test"
timestamp_from:$timestamp
livequery:true
) {
result
message
emails {
from
from_parsed {
address
name
}
subject
}
}
}
GRAPHQL
result = Client.query(EmailQuery, variables: {timestamp: Time.now.to_i * 1000})
p result.data
Expect something like this (example):
{
"result": "success",
"message": null,
"emails": [
{
"from": "testmail app <[email protected]>",
"from_parsed": {
"address": "[email protected]",
"name": "testmail app"
},
"subject": "Please confirm your email"
}
]
}
When you’re expecting to receive an email, instead of repeatedly querying the inbox, use a “live” query.
“Live” queries wait until at least one email is matched before returning. In the example on the right, the testmail API will wait until a new email addressed to [email protected]
is received before returning.
“Live” queries are implemented using HTTP 307 redirects. Here’s how it works:
- The API receives your live query and begins waiting for new emails that match your query.
- As soon as at least one matching email is found, the API returns the result immediately.
- After one minute of waiting, the API returns an HTTP 307 redirect to itself - so your GraphQL/HTTP client resends the same query again. Make sure your client follows 307 redirects.
- The above process repeats itself indefinitely, so make sure you set a timeout in your testing suite!
“Live” queries can return multiple emails. This happens either because there are already multiple emails matching the query (so the “live” query returns immediately) or because multiple new emails were received and processed at exactly the same time (this is rare).
Sorting and filtering
curl -X POST \
https://api.testmail.app/api/graphql \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer '$YOUR_API_KEY'' \
-d '{
"query": "query getInbox($namespace : String!){ inbox ( namespace:$namespace livequery:true advanced_sorts:[{ field:tag, order:asc }, { field:timestamp, order:desc }] ) { result message count emails { tag timestamp } } }",
"variables":{
"namespace": "YOUR_NAMESPACE"
}
}'
$query = <<<'QUERY'
{
inbox (
namespace:"YOUR_NAMESPACE"
tag_prefix:"peter"
advanced_filters:[{
field:subject
match:exact
action:include
value:"Please confirm your email"
}]
advanced_sorts:[{
field:tag,
order:asc
}, {
field:timestamp,
order:desc
}]
) {
result
message
count
emails {
tag
timestamp
}
}
}
QUERY;
$variables = [];
$response = $client->query($query, $variables);
$r = $response->getData();
echo "<pre>".json_encode($r, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES)."</pre>";
?>
/**
* The following GraphQL query should be present in a graphql file along with the schema.
query FetchQuery($namespace: String!, $tag: String!, $subject: String!) {
inbox(
namespace: $namespace
tag_prefix: $tag
advanced_filters: [
{ field: subject, match: exact, action: include, value: $subject }
]
advanced_sorts: [
{ field: tag, order: asc }
{ field: timestamp, order: desc }
]
) {
result
message
count
emails {
tag
timestamp
}
}
}
*
*/
apolloClient.query(new FetchQuery("NAMESPACE_HERE", "TAG_HERE", "SUBJECT_HERE"))
.enqueue(new ApolloCall.Callback < FetchQuery.Data > () {
@Override
public void onResponse(@NotNull Response < FetchQuery.Data > response) {
LOGGER.log(Level.INFO, "Output: " + response.getData());
}
@Override
public void onFailure(@NotNull ApolloException e) {
LOGGER.log(Level.INFO, "Error", e);
}
});
/*
* This example uses the genqlient library to generate a strongly-typed Go client
* from the GraphQL schema.
* The genqlient.schema file is contains the following schema:
query getInbox($namespace : String!){
inbox (
namespace:$namespace
tag_prefix:"peter"
advanced_filters:[{
field:subject
match:exact
action:include
value:"Please confirm your email"
}]
advanced_sorts:[{
field:tag,
order:asc
}, {
field:timestamp,
order:desc
}]
) {
result
message
count
emails {
tag
timestamp
}
}
}
*/
func main() {
// create a client (safe to share across requests) by passing in an HTTP client that adds the auth header
client := graphql.NewClient("https://api.testmail.app/api/graphql",
&http.Client{Transport: &authedTransport{wrapped: http.DefaultTransport}})
// getInbox is a function generated by genqlient from the GraphQL schema.
resp, err := getInbox(context.Background(), client, "YOUR_NAMESPACE")
if err != nil {
return
}
fmt.Println(resp.Inbox)
}
# Provide the query to fetch inbox details with livequery as true.
query = gql(
"""
{ inbox (
namespace:"YOUR_NAMESPACE"
tag_prefix:"peter"
advanced_filters:[{
field:subject
match:exact
action:include
value:"Please confirm your email"
}]
advanced_sorts:[{
field:tag,
order:asc
}, {
field:timestamp,
order:desc
}]
) {
result
message
count
emails {
tag
timestamp
}
}
}
"""
)
# Execute the query on the transport
result = client.execute(query)
print(json.dumps(result, indent=4))
testmailClient
.request(
`{
inbox (
namespace:"YOUR_NAMESPACE"
tag_prefix:"peter"
advanced_filters:[{
field:subject
match:exact
action:include
value:"Please confirm your email"
}]
advanced_sorts:[{
field:tag,
order:asc
}, {
field:timestamp,
order:desc
}]
) {
result
message
count
emails {
tag
timestamp
}
}
}`
)
.then(data => {
console.log(data.inbox);
});
EmailQuery = Client.parse <<-'GRAPHQL'
query {
inbox (
namespace:"YOUR_NAMESPACE"
tag_prefix:"peter"
advanced_filters:[{
field:subject
match:exact
action:include
value:"Please confirm your email"
}]
advanced_sorts:[{
field:tag,
order:asc
}, {
field:timestamp,
order:desc
}]
) {
result
message
count
emails {
tag
timestamp
}
}
}`
GRAPHQL
result = Client.query(EmailQuery)
p result.data
var inboxRequest = new GraphQLRequest
{
Query = @"
query getInbox($namespace : String!,$tag : String!, $subject : String!){
inbox (
namespace:$namespace
tag_prefix:$tag
advanced_filters:[{
field:subject
match:exact
action:include
value:$subject
}]
advanced_sorts:[{
field:tag,
order:asc
}, {
field:timestamp,
order:desc
}]
) {
result
message
count
emails {
tag
timestamp
}
}
}
",
Variables = new {
Tag = "peter",
Namespace = "YOUR_NAMESPACE",
Subject = "Please confirm your email"
}
};
var graphQLResponse = await graphQLClient.SendQueryAsync<InboxType>(inboxRequest);
Console.WriteLine(System.Text.Json.JsonSerializer.Serialize(graphQLResponse.Data, new JsonSerializerOptions { WriteIndented = true }));
Expect something like this (example):
{
"result": "success",
"message": null,
"count": 4,
"emails": [
{
"tag": "peter.pan",
"timestamp": 1565139749955
},
{
"tag": "peter.pan",
"timestamp": 1565136514957
},
{
"tag": "peter.parker",
"timestamp": 1565140065791
},
{
"tag": "peter.parker",
"timestamp": 1565140041284
}
]
}
The GraphQL API has excellent options for sorting and filtering emails. A couple of typical use cases:
- (Livequery) You’re waiting for an email with the subject “Please confirm your email” to hit the inbox.
If you don’t use sufficiently specific filters, the live query will return any new email that hits the inbox - not necessarily the email you’re expecting. This is especially important when you have many back-to-back email tests (you could receive emails in an unexpected order) or when you have parallel builds in your CI pipelines (you could receive emails from another test).
- You can let testmail do the matching for you by just querying for the count instead of downloading the full email.
If you want to check whether an email with <h1>Getting started</h1>
in the html body just hit the inbox, you can use the wildcard filter on the html field and query the count of matched emails instead of downloading the full html content and searching for that string. However, note that if the html field is greater than 30kb, it might not be indexed for search! If you need full text search over large emails, please contact us.
Note that the timestamp_from
, timestamp_to
, tag
, and tag_prefix
options are the most performant options for filtering emails.
Tags are indexed using n-grams - so tags and subsets of tags can be queried incredibly efficiently. Here is a typical usage pattern that takes advantage of this:
Send emails to: [email protected]
(where the tag is category.subcategory
), and then query for emails using the tag_prefix:
tag_prefix:"category"
will return all emails that fall within the categorytag_prefix:"category.subcategory"
will return all emails that fall within the category and subcategory- (and so on)
Nesting tags like this allows you to group/categorize inboxes, reduce conflicts, and use fewer advanced_filters
(which are less performant).
Emails are sorted by the timestamp
field in descending order by default (newest emails first). You can change this using advanced_sorts
; you can also use multiple sorts (see the example on the right).
GraphQL API: Reference
The complete GraphQL API reference is available in the GraphQL playground (see the schema and docs popovers on the right).
Spam Testing Guide
Spam filters have become fairly good at differentiating “spam” (bad) from “ham” (good): most modern spam detection tools incorporate machine learning and sophisticated filters that go beyond simple rules like whether the subject line includes “Congratulations!!!” or whether the body mentions a “Nigerian prince” who wants to send money for “safekeeping”.
Generally you don’t have to worry about spam filters if:
- Your emails are genuinely relevant and not spammy.
- You have correctly configured DKIM and SPF.
- Your domain, email servers, and IPs have good reputations (they are not on blacklists).
To help you improve deliverability, we spam-test every email and attach spam scores and detailed spam reports that you can query via API.
How it works
We use SpamAssassin (the most popular open-source anti-spam platform) to spam-test emails.
Spam scores and reports generated by SpamAssassin are not “universal” - they are highly dependent on the enabled rule sets and plugins. Each mailbox provider configures their installation differently: some build their own rule sets, some set higher or lower thresholds for classifying spam, etc.
Our configuration includes the most commonly enabled rule sets and uses the default spam threshold of 5 points. We add and update rules from time to time - so spam scores can change over time for the same email.
We recommend using these spam reports as guidelines: a high score suggests a high probability of being marked as spam - but a low score does not guarantee inbox placement.
TL;DR
- Don’t worry about spam scores below 5.
- If the spam score exceeds 5: look at which rules were triggered in the spam_report and fix the problems with the greatest impact.
- Negative spam scores are possible (a good thing).
Examples
See the following examples of common end-to-end tests using some of the most used testing frameworks - cypress, selenium and testcafe.
You can use testmail.app with any testing framework. These examples are just to give you an idea of how you can use testmail.app in your tests. Here we are using a simple example of the Welcome Aboard
(shown on the right) email that we send to new users. In your case, you can use testmail API to test any email that you send from your application. We have examples of testing emails with the help of both JSON API and GraphQL API.
Examples - JSON API
See the example on the right of an end-to-end test for the testing of our Welcome Aboard mail via JSON API. The same example via GraphQL API is available here.
Notes:
- You can use any HTTP client of your choice.
- We use the chance package to randomly generate tags. This allows us to retrieve the email based on this random/unique tag.
- You can do a lot more with tags:
- You can use tags to categorize your emails (for example:
namespace.category.subcategory
) and retrieve emails by adding&tag_prefix=namespace.category
to the url (as a query parameter). - You can group emails by test and retrieve them together like this:
namespace.test_id.category
(retrieve using&tag_prefix=namespace.test_id
)
- You can use tags to categorize your emails (for example:
- We use a test timeout of 5 minutes: this is the maximum time we will wait for the email to be delivered. The vast majority of emails will be received within seconds (we are allowing up to 5 minutes for edge cases).
- We use
timestamp_from
to ensure that we are retrieving emails that were received after this test began. - The combination of using
timestamp_from
and a unique randomly generated string in the tag minimizes conflicts between parallel tests (queries from one test should not retrieve emails from another test).
Using Cypress
Please switch to the Javascript tab to see the code for using Cypress with JSON API.
Please switch to the Javascript tab to see the code for using Cypress with JSON API.
Please switch to the Javascript tab to see the code for using Cypress with JSON API.
Please switch to the Javascript tab to see the code for using Cypress with JSON API.
Please switch to the Javascript tab to see the code for using Cypress with JSON API.
Please switch to the Javascript tab to see the code for using Cypress with JSON API.
Please switch to the Javascript tab to see the code for using Cypress with JSON API.
/**
* Required environment variables: TESTMAIL_APIKEY and TESTMAIL_NAMESPACE
*/
// Use any HTTP client
const axios = require('axios').default;
// Setup our JSON API endpoint
const ENDPOINT = `https://api.testmail.app/api/json?apikey=${Cypress.env(
'TESTMAIL_APIKEY'
)}&namespace=${Cypress.env('TESTMAIL_NAMESPACE')}`;
// Randomly generating the tag...
const ChanceJS = require('chance');
const chance = new ChanceJS();
const TAG = chance.string({
length: 12,
pool: 'abcdefghijklmnopqrstuvwxyz0123456789'
});
// This is our email address (for this test)
const TESTEMAIL = `${Cypress.env(
'TESTMAIL_NAMESPACE'
)}.${TAG}@inbox.testmail.app`;
// (Optional) Record the start time for the timestamp_from filter
const startTimestamp = Date.now();
// Use any test environment (this example uses cypress)
context('Some testing suite', () => {
before(() => {
// Call to your mail send function in the API:
cy.task('mailSend'); // This is a custom task, propbably a nodejs function that sends email to the TESTEMAIL
});
context('Verify email', () => {
let inbox;
before(done => {
// Create a spy for the API call
cy.intercept({
url: 'https://api.testmail.app/api/json'
}).as('getEmail');
// Query the inbox
axios
.get(
`${ENDPOINT}&tag=${TAG}×tamp_from=${startTimestamp}&livequery=true`
)
.then(response => {
console.log(response.data);
inbox = response.data;
done();
})
.catch(err => {
console.log(err);
done(err);
});
// Wait for the API call to finish
cy.wait('@getEmail')
.its('response.statusCode')
.should('be.oneOf', [200, 307])
.then(statusCode => {
if (statusCode === 307) {
// If the API returns 307, it means that the inbox is empty.
// We need to wait for the email to arrive.
cy.wait(5000);
}
});
});
it('The result should be successful', () => {
// Check the result
expect(inbox.result).to.equal('success');
});
it('There should be one email in the inbox', () => {
// Check the count of emails
expect(inbox.count).to.equal(1);
});
it('Get the email verification link', () => {
// Check the email details
expect(inbox.emails[0].from).to.equal(
'testmail app <[email protected]>'
);
expect(inbox.emails[0].subject).to.equal('Welcome aboard!');
// Extract the mail content
cy.wrap(inbox.emails[0].html).as('mailContent');
// Check the mail content
cy.get('@mailContent')
.should('contain', 'h1')
.should('contain', 'Welcome aboard!');
// Other checks...
});
});
});
Cypress is a free and open source, JavaScript-based testing tool with an MIT licence. Cypress makes it simple to write and debug high-quality E2E, integration, and unit tests.
Please have a look on the right for an example of how to use Testmail.app with Cypress. In this particular example, we are testing the Welcome Aboard mail via JSON API. We have also shown the use of NodeJs tasks in Cypress. You can assign any task to a NodeJs function and call it from your test. Here we used the NodeJs function to send the email. In your case, an email will be sent by your API.
Using Selenium
Please switch to the Javascript tab to see the code for using Selenium with JSON API.
Please switch to the Javascript tab to see the code for using Selenium with JSON API.
Please switch to the Javascript tab to see the code for using Selenium with JSON API.
Please switch to the Javascript tab to see the code for using Selenium with JSON API.
Please switch to the Javascript tab to see the code for using Selenium with JSON API.
Please switch to the Javascript tab to see the code for using Selenium with JSON API.
Please switch to the Javascript tab to see the code for using Selenium with JSON API.
/**
* Required environment variables: TESTMAIL_APIKEY and TESTMAIL_NAMESPACE
*/
require('chromedriver');
const axios = require('axios');
const ChanceJS = require('chance');
// import this classes from selenium
const { Builder, By, Key, until } = require('selenium-webdriver');
var assert = require('assert');
require('dotenv').config();
// import the function that sends email
const mailSend = require('./mailSend');
// Setup our JSON API endpoint
const ENDPOINT = `https://api.testmail.app/api/json?apikey=${process.env.TESTMAIL_APIKEY}&namespace=${process.env.TESTMAIL_NAMESPACE}`;
// Randomly generating the tag...
const chance = new ChanceJS();
const TAG = chance.string({
length: 12,
pool: 'abcdefghijklmnopqrstuvwxyz0123456789'
});
// (Optional) Record the start time for the timestamp_from filter
const startTimestamp = Date.now();
// describe test
describe('Test related to mail', function() {
let inbox, driver;
// create a selenium driver before the test starts
before(function() {
driver = new Builder().forBrowser('chrome').build(); // open chrome driver
mailSend(); // this is a function that sends email to the TESTEMAIL
// In this case, the email is sent before the test starts using the function
// but it will be sent by the website being tested using selenium in your case
});
// it describes expected behaviour when user perfroms search on google
it('Fetch mail from API', async function() {
// Query the inbox
const response = await axios.get(
`${ENDPOINT}&tag=${TAG}×tamp_from=${startTimestamp}&livequery=true`
);
inbox = response.data;
// Check if the response is successful
assert.equal(inbox.result, 'success');
});
it('Check if only one email is received', function() {
assert.equal(inbox.count, 1);
});
it('Check if email is received from the correct sender', function() {
assert.equal(
inbox.emails[0].from,
'testmail app <[email protected]>'
);
});
it('Check if email is received with the correct subject', function() {
assert.equal(inbox.emails[0].subject, 'Welcome aboard!');
});
it('Check if email is received with the correct body', function() {
// Check if the email body contains the expected text
let contentCheck = inbox.emails[0].html.includes('Welcome aboard!');
assert.equal(contentCheck, true);
});
// close the driver after the test is done
after(() => driver.quit());
});
Selenium WebDriver is a web-based automation testing platform that can test web pages launched in a variety of web browsers and operating systems. Furthermore, you are free to create test scripts in a variety of programming languages, including Java, Perl, Python, Ruby, C#, PHP, and JavaScript.
Please have a look on the right for an example of how to use Testmail.app with Selenium. In this particular example, we are testing the Welcome Aboard mail via JSON API. Here, a NodeJs function has been used to send the email. In your case, an email will be sent by your API when the automated test is running in Selenium.
Using TestCafe
/**
* Required environment variables: TESTMAIL_APIKEY and TESTMAIL_NAMESPACE
*/
require('dotenv').config();
const axios = require('axios');
const ChanceJS = require('chance');
// Setup our JSON API endpoint
const ENDPOINT = `https://api.testmail.app/api/json?apikey=${process.env.TESTMAIL_APIKEY}&namespace=${process.env.TESTMAIL_NAMESPACE}`;
// import the function that sends email
const mailSend = require('./mailSend');
// Randomly generating the tag...
const chance = new ChanceJS();
const TAG = chance.string({
length: 12,
pool: 'abcdefghijklmnopqrstuvwxyz0123456789'
});
// (Optional) Record the start time for the timestamp_from filter
const startTimestamp = Date.now();
let inbox;
// describe test
fixture('Test related to mail').before(() => {
mailSend(); // this is a function that sends email to the TESTEMAIL
// In this case, the email is sent before the test starts using the function
// but it will be sent by the website being tested using selenium in your case
});
test('Fetch mail from API', async t => {
// Query the inbox
// Query the inbox
const response = await axios.get(
`${ENDPOINT}&tag=${TAG}×tamp_from=${startTimestamp}&livequery=true`
);
inbox = response.data;
// Check if the response is successful
await t.expect(inbox.result).eql('success', { timeout: 5 * 60 * 1000 });
});
test('Check email details', async t => {
// Check the number of emails
await t.expect(inbox.count).eql(2);
// check the subject and from fields
await t.expect(inbox.emails[0].subject).eql('Welcome aboard!');
await t
.expect(inbox.emails[0].from)
.eql('testmail app <[email protected]>');
});
test('Check email body', async t => {
// Check the email content
await t.expect(inbox.emails[0].html).contains('Welcome aboard');
// Other checks
});
Please switch to the Javascript tab to see the code for using Testcafe with JSON API.
Please switch to the Javascript tab to see the code for using Testcafe with JSON API.
Please switch to the Javascript tab to see the code for using Testcafe with JSON API.
Please switch to the Javascript tab to see the code for using Testcafe with JSON API.
Please switch to the Javascript tab to see the code for using Testcafe with JSON API.
Please switch to the Javascript tab to see the code for using Testcafe with JSON API.
Please switch to the Javascript tab to see the code for using Testcafe with JSON API.
TestCafe is a complete end-to-end node.js solution for testing web applications. It handles each step, including launching browsers, starting tests, gathering test data, and producing reports. TestCafe doesn’t require any browser plugins; it functions right out of the box in all widely used modern browsers.
Please have a look on the right for an example of how to use Testmail.app with TestCafe. In this particular example, we are testing the Welcome Aboard mail via JSON API. Here, a NodeJs function has been used to send the email. In your case, an email will be sent by your API when the automated test is running in TestCafe.
Examples - GraphQL API
See the example on the right of an end-to-end test for our Welcome Aboard mail via GraphQL API. The same example via JSON API is available here.
Notes:
- You can use any GraphQL (or HTTP) client of your choice. The
@testmail.app/graphql-request
client used in this example is a fork of the popular graphql-request client with added features like built-in retries. - We use the chance package to randomly generate tags. This allows us to retrieve the email based on this random/unique tag.
- You can do a lot more with tags:
- You can use tags to categorize your emails (for example:
namespace.category.subcategory.randomstring
) and retrieve emails by category usingtag_prefix:"namespace.category"
- You can group emails by test and retrieve them together like this:
namespace.test_id.category
(retrieve usingtag_prefix:"namespace.test_id"
)
- You can use tags to categorize your emails (for example:
- We use a test timeout of 5 minutes: this is the maximum time we will wait for the email to be delivered. The vast majority of emails will be received within seconds (we are allowing up to 5 minutes for edge cases).
- We use
timestamp_from
to ensure that we are retrieving emails that were received after this test began. - The combination of using
timestamp_from
and a unique randomly generated string in the tag minimizes conflicts between parallel tests (queries from one test should not retrieve emails from another test).
Using Cypress
Please switch to the Javascript tab to see the code for using Cypress with GraphQL API.
Please switch to the Javascript tab to see the code for using Cypress with GraphQL API.
Please switch to the Javascript tab to see the code for using Cypress with GraphQL API.
Please switch to the Javascript tab to see the code for using Cypress with GraphQL API.
Please switch to the Javascript tab to see the code for using Cypress with GraphQL API.
Please switch to the Javascript tab to see the code for using Cypress with GraphQL API.
Please switch to the Javascript tab to see the code for using Cypress with GraphQL API.
/**
* Required environment variables: TESTMAIL_APIKEY and TESTMAIL_NAMESPACE
*/
// Use any GraphQL client (or HTTP client - but let's keep this simple)
const GraphQLClient = require('@testmail.app/graphql-request').GraphQLClient;
const testmailClient = new GraphQLClient(
// API endpoint:
'https://api.testmail.app/api/graphql',
// Use your API key:
{ headers: { Authorization: `Bearer ${Cypress.env('TESTMAIL_APIKEY')}` } }
);
// Randomly generating the tag...
const ChanceJS = require('chance');
const chance = new ChanceJS();
const TAG = chance.string({
length: 12,
pool: 'abcdefghijklmnopqrstuvwxyz0123456789'
});
// This is our email address (for this test)
const TESTEMAIL = `${Cypress.env(
'TESTMAIL_NAMESPACE'
)}.${TAG}@inbox.testmail.app`;
// (Optional) Record the start time for the timestamp_from filter
const startTimestamp = Date.now();
// Use any test environment (this example uses cypress)
context('Some testing suite', () => {
before(() => {
// Call to your mail send function in the API:
cy.task('mailSend'); // This is a custom task, propbably a nodejs function that sends email to the TESTEMAIL
});
context('Verify email', () => {
let inbox;
before(done => {
// Create a spy for the API call
cy.intercept({
url: 'https://api.testmail.app/api/graphql'
}).as('getEmail');
// Query the inbox
testmailClient
.request(
`{
inbox (
namespace:"${Cypress.env('TESTMAIL_NAMESPACE')}"
tag:"${TAG}"
timestamp_from:${startTimestamp}
livequery:true
) {
result
count
emails {
subject
html
text
from
}
}
}`
)
.then(data => {
console.log(data);
inbox = data.inbox;
done();
})
.catch(err => {
done(err);
});
// Wait for the API call to finish
cy.wait('@getEmail')
.its('response.statusCode')
.should('be.oneOf', [200, 307])
.then(statusCode => {
if (statusCode === 307) {
// If the API returns 307, it means that the inbox is empty.
// We need to wait for the email to arrive.
cy.wait(5000);
}
});
});
it('The result should be successful', () => {
// Check the result
expect(inbox.result).to.equal('success');
});
it('There should be one email in the inbox', () => {
// Check the count of emails
expect(inbox.count).to.equal(1);
});
it('Get the email verification link', () => {
// Check the email details
expect(inbox.emails[0].from).to.equal(
'testmail app <[email protected]>'
);
expect(inbox.emails[0].subject).to.equal('Welcome aboard!');
// Extract the mail content
cy.wrap(inbox.emails[0].html).as('mailContent');
// Check the mail content
cy.get('@mailContent')
.should('contain', 'h1')
.should('contain', 'Welcome aboard!');
// Other checks...
});
});
});
Cypress is a popular test automation framework. It is easy to use and has a lot of features. It is also very popular in the testing community. It is a great choice for testing your application.
The example on the right shows how to use Cypress with the GraphQL API. In this example we are using the graphql-request package to make the API call. The mail in this example is sent using a custom task. In your case, the mail will be sent by your application.
In this example, we are using the cypress-intercept
to intercept the API call and wait for the email to arrive. After the email arrives, we can check the email details and the mail content.
Using Selenium
Please switch to the Javascript tab to see the code for using Selenium with GraphQL API.
Please switch to the Javascript tab to see the code for using Selenium with GraphQL API.
Please switch to the Javascript tab to see the code for using Selenium with GraphQL API.
Please switch to the Javascript tab to see the code for using Selenium with GraphQL API.
Please switch to the Javascript tab to see the code for using Selenium with GraphQL API.
Please switch to the Javascript tab to see the code for using Selenium with GraphQL API.
Please switch to the Javascript tab to see the code for using Selenium with GraphQL API.
/**
* Required environment variables: TESTMAIL_APIKEY and TESTMAIL_NAMESPACE
*/
require('chromedriver');
const GraphQLClient = require('@testmail.app/graphql-request').GraphQLClient;
const ChanceJS = require('chance');
// import this classes from selenium
const { Builder, By, Key, until } = require('selenium-webdriver');
var assert = require('assert');
require('dotenv').config();
const testmailClient = new GraphQLClient(
// API endpoint:
'https://api.testmail.app/api/graphql',
// Use your API key:
{ headers: { Authorization: `Bearer ${process.env.TESTMAIL_APIKEY}` } }
);
// import the function that sends email
const mailSend = require('./mailSend');
// Randomly generating the tag...
const chance = new ChanceJS();
const TAG = chance.string({
length: 12,
pool: 'abcdefghijklmnopqrstuvwxyz0123456789'
});
// (Optional) Record the start time for the timestamp_from filter
const startTimestamp = Date.now();
// describe test
describe('Test related to mail', function() {
let inbox, driver;
// create a selenium driver before the test starts
before(function() {
driver = new Builder().forBrowser('chrome').build(); // open chrome driver
mailSend(); // this is a function that sends email to the TESTEMAIL
// In this case, the email is sent before the test starts using the function
// but it will be sent by the website being tested using selenium in your case
});
// it describes expected behaviour when user perfroms search on google
it('Fetch mail from API', async function() {
// Query the inbox
const response = await testmailClient.request(
`{
inbox (
namespace:"${process.env.TESTMAIL_NAMESPACE}"
tag:"${TAG}"
timestamp_from:${startTimestamp}
livequery:true
) {
result
count
emails {
subject
html
text
from
}
}
}`
);
inbox = response.inbox;
// Check if the response is successful
assert.equal(inbox.result, 'success');
});
it('Check if only one email is received', function() {
assert.equal(inbox.count, 1);
});
it('Check if email is received from the correct sender', function() {
assert.equal(
inbox.emails[0].from,
'testmail app <[email protected]>'
);
});
it('Check if email is received with the correct subject', function() {
assert.equal(inbox.emails[0].subject, 'Welcome aboard!');
});
it('Check if email is received with the correct body', function() {
// Check if the email body contains the expected text
let contentCheck = inbox.emails[0].html.includes('Welcome aboard!');
assert.equal(contentCheck, true);
});
// close the driver after the test is done
after(() => driver.quit());
});
Selenium WebDriver is a web-based automation testing platform that can test web pages launched in a variety of web browsers and operating systems. Furthermore, you are free to create test scripts in a variety of programming languages, including Java, Perl, Python, Ruby, C#, PHP, and JavaScript.
Please have a look on the right for an example of how to use Testmail.app with Selenium. In this particular example, we are testing the Welcome Aboard mail via GraphQL API. Here, a NodeJs function has been used to send the email. In your case, an email will be sent by your API when the automated test is running in Selenium.
Using Testcafe
Please switch to the Javascript tab to see the code for using TestCafe with GraphQL API.
Please switch to the Javascript tab to see the code for using TestCafe with GraphQL API.
Please switch to the Javascript tab to see the code for using TestCafe with GraphQL API.
Please switch to the Javascript tab to see the code for using TestCafe with GraphQL API.
Please switch to the Javascript tab to see the code for using TestCafe with GraphQL API.
Please switch to the Javascript tab to see the code for using TestCafe with GraphQL API.
Please switch to the Javascript tab to see the code for using TestCafe with GraphQL API.
/**
* Required environment variables: TESTMAIL_APIKEY and TESTMAIL_NAMESPACE
*/
require('dotenv').config();
const GraphQLClient = require('@testmail.app/graphql-request').GraphQLClient;
const ChanceJS = require('chance');
const testmailClient = new GraphQLClient(
// API endpoint:
'https://api.testmail.app/api/graphql',
// Use your API key:
{ headers: { Authorization: `Bearer ${process.env.TESTMAIL_APIKEY}` } }
);
// import the function that sends email
const mailSend = require('./mailSend');
// Randomly generating the tag...
const chance = new ChanceJS();
const TAG = chance.string({
length: 12,
pool: 'abcdefghijklmnopqrstuvwxyz0123456789'
});
// (Optional) Record the start time for the timestamp_from filter
const startTimestamp = Date.now();
let inbox;
// describe test
fixture('Test related to mail').before(() => {
mailSend(); // this is a function that sends email to the TESTEMAIL
// In this case, the email is sent before the test starts using the function
// but it will be sent by the website being tested using Testcafe in your case
});
test('Fetch mail from API', async t => {
// Query the inbox
const response = await testmailClient.request(
`{
inbox (
namespace:"${process.env.TESTMAIL_NAMESPACE}"
tag:"${TAG}"
timestamp_from:${startTimestamp}
livequery:true
) {
result
count
emails {
subject
html
text
from
}
}
}`
);
inbox = response.inbox;
// Check if the response is successful
await t.expect(inbox.result).eql('success', { timeout: 5 * 60 * 1000 });
});
test('Check email details', async t => {
// Check the number of emails
await t.expect(inbox.count).eql(2);
// check the subject and from fields
await t.expect(inbox.emails[0].subject).eql('Welcome aboard!');
await t
.expect(inbox.emails[0].from)
.eql('testmail app <[email protected]>');
});
test('Check email body', async t => {
// Check the email content
await t.expect(inbox.emails[0].html).contains('Welcome aboard');
// Other checks
});
TestCafe is a complete end-to-end node.js solution for testing web applications. It handles each step, including launching browsers, starting tests, gathering test data, and producing reports. TestCafe doesn’t require any browser plugins; it functions right out of the box in all widely used modern browsers.
Please have a look on the right for an example of how to use Testmail.app with TestCafe. In this example, we are using the GraphQL API to fetch the email. We are using the before
hook to send the email before the test starts. We are using the test
hook to fetch the email and check the email details. In your case, you will be using the test
hook to run the test on the website and the email will be sent by your API.
Troubleshooting
Power users may occasionally encounter the issues described below (they should be extremely rare for most users).
Timeouts (307 loops)
Timeouts during live queries (in the form of 307 redirects every minute forever) are the most common issue. There are several reasons for this:
- Your emails can simply fail to send.
Of course: this is what we are testing in the first place! If your app is not sending the right emails or fails to send emails at all, the tests will naturally timeout or fail (as they should).
- Time to inbox can vary occasionally.
While the vast majority of emails are received within a few seconds, some emails can take 5 minutes or more, and on rare occasions emails can take days to deliver! When emails fail to deliver (e.g. because of a connection issue), the sending server usually makes a certain number of retry attempts with an increasing time interval between retries. So when you send many emails, and when you’ve integrated testmail in your CI/CD pipeline, you will inevitably encounter edge cases.
To minimize build timeouts while waiting to receive emails, make sure the timeout configured in your test environment is sufficiently high. We use 5 minutes in our pipeline, but we can’t recommend this number for everyone. Here’s the tradeoff: if the timeout is too low, you’ll get more build failures (you’ll just have to retry/restart these builds). If the timeout is too high, the edge cases can result in builds that take very long and clog your pipeline.
- The quality (and configuration) of your email sending server/provider can make a big difference.
If your mail server IP address or ISP has a bad reputation, spam filters can reject or delay your emails. Misconfigurations on the sending server can cause delays, deliverability problems, and failures. One key benefit of end-to-end testing is discovering these problems in advance - before your users experience them.
Here’s a real example: we signed up for Mailgun and started testing email sends. We noticed that many of our emails were not reaching the inbox. This was because, as a new customer, Mailgun had put us in a shared pool of IPs with a poor spam reputation. After reaching out to their support team and verifying our business (the types of emails we send, email list management policies, privacy policy, etc.), they upgraded us to an IP pool with a better reputation. This is not an endorsement or criticism of Mailgun; every good email provider has to do this (in some way) to prevent spammers from signing up and sending bulk emails from their quality IPs.
Rate limiting (429 errors)
We use API rate limits to prevent abuse and survive common developer mistakes (like infinite loops) that we are exposed to as a developer tool with public APIs.
We have two layers of rate limits that mitigate different types of stresses on our system:
Global API rate limits
You may trigger global rate limits if you sustain >10 requests/second for many minutes from the same IP. Global rate limits apply to all requests from the same IP across all APIs. Please reach out if your use case requires higher global rate limits - we can whitelist select IPs. Note that we may adjust global rate limits occasionally to mitigate DDoS attacks and other malicious activity.
API key rate limits
On paid plans you may trigger the API key rate limit if you exceed:
- >5 requests/second per API key sustained for an hour or more across all IPs.
We can increase this limit for your API keys if your use case requires it - just let us know.
Individual API keys that breach this limit are temporarily blacklisted for up to an hour. We implemented this limit to survive edge cases like infinite loops inside matrix builds in CI/CD pipelines where the loop is parallelized across many machines with different IPs - creating DDoS monsters!
On free plans you may trigger API key rate limits if you exceed:
- >1,000 requests/hour
- >10,000 requests/day
- >100,000 requests/month
We implemented these free plan limits to prevent anti-pattern use that consumes excessive resources (e.g. querying the inbox for new emails every second). Most free users never exceed these limits. If you breach them, please upgrade to any paid plan.
5xx status codes
5xx class errors (500, 502, 503, 504) should be extremely rare: we go to great lengths to maintain high availability and uptime. However, a small number of these are inevitable in any environment at scale.
To avoid these edge cases, we recommend configuring automatic retries on your client. Our @testmail.app/graphql-request
client includes automatic retries with exponential backoff by default.
Performance
You can expect excellent performance when searching, retrieving, and filtering emails by namespace, tag, and timestamp. However, using advanced_filters
on other fields is generally less performant - but you’re unlikely to notice this until you’re working with very large result sets.
Full-text search
The GraphQL API includes features like wildcard search on the text/html email body. These features have some limits:
- Using wildcard search on a large result set will compromise your query’s performance. To avoid this, use the namespace, tag, and timestamp range filters alongside to reduce the size of the result set.
- If the size of the field (e.g. text or html) exceeds 30kb, the field might not be indexed for search (wildcards won’t work). You can still query these emails - just use any other field (namespace, tag, from, to, subject, etc.)
Consistency
In very rare cases, two identical API calls may return different results. This is an inevitable tradeoff from prioritizing high availability in distributed systems (see CAP theorem). We have not seen any use cases for testmail.app where this might be a problem - if you can think of one, let us know :)
Get Help
If you cannot find answers, please reach out - we are happy to help. You can reach us via email or live chat. Pro and enterprise customers should use their registered emails or signin for priority support.
Support requests are handled by software engineers who actively build and maintain testmail.app
We typically respond within a few hours. Priority support requests are often answered within minutes.