Transactional email service

EmailEngine is a self-hosted open source application that provides a REST interface on top of any email account you have access to read and send emails. Combining these features would give you a similar email sending experience you'd get from a transactional email service.

EmailEngine allows converting any email account into a poor-man's transactional email service. With it you can submit an email message for delivery, EmailEngine queues it and then tries to relay that message to the SMTP server of the sending account.

  • Emails can be submitted for delivery either via REST API or SMTP
  • Sending can be scheduled by providing a future date for delivery
  • When an email bounces a webhook notification is sent with the details of that bounce
  • Sent emails are automatically uploaded to the "Sent Mail" folder
  • If the sent email has a reference to a previous email then this email is marked as Answered

Delivery via API

To submit an email for delivery using the EmailEngine API you can POST structured message details to the /v1/account/{account}/submit endpoint (see the details for that endpoint here). As the message is provided as a structured object then you do not need to know anything MIME specific. EmailEngine takes the input (Unicode for strings, base64 for binary data like attachments) and converts it to a valid RFC822 message file that is then sent to SMTP.

Here's an example of such a submission. Note that this example has no subject value set as the subject can be derived from the referenced message. In this case, EmailEngine would also make sure that headers like In-Reply-To and References are properly set. If you do want to use a different subject you can still set the subject property to do so.

$ curl -XPOST "localhost:3000/v1/account/example/submit" -H "content-type: application/json" -d '{
    "reference": {
        "message": "AAAAAQAAP1w",
        "action": "reply"
    },
    "from": {
        "name": "Example Sender",
        "address": "andris@kreata.ee"
    },
    "to": [{
        "name": "Andris Reinman",
        "address": "andris@ethereal.email"
    }],
    "text": "test",
    "html": "<p>test</p>",
    "attachments": [
        {
            "filename": "checkmark.png",
            "content": "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQAQMAAAAlPW0iAAAABlBMVEUAAAD///+l2Z/dAAAAM0lEQVR4nGP4/5/h/1+G/58ZDrAz3D/McH8yw83NDDeNGe4Ug9C9zwz3gVLMDA/A6P9/AFGGFyjOXZtQAAAAAElFTkSuQmCC"
        }
    ]
}'

For successful submission you should get a response that looks like the following:

{
  "response": "Queued for delivery",
  "messageId": "<188db4df-3abb-806c-94c8-7a9303652c50@kreata.ee>",
  "sendAt": "2021-08-16T15:04:46.161Z",
  "queueId": "24279fb3e0dff64e"
}

This does not yet mean that the message would have been relayed to the SMTP server. It means that EmailEngine has queued this message internally and tries to deliver it to the SMTP server as soon as possible (or if sendAt value was used then at the specified time). Once the message has been actually passed over to the account's SMTP server (or the server rejects it) EmailEngine sends a webhook  notification.

{
  "account": "example",
  "date": "2021-08-16T15:04:47.047Z",
  "event": "messageSent",
  "data": {
    "messageId": "<188db4df-3abb-806c-94c8-7a9303652c50@kreata.ee>",
    "response": "250 2.0.0 OK  1629126286 z8sm965227lfs.177 - gsmtp",
    "queueId": "24279fb3e0dff64e",
    "envelope": {
      "from": "andris@kreata.ee",
      "to": [
        "andris@ethereal.email"
      ]
    }
  }
}

Delivery via SMTP

SMTP sending is not enabled by default with EmailEngine. In order to enable it you would have to provide a secret for the SMTP server that is then used as the authentication password for SMTP clients (authentication username is the account Id you'd want to use to send email).

To enable the SMTP server, set the environment variables EENGINE_SMTP_ENABLDED and EENGINE_SMTP_SECRET:

$ export EENGINE_SMTP_ENABLED=true
$ export EENGINE_SMTP_SECRET=saladus
$ emailengine

By default EmailEngine's SMTP relay is listening on port 2525, localhost only. If you want to change the port or host use EENGINE_SMTP_PORT and EENGINE_SMTP_HOST variables.

In general, you would be using EmailEngine's SMTP like any other SMTP server. There are some things to consider though.

  • Use S-EE-Send-At header to schedule future delivery time
  • If your message includes a Bcc header then it is removed from the message automatically
  • EmailEngine ensures that mandatory headers like Message-Id, MIME-Version and Date would be set
  • There is no TLS support, make sure that your SMTP client is configured to use cleartext connection
  • If you really have to support TLS for SMTP then you can use HAProxy with send-proxy option in front of EmailEngine (make sure though that EENGINE_SMTP_PROXY value is set to true so that EmailEngine would be able to use the HAProxy protocol)

As SMTP is a text-based protocol you can test it straight from the shell. First, you'd need the authentication argument for the AUTH PLAIN command to authenticate yourself. Other email protocols allow authentication with regular text username and password which makes it easier to do manually but all SMTP authentication mechanisms require these to be encoded in base64.

If account ID is example and EENGINE_SMTP_SECRET value is saladus then you can generate the authentication argument in the shell like this (note the mandatory \0 sequences in front of the values):

$ echo -ne "\0example\0saladus" | base64
AGV4YW1wbGUAc2FsYWR1cw==

Next, you can connect to the SMTP relay with telnet or netcat:

$ telnet localhost 2525

or

$ nc -c localhost 2525

Once connected run the SMTP handshake and authenticate yourself with the following commands

EHLO foo
AUTH PLAIN AGV4YW1wbGUAc2FsYWR1cw==

The server should respond to the second command with 235 Authentication successful.

To send an actual email use the following command sequence. MAIL FROM can only be issued once. RCPT TO can be run many times. Once you send the DATA command you can't change the recipients list anymore, instead, you'd have to enter the MIME formatted message, and then, on a separate line, send the dot symbol to complete the transaction.

MAIL FROM:<andris@kreata.ee>
RCPT TO:<andris@ethereal.email>
DATA
From: andris@kreata.ee
To: andris@ethereal.email
Subject: hello world!

Hello!
.

Missing mandatory headers like "Date" are inserted by EmailEngine automatically
If you have previously not used SMTP directly then beware that the email is only sent to addresses submitted using the RCPT TO command. The addresses in the message's To, Cc and Bcc headers are informational only and not used in email routing

The server should respond with a success message

250 Message queued for delivery as 88f430cc6dbe9a65 (2021-08-16T12:15:17.408Z)
The timestamp in the response message shows the scheduled sending time

Scheduled sending

In general, EmailEngine tries to relay the message as soon as possible. If you want you can delay that sending to a suitable future time by using message scheduling. Emails can be scheduled both when sending via the API and SMTP.

Scheduling with the API can be done using the sendAt property, where the value would be the ISO formatted timestamp of the time the message, should be sent.

$ curl -XPOST "localhost:3000/v1/account/example/submit" -H "content-type: application/json" -d '{
    "from": {
        "name": "Example Sender",
        "address": "andris@kreata.ee"
    },
    "to": [{
        "name": "Andris Reinman",
        "address": "andris@ethereal.email"
    }],
    "subject": "Scheduled email",
    "text": "test",
    "html": "<p>test</p>",
    "sendAt": "2021-08-18T06:42:25.408Z"
}'

For SMTP add an additional X-EE-Send-At header with the timestamp value to the message to schedule it. That header is then removed by the server from the message before delivering it.

From: andris@kreata.ee
To: andris@ethereal.email
Subject: hello world!
X-EE-Send-At: 2021-08-18T06:42:25.408Z

Hello!

Once the message gets delivered (or rejected) you'd get a regular webhook notification about it.

Bounce detection

As EmailEngine can access both SMTP and IMAP data then one of the nifty features is to automatically pick up information about bounced emails from the IMAP account. For example, when you try to send to a non-existing email address, the email is at first accepted for delivery and if it later fails with a bounce response you'd get a notification about the failure.

{
  "account": "example",
  "date": "2021-08-16T11:55:06.107Z",
  "event": "messageBounce",
  "data": {
    "bounceMessage": "AAAAAgAAxxk",
    "recipient": "asfassdf@ethereal.email",
    "action": "failed",
    "response": {
      "source": "smtp",
      "message": "550 5.1.1 No such user",
      "status": "5.1.1"
    },
    "mta": "mx.ethereal.email. (13.49.22.0, the server for the domain ethereal.email.)",
    "messageId": "<19f1157c-d72b-50eb-74d5-d30f9ec816d3@kreata.ee>"
  }
}

Note that though the bounce info includes a reference ID to the bounce response email (that is the email that was sent by the MTA server and includes the bounce response, in this case "AAAAAgAAxxk") it does not have a reference to the original message that was sent and that got rejected. Instead, you get only Message-ID value that you can use to match the original message yourself. This is because EmailEngine does not keep a registry of Message-ID values and it does not know from which folder to look for that message. This behavior might change in the future though.