Measuring Inbox/Spam placement

One common use case for syncing an IMAP mailbox with EmailEngine is to measure if emails are going to INBOX (or in Gmail's case, to promotions) or to the Spam folder.

EmailEngine is a self hosted application to access IMAP accounts over a REST API. EmailEngine also sends webhooks about any changes on the account.

One common use case for syncing an IMAP mailbox with EmailEngine is to measure if emails are going to INBOX (or in Gmail's case, to promotions) or to the Spam folder. This way we can send emails to that account and see what the email service provider thinks of us.

Such an analysis is a bit different from normal syncing. Usually, there is no noticeable delay when an email is put into one of these target folders and we would like to get the results immediately as well. It is simpler with INBOX placement as EmailEngine is already following that folder but to detect new messages in the Spam folder EmailEngine would have to perform periodic polling checks which always happen with a noticeable delay.

The solution is to add multiple EmailEngine checks, one for each target folder.

At first, we should check which kind of folders there even are on the account. We can use /v1/verifyAccount endpoint with this by setting mailboxes option to true

curl -XPOST "http://127.0.0.1:3000/v1/verifyAccount" -H "content-type: application/json" -d '{
    "imap": {
        "host": "imap.ethereal.email",
        "port": 993,
        "secure": true,
        "auth": {
            "user": "pansy.watsica@ethereal.email",
            "pass": "h1dW7F8GVfyfjn8ENR"
        }
    },
    "mailboxes": true
}'

If IMAP settings check out then we should get a successful result together with a folder listing

{
  "imap": {
    "success": true
  },
  "mailboxes": [
    {
      "path": "INBOX",
      "delimiter": "/",
      "listed": true,
      "specialUse": "\\Inbox",
      "name": "INBOX",
      "subscribed": true
    },
    ...
    {
      "path": "Junk",
      "delimiter": "/",
      "listed": true,
      "name": "Junk",
      "specialUse": "\\Junk",
      "subscribed": true
    },
    ...
  ]
}

So now we know that there are folders INBOX and "Junk" that interests us.

Next, we should register 2 EmailEngine accounts, one for each folder. Normally EmailEngine syncs all the folders on an account but in this case, we are only interested in the contents of these specific folders and want to ignore everything else.

When registering the account we should use the path property – this ensures that EmailEngine is syncing data only from that specific path and does not touch other folders.

First, we register an account for the INBOX folder:

curl -XPOST "http://127.0.0.1:3000/v1/account" -H "content-type: application/json" -d '{
    "account": "pansy_inbox",
    "name": "Pansy Watsica INBOX",
    "path": "INBOX",
    "imap": {
        "host": "imap.ethereal.email",
        "port": 993,
        "secure": true,
        "auth": {
            "user": "pansy.watsica@ethereal.email",
            "pass": "h1dW7F8GVfyfjn8ENR"
        }
    }
}'

and then the same but for the Spam folder. Also be aware that even though it's the same IMAP account we have to use a different account id for EmailEngine, otherwise we would be only modifying the already existing INBOX handling account.

curl -XPOST "http://127.0.0.1:3000/v1/account" -H "content-type: application/json" -d '{
    "account": "pansy_spam",
    "name": "Pansy Watsica SPAM",
    "path": "Junk",
    "imap": {
        "host": "imap.ethereal.email",
        "port": 993,
        "secure": true,
        "auth": {
            "user": "pansy.watsica@ethereal.email",
            "pass": "h1dW7F8GVfyfjn8ENR"
        }
    }
}'

Now, whenever a message lands in the INBOX or Spam folder we immediately get a webhook notification. If the email was sent by us and the webhook notification is from the account "pansy_inbox" then the email ended up in INBOX, if it's from "pansy_spam" then we failed and were deemed as a spammer.

It's a bit more complicated in case of Gmail if we want to get the promotion tab info as well and not just the INBOX info. In this case we can register an account not for the INBOX folder but for the All Mail folder and when receiving notifications about new emails we should check the Gmail Labels property to determine the actual placement of that email.