Email Delivery Configuration — Email Relay
Introduction
This a high level overview of how Email Delivery will flow from LiquidFiles when using an Email Relay Server:
Please note that an Email Relay Server in this example can be a local Microsoft Exchange Server, Microsoft Office 365 cloud email server, Google Mail, Amazon SES (Simple Email Service) or pretty much any other email server or service that can accept emails and forward them on your behalf. Feel free to do a web search for SMTP Relay Service to get more options.
Fundamental Concepts
(SMTP) Email is a pass the bucket system. You deliver the email to a Mail Server, that delivers it to the next Mail Server, that delivers it to the next Mail Server... Until it has reached the final mailbox, or one of the Mail Servers decides to either Drop or Reject the email.
Each Mail server on the way are responsible to deliver the email to the next Mail Server. After the email has been accepted by the next mail server the email is no longer the original or current mail servers responsibility.
Further to this concept, once an email has been accepted by the next mail server, there's Nothing that LiquidFiles or the current mail server can do about that email.
Troublshooting Email Delivery then boils down to investigate if the email was Accepted by the next mail server or not. If it was accepted, we have to move to the next mail server and investigate there, and the next mail server, until we eventually find either that the email was successfully delivered, or it was dropped or rejected by the email server we're currently investigating. If an email is dropped or rejected, there will be a log to why the email was dropped or rejected.
If a mail server drops an email it "should" send a non-delivery message back to sender, but for security reasons many do not.
Configuration
Network Level Configuration
In most cases, all the configuration that's required to configure sending emails using an Email Relay Server or Service is in the Email Relay section in Admin → Configuration → Email, looking like this:
The only required options is Email Relay host which can be something like:
- mail.liquidftest.com — lookup the name mail.liquidftest.com and use the default SMTP port TCP 25 to deliver the email to the Email Relay host.
- mail.liquidftest.com:587 — lookup the name mail.liquidftest.com and use the specified TCP port 587 to deliver the email to the Email Relay host.
- 10.2.3.20:587 — use the specified IP address and TCP port 587 to deliver the email to the Email Relay host.
Some Email Relay Servers or Services will require Authentication and if required you can specify a username and password that will be used for
Email Security Level
By default, LiquidFiles will try to deliver the email with TLS encryption. In some cases, you may need to tweak this configuration to match your Email Relay Server setting, please consult the support or documentation of the Email Relay Server or Service if you think you need to adjust this configuration.
Email Sender Configuration
When an email is being delivered, it has a typical header looking like:
From: Friendly Name <someuser@mycompany.com> To: somerecipient@theircompany.com Subject: Some Subject
The important bit for this configuration is the From: header. The From: header is made up of a Friendly Name, that could be anything, and the actual email address within brackets.
For security reasons many Email Servers will only allow you to send as the user you've authenticated as. In the Email Relay Username above,
you can see the username being
This setting can be configured at the top of Admin → Configuration → Email:
If the "Use The Email Sender Address as from address for all emails" is enabled and the Email Sender address is set as: support@liquidftest.com, if a user john.doe@liquidftest.com is logged in and sends a message, the Email Header From address will read:
From: john.doe@liquidftest.com <support@liquidftest.com> To: somerecipient@theircompany.com Subject: Some Subject Reply-To: john.doe@liquidftest.com
This satisfies the requirement that all emails will use the Email Sender Address as the From address when sending emails, while still showing the end recipient who the real sender is, and also allows them the hit "Reply" in their email client and having the emails come back to john.doe@liquidftest.com.
Some people don't like this, and strictly speaking, sending on behalf of someone else could be seen as not correct from a strict email
policy perspective, and LiquidFiles will cater for this stand as well. If you disable the "Use The Email Sender Address as from
address for all emails" setting, with the same premise that
From: john.doe@liquidftest.com To: somerecipient@theircompany.com Subject: Some Subject
Please check with your email relay server or service. Some have configuration that will enable them to act as an "unrestricted" email relay server or service, allowing any From address to being used. If you can configure your Email relay server or service in this way, feel free to use this configuration.
Mail Relay Delivery & Troubleshooting
The first place to look is in the Admin → System → Email Queue, where you will see something like this:
If you see any emails in the Email Queue, that means that they haven't been able to be delivered. You will also see an explanation in the Info field why this particular email hasn't been able to be delivered.
Often these errors are pretty self-explanatory if you have experience troubleshooting Internet network related issues. Typically the errors will be something along the lines of:
- Connection Timed Out — meaning that a firewall or similar has blocked the connection.
- Connection Refused — meaning that the connection was permitted by any firewall but blocked by the server it was trying to connect to. Best guess would be incorrect hostname or IP address for the email relay server.
- Lookup Error — meaning that the LiquidFiles system could not lookup the hostname using the DNS server you've configured.
- Permission Denied — meaning that the Email server in the other and did respond but refused the email. This means that there's something wrong in either the configuration in the LiquidFiles end (most email relay servers require that you use the same Email Sender Address as the Email Relay username), or in the other end (you need to enable email relay from the LiquidFiles system).
Also, please note that SMTP or Internet Based Email systems have builtin resiliency and just because an Email is listed in the Email Queue it doesn't always because of a problem. There could just as well be a temporary routing issue, connection issue, power outtage, reboot of some network equipemnt or the recipient email server that is causing the connection to fail temporarily. If it does, the email will be queued for a while and then a redelivery will be attempted.
Investigating Email Delivery Issues using the System Log
To further expand on the troubleshooting, we can go back to the diagram at the top of the page, and look at the steps taken when delivering an email using an Email Relay server, the following steps will be taken:
- The LiquidFiles Appliance delivers the email to the configured email relay server.
- The email relay server will either relay the email to the next mail server or lookup in DNS and send the email towards the recipient.
Responses
To see if an Email has been successfully been delivered or not, we will need to look in the Email Log, available in the Admin → Activity Log.
As explained in the Fundamental Concepts above, what we're investigating is if the email was accepted by the Email Relay Server or not:
The highlighted row shows an email that has been successfully delivered using an Email Relay Server with IP: 172.16.5.1.
What we're looking for in this instance is the status=sent, that determines that the email was successfully delivered.
Looking in any (SMTP) Email Log, there are only three types of responses:
- status=sent – Message has been delivered successfully
- status=deferred – Message has not been delivered with a temporary failure
- status=rejected – Message has not been delivered with a permanent failure
When we get a status=sent (will also be listed as a 2xx (200-299) Ok message), this means that the Email Relay has accepted the email for delivery, and all we can really do from the LiquidFiles server itself.
A 4xx temporary (Deferred) failure means that there's a temporary failure in the Email Relay Server. What happens with temporary failure is that the LiquidFiles appliance will queue the message and try to deliver again. After retrying for a couple of days, the LiquidFiles appliance will drop (delete) the message and send a bounce message (from Mailer Daemon) that the message could not be delivered.
A 5xx permanent (Reject) failure means that the receiving mail server is not accepting the message at all. This could be anything from that our sending ip address is in a list of addresses that the receiving mail server things is from a known spammer (or home network), to that the email FROM address is invalid, or that we're trying to send to an non-existent email address (there's no email address homer@simpsons.com in the simpsons.com domain).
In all of these cases, the mail log is your friend.
The Mail Log — Successful Delivery
The by far best way to troubleshoot mail delivery problems is by looking at the maillog. In the LiquidFiles appliance, this can be done in the Admin → System Log, and filter on "Mail". Then we can search on the recipient, to see what's going on. Please see the following:
In this example, the important thing to notice is the status=sent followed by the code 250 2.0.0 Ok. Any code with status=sent means that the email was delivered as expected.
At this point, there's nothing more that can be done from the LiquidFiles appliance point of view. It should now be delivered to the end user. But if it isn't, it's because a receiving mail server up the path is not delivering it. If the message does not appear, it would have to be troubleshooted, first in the mail.simpsons.com email server.
The Mail Log — Warnings
An Example warning (temporary failure would look something like this):
In this example, you can see the status=deferred which means that there's a temporary failure. In this case the error is "Connection Refused" which means that LiquidFiles could not connect to the configured Email Relay Server, 172.16.5.1 on port 1025. Connection Refused, as in this case, is typically because of a firewall or similar blocking the connection but you would need to start troubleshooting on the network layer between LiquidFiles and the Email Relay Server.
Mail Queue
Another good place to look is in Admin → System → Mail Queue:
The Mail Queue will list all emails that are currently in Queue to be delivered together with the reason they haven't yet been delivered.
The Mail Log — Gmail Failure
If you want to relay emails using Google, you can follow their guide here.
In this first example, let's purposefully configure things with the wrong name:
You can look either in Admin → Activity Log, or in this case, we can see a mail in the Admin → System → Mail Queue:
The error described here is that the LiquidFiles system cannot find the Host or Domain name for: smtp.google.com. Which makes sense because their actual hostname is smtp-relay.google.com:
But still, we get errors, looking in the Mail Log:
You can see that we can now connect just fine to smtp-relay.google.com (using TLSv1.2) but we still get a status=bounced with a 5xx error (550 in this case), and reading further we can see Invalid Credentials (wrong username and/or password) or that the IP address isn't registered as required in the Google G-Suite configuration.
At this point, there's not a lot that can be done from the LiquidFiles system. This is all configuration now that needs to happen within Google, and if you update the Google G-Suite configuration using this information, you could use LiquidFiles to relay emails using smtp-relay.google.com. But the point here was more to show different error messages.
Conclusion
A few things to note from this article, first — please make sure that you match the configuration that your chosen Email Relay Server or Service requires. Different products and services have different requirements and configuration so please make sure you consult either the documentation or the respective vendor support. Once you have the required configuration, it should be pretty simple to match the configuration in LiquidFiles.
The second thing to note is — the Mail Log and the Mail Queue is your friend. If there's delivery errors or problems, it will be highlighted there. You can search for the sender or recipient email address and it should be easy to find information if a message has been sent successfully (status=sent) or not, and then adjust the configuration accordingly.