...making Linux just a little more fun!

Getting Started with the Exim Mail Server

By Neil Youngman

Introduction

Most Linux users don't need to think about mail servers. A typical Linux installation will just have the distribution's default mail server installed, and it will do very little.

If you need more than that, and you want a FOSS mail server, you will find that there are 3 popular choices. They are Sendmail, Postfix, and Exim. This article is about Exim, which is my mail server of choice.

Exim is a very full-featured mail server. It is suitable for most sites, although very small sites might prefer a simpler alternative such as msmtp.

I have been using Exim for over 3 years. It has proved to be efficient, flexible, and reliable, and it provides all the features I require. Some of the features that you may find useful are Sendmail compatibility, authentication, encryption, mail relaying, and easy integration with a number of popular databases, as well as virus and spam scanners.

Exim is highly configurable. The Exim site hosts a full manual online, but wading through it for the first time can be quite daunting. This article aims to give a quick introduction to the key configuration items. Once you are comfortable with those, you can browse the manual and the Wiki to find other features which may be useful to you.

The configuration file provides a wide range of built-in options, including hooks allowing file and database lookups and easy integration with SpamAssassin, ClamAV, and other spam and virus scanning programs. Some options control which features are provided, others allow Exim to handle all sorts of misfeatures in other mail programs with which it needs to communicate. If the right build options are selected, it also provides hooks into Perl and C programs that can be used to provide custom functionality.

At the time of writing, the current version of Exim is 4.69. If you are using any other version you may find that some features differ, but Exim is very stable and I would expect any recent version to support all the features in this article.

The Mail Life Cycle

Exim handles mail in 3 phases. They are acceptance, routing, and delivery.

Acceptance is controlled by Access Control Lists (ACLs). ACLs determine from whom you accept email, for which destinations, and any other conditions you care to impose.

Routing is controlled by routers. Routers determine where Exim will deliver the mail. Usually this will be either to another mail server or to a local mailbox.

Delivery is controlled by transports. These determine the actual delivery mechanism. Mail may be passed to another mail server, written to a mailbox, or piped to a command to process it.

The Configuration File

The configuration file comprises five main sections: global configuration options, ACLs, routers, transports, and retry rules.

The default configuration file shipped with the source is extremely well commented. I would not recommend starting a new configuration from scratch; it will almost always be better to start with the default configuration and adapt that to your requirements. The default configuration for Exim 4.69 can be found here.

Note that the configuration file shipped with the Exim package for your distribution may not be the default configuration shipped with the source.


[Sidebar begins here]

SMTP and ESMTP

The Simple Mail Transfer Protocol (SMTP) and the Extended Simple Mail Transfer Protocol (ESMTP) are the main protocols used by mail servers to communicate with each other and by mail clients to submit messages to a server. A typical SMTP session looks like

220 mail.example.net ESMTP Exim 4.69 Sun, 23 Nov 2008 14:29:20 +0000
HELO client.example.net
250 test.wirefast.net Hello client.example.net [10.20.30.40]
MAIL FROM:<gazetteer@example.net>
250 OK
RCPT TO:<user@example.net>
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself
From:<gazetteer@example.net>
To:<user@example.net>
Subject: Isn't Exim great

You really should check out the article on Exim in this month's
Linux Gazette.

Sincerely

Gaz E Teer
.
250 Message accepted
QUIT
221 mail.example.net closing connection

This can be summarised as the server identifying itself (Exim banner), after which the sender introduces itself (the HELO command). When the HELO command has been accepted, the sender says, I have some mail from gazetteer@example.net (the MAIL FROM command). The server accepts this and the sender asks can you deliver it to user@example.net (the RCPT TO command)? The server says it can, so the sender gives it the message (the DATA command). A line consisting of a single '.' marks the end of the message and the sender ends the conversation with the QUIT command.

The first thing to notice is that the banner and all the responses from the mail server start with a 3-digit code. These codes tell the client whether a command or connection has been accepted. The human-readable text in server responses is not a significant part of the protocol, but it can be useful when a human is following the session. The first digit indicates success or failure. Codes starting with 2 indicate success, 3 indicates a prompt for further data. 4 indicates a temporary rejection, such as an over quota mailbox and 5 indicates a permanent rejection, such as a non-existent user.

RFC 5321 now recommends that EHLO (extended HELLO) should be used in place of the original HELLO command, but servers are still required to accept the old HELO command, as used in this example. Where the extended HELLO command is used, the server may offer a list of optional protocol extensions such as encryption (TLS) and pipelining.

This example shows a message with a single recipient, but the RCPT TO command may be repeated for messages with multiple recipients.

Similarly more than one message can be submitted per session. After a message has been accepted by the server, the sender can issue another MAIL FROM command, instead of a QUIT command.

Note that the addresses in the To and From headers can be completely different from those given in the MAIL FROM and RCPT TO commands for a number of reasons, including mailing lists, aliases, and bcc addresses.

[Sidebar ends here]


Access control lists allow you to define rules for each stage of the SMTP protocol [see the sidebar] and some other events for non-SMTP messages. The configuration contains named ACLs and there are 18 different options to connect a named ACL to specific events, e.g. the option acl_smtp_mail determines which ACL is run to decide whether to accept or reject a RCPT FROM command.

ACLs will typically return accept or deny, but they may also return defer or discard. Accept and discard cause a 2xx (success) response to be returned, but discard will then discard the message or recipient. Defer causes a 4xx (temporary rejection) response and deny causes a 5xx (permanent rejection) response.

An ACL will contain a list of rules, each rule consists of a verb and a set of conditions. If the conditions match the rule is triggered. The rules are processed in order until a rule returns an accept, deny, or defer, or there are no further rules. If there are no further rules, then there is an implicit deny.

The SMTP CONNECT ACL is specified by the acl_smtp_connect option. It is used to decide whether or not to accept a connection. Normally this would not be used, as most mail administrators don't know everybody who might want to send them emails, so they would accept connections from anywhere. A situation where you might want to use this would be an internal mail server that does not connect directly to the Internet, which would accept mail from clients on its subnet and from one or two external mail servers. To implement this you might set

acl_smtp_connect = check_connection

in the options and

check connection:
    accept hosts = 192.168.53.0/24: 10.10.10.10 : 10.11.12.13 

    deny

in the ACL section.

This ACL starts with an accept verb and a condition that the connecting host is on subnet 192.168.53.0/24, or it is one of 10.10.10.10 or 10.11.12.13. The only other rule is an unconditional deny, so if the connecting host does not match the given addresses the connection will be denied.

The SMTP MAIL ACL is specified by the acl_smtp_mail option. It is used to decide whether or not to accept a MAIL FROM command from a specified email address. This would not be an ACL I would expect to use in normal operation, but I did have a problem where a particular user had sent an email which was then being resent around a thousand times per hour. We didn't want to block any other users sending from that host, so to block just that user we could set

acl_smtp_mail = check_sender

in the options and

check_sender:
    deny message = User has been blocked
         sender  = user@example.com

    accept

The first rule obviously denies all requests to send mail from user@example.com, with the message "User has been blocked" and the second rule then accepts all other senders.

In practice, maintaining a list of blocked users in the configuration file is unwieldy. It is almost always better to maintain the list in an external file. This can be done by changing the sender line to

         sender  = lsearch;/etc/mail/sender_blacklist

As well as file based lookups Exim can query a number of popular databases and other data sources, such as LDAP. These are listed in chapter 9 of the manual.

The SMTP RCPT ACL is specified by the acl_smtp_rcpt option. It is used to decide whether or not to accept a RCPT TO command.

The following example comes from the default configuration, where it is fully commented. I have omitted some rules that restrict addresses containing certain characters.

Before defining the ACL we define some domain lists that are used in the ACL.

domainlist local_domains = exim.example.com
domainlist relay_to_domains = 
hostlist   relay_from_hosts = 127.0.0.1

acl_smtp_rcpt = acl_check_rcpt

and the ACL is

  accept  hosts = :

  accept  local_parts   = postmaster
          domains       = +local_domains

  require verify        = sender

  accept  hosts         = +relay_from_hosts
          control       = submission

  accept  authenticated = *
          control       = submission

  require message = relay not permitted
          domains = +local_domains : +relay_to_domains

  require verify = recipient

  accept

The first rule accepts any email with an empty sending host field. This supports mail from local user agents that submit messages via pipes (standard input and output), rather than via a TCP/IP connection.

The second rule accepts messages to postmaster for any domain in the domain list local_domains. This is a requirement of RFC 5321.

The third rule requires that the sender address is verifiable. Sender addresses are verified by running the routers for the address. If the address is not routable verification fails. For advanced users it is possible to set routing options specifically for verification.

The fourth rule accepts messages from any hosts in domain list relay_from_hosts. This rule is used to allow Exim to relay mail from other hosts, e.g. Exim may be used as a gateway to relay email from internal mail servers that are not exposed directly to the Internet. In this instance relay_from_hosts has been left empty, so Exim will not relay mail for any other hosts.

The fifth rule accepts mail from anywhere, provided that the user is authenticated. This would typically be used to support email from laptops that are connected via the the Internet, rather than via the local network.

If we do not configure Exim to accept mail authentication, this will not accept any mail. This article will not cover authentication in Exim.

The sixth rule rejects any recipients that are not for either one of our local domains or a domain for which we are configured to relay. The message option on the rule sets the message that will be returned for any recipient that is rejected by this rule.

The seventh rule rejects any recipients we can not verify.

The final rule accepts any mail that has passed through all the preceding checks without being accepted or rejected.

Note that where a message has multiple recipients each recipient can be accepted or rejected separately. Only one recipient has to be accepted for the message to be accepted. Obviously the message will only be delivered to those recipients that were accepted.

The DATA ACL can be used to reject messages based on their contents, e.g. to reject messages containing "bad words" or "naughty words", set

acl_smtp_data = acl_check data

in the options and

  deny message = Bad words in message
       regex   = (?i)\\w(bad|naughty)\\wwords\\w

  accept

The first rule uses a a regular expression to detect the phrase "bad words" or the phrase "naughty words" and rejects the message with the message "Bad words in message".

The second rule accepts all messages that have not been denied by the first rule.

Routers

Once a message has been accepted, Exim will store the message in its spool directory for further processing. The next processing step is determined by the routers.

Routers process addresses. They can do 3 things with an address. The address can be assigned to a transport for delivery, it can be rejected, or the address can be converted into a new address (or multiple addresses).

The configuration will contain a number of routers and they are processed in the order in which they appear. The default configuration defines the following 4 routers.

dnslookup:
  driver = dnslookup
  domains = ! +local_domains
  transport = remote_smtp
  ignore_target_hosts = 0.0.0.0 : 127.0.0.0/8
  no_more

system_aliases:
  driver = redirect
  allow_fail
  allow_defer
  data = ${lookup{$local_part}lsearch{/etc/aliases}}
  file_transport = address_file
  pipe_transport = address_pipe

userforward:
  driver = redirect
  check_local_user
  file = $home/.forward
  no_verify
  no_expn
  check_ancestor
  file_transport = address_file
  pipe_transport = address_pipe
  reply_transport = address_reply

localuser:
  driver = accept
  check_local_user
  transport = local_delivery
  cannot_route_message = Unknown user

These routers are fully commented in the default configuration.

All routers start with the driver option. This specifies which Exim code module is used to route the address.

The first router, the dnslookup router, looks for remote deliveries. It matches any addresses whose domains are not listed in the local_domains domain list and assigns them to the remote_smtp mail transport. The ignore target hosts option ensures that the address 0.0.0.0 and the subnet 127.0.0.0/8 are treated as DNS lookup failures and the no_more option ensures that they are not then passed to the following routers, thus they are rejected as unroutable.

The second router, the system_aliases router, looks for aliases in /etc/aliases, and replaces the address with the address found in the aliases file. If the lookup does not find a match, the router will decline the address, allowing it to be processed by the next router.

The allow_fail and allow_defer options cause the router to fail or defer if the lookup returns :fail: or :defer:. The file_transport and pipe_transport options specify the transports to use if the lookup returns a filename or a pipe. For these purposes a filename is a string that starts with a slash but does not end with a slash and does not parse as a valid RFC2822 address, with a domain. A pipe is a string starting with a vertical bar character.

The third router, the userforward router, looks for .forward files in users' home directories and uses the contents of those files to replace the address. As you would expect, if a matching .forward file is not found the router will decline the address, allowing it to be processed by the next router.

The check_local_user option is used to ensure that this router declines the address if the local part does not match a valid user on the system, e.g. an alias. The no_verify option ensures that this router declines when run for address verification, not delivery and no_expn causes it to decline if Exim is processing an EXPN command. The check_ancestor option means that the forward file can not redirect to a previously redirected address, i.e. it will break a redirection loop.

Finally the three transport options specify the transports to use for file names, pipes, and auto-replies.

The last router attempts to deliver a message to a local user.

Transports

Transports transmit messages to a destination. Local transports transmit messages to a file or a pipe on the local host. Remote transports transmit messages to another host.

The default configuration defines 5 transports.

remote_smtp:
  driver = smtp

local_delivery:
  driver = appendfile
  file = /var/mail/$local_part
  delivery_date_add
  envelope_to_add
  return_path_add

address_pipe:
  driver = pipe
  return_output

address_file:
  driver = appendfile
  delivery_date_add
  envelope_to_add
  return_path_add

address_reply:
  driver = autoreply

As with routers, all transports start with the driver options, which determines which Exim code module will handle the delivery.

The remote_smtp transport uses the smtp driver to deliver to remote hosts using the SMTP protocol.

The local_delivery transport uses the appendfile driver to deliver to local mailboxes in BSD mailbox format. The delivery_date_add, envelope_to_add and return_path_add options add Delivery-Date, Envelope-To and Return-Path headers. Note that RFC5321 requires the Return-Path header for final delivery, but Delivery-Date and Envelope-To are non-standard.

Exim can also support mail deliveries in MBX format, Maildir format, and mailstore format, when built with the appropriate configuration options.

The address_pipe transport handles any deliveries to pipes, when pipes are specified in aliases or .forward files. The return output option treats any text on the pipe's standard output as a delivery failure and returns the text to the message sender as an error. An alternative is return_fail_output, which only returns the output as an error if the pipe returns a non-zero status code on completion.

The address_file transport is used to deliver to files, when file names are specified in aliases or .forward files. It is very similar to the local_delivery transport, but uses the file name from the alias or .forward file, instead of the user's normal mail spool. files

Finally, the address_reply transport uses the autoreply driver, which generates automated replies when required by the userforward router. It is not a proper transport because it does not actually deliver the original message. To generate an automated reply and still deliver the original message routers calling an autoreply transport should use the unseen option, so the message will also be processed by later routers.

Debugging

This article has tried to make it clear that basic Exim configuration is fairly simple; however, any new configuration is likely to contain some errors and will require testing. Exim provides a whole host of options to help you to test and debug your configuration.

The most basic test of any new configuration is a check for syntax errors. You can run Exim with the options -bv -C /path/to/file to check the basic syntax.

A fake SMTP session can be run with exim -bh ip_address. This simulates an incoming message from the given IP address, using standard input and output, standard and diagnostic information is written to standard error. It does not deliver the message or write to the real log files.

Addresses can be tested for deliverability using exim -bt. This can combined with -t sender_address if the routers use tests on the sender address.

An extremely useful option is -d. This will print diagnostic information to standard error. The information to be printed can be controlled by appending +option and -option, e.g. exim -bd -d-all+acl would provide debugging information for ACLs.

There are other options that may be used to debug particular aspects of the configuration. These are all listed in the manual.

Conclusion

I hope this article has made clear that getting started with Exim is not terribly difficult, and also made clear that more experienced Exim administrators will find a wealth of features that can be used to achieve a wide range of goals from spam filtering to building applications based on automated email processing.

Although getting Exim running is easy, that is only the start. Every mail administrator needs to be aware of their responsibilities to the wider Internet community. A badly set up mail server may find that it is blacklisted for a variety of reasons. The worst offenses are relaying and backscatter, but many sites will also blacklist servers that are on dialup connections, have poor DNS configurations or are non-compliant with various RFCs.

Most mail administrators will also need to have some anti-spam measures in place such as greylisting and scanning with tools like SpamAssassin.

The author would like to thank Dr. Philip Hazel and the many contributors to Exim's code and documentation. Thanks are also due to the many subscribers to the exim-users mailing list who have assisted the author and others to get the best from Exim.


Talkback: Discuss this article with The Answer Gang


Bio picture

Neil is a programmer, specialising in C++ on Unix and Linux. He has degrees in Computer science and Next Generation Computing.

Neil has worked on a wide range of systems from the control system for the British Gas national grid to video servers for the Home Choice video on demand service. He first programmed computers in 1980 with his school General Studies class, which was allowed access to a mainframe at The National Institute of Oceanography, programmed in Fortran on punch cards.

A computer science degree followed at Queen Mary College, London, then Neil worked for Logica for 3 years before taking an MSc in New Generation Computing at Exeter University.

The next 5 years saw Neil researching parallel simulation algorithms at the Royal Signals and Radar Establishment, initially on transputers and subsequently on SPARC based parallel systems. Since leaving RSRE, Neil has mostly worked freelance and has worked on financial data feeds, video servers and virus scanning proxies.

Neil first used Unix at college in 1982 and started working on Linux in 1996.

As of May 2004, Neil is working for Wirefast a global messaging company.

Outside of computing, Neil is into motor sport, particularly Formula 1, the World Rally Championship and the British Touring Car Championship. He doesn't race himself. If you've seen Neil's driving, you'll understand why.

Copyright © 2009, Neil Youngman. Released under the Open Publication License unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 158 of Linux Gazette, January 2009

Tux