Glen Pitt-Pladdy :: Blog

Basic Postfix config guide for Cacti, Spam Blocking, TLS etc.

I often find myself helping sort out some Postfix config problems and it occured to me that much of what I do regularly with Postfix is probably not obvious to the average sysadmin, plus the config that works with my Postfix Cacti Templates may need a little explanation.

This article gives the basic nuts and bolts of a Postfix config beyond shuffling mail around which most installs will do out the box anyway. It is primarily aimed at systems based on Debian (eg. Ubuntu and relatives), but will probably also be relevant to most other platforms. If configuration is applied wrongly for your situation then you may end up with lost mail or worse, so ensure you understand what you are doing and test your confguration carefully.

This could be the basis of a standalone mailserver for a small business or home, as well as Edge Relays for a corporate network, or any other major mail system.

The main things I will look at here:

  • General DNS stuff for mail
  • TLS (encryption) for smtpd (inbound mail)
  • TLS (encryption) for smtp (outbound mail)
  • Virus scanning with ClamAV milter
  • Basic sender, recipient and header filtering
  • Greylisting with Postgrey (SPAM blocking)
  • DNS RBLs (SPAM blocking)
  • Rule ordering

DNS General comments

I regularly come across mail admins that don't fully understand the relationship of DNS and mail, and unsurprisingly many mail systems have rather strange DNS setup.

To a large extent, much of the DNS side often ends up as a case of what people can get away with since it depends on how strict the checks on the other side are. Often bad DNS config is just going to mean that other systems score mail from your system as more likely to be SPAM, or it can't be as well secured.

The basic "best practice" rule for DNS is that everything is consistent:

  • You can call your machine whatever you like, it doesn't have to be mail.company.com or similar. The choice of host name for the system doesn't affect the reliability of mail delivery despite what some believe.
  • Often, having a separate name for the service (eg. mail1.company.com, mail2.company.com etc.) can be useful in that it allows the service (and it's certificates) to be moved between servers independent of the server names, and in larger networks, to be easily shared across many servers simply with multiple A records. This is not a necessity, but may be useful for long term flexibility and scalability.
  • It's best that the MX records for a domain point to a A (and AAAA) records (rather than CNAMEs)
  • Make sure that the machines are configured to operate under the name that the MX record and acompanying A record points to. ie. if it connects to another server and says "HELO hunkofjunk.mycompany.local" then the other side may decide that it's identifying it's self with a fake name (no A record out on the internet) and may get grumpy about talking to it.
  • It's nice if your PTR record (reverse lookup - turns IP addresses into names) for the machine also resolves to the same name, but this is not always possible since this is often at the mercy of your ISP so hopefully not too many servers get paranoid over PTR records
  • Ensure that FQDNs are always used. Where a server says "HELO hunkofjunk", many servers will reject this requiring a FQDN, and it won't look up so may be rejected as invalid

Postfix will normally be quite tidy on it's own. If you are have a different MX record (with matching A record) to what the actual machine host name is then you can tell Postfix to use a different hostname.

eg. If the machine's hostname is carrot.mycompany.com, and the MX  record points to mail.mycompany.com (which obviously should also have an A / AAAA record that resolves to the IP of the server) then you should add / change this in your /etc/postfix/main.cf:

myhostname = mail.mycompany.com

Then if you telnet to the smtp port you should see something like this:

$ telnet mail.mycompany.com smtp
Trying XXXX:XXXX:XXXX:XXXX:XXXX::XXXX...
Connected to mail.mycompany.com.
Escape character is '^]'.
220 mail.mycompany.com ESMTP Postfix (Debian/GNU)
quit
221 2.0.0 Bye
Connection closed by foreign host.

At least now the DNS matches what the machine is identifying it's self as and the name used for HELO/EHLO should also match the DNS making mail less likely to be scored as SPAM.

This is good stuff for any mail system, not just postfix, but due to restrictions which some vendors impose on their systems this is not always practical.

TLS General comments

I currently look after the techy side of a leading financial software company and am a big believer in doing things properly from the start. Bodging stuff until it looks like it is sort-of-working is the industry norm, and really annoys me.

TLS (Transport Layer Security) is widely available on all decent (and even on many not so decent) SMTP mail systems these days. Despite this, the mail system monitoring that I have in place shows that poor configuration of TLS is widespread. Even some of the worlds biggest and most respected internet brands have misconfigured TLS which flags up in the monitoring.

This covers the basics of getting it right with Postfix, primarily on a Debian based system (ie. includes Ubuntu and relatives).

The first thing you need is a certificate for your server. Although self signed certificates are better than nothing, these days it's easy to get a legitimate certificate from a recognised CA for the cost of a round down the pub.

To create a key for the server and CSR (Certificate Signing Request) to upload to the CA you can do something like:

$ openssl req -nodes -newkey rsa:2048 -keyout host.key -out host.csr
.......

Make sure the "Common Name" that it requests for the certificate is the name it will be serving as to the outside world - ie. the name the MX record points to.

Once you have your certificate from the CA, also grab the CA's certificate, and put them in /etc/ssl/certs, and put the key you created in /etc/ssl/private. Ensure that the key is only readable by root. This is important since if the key gets out then all bets are off.

TLS for smtpd (inbound)

The basics are simple - this is what goes in /etc/postfix/main.cf:

smtpd_tls_security_level = may
smtpd_tls_CAfile = /etc/ssl/certs/Your_CA_Certificate
smtpd_tls_cert_file = /etc/ssl/certs/Your_Certificate
smtpd_tls_key_file = /etc/ssl/private/Your_Key

Now, that turns on TLS and gives it some certificates which is the basics covered. It doesn't enforce the use of TLS, but if you are accepting mail from the internet (or even local devices on your network that don't support TLS), then enforcing means you are going go loose out on a lot of your mail.

At this stage, you may be encrypting messages, but how do you know if the sender is who they say, or there is a man in the middle and you are receiving the mail second hand? This is where verifying the certificates comes in. If you check that the sender's certificate has been signed by some authority you trust (a reputable CA), then you can have more confidence in it being legitimate. There are many badly setup systems and people with self-signed certificates so not a good idea to reject mail on this basis unless you are within a network where all servers have legitimate certificates.

To do this you will need a bunch of trusted certificates. On Debian based systems, the package ca-certificates provides this and you can then configure which ones you trust and it will do the rest for you. The trusted certificates are symlinked in /etc/ssl/certs, but keep in mind that Postfix runs chroot /var/spool/postfix so can't see this directory (individual files are read before chroot when Postfix starts, but your directory of trusted certificates is read as needed after chroot), so you need to copy the contents of this directory (or even setup a periodic cron job to rsync it) from /etc/ssl/certs to /var/spool/postfix/etc/ssl/certs.

Then add this to your /etc/postfix/main.cf:

smtpd_tls_CApath = /etc/ssl/certs
smtpd_tls_ask_ccert = yes

Although not entirely necessary, smtpd_tls_ask_ccert will request the certificate from the other side (client) which will allow smtpd to validate it. Without this you will always end up with anonymous TLS. The only likely side-effect you may see is that some clients (gerally end user Mail clients rather than other MTAs) is that they may complain about this. If you are not relaying for end users then this is not likely to be a problem, and every modern mail client I have used including the tiny one on my phone works fine with this so I suspect it is only really old mail clients which have problems.

And for the sake of providing some details for Cacti to chew on:

smtpd_tls_loglevel = 1

That should give you the basics to get your Postfix accepting mail with TLS.

TLS for smtp (outbound)

Although many servers seem to be configured to accept TLS on inbound connections, outbound connections often seem neglected. It does rather defeat the point if everyone can accept TLS mail, but nobody bothers sending with TLS. The common flaw is to configure inbound mail (smtpd) with a certificate and leave outbound mail (smtp) to do anonymous connections.

The basics are much the same as for inbound mail, in fact you could just set the smtp configuration to use the same settings as you already configured for smtpd:

smtp_tls_security_level = may
smtp_tls_CAfile = $smtpd_tls_CAfile
smtp_tls_cert_file = $smtpd_tls_cert_file
smtp_tls_key_file = $smtpd_tls_key_file
smtp_tls_CApath = $smtpd_tls_CApath

Again, some more logging is useful, and gives Cacti something to analyse:

smtp_tls_note_starttls_offer = yes
smtp_tls_loglevel = 1

Common causes of TLS problems

I see these all the time with other systems:

  • The certificate does not match the name from the MX record which you are making the connection to
  • The server identifies it's self with a different name to the certificate and/or MX record
  • The CA certificate is not available so can't be verified
  • Outbound SMTP has not been configured to use TLS, or if it has, only to use Anonymous TLS rather than using a valid certificate
  • People have not tested their TLS configuration so have not found this stuff out and have just left their systems in a state of "it seems to work"

Virus / Malware scanning

Malware is all to common these days along with Phishing and other bad stuff coming in through mail. While virus scanners are not nearly as effective as everyone (including reviewers) think, they do provide a some degree of protection for vulnerable users you are serving to, and may also help contain an outbreak on your network. Even if you are a vigilant user on a secure platform, virus scanning can help remove the annoying deluge of Malware on other peope's systems spewing their mail at you.

Assuming you have clamav-milter and clamav-daemon (plus of course clamav-freshclam) installed, there are a few thing that need to be done to make it work with Postfix.

The first thing is that the milter socket needs to be accessible by Postfix. By default it runs with with ownership and group of clamav. To change this, edit /etc/defalt/clamav-milter and change (or add) the line:

SOCKET_RWGROUP=postfix

Next, edit /etc/postfix/main.cf and change / add:

smtpd_milters = unix:/var/run/clamav/clamav-milter.ctl
milter_default_action = accept

This tells Postfix's smtpd to filter the mail, and where to find the socket, and if all else fails (eg. the milter is not running) to accept the mail.

To get a more detailed record of what is going on (and allow Cacti to gather info) you may also want to do some logging in /etc/clamav/clamav-milter:

LogSyslog true
LogFacility LOG_MAIL
LogInfected Full

These will log more detail, and log them to the mail logs where my Cacti scripts (in the next update) can pick them up.

When bad messages are found, they will be quarantined in the HOLD queue /var/spool/postfix/hold/. These can be released again by following this sequence (or you could even make a script to do this if it is a regular occurrence):

~# cd /var/spool/postfix
/var/spool/postfix# mv hold/3C45430011 incoming/
/var/spool/postfix# echo -n I >public/qmgr

Note that the file in the queue should be executable so that Postfix will pick it up for delivery. If you change this then it will not work.

Also, you may want to prune rejected mails after say 30 days. To do this you can put a simple script in /etc/cron.daily/ with something like:

#!/bin/sh
find /var/spool/postfix/hold -mtime +30 -printf 'Pruning Postfix HOLD: %fn' -exec rm {} ;

Implementing restrictions

With Postfix it is easy to apply comprehensive restrictions to accepting mail. This can be very useful for SPAM blocking as well as many other things (eg. giving a custom bounce notice for an old address). There are a number of points in the process where the checks can be applied, and personally I prefer to do this late in the process so I capture as much information about the rejected mail as possible. This is useful both for diagnosing problems with blocked mail as well as giving more information to adapt strategies in the long run.

Almost all my filtering happens with smtpd_recipient_restrictions which happens after "RCPT TO" in the SMTP conversation and gives us all the server, sender and recipient information that is practical without allowing message delivery to start.

Almost all the following sections add rules into smtpd_recipient_restrictions unless otherwise noted. The ordering of the rules is important so please see below on this.

Basic Filtering

Some times basic sender or recipient filtering (eg. with an appropriate bounce message) is very useful, and also very simple to implement. There are two main approaches to filtering senders and recipients. The choice is between flexibility and performance.

Regular expressions allows comprehensive patterns to be matched, but in turn has far higher overheads. This is probably not a problem unless you are handling large volumes of mail and/or have a large number of checks to process. The config for these in /etc/postfix/main.cf is:

smtpd_recipient_restrictions =
    # ....... various other rules ......

    check_sender_access hash:/etc/postfix/fromaccess,
    check_sender_access pcre:/etc/postfix/fromaccess_pcre,
    check_recipient_access hash:/etc/postfix/toaccess,
    check_recipient_access pcre:/etc/postfix/toaccess_pcre,
    # ....... various other rules ......

The pcre: stuff refers to  Perl Compatible Regular Expressions and requires the postfix-pcre package. There is also a generic regex: for Postfix but I like the Perl extensions. These filters allow all sorts of patterns to be matched. The basic syntax of the *access_pcre files is:

# Comments start with a hash, blank lines are ignored
# this rejects with no given reason (Postfix puts a generic one in)
/regularexpression1/ REJECT
# this accepts a mail which means that later rules are not processed
/regularexpression2/ OK
# this rejects the mail with a message which hopefully bounces
# back to the sender depending on their server
/regularexpression3/ REJECT some message sent back to the sending server

For more information on this see the Postfix man pages for pcre_table and access. How to write these is best left up to the Perl Regular Expressions docs.

The hash: stuff is a straight pattern match with similar basic syntax:

# Comments start with a hash, blank lines are ignored
# this rejects with no given reason (Postfix puts a generic one in)
addres1@to.match REJECT
# this accepts a mail which means that later rules are not processed
addres2@to.match OK
# this rejects the mail with a message which hopefully bounces
# back to the sender depending on their server
addres3@to.match REJECT some message sent back to the sending server

These are rather simpler and only provide a straight address match, but in turn are far more efficient. All hash: tables require a .db file to be created from the plain text file using postmap once you have edited them:

# postmap /etc/postfix/toaccess

Finally, we can also do regheader checks which are useful if you want to block certain patterns in the header (eg. the subject or know faking of the header). These go in /etc/postfix/main.cf with:

header_checks = pcre:/etc/postfix/header_checks

Once again this is a pcre: so the same rules apply, but this time lines of the message header are checked which means that the sending server has already started sending the message. It is generally cleaner (won't waste bandwidth etc.) if you can reject bad mail earlier, but header_checks does allow an additional level of checking that is none the less useful.

Greylisting

This is simply a trick to verify compliance with current SMTP standards which require temporary or retryable failures (4xx codes - normally 450, 451 or similar) to be retried. The main side-effect of this is that mails get delayed because we fake a failure the first time a server tries to deliver mail. After time we can begin to trust servers we see regularly and allow them to deliver mail without delay.

Greylisting is widely used by major ISPs and mail providers and on it's own can result in reductions of SPAM of as much as a factor of 10 to 20. As it is so widely used, the small number servers which do not comply with standards or are very badly configured and fail to attempt re-delivery within a reasonable time will be loosing lots of mail to many destinations. I don't believe that it is my job to compensate for others who deploy badly setup or buggy systems, especially when the problems with their systems are going to be far more widespread than just sending mail to my servers. Their systems are broken and it is their responsibilty to fix them, but your policy may differ so use Greylisting as you feel appropriate.

For more detailed information on how this works see Wikipedia on Greylisting.

There are a number of Greylisting implementations, but postgrey is an extremely easy one to use - simply install it and add the restriction in your /etc/postfix/main.cf of:

smtpd_recipient_restrictions =
    # ....... various other rules ......
    check_policy_service inet:127.0.0.1:60000,
    # ....... various other rules ......

If your postgrey runs on a different port then you need to adjust this accordingly. You may also like to adjust the message and Greylisting time (delay) in /etc/default/postgrey:

POSTGREY_TEXT="Greylisted for 42 seconds (see http://your.own.url/about/greylisting/)"
POSTGREY_OPTS="--inet=127.0.0.1:60000 --delay=42"

.... and then restart postgrey to bring those into effect.

DNS RBLs

There are a number of abuse blacklists (RBLs) available via DNS lookups. Many of these are free for non-commercial use, and provide a collective, often community maintained means of deciding who we trust.

The choice of what RBLs to use is up to you as different ones will suit different situations. There is always the risk of someone you do want to accept mail from being blacklisted, faults at RBL providers and other factors to consider if you decide to use them. To use them in Postfix simply add the following restriction:

smtpd_recipient_restrictions =
    # ....... various other rules ......
    reject_rbl_client rbl.provider1,
    reject_rbl_client rbl.provider2,
    reject_rbl_client rbl.provider3,
    # ....... various other rules ......

You can use multiple providers, but in the end it can start getting a bit silly, and the more providers you use the higher the risk of false positives (legitimate mail being rejected).

General sanitisation

SPAM is often delivered with little regard to strict adhearance to SMTP standards which often provides a good way of removing large amounts of it, but like all SPAM blocking methods, may be prone to rejecting legitimate mail with badly configured (or just buggy) servers trying to send you mail.

Some general sanity checks that are often useful can be added to /etc/postfix/main.cf, but do make sure you understand what these do and the consequences of using them:

smtpd_recipient_restrictions =
    # ....... various other rules ......
    reject_invalid_hostname,
    reject_non_fqdn_sender,
    reject_non_fqdn_recipient,
    reject_unknown_sender_domain,
    reject_unknown_recipient_domain,
    permit_mynetworks,
    reject_non_fqdn_hostname,
    permit_sasl_authenticated,
    reject_unauth_destination,
    # ....... various other rules ......

The permit_mynetworks means any local machines you have in mynetworks will be accepted before continuing with later rules. As these may include devices such as UPS', routers, printers, and others which  are not full blown MTAs and have very limited SMTP capability, this rule needs to be before any rules which would reject mail from these devices.

The permit_sasl_authenticated rule is for those who use SASL authentication. This allows for example, remote users to authenticate themselves to the server to use it as a mail relay, and is a whole other subject. If you don't use SASL then leave this out.

You may also like to be strict about HELO:

smtpd_helo_required = yes

And make sure that mailers are only pipeling when allowed (spammers often don't adhere to this):

smtpd_data_restrictions =
    reject_unauth_pipelining,
    permit

It is often also useful not to give away information about your users unneccessarily:

disable_vrfy_command = yes

There are a number of useful articles on SPAM / UCE control on the Postfix site.

Rule ordering

The order in which you check restrictions can have significant implications, especially where there is a rule that may accept mail among the checks. The exact ordering may vary depending on your situation, but is worth thinking through and testing thouroughly.

My approach is:

  1. Do basic sanity checks (note, these may reject mail from badly setup kit, networks, or just simplistic SMTP clients):
    • reject_invalid_hostname
    • reject_non_fqdn_sender
    • reject_non_fqdn_recipient
    • reject_unknown_sender_domain
    • reject_unknown_recipient_domain
  2. Allow any local machines or kit that may not be perfect SMTP clients:
    • permit_mynetworks
  3. Some strict checks that are more likely to reject simplistic SMTP clients on printers, routers, etc.:
    • reject_non_fqdn_hostname
  4. Allow SASL authenticated users (optional):
    • permit_sasl_authenticated
  5. Restrict mail relaying:
    • reject_unauth_destination
  6. Sender / recipient filtering:
    • check_sender_access hash:/etc/postfix/fromaccess
    • check_sender_access pcre:/etc/postfix/fromaccess_pcre
    • check_recipient_access hash:/etc/postfix/toaccess
    • check_recipient_access pcre:/etc/postfix/toaccess_pcre
  7. Greylisting. No point in wasting time doing this with mail that we already know is dodgy:
    • check_policy_service inet:127.0.0.1:60000
  8. DNS RBLs. I don't like to hammer free services unneccessarily, so I make sure that reasonable effort is made to reject bad mail with other checks have all passed before checking blacklists:
    • reject_rbl_client rbl.blacklist.provider
  9. And finally accpet the message (default):
    • permit

Once you have set the order of your checks, test it thouroughly as bad ordering will result in  undesired mail getting accepted or worse yet, lost mail.

Other Refinements

With the widespread use of Greylisting, it is often desirable to have more agressive retry times when mail delivery fails:

queue_run_delay = 120s
minimal_backoff_time = 120s

Many users these days expect to be able to throw huge mails around. Depending on your network (ever seen what happens when the sales guy copies 30 people on a presentation on a 20MB presentation.... when connected via a single channel of ISDN or dial-up!), how silly you want to allow your users to be, and how much you want to waste bandwidth on destination servers that won't accept mail that big anyway:

message_size_limit = 20971520

Yes folks - that allows mail of 20MiB! Many other systems will not accept mail that big so expect some bounces when users start throwing around mails of that sort of size.

Monitoring things closely is generally a good idea. Using a package like logcheck to report problems. Graphs can also give invaluable information on trends and unexpected events, Malware outbreaks and attacks. See my Cacti templates for Postfix which are built around using the sort of config described here.

Comments:




Are you human? (reduces spam)
Note: Identity details will be stored in a cookie. Posts may not appear immediately