Previously I'd written about SPF and DKIM which cover different aspects of determining the legitimacy of mails. What has since happened is that adoption and real-world experience of these has been bumpy to say the least. To a large extent they've just ended up being used as another heuristic in spam filters and I routinely see major mail providers quarantining legitimate mail with good SPF and DKIM so I'm not even sure they count for much.

Both SPF and DKIM have facilities to reject mail (eg. in SPF a record ending "-all" means to reject the mail which should hopefully see the upstream server bouncing it back to the sender), and that would hopefully alert people to the problem. In reality, because of the use as a heuristic instead, more often it quietly vanishes into a Junk folder and is never seen again. That means that there's an enormous amount of broken SPF and DKIM out there, and with no feedback mechanism, it never gets corrected. Add to that the number of problem mail servers still in operation (eg. that don't properly retry 4xx responses) and lack of monitoring, and the value is undermined.

Even where servers do obey the rules strictly, the sending end often creates a "friendly" bounce message which hides any kind of useful information about what went wrong and how to correct it, once again defeating the feedback mechanism.

Essentially, the whole chain is riddled with things that undermine the value, and there's little to be gained by fixing any of them when the others (which are often the responsibility of someone else) remain unfixed.

DMARC (Domain-based Message Authentication, Reporting and Conformance) aims to address a lot of these problems in the adoption of SPF and DKIM and theoretically adds a feedback/reporting mechanism. I have a hunch that once again similar things could happen on contact with the real world: the major players that defeated feedback in SPF and DKIM are still involved, rumour is that the reporting for DMARC is not being implemented widely, the same flaky mail servers remain in operation, and there's no reason that all the broken config we've seen with SPF and DKIM will be any different when it's still needed in addition to even more config to implement DMARC. This is more a problem of human nature and behavioural psychology rather than a technical one and I think there are very real limits to how much things can be improved by adding more technical layers.

None the less, it would be an exceptionally good thing for everyone if a mechanism for reliably establishing the authenticity of mail was widely adopted, and worth the tiny bit of effort to for everyone to contribute to it getting traction.... even if I am likely way over optimistic given the track record.

How it works

DMARC doesn't replace the authentication mechanisms provided by SPF and DKIM. You still need to implement those, but you may want to do things slightly differently with DMARC which is something we'll get to.

What DMARC changes is how we go about enforcing SPF and DKIM. Instead of SPF and DKIM providing rules (which seem to mostly get ignored by major providers) for accepting mail, the decision is passed on to DMARC to accept, quarantine or reject the mail, then to report back (assuming everyone implements that).

That means that we probably want to remove or back off on enforcement rules for SPF (major providers seem to be going for "~all") and DKIM (major provider seem to have validation keys only and no ADSP and no kind of other enforcement).

It also means that we might want to configure SPF and DKIM not to enforce themselves due to flaky configuration that seems to be so prevalent. None of these are definite, but might help with unnecessary loss of mail due to the prevalence of bad configuration and worth giving some thought.

Implementation Details

This is based on Debian Jessie (8.x) which contains all the packages needed. The principles should be the same for other distros, but there have been changes in some configuration of different packages so some extra care may be needed.

SPF

This is easy for outbound mail. You need a TXT record in your DNS for the domain the mail is coming from which lays down some rules. This is widely documented and is largely a list of legitimate sources for mail, and finally a catch-all rule to determine what happens if the rest fail.

The catch all rule is likely best "~all" (return 4xx temporary failure) rather than "-all" (return 5xx permanent failure) which will allow you to monitor your mail queues for problems (you should do that!) and correct them without mail being bounced. It should end up with a record (bind format) looking something like this:

@ IN TXT "v=spf1 a:outboundmail1.example.com a:outboundmail2.example.com include:spf.mailinglist.provider.tld ~all"

There's a lot of other options so do take a look at the docs for SPF.

On the inbound side, we will need to implement SPF checking against the senders DNS records. While opendmarc can theoretically handle this, there have been reports of problems and my own tests also show some reliability risks with this with the version shipping in Debian Jessie, so for safety until the problems get worked out, we can use postfix-policyd-spf-python to add authentication headers to the mail before opendmarc processes it to take over the SPF authentication. The Perl based policy daemon, postfix-policyd-spf-perl is something that I've described previously, but opendmarc is only tested against the Python one.

There are plenty of resources around describing what is needed, but after installing postfix-policyd-spf-python the following lines can be added to /etc/postfix-policyd-spf-python/policyd-spf.conf to prepare it for use with opendmarc:

# format headers to be compatible with opendmarc
Header_Type = AR
# specify the server name for the headers
Authserv_Id = mailserver.example.com
# set test mode so we don't actually enforce SPF and leave things up to opendmarc (optional)
# note: this option changes in later versions
defaultSeedOnly = 0

We need the Postfix master process to manage running of the policy daemon so ensure that /etc/postfix/master.cf contains an entry like:

policyd-spf  unix  -       n       n       -       0       spawn
    user=policyd-spf argv=/usr/bin/policyd-spf

Then to wire this in to Postfix we need to add it in /etc/postfix/main.cf to smtpd_recipient_restrictions and/or smtpd_relay_restrictions as appropriate:

smtpd_recipient_restrictions =
        ..... various rules
        check_policy_service unix:private/policy-spf
        ..... various other rules

smtpd_relay_restrictions =
        ..... various rules
        check_policy_service unix:private/policy-spf
        ..... various other rules

The exact location of of the check will depend on your existing configuration. I put it after the basic checks for header items like legitimate domains, destinations, etc.

Then reload postfix to ensure the new config is picked up:

# systemctl reload postfix

After this your mails should contain a header like:

Authentication-Results: mailserver.example.com; spf=pass (sender SPF authorized) smtp.mailfrom=....

This is what opendmarc will be looking for in the mails rather than doing it's own SPF checks.

DKIM

This can be implemented with opendkim as I described before. The only thing that you probably want is to avoid any enforcement in your DNS records (DMARC will do that) so don't add records like:

_domainkey IN TXT "t=y;o=~"
_adsp._domainkey IN TXT "dkim=all"

Only add the keys needed to authenticate your outbound mail like:

mailservername._domainkey IN TXT "v=DKIM1; k=rsa; p=MIGf....

You should see headers in your inbound mail like:

Authentication-Results: mailserver.example.com; dkim=pass

And when you send mail to others they should also have similar headers added by their mail provider indicating successful authentication of your mail.

If you don't want to enforce DKIM (and leave it purely to DMARC) on inbound mail then you might want to consider adding to /etc/opendkim.conf the line:

On-BadSignature        accept

And there are others like "On-InternalError" that you could do similar with.

DMARC Outbound

Since the only additions to the outbound mail will be the DKIM signature, the only configuration needed with be the DNS record for others to enforce your authentication. This will need some addresses for sending reports (feedback), so ensure you have those in place first. Initially on a small scale you may want to make these a real mailbox or alias, but in the longer term or on large scale process them automatically.

Your DMARC record will likely start off something like:

_dmarc IN TXT "v=DMARC1; p=none; rua=mailto:dmarcreport@example.com"

Where "rua" is the address for aggregate reports. You can also specify "ruf" for forensic reports. The "p" parameter is for policy and this will likely start of as "none" and advance to "reject" once things are known to be operating reliably. Remember to make sure there are mailboxes to receive the reports. There are also scripts around for automatically processing the reports which will be useful for producing metrics on larger scale deployments.

It seems not many providers send reports, but I have found that mail sent to Yahoo! sees periodic aggregate report mails returned confirming the status of the mails. This provides useful confirmation that all your configuration is working as expected.

More detail is available in the DMARC documentation (see "Anatomy of a DMARC resource record in the DNS").

DMARC Inbound

This is where we need to configure opendmarc to check our inbound mail. This will use the headers we're inserting with validating SPF and DKIM, and then apply the relevant policy.

First up we need to get opendmarc installed:

# apt-get install opendmarc

Then we need to tell it to listen on a socket by editing /etc/default/opendmarc and setting:

SOCKET="inet:8892"

Listening on 8892 is fairly arbitrary, but there seems to be a trend to putting milters on 8891, 8892 .... etc. By listening on a TCP socket rather than a Unix socket, we avoid a whole lot of complexities around permissions, chroot etc.

You might also want to tweak /etc/opendmarc.conf and add some or all of these:

# avoid checking endpoints ... but this seems to be buggy
IgnoreAuthenticatedClients true
# send failure reports out to others - consider this carefully before use
#FailureReports true

There's a whole bunch of extra config (see man page) that you can add, but this should be sufficient from the default Debian Jessie shipped config.

You should be able to restart opendmarc and it's mostly ready to go:

# systemctl restart opendmarc

Now we should have opendmarc configured and waiting for connections. In Postfix we need to configure it to use the milter. Your exact configuration may vary, but assuming you have some existing milters (opendkim for one), you would add to /etc/postfix/main.cf something along the lines of this, adding it after the other milters that are needed to do DKIM:

smtpd_milters =
    unix:some-other-milter/milter.sock
    inet:localhost:8891
    inet:localhost:8892
non_smtpd_milters =
    inet:localhost:8891
    inet:localhost:8892
milter_default_action = accept

Now inbound mail should get a header added something like:

Authentication-Results: mail.yourdomain.com; dmarc=pass header.from=dmarc.example.com

Out the box opendmarc will not try and do anything with the mail (eg. reject it) if things fail to authenticate, but only adds this header. You could set your own mail filtering or further processing to quarantine or handle it as you prefer.

Bugs

The version of opendmarc shipped with Debian Jessie seems a bit buggy and it attempts checks on mail from authenticated clients sometimes and adds headers to outbound mail reporting failures.

Fortunately Debian Stretch is nearing release so we might have a newer, hopefully less buggy version available with that.