My Experience of Setting Up DKIM and SPF with Postfix and Ubuntu For Sending Email

I am developing a web application which needs to send email. The application is running on Ubuntu 12.04 64-bit and is using Postfix as the MTA, and therefore the web application is using Postfix to send outgoing emails.

To raise the chance of successful delivery of mail to the recipients, it is advised to setup DKIM and SPF for my environment.

This will be a long article to go, but I will make it easy for you to perform step-by-step.

My Environment

  • Computer running Ubuntu 12.04 LTS 64 bit
  • Postfix version 2.9.6
  • Web application is running on the same host.
  • My DNS server is not hosted on the host nor local network, but on GoDaddy.com.

Goal

  • Send a mail from my Ubuntu host to Gmail, and in Gmail web interface, check the mail’s raw message to ensure the dkim “Authentication-Results” header contains “dkim=pass“, and “spf=pass”.
  • This example use a sub-domain “test.example.com” as showcase which is a bit more complicated than just use example.com. As a result, when you are going to use root domain “example.com”, you will be able to do this with this post.

Verify Your Current Setup is Wrong

It is good to face some failure first, so you will know what’s correct.

I send an email from my Ubuntu host with sender email address no-reply@test.example.com to my email e.g. fake@gmail.com .

echo -e "Subject: DKIM Verification Email\nDKIM test mail content. Please read the message header.\nSent at: $(date)" | sendmail -f no-reply@test.example.com -t <your-email-address>@gmail.com

Inspect the log /var/log/mail.log, make sure postfix has delivered the mail to Gmail.

Then I open my email and view the raw message/message source, and do not find any traces of dkim wordings in the mail headers. (remember to look in your Spam folder if you can’t find the mail!)

First I install OpenDKIM related packages:

apt-get install opendkim opendkim-tools

Now I generate the keys using OpenDKIM’s tools. There are a few things for me to consider first:

  • I need to store the generated DKIM public and priavte key in some directories. Its seems that there is no standard directories defined by Ubuntu packages, so I create a directory /etc/opendkim/ for this purpose.
  • I need to know the domain I am working on. In this article, it is test.example.com. Change to suits your need.
  • After reading some DKIM documentation, I need to specify the name of something called “selector”. I found that the selector name need not to be made the same as anything (e.g. sub-domain name). So I choose “default” as my choice. Now run the following commands:
mkdir -p /etc/opendkim
opendkim-genkey -s default -d test.example.com -D /etc/opendkim
ls -l /etc/opendkim
-rw------- 1 root root 887 Oct 2 16:10 default.private
-rw------- 1 root root 310 Oct 2 16:10 default.txt

There will be 2 files generated: default.private and default.txt. The .private file is the private key, while the .txt file is the public key. Keep these files and make a backup!

There are some file permission things to set. The private key should be made accessible by the opendkim user only so I run the following command:

chown opendkim:opendkim /etc/opendkim/default.private

I need to tell OpenDKIM about the locatin of private key, name of selector and domain name in order to function properly, so I edit the file /etc/opendkim.conf and ensure the following lines are present

Domain test.example.com
KeyFile /etc/opendkim/default.private
Selector default

Now OpenDKIM knows where to find my private key as well as the selector name plus domain name to use. This will instruct OpenDKIM to sign outgoing emails with DKIM signatures.

Now we need to configure postfix to work with DKIM properly. Edit the file /etc/postfix/main.cf and make sure the following lines are present:

milter_default_action = accept
milter_protocol = 6
smtpd_milters = unix:/var/run/opendkim/opendkim.sock
non_smtpd_milters = unix:/var/run/opendkim/opendkim.sock

Note that the value for option “milter_protocol” depends on your postfix version. Please check this link. Also we decided to use local UNIX socket for communication between postfix and OpenDKIM for cleaner and more secure setup.

Restart postfix, and then send an test email again.

service postfix restart
echo -e "Subject: DKIM Verification Email\nDKIM test mail content. Please read the message header.\nSent at: $(date)" | sendmail -f no-reply@test.example.com -t <your-email-address>@gmail.com

Let’s check again the /var/log/mail.log postfix log file… oops, warning appeared!?

precise64 postfix/cleanup[5725]: warning: connect to Milter service unix:/var/run/opendkim/opendkim.sock: No such file or directory

Er, what’s up? A bit of google give me a hint that if postfix is running in chroot environment, it looks for files  under the directory /var/spool/postfix! Therefore I need to create the directory hierarchy:

mkdir -p /var/spool/postfix/var/run/opendkim
chown opendkim:opendkim /var/spool/postfix/var/run/opendkim

Since that directory is owned by the user ‘opendkim’, the user ‘postfix’ which runs the postfix daemon cannot write to it. We solved this by adding the system user ‘postfix’ to the ‘opendkim’ group:

usermod -G opendkim postfix

Since OpenDKIM is an independent process, we need to configure OpenDKIM to tell the correct location of local UNIX socket to use. Edit file /etc/default/opendkim and add the following line:

SOCKET="local:/var/spool/postfix/var/run/opendkim/opendkim.sock"

With this setup, OpenDKIM will open a local UNIX socket under postfix’s chroot directory /var/spool/postfix, and allow postfix to open that local socket.

Restart opendkim and… done! Try to send an email again, no more warning messages.

DKIM DNS Setup

Now I send another email from my Ubuntu box again… and the email header contains some dkim related entries finally:

Delivered-To: fake@gmail.com
..
Return-Path: <root@test.example.com>
Received: from test.example.com (<hide-for-security-reasons>)
        by mx.google.com with ESMTP id a4si372057pbj.197.1969.12.31.16.00.00;
        Wed, 02 Oct 2013 01:26:33 -0700 (PDT)
Authentication-Results: mx.google.com;
       dkim=fail header.i=@test.example.com
Received: by test.example.com (Postfix, from userid 0)
	id 3B2A43A0BC6; Wed,  2 Oct 2013 16:26:32 +0800 (HKT)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=test.example.com;
	s=default; t=1380702392;
	bh=APm/xHDwsW6xiUIVvtWoOZXf5R5WMJGjvbOfV3iMrfk=;
	h=Subject:Date:From:From;
	b=qnhOaHJwxean2fiY7cUJSP923DLmP5hAsRvg7vkHdY66Fi4Ksljvelb1x/aQng/Ko
	 oAszDkmGyBkmkajHcrlbY6cyxJSvQZfTnjIZFV1+/ZkD/PG/B8v93bHuEukcx1RUm1
	 TtxDfbwIxtYjq0P3kKq9RezpowSPIMW8YThI5WX4=
Subject: DKIM Test
Message-Id: <20131002082632.3B2A43A0BC6@test.example.com>
Date: Wed,  2 Oct 2013 16:26:32 +0800 (HKT)
From: root@test.example.com (Me)

DKIM test mail content one-liner

Notice the line “dkim=fail” and the mail header “DKIM-Signature”, now there are DKIM related headers in our mail which is good, however Gmail does not pass the DKIM authentication.

Reason is: I need to install my public key to the DNS server, which is part of DKIM configuration.

I logged in to GoDaddy and go to the domain I managed, which is example.com. The next thing to do is to add a TXT record with the following:

  • Host: default._domainkey.test. The format is actually in <selector>._domainkey . Since we are setting up DKIM for sub-domain test.example.com under example.com, so we need to append .test after _domainkey. If you setup is for the root domain ‘example.com’, you can skip that suffix.
  • TXT record value: v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDYArsr2BKbdhv9efugByf7LhaKtxFUt0ec5+1dWmcDv0WH0qZLFK711sibNN5LutvnaiuH+w3Kr8Ylbw8gq2j0UBokFcMycUvOBd7nsYn/TUrOua3Nns+qKSJBy88IWSh2zHaGbjRYujyWSTjlPELJ0H+5EV711qseo/omquskkwIDAQAB. This value comes from the file default.txt.
  • TTL: I set it as 0.5 hour as it is the minimum value I can set. It is recommend to set this to a small value.

You can find more about the specification of DKIM TXT record here, if you are interested.

You can quickly test if your DNS setup is reachable by remote server correctly by using this tool and go to the section “Check a published DKIM Core Key”, input default in Selector and test.example.com in Domain name field. Click “Check” button and you should be presented with a screen telling you “This is a valida DKIM key record”. However, this does not indicate your DKIM is 100% correct. This just verify that the DNS setup is correct, but does not check the messages sent by my Ubuntu Postfix server matches my DNS DKIM TXT record. So, I send an email (again) to verify it.

Check Your Mailbox for Victory

Send an email to your mail

echo -e "Subject: DKIM Test\nDKIM test mail content one-liner" | sendmail -F Me -t fake@gmail.com

Check your mailbox, and view the raw message, you should see something like this in your message’s headers:

Authentication-Results: mx.google.com;
       dkim=pass header.i=@test.example.com

If you see this line, congratulations! Your DKIM setup is working!

Setup SPF (Sender Policy Framework) to Allow Third-Party Servers to Verify Our Outbound Emails

Read the wiki for understanding of SPF if you are interested. You don’t need to read that before proceeding the steps below.

Before I have setup SPF, any email sent from my Ubuntu box will have the following headers when delivered to the recipient mailbox:

Received-SPF: neutral (google.com: <ip-address-hidden> is neither permitted nor denied by best guess record for domain of no-reply@test.example.com)
Authentication-Results: mx.google.com; spf=neutral (google.com: <ip-address-hidden> is neither permitted nor denied by best guess record for domain of no-reply@test.example.com)

Goal

Configure the environment such that recipient’s mail server will pass the SPF test for mail sent from my postfix server.

Installation

Since my goal is to allow third-parties (more specifically, third party receiving mail servers) to verify my emails to pass the SPF test, I just need to set it up in my DNS server. If you need to setup your postfix mail server to validate incoming (received) emails, you need to install additional packages which will not be covered there as I use my postfix for outbound email only ;P

The setup is ultra easy: Just add a DNS TXT entry and it is done!

I logged to GoDaddy again, and add the follow TXT record:

  • Host: @
  • TXT Value: v=spf1 a mx ptr -all
  • TTL: 1/2 hour

When done, test the setup using this link. I entered my Ubuntu server’s IP address and the sender email address (e.g. no-reply@test.example.com). The test should pass.

For your interest, this is the raw mail message (with sensitive data filtered):

Delivered-To: fake@gmail.com
Received: by 10.70.80.8 with SMTP id n8csp88731pdx;
        Wed, 2 Oct 2013 03:09:05 -0700 (PDT)
X-Received: by 10.68.135.100 with SMTP id pr4mr1679936pbb.62.1380708545656;
        Wed, 02 Oct 2013 03:09:05 -0700 (PDT)
Return-Path: <no-reply@test.example.com>
Received: from test.example.com (<sender-host-hidden> [<sender-ip-hidden>])
        by mx.google.com with ESMTP id m3si1497136pan.56.1969.12.31.16.00.00;
        Wed, 02 Oct 2013 03:09:05 -0700 (PDT)
Received-SPF: neutral (google.com: <sneder-ip-hidden> is neither permitted nor denied by best guess record for domain of no-reply@test.damingbo.com) client-ip=<sender-ip-hidden>;
Authentication-Results: mx.google.com;
       spf=neutral (google.com: <sender-ip-hidden> is neither permitted nor denied by best guess record for domain of no-reply@test.example.com) smtp.mail=no-reply@test.example.com;
       dkim=pass header.i=@test.example.com
Received: by test.example.com (Postfix, from userid 0)
	id 9A6113A0BC6; Wed,  2 Oct 2013 18:09:04 +0800 (HKT)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=test.example.com;
	s=default; t=1380708544;
	bh=csHn8kLI2tSpw+z0K6LgLtRp2SYOJSXxsE2HTFqDvr4=;
	h=Subject:Date:From:From;
	b=LvJsYtpNHYNRAeHqRHbfC0kZpcLWloUAZvomn+kAygnA+ySZusXtieFDGuMMa0JIs
	 kr87aoILReKDCaKxf6YuijMM3h66OFSSDsOMSJe16nXH2BVHGYkgGSQi/6Xb6Hk4ul
	 b2j63RY/eAlOyStxK/L5eK5jr7k0NcqCm3oc6ZTY=
Subject: Postfix Outbound Mail Test (Wed Oct  2 18:09:04 HKT 2013)
Message-Id: <20131002100904.9A6113A0BC6@test.example.com>
Date: Wed,  2 Oct 2013 18:09:04 +0800 (HKT)
From: no-reply@test.example.com (root)

Mail Content

References

Advertisements

8 thoughts on “My Experience of Setting Up DKIM and SPF with Postfix and Ubuntu For Sending Email

  1. thanks for your blog post. Helped me in setting up mail to gmail.com…
    but unfortunately since I already mailed to gmail.com several times before and it ended up in spam….even after everything is OK(SPF AND DKIM) it still ends up in spam…did you get this problem…?

    • Do you mean you saw your email ends up in the “Spam” folder in Gmail? Haha… spam is another thing… during the test I sent some email with simple contents like “Test”… and it ends up in Gmail’s “Spam” folder. To my understanding we can do nothing as it is Gmail’s mail server (?) logic to determine what is that. DKIM and SPF is not a 100% tool to fight all kind of spams.

      • As you may already know, DKIM or SPF only authenticates your domain, essentially telling the server that you own the domain and are not sending from random account using hotmail, gmail or some such thing. It’s the content that makes the servers categorize your email as spam or otherwise. Check your headers. A lot of times it could be a problem with your headers that ends up making it look like it’s not signed properly. This document http://media.amazonwebservices.com/AWS_Amazon_SES_Best_Practices.pdf from Amazon SES gives tips for being a good SES citizen but a lot of it should apply to any email server. I found another link on how to avoid Postini from filtering your email as spam http://blog.returnpath.com/blog/tonya-mitchell/8-ways-to-help-avoid-postini-filtering.
        As these two articles suggest, you may have to make sure your email is not already blacklisted.
        Hope this gets you in the direction to answer why your email ended up in spam.

  2. Thanks for a good tutorial. I just want to tell you that you should fix;

    this:
    “/etc/default/opendkim and add the following line:”

    with this:
    “/etc/opendkim.conf and add the following line:”

  3. I forgot to mention; mode is also important. If you want DKIM for only outgoing mails, mode s probably has better performance than the default sv (verify & sign) options.

  4. I’ve followed your guide, but the DKIM part of the header simply will not appear in my test mails, and dkimcore.org says DNS query failed for ‘default._domainkey.***.**’:NXDOMAIN (domain replaced by *’s). Do you have any idea what could be wrong?

  5. stiil getting
    postfix/smtpd[7505]: warning: connect to Milter service unix:/var/run/opendkim/opendkim.sock: No such file or directory

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s