So tonight I figured it was time to configure my mail server to look at SPF records. While searching around for a SPF implementation that seemed reasonable (ie, not written in Perl), I found python-postfix-policyd-spf, which is written in Python (although the code is not Pythonic at all).

After installing the PyDNS and PySPF module dependencies, I installed python-postfix-policyd-spf by running ‘python setup.py install‘ and then things were ready to be configured.

The next step was to configure Postfix, which was actually very easy. I added the following line to my /etc/postfix/master.cf file:

spfpolicy unix  -       n       n       -       -       spawn
        user=nobody argv=/usr/bin/python /usr/bin/policyd-spf

And the following to /etc/postfix/main.cf:

smtpd_recipient_restrictions =  permit_mynetworks,
                                permit_sasl_authenticated,
                                check_client_access hash:/etc/postfix/pop-before-smtp-relays,
                                check_recipient_maps,
                                reject_unauth_destination,
                                check_recipient_access hash:/etc/postfix/badmailto,
                                check_policy_service inet:127.0.0.1:60000,
                                check_policy_service unix:private/spfpolicy,
                                permit
spfpolicy_time_limit = 3600

Be sure to add the check_policy_service unix:private/spfpolicy line after the reject_unauth_destination line, otherwise you’ll be an open-relay.

Run ‘postfix reload‘ to get Postfix to acknowledge the changes, and that’s it.

After setting this up and watching the logs for a while tonight, I noticed a few things.

First, there are a lot more domains using SPF than I thought. I know it’s been out for a few years now, but apparently it has really taken off. The only shame is that most of the domains that have SPF records seem to still be using the ~all code, which basically makes it pointless.

The second thing I noticed tonight isn’t quite as shocking: spammers are really careless and dumb. The first two messages that were rejected via SPF were because the spammer actually set up SPF records for their spamming domains, but they screwed it up. The log messages for those two are below:

Mar 29 01:07:05 silver policyd-spf[9260]: SPF fail - not authorized:QUEUEID=;
       identity=mailfrom; client-ip=65.254.160.36; helo=mail.meckcom.net;
       envelope-from=colonproblems@jermaynepaganochristianism.com;
       receiver=victim@example.com;
Mar 29 01:19:00 silver policyd-spf[9508]: SPF fail - not authorized:QUEUEID=;
       identity=mailfrom; client-ip=65.254.160.36; helo=mail.meckcom.net;
       envelope-from=mortgagenewsfromlowermybills@trythetips.com;
       receiver=victim@example.com;

Investigating the first one, I found this:

jermaynepaganochristianism.com. 600 IN  TXT     "v=spf1 a mx ip4:38.98.2.0/24 -all"

So the spammer who bought jermaynepaganochristianism.com (which was registered earlier this month) decided to set up a record specifying which hosts could send mail for that domain, and then send the spam through a different server. Brilliant!

Here is another interesting one:

Mar 29 00:58:21 silver policyd-spf[9071]:
       SPF Permanent Error: Invalid IP4 address: ip4:72.11.154.128/25-all:
       QUEUEID=; identity=mailfrom; client-ip=72.11.154.232; helo=mail.anbermedia.com;
       envelope-from=smtp@anbermedia.com; receiver=victim@example.com;

In this case the spammer who bought anbermedia.com (which was registered today) set up a SPF record, but screwed it up by not placing a space between /25 and -all, thereby making it an invalid record and causing mail to be rejected. Sweet!

In the 2 hours I have had SPF in place, its blocked 10 messages or so. That isn’t a huge amount, but my server doesn’t move a tremendous amount of mail, especially around midnight. It will be interesting to see how well this works when the server is busy.

Next I plan to try out DomainKeys and see if that helps any.

I’ve been using Mail.app for about 3 years now and after running the sqlite command to vacuum the index I noticed a pretty significant speedup.

Update: I showed this to Will and he tried vacuuming all the other tables as well. I tried it too and it made things even faster! Here are the steps:

cd ~/Library/Mail
sqlite3 Envelope\ Index
sqlite> vacuum;
sqlite> .quit