Anyone expert in SFP, DKIM, and DMARC?

TidBITS Talk
1

After hearing about a few instances where our email (both mine and Tonya’s personally, and transactional and bulk mail from our systems) wasn’t being delivered properly, I dove into my SPF, DKIM, and DMARC setup. I think I’ve done everything right, and checkers all claim that I have, but I’m still seeing some problems.

SPF: https://easydmarc.com/tools/spf-lookup?domain=tidbits.com
DKIM: https://easydmarc.com/tools/dkim-lookup?domain=tidbits.com
DMARC: https://easydmarc.com/tools/dmarc-lookup?domain=tidbits.com

A few of the problems are probably impossible to avoid, since if someone at cornell.edu forwards their email to a Gmail address, for instance, mail from me will fail authentication because I don’t have cornell.edu allowed to send tidbits.com mail. If SPF allowed more than 10 lookups, I might special-case a few domains like cornell.edu, but I’m already having to specific IP addresses manually to get around the 10-lookup limit.

Where I get worried, though, is when I look at reports. For instance, Cloudflare shows only 93% of email sent through Gmail and 72% sent through Microsoft has passing DMARC. (I’m not even sure what tidbits.com mail gets sent through Microsoft, since Tonya and I both use Gmail as the backend of our tidbits.com accounts.) Maybe it’s just forwarding, but if not, that’s 20 failed messages through Gmail and 60 through Microsoft in a week. That’s a lot, and it’s hard to know when a message you send doesn’t arrive. Even with Amazon SES, which handles most of our outgoing bulk and transactional mail, the .32% failure rate works out to over 1300 failed messages in the week.

Arc 2023-07-18 at 15.32.54@2x
Arc 2023-07-18 at 15.32.54@2x2550×1186 275 KB

But why do Amazon and Google fail SPF when I have them included in my SPF record with include:_spf.google.com and include:amazonses.com?

v=spf1 ip4:64.68.200.0/22 ip4:64.68.200.41 ip4:64.68.198.23 ip4:64.68.203.22 ip4:64.68.198.24 ip4:64.68.200.59 ip4:64.68.200.63 ip4:205.201.128.0/20 ip4:198.2.128.0/18 ip4:148.105.8.0/21 ip4:64.62.152.51 include:easymail.ca include:_spf.google.com include:amazonses.com include:mailchannels.net ~all

And then I get this report from Postmark DMARC, and the only thing I can pull out as an actionable item is perhaps setting up a Return-Path with Amazon SES. But that gets into MX record juju, which I really don’t want to mess up. And other stuff I just don’t understand, such as how just a couple of IPs at google.com can fail DKIM—if I’ve set it up correctly, how can it fail for just a few? Similarly, Postmark DMARC claims that easyDNS’s 64.68.200.41 server is failing SPF, but it’s explicitly listed in my SPF record with ip4:64.68.200.41. Augh!

Anyone have any experience with SPF, DKIM, and DMARC that would provide insights into what’s happening here? Thanks!

Your sources

These are sources that we know belong to you based on the DNS checks we do.

amazonses.com TOTAL SPF ALIGNED DKIM ALIGNED
54.240.11.69 509 0% 99.8%
54.240.8.32 505 0% 100%
54.240.10.1 503 0% 100%
54.240.11.44 503 0% 100%
54.240.11.60 503 0% 100%
43 more IPs 19,891 0% 99.8%
amazonses.com is authorized to send on behalf of tidbits.com, however it looks like SPF is still failing DMARC’s alignment test. DMARC looks at the Return-Path of a message to make sure the domain there matches the domain in your From address. If the Return-Path path doesn’t match your From address, those messages will fail DMARC’s SPF alignment test. Check with this source because you may need to set up a custom Return-Path.
google.com TOTAL SPF ALIGNED DKIM ALIGNED
209.85.220.41 560 0% 98.2%
209.85.220.69 9 11.1% 0%
209.85.220.97 5 0% 100%
209.85.128.172 2 0% 100%
209.85.128.177 2 0% 100%
90 more IPs 106 0% 98.1%
easydns.com TOTAL SPF ALIGNED DKIM ALIGNED
64.68.202.10 128 100% 0%
64.68.200.41 123 0% 76.4%
Not Resolved TOTAL SPF ALIGNED DKIM ALIGNED
64.62.152.51 8 0% 0%
192.92.191.183 2 0% 0%
170.158.12.141 1 0% 0%
185.218.11.138 1 0% 0%
192.92.191.182 1 0% 0%
4 more IPs 4 0% 0%
:warning: Not Resolved is authorized to send on behalf of tidbits.com, however it looks like SPF and DKIM are still failing DMARC’s alignment test. DMARC looks at the Return-Path of a message to make sure the domain there matches the domain in your From address. If the Return-Path path doesn’t match your From address, those messages will fail DMARC’s SPF alignment test. Set up a DKIM record and check with this source about setting up custom Return-Path.
mailchannels.net TOTAL SPF ALIGNED DKIM ALIGNED
23.83.209.13 1 0% 100%
23.83.209.150 1 100% 0%
23.83.212.19 1 100% 0%
23.83.212.4 1 100% 0%
23.83.212.50 1 100% 0%
4 more IPs 4 100% 0%
zoneedit.com TOTAL SPF ALIGNED DKIM ALIGNED
64.68.198.23 6 0% 100%
64.68.198.24 2 0% 100%
zoneedit.com is authorized to send on behalf of tidbits.com, however it looks like SPF is still failing DMARC’s alignment test. DMARC looks at the Return-Path of a message to make sure the domain there matches the domain in your From address. If the Return-Path path doesn’t match your From address, those messages will fail DMARC’s SPF alignment test. Check with this source because you may need to set up a custom Return-Path.
easydns.net TOTAL SPF ALIGNED DKIM ALIGNED
64.68.203.22 5 0% 100%
easydns.net is authorized to send on behalf of tidbits.com, however it looks like SPF is still failing DMARC’s alignment test. DMARC looks at the Return-Path of a message to make sure the domain there matches the domain in your From address. If the Return-Path path doesn’t match your From address, those messages will fail DMARC’s SPF alignment test. Check with this source because you may need to set up a custom Return-Path.
easymail.ca TOTAL SPF ALIGNED DKIM ALIGNED
64.68.200.59 3 0% 100%
64.68.200.63 1 0% 100%
easymail.ca is authorized to send on behalf of tidbits.com, however it looks like SPF is still failing DMARC’s alignment test. DMARC looks at the Return-Path of a message to make sure the domain there matches the domain in your From address. If the Return-Path path doesn’t match your From address, those messages will fail DMARC’s SPF alignment test. Check with this source because you may need to set up a custom Return-Path.

Other sources

These sources are sending emails saying they are from tidbits.com, but we couldn’t verify that they belong to you. It’s important to go through each source and set up SPF and DKIM records for sources that you control.

maaat.com TOTAL SPF ALIGNED DKIM ALIGNED
23.239.4.200 44 0% 0%
Use the API to fetch the 23 remaining domains.

Forwarded email sources

These sources might be from forwarded emails. The email headers are often preserved when emails are forwarded causing DKIM to pass and SPF to fail DMARC alignment because the From address no longer matches tidbits.com’s SPF record. Generally, there’s nothing to worry about if you already have SPF and DKIM set up on your sources.

Learn more about forwarded sources →

me.com TOTAL SPF ALIGNED DKIM ALIGNED
17.58.23.191 31 0% 100%
17.58.23.183 28 0% 100%
17.58.23.193 25 0% 100%
17.58.23.182 23 0% 100%
17.58.23.188 23 0% 100%
85 more IPs 565 0% 100%
cornell.edu TOTAL SPF ALIGNED DKIM ALIGNED
3.224.229.126 178 0% 0%
128.253.150.158 21 0% 28.5%
outlook.com TOTAL SPF ALIGNED DKIM ALIGNED
2a01:111:f400:7eae::700 4 0% 75%
2a01:111:f400:7eae::601 3 0% 100%
2a01:111:f400:7e8a::613 2 0% 100%
2a01:111:f400:7e8b::617 2 0% 50%
2a01:111:f400:7eae::600 2 0% 100%
119 more IPs 120 0% 62.5%
pobox.com TOTAL SPF ALIGNED DKIM ALIGNED
173.228.157.40 16 0% 100%
173.228.157.39 15 0% 100%
64.147.108.55 15 0% 100%
64.147.108.50 14 0% 100%
64.147.108.51 12 0% 100%
3 more IPs 26 0% 100%
umich.edu TOTAL SPF ALIGNED DKIM ALIGNED
52.37.117.118 13 0% 100%
13.58.214.35 11 0% 100%
18.219.197.222 11 0% 100%
18.188.205.92 10 0% 100%
35.160.237.124 10 0% 100%
2 more IPs 9 0% 100%
elnk.net TOTAL SPF ALIGNED DKIM ALIGNED
24.41.67.42 21 0% 100%
24.41.67.41 18 0% 100%
1e100.net TOTAL SPF ALIGNED DKIM ALIGNED
2600:1901:101::11 19 0% 100%
2600:1901:101::10 15 0% 100%
108.177.16.12 1 0% 0%
108.177.16.13 1 0% 100%
2600:1901:101:: 1 0% 100%
1 more IP 1 0% 100%
yahoo.com TOTAL SPF ALIGNED DKIM ALIGNED
74.6.132.42 3 0% 100%
74.6.128.205 2 0% 100%
74.6.135.82 2 0% 100%
66.163.184.147 1 0% 100%
66.163.186.206 1 0% 100%
25 more IPs 25 0% 100%
comcast.net TOTAL SPF ALIGNED DKIM ALIGNED
2001:558:fd00:56::b 2 0% 100%
2001:558:fd01:2bb4::4 2 0% 100%
2001:558:fd01:2bb4::d 2 0% 100%
2001:558:fd02:2446::3 2 0% 100%
2001:558:fd02:2446::8 2 0% 100%
19 more IPs 21 0% 100%
Use the API to fetch the 230 remaining domains.
2

I can only offer a lot of sympathy, and be relieved all over again that I’m finally no longer running a mail server.

At least part of problem is that large players don’t actually follow the protocols. E.g. gmail will happily tell your log that your mail was accepted for delivery, but sometimes to often sends the message to dev/null instead of the user (especially but not exclusively for mailing lists). There was no pattern that I was able to determine in a fair bit of testing for a month with a half dozen people.

Another part is that a lot of biggish entities (university/corporate scale) subscribe to extra filtering layers such as Proofpoint. Fortunately I retired before having to deal with the horrendous issues that Proofpoint caused for the university by blocking email both in and out for nebulous never defined criteria (we can’t tell you, the bad guys would work around it!) They also rewrote urls in the email so that if you forwarded it, the links would break. I know that at least one grant came within a couple of hours of being rejected because the PI didn’t receive an email asking for a clarification. Phone call to the rescue, but that was luck. I wouldn’t be surprised if other departments did lose grants. I’ve heard that it seems to have settled down now, but it took about a year.

Even Fastmail isn’t perfect. I don’t know of any personal or shopping type mail loss, but they do drop some tidbits mail.

I wonder what groups.io delivery success rate is? I’m on a couple of their mailing lists, but haven’t bothered to find out if any gets lost compared to the web forum interface.

It’s been sad watching email go from the most reliable long distance form of communication ever to one of the least.

3

Yeah, and I’m not even running a real mail server, just services that send a lot of transactional and bulk mail, coupled with a custom domain. A bit of the SPF problem is that long ago, we gave tidbits.com email addresses to various family and friends, so even though there’s no actual mailserver behind it any more (we run it through easyDNS’s easyMail system with forwarding), we still have to account for their forwarding setups.