Re: SPF/DKIM/DMARC et.al.: REALLY LONG [was: is it just me or...]

[Tom Beecher via NANOG <nanog () lists nanog org>]

Sir, this is a Wendy's.

On Wed, Jul 2, 2025 at 3:47 PM Rich Kulawiec via NANOG <
nanog () lists nanog org> wrote:

> On Sun, May 25, 2025 at 11:20:16AM +0200, Tom Ivar Helbekkmo via NANOG
> wrote:
> > First: SPF/DKIM/DMARC are not about spam, so that part is irrelevant.
>
> Perhaps you don't remember this, but when SPF was announced, its home
> page read:
>
>         "Spam as a technical problem is solved by SPF."
>
> Shortly thereafter, spammers became far and away the largest early
> adopters of SPF.
>
> Of course, that claim was eventually disappeared down the memory hole.
>
>
> Now onto the long, and I do mean REALLY LONG part of this.  Sorry,
> it's a complex issue and it takes time to explain all the pieces.
>
> Here's a one-sentence summary:
>
> All these alleged anti-email-forgery technologies have accomplished is
> to make the email forgery problem MUCH worse than before they existed.
>
>
>
> Introduction
> ------------
>
> I've never considered email forgery to be a significant problem --
> not when compared to the other problems we face.
>
> But let's put my opinion aside for a moment, and let's presume that email
> forgery really is a significant problem -- one so serious that it's worth
> adding an enormous amount of fragile complexity to an ecosystem already
> under serious stress from spam and other attacks/abuse.  Let's assume
> that it's worth breaking email forwarding (working fine for decades)
> and mailing lists (working fine for decades, and clearly the best mass
> collaboration/communication mechanism we have) and adding enormous cost,
> effort, and complexity to every email system.
>
> There's a problem with that: email forgery can't be solved.
>
> Even if if these byzantine hacks were perfectly designed (they're not)
> and globally deployed (not happening) and correctly deployed everywhere
> (nope) and always worked perfectly (also no) and weren't attacked (they
> are) and so on, even if everything worked *exactly* as intended...the
> overall impact of all this effort and expense and brittle complexity is
> to make the email forgery problem MUCH worse.
>
> I've explained this before (briefly) but let me add a lot more detail
> this time around.
>
>
> Part A: contributory factors
> ----------------------------
>
> A1. We've had a problem with fake/junk domains for decades.  It was bad
> enough when there were only 7 gTLDs, but now there are now ~1000 new gTLDs
> that nobody needed or wanted except (a) registrars, who of course want to
> make the cash registers ring and (b) abusers/attackers, who buy domains in
> bulk, burn through them, and then buy more.  This symbiotic relationship
> works beautifully for both parties involved but not for anyone else.
> And of course registrars are performing no due diligence whatsoever
> (why should they?)  so it's trivial for abusers/attackers to register
> example.xyz or example.lol or example-com.xyz or example-site.red or
> whatever, where "example" is a name explicitly designed to suggest that
> they're some other Internet operation -- e.g., Amazon, Paypal, Google,
> Joe's Donuts, etc.
>
> I've been studying large samples of domain data for multiple decades,
> and there are enormous numbers of these fake/junk domains.  Many are
> abandoned once they've served their purpose, but new ones are being
> registered constantly.  And every time a new TLD is put into production,
> there's a land rush as abusers/attackers swarm it in an attempt to be
> the first to register the optimal fake domains.
>
> I see no signs that this problem will be solved, because the people
> responsible for creating it don't want it to be solved.  If these
> registrars performed any kind of due diligence, it would (a) cost them
> money and (b) reduce their sales.  There's no reason in the world for
> them to do that.
>
> Bottom line: it's cheap, simple, and easy to register an arbitrary number
> of domains whose names correspond to any operation that abusers/attackers
> wish to target.
>
> You want a real-world example?  Okay.  Here's an example:
>
>         Infrastructure Laundering: Blending in with the Cloud
>
> https://krebsonsecurity.com/2025/01/infrastructure-laundering-blending-in-with-the-cloud/
>
>         U.S. Sanctions Cloud Provider "Funnull" as Top Source of "Pig
> Butchering" Scams
>
> https://krebsonsecurity.com/2025/05/u-s-sanctions-cloud-provider-funnull-as-top-source-of-pig-butchering-scams/
>
>         Complete List of Domains Attributed to Funnull
>
> https://www.ic3.gov/CSA/2025/Complete_List_of_Domains_Attributed_to_Funnull.zip
>
> There are 332,696 lines in that list.  Well over a quarter million
> fake/junk
> domains used by just one scamming operation.  And as large as this is,
> it's the
> tip of the tip of the tip of the iceberg.
>
> Let me also ask a very pointed question: if someone shows up at your
> hosting
> or cloud operation, and they want to deploy, let's say, 52,782 domains: do
> you
> think that maybe, just *maybe*, you ought to ask a few questions about
> what the heck they're up to?
>
> If you're Microsoft or Amazon (see Krebs articles above): nope.  Which
> brings
> me to:
>
> A2. There are all kinds of hosting/cloud operations out there which will
> happily take example.xyz's money because they perform the same level of
> due diligence as registrars in (1): none.  These operations know full well
> that example.xyz is not Example Inc, and they know full well what they're
> doing, but because example.xyz pays the bills and they're repeat/bulk
> customers, the hosts/clouds just take the money and look the other way.
>
> This isn't a small problem.  It's massive.  And it includes a lot of
> hosting/cloud operations that should know better.  And probably *do*
> know better, but: due diligence costs money and takes effort, it's
> cheaper and easier to just blow it off and only take action belatedly --
> long after the damage is done -- if there's any bad press.
>
> Bottom line: even the most obvious scam/fraud operations have no problem
> finding hosting.
>
> A3.  The combination of (A1) and (A2) mean that you can go out today
> and register 1000, 10000, 100000 fake domains impersonating the Example,
> Inc's of the world and you can find hosting for them, and you can do it
> all at a bulk rate discount.  Your only real difficulty will be competing
> with the other people doing the same thing.  And unless you do something
> that generates a sufficient amount of sufficiently bad press, it's very
> unlikely that a registrar or host will take any action to shut down
> anything you're during, and even if they do that, they'll only shut down
> the small part directly involved -- everything else you have in place will
> be just fine.  And then, on top of all this, they'll slow-walk whatever
> they do in order to keep the income coming as long as they possibly can.
>
> A4. User interfaces in email clients -- whether standalone pieces
> of software, or webmail interfaces that run in a web browser -- are
> increasingly being modified to obfuscate sender addresses.  In other
> words, rather than displaying:
>
>         From: Joe Smith <joe () example com>
>
> they display:
>
>         From: Joe Smith
>
> and some kind of user action is required to actually see the address.
>
> The aggregate effect of this, across all UIs across all clients across
> all users, is to train users to ignore actual email addresses in favor
> of the text strings that purport to be someone.
>
> (Did YOU check the actual email address in this message to see if it's
> really from me?)
>
> A5. One consequence of (A4) is that users are increasingly unable
> and/or unwilling to understand email addresses.  They can't distinguish
> @example.com from @example.xyz or @example-com.lol or @exammple.com
> or @exammple.somethingsomething.fun.
>
> (Yes, yes, there are *some* users that can do this.  A few clueful
> ones.  As in: "A person is smart.  People are..."  C'mon, you know
> the rest of this.)
>
> This loss of comprehension/function resulting from (A4) and (A5) works
> beautifully for the operators of domains in (A1) and (A2): they don't
> even have to try hard.  Even sloppy half-ass typosquatted domains (and
> subdomains) work just fine.
>
>         [ Let me interject that while I was drafting this I read about the
>         plans by Mozilla to completely screw up the address bar in Firefox
>         -- because of course leaving the simplest, most fundamental, and
>         most important thing in the browser alone was never a choice for
>         these incompetent clowns.  If I understand what they're going to
>         do correctly, and I really hope I don't, they're going to make
>         life MUCH easier for forgers/scammers/et.al. who are deploying
>         fake domains. ]
>
> A6. User interfaces in email clients are being modified to provide
> signaling to users that a given message has been "authenticated" or
> "verified" or "signed" or "certified" or some similar designation.
> This is of course the result of a successful check of whichever
> anti-forgery mechanism(s) the receiving email server chose to utilize.
>
> And just as users being trained to accept (A4), they're being trained to
> accept this.  They believe what's in front of them on the screen.
>
> But of course this means *nothing*: a message from joe () example xyz will
> be dutifully marked (provided the keepers of example.xyz set things up
> correctly and everything's working) as authentic even though example.xyz
> is a fake domain that has nothing to do with the real Example Inc. and
> Joe is a fake person.
>
> In other words, obviously fake messages from obviously fake domains are
> treated the same as authentic messages from authentic domains.
>
> ("But couldn't...".  No.  It's not a tractable problem, because it would
> require enumerating the authentic domain(s) of every operation on the
> Internet, i.e. determining that example.com is the "real Example Inc. --
> for millions of example.com's, example.edu's, example.net's, etc.  And not
> only would this be incredibly expensive, it would break badly as companies
> and domains change ownership or go out of business, it would greatly
> favor large operations over small ones, it would quickly be corrupted by
> the people with the biggest wallets, it would...  So don't even go there.)
>
> This is analogous to the wry saying about https:// -- yeah, it means
> you have a (putatively) secure connection, but you could have a secure
> connection to the worst people in the world.
>
> A7. Many Example Inc's of the world don't send plaintext email messages;
> they mark them up with HTML and graphics and so on, which means that
> they're teaching their users that any message which looks like it's from
> them is really from them.
>
> They're training their users to fall for forgeries/phishes.
>
> And years (decades now) of this training have paid off: it's much,
> MUCH easier to trick users now than it was 20 years ago.
>
> A8. Similarly, many Example Inc's of the world include URLs in their
> messages.  They've spent years (decades now) training users to utilize
> those URLs, and of course users have dutifully learned to do this,
> and once again:
>
> They're training their users to fall for forgeries/phishes.
>
> And years (decades now) of *this* training have also paid off: again,
> it's much easier to trick users now than it was 20 years ago.
>
> A9. The combination of (A7) and (A8) means that there's no help coming
> from the Example Inc's of the world.  They'll continue to train
> their users to be victims while exhorting them to not be victims.
> The relevance to my point here?  These email worst practices distract
> users from the small pieces of critical information that *might* give
> them a slim chance of spotting a forgery.  They don't cause the problem,
> but they certainly exacerbate it.
>
> A10.  The aggregate impact of (A1)-(A9) is: users see a message that looks
> as they expect it to -- it has the pretty graphics and typography of
> Example Inc.  It claims to be from Example Inc.  It's marked as "verified"
> in their email UI.  It has a domain that looks kinda like Example Inc's
> (if they even bother to check, which almost none of them will).
>
> What do you *think* they're going to do?  I can tell you what they won't
> do: they won't take the time to actually study the email address and see
> that it's @example-com-completely-fake.xyz, or to study the embedded URLs,
> and even if they do, a substantial fraction of them Will Not Get It.
>
> If you think I'm wrong about this, then I must ask: have you actually
> spent any time with a significant number of users?
>
>
> Part B: large email operations
> ------------------------------
>
> It's not always necessary to go through the trouble and expense of
> registering domains, arranging for hosting, etc.
>
> It's easy to set up an arbitrary number of accounts that claim to be
> arbitrary people/companies and spam/phish/whatever from them at will,
> because there are some very large mail operations that make this easy.
>
> And because those same operations have implemented all this anti-forgery
> tech, those messages will dutifully pass every check that can be done
> on them.
>
> And before someone trots out the "...but they're so large and it's SOOOO
> hard..." nonsense as an excuse for the malfeasance of these very large
> email operations: no.  If you can't run an enormous operation properly,
> you shouldn't *build* an enormous operation.  It's irresponsible.
> It's unprofessional.  It's unethical.
>
> Especially when it would be trivial for these operations to throw 1,000
> people and a billion dollars at the problem tomorrow.  (We *know* they
> have those resources, because they're already wasting them on bogus
> "AI" development.)
>
> Let me note, in fairness, that there are also some other rather large
> operations that -- as far as I can tell by measuring across hundreds of
> domains over several decades -- do a very good job.  The problem is that
> they're the exception.
>
>
> Recap of part A and part B:
> ---------------------------
>
> The combination of (A) and (B) means that we have an environment where
> it's cheap, easy, and simple to register an arbitrary number of domains
> masquerading as Example Inc.  It's also cheap, easy, and simple to find
> hosting for them.  And then -- thanks to very poor choices in user
> interfaces combined with lack of user knowledge/effort, it's cheap,
> easy, and simple to convince lots of people that completely fake messages
> from completely fake domains are authentic.  And there's even a pretty
> good chance that forgeries which don't even use a fake domain will work:
> there are certainly unlimited opportunities to try -- also cheap, easy,
> and simple.
>
> The sad reality is that while it's true that thanks to this anti-forgery
> tech, abusers will have a hard time successfully forging messages from
> the real example.com and getting them delivered: they don't have to.
>
>
> Part C: bots
> ------------
>
> And then it gets even worse, because of a problem that still exists
> but doesn't garner the headlines that it did years ago.
>
> The problem is bots.  Remember this?
>
>         Vint Cerf: one quarter of all computers part of a botnet
>         http://arstechnica.com/news.ars/post/20070125-8707.html
>
> That was 18 years ago.  And in the ensuing time, *nothing* has happened
> to seriously address that problem.  (Yes, yes, I'm aware of all the
> headline-grabbing when someone somewhere dismantles a botnet, but
> those are momentary disruptions at best.)
>
> On the other hand, lots of things have happened to make the bot problem
> worse, including (1) more computers (2) more users (3) vastly more
> sophisticated bot-creating malware (4) bot-creating malware targeting
> non-Windows systems (5) major improvements in botnet C&C (6) the IOT
> -- remember, the "S" in "IOT" stands for security.  And (7) thanks to
> Microsoft's "Recall", things will get worse yet again.
>
> Oh, and let's (8) toss in AI, because the people frantically developing
> those systems want to spend lots of time and money hyping them but can't
> be bothered to put in any guardrails or do any real testing or even
> concern themselves for a moment about how this tech will be misused by
> some of the worst people on the planet.  I'll bet even money that there's
> already a botnet being run by a commercial AI.  It's an obvious move,
> don't you think?
>
> Note that:
>
> C1. Anyone in control of a bot can utilize any email credentials stored
> on or transiting that bot at will.  If Joe has an address @example.org
> and one at @aol.com, then the new owner of Joe's system can send outbound
> messages via those at will.
>
> C2. They can also rummage through all email stored on that system or
> in the accounts from (1), which means that -- if they want -- they can
> easily make some first-order guesses at who's likely to believe that
> a message which claims to be from Joe AND that their email system's UI
> says is from Joe, really is from Joe.
>
> C3. This is already bad if Joe is just some guy and Joe's system is just
> some random laptop somewhere.  But what if it's not?  What if Joe works
> at Example Inc. and this is his desktop system and it's loaded with
> customer correspondence and it's connected to his work email account,
> also loaded with customer correspondence?  *Now* the new owner of Joe's
> system is in an ideal situation to phish or scam all of those people,
> because such a message really will come from Joe's email address via
> Joe's work email server, it'll pass all the anti-forgery checks anyone
> cares to run, and it'll dutifully be marked as "authentic" in most/all
> those email UIs used by its recipients.
>
> So even if the content is a little sketchy -- maybe enough to raise
> an eyebrow or two among recipients -- they'll look at their screen,
> see that the message is marked as real, see that the fancy graphics
> and typography look real, AND THEY'LL BELIEVE IT.  Of course they will,
> they've been trained to, see A7 and A8 and above
>
> C4. And what about the tiny fraction of people who will catch on that
> think that's something amiss?  What, EXACTLY, do you propose that they
> do about it?  Example Inc has probably done its very best to turn itself
> into an impenetrable corporation where it's nearly impossible to reach
> anyone in a position to understand a problem like this and take prompt
> action to fix it.  Do they have working postmaster@, security@, abuse@
> addresses -- something that every novice sysadmin on this planet should
> know about?  Probably not.  Do they have a published security contact?
> Probably not.  Do they have any mechanism whereby someone who calls in can
> reach anyone useful?  Probably not.  So even if one of those rare, clueful,
> diligent people spots this and tries to report it, they're probably
> going to give up in frustration long before they achieve anything.
>
> C5. And even if -- by some miracle of science -- someone in (C4) manages
> to reach a contact inside Example Inc who understands the problem...the
> most likely outcome is that Example Inc will ignore it and pretend it
> doesn't exist.  The next most likely outcome is that they'll quietly fix
> it but say nothing.  Why should they inform or warn anyone?  That's bad
> for business.  And since there are no significant penalties for anyone
> involved, their best bet is to disappear the problem and claim it
> never happened.  (If caught?  Deny everything.  If REALLY caught?
> Blame a miscommunication and throw a third-level manager under the bus.)
>
> Sadly, (C4) and (C5) are not hypothetical.  They're why I don't even
> bother any more.  Why should I invest my time and unpaid labor when
> these operations can't even be bothered to do the simplest, easiest,
> most obvious  things like supporting RFC 2142 addresses and reading
> what shows up there, acting on it promptly, and publicly acknowledging
> what's happened?
>
> How many email accounts are affected by bot proliferation?  A first-order
> estimate may be found by multiplying (a) an estimate of the worldwide
> population of bots by (b) an estimate of the average number of email
> accounts used/present on those bots.  I think the floor for (a) is 750M
> and 1.5B is much more likely.  (Go back to Cerf's estimate from 18 years
> ago and extrapolate.)  As to (b) I think it's probably between 1 and 2 --
> let's go with 1.5, probably low but that may also account for duplicate
> email accounts across bots.
>
> This gives us a quite conservative back-of-the-envelope guesstimate
> somewhere north of 1B.
>
> One billion compromised email accounts.
>
> And you want to tell me it's possible to do something truly meaningful
> about email forgery?  Okay.  Fine.  *Fix this problem first*, then
> we'll talk.
>
> I'll wait.  Let me know when you're done.
>
> Note that not only do you have to (1) remediate all those accounts,
> you'll also need to (2) remediate all the bot'd systems and then
> (3) keep them that way -- which means discovering the root cause(s) of
> their
> compromise and fixing it.  (3) will be very difficult if the root cause is
> something like "the user clicks on every shiny thing they see".  But if
> you're going to claim it's possible to do something truly meaningful
> about email forgery, then you must do all those steps first.
>
>
> Part D: email account compromises
> ---------------------------------
>
> And (C) isn't even all the compromised email accounts, because we also
> have to think about all the cases where only an email account (not an
> entire system) has been compromised.  I don't know how many there are
> (nobody does), but I strongly suspect it's also an enormous (hundreds of
> millions or more) number.  One way to approach an estimate of this is to
> observe the marketplaces where they're being sold, and in those places,
> there are lots people selling them in bulk: how many hundred thousand
> would you like?
>
> (Do recall that a few years ago, EVERY Yahoo email account was
> compromised.  That was a neat 3 billion accounts right there.
>
>         Every single Yahoo account was hacked - 3 billion in all
>
> https://money.cnn.com/2017/10/03/technology/business/yahoo-breach-3-billion-accounts/index.html
>
> And as anyone who's paying attention to the daily parade of security
> breaches and dataloss incidents is painfully aware: this isn't an
> isolated case.  It's the norm.)
>
> This problem is exacerbated by the complete lack of support at some very
> large email operations: even when someone who's the real owner of one
> of these compromised email accounts detects it and make a good-faith
> effort to fix the problem...yeah, good luck with that.  Their chances
> of recovering their own email account are pretty much zero thanks to
> awful procedures and non-existent customer service.  Thus there's a
> high probabilty that any compromised email account is going to stay
> compromised until it no longer exists.
>
> Again, if you want to tell me that something serious is going to happen
> WRT email forgery: *you have to fix this too*.  Let me know when that's
> done as well.
>
>
> Part E: email server compromises
> --------------------------------
>
> And *then*, on top of (C) and (D), we have to consider the cases where
> email servers themselves have been compromised, and there's a steady
> parade of those.  Hardly a day goes without another announcement of
> a massive security breach that involves most/all of some operation's
> infrastructure,
>
> Those announcements are only the tip of the iceberg, since pretty
> much everyone does their best to conceal such things until either the
> press reports on them and/or they're legally required to report them...
> and even *then* they'll sometimes persist in denial and minimization.
>
> I'm looking right at you, Oracle.
>
> Anyway, compromise of an email server means that it's not only possible to
> undetectably forge email from every account on it, it's also possible to
> create new ones and do the same with those.  And of course the new owners
> of these servers have access to everything stored on or transiting them,
> which provides all kinds of resources that are useful in figuring out
> who to target, how best to target them, etc.
>
> And yet once again, if you want to tell me that something serious is
> going to happen WRT email forgery: *you have to fix this too*.  First.
>
> Yeah, I'll wait for this too.  Sure I will.
>
>
> Summary of (C) (D) (E):
> ------------------------
>
> So whether (C) email accounts have been compromised via bot'd systems
> or (D) just the email accounts themselves have been taken over or
> (E) entire email servers have been comprommised, outbound messages from
> all of those will pass any anti-forgery check anybody feels like running.
> This problem is already well past "enormous" and will continue to
> get worse, as various ill-conceived bandaids (e.g. "let's get rid of
> passwords and use passkeys", a breathtakingly stupid and cruel idea)
> are haphazardly slapped on the problem because nobody wants to spend
> the massive resources it would take to solve it properly -- one system,
> one account, one server at a time.  Nobody wants to do the hard boring
> work that might actually succeed in the real world *without* creating
> more problems than it claims to solve.
>
> Until then, we're living in a world where "one billion compromised email
> accounts" is very likely a serious underestimate.  And there will be more
> tomorrow -- doubly so because attackers know everything I just said.
> They recognize that the ill-advised rollout of anti-forgery tech that
> doesn't actually solve the forgery problem has created a great opportunity
> for them...and they're all over it.
>
>
> Conclusions?  Well, maybe. ;)
> -----------------------------
>
> As I've said before, if you want to solve email forgery for a real-world
> value of "solve" at the scale of the Internet, then you have to solve
> the underlying security problems first.
>
> ALL OF THEM.
>
> And nobody's interested in doing that, because "doing nothing" and
> "pretending those problems don't exist" is not only far easier, it's
> far more profitable.  Plus it's much more fun to invent technology
> and promote it and deploy it and declare "Mission Accomplished" and
> take credit for solving the problem than it is to do the hard work
> of actually solving the problem.
>
> So what's really transpired here, as a consequence of this headlong rush
> to "solve" email forgery, is that all these elaborate mechanisms have
> been deployed on top of an ecosystem that's broken all the way down.
>
> They're gold leaf on a rotting garbage pile.
>
> They don't work because they can't work.  They've make-believe -- and
> unfortunately *users are being trained to believe in them*, which is
> going to lead to very bad real-world outcomes.
>
> Note well: none of this commentary has anything to do with the actual
> mechanics of SPF or DMARC or DKIM or anything else, which is why
> I haven't discussed their specifics.  The specifics *do not matter*.
> If someone comes up with another one of these tomorrow and writes an
> RFC and releases code and tells us all about it: that's not going to
> work either -- for the same reasons.
>
> Yes, in some ideal best-case world, maybe, MAYBE, this elaborate
> anti-forgery tech might have a fighting chance.  (Although: if we were
> living in that ideal best-case world, we probably wouldn't need it.
> Do recall that we did just fine for many years without any of this,
> modulo the occasional prankster or idiot.)
>
> But this is not the ideal best-case world.  This is a world with massive
> problems that nobody wants to solve -- and as long as this writeup is,
> I've only listed some of them.  (Yeah, it's worse.  It's always worse.)
>
> ---rsk
>
> _______________________________________________
> NANOG mailing list
>
> https://lists.nanog.org/archives/list/nanog () lists nanog org/message/J4QLVGCKA6NG4WQY5WGANML2DYLODWUY/
_______________________________________________
NANOG mailing list 
https://lists.nanog.org/archives/list/nanog () lists nanog org/message/E4QDYX4Z3DSTXWM2UYFXBEJSUYIBAJE2/