On 2/1/2017 12:56 AM, Dave Warren wrote:
They publish SPF records and DKIM sign everything for competent SMTP
receivers to handle in real-time, AND they publish a HTML version for
humans, and yet someone still finds a reason to complain?
Dave,
After the initial question was raised, it took about 11 posts and almost
24 hours for someone to notice the discussion who happened to know about
the "HTML version for humans" and mention that. During those 11 posts, a
well-respected and knowledgeable person was actually defending Yahoo for
NOT having such a page, which gave the impression that such didn't
exist. (certainly, that was a head-fake that I fell for, even if such
was very innocent)
So I think there is a strong argument that the existence of this page
page isn't exactly common knowledge. Archive.org suggests that this page
has only existed for a couple of years. I've been looking for it
(occasionally) for the past 10 years - so I think all my memories of
past discussions in past years about such a page not existing - were
probably accurate. By the time this page existed, I had given up on
finding it. (not that I spend every waking hour looking for it - I think
I probably looked for it about once every year or two - for some time -
and the need for this isn't so great with other senders - because few
senders [even large ones] have such a MASSIVE amount of sending IPs that
are so particularly hard to find)
Regarding your references about such a page not being needed - all I'm
going to say is that some systems benefit from having large IP ranges
preemptively whitelisted for the sake of efficiency. There are scenarios
in certain very high volume systems where this enables the processing of
messages at order of magnitudes faster rates than if SPF and DKIM and
FCrDNS-confirmation had to be checked on every sending IP. MUCH of that
relies on the response times of 3rd party servers - which (even at
best!) is order of magnitudes slower than a local rbldnsd query - or
than an optimized binary search of an in-memory array - which is even
faster than rbldnsd or even a high-end in-memory database. Sometimes,
such 3rd party servers can "freeze up" in their responses, or rate limit
queries - or firewall such lookups for what is perceived as abuse -
causing further complications. Caching only does so much to prevent this!
That kind of need for speed is the world in which I live. At
invaluement, I'm processing dozens of spams per second - and since much
of these are ones where the "low-hanging fruit" - such as ALREADY
heavily blacklisted botnet-sent spams are ALREADY filtered out before
they get to my system - that means that the processing resources per
spam is already much higher for my system than that of a typical ISP or
hoster's natural incoming spam. (I process a higher concentration of the
more sneaky spams and the newer emitters)
With this in mind... if I deleted my IP whitelist, and had to rely on
SPF and DKIM and FCrDNS-verification for EVERY message, my queues would
back up considerably - and a lot of worthy blacklistings of IPs and
domains from new incoming spams would get considerably delayed. (again,
inevitably - at this volume - issues come up where such
queries/verification suddenly "freeze up" or get rate limited,
firewalled, etc)
And I think my need for efficiency is probably not much different than
some very large hosters and ISPs - who process mail for millions of users?
And I think we've already established that there is no possible way to
generate "on demand" and remotely efficiently the information on that
HTML page just via Yahoo's SPF records.
iow - maybe you should have a little more respect and try to be a little
less snarky in the future - when you don't necessarily know/understand
others' situation/requirements that may be a little different than your
particular situation/requirements.
--
Rob McEwen
