On 9/22/2018 5:55 PM, Michael Grant wrote:
The URIBL plugin looks for URLs in the subject and message body.
Is there some way to coax it to look in the other headers as well, for
example the From: Reply-to: or the Received headers?
Michael,
This reminds me of that saying, "just because you can, doesn't mean you
should" - and along those lines, I have some interesting observations
about this:
(1) some URI/domain blacklists are ONLY intended for blocking on the
domain or IP that is at the base of clickable links inside the body of
the message. These will often have a small (but critical) uptick in
false positives if used to check against domains found in the SMTP
envelope (FROM, PTR record, HELO), with typically a very small increase
in additional spams blocked. SO BE CAREFUL -AND- if you use a URI/domain
blacklist in that way and they don't prescribe that type of usage, don't
complain to them or anyone about any resulting false positives - because
it would then be your MIS-usage off their list that caused those false
positive.
(2) Even so, there really are SOME series of spams that can be safely
blocked based on domains that are in the SMTP envelope (FROM, PTR
record, HELO). In some cases, these are snowshoe spammers who are
sending from their own spammy domain - but where this domain is NOT
found in a clickable link inside the body of the message - they really
are trying to get the user to hit "reply". So there really is a purpose
for this, even if it is is a very small percentage of all spam
(3) However, even with that being a very small percentage of all - LARGE
mail hosters LOVE THIS IDEA? Why? Because it is SO EFFICIENT for them to
be able to block MORE spam based on information in the SMTP envelope -
BEFORE the "data" command. Sometimes, this helps block messages where
the domain was in a clickable link inside the body of the message - but
it is still MORE EFFICIENT to block that based on the domain also being
in the SMTP envelop.
(4) ABOUT THOSE FALSE POSITIVES: One of the main reasons that this is so
risky for False Positives... is because two things are epidemic in
recent years: (a) web site gets hijacked by criminal spammer, who
installed pages there that redirect to pornographic dating sites or pill
spam websites -AND/OR- (b) email account on the mail server gets
credentials hijacked and starts spewing spam. HERE IS THE PROBLEM:
*MOST* of the time, one or the other happens, (a or b) but not both.
Therefore, if (a) happens, they are sure to land on traditional URI
blacklists like SURBL, URIBL, and ivmURI. But this company - whose web
site was hacked - might not have a single spam coming from their mail
server. Yet, if you do the SMTP envelope checking against such URI
blacklists - you're going to have a substantially higher amount of false
positives due to blocking ALL of those emails that merely have a "FROM"
address ending in that domain name - even though NONE of THOSE messages
are spam.
(5) So which lists *DO* support blocking on the SMTP envelope? Spamhaus'
DBL list is designed for this. However, invaluement's ivmURI list is NOT
supposed to be used in this manner. SURBL and URIBL were originally
designed to not be used in this way - but that might have changed in
recent years? I recommend checking on that. In the meantime, I recommend
*ONLY* using Spamhaus' DBL list in this way. (possibly SURBL or URIBL
too? - but double check on that!)
(6) QUESTION: So why would a list not support both blocking methods? For
example, why wouldn't ivmURI support this method?
ANSWER: What Spamhaus did with DBL, while interesting, put them at a
strategic disadvantage, and there isn't a thing they can do about that
without making fundamental changes to their strategy. Recall that false
positive scenario mentioned earlier, where a hacked web site causing a
URI-list blacklisting can lead to substantially more false positives due
to only hitting on legit mails when blocking based on this domain being
in the SMTP envelope? Well.. the OPPOSITE situation ALSO causes more
false positives. When their email system has a hijacked email account,
but their web site was NOT hacked - then domain blacklists that
prescribe BOTH blocking methods and blacklist that domain... are going
to then start blocking ALL messages that have that domain as a hyperlink
inside the body of the message, even if THOSE messages are legit. This
will then cause a substantial number of false positives that were not
part of those hijacked outbound messages. So this works both ways. The
problem with such domain blacklists that prescribe both uses... is that
they either have to settle for (a) more false positives -OR- (b) more
false negatives. In other words, the higher collateral damage potential
means that there is going to be more collateral damage when they "take
the bait" and blacklist the domain -OR- their desire to limit false
positives will cause them to defer on the listing - even though it would
have been an excellent and justified ratio of spam-to-ham blocked, with
little collateral damage if the mail systems using that list could have
ONLY blocked using one method or the other, NOT both! DBL likely errs on
the side of less collateral damage - so it is should be safe to use DBL
for blocking based on both methods, as they prescribe, especially
considering Spamhaus' reputation for extremely low false positives.
Then, other URI lists can pick up the slack on the occasional False
Negatives.
(7) Given this information, at invaluement, we have solved this problem
by creating a new domain blacklist ("ivmSED") that is independent of
ivmURI, where ivmSED is a domain blacklist used ONLY for blocking based
on the domains found on the SMTP envelope (FROM, PTR record, HELO) - and
where ivmSED NOT be used for blocking domains in clickable links in the
body of the message, since that is the job of our ivmURI list. That way,
ivmSED and ivmURI are independent, and we then have the flexibility to
block a domain using either method independently, or both together, for
the approach that most surgically targets the spam, keeps collateral
damage to a minimum, and without compromises that lead to more false
negatives. ivmSED has just recently entering beta testing. (SED =
"Sender's Envelope Domain").
--
Rob McEwen
https://www.invaluement.com
