On 2/21/2018 1:17 AM, @lbutlr wrote:
goo.gl (and other shorteners) are used for far more than email.
That said, most my incoming email is rejected long before it get to
any sort of URI lookups based on just the transaction information,
That is to say, upwards of 90% of incoming mail is rejected before DATA.
That is besides the point. I'm discussing the more elusive spams that
often slip past spam filters and are difficult to block. I'm not very
concerned with the "low hanging fruit" that you're describing - that is
ANOTHER topic!
286 total spams blocked that had a shortner,
That's not enough to have any sort of reliable statistical data.
This is what was pulled from a corpus of 5K spams. Are you sure you
didn't get confused and think I said that my total corpus I searched
only involved 286 spams to start with?
Also, yes, different data batches from different sources can have
different idiosyncrasies - but to discount this as not reliable due to
being too small of sample size - is laughable. The server has a good
variety of dozens of different domains and thousand of different users.
These are actual "brick and mortar" businesses in the US with real
people: Schools, law firms, real estate companies, manufacturing
companies, service-oriented companies - so there is a SUBSTANTIAL
variety. At the same time, I've had a distinct uptick in complaints
about spam where the goo.gl shortner was a large factor in recent
unblocked spams complaints - where those complaints are normally few and
far between. With changes I've made in the past few days, the number of
such spams slipping past my filter was been sharply reduced. Also, over
the years, I've noticed that patterns I seen on my server, like this
one, often match patterns that others report. It defies Occam's Razor to
suggestion that I have this magical concentration of egregious goo.gl
shortner spams that is magically hitting these very diverse and
unrelated companies for whom I host mail, and not hitting elsewhere.
That is how statistics tend to work. The next time you read about a poll
that a newpaper or news show conducted, you may be shocked to learn that
this poll that gave a nationwide estimate with a small margin of error -
often only had about a thousand participants. My study isn't as
scientific because my users aren't perfectly random. But it is diverse
enough to simulate a sufficient amount of randomness so as to be an
extremely good indicator. Likewise, whenever I hear about a new spam
trend that is effecting many people, I pretty much never come up empty
when searching for examples of that hitting my servers.
187 total legit messages had a hit on at least one of hundreds of URL shortners
So the use of a shortner is a poor spam indicator. Even in your corpus, and a
negligible indicator even when specifically looking at goo.gl
You have an amazing talent for missing the point. Most often, when used
in legit mail... it was more of an afterthought - such as someone
sending a link to a Google maps reference - but yet where the sender was
NOT using the shortner for obfuscation of spammer's domains, or even for
their mail hyperlinks. But THAT is what happened when these were used in
spams, where it was used to HIDE identity by obfuscating their main
links. So there is a strong correlation right there. But you want an
"all or none" strawman to argue against, while you conveniently ignore
many points I've made. Yes - it is frustrating that it is NOT extremely
easy to differentiate the ones who use it for obfuscation from the ones
have more appropriate usages. But there are ways. This is why I had to
spend some hours making adjustments to my system - not some minutes. You
make a good point that it is unwise to outright block ALL messages due
to them having a shortner, or having the goo.gl shortner - but I'm light
years more sophisticated than that - and I haven't argued that even one
time in this thread.
Another problem with your statement is that if this loophole isn't
addressed it will only get larger. Also, many certain of these spams are
already very difficult to block - but would be very easy to block if the
spammer wasn't hiding behind the shortner. That is a real problem that
STILL exists regardless of how often shortners are used in legit mail -
and regardless of how often they are used in spams - but you don't want
to face that truth, it seems?
The BOTTOM LINE is that anything that (a) helps blacklist those senders
who are doing this obfuscation in order to facilitate sending spam -AND-
(b) motivate marketers and ESPs (& their clients) to avoid this tactic -
is a step in the right direction.
Google's shortner is DOMINATING in its spam usage, where 92% (262 of 286) of
ALL spam that contained shortners used Google.
But about 25% of goo.gl containing email is not spam, by your own numbers. So,
a very poor metric.
A poor metric if only idiots ran spam filters and DNSBLs. But,
thankfully, I'm able to separate the legit uses - from those spammers
who are trying to hide their identities in order to facilitate spamming
- a point lost on you. I've NEVER ONCE argued for creating rules that
blocked ALL messages that used shortners - but that is the "straw man"
that you're arguing against. This is also one of the very reasons that
spammers are so happy with this loophole - they know that nobody is
going to create a rule that blocks goo.gl (or even scores that high
against it). But thankfully, they'll get SOME relief as more of those
who abuse it find their IPs blacklisted at invaluement (and hopefully
other places), and rightly so! (yet without the collateral damage you
keep predicting or implying)
If we're going to continue this conversation, can you please STOP
putting words in my mouth and arguing against "straw men"? Also, I
understand your very valid concerns about collateral damage. I've
addressed that numerous times and in numerous ways, in numerous posts.
This is getting tiresome.
--
Rob McEwen
https://www.invaluement.com
