On 03/11/10 21:54, Stan Hoeppner wrote:
Ned Slider put forth on 11/3/2010 3:11 PM:
Stan, and others who are using this file - have any of you looked at the
overlap with greylisting? I would imaging that the vast majority of
clients with dynamic/generic rDNS would be spambots and as such I would
expect greylisting to block the vast majority anyway, and without the
risk of FPs. IOW what percentage of connection attempts from clients
with dynamic/generic rDNS will retry? Of course the benefits of growing
such a list now would become immediately apparent the day spambots learn
to retry to overcome greylisting.
Hmm. The CBL still exists. The PBL and other "dynamic" dnsbls still
exist. I've guess they've not heard of this master of zombie killers
called greylisting. ;)
The performance impact of greylisting is substantially higher than an
access table lookup, yes, even the caching ones such as postgrey. You
also have the retry interval finger tapping with greylisting, waiting
for order confirmation, list subscription, airline reservation, etc.
Greylisting is simply not an option at some organizations due to
management policy. Greylisting is not a panacea.
Yes, I take your point, greylisting is not for everyone and can in
itself cause issues.
If this expression file is to evolve, some of the first additions will
likely be patterns matching snowshoe farms and other spam sources
different from the generic broadband type hosts targeted by the current
expressions.
Personally, I'd be in favour of splitting things like snowshoe into a
separate file rather than having one file for everything. WRT snowshoe,
I've had more success listing by IP (cidr table) rather than trying to
list/match domains but I do block by domain where blocking by IP isn't
effective - I guess a case of using the right tool for the job in hand
(for example, Spamhaus CSS vs DBL)
Regarding the FP issue you raise, I think you're approaching it from the
wrong perspective. These are regular expressions. They match rDNS
naming patterns. We're not talking about something sophisticated like
Spamhaus Zen or the CBL which exclusively use spamtraps and auto expire
listings a short while after the emission stops. For just about every
/21 consumer/small biz broadband range that any of these patterns may
match, there is likely going to be a ham server or 3 nestled in there,
maybe small biz or a kid with a Linux box and Postfix in the basement.
I understand your view, I think it's just I have a lower tolerance for
collateral damage if that's an appropriate term. Having looked for
ranges that are likely to be sending *me* ham, I could probably
whitelist or comment out a handful of major UK ISPs and I'd be good to
go. The chances of me receiving ham from a small biz or a kid with a
Linux box and Postfix in the basement in Russia or China or many Eastern
European countries is extremely slim.
The main "issue" I have is that I'm largely able to filter spam whilst
staying within the confines of the RFCs. I'm not aware of an RFC that
states the rDNS of an smtp server should be of the form mail.example.com
and not dynamic/generic in nature. Sure, we all understand that's best
practice and desirable but it's also not always the reality as you
yourself know only too well.
That's why I say "whitelist where necessary" when promoting use of this
regex set. I haven't checked the entire list, but I'm pretty sure all
the patterns match mostly residential type ranges. Some ISPs mix small
biz in with residential, which is stupid, but they do it. My
residential ISP is an example of such, as we discussed. With a method
this basic, there's no way around rejecting the ham servers without
whitelisting. If you start removing patterns due to what you call FPs,
pretty soon there may be few patterns left. If you start adding
patterns to the top of the file to specifically whitelist those ham
sources, you're now starting to duplicate the DNSWL project, with the
exception that such regex patterns will only be realized retroactively
after an FP. Such a method of weeding out the ham servers is absolutely
the opposite of scale. Any ham server within a residential type range
should be listed by its OP at dnswl anyway. Do you query dnswl Ned? If
not, I wonder how many of your FPs wouldn't be rejected if you did.
Yes, I have no problem whitelisting - as you say it's knowing who/what
to whitelist in advance so you don't lose mail. I do check DNSWL (I'm
listed there myself), but I check and score it in SpamAssassin, not from
within Postfix. Unfortunately none of the FPs I discovered were listed
on DNSWL - all but one were hosted on BTOpenWorld.com. My guess is that
those who know about rDNS PTR records but are unable to change them
might request a listing on DNSWL, and then there's the rest.
Unfortunately I suspect the vast majority have no clue what a rDNS PTR
record is.
My interest is not so much in how I whitelist and solve the issue for
myself, but rather how the issue of potential FPs could be addressed for
the wider usage of the file. As you outline above, there isn't an
obvious solution. I don't know if it would be possible to call the file
as part of a separate postfix restriction class that first checks DNSWL
before rejecting on the regex file, or even how effective that would be.
My other thought was to simply comment (or document) ranges known to
contain FPs and then the user can make a judgement call whether they
want to comment out that particular regex based on their circumstances.
Not a very elegant solution.
To date, I can't recall a single FP here due to these regexes. This is
one of the reasons I like it so well and promote it. As always, YMMV.
Indeed, lets not detract from the fact that these regexes are very
effective. You implied earlier in your reply that this wasn't a
"sophisticated" solution but I have to admit I'm surprised just how
effective they are and just how *few* FPs there are for something not
sophisticated.
I'm also mindful that we might be getting off topic for the postfix
users list?
