... Date: Sun, 27 Mar 2005 00:51:25 -0500 From: "Daryl C. W. O'Shea" <[EMAIL PROTECTED]> ... To: List Mail User <[EMAIL PROTECTED]> Cc: [EMAIL PROTECTED], [EMAIL PROTECTED], users@spamassassin.apache.org Subject: Re: Webmail and IP rules ...
Dayrl,
As is usually the case, I didn't give all of the details which may apply.
List Mail User wrote:
Dave,
You have a few valid points, and the rule may be misnamed with HELO at its prefix; But look at some email coming from the free services like Yahoo!, Hotmail or Gmail and you will see HTTP (as well as other protocols; Hotmail/MSN also uses both of the MS proprietary protocols "DAV" and "SMTPSVC"). I also have copies of some messages from "mindspring.com" (re. Earthlink), "gmx.com" and "mail.ru" where the initial hop was HTTP and these are "ham". In addition, I also have *lots* of saved spam from "mail.ru" where the transfer initially was HTTP.
I don't recall seeing 'with DAV' in anything but Hotmail headers. Is a product that uses this availble to the public? Is it _always_ an authenticated hop?
This I don't know, but likely can call connections at MS and ask (which unfortunately does not guarantee and answer, but makes one likely).
Exchange, or even the stripped down SMTP service in Windows Server 2000/2003 all use 'with SMTPSVC', but it's not always an authenticated hop, so it's useless as an authentication token.
The name "HELO_xxx" may be bad, but the rule is good. The problems lie with IMP and its attempts to "create/forge" an initial header. Of course, an "easy" work around is to simply "unmark" the next hop as trusted, then the message will look like any message forwarded trough a legitimate commercial "free" system or from a dynamic/dial-up user to his ISP's web interface and your "problem" will disappear. Personally, I don't even mark my "back-up" 'MX's as trusted, and have never seen a problem with this (though some extra computation and DNS lookups are performed as a result).
Not including your 'back-up' MX is a bad idea. You'll loose lots of tests, such as the tests for mail coming straight from dynamic IPs, and you'll cause SPF checks to result in an SPF fail.
Here is an important missing detail - the 'back-up' MXs not under my direct control do their own SPF and DUL checks and add headers, which I have local rules for and believe (though just two weeks ago a spammmer setup a /29 with my ISP and was using my 'back-up' as his relay! It took a quick note to the VP of engineering and the spammer was gone inside of 15 minutes (good ISP, now *very* large, but I've been dealing with them for >12 years and remember when the VP was a tech. who answered the telephone - also they don't sell and won't do the various things that they do for me for new customers). Also, the DUL rules (despite my even adding local checks) don't apply, because I rbl DULs in Postfix far before anything hits SA; In fact, while it is down from what I was getting, Postfix still refuses about half of all connection attempts for me, most due to bad rDNS. Also, I see other people reporting all kinds of numbers - I guess it depends on the type and mix of email a site gets, but I find SA >99.5% effective (i.e. only one or two spams a week get to mailboxes). And I do take the time to "hand" train bayes with spam it doesn't autolearn (but does catch), but which falls into certain categories (e.g. stock pumps and viruses) which don't generate a high enough "header" score to be autolearned. Generally, except for the need for a dedicated or at least powerful server and lots of memory (and Perl 5.8.x seems to need more memory than 5.6.x did), I find SA quite amazing - but still it is only at the third level of four levels of checks before I will deliver mail to a mail box (there is still a virus checker afterward, though SA gets almost 80% of the viruses itself). And as has been brought up slightly on the list and with a few people in private, as my confidence in SA has increased, my level of "pre-filtering" has in some cases decreased.
A quick check of plectere.com will show 4 MXs under my direct control and despite having different priorities, from outside, they really map to 3 machines which are dynamically load balanced, and which I tend to think of as a single unit (i.e. they are setup to share configs, merge logs, and generally act "almost" like a cluster would); My wording of 'back-up' was meant to be only those machines not at my site (and behind my own firewalls) - also they never pass mail to one another, only to a further internal MUA/MTA delivery machine (i.e. they "see" a different DNS than is published, with equal priorities for all of them).
Although 'unmarking' the next hop as trusted would prevent the dynamic tests from firing on IMP mail, it would wreck havoc on the tests mentioned above for all mail coming from remote networks.
Would it? It seems that the "notfirsthop" qualifier would do the right thing for those cases. i.e. with the IMP machine untrusted, it would look like any ISP's relay. It seems that it would only be a problem if you did not "believe" the authentication that IMP uses itself.
To get the result described above you'd have to untrust the machine after the IMP machine/header. If you do this mail coming from the IMP machine will have good results, but all other mail passing through the newly untrusted machine (the one after the IMP machine) will have incorrect results.
Actually... setting trusted networks manually would fix Shane's problem anyway, since the bug is (well, was) only in the inferral code.
Another thing I do (and while I sure others do also, it is not the common case), is I have separate servers for incoming and outgoing mail. So being paranoid, I don't even trust my own firewalls, who have to send the nightly logs through the entire gamut of tests (now, my outgoing server and the final MUA *do* (mostly) trust the incoming servers, but the incoming servers don't trust anybody).
I don't think it's too popular with small installations, but every even small ISP I've setup has been this way. I do the same for medium sized businesses too.
BTW. at first read, I saw "inferral" above as "infernal":)
Due to the inherent problem with detecting half of the NATed hosts, that's pretty much how I feel about it.
I can't see any possible reason why blanca.unet.brandeis.edu is a trusted host for server.home.jay.fm; But I can see many reasons why it should not be.
Also, by including the original post, you have done us all a service. I see leakage of the literal 127.0.0.1 and evidence of a misconfigured DNS or hosts file - i.e. the resolution of 127.0.0.1 to localhost.localdomain: While the hostname "localhost" is part of several standards, the ".localdomain" part is intended as an example only, is not legal and shows that there *do* exist other administrative problems in his environment.
'localhost.localdomain' is a RedHat default. Why they ever did that I don't know.
I can say I'm glad I'm not a Linux user, but if I were, I would probably choose SUSE or Debian - I spent too much time helping people fix Redhat/Fedora installs (and their marketing, with different classes of servers for paying vs. none-paying customers is a serious PITA).
Simply, if you intend to setup a web mail interface, you need to "fix" the other problems too (and probably first, before complaining about a bug).
None of his other issues are actually related to the problem or the bug that was present in the inferral code.
Yea, I actually thought this thread had died. My simple example in the post after the one quoted here seem to show that it was related entirely to "trust" (i.e. if I "trusted" the same machine(s) the rules didn't fire, otherwise they did).
Yeah, it was. I just came across it though and realized there actually was a bug present in the infernal code (but not the manually trusted_networks code), fixed the bug, then saw your post recommending pushing the trust boundary back which I don't think is a good thing to recommend to the majority of users (many of which rely on SA for DUL & SPF checks).
Daryl
