On Wed, June 19, 2013 2:33 pm, Axb wrote: > imo, it makes little sense to write rules to catch these hashbusters. As
If the rule is sufficiently broad, it will catch them. If the rule is so strict that it catches only one trailing slash or something, then yes, it makes little sense... but I think it should be possible to write the rule to be sufficiently generic. I'm hoping John is trying to be as generic as possible (while obviously minimizing FPs). Basically, look for long strings of stuff that cannot possibly be a valid HTML or CSS tag... if it's there, consider it gibberish and spammy. There are known regexps for valid HTML/CSS markup; the rule could, in principle, simply match on the negation of those regexps, with sufficient repetition. (This is the same reason why I think we need an HTML comment gibberish rule, and how it could be implemented.) > I'd suggest you disable MailScanner's remote img munging - this is so > 2004... (MUAS block remote images anyway) Mail clients only block remote images if they are set to do so. While this may be the default setting on most clients, it's not the default on all, and it can be overridden by the user (globally or on a per-message basis). Web bugs embedded in an email server only one purpose: to verify that an email has been read. For legitimate emails, they're basically innocuous; for spam, they are potentially harmful since they verify the spam recipient address as valid. I don't want _ANY_ of my users interacting with web bugs, whether because they deliberately turned on the "view remote images" feature of their client, either globally or even just for single message (but they most probably don't understand that this exposes web bugs), or because that feature was somehow enabled by default and they don't know enough to turn it off. Either way, I don't want the web bugs followed, and hence I prefer to retain this (perhaps outdated but IMHO still useful) feature of MailScanner. > the image URL may contain a listed domain and you'll miss it. You're right that SA may miss it, but in my experience, the spam body typically contains that same domain in (often many) other links or image tags (not web bugs, meaning MailScanner won't munge them), so SA will usually pick it up anyway. Perhaps SA should include a module/plugin to "unmunge" MailScanner munging? Has anyone written one, or if not, would anyone like to? ;-) (Since MailScanner is open-source perl, I imagine it should be relatively straightforward to find the munging code, write the reverse of it, and make that an SA plugin... I'm not sufficiently experienced to do it at the moment, but maybe someone else is interested.) > As this is applied to ham as well as spam, your bayes will learn > mailscanner.tv as spam AND ham making it harder to be effective. In other words, the munging won't have any effect on the Bayes DB since it's applied to both ham and spam. So, I don't quite see the problem. If I remove munging, it has no effect on spam or ham... if I retain it, it has basically no effect on spam or ham. So, Bayes will pretty much just ignore that token. But, per above, the same domain is generally mentioned elsewhere in the message, so the appropriate token should still get picked up. As above, I prefer to retain this feature to prevent any interaction with web bugs, since mail clients CAN load remote images (on purpose or not). > Are you using RAZOR? if not, it may be time to deploy. Yes, I am using both Razor and Pyzor. Both of them are getting positive hits on a lot of received spam (Razor more often than Pyzor, but both do hit). Thanks. --- Amir