Re: Rule for non-DK-signed mail from yahoo
Justin Mason wrote: That should not be a problem - if the message is re-signed, and the resigner inserts his own Sender header field as it is supposed to do, outer DK and DKIM signatures will succeed and the rule will not fire yeah -- in a perfect world, maybe ;) How does one move a mountain? Stone by stone. Checking the last 12 hours of the log, I found two false positives, one was a yahoo user with a regular yahoo account, who posted directly through his home ISP's mailer (not through yahoo), but provided his yahoo From address. ... The former is pretty common, fwiw. All solutions (including SPF) seem to go in the direction that the roaming poster will need to submit his mail through the provider/service/domain whose domain name is used in his sending address. SASL and TLS are common now, most popular mail readers support it, and most MTAs. People will need to adjust, to avoid their mail being treated as second-class. Until then, giving such second-class mail small number of positive score points is not too bad - well intended mail still passes with no trouble. # give some incentive for people to start signing their mail: score DKIM_VERIFIED -1.5 score DK_VERIFIED -1.0 SpamAssassin has some merit and influence on the population, so it may just as well be setting some trends. If spamers start signing their mail, so much the better. I'd prefer not to do this without some kind of DKIM reputation service up and running, so that we don't give bonuses to spammers who sign their mails. In our experience, spammers will quickly exploit any SpamAssassin bonuses available, and this would be pretty easy. Give a little - gain a lot! If they start signing their mail with a valid signature corresponding to a sending domain, that is perfect: they get 1.5 points of a bonus, we get the assurance that whoever placed a domain name in From header field actually owns (or ownz) that domain. So how can a spammer validly sign a message: - by owning a legitimate permanent domain, such as genuine commercial bulk mailers do, not trying to hide it - good, they get 1.5 bonus points, we may blacklist them if we want; - by using a public mail service such as Yahoo - good, they get 1.5 bonus points, there may be increased interest of spammers on legitimate Yahoo accounts, Yahoo gets more pressure from other legitimate users since its reputation is at stake, they take action to minimize misuse of their accounts - we all win (except for spammers); - by stealing a private key from legitimate domain - not likely, and easy to fix by switching to a new key; - by using throw-away domains, like being discussed on this ML not that long ago. Countermeasures are already being build, RBL lists of few-days old domains. And registrats may get bored with the scheme and actually do something about it. - by using zombies sending through legitimate company's mailer with a legitimate sender address of that company - the company's reputation is at stake, and if it goes on for a long time, we may add few positive score points for such domains (RBL) or just blacklist them. Is there another way? For now I'd stick with Give a little - gain a lot! Mark
Re: Rule for non-DK-signed mail from yahoo
Mark Martinec writes: Thank you all for the feedback. FWIW, I've seen a few mails that had multiple DK signatures, apparently as the result of going through a DK signed mailing list when the original message had also been signed. yeah, I think if the list re-signs the message, that's ok, because it then doesn't matter if the internal signature fails (there being no need to check that). That may be a DKIM interpretation btw. That should not be a problem - if the message is re-signed, and the resigner inserts his own Sender header field as it is supposed to do, outer DK and DKIM signatures will succeed and the rule will not fire thanks to !DK_VERIFIED. yeah -- in a perfect world, maybe ;) DK verification may fail if the mail goes through mailing lists. ah. Here's another one that just occurred to me -- (c): if you're keying off the From: header, watch out for mailing list traffic that appends a footer to the body. That will cause a verification failure, and fire the rule. Bear in mind the list/forwarding issue I also noted; most list installs don't re-sign mails, so an additional exemption for messages that contain List-Id might be worthwhile. mass-check should give a good idea, anyway. Some (most?) mailing lists are indeed problematic, so the rule should not fire if it looks like the message was passed through a mailing list. I'm glad that this ML seems to do pretty well in avoiding breaking of original signatures. (and the postfix-users ML for DKIM, but not for DK, because it appends a Sender:) This is what I have now: header __L_ML0Precedence=~ /\b(list|bulk)\b/i header __L_ML1exists:List-Id header __L_ML2exists:List-Post header __L_ML3exists:Mailing-List header __L_HAS_SENDER exists:Sender meta __L_VIA_ML __L_ML0 || __L_ML1 || __L_ML2 || __L_ML3 || __L_HAS_SENDER header __L_FROM_YAHOO From:addr =~ /@yahoo\.com$/i header __L_FROM_GMAIL From:addr =~ /@gmail\.com$/i meta UNVERIFIED_YAHOO __L_FROM_YAHOO !__L_VIA_ML !DK_VERIFIED priority UNVERIFIED_YAHOO 500 scoreUNVERIFIED_YAHOO 2.5 meta UNVERIFIED_GMAIL __L_FROM_GMAIL !__L_VIA_ML !DK_VERIFIED priority UNVERIFIED_GMAIL 500 scoreUNVERIFIED_GMAIL 2.5 Checking the last 12 hours of the log, I found two false positives, one was a yahoo user with a regular yahoo account, who posted directly through his home ISP's mailer (not through yahoo), but provided his yahoo From address. The other was a forwarding through a gmail account, which did not (re)sign the message. Seems pretty good - and 2.5 score points is not too bad for an otherwise healthy message. The former is pretty common, fwiw. OK -- someone who would know, tells me: Pretty much all user-generated mail From: yahoo-owned domains is [now] signed, but that's not intended as a statement of spam/non-spam. Certainly not, but either way, we can be certain that the massage came from the signing domain it claims to be, which makes it easier to apply other rules like blacklisting etc, if mail happens to be spam. Which is why I'd suggest something like: # give some incentive for people to start signing their mail: score DKIM_VERIFIED -1.5 score DK_VERIFIED -1.0 SpamAssassin has some merit and influence on the population, so it may just as well be setting some trends. If spamers start signing their mail, so much the better. I'd prefer not to do this without some kind of DKIM reputation service up and running, so that we don't give bonuses to spammers who sign their mails. In our experience, spammers will quickly exploit any SpamAssassin bonuses available, and this would be pretty easy. --j.
Re: Rule for non-DK-signed mail from yahoo
Justin Mason wrote: I'd prefer not to do this without some kind of DKIM reputation service up and running, so that we don't give bonuses to spammers who sign their mails. In our experience, spammers will quickly exploit any SpamAssassin bonuses available, and this would be pretty easy. --j. So what is involved in establishing one?
Re: Rule for non-DK-signed mail from yahoo
Mark Martinec writes: Thanks Justin and Daryl. (a) Is From:addr rather than EnvelopeFrom:addr the right header to use? I'd say yes. DK signs the message, not the envelope. I'm pretty sure the current milters look for a From: header to decide on what selector/etc to use. Right, DK (as well as DKIM) uses addresses in the header, not envelope. DK would choose Sender if it exists, otherwise a From, to obtain the signer domain. DKIM is more sophisticated (could use Resent-From,...), but basically, for direct mail the From header field is the most important one. (b) are Y! signing all mail? I would have assumed some systems are not yet using DK. This is a key question here. I'd hope yes, since Yahoo was the leading proponent in establishing this technology (now aiming for DKIM). Although their policy record still says 'testing' and 'signs SOME mail': $ host -t txt _domainkey.yahoo.com t=y\; o=~\; n=http://antispam.yahoo.com/domainkeys I think they are just conservative, trying to avoid some broken recipient's mailer from rejecting their genuine mail, or to avoid problems with mailing lists invalidating signatures when their user posts there. In 3.1.x, you have to set priority manually, unfortunately, to be higher than both of the subrules. in 3.2.x, it'll do that automatically for you. Thanks for the info. Personally I'd cut the score in half. Ok, perhaps. Slow DNS could cause FPs -- I've seen it happen on mail from rogers.com which Y! runs. Interesting. Further experience is welcome. The _domainkey.yahoo.com TXT policy record has TTL set to two hours, and one of their public keys (s1024._domainkey.yahoo.com) has a lifetime of 24 hours - so a local caching DNS resolver is likely to retrieve the policy from its cache, or from any one of the 5 registered Yahoo name servers. As far as I can tell, it is a global Yahoo thing, not something pertaining to one or another of their servers. What about gmail.com? They seem to be signing their mail too (see: host -t txt beta._domainkey.gmail.com) but also avoid full commitment in their policy (no policy = default policy). Any experience there? ah. Here's another one that just occurred to me -- (c): if you're keying off the From: header, watch out for mailing list traffic that appends a footer to the body. That will cause a verification failure, and fire the rule. in other words: - sender @ yahoo.com sends mail to mailmanlist @ somelist.com; - mailmanlist @ somelist.com appends the mailman footer to the body text/plain part; - recipient gets message, reads From addr, verifies DK sig, which now fails. --j.
Re: Rule for non-DK-signed mail from yahoo
in other words: - sender @ yahoo.com sends mail to mailmanlist @ somelist.com; - mailmanlist @ somelist.com appends the mailman footer to the body text/plain part; - recipient gets message, reads From addr, verifies DK sig, which now fails. FWIW, I've seen a few mails that had multiple DK signatures, apparently as the result of going through a DK signed mailing list when the original message had also been signed. Loren
Re: Rule for non-DK-signed mail from yahoo
Loren Wilton writes: in other words: - sender @ yahoo.com sends mail to mailmanlist @ somelist.com; - mailmanlist @ somelist.com appends the mailman footer to the body text/plain part; - recipient gets message, reads From addr, verifies DK sig, which now fails. FWIW, I've seen a few mails that had multiple DK signatures, apparently as the result of going through a DK signed mailing list when the original message had also been signed. yeah, I think if the list re-signs the message, that's ok, because it then doesn't matter if the internal signature fails (there being no need to check that). That may be a DKIM interpretation btw. --j.
Re: Rule for non-DK-signed mail from yahoo
Mark Martinec writes: Thanks Justin and Daryl. (a) Is From:addr rather than EnvelopeFrom:addr the right header to use? I'd say yes. DK signs the message, not the envelope. I'm pretty sure the current milters look for a From: header to decide on what selector/etc to use. Right, DK (as well as DKIM) uses addresses in the header, not envelope. DK would choose Sender if it exists, otherwise a From, to obtain the signer domain. DKIM is more sophisticated (could use Resent-From,...), but basically, for direct mail the From header field is the most important one. (b) are Y! signing all mail? I would have assumed some systems are not yet using DK. This is a key question here. I'd hope yes, since Yahoo was the leading proponent in establishing this technology (now aiming for DKIM). Although their policy record still says 'testing' and 'signs SOME mail': $ host -t txt _domainkey.yahoo.com t=y\; o=~\; n=http://antispam.yahoo.com/domainkeys I think they are just conservative, trying to avoid some broken recipient's mailer from rejecting their genuine mail, or to avoid problems with mailing lists invalidating signatures when their user posts there. OK -- someone who would know, tells me: Pretty much all user-generated mail From: yahoo-owned domains is [now] signed, but that's not intended as a statement of spam/non-spam. Bear in mind the list/forwarding issue I also noted; most list installs don't re-sign mails, so an additional exemption for messages that contain List-Id might be worthwhile. mass-check should give a good idea, anyway. Dunno about gmail, yet. --j. In 3.1.x, you have to set priority manually, unfortunately, to be higher than both of the subrules. in 3.2.x, it'll do that automatically for you. Thanks for the info. Personally I'd cut the score in half. Ok, perhaps. Slow DNS could cause FPs -- I've seen it happen on mail from rogers.com which Y! runs. Interesting. Further experience is welcome. The _domainkey.yahoo.com TXT policy record has TTL set to two hours, and one of their public keys (s1024._domainkey.yahoo.com) has a lifetime of 24 hours - so a local caching DNS resolver is likely to retrieve the policy from its cache, or from any one of the 5 registered Yahoo name servers. As far as I can tell, it is a global Yahoo thing, not something pertaining to one or another of their servers. What about gmail.com? They seem to be signing their mail too (see: host -t txt beta._domainkey.gmail.com) but also avoid full commitment in their policy (no policy = default policy). Any experience there? Mark
Re: Rule for non-DK-signed mail from yahoo
Thank you all for the feedback. FWIW, I've seen a few mails that had multiple DK signatures, apparently as the result of going through a DK signed mailing list when the original message had also been signed. yeah, I think if the list re-signs the message, that's ok, because it then doesn't matter if the internal signature fails (there being no need to check that). That may be a DKIM interpretation btw. That should not be a problem - if the message is re-signed, and the resigner inserts his own Sender header field as it is supposed to do, outer DK and DKIM signatures will succeed and the rule will not fire thanks to !DK_VERIFIED. DK verification may fail if the mail goes through mailing lists. ah. Here's another one that just occurred to me -- (c): if you're keying off the From: header, watch out for mailing list traffic that appends a footer to the body. That will cause a verification failure, and fire the rule. Bear in mind the list/forwarding issue I also noted; most list installs don't re-sign mails, so an additional exemption for messages that contain List-Id might be worthwhile. mass-check should give a good idea, anyway. Some (most?) mailing lists are indeed problematic, so the rule should not fire if it looks like the message was passed through a mailing list. I'm glad that this ML seems to do pretty well in avoiding breaking of original signatures. (and the postfix-users ML for DKIM, but not for DK, because it appends a Sender:) This is what I have now: header __L_ML0Precedence=~ /\b(list|bulk)\b/i header __L_ML1exists:List-Id header __L_ML2exists:List-Post header __L_ML3exists:Mailing-List header __L_HAS_SENDER exists:Sender meta __L_VIA_ML __L_ML0 || __L_ML1 || __L_ML2 || __L_ML3 || __L_HAS_SENDER header __L_FROM_YAHOO From:addr =~ /@yahoo\.com$/i header __L_FROM_GMAIL From:addr =~ /@gmail\.com$/i meta UNVERIFIED_YAHOO __L_FROM_YAHOO !__L_VIA_ML !DK_VERIFIED priority UNVERIFIED_YAHOO 500 scoreUNVERIFIED_YAHOO 2.5 meta UNVERIFIED_GMAIL __L_FROM_GMAIL !__L_VIA_ML !DK_VERIFIED priority UNVERIFIED_GMAIL 500 scoreUNVERIFIED_GMAIL 2.5 Checking the last 12 hours of the log, I found two false positives, one was a yahoo user with a regular yahoo account, who posted directly through his home ISP's mailer (not through yahoo), but provided his yahoo From address. The other was a forwarding through a gmail account, which did not (re)sign the message. Seems pretty good - and 2.5 score points is not too bad for an otherwise healthy message. OK -- someone who would know, tells me: Pretty much all user-generated mail From: yahoo-owned domains is [now] signed, but that's not intended as a statement of spam/non-spam. Certainly not, but either way, we can be certain that the massage came from the signing domain it claims to be, which makes it easier to apply other rules like blacklisting etc, if mail happens to be spam. Which is why I'd suggest something like: # give some incentive for people to start signing their mail: score DKIM_VERIFIED -1.5 score DK_VERIFIED -1.0 SpamAssassin has some merit and influence on the population, so it may just as well be setting some trends. If spamers start signing their mail, so much the better. Mark
RE: Rule for non-DK-signed mail from yahoo
-Original Message- From: Mark Martinec [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 15, 2006 7:38 PM To: users@spamassassin.apache.org Subject: Re: Rule for non-DK-signed mail from yahoo This is what I have now: I get this on a lint with SA 3.13: Does it need escape in front of '@'? [38743] warn: Possible unintended interpolation of @gmail in string at /usr/local/etc/mail/spamassassin/99_dk_signs.cf, rule __L_FROM_GMAIL, line 1. [38743] warn: Possible unintended interpolation of @yahoo in string at /usr/local/etc/mail/spamassassin/99_dk_signs.cf, rule __L_FROM_YAHOO, line 1. [38743] warn: rules: failed to run header tests, skipping some: Global symbol @gmail requires explicit package name at /usr/local/etc/mail/spamassassin/99_dk_signs.cf, rule __L_FROM_GMAIL, line 1. [38743] warn: Global symbol @yahoo requires explicit package name at /usr/local/etc/mail/spamassassin/99_dk_signs.cf, rule __L_FROM_YAHOO, line 1. [38743] warn: lint: 2 issues detected, please rerun with debug enabled for more information
Re: Rule for non-DK-signed mail from yahoo
On Wed, 2006-08-16 at 01:37 +0200, Mark Martinec wrote: header __L_FROM_YAHOO From:addr =~ /@yahoo\.com$/i header __L_FROM_GMAIL From:addr =~ /@gmail\.com$/i You should escape the @ signs in the expression: /[EMAIL PROTECTED]/i Chris
Re: Rule for non-DK-signed mail from yahoo
On Wednesday August 16 2006 01:47, Chris Stone wrote: On Wed, 2006-08-16 at 01:37 +0200, Mark Martinec wrote: header __L_FROM_YAHOO From:addr =~ /@yahoo\.com$/i header __L_FROM_GMAIL From:addr =~ /@gmail\.com$/i You should escape the @ signs in the expression: /[EMAIL PROTECTED]/i Yes, just found out the minute after I posted, last minute typo.
Rule for non-DK-signed mail from yahoo
Having received a couple of messages faking to be from yahoo, despite FORGED_YAHOO_RCVD and few other rules firing, the final score was not high enough. Since Yahoo! is signing their outgoing mail with DomainKeys, I came up with: header __L_FROM_YAHOOFrom:addr =~ /[EMAIL PROTECTED]/i meta UNVERIFIED_YAHOO __L_FROM_YAHOO !DK_VERIFIED priority UNVERIFIED_YAHOO 500 scoreUNVERIFIED_YAHOO 5.0 which seems to do its job. I had to experiment with priority - are there any guidelines fo this? Is this a way to go? - any obvious improvements? Mark
Re: Rule for non-DK-signed mail from yahoo
Mark Martinec writes: Having received a couple of messages faking to be from yahoo, despite FORGED_YAHOO_RCVD and few other rules firing, the final score was not high enough. Since Yahoo! is signing their outgoing mail with DomainKeys, I came up with: header __L_FROM_YAHOOFrom:addr =~ /[EMAIL PROTECTED]/i meta UNVERIFIED_YAHOO __L_FROM_YAHOO !DK_VERIFIED priority UNVERIFIED_YAHOO 500 scoreUNVERIFIED_YAHOO 5.0 which seems to do its job. I had to experiment with priority - are there any guidelines fo this? Is this a way to go? - any obvious improvements? makes sense to me, although -- (a) Is From:addr rather than EnvelopeFrom:addr the right header to use? (b) are Y! signing all mail? I would have assumed some systems are not yet using DK. In 3.1.x, you have to set priority manually, unfortunately, to be higher than both of the subrules. in 3.2.x, it'll do that automatically for you. --j.
Re: Rule for non-DK-signed mail from yahoo
On 8/14/2006 2:23 PM, Justin Mason wrote: Mark Martinec writes: Having received a couple of messages faking to be from yahoo, despite FORGED_YAHOO_RCVD and few other rules firing, the final score was not high enough. Since Yahoo! is signing their outgoing mail with DomainKeys, I came up with: header __L_FROM_YAHOOFrom:addr =~ /[EMAIL PROTECTED]/i meta UNVERIFIED_YAHOO __L_FROM_YAHOO !DK_VERIFIED priority UNVERIFIED_YAHOO 500 scoreUNVERIFIED_YAHOO 5.0 which seems to do its job. I had to experiment with priority - are there any guidelines fo this? Is this a way to go? - any obvious improvements? Personally I'd cut the score in half. Slow DNS could cause FPs -- I've seen it happen on mail from rogers.com which Y! runs. makes sense to me, although -- (a) Is From:addr rather than EnvelopeFrom:addr the right header to use? I'd say yes. DK signs the message, not the envelope. I'm pretty sure the current milters look for a From: header to decide on what selector/etc to use. Daryl
Re: Rule for non-DK-signed mail from yahoo
Thanks Justin and Daryl. (a) Is From:addr rather than EnvelopeFrom:addr the right header to use? I'd say yes. DK signs the message, not the envelope. I'm pretty sure the current milters look for a From: header to decide on what selector/etc to use. Right, DK (as well as DKIM) uses addresses in the header, not envelope. DK would choose Sender if it exists, otherwise a From, to obtain the signer domain. DKIM is more sophisticated (could use Resent-From,...), but basically, for direct mail the From header field is the most important one. (b) are Y! signing all mail? I would have assumed some systems are not yet using DK. This is a key question here. I'd hope yes, since Yahoo was the leading proponent in establishing this technology (now aiming for DKIM). Although their policy record still says 'testing' and 'signs SOME mail': $ host -t txt _domainkey.yahoo.com t=y\; o=~\; n=http://antispam.yahoo.com/domainkeys I think they are just conservative, trying to avoid some broken recipient's mailer from rejecting their genuine mail, or to avoid problems with mailing lists invalidating signatures when their user posts there. In 3.1.x, you have to set priority manually, unfortunately, to be higher than both of the subrules. in 3.2.x, it'll do that automatically for you. Thanks for the info. Personally I'd cut the score in half. Ok, perhaps. Slow DNS could cause FPs -- I've seen it happen on mail from rogers.com which Y! runs. Interesting. Further experience is welcome. The _domainkey.yahoo.com TXT policy record has TTL set to two hours, and one of their public keys (s1024._domainkey.yahoo.com) has a lifetime of 24 hours - so a local caching DNS resolver is likely to retrieve the policy from its cache, or from any one of the 5 registered Yahoo name servers. As far as I can tell, it is a global Yahoo thing, not something pertaining to one or another of their servers. What about gmail.com? They seem to be signing their mail too (see: host -t txt beta._domainkey.gmail.com) but also avoid full commitment in their policy (no policy = default policy). Any experience there? Mark
Re: Rule for non-DK-signed mail from yahoo
At 11:03 14-08-2006, Mark Martinec wrote: Having received a couple of messages faking to be from yahoo, despite FORGED_YAHOO_RCVD and few other rules firing, the final score was not high enough. Since Yahoo! is signing their outgoing mail with DomainKeys, I came up with: header __L_FROM_YAHOOFrom:addr =~ /[EMAIL PROTECTED]/i meta UNVERIFIED_YAHOO __L_FROM_YAHOO !DK_VERIFIED priority UNVERIFIED_YAHOO 500 scoreUNVERIFIED_YAHOO 5.0 which seems to do its job. The score is too high. Some From: yahoo.com mail may not be DK signed. DK verification may fail if the mail goes through mailing lists. Regards, -sm
