Re: Airline reservations get tagged
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hamish Marson wrote: > Loren Wilton wrote: Yeah, I know about the SPF checks... But I meant does SA currently do anything with digital signatures to verify that the sender really is the sender & apply a -ve score. >>> Other than the SPF type header checks I don't believe so. >>> Certainly not any pgp blocks or the like in the body of the >>> mail. >>> >>> This probably wouldn't be too hard to do in a plugin if someone >>> wanted to. >>> >>> Of course, like SPF, this really isn't an anti-spam sort of >>> thing. All we would know is that the spammer bothered to get >>> his own pgp key or the like. (That said, a lot of spammers are >>> stupid, so giving positive points to failed checks might be >>> useful.) > > More authentication... Anti-spoofing... Which helps when trying to > differentiate what only looks spammy... > > I went through a few airlines & travelagents... I could only find > one with an SPF record (Although I didn't do an exhaustive search, > just ones I could think of off hand). Any chance it could be added > to the SPF whitelists? (Two records, because I'm not sure which one > is used for eTicketing & disruption notices etc). > > ba.com. 86400 IN TXT "v=spf1 mx > ip4:163.166.43.0/24 -all" britishairways.com. 86400 IN TXT > "v=spf1 mx ip4:163.166.43.0/24 -all" > > (There's no digital signing on the emails AFAIK, so dkim isn't an > option yet). > Apologies... Travelocity also have an SPF record travelocity.com.900 IN TXT "v=spf1 ip4:151.193.165.14 ip4:151.193.165.154 ip4:151.193.165.224 ip4:151.193.165.236 ip4:151.193.165.237 ip4:151.193.165.238 ip4:151.193.220.17 ip4:151.193.220.19 ip4:151.193.165.210 ip4:151.193.165.211 ip4:151.193.167.5" > regards Hamish. > > -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEo8rO/3QXwQQkZYwRApxxAJ41D8u2Gl7JRtBVC57oHFGsaNT/UACfY4Uf nFDBoy7xP5FNBOHwVejDPt4= =ILYZ -END PGP SIGNATURE-
Re: Airline reservations get tagged
Hamish Marson writes: > Loren Wilton wrote: > >> Yeah, I know about the SPF checks... But I meant does SA > >> currently do anything with digital signatures to verify that the > >> sender really is the sender & apply a -ve score. > > > > Other than the SPF type header checks I don't believe so. > > Certainly not any pgp blocks or the like in the body of the mail. > > > > This probably wouldn't be too hard to do in a plugin if someone > > wanted to. > > > > Of course, like SPF, this really isn't an anti-spam sort of thing. > > All we would know is that the spammer bothered to get his own pgp > > key or the like. (That said, a lot of spammers are stupid, so > > giving positive points to failed checks might be useful.) > > More authentication... Anti-spoofing... Which helps when trying to > differentiate what only looks spammy... > > I went through a few airlines & travelagents... I could only find one > with an SPF record (Although I didn't do an exhaustive search, just > ones I could think of off hand). Any chance it could be added to the > SPF whitelists? (Two records, because I'm not sure which one is used > for eTicketing & disruption notices etc). > > ba.com. 86400 IN TXT "v=spf1 mx > ip4:163.166.43.0/24 -all" > britishairways.com. 86400 IN TXT "v=spf1 mx > ip4:163.166.43.0/24 -all" > > (There's no digital signing on the emails AFAIK, so dkim isn't an > option yet). (a) first off, check to ensure that the etickets/notices really *are* coming from the SPF-listed ranges. Many senders have outsourced this kind of function, have different depts working on the SPF record vs the eticketing systems, and some senders are -- to be honest -- quite incompetent in this respect. ;) Not that I'm saying BA are, but it's worth checking anyway... (b) also, if the etickets do *not* hit many rules, and are safely marked as nonspam, it's best not to add a whitelisting when it's not required -- since there's no guarantee the whitelist will always match those mails in future (a side-effect of the (a) problem). (c) if it does work out as a good idea, open an enhancement request at our bugzilla to ensure the request doesn't get forgotten. Feel free to attach sample mail(s), with sensitive info removed or obscured; we're more likely to add the whitelisting if we can verify that it works. --j.
Re: Airline reservations get tagged
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Loren Wilton wrote: >> Yeah, I know about the SPF checks... But I meant does SA >> currently do anything with digital signatures to verify that the >> sender really is the sender & apply a -ve score. > > Other than the SPF type header checks I don't believe so. > Certainly not any pgp blocks or the like in the body of the mail. > > This probably wouldn't be too hard to do in a plugin if someone > wanted to. > > Of course, like SPF, this really isn't an anti-spam sort of thing. > All we would know is that the spammer bothered to get his own pgp > key or the like. (That said, a lot of spammers are stupid, so > giving positive points to failed checks might be useful.) More authentication... Anti-spoofing... Which helps when trying to differentiate what only looks spammy... I went through a few airlines & travelagents... I could only find one with an SPF record (Although I didn't do an exhaustive search, just ones I could think of off hand). Any chance it could be added to the SPF whitelists? (Two records, because I'm not sure which one is used for eTicketing & disruption notices etc). ba.com. 86400 IN TXT "v=spf1 mx ip4:163.166.43.0/24 -all" britishairways.com. 86400 IN TXT "v=spf1 mx ip4:163.166.43.0/24 -all" (There's no digital signing on the emails AFAIK, so dkim isn't an option yet). regards Hamish. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEo8Ls/3QXwQQkZYwRAofWAJ0RW4X6X5pgUkddMwTdAQKhQ4haKwCgqQ8l HiOkaY+bjupCIiAsGAXA4ok= =8XKs -END PGP SIGNATURE-
Re: Airline reservations get tagged
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Justin Mason wrote: > Hamish Marson writes: >> Justin Mason wrote: >>> Hamish writes: On Wednesday 28 June 2006 08:48, Ralf Hildebrandt wrote: > * [EMAIL PROTECTED] <[EMAIL PROTECTED]>: >> Given that airline messages are important, are related to >> meney, and recipients dont want to get forged ones, it >> would be a great idea to start a campaign with airlines / >> travel agents to use some sort of proof of origin (spf, >> digital signature, whatnot) Recipients could then apply >> whitelists > Amen to that! Does SA do anything with digital signatures to deduct scores? If it's worthwhile, I'm game to play. >>> It allows us (and third parties, and individual site admins) to >>> reliably whitelist sources safely. See >>> 'rules/60_whitelist_spf.cf', e.g.: >>> >>> def_whitelist_from_spf [EMAIL PROTECTED] >> Yeah, I know about the SPF checks... But I meant does SA >> currently do anything with digital signatures to verify that the >> sender really is the sender & apply a -ve score. > > hmm. I thought 'def_whitelist_from_spf' also checked DK and DKIM > sigs, but it appears not :( > > It appears that 'def_whitelist_from_dkim' is in place for this > purpose, instead. -- at least that's the plan... no orgs are yet > listed in 'rules/60_whitelist_dkim.cf' though. > > But yes, given a DKIM sig, and a 'def_whitelist_from_dkim' line for > that sender, it'll apply a negative bonus. > Ah ha! Great. Thanks for that. Hamish. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEo75x/3QXwQQkZYwRAlIGAKDY23BHNE7JF7zc44xLxLap9P5voACguC/E u3pcbI4er9+rnwyPDXAwl/c= =ChIz -END PGP SIGNATURE-
Re: Airline reservations get tagged
Hamish Marson writes: > Justin Mason wrote: > > Hamish writes: > >> On Wednesday 28 June 2006 08:48, Ralf Hildebrandt wrote: > >>> * [EMAIL PROTECTED] <[EMAIL PROTECTED]>: > Given that airline messages are important, are related to > meney, and recipients dont want to get forged ones, it would > be a great idea to start a campaign with airlines / travel > agents to use some sort of proof of origin (spf, digital > signature, whatnot) Recipients could then apply whitelists > >>> Amen to that! > >> Does SA do anything with digital signatures to deduct scores? If > >> it's worthwhile, I'm game to play. > > > > It allows us (and third parties, and individual site admins) to > > reliably whitelist sources safely. See > > 'rules/60_whitelist_spf.cf', e.g.: > > > > def_whitelist_from_spf [EMAIL PROTECTED] > > Yeah, I know about the SPF checks... But I meant does SA currently do > anything with digital signatures to verify that the sender really is > the sender & apply a -ve score. hmm. I thought 'def_whitelist_from_spf' also checked DK and DKIM sigs, but it appears not :( It appears that 'def_whitelist_from_dkim' is in place for this purpose, instead. -- at least that's the plan... no orgs are yet listed in 'rules/60_whitelist_dkim.cf' though. But yes, given a DKIM sig, and a 'def_whitelist_from_dkim' line for that sender, it'll apply a negative bonus. --j.
Re: Airline reservations get tagged
> Yeah, I know about the SPF checks... But I meant does SA currently do > anything with digital signatures to verify that the sender really is > the sender & apply a -ve score. Other than the SPF type header checks I don't believe so. Certainly not any pgp blocks or the like in the body of the mail. This probably wouldn't be too hard to do in a plugin if someone wanted to. Of course, like SPF, this really isn't an anti-spam sort of thing. All we would know is that the spammer bothered to get his own pgp key or the like. (That said, a lot of spammers are stupid, so giving positive points to failed checks might be useful.) Loren
Re: Airline reservations get tagged
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Justin Mason wrote: > Hamish writes: >> On Wednesday 28 June 2006 08:48, Ralf Hildebrandt wrote: >>> * [EMAIL PROTECTED] <[EMAIL PROTECTED]>: Given that airline messages are important, are related to meney, and recipients dont want to get forged ones, it would be a great idea to start a campaign with airlines / travel agents to use some sort of proof of origin (spf, digital signature, whatnot) Recipients could then apply whitelists >>> Amen to that! >> Does SA do anything with digital signatures to deduct scores? If >> it's worthwhile, I'm game to play. > > It allows us (and third parties, and individual site admins) to > reliably whitelist sources safely. See > 'rules/60_whitelist_spf.cf', e.g.: > > def_whitelist_from_spf [EMAIL PROTECTED] > Yeah, I know about the SPF checks... But I meant does SA currently do anything with digital signatures to verify that the sender really is the sender & apply a -ve score. Hamish. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEo7Le/3QXwQQkZYwRAiZpAJ9hYQLMlK18VDGG72OeuBWhhfE5fwCfVO5h +qZu4jbJlyJloRSKhBEu+V8= =xZ5R -END PGP SIGNATURE-
Re: Airline reservations get tagged
Hamish writes: > On Wednesday 28 June 2006 08:48, Ralf Hildebrandt wrote: > > * [EMAIL PROTECTED] <[EMAIL PROTECTED]>: > > > Given that airline messages are important, are related to meney, and > > > recipients dont want to get forged ones, it would be a great idea to > > > start a campaign with airlines / travel agents to use some sort of > > > proof of origin (spf, digital signature, whatnot) Recipients could then > > > apply whitelists > > > > Amen to that! > > Does SA do anything with digital signatures to deduct scores? If it's > worthwhile, I'm game to play. It allows us (and third parties, and individual site admins) to reliably whitelist sources safely. See 'rules/60_whitelist_spf.cf', e.g.: def_whitelist_from_spf [EMAIL PROTECTED] --j.
Re: Airline reservations get tagged
On Wednesday 28 June 2006 08:48, Ralf Hildebrandt wrote: > * [EMAIL PROTECTED] <[EMAIL PROTECTED]>: > > Given that airline messages are important, are related to meney, and > > recipients dont want to get forged ones, it would be a great idea to > > start a campaign with airlines / travel agents to use some sort of > > proof of origin (spf, digital signature, whatnot) Recipients could then > > apply whitelists > > Amen to that! Does SA do anything with digital signatures to deduct scores? If it's worthwhile, I'm game to play. Hamish. pgpRP8SW9ERLm.pgp Description: PGP signature
Re: Airline reservations get tagged
Hi Loren, everyone Loren Wilton wrote: bayes token 'visa' => 0.997839158297152 bayes token 'refund' => 0.997646909307943 bayes token 'drinks' => 0.997585038685398 bayes token 'NUMBER' => 0.990398319296953 bayes token 'nights' => 0.98853871069642 This suggests you are still on 2.6x. It is possible that upgrading to 3.x or 3.1.x might get spam scores more in alignment with your actual incoming mail. No, we're running SA 3.04 in here. This is the output from SpamAssassin -D on an email, not a dump from the (now hashed) Bayes database. Regards, Paul Boven.
Re: Airline reservations get tagged
* [EMAIL PROTECTED] <[EMAIL PROTECTED]>: > Given that airline messages are important, are related to meney, and > recipients dont want to get forged ones, it would be a great idea to > start a campaign with airlines / travel agents to use some sort of > proof of origin (spf, digital signature, whatnot) Recipients could then > apply whitelists Amen to that! -- Ralf Hildebrandt (i.A. des IT-Zentrums) [EMAIL PROTECTED] Charite - Universitätsmedizin BerlinTel. +49 (0)30-450 570-155 Gemeinsame Einrichtung von FU- und HU-BerlinFax. +49 (0)30-450 570-962 IT-Zentrum Standort CBF send no mail to [EMAIL PROTECTED]
Re: Airline reservations get tagged
>> > Although our SA setup works very well in general, one issue that has >> > come up a few times recently is airline E-tickets/reservations. >> >> Travel stuff in general seems to be designed specifically to hit as many >> spam rules as possible. *Everything* from Travelocity and Alaska Air get >> around 20 points here on average. Ticket confirmations add more from the >> all-caps subjects and the like. >> >> > X-Spam-Score: * (5.696) BAYES_99,HTML_30_40,HTML_MESSAGE,NO_REAL_NAME, >> > SARE_OBFU_TBL_03,UPPERCASE_50_75,autolearn=no >> >> In your specific case I'd fix the bayes_99 problem, and that should get you >> clean. There isn't any particular reason I can think of why ticket >> confirmations should be getting tagged as spam by Bayes. >> >> Loren >> >> Given that airline messages are important, are related to meney, and recipients dont want to get forged ones, it would be a great idea to start a campaign with airlines / travel agents to use some sort of proof of origin (spf, digital signature, whatnot) Recipients could then apply whitelists Wolfgang Hamann
Re: Airline reservations get tagged
> bayes token 'visa' => 0.997839158297152 > bayes token 'refund' => 0.997646909307943 > bayes token 'drinks' => 0.997585038685398 > bayes token 'NUMBER' => 0.990398319296953 > bayes token 'nights' => 0.98853871069642 This suggests you are still on 2.6x. It is possible that upgrading to 3.x or 3.1.x might get spam scores more in alignment with your actual incoming mail. Loren
Re: Airline reservations get tagged
> Although our SA setup works very well in general, one issue that has > come up a few times recently is airline E-tickets/reservations. Travel stuff in general seems to be designed specifically to hit as many spam rules as possible. *Everything* from Travelocity and Alaska Air get around 20 points here on average. Ticket confirmations add more from the all-caps subjects and the like. > X-Spam-Score: * (5.696) BAYES_99,HTML_30_40,HTML_MESSAGE,NO_REAL_NAME, > SARE_OBFU_TBL_03,UPPERCASE_50_75,autolearn=no In your specific case I'd fix the bayes_99 problem, and that should get you clean. There isn't any particular reason I can think of why ticket confirmations should be getting tagged as spam by Bayes. Loren
RE: Airline reservations get tagged
Title: RE: Airline reservations get tagged > -Original Message- > From: Paul Boven [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, June 27, 2006 10:13 AM > To: Spamassassin Users List > Subject: Airline reservations get tagged > > > Hi everyone, > > Although our SA setup works very well in general, one issue that has > come up a few times recently is airline E-tickets/reservations. These > tend to be ALL CAPS and have quite a few other trigger words. Our > company seems to do business with more than one travel-agent, so just > whitelisting isn't quite enough. These mails hit the following rules: > > X-Spam-Score: * (5.696) > BAYES_99,HTML_30_40,HTML_MESSAGE,NO_REAL_NAME, > SARE_OBFU_TBL_03,UPPERCASE_50_75,autolearn=no > > Given the sensitive nature of these emails, I'd rather not post it on > the list. My question is: do other people get the same FPs? Do any of > the current rules need to take this in account? A publicly available > negative scoring rule would probably just be abused by the > spammers, so > what would be the best way to fix this, not just for me but > in general? A big resounding YES! I used to get these FPs. I have written rules to reduce the points on these. Obviously I'm not going to post them here, as spammers would just insert that stuff into their spam. I suggest you just write a few simple rules that reduce the points for airline emails. I really wish the airlines would learn to write better confirmations emails. Chris Santerre SysAdmin and SARE/URIBL ninja http://www.uribl.com http://www.rulesemporium.com
Re: Airline reservations get tagged
Hi Ralf, Ralf Hildebrandt wrote: Although our SA setup works very well in general, one issue that has come up a few times recently is airline E-tickets/reservations. These tend to be ALL CAPS and have quite a few other trigger words. Our company seems to do business with more than one travel-agent, so just whitelisting isn't quite enough. These mails hit the following rules: X-Spam-Score: * (5.696) BAYES_99,HTML_30_40,HTML_MESSAGE,NO_REAL_NAME, SARE_OBFU_TBL_03,UPPERCASE_50_75,autolearn=no You could feed these to the bayes DB as "ham" You are right, of course. But Bayes is more of a statistical tool, and given the total number of mails stored in Bayes already, I fear it will take quite a bit of learning to offset the current high scoring. Our current Bayes setup is: Company-wide Bayes database bayes_auto_learn 1 bayes_auto_learn_threshold_nonspam -0.1 score BAYES_99 5.0 score BAYES_95 4.0 Perhaps I should lower my BAYES_99 and BAYES_95 a bit, though these settings are based on past experience where Bayes alone was not able to put clearly spammy mails over the threshold. These E-tickets just look terribly spammy to Bayes because of the languaged used, it seems. Some high-scoring words for this one are: bayes token 'visa' => 0.997839158297152 bayes token 'refund' => 0.997646909307943 bayes token 'drinks' => 0.997585038685398 bayes token 'NUMBER' => 0.990398319296953 bayes token 'nights' => 0.98853871069642 Regards, Paul Boven.
Re: Airline reservations get tagged
* Paul Boven <[EMAIL PROTECTED]>: > Hi everyone, > > Although our SA setup works very well in general, one issue that has > come up a few times recently is airline E-tickets/reservations. These > tend to be ALL CAPS and have quite a few other trigger words. Our > company seems to do business with more than one travel-agent, so just > whitelisting isn't quite enough. These mails hit the following rules: > > X-Spam-Score: * (5.696) BAYES_99,HTML_30_40,HTML_MESSAGE,NO_REAL_NAME, > SARE_OBFU_TBL_03,UPPERCASE_50_75,autolearn=no You could feed these to the bayes DB as "ham" -- Ralf Hildebrandt (i.A. des IT-Zentrums) [EMAIL PROTECTED] Charite - Universitätsmedizin BerlinTel. +49 (0)30-450 570-155 Gemeinsame Einrichtung von FU- und HU-BerlinFax. +49 (0)30-450 570-962 IT-Zentrum Standort CBF send no mail to [EMAIL PROTECTED]
