Re: can we make AWL ignore mail from self to self?
You've presented good logic for acceping mail from self to self. But you haven't explained by using the AWL for mail from self to self is better than not having it. On Jun 2, 2008, at 4:02 AM, Jonas Eckerman wrote: Because it can help discriminate between spam and ham addressed from self to self. Heres an example: StupidWebService send self-self addressed ham from relay 1.2.3.4 EvilSpammer send self-self addressed spam from relay 5.6.7.8 (wich, unfortunately, belongs to a big ISP so the relay doesn'ät get blocked). One day StupidWebService send a ham that triggered a bunch of positive hits (including BAYES_99). Since mail from [EMAIL PROTECTED] has a negative score in the AWL, the mail gets though all right. One day EvilSpammer manages to send a mail that doesnät hit any positive rules, but does hit BAYES_00. Since [EMAIL PROTECTED] has a high positive score in the AWL, the mail still gets flagged as spam. If the AWL ignore mail from self-self, the two mails in the above example would have been misclassified. Indeed. I submit you are right. FYI: I still haven't had another misclassification since the first, so I'm beginning to think that this was a lark. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: And considering that SpamAssassin doesn't (in many configurations) even know what recipient address a message has, it might actually be easier than having the AWL ignore mail from self-self. It has to, for the AWL to work. No, it hasn't. The AWL only uses the *senders* address and the IP address of the client. It doesn't use the recipients address. The AWL helps discriminate between senders. Not sender-recipient pairs. As long as the MSA adds authentication info in it's received header, this could be fetched from X-Spam-Relays-Trusted pseudo header. The changes to do this would not be more difficult or invlolved than the changes necessary to exempt self-self mail from the AWL AFAICS. Easy or not, I don't see the value just yet. Including the authentication state in the AWL key would 1: Fix the problem you reported (unless I misunderstood you) 2: Fit with the current function of the AWL (discriminating between senders with no regard for recipient addresses). The AWL wouldn't work if it didn't know the recipient. Since this is something it stores in the AWL database we know that the recipient information is there. That's strange, considering that the AWL does work now, and it doesn't know the recipient. Also, the AWL doesn't store the recipient address in the database. If you use SQL base AWL, Mail::SpamAssassin::SQLBasedAddrList will store a username in the database, but neither Mail::SpamAssassin::Plugin::AWL nor Mail::SpamAssassin::AutoWhitelist knows anything about that AFAICS. Also, the username in the database might or might not be the recipients address or username. This depends entirely on how the system is setup. Here it is either mdf or spamd, and never the recipients address or local username (the local users aren't on the same machine as SA, so it knows nothing about them). You've presented good logic for acceping mail from self to self. But you haven't explained by using the AWL for mail from self to self is better than not having it. Because it can help discriminate between spam and ham addressed from self to self. Heres an example: StupidWebService send self-self addressed ham from relay 1.2.3.4 EvilSpammer send self-self addressed spam from relay 5.6.7.8 (wich, unfortunately, belongs to a big ISP so the relay doesn'ät get blocked). One day StupidWebService send a ham that triggered a bunch of positive hits (including BAYES_99). Since mail from [EMAIL PROTECTED] has a negative score in the AWL, the mail gets though all right. One day EvilSpammer manages to send a mail that doesnät hit any positive rules, but does hit BAYES_00. Since [EMAIL PROTECTED] has a high positive score in the AWL, the mail still gets flagged as spam. If the AWL ignore mail from self-self, the two mails in the above example would have been misclassified. Regards /Jonas -- Jonas Eckerman, FSDB Fruktträdet http://whatever.frukt.org/ http://www.fsdb.org/ http://www.frukt.org/
Re: can we make AWL ignore mail from self to self?
On May 29, 2008, at 4:18 AM, Jonas Eckerman wrote: Please do remember that I am in no way trying to stop or hinder you in implementing your fix. The fact that I have other suggestions does not mean that I'm opposing you. Of course. This is normal discussion. A lot of work to hack around a simple problem. The AWL works just fine for mail from my users to other my users. In fact, it works exceedingly well for that. What value is there in separating them? It would create a difference (a regards the AWL) between self-self addressed mail sent from authenticated/local users ans similar mail from other systems. I understand the concept, I don't see the value. And considering that SpamAssassin doesn't (in many configurations) even know what recipient address a message has, it might actually be easier than having the AWL ignore mail from self-self. It has to, for the AWL to work. As long as the MSA adds authentication info in it's received header, this could be fetched from X-Spam-Relays-Trusted pseudo header. The changes to do this would not be more difficult or invlolved than the changes necessary to exempt self-self mail from the AWL AFAICS. Easy or not, I don't see the value just yet. Also, while the adressee of a mail is often available with PerMsgStatus all_to_addrs, this function is not very reliable. It actually extracts a whole bunch of addresses that might be the recipient from the mail header. There is no guarantee that any of the returned addresses really are the recipient of the mail. So, to implement exemption of self-self-mail you first have to implement a way for SpamAssassin to know what the recipient address is in order to know if a mail is self-self-addressed. The AWL wouldn't work if it didn't know the recipient. Since this is something it stores in the AWL database we know that the recipient information is there. I want the AWL to apply to mail that is addressed from self-self. Since the AWL also takes the IP address into account and since all mail from authenticated/local users here comes from 127.0.0.1 to the software calling SpamAssassin, I do not have your problem here and would not benefit from your fix. While most mail addressed self-self that comes from external systems is spam, every now and then ham addressed from self-self do come in from idiotic systems and sometimes from users who for some reason is not using our servers when sending mail. The AWL as it is now does distinguish between good and bad mail that are or pretends to be from our users, and I see no reason to remove possible benefits of that distinction for mail that happens to be addressed to the same user as it's addressed from. You've presented good logic for acceping mail from self to self. But you haven't explained by using the AWL for mail from self to self is better than not having it. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
Please do remember that I am in no way trying to stop or hinder you in implementing your fix. The fact that I have other suggestions does not mean that I'm opposing you. Jo Rhett wrote: I don't trust my users in this context. Nothing I said implied or required trust in your users. A lot of work to hack around a simple problem. The AWL works just fine for mail from my users to other my users. In fact, it works exceedingly well for that. What value is there in separating them? It would create a difference (a regards the AWL) between self-self addressed mail sent from authenticated/local users ans similar mail from other systems. And considering that SpamAssassin doesn't (in many configurations) even know what recipient address a message has, it might actually be easier than having the AWL ignore mail from self-self. It also might (depedning on configuration) not require any changes at all to SpamAssassin. What alternatives? So far I've only heard (a) disable the AWL (b) don't use AWL it sucks and (c) hack the system to use different AWLs. None of which really make any logical sense to solve the problem. I also mentioned the having the AWL include the authentication state in AWL data key. As long as the MSA adds authentication info in it's received header, this could be fetched from X-Spam-Relays-Trusted pseudo header. The changes to do this would not be more difficult or invlolved than the changes necessary to exempt self-self mail from the AWL AFAICS. Also, while the adressee of a mail is often available with PerMsgStatus all_to_addrs, this function is not very reliable. It actually extracts a whole bunch of addresses that might be the recipient from the mail header. There is no guarantee that any of the returned addresses really are the recipient of the mail. So, to implement exemption of self-self-mail you first have to implement a way for SpamAssassin to know what the recipient address is in order to know if a mail is self-self-addressed. If you do implement your fix and submit it, please make it an option. I for one would turn it off since it would not improve things here. You are the first person to say so. Can you explain why? I want the AWL to apply to mail that is addressed from self-self. Since the AWL also takes the IP address into account and since all mail from authenticated/local users here comes from 127.0.0.1 to the software calling SpamAssassin, I do not have your problem here and would not benefit from your fix. While most mail addressed self-self that comes from external systems is spam, every now and then ham addressed from self-self do come in from idiotic systems and sometimes from users who for some reason is not using our servers when sending mail. The AWL as it is now does distinguish between good and bad mail that are or pretends to be from our users, and I see no reason to remove possible benefits of that distinction for mail that happens to be addressed to the same user as it's addressed from. Regards /Jonas -- Jonas Eckerman, FSDB Fruktträdet http://whatever.frukt.org/ http://www.fsdb.org/ http://www.frukt.org/
Re: can we make AWL ignore mail from self to self?
On May 23, 2008, at 3:45 AM, Jonas Eckerman wrote: 1: Just read it as of when I said your own users I meant the users of the host in question (the ones you mention above). More specifically, the users using your host as a MSA (authenticated or locally). I don't trust my users in this context. 2: I never suggested disabling the AWL entirely. I suggested disabling it for the above mentioned users. I also suggested (and this is prefferable to disabling it in my opinion) to separate the AWL so that you use one AWL for mail from the above mentioned users and another for unathenticated mail from external relays. Is there any specific reason you do not want to use two different AWLs for those two different types of traffic? Non-standard configuration/setup I would have to maintain *AND* A lot of work to hack around a simple problem. The AWL works just fine for mail from my users to other my users. In fact, it works exceedingly well for that. What value is there in separating them? A more involved change would be to have the AWL store the authentication state as well as mail address and relay IP/16. When scanning mail from your own users using the same AWL database as for for mail to your users, this seems necessary to me. Again, this seems to be a lot of work for no real gain. What I have proposed makes sense for widespread use. Why hack/slash/burn when a good fix would improve it for everyone? In case you haven't noticed it, your suggestion is not seen as a good fix for the problem by everyone. I was merely suggesting other ways to go about this. Actually, that's not true. Nobody has suggested that this fix would be bad. Matt was querying me thinking I had screwed up my trusted hosts, but not a single person has suggested that this change would be bad. If you wish other peoiple to implement/accept something that fixes your problem and you can't convince them that your own ideas are good, it may be that alternative means of fixing the problem are seen as better and therefore stand a bigger chance of being implemented/eccepted. What alternatives? So far I've only heard (a) disable the AWL (b) don't use AWL it sucks and (c) hack the system to use different AWLs. None of which really make any logical sense to solve the problem. If you do implement your fix and submit it, please make it an option. I for one would turn it off since it would not improve things here. You are the first person to say so. Can you explain why? -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: Lots of users of this host have Windows PCs, Another way to do it would be to use different AWLs, or disabling AWL, for mail from your own users (either authenticated or locally submitted). This makes a lot of sense to me. Have no my own users except me ;-) And disabling AWL entirely is again a hack. Let's focus on a fix. 1: Just read it as of when I said your own users I meant the users of the host in question (the ones you mention above). More specifically, the users using your host as a MSA (authenticated or locally). 2: I never suggested disabling the AWL entirely. I suggested disabling it for the above mentioned users. I also suggested (and this is prefferable to disabling it in my opinion) to separate the AWL so that you use one AWL for mail from the above mentioned users and another for unathenticated mail from external relays. Is there any specific reason you do not want to use two different AWLs for those two different types of traffic? A more involved change would be to have the AWL store the authentication state as well as mail address and relay IP/16. When scanning mail from your own users using the same AWL database as for for mail to your users, this seems necessary to me. Again, this seems to be a lot of work for no real gain. What I have proposed makes sense for widespread use. Why hack/slash/burn when a good fix would improve it for everyone? In case you haven't noticed it, your suggestion is not seen as a good fix for the problem by everyone. I was merely suggesting other ways to go about this. If you wish other peoiple to implement/accept something that fixes your problem and you can't convince them that your own ideas are good, it may be that alternative means of fixing the problem are seen as better and therefore stand a bigger chance of being implemented/eccepted. I am not, however, trying to stop you from implementing ignoring self-self mail by the AWL. If you do implement your fix and submit it, please make it an option. I for one would turn it off since it would not improve things here. Regards /Jonas -- Jonas Eckerman, FSDB Fruktträdet http://whatever.frukt.org/ http://www.fsdb.org/ http://www.frukt.org/
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: I'm not -- my Treo delivers mail directly to my mail server. From DHCP-assigned addresses all over the world. I enjoy travel ;-) Then I guess you use authenticated SMTP for that. The easiest way to handle this probably is to simply avoid calling SA for authenticated mail. Another way to do it would be to use different AWLs, or disabling AWL, for mail from your own users (either authenticated or locally submitted). This makes a lot of sense to me. A more involved change would be to have the AWL store the authentication state as well as mail address and relay IP/16. When scanning mail from your own users using the same AWL database as for for mail to your users, this seems necessary to me. Regards /Jonas -- Jonas Eckerman, FSDB Fruktträdet http://whatever.frukt.org/ http://www.fsdb.org/ http://www.frukt.org/
Re: can we make AWL ignore mail from self to self?
On May 22, 2008, at 7:29 AM, Jonas Eckerman wrote: Jo Rhett wrote: I'm not -- my Treo delivers mail directly to my mail server. From DHCP-assigned addresses all over the world. I enjoy travel ;-) Then I guess you use authenticated SMTP for that. The easiest way to handle this probably is to simply avoid calling SA for authenticated mail. That's a hack with consequences. Like just disable the firewall. Uh, no ;-) Lots of users of this host have Windows PCs, and running SA on all outbound mail has both alerted them quickly to the problem and avoided nailing other people with spam and/or virus runs. Another way to do it would be to use different AWLs, or disabling AWL, for mail from your own users (either authenticated or locally submitted). This makes a lot of sense to me. Have no my own users except me ;-) And disabling AWL entirely is again a hack. Let's focus on a fix. A more involved change would be to have the AWL store the authentication state as well as mail address and relay IP/16. When scanning mail from your own users using the same AWL database as for for mail to your users, this seems necessary to me. Again, this seems to be a lot of work for no real gain. What I have proposed makes sense for widespread use. Why hack/slash/burn when a good fix would improve it for everyone? -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: That's a hack with consequences. Like just disable the firewall. Uh, no ;-) Lots of users of this host have Windows PCs, and running SA on all outbound mail has both alerted them quickly to the problem and avoided nailing other people with spam and/or virus runs. Something seems out of order here. First, even if this isn't what you meant, I must set the record straight... requiring SMTP password-authentication is NOT a hack. Instead, that is a security feature. I'm not sure if you meant that differently, but I state this just to be on the safe side. Second, you do require SMTP authentication, right? Because not doing so would likely open up your server as an open relay. Additionally, the vast majority of the spams and viruses that you referred to would not have a chance of using your server to nail other people with spams or viruses if you required SMTP authentication. Most not-large-isp mail servers do just fine NOT spam filtering SMTP password-authenticated messages with many years going by between any single incident of a spam or virus being sent from that server. The main reason larger ISPs must do some spam filtering on their outbound mail sent from members of that ISP is because (a) they do NOT use SMTP password-authentication and, instead, allow relaying simply based on the message originating from a particular block of IPs (very bad form... but the large ISPs can't find an easy way to convert millions of users over to SMTP authentication). If that is your situation, then I probably stand corrected as far as your situation is concerned. ..OR.. (b) they are a heavily abused service.. such as freemail providers where criminals sign up to try to send spam. Therefore, they should do outbound filtering even on authenticated mail. Otherwise, SMTP password-authenticated e-mail should almost always not be filtered, or be minimally filtered. Rob McEwen
Re: can we make AWL ignore mail from self to self?
On Thu, 22 May 2008, Jo Rhett wrote:
Then I guess you use authenticated SMTP for that.
The easiest way to handle this probably is to simply avoid calling SA for
authenticated mail.
That's a hack with consequences. Like just disable the firewall. Uh, no
;-)
Lots of users of this host have Windows PCs, and running SA on all outbound
mail has both alerted them quickly to the problem and avoided nailing other
people with spam and/or virus runs.
Genuine curiosity Jo, have you seen instances of viruses/trojans sending
-authenticated- mail? Have they learned how to read users' passwords, etc?
We require our PC users to authenticate when sending and I had assumed
that would stop viruses/trojans. Am I being naive?
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Re: can we make AWL ignore mail from self to self?
At 13:23 22-05-2008, Dave Funk wrote: We require our PC users to authenticate when sending and I had assumed that would stop viruses/trojans. Am I being naive? No. But it's only one extra step for malware to capture SMTP authentication information. Regards, -sm
Re: can we make AWL ignore mail from self to self?
On May 22, 2008, at 12:42 PM, Rob McEwen wrote: First, even if this isn't what you meant, I must set the record straight... requiring SMTP password-authentication is NOT a hack. Instead, that is a security feature. I'm not sure if you meant that differently, but I state this just to be on the safe side. Second, you do require SMTP authentication, right? Because not doing so would likely open up your server as an open relay. Rob, please read what you reply to. I've been doing SMTP AUTH since before we got it standardized. I said that disabling running SA for SMTP-AUTH users is a hack much like disabling a firewall and I won't do it. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On May 22, 2008, at 1:23 PM, Dave Funk wrote: Lots of users of this host have Windows PCs, and running SA on all outbound mail has both alerted them quickly to the problem and avoided nailing other people with spam and/or virus runs. Genuine curiosity Jo, have you seen instances of viruses/trojans sending -authenticated- mail? Have they learned how to read users' passwords, etc? We require our PC users to authenticate when sending and I had assumed that would stop viruses/trojans. Am I being naive? Yes, you are. Most of the viri use the existing Outlook configuration, which includes the user's saved SMTP AUTH passwords. Like I said, SA has saved our butt each time it happened. I wouldn't say that without it having happened multiple times... -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On May 20, 2008, at 1:07 PM, Justin Mason wrote: 1. How does AWL deal with forgery (other than by saving a /16 of the source IP) No other way. What's wrong with saving a /16? In my experience it's worked pretty well for the past few years... Seems to. I can logically think of ways it would/should break (ie public wireless networks) but I haven't seen any real problems until now, and the problem is specific to self-self. My comment was only because Matt kept insisting that AWL prevents forgery... 2. How can I easily see the AWL database for a given destination address? tools/check_whitelist Where can I find this? It's not in the Mail-SpamAssassin tarfile... -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: Matt, how can I possibly get you to move past this unfounded assumption that my trust path is broken and focus on the real problem? The trust path is not broken, it's just fine. On May 20, 2008, at 5:47 PM, Matt Kettler wrote: Ok, then the AWL code is *SEVERELY* bugged. The question then becomes why isn't the source address part of the AWL working properly. I'm not sure I know this or can agree. I'm fairly sure its orthagonal, but I may be wrong. That IP range is what would detect the forgeries, or at least give the forgeries a different AWL entry than email you really sent yourself. I only send mail to myself from my wireless provider or open WiFi networks. e.g. note to self while I am on the road. The source IPs are different, so your real self-to-self should be handled independently, with a completely separate AWL entry, from the spammer forged self-to-self. You're assuming I use the same source IP when I send myself mail, and that just isn't true. Or that you receive e-mail from the very same public wireless and/ or phone providers as everyone else does. My trust path doesn't have to be broken if the networks used to send the e-mail are public networks. (if you can laugh == welcome to the 21st century and the Crackberry/Treo/iPhone) Not trying to be snide. If you're using any kind of forwarder, including crackberry, their servers should be trusted by you so that SA's checks get applied to the mailserver that dropped mail off at them. That's the purpose of the trust path, to allow you to trust the headers of those systems receiving mail on your behalf and forwarding it to you. I'm not -- my Treo delivers mail directly to my mail server. From DHCP-assigned addresses all over the world. I enjoy travel ;-) I'd also like to point out that no provider is willing to share their server lists openly and consistently enough for this to occur. We have to put crackberry users in their own domain because we use SPF on the main domains and crackberry keeps changing their servers. no provider == crackberry, verizon, sprint, etc... the wireless providers who intercept outbound mail. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On 21 May 2008, Jo Rhett stated: On May 20, 2008, at 1:07 PM, Justin Mason wrote: 2. How can I easily see the AWL database for a given destination address? tools/check_whitelist Where can I find this? It's not in the Mail-SpamAssassin tarfile... It's in SVN. -- `If you are having a ua luea luea le ua le kind of day, I can only assume that you are doing no work due [to] incapacitating nausea caused by numerous lazy demons.' --- Frossie
Re: can we make AWL ignore mail from self to self?
On May 3, 2008, at 7:59 PM, Matt Kettler wrote: Have you tried running one of the forged messages, and an actual legitimate message through SA manually with the -D flag to see what the trusted and untrusted hosts are, as SA sees it? Yes. Many times. That's not the point of this thread. I still think it is. Matt, how can I possibly get you to move past this unfounded assumption that my trust path is broken and focus on the real problem? The trust path is not broken, it's just fine. If your AWL is applying the same history data to forged email as unforged email, either there's a *major* bug in the AWL code, or your trust path is broken. Period. The AWL is designed to be able to distinguish forged mail from nonforged mail. If that's not working, that's a major problem. I've read the code and I see nothing designed to determine forgeries. There is code to save data with an IP range, but that's not relevant to this issue. The point of this thread is the obvious ease of forging e-mail from recipient to (same) recipient. It's one situation where the AWL wouldn't work very well. Actually, it's very difficult to forge in a way that will confuse the AWL, if your trust path and the AWL code is working properly. After all, it looks at the combination of email address and first untrusted IP. Forged email will not be from the same IP as legitimate email, unless your trust path is broken and SA always sees all mail as entering your network from the same IP. Or that you receive e-mail from the very same public wireless and/or phone providers as everyone else does. My trust path doesn't have to be broken if the networks used to send the e-mail are public networks. (if you can laugh == welcome to the 21st century and the Crackberry/ Treo/iPhone) Not trying to be snide. It would be fairly easy to forge, and worthwhile enough for botnets to just do this (which they are, in force, for the last month) I personally see no value in applying AWL to messages from self to self. I agree, but I see no value in applying the exception. I'd rather try to fix the more general problem of your AWL not distinguishing message sources properly. I see no evidence of this. My trust path is just fine (ie nonexistent == all mail not from localhost isn't trusted) I may be wrong, and I'm open to arguements against this, but I am suggesting that the AWL module should skip over self-self messages. It seems too easy to forge, and no gain in doing so. You're overlooking how the AWL works. It's actually really hard to forge. However, I will agree with you there's limited value in self-to-self AWL records.. but there's also no harm in them if the AWL is working properly. Instead of making statements like this, please explain how the AWL deals the forgery. Because I have the code right in front of me and I see absolutely nothing in the AWL code that tries to identify forgeries. Instead of making unfounded statements, can you be specific about the issues?
Re: can we make AWL ignore mail from self to self?
Let's focus this on specific technical details: 1. How does AWL deal with forgery (other than by saving a /16 of the source IP) 2. How can I easily see the AWL database for a given destination address?
Re: can we make AWL ignore mail from self to self?
Jo Rhett writes: Let's focus this on specific technical details: 1. How does AWL deal with forgery (other than by saving a /16 of the source IP) No other way. What's wrong with saving a /16? In my experience it's worked pretty well for the past few years... 2. How can I easily see the AWL database for a given destination address? tools/check_whitelist --j.
Re: can we make AWL ignore mail from self to self?
On Tue, May 20, 2008 22:07, Justin Mason wrote: No other way. What's wrong with saving a /16? In my experience it's worked pretty well for the past few years... when mails is from [EMAIL PROTECTED] to [EMAIL PROTECTED] this should kill the attempt to get negative scores but positive should still be added :-) else turn down AWL_factor in the plugin, default is 0.5, but better imho is 0.25 Benny Pedersen Need more webspace ? http://www.servage.net/?coupon=cust37098
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: On May 3, 2008, at 7:59 PM, Matt Kettler wrote: Have you tried running one of the forged messages, and an actual legitimate message through SA manually with the -D flag to see what the trusted and untrusted hosts are, as SA sees it? Yes. Many times. That's not the point of this thread. I still think it is. Matt, how can I possibly get you to move past this unfounded assumption that my trust path is broken and focus on the real problem? The trust path is not broken, it's just fine. Ok, then the AWL code is *SEVERELY* bugged. The question then becomes why isn't the source address part of the AWL working properly. If your AWL is applying the same history data to forged email as unforged email, either there's a *major* bug in the AWL code, or your trust path is broken. Period. The AWL is designed to be able to distinguish forged mail from nonforged mail. If that's not working, that's a major problem. I've read the code and I see nothing designed to determine forgeries. There is code to save data with an IP range, but that's not relevant to this issue. That's entirely relevant. That IP range is what would detect the forgeries, or at least give the forgeries a different AWL entry than email you really sent yourself. The source IPs are different, so your real self-to-self should be handled independently, with a completely separate AWL entry, from the spammer forged self-to-self. The point of this thread is the obvious ease of forging e-mail from recipient to (same) recipient. It's one situation where the AWL wouldn't work very well. Actually, it's very difficult to forge in a way that will confuse the AWL, if your trust path and the AWL code is working properly. After all, it looks at the combination of email address and first untrusted IP. Forged email will not be from the same IP as legitimate email, unless your trust path is broken and SA always sees all mail as entering your network from the same IP. Or that you receive e-mail from the very same public wireless and/or phone providers as everyone else does. My trust path doesn't have to be broken if the networks used to send the e-mail are public networks. (if you can laugh == welcome to the 21st century and the Crackberry/Treo/iPhone) Not trying to be snide. If you're using any kind of forwarder, including crackberry, their servers should be trusted by you so that SA's checks get applied to the mailserver that dropped mail off at them. That's the purpose of the trust path, to allow you to trust the headers of those systems receiving mail on your behalf and forwarding it to you. It would be fairly easy to forge, and worthwhile enough for botnets to just do this (which they are, in force, for the last month) I personally see no value in applying AWL to messages from self to self. I agree, but I see no value in applying the exception. I'd rather try to fix the more general problem of your AWL not distinguishing message sources properly. I see no evidence of this. My trust path is just fine (ie nonexistent == all mail not from localhost isn't trusted) Agreed that's probably reasonable in many networks. I may be wrong, and I'm open to arguements against this, but I am suggesting that the AWL module should skip over self-self messages. It seems too easy to forge, and no gain in doing so. You're overlooking how the AWL works. It's actually really hard to forge. However, I will agree with you there's limited value in self-to-self AWL records.. but there's also no harm in them if the AWL is working properly. Instead of making statements like this, please explain how the AWL deals the forgery. Because I have the code right in front of me and I see absolutely nothing in the AWL code that tries to identify forgeries. Instead of making unfounded statements, can you be specific about the issues?
Re: can we make AWL ignore mail from self to self?
On Apr 29, 2008, at 7:40 PM, Matt Kettler wrote: I'm not repeating for the 5th time that there are no trusted mailservers. Only this host. That's a contradiction, because this host is a mailserver. Clearly you have a trusted mailserver. However, in the interest of moving the discussion forward, you have exactly one trusted mailserver, your MX, which is perfectly valid. Yes. I'm sorry but this is obvious. I don't know how to pick the words exactly as you want them, but most people understood what I meant 5 or 6 replies ago ;-) The question lies in why does the AWL seem to be confusing forged email with your own email. That's generally quite critically dependent on the trust path. No, that's not the question at all. (more below) Have you tried running one of the forged messages, and an actual legitimate message through SA manually with the -D flag to see what the trusted and untrusted hosts are, as SA sees it? Yes. Many times. That's not the point of this thread. The point of this thread is the obvious ease of forging e-mail from recipient to (same) recipient. It's one situation where the AWL wouldn't work very well. It would be fairly easy to forge, and worthwhile enough for botnets to just do this (which they are, in force, for the last month) I personally see no value in applying AWL to messages from self to self. I may be wrong, and I'm open to arguements against this, but I am suggesting that the AWL module should skip over self-self messages. It seems too easy to forge, and no gain in doing so. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: On Apr 29, 2008, at 7:40 PM, Matt Kettler wrote: I'm not repeating for the 5th time that there are no trusted mailservers. Only this host. That's a contradiction, because this host is a mailserver. Clearly you have a trusted mailserver. However, in the interest of moving the discussion forward, you have exactly one trusted mailserver, your MX, which is perfectly valid. Yes. I'm sorry but this is obvious. I don't know how to pick the words exactly as you want them, but most people understood what I meant 5 or 6 replies ago ;-) The question lies in why does the AWL seem to be confusing forged email with your own email. That's generally quite critically dependent on the trust path. No, that's not the question at all. (more below) Have you tried running one of the forged messages, and an actual legitimate message through SA manually with the -D flag to see what the trusted and untrusted hosts are, as SA sees it? Yes. Many times. That's not the point of this thread. I still think it is. If your AWL is applying the same history data to forged email as unforged email, either there's a *major* bug in the AWL code, or your trust path is broken. Period. The AWL is designed to be able to distinguish forged mail from nonforged mail. If that's not working, that's a major problem. The point of this thread is the obvious ease of forging e-mail from recipient to (same) recipient. It's one situation where the AWL wouldn't work very well. Actually, it's very difficult to forge in a way that will confuse the AWL, if your trust path and the AWL code is working properly. After all, it looks at the combination of email address and first untrusted IP. Forged email will not be from the same IP as legitimate email, unless your trust path is broken and SA always sees all mail as entering your network from the same IP. It would be fairly easy to forge, and worthwhile enough for botnets to just do this (which they are, in force, for the last month) I personally see no value in applying AWL to messages from self to self. I agree, but I see no value in applying the exception. I'd rather try to fix the more general problem of your AWL not distinguishing message sources properly. I may be wrong, and I'm open to arguements against this, but I am suggesting that the AWL module should skip over self-self messages. It seems too easy to forge, and no gain in doing so. You're overlooking how the AWL works. It's actually really hard to forge. However, I will agree with you there's limited value in self-to-self AWL records.. but there's also no harm in them if the AWL is working properly.
Re: can we make AWL ignore mail from self to self?
On Apr 21, 2008, at 10:01 PM, Theo Van Dinter wrote: Actually I don't think it's that hard, at least for conversations on public lists. Right now it seems to be more work than they bother with. As I've noted, I read all my spam looking at the latest techniques and I've never seen this. (I have a 20-year-old mail address which gets thousands per hour unfiltered which I use to test my ideas with) Also, I've had spammers forge my email address from work to mail my personal account. Do you have the same lhs? At least one of the botnets tries to match lhs for the forged sender. A few of my messages came from my other accounts, many others (in the same spam run) came from people I didn't know with the same lhs. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On Apr 21, 2008, at 10:46 PM, Bob Proulx wrote: Jo Rhett wrote: Bob Proulx wrote: Who to forge? The answer is Everyone! Any address that can be You're going out of your way to miss the point. That's hard work It is you who are missing the point. When spammers generate mail from and to every possible combination they will eventually hit a combination that you will see. The distributed spamming engines of the 'bot-nets are quite powerful and can generate this volume of traffic. Eventually is the big word. If we succeed in making spam eventually get through then we would have won this war. I'm saying that I've never seen this in the wild, and the AWL has been 99% or greater effective for me, so I'm not going to throw away a good tool because it is theoritically possible to fit another angel on that pinhead. It works today. Now please stop arguing that AWL is useless. It works for me. If it doesn't work for you, then you have no reason to reply on this thread. (not trying to be rude, but this conversation is pointless) -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On Apr 22, 2008, at 12:06 AM, Matus UHLAR - fantomas wrote: On 21.04.08 23:46, Bob Proulx wrote: It is you who are missing the point. When spammers generate mail from and to every possible combination they will eventually hit a combination that you will see. The distributed spamming engines of the 'bot-nets are quite powerful and can generate this volume of traffic. especially when they start collecting people's addressbooks to see who sends mail to whom. In which case I will know and inform my friend that their system has been compromised. And again, irrelevant to the topic. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On Apr 23, 2008, at 3:27 PM, Matt Kettler wrote: How and why? Are you saying I *must* have a 2nd-level MX host for SA to work? That's not my experience, and 2-layer relays are backscatter sources. Milter from the local MTA works just fine. No, you don't need a second-level MX. However, to work properly, SA must trust everything up to an including your MX, and all your trusted mailservers need to generate Received: headers that SA can then make sense of. I'm not repeating for the 5th time that there are no trusted mailservers. Only this host. This isn't about SA trusting the originating source of the message. it's about SA trusting that at least one trusted mailserver actually received the message. ie: the message has to have actually arrived at your server, and not been transplanted from nowhere by magic. If there's no trusted headers, then all messages are equally magic to SA, and it will never distinguish mail you sent as compared to mail an outsider forged as you. Yes, it knows the localhost received header is valid. Basics of SA setup 101. Now can we return to the topic? -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On Tue, 29 Apr 2008 at 17:53 -0700, [EMAIL PROTECTED] confabulated: Now please stop arguing that AWL is useless. It works for me. If it doesn't work for you, then you have no reason to reply on this thread. (not trying to be rude, but this conversation is pointless) Works for me too. I was going to reply to the thread days weeks ago and didn't have time. Some of my AWL scores on spam have been in the negative. However, bayes_99 quickly brings it back onto the positive side for the rest of the positive scoring rules.
Re: can we make AWL ignore mail from self to self?
On Tue, 29 Apr 2008 at 17:58 -0700, [EMAIL PROTECTED] confabulated: I'm not repeating for the 5th time that there are no trusted mailservers. Only this host. Correct. On our filter server(s) which are strictly inbound only (nothing trusted but itself): # Begin SA Network Settings clear_trusted_networks clear_internal_networks clear_msa_networks trusted_networks 192.168.1.100 # smtpgate.ndunet.com internal_networks 192.168.1.100 # smtpgate.ndunet.com
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: On Apr 23, 2008, at 3:27 PM, Matt Kettler wrote: How and why? Are you saying I *must* have a 2nd-level MX host for SA to work? That's not my experience, and 2-layer relays are backscatter sources. Milter from the local MTA works just fine. No, you don't need a second-level MX. However, to work properly, SA must trust everything up to an including your MX, and all your trusted mailservers need to generate Received: headers that SA can then make sense of. I'm not repeating for the 5th time that there are no trusted mailservers. Only this host. That's a contradiction, because this host is a mailserver. Clearly you have a trusted mailserver. However, in the interest of moving the discussion forward, you have exactly one trusted mailserver, your MX, which is perfectly valid. The question lies in why does the AWL seem to be confusing forged email with your own email. That's generally quite critically dependent on the trust path. Have you tried running one of the forged messages, and an actual legitimate message through SA manually with the -D flag to see what the trusted and untrusted hosts are, as SA sees it?
Re: can we make AWL ignore mail from self to self?
On Tue, Apr 29, 2008 at 05:51:17PM -0700, Jo Rhett wrote: Do you have the same lhs? At least one of the botnets tries to match lhs for the forged sender. A few of my messages came from my other accounts, many others (in the same spam run) came from people I didn't know with the same lhs. No, different usernames. -- Randomly Selected Tagline: You're one hundred percent positive that the ship which is crashed on the bottom of this ocean is the ship which you said you were one hundred percent positive could one hundred percent positively never crash? pgpsrPqrat1Sd.pgp Description: PGP signature
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: Matt Kettler wrote: There's nothing in trusted networks, I don't trust anything... Jo, that's impossible in spamassasin. You cannot have an empty trust, it doesn't make any logical sense, and would cause spamassassin to fail miserably. I should rather have said trust is only localhost. If you don't declare a trusted_networks, SA will auto-guess for you. (And the auto-guesser is notorious for failing if your MX is NAT mapped) And please, understand that trust here means trusted to never forge a received header not trusted to never relay any spam. I know this. In spamassassin, under trusting is BAD. It is just as bad as over-trusting. SA needs at least one trustworthy received header to work with. How and why? Are you saying I *must* have a 2nd-level MX host for SA to work? That's not my experience, and 2-layer relays are backscatter sources. Milter from the local MTA works just fine. No, you don't need a second-level MX. However, to work properly, SA must trust everything up to an including your MX, and all your trusted mailservers need to generate Received: headers that SA can then make sense of. Also, to work properly, SA needs to be able to determine what is a part of your network, and what isn't. Unless you declare internal_networks separately, it bases internal vs external on the trust. There is no network. There is only a single host. I don't control any other host on the subnet. trust no-one is NOT a valid option, and would actually result in the problem you're suffering from. After all, if no headers are trusted, all email comes from no server, so SA would never be able to tell the difference between an email you really sent, vs a forgery from the outside. This statement parses as nonsense. SA can't parse an e-mail because it doesn't trust the source? Isn't that all e-mail? Erm, how did you mis-parse that statement? This isn't about SA trusting the originating source of the message. it's about SA trusting that at least one trusted mailserver actually received the message. ie: the message has to have actually arrived at your server, and not been transplanted from nowhere by magic. If there's no trusted headers, then all messages are equally magic to SA, and it will never distinguish mail you sent as compared to mail an outsider forged as you.
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: Bob Proulx wrote: Who to forge? The answer is Everyone! Any address that can be You're going out of your way to miss the point. That's hard work On 21.04.08 23:46, Bob Proulx wrote: It is you who are missing the point. When spammers generate mail from and to every possible combination they will eventually hit a combination that you will see. The distributed spamming engines of the 'bot-nets are quite powerful and can generate this volume of traffic. especially when they start collecting people's addressbooks to see who sends mail to whom. -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Microsoft dick is soft to do no harm
Re: can we make AWL ignore mail from self to self?
Matt Kettler wrote: There's nothing in trusted networks, I don't trust anything... Jo, that's impossible in spamassasin. You cannot have an empty trust, it doesn't make any logical sense, and would cause spamassassin to fail miserably. I should rather have said trust is only localhost. If you don't declare a trusted_networks, SA will auto-guess for you. (And the auto-guesser is notorious for failing if your MX is NAT mapped) And please, understand that trust here means trusted to never forge a received header not trusted to never relay any spam. I know this. In spamassassin, under trusting is BAD. It is just as bad as over-trusting. SA needs at least one trustworthy received header to work with. How and why? Are you saying I *must* have a 2nd-level MX host for SA to work? That's not my experience, and 2-layer relays are backscatter sources. Milter from the local MTA works just fine. Also, to work properly, SA needs to be able to determine what is a part of your network, and what isn't. Unless you declare internal_networks separately, it bases internal vs external on the trust. There is no network. There is only a single host. I don't control any other host on the subnet. trust no-one is NOT a valid option, and would actually result in the problem you're suffering from. After all, if no headers are trusted, all email comes from no server, so SA would never be able to tell the difference between an email you really sent, vs a forgery from the outside. This statement parses as nonsense. SA can't parse an e-mail because it doesn't trust the source? Isn't that all e-mail? If your trust path is working properly, SA knows the difference. If it's not working, you get a broken AWL, broken RBLs, broken ALL_TRUSTED, and dozens of other broken things. Okay, seriously I think you're both underestimating my understanding of this and further confusing the matter by making all sorts of unclear claims that don't reflect in reality. I get trust paths. This issue I reported is not related to trust paths. It's not a broken trust path problem. The e-mail came from an untrusted source, but was given a negative AWL score based on the sender name. That has nothing to do with trust.
Re: can we make AWL ignore mail from self to self?
John Hardin wrote: I'm only suggesting bypassing SA for mail that originates on the local network and is destined to the local network. No. I don't trust every user who can authenticate to this host to run active anti-virus on their hosts. I scan all mail, everywhere. And again, this isn't about local mail marked as spam. It's about non-local mail being marked as ham.
Re: can we make AWL ignore mail from self to self?
Bob Proulx wrote: Who to forge? The answer is Everyone! Any address that can be obtained from a spam-virus infected PC and any address that can be harvested from a web page. Forge them all. They are (mostly) valid email addresses and will pass sender verification. Send To: and From: all of them. You're going out of your way to miss the point. That's hard work Yes, a spammer can forge anyone. Can they forge the exact e-mail addresses used by people I correspond with regularly? Not in my experience. Can they forge my e-mail to me? Easily.
Re: can we make AWL ignore mail from self to self?
Justin Mason wrote: hmm, I'm not sure. It depends on your trusted_networks setting. try running spamassassin -D and see what it logs... I'm sorry -- feeling dense, how is this supposed to help? From the headers quoted below you know what spamassassin is seeing. There's nothing in trusted networks, I don't trust anything... No, I don't know. I'd have to run SpamAssassin to find out. Since you're asking, you can run it ;) I would, but I can't find the exact situation that made this work nor the original message. My other testing doesn't reproduce anything near a -10 score. Is there any useful way to query the AWL database to find how this might have occurred? trusted networks is just localhost, which is what Darryl recommended for single hosts without any trusted hosts.
Re: can we make AWL ignore mail from self to self?
On Mon, Apr 21, 2008 at 09:56:39PM -0700, Jo Rhett wrote: Yes, a spammer can forge anyone. Can they forge the exact e-mail addresses used by people I correspond with regularly? Not in my experience. Can they forge my e-mail to me? Easily. Actually I don't think it's that hard, at least for conversations on public lists. Also, I've had spammers forge my email address from work to mail my personal account. fwiw. -- Randomly Selected Tagline: It's not you Bernie. I guess I'm just not used to being chased around a mall at night by killer robots. - Linda from the movie Chopping Mall pgpx9NISETr4Y.pgp Description: PGP signature
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: Bob Proulx wrote: Who to forge? The answer is Everyone! Any address that can be You're going out of your way to miss the point. That's hard work It is you who are missing the point. When spammers generate mail from and to every possible combination they will eventually hit a combination that you will see. The distributed spamming engines of the 'bot-nets are quite powerful and can generate this volume of traffic. Bob
Re: can we make AWL ignore mail from self to self?
Jo Rhett writes: On Apr 1, 2008, at 3:14 PM, Justin Mason wrote: Sorry, I don't the original messages any more. (I looked) But it wouldn't surprise me if the /16 matched. The mail I send myself is usually from Wifi or my phone carrier's GSM network, but accepted via SMTP AUTH on the local machine. So which address are you using? hmm, I'm not sure. It depends on your trusted_networks setting. try running spamassassin -D and see what it logs... I'm sorry -- feeling dense, how is this supposed to help? From the headers quoted below you know what spamassassin is seeing. There's nothing in trusted networks, I don't trust anything... No, I don't know. I'd have to run SpamAssassin to find out. Since you're asking, you can run it ;) Here's an example. Return-Path: [EMAIL PROTECTED] Received: from mail.netconsonance.com ([unix socket]) by triceratops.netconsonance.com (Cyrus v2.3.9) with LMTPA; Tue, 01 Apr 2008 13:14:34 -0700 X-Sieve: CMU Sieve 2.3 Received: from [10.178.18.103] (m4a0e36d0.tmodns.net [208.54.14.74]) (authenticated bits=0) by mail.netconsonance.com (8.14.1/8.14.1) with ESMTP id m31KE4ui014296 for [EMAIL PROTECTED]; Tue, 1 Apr 2008 13:14:27 -0700 (PDT) (envelope-from [EMAIL PROTECTED]) X-Virus-Scanned: amavisd-new at netconsonance.com X-Spam-Flag: NO X-Spam-Score: -0.72 X-Spam-Level: X-Spam-Status: No, score=-0.72 tagged_above=-999 required=3.8 tests=[ALL_TRUSTED=-1.44, AWL=0.720] From: Jo Rhett [EMAIL PROTECTED] Subject: test awl Date: 01 Apr 2008 13:14:00 -0700 To: [EMAIL PROTECTED] X-Mailer: ChatterEmail+ for Treo 6xx/700p (3.0.8) Message-ID:[EMAIL PROTECTED] -- from the cell phone of Jo Rhett Network/Software Engineer Network Consonance -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: On Apr 1, 2008, at 3:14 PM, Justin Mason wrote: Sorry, I don't the original messages any more. (I looked) But it wouldn't surprise me if the /16 matched. The mail I send myself is usually from Wifi or my phone carrier's GSM network, but accepted via SMTP AUTH on the local machine. So which address are you using? hmm, I'm not sure. It depends on your trusted_networks setting. try running spamassassin -D and see what it logs... I'm sorry -- feeling dense, how is this supposed to help? From the headers quoted below you know what spamassassin is seeing. There's nothing in trusted networks, I don't trust anything... Jo, that's impossible in spamassasin. You cannot have an empty trust, it doesn't make any logical sense, and would cause spamassassin to fail miserably. If you don't declare a trusted_networks, SA will auto-guess for you. (And the auto-guesser is notorious for failing if your MX is NAT mapped) And please, understand that trust here means trusted to never forge a received header not trusted to never relay any spam. In spamassassin, under trusting is BAD. It is just as bad as over-trusting. SA needs at least one trustworthy received header to work with. Also, to work properly, SA needs to be able to determine what is a part of your network, and what isn't. Unless you declare internal_networks separately, it bases internal vs external on the trust. This is why trust is important, and it's important to get it right. trust no-one is NOT a valid option, and would actually result in the problem you're suffering from. After all, if no headers are trusted, all email comes from no server, so SA would never be able to tell the difference between an email you really sent, vs a forgery from the outside. If your trust path is working properly, SA knows the difference. If it's not working, you get a broken AWL, broken RBLs, broken ALL_TRUSTED, and dozens of other broken things. See also: http://wiki.apache.org/spamassassin/TrustPath
Re: can we make AWL ignore mail from self to self?
On Thu, April 3, 2008 05:33, Bob Proulx wrote: Who to forge? The answer is Everyone! Any address that can be obtained from a spam-virus infected PC and any address that can be harvested from a web page. Forge them all. yes a big problem without spf They are (mostly) valid email addresses and will pass sender verification. Send To: and From: all of them. shame that sender host does not use spf when verifying it Benny Pedersen Need more webspace ? http://www.servage.net/?coupon=cust37098
Re: can we make AWL ignore mail from self to self?
On Apr 1, 2008, at 3:00 PM, Bob Proulx wrote: I have never been fond of AWL because the information it relies upon, the mail headers, is very easy to forge. It depends too much upon Yes, but they have to know who to forge. Anyway, I'm not debating its merits. It works very, very well in our experience. Except for this one situation. What I am pointing out is that AWL should not be used for mail from self to self, because this is an easy forgery. It is all very easy to forge. But self to self is very easy for the recipient to spot as a forgery. (Unless they have a short memory and are very gullible. :-) Not guillable, but don't want to get an obvious spam in my mailbox. SA knew it was spammy, but the AWL discounted the score. I disagree with the premise that it is hard to forge mail from someone you correspond with frequently. It is equally easy to forge. Easy to forge, but who to forge? Hard for a spammer to know who I correspond with frequently. Myself is the only one a spammer could guess. Again, not debating its merits just the implementation. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On Apr 1, 2008, at 3:14 PM, Justin Mason wrote: Sorry, I don't the original messages any more. (I looked) But it wouldn't surprise me if the /16 matched. The mail I send myself is usually from Wifi or my phone carrier's GSM network, but accepted via SMTP AUTH on the local machine. So which address are you using? hmm, I'm not sure. It depends on your trusted_networks setting. try running spamassassin -D and see what it logs... I'm sorry -- feeling dense, how is this supposed to help? From the headers quoted below you know what spamassassin is seeing. There's nothing in trusted networks, I don't trust anything... Here's an example. Return-Path: [EMAIL PROTECTED] Received: from mail.netconsonance.com ([unix socket]) by triceratops.netconsonance.com (Cyrus v2.3.9) with LMTPA; Tue, 01 Apr 2008 13:14:34 -0700 X-Sieve: CMU Sieve 2.3 Received: from [10.178.18.103] (m4a0e36d0.tmodns.net [208.54.14.74]) (authenticated bits=0) by mail.netconsonance.com (8.14.1/8.14.1) with ESMTP id m31KE4ui014296 for [EMAIL PROTECTED]; Tue, 1 Apr 2008 13:14:27 -0700 (PDT) (envelope-from [EMAIL PROTECTED]) X-Virus-Scanned: amavisd-new at netconsonance.com X-Spam-Flag: NO X-Spam-Score: -0.72 X-Spam-Level: X-Spam-Status: No, score=-0.72 tagged_above=-999 required=3.8 tests=[ALL_TRUSTED=-1.44, AWL=0.720] From: Jo Rhett [EMAIL PROTECTED] Subject: test awl Date: 01 Apr 2008 13:14:00 -0700 To: [EMAIL PROTECTED] X-Mailer: ChatterEmail+ for Treo 6xx/700p (3.0.8) Message-ID:[EMAIL PROTECTED] -- from the cell phone of Jo Rhett Network/Software Engineer Network Consonance -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
I'm not worried about mail from self to self. I'm annoying because
AWL is decreasing forged spam score so far that the SPF failure
doesn't catch.
On Apr 1, 2008, at 3:14 PM, Benny Pedersen wrote:
INSERT INTO `awl` VALUES('amavis', '[EMAIL PROTECTED]', '80.166', 4, -14,
'2008-04-02 00:02:15');
INSERT INTO `awl` VALUES('amavis', '[EMAIL PROTECTED]', 'none', 1, -8.5,
'2008-04-01
23:55:23');
it seems it works here, none is when its sent from localhost,
80.166 is when
sent outside localhost, so problem is ?
Sorry, I don't understand your question.
I also don't see the value in having every possible mail account need
a setting like this manually inserted. That's why I'm asking about a
fix in the module...
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: can we make AWL ignore mail from self to self?
On Apr 1, 2008, at 4:03 PM, John Hardin wrote: If you don't scan mails that you know originated from you, then they won't affect AWL for a forged message... Sorry, I'm not going to disable virus and bot protection just to avoid a mis-feature in another module. The right answer is a fix in the module. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On Apr 1, 2008, at 5:46 PM, Benny Pedersen wrote: What I am pointing out is that AWL should not be used for mail from self to self, because this is an easy forgery. explain why its a problem when awl logs ip AWL counts on the spammer not being able to forge someone you correspond with normally. so problem is that awl tracks /16 with is mostly to wide ? will problem be solved if it was /32 ? The answer to these questions is I don't know. It's not clear to me how spamassassin deals with SMTP AUTH messages from localhost. It appears that in some situations SA skips the first Received header and goes to the previous one. That's why I asked the question about which IP is used. This is usually true, but forging your own address is trivial. yep, but ip should still limit the problem very much I agree. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On Wed, 2 Apr 2008, Jo Rhett wrote: On Apr 1, 2008, at 4:03 PM, John Hardin wrote: If you don't scan mails that you know originated from you, then they won't affect AWL for a forged message... Sorry, I'm not going to disable virus and bot protection just to avoid a mis-feature in another module. Since when is SA an antivirus tool? I'm only suggesting bypassing SA for mail that originates on the local network and is destined to the local network. The right answer is a fix in the module. I don't disagree. However, it's not the only way to address the problem. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The difference between ignorance and stupidity is that the stupid desire to remain ignorant. -- Jim Bacon --- 11 days until Thomas Jefferson's 265th Birthday
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote: Bob Proulx wrote: I disagree with the premise that it is hard to forge mail from someone you correspond with frequently. It is equally easy to forge. Easy to forge, but who to forge? Hard for a spammer to know who I correspond with frequently. Myself is the only one a spammer could guess. Who to forge? The answer is Everyone! Any address that can be obtained from a spam-virus infected PC and any address that can be harvested from a web page. Forge them all. They are (mostly) valid email addresses and will pass sender verification. Send To: and From: all of them. Bob
Re: can we make AWL ignore mail from self to self?
On Mar 28, 2008, at 6:21 PM, Theo Van Dinter wrote: On Fri, Mar 28, 2008 at 06:09:03PM -0700, Jo Rhett wrote: I think that mail from self to self should be ignored by the AWL. (it's harder to forged mail from a regular correspondent, so this makes AWL more useful) If you know the mail is from you, don't waste the resources scanning the message at all. This was a spam I'm talking about. I'm not worried about mail from self to self. I'm annoying because AWL is decreasing forged spam score so far that the SPF failure doesn't catch. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
Benn, you are missing the point. AWL is working very well for our needs. What I am pointing out is that AWL should not be used for mail from self to self, because this is an easy forgery. AWL counts on the spammer not being able to forge someone you correspond with normally. This is usually true, but forging your own address is trivial. On Mar 28, 2008, at 6:48 PM, Benny Pedersen wrote: On Sat, March 29, 2008 02:09, Jo Rhett wrote: I send myself a lot of email from my phone. So AWL properly scores me well. and the sender ip with a fuss of /16 I just got a piece of SPAM which should have scored 12.something that got a -6 from the AWL. ok I think that mail from self to self should be ignored by the AWL. (it's harder to forged mail from a regular correspondent, so this makes AWL more useful) better configure awl to weight scores better to what trustness you want from it perldoc Mail::SpamAssassin::Plugin::AWL see the factor setting in usersettings Benny Pedersen Need more webspace ? http://www.servage.net/?coupon=cust37098 -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On Mar 29, 2008, at 3:21 AM, Justin Mason wrote: the AWL is keyed on email address and /16 of the sending IP address, so this may warrant more investigation. could you post the Received hdrs from the spam that hit the AWL, and a ham that properly hits the AWL? I still believe that self-self would make a good exemption for AWL. Sorry, I don't the original messages any more. (I looked) But it wouldn't surprise me if the /16 matched. The mail I send myself is usually from Wifi or my phone carrier's GSM network, but accepted via SMTP AUTH on the local machine. So which address are you using? Here's an example. Return-Path: [EMAIL PROTECTED] Received: from mail.netconsonance.com ([unix socket]) by triceratops.netconsonance.com (Cyrus v2.3.9) with LMTPA; Tue, 01 Apr 2008 13:14:34 -0700 X-Sieve: CMU Sieve 2.3 Received: from [10.178.18.103] (m4a0e36d0.tmodns.net [208.54.14.74]) (authenticated bits=0) by mail.netconsonance.com (8.14.1/8.14.1) with ESMTP id m31KE4ui014296 for [EMAIL PROTECTED]; Tue, 1 Apr 2008 13:14:27 -0700 (PDT) (envelope-from [EMAIL PROTECTED]) X-Virus-Scanned: amavisd-new at netconsonance.com X-Spam-Flag: NO X-Spam-Score: -0.72 X-Spam-Level: X-Spam-Status: No, score=-0.72 tagged_above=-999 required=3.8 tests=[ALL_TRUSTED=-1.44, AWL=0.720] From: Jo Rhett [EMAIL PROTECTED] Subject: test awl Date: 01 Apr 2008 13:14:00 -0700 To: [EMAIL PROTECTED] X-Mailer: ChatterEmail+ for Treo 6xx/700p (3.0.8) Message-ID:[EMAIL PROTECTED] -- from the cell phone of Jo Rhett Network/Software Engineer Network Consonance -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
Jo Rhett wrote:
Benn, you are missing the point. AWL is working very well for our
needs.
I have never been fond of AWL because the information it relies upon,
the mail headers, is very easy to forge. It depends too much upon
trusting the sender. And in the case of spam that trust model is
already a problem. I normally disable AWL entirely. I just don't
feel it is benefiting enough to be worth the problems. It was an idea
that made a good experiment but unfortunately IMNHO didn't work out
very well in practice.
If the mail headers could be trusted then the result would be
completely different. Perhaps AWL would be very effective if it were
only applied to headers when those headers could be trusted. Such as
when trusted_networks, DKIM or other trust model exists.
What I am pointing out is that AWL should not be used for mail from
self to self, because this is an easy forgery.
It is all very easy to forge. But self to self is very easy for the
recipient to spot as a forgery. (Unless they have a short memory and
are very gullible. :-)
AWL counts on the spammer not being able to forge someone you
correspond with normally. This is usually true, but forging your
own address is trivial.
I disagree with the premise that it is hard to forge mail from someone
you correspond with frequently. It is equally easy to forge. With
signed headers, whitelist_from_{rcvd,spf,other} it may be possible to
catch targeted forgeries but in general it isn't solved yet.
Bob
Re: can we make AWL ignore mail from self to self?
Jo Rhett writes: On Mar 29, 2008, at 3:21 AM, Justin Mason wrote: the AWL is keyed on email address and /16 of the sending IP address, so this may warrant more investigation. could you post the Received hdrs from the spam that hit the AWL, and a ham that properly hits the AWL? I still believe that self-self would make a good exemption for AWL. Sorry, I don't the original messages any more. (I looked) But it wouldn't surprise me if the /16 matched. The mail I send myself is usually from Wifi or my phone carrier's GSM network, but accepted via SMTP AUTH on the local machine. So which address are you using? hmm, I'm not sure. It depends on your trusted_networks setting. try running spamassassin -D and see what it logs... --j. Here's an example. Return-Path: [EMAIL PROTECTED] Received: from mail.netconsonance.com ([unix socket]) by triceratops.netconsonance.com (Cyrus v2.3.9) with LMTPA; Tue, 01 Apr 2008 13:14:34 -0700 X-Sieve: CMU Sieve 2.3 Received: from [10.178.18.103] (m4a0e36d0.tmodns.net [208.54.14.74]) (authenticated bits=0) by mail.netconsonance.com (8.14.1/8.14.1) with ESMTP id m31KE4ui014296 for [EMAIL PROTECTED]; Tue, 1 Apr 2008 13:14:27 -0700 (PDT) (envelope-from [EMAIL PROTECTED]) X-Virus-Scanned: amavisd-new at netconsonance.com X-Spam-Flag: NO X-Spam-Score: -0.72 X-Spam-Level: X-Spam-Status: No, score=-0.72 tagged_above=-999 required=3.8 tests=[ALL_TRUSTED=-1.44, AWL=0.720] From: Jo Rhett [EMAIL PROTECTED] Subject: test awl Date: 01 Apr 2008 13:14:00 -0700 To: [EMAIL PROTECTED] X-Mailer: ChatterEmail+ for Treo 6xx/700p (3.0.8) Message-ID:[EMAIL PROTECTED] -- from the cell phone of Jo Rhett Network/Software Engineer Network Consonance -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness
Re: can we make AWL ignore mail from self to self?
On Tue, April 1, 2008 21:43, Jo Rhett wrote:
On Mar 28, 2008, at 6:21 PM, Theo Van Dinter wrote:
On Fri, Mar 28, 2008 at 06:09:03PM -0700, Jo Rhett wrote:
I think that mail from self to self should be ignored by the AWL.
(it's harder to forged mail from a regular correspondent, so this
makes AWL more useful)
If you know the mail is from you, don't waste the resources
scanning the
message at all.
This was a spam I'm talking about.
I'm not worried about mail from self to self. I'm annoying because
AWL is decreasing forged spam score so far that the SPF failure
doesn't catch.
INSERT INTO `awl` VALUES('amavis', '[EMAIL PROTECTED]', '80.166', 4, -14,
'2008-04-02 00:02:15');
INSERT INTO `awl` VALUES('amavis', '[EMAIL PROTECTED]', 'none', 1, -8.5,
'2008-04-01
23:55:23');
it seems it works here, none is when its sent from localhost, 80.166 is when
sent outside localhost, so problem is ?
well i added the last field for when it was last updated to be used when i
cleanup the awl table
Benny Pedersen
Need more webspace ? http://www.servage.net/?coupon=cust37098
Re: can we make AWL ignore mail from self to self?
On Tue, 1 Apr 2008, Jo Rhett wrote: On Mar 28, 2008, at 6:21 PM, Theo Van Dinter wrote: On Fri, Mar 28, 2008 at 06:09:03PM -0700, Jo Rhett wrote: I think that mail from self to self should be ignored by the AWL. (it's harder to forged mail from a regular correspondent, so this makes AWL more useful) If you know the mail is from you, don't waste the resources scanning the message at all. This was a spam I'm talking about. I'm not worried about mail from self to self. I'm annoying because AWL is decreasing forged spam score so far that the SPF failure doesn't catch. If you don't scan mails that you know originated from you, then they won't affect AWL for a forged message... -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...every time I sit down in front of a Windows machine I feel as if the computer is just a place for the manufacturers to put their advertising.-- fwadling on Y! SCOX -- Today: April Fools' day
Re: can we make AWL ignore mail from self to self?
On Tue, April 1, 2008 21:45, Jo Rhett wrote: Benn, you are missing the point. AWL is working very well for our needs. good What I am pointing out is that AWL should not be used for mail from self to self, because this is an easy forgery. explain why its a problem when awl logs ip AWL counts on the spammer not being able to forge someone you correspond with normally. so problem is that awl tracks /16 with is mostly to wide ? will problem be solved if it was /32 ? This is usually true, but forging your own address is trivial. yep, but ip should still limit the problem very much Benny Pedersen Need more webspace ? http://www.servage.net/?coupon=cust37098
Re: can we make AWL ignore mail from self to self?
Jo Rhett writes: I send myself a lot of email from my phone. So AWL properly scores me well. I just got a piece of SPAM which should have scored 12.something that got a -6 from the AWL. I think that mail from self to self should be ignored by the AWL. (it's harder to forged mail from a regular correspondent, so this makes AWL more useful) the AWL is keyed on email address and /16 of the sending IP address, so this may warrant more investigation. could you post the Received hdrs from the spam that hit the AWL, and a ham that properly hits the AWL? --j.
Re: can we make AWL ignore mail from self to self?
On Fri, Mar 28, 2008 at 06:09:03PM -0700, Jo Rhett wrote: I think that mail from self to self should be ignored by the AWL. (it's harder to forged mail from a regular correspondent, so this makes AWL more useful) If you know the mail is from you, don't waste the resources scanning the message at all. -- Randomly Selected Tagline: Any sufficiently perverted technology is indistinguishable from Perl. - Unknown pgpurwNOMgu0U.pgp Description: PGP signature
Re: can we make AWL ignore mail from self to self?
On Sat, March 29, 2008 02:09, Jo Rhett wrote: I send myself a lot of email from my phone. So AWL properly scores me well. and the sender ip with a fuss of /16 I just got a piece of SPAM which should have scored 12.something that got a -6 from the AWL. ok I think that mail from self to self should be ignored by the AWL. (it's harder to forged mail from a regular correspondent, so this makes AWL more useful) better configure awl to weight scores better to what trustness you want from it perldoc Mail::SpamAssassin::Plugin::AWL see the factor setting in usersettings Benny Pedersen Need more webspace ? http://www.servage.net/?coupon=cust37098
