DOS_OE_TO_MX rule and trusted_networks
I'm using SpamAssassin 3.3.1 together with Amavis 2.6.4 on one server with Postfix. All our customers have public static IP addresses on their PC's 213.x.x.x/y. We use only one mailserver with one public IP address from the 213.x.x.x/y range mentioned earlier for both the incoming and outgoing mail traffic to/from all of our domains. We are ISP. Our customer complained about false positive mail with DOS_OE_TO_MX. How exactly this rule works? Should I add all my range 213.x.x.x/y to the trusted_networks and my mailserver should be added to the internal_networks? I guess, that the DOS_OE_TO_MX rule says, that someone from the internet/outside world is connected directly to my mailserver, says it sends mail using Outlook Express and sends the mails to my domains. He does not uses his ISP's mailserver for sending mails. Right? I suggest something like this: trusted_networks 213.x.x.x/y # all our public ip addresses range internal_networks 213.0.0.5 # let's say that's our mailserver's IP I have none lines with trusted_networks and internal_networks in my config now. The doc says: Trusted in this case means that relay hosts on these networks are considered to not be potentially operated by spammers, open relays, or open proxies. A trusted host could conceivably relay spam, but will not originate it, and will not forge header data But I think, that almost everone is sometimes infected and sends spam... So I'm confused howto setup my system. Kind regards, Tomas
Blacklisting By Mail Server Rather Than By An Email Address
I have recently changed the company that hosts my email and they use SpamAssassin. In the SpamAssassin Configuration I have entered several email addresses in the Blacklist which has worked fine. One thing I have noticed with one particular spammer is that they send out their emails using fake sender addresses which bear no resemblance to either their domain (XXX.com) or their mail server address (mail.XXX.com). When an email is received in this form it is of no use to enter the fake sender email address into the blacklist because they use something very different the next time. Is there a way of blacklisting the mail server which would prevent any mail originating from that server being received? Many thanks -- View this message in context: http://old.nabble.com/Blacklisting-By-Mail-Server-Rather-Than-By-An-Email-Address-tp32622830p32622830.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: DOS_OE_TO_MX rule and trusted_networks
On Mon, 10 Oct 2011 12:19:56 +0200 (CEST), Tomas Macek wrote: I suggest something like this: trusted_networks 213.x.x.x/y # all our public ip addresses range internal_networks 213.0.0.5 # let's say that's our mailserver's IP the above should only list all the mailserver(s) you have as isp, not custommers ips in network, same with trusted_network if you forward mails in clusters add cluster ips as trusted_networks, not internal_network But I think, that almost everone is sometimes infected and sends spam... So I'm confused howto setup my system. verify with above change spamassassin -t msg | less your clients still use sasl auth even from isp ip ranges ?, thats the correct way to solve most problems, but are unrelated to the error you see, here i use amavisd-new and have seperate policy banks for submision and smtpd incomming mails, smtpd is never originating mails here, submission reject non sasl authed clients hope that helps, if not post sample on pastebin, and just mangle sender donain with example.org
Re: Blacklisting By Mail Server Rather Than By An Email Address
On Mon, 10 Oct 2011 03:47:27 -0700 (PDT), johnjinsf wrote: Is there a way of blacklisting the mail server which would prevent any mail originating from that server being received? is sender domain(s) rfc-ignorant ?, sendmail -bv ab...@example.org sendmail -bv postmas...@example.org, what results come back to you ?, root is geting this mails :=) if domain is rfc-ignorant please list them as so here http://www.rfc-ignorant.org/ next step depends on meta rules in spamassassin i got a new email address was something i seen last year, but not much anymore :=) suggested plugins to add SAGREY, with caches first time senders
Re: DOS_OE_TO_MX rule and trusted_networks
On Mon, 10 Oct 2011, Benny Pedersen wrote: On Mon, 10 Oct 2011 12:19:56 +0200 (CEST), Tomas Macek wrote: I suggest something like this: trusted_networks 213.x.x.x/y # all our public ip addresses range internal_networks 213.0.0.5 # let's say that's our mailserver's IP the above should only list all the mailserver(s) you have as isp, not custommers ips in network, same with trusted_network OK, this should be good: trusted_networks 213.0.0.5 213.0.0.10 # primary mx IP and backup mx IP internal_networks 213.0.0.5 # only the IP of primary mx Right? if you forward mails in clusters add cluster ips as trusted_networks, not internal_network But I think, that almost everone is sometimes infected and sends spam... So I'm confused howto setup my system. verify with above change spamassassin -t msg | less your clients still use sasl auth even from isp ip ranges ?, thats the correct way to solve most problems, but are unrelated to the error you see, here i use amavisd-new and have seperate policy banks for submision and smtpd incomming mails, smtpd is never originating mails here, submission reject non sasl authed clients I know, that smtp auth often solves many problems, but today I cannot force all our clients to use it. So that means, that someone uses it, but mostly not. hope that helps, if not post sample on pastebin, and just mangle sender donain with example.org But there is still the question what bad happened when DOS_OE_TO_MX matched the message? The client sent the mail from internal network 213.x.x.x/y from his public static IP through our mailserver into some mailbox hosted on our mailserver. I think I must have some misconfiguration in spamassassin...
Re: Blacklisting By Mail Server Rather Than By An Email Address
On Mon, 10 Oct 2011 03:47:27 -0700 (PDT) johnjinsf wrote: Is there a way of blacklisting the mail server which would prevent any mail originating from that server being received? I don't think there is a way to blacklist a server unless the provider allows you to create SA rules. If it does then: header BAD_SERVER X-Spam-Relays-Untrusted =~ /helo=mail.XXX.com /i score BAD_SERVER 3.0
Re: Blacklisting By Mail Server Rather Than By An Email Address
On Mon, 10 Oct 2011 13:47:28 +0100 RW wrote: On Mon, 10 Oct 2011 03:47:27 -0700 (PDT) johnjinsf wrote: Is there a way of blacklisting the mail server which would prevent any mail originating from that server being received? I don't think there is a way to blacklist a server unless the provider allows you to create SA rules. If it does then: header BAD_SERVER X-Spam-Relays-Untrusted =~ /helo=mail.XXX.com /i score BAD_SERVER 3.0 Forgot to escape the dots /helo=mail\.XXX\.com/
Re: DOS_OE_TO_MX rule and trusted_networks
On Oct 10, 2011, at 1:14 PM, Tomas Macek wrote: hope that helps, if not post sample on pastebin, and just mangle sender donain with example.org But there is still the question what bad happened when DOS_OE_TO_MX matched the message? The client sent the mail from internal network 213.x.x.x/y from his public static IP through our mailserver into some mailbox hosted on our mailserver. I think I must have some misconfiguration in spamassassin... I believe the right variable to set up would be msa_networks instead of trusted_networks. We had similar issues, with our pop-before-smtp users relaying through our mail servers to domains that we hosted. In the end, we lowered the DOS_OE_TO_MX score, since we were unable to dynamically specify msa_networks variable. I know that there was a plugin for that: http://wiki.apache.org/spamassassin/POPAuthPlugin We eventually developed the SA plugin, that could dynamically specify the msa_networks, but in the end we settled for lowering the score and advising the users to use SMTP AUTH which bypasses the DOS_OE_TO_MX... regards, Jernej
Re: New Bayes like paradigm
On 9/28/2011 8:02 AM, dar...@chaosreigns.com wrote: On 09/28, Marc Perkel wrote: You would only have to test the rule combinations that the message actually triggered. So if it hit 10 rules then it would be 1024 combinations. Seems not to be unreasonable to me. You definitely have a good point that it would only be necessary to track the combinations that actually show up in emails, however 1024 is only the possible combinations from one set of 10 rules. The number of combinations in the actual corpora would be much higher. I'll try to get you a number. You wouldn't have to store all combinations. You could just do up to 3 levels and only the combinations that actually occur and use a hash to look up the combinations. -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Blacklisting based on SPF
On 10/7/2011 12:50 AM, Benny Pedersen wrote: On 7 Oct 2011 00:28:49 -, John Levine wrote: Nobody with any interest in delivering the mail that their users want. The error rate is much, much too high. how ? All forwarded email would fail SPF testing. You would be blocking all hosted spam filtering services for example. -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Blacklisting based on SPF
On 10/10/11 9:00 AM, Marc Perkel supp...@junkemailfilter.com wrote: On 10/7/2011 12:50 AM, Benny Pedersen wrote: On 7 Oct 2011 00:28:49 -, John Levine wrote: Nobody with any interest in delivering the mail that their users want. The error rate is much, much too high. how ? All forwarded email would fail SPF testing. You would be blocking all hosted spam filtering services for example. then you aren't doing it right. If the hosted filtering is egress, then the address ranges of your egress filter provider should be in your SPF statement. If the hosted filtering is ingress, then the address ranges of your ingress filter provider should be in your trusted-networks, so that spf will look at the last-untrusted address for the source. Mail-lists running on sane software will change the envelope address, so there is no problem there. So, what other bizarre corner cases are you talking about that break SPF? -- Daniel J McDonald, CCIE # 2495, CISSP # 78281
Re: DOS_OE_TO_MX rule and trusted_networks
On Mon, 10 Oct 2011 13:14:21 +0200 (CEST), Tomas Macek wrote: On Mon, 10 Oct 2011, Benny Pedersen wrote: On Mon, 10 Oct 2011 12:19:56 +0200 (CEST), Tomas Macek wrote: I suggest something like this: trusted_networks 213.x.x.x/y # all our public ip addresses range internal_networks 213.0.0.5 # let's say that's our mailserver's IP the above should only list all the mailserver(s) you have as isp, not custommers ips in network, same with trusted_network OK, this should be good: trusted_networks 213.0.0.5 213.0.0.10 # primary mx IP and backup mx IP internal_networks 213.0.0.5 # only the IP of primary mx Right? backup is imho also internal, only ecception is if its another isp if you forward mails in clusters add cluster ips as trusted_networks, not internal_network But I think, that almost everone is sometimes infected and sends spam... So I'm confused howto setup my system. verify with above change spamassassin -t msg | less your clients still use sasl auth even from isp ip ranges ?, thats the correct way to solve most problems, but are unrelated to the error you see, here i use amavisd-new and have seperate policy banks for submision and smtpd incomming mails, smtpd is never originating mails here, submission reject non sasl authed clients I know, that smtp auth often solves many problems, but today I cannot force all our clients to use it. So that means, that someone uses it, but mostly not. i remember clients problems can be isps problems understanding as well :=) what kind of problems remain to be solved if sasl auth is only way to send mail ? what logs say ? hope that helps, if not post sample on pastebin, and just mangle sender donain with example.org But there is still the question what bad happened when DOS_OE_TO_MX matched the message? yes, check if msg is with ALL_TRUSTED test or not The client sent the mail from internal network 213.x.x.x/y from his public static IP through our mailserver into some mailbox hosted on our mailserver. I think I must have some misconfiguration in spamassassin... if ALL_TRUSTED agree its sure a bug, but imho its not scoring 5.0 ?
Re: sa-learn --backup
On Mon, 2011-10-10 at 01:03 +0100, RW wrote: On Sun, 09 Oct 2011 08:31:16 -0500 Chris wrote: I've been running the above command as a cronjob nightly for years now suddenly in the past few days I've been getting this as the output of the cronjob: bayes: cannot open bayes databases /home/chris/.spamassassin/bayes_* R/W: lock failed: Interrupted system call ERROR: Bayes backup returned an error, please re-run with -D for more information The command line I'm using is: cd /home/chris/.spamassassin /usr/local/bin/sa-learn --backup /home/chris/SABackup/backup.txt Running with -D shows nothing and running from the command line works fine. What could be causing the error output? Have you tried changing the crontab entry to: sa-learn -D --backup /home/chris/SABackup/backup.txt 2/home/chris/SABackup/debug Thanks, the cronjob ran ok last night, I'll keep an eye on it and check the debug output next time it fails. -- Chris KeyID 0xE372A7DA98E6705C 31.11°N 97.89°W (Elev. 1092 ft) 09:55:37 up 8 days, 18:05, 1 user, load average: 0.32, 0.59, 0.52 signature.asc Description: This is a digitally signed message part
Re: Blacklisting based on SPF
On Mon, 10 Oct 2011 07:00:48 -0700 Marc Perkel supp...@junkemailfilter.com wrote: [Blocking SPF fail mail] All forwarded email would fail SPF testing. You would be blocking all hosted spam filtering services for example. Nonsense. If someone uses a hosted spam filtering servic for inbound mail, then that person should turn off SPF checking on the back-end completely; checking SPF and applying policy is the job of the hosted spam filter. (If you're using a hosted anti-spam service that does *not* allow you to apply fine-grained SPF policies, then it's time to switch.) If someone uses a hosted filtering service for outbound mail, then he/she just needs to publish appropriate SPF records listing the service's egress IP addresses. Regards, David.
Re: sa-learn --backup
On 10/9/2011 9:31 AM, Chris wrote: cd /home/chris/.spamassassin /usr/local/bin/sa-learn --backup /home/chris/SABackup/backup.txt Out of curiosity, is there a reason for the 'cd'? sa-learn gets the database path from the config (or defaults to ~/.spamassassin/bayes). The current directory is irrelevant. -- Bowie
How to create spam score list for sample email messages
Hi, I have a mail receiving setup where in Postfix (2.6.6) is the MTA and then I have amavisd-new (with spamassassin and CLamAV) as content filter. I have enabled spam report header in my amavisd-new conf file. I want to create a report of sample emails with the spam scores generated in accordance with permissible limits after deploying the spamassassin updated rulesets. For that I am trying out on a shell script providing with my test email messages to the following (with following command): spamassassin -C /etc/amavisd.conf -e --progress testemail.eml and be able to create a report, that would enlist the spam scores of all email messages that have been parsed by the above tool. Is it possible?, actually I am unable to generate the spam scores in any output form via the above command to be added in the report. Moreover I am using amavisd-new config file here, is it a right approach? Will the above command affect any kind of Bayesian learning of the spamassassin setup ?, I don't want to do that. Thanks Ashish Sharma
Re: New Bayes like paradigm
On 10/10, Marc Perkel wrote: On 9/28/2011 8:02 AM, dar...@chaosreigns.com wrote: On 09/28, Marc Perkel wrote: You would only have to test the rule combinations that the message actually triggered. So if it hit 10 rules then it would be 1024 combinations. Seems not to be unreasonable to me. You definitely have a good point that it would only be necessary to track the combinations that actually show up in emails, however 1024 is only the possible combinations from one set of 10 rules. The number of combinations in the actual corpora would be much higher. I'll try to get you a number. You wouldn't have to store all combinations. You could just do up to 3 levels and only the combinations that actually occur and use a hash to look up the combinations. I never said storage would be a problem. I agree you could just store a relatively small number that were most useful. The problems are: 1) The many years it would take to find useful rule combinations by trying one possibility per masscheck run. 2) The hundreds of times as much (masscheck) data we'd need to get an accurate re-score using all rule combinations existing in the corpora. There is still the possibility of doing an analysis of what combinations of rules hit false-negatives significantly more often than they hit non-spam. (Or false-positives vs. spam.) -- Immorality: The morality of those who are having a better time - Henry Louis Mencken http://www.ChaosReigns.com
Re: How to create spam score list for sample email messages
On Mon, 2011-10-10 at 15:36 +, Sharma, Ashish wrote: I want to create a report of sample emails with the spam scores generated in accordance with permissible limits after deploying the spamassassin updated rulesets. For that I am trying out on a shell script providing with my test email messages I do something similar, but keep my test messages as separate text files in a directory because I find that easier to manage. I do approximately this on a computer that's entirely separate from my mail host and runs its own copy of spamd so I can mess around with its rule sets and configuration without upsetting the live copy of SA. The testing SA runs in effectively the same configuration as my live SA because the test rig has an identical set of SA config files: when I'm happy with the test operation I export the entire set of configuration files to the live system and then restart spamd. Here's the guts of the test system: for f in testdata/*.txt do spamc testdata/$f | grep '^X-spam-status: ' result.txt done analysis_prog result.txt rm result.txt My analysis program is an awk script: that or Perl are probably the weapons of choice for writing this type of program. You probably need to feed the messages to amavis-new since it is creating a special header, rather than to spamc/spamd as I do, but I question whether your command line is right since amavis has direct access to the Perl modules that make up spamassassin. Disclaimer: the previous paragraph contains almost everything I know about amavis-new. Somebody else may be able to help with the amavis-new command line, but not me since I don't use it. What I do know is that Postfix passes a message at a time to spamc/spamd so its entirely probable it does the same with amavis-new if you're running that as a Postfix service. Martin
Re: How to create spam score list for sample email messages
On Mon, 10 Oct 2011 17:29:08 +0100 Martin Gregorie wrote: for f in testdata/*.txt do spamc testdata/$f | grep '^X-spam-status: ' result.txt For that to work you need the setting fold_headers 0
Re: Blacklisting By Mail Server Rather Than By An Email Address
RW-15 wrote: I don't think there is a way to blacklist a server unless the provider allows you to create SA rules. Many thanks for your replies and suggestions. I haven't seen where my hoster allows for users to create rules, but I'll open a ticket with their help desk to ask if they do. -- View this message in context: http://old.nabble.com/Blacklisting-By-Mail-Server-Rather-Than-By-An-Email-Address-tp32622830p32627953.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: How to create spam score list for sample email messages
On Mon, 2011-10-10 at 20:08 +0100, RW wrote:
On Mon, 10 Oct 2011 17:29:08 +0100
Martin Gregorie wrote:
for f in testdata/*.txt
do
spamc testdata/$f | grep '^X-spam-status: ' result.txt
For that to work you need the setting
fold_headers 0
Fair comment: I use gawk rather than grep and my filter looks like this:
spamc -l $s | gawk '
BEGIN { tag=0 }
/^X-Spam/ { tag=1; print; next }
/^ / || /^\t/ { if (tag==1) { print } next }
{ tag = 0 }
' | where_ever
I don't use 'fold_headers 0' because I don't want *any* differences
between my test config and the live one. From the look of that filter I
obviously ran into the folded line thing solved the problem with gawk.
Martin
Re: DOS_OE_TO_MX rule and trusted_networks
On Mon, 10 Oct 2011, Benny Pedersen wrote: On Mon, 10 Oct 2011 13:14:21 +0200 (CEST), Tomas Macek wrote: On Mon, 10 Oct 2011, Benny Pedersen wrote: On Mon, 10 Oct 2011 12:19:56 +0200 (CEST), Tomas Macek wrote: I suggest something like this: trusted_networks 213.x.x.x/y # all our public ip addresses range internal_networks 213.0.0.5 # let's say that's our mailserver's IP the above should only list all the mailserver(s) you have as isp, not custommers ips in network, same with trusted_network OK, this should be good: trusted_networks 213.0.0.5 213.0.0.10 # primary mx IP and backup mx IP internal_networks 213.0.0.5 # only the IP of primary mx Right? backup is imho also internal, only ecception is if its another isp it's our server in another isp's network :-)) if you forward mails in clusters add cluster ips as trusted_networks, not internal_network But I think, that almost everone is sometimes infected and sends spam... So I'm confused howto setup my system. verify with above change spamassassin -t msg | less your clients still use sasl auth even from isp ip ranges ?, thats the correct way to solve most problems, but are unrelated to the error you see, here i use amavisd-new and have seperate policy banks for submision and smtpd incomming mails, smtpd is never originating mails here, submission reject non sasl authed clients I know, that smtp auth often solves many problems, but today I cannot force all our clients to use it. So that means, that someone uses it, but mostly not. i remember clients problems can be isps problems understanding as well :=) what kind of problems remain to be solved if sasl auth is only way to send mail ? At the early begginings we did not have smtp auth, so now it's too late to force the clients to setup this on their OE or Thunderbird or something similar. So the solution is easy: to setup about ten thousands clients to use smtp auth :-)) what logs say ? hope that helps, if not post sample on pastebin, and just mangle sender donain with example.org But there is still the question what bad happened when DOS_OE_TO_MX matched the message? yes, check if msg is with ALL_TRUSTED test or not The client sent the mail from internal network 213.x.x.x/y from his public static IP through our mailserver into some mailbox hosted on our mailserver. I think I must have some misconfiguration in spamassassin... if ALL_TRUSTED agree its sure a bug, but imho its not scoring 5.0 ? No, there is not ALL_TRUSTED in the headers. I'm sorry, I did not write here the rules that matched the message, so here it is: X-Spam-Status: Yes, score=5.988 tagged_above=3 required=5 tests=[DOS_OE_TO_MX=3.086, FSL_HELO_NON_FQDN_1=0.001, HELO_NO_DOMAIN=0.001, HELO_OEM=2.899, HTML_MESSAGE=0.001] autolearn=disabled What the rule DOS_OE_TO_MX exactly does? This (http://www.gossamer-threads.com/lists/spamassassin/users/110344) for example says, that OE from some outside network used our mailserver to send mail to our network. The outside OE client did not used it's isps mailserver. Is that right? If so, I think I must somewhere say, what our network and I don't know where... Tomas
