Spam Assassin
> > Does the Spam Assassin work on Mac's? > > Thank you for your help. -- Michelle Acosta Bookkeeper/Office Manager The TEAK Fellowship 16 West 22nd Street, 3rd Fl. New York, NY 10010 Tel: (212) 288-6678, ext. 109 Fax: (212) 288-5058
Re: Spam Assassin
Am/On Fri, 16 May 2008 11:18:04 -0400 schrieb/wrote Michelle Acosta: >> >> Does the Spam Assassin work on Mac's? sure it does. <http://wiki.apache.org/spamassassin/SpamAssassin_on_Mac_OS_X_Server> <http://osx.topicdesk.com/content/category/4/18/41/> Thanks and all the best Matthias
spam assassin dies
I'm getting occasional spam in my inbox that doesn't run through spam assassin. I turned on procmail and discovered that spam assassin is dying with -6. Details from the logs: procmail: Executing "/usr/local/bin/spamassassin" __db_assert: "0" failed: file "../dist/../common/db_err.c", line 200 procmail: [8883] Wed Oct 18 07:29:13 2006 procmail: Program failure (-6) of "/usr/local/bin/spamassassin" procmail: Rescue of unfiltered data succeeded Spam Assassin does run on many occasions. The problem is these failures result in high levels of spam. Any thoughts? Jeff
Parallelizing Spam Assassin
Hi I was measuring how quickly could SA [spam assassin] process spams when several SA processes are run in parallel over separate mbox files. I used a 8 core machine. Below are the numbers when I forked different number of processes. Fork = 8; Rate = 57 msgs/sec Fork = 4; Rate = 44 msgs/sec Fork = 1; Rate = 22 msgs/sec I ran freshly build SA with Bayes and DNSBL turned off. Why am I not seeing a linear increase in the throughput? Is a file locking creating the bottleneck? If yes, which particular file is being locked? If no, what could be the reason for this? thnx -- View this message in context: http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24751958.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Spam Assassin/Plesk
Hi All, I am hoping someone can help me with a couple of newbie questions. In Plesk, I cannot see the user’s email in the Bayesian Training portion All it shows is a button to clear the database, but what I want is a way to retrain the ‘spam’ham’ etc…Any ideas? 2. Can anybody elaborate on this -RPM: SpamAssassin RPMs can be built directly from the tar file. · Simply run: rpmbuild -tb Mail-SpamAssassin-3.1.0.tar.gz Do I run this after having uploaded the .tar to my server? I realize these query’s are a bit disjointed, but any help is much appreciated. Peter --- Peter McEwen Web Developer Nuclear Age Peace Foundation PMB 121 1187 Coast Village Rd., Suite 1 Santa Barbara, CA 93108-2794 Ph: 805 965-3443 Fax: 805 568-0466 www.wagingpeace.org
Spam Assassin Load Balancing
I've got a few questions for everyone concerning load balancing Spam Assassin. I've currently got a 10 node spam assassin cluster, and have been using round-robin DNS to load balance across them, but Im wondering what other people recommend? The benefit of DNS seems to be that spamc will try all nodes in the cluster, as each proceeding node times out. With a rounb robin policy on a hardware load balancer, once the connection is routed to a specific 'worker bee', if that machine times out, the request will fail, and the mail wont get scanned. However, more intelligent hardware load balancing setups can monitor the work on each node, and take it out of service as necessary. What methods do other people use? Also, when running a round-robin based cluster, is there any problem having a mix of machines with different performance capacities? i.e. If I have a 10 node cluster, and 3 of the servers are much slower than the others, will it impact performance of the cluster as a whole? Even if I limit the number of spamd that run to a lower value than the higher performance machines? Thanks in advance!
Re: spam assassin dies
Jeff Fulmer wrote: I'm getting occasional spam in my inbox that doesn't run through spam assassin. I turned on procmail and discovered that spam assassin is dying with -6. Details from the logs: procmail: Executing "/usr/local/bin/spamassassin" __db_assert: "0" failed: file "../dist/../common/db_err.c", line 200 procmail: [8883] Wed Oct 18 07:29:13 2006 procmail: Program failure (-6) of "/usr/local/bin/spamassassin" procmail: Rescue of unfiltered data succeeded Spam Assassin does run on many occasions. The problem is these failures result in high levels of spam. Any thoughts? It looks like a problem with whatever storage backend you're using for bayes or awl. Daryl
Problems running Spam Assassin
Hi, I installed the latest SpamAssassin on my server. At first all my tests looked good, apart from load. So I setup spamc and spamd and everything seemed great, for a short while at least. A day later my mqueue had about 1500 messages in it, most with the error "local mailer (/usr/bin/procmail) exited with EX_TEMPFAIL". This seems to be coming up if the mailbox is full or the email is to an address that doesn't exist. It seemed that about every hour or so Sendmail was trying to flush out these messages, causing 1000's of processes to be started and making the server freeze up. Despite my Sendmail config having define(`confMAX_DAEMON_CHILDREN', `12')dnl In my procmailrc file I have:- DROPPRIVS=yes :0fw: spamassassin.lock * < 256000 | spamc The SpamAssassin daemon was started with /usr/bin/spamd -d -u nobody At some point all mail stopped coming in. When I looked at the maillog file it had lots of lines like:- mkdir /root/.spamassassin: Permission denied Which I guess was causing the problem. This wasn't a problem before so I'm not sure why it happened. Any clues? Basically I need to set things up so that when sendmail trys to flush I don't get my server falling over. Emails that are sent to addresses that don't exist that are currently getting the error "local mailer (/usr/bin/procmail) exited with EX_TEMPFAIL" be delete from the queue automatically. Ideally I'd like to give each different virtual server I have it's own possibly spam folder. I'm using Webmin and have a 100 or so Virtual servers so if anyone knows a good automated way of doing this that would be great. Either way I can't have things go down again otherwise I'll loose all my clients! And SpamAssassin working again. At first it was just marking emails with [spam] in the subject. Then Yesterday It then also started changing the message to an attachment and having "Spam detection software, running on the system "ns.cosmicsitehosting.com", has identified this incoming email as possible spam..." in the message text. I've no idea what was changed so that this started happening. I didn't think I changed anything. Then last night it stopped sending any emails. Please help! Thanks in advance. Oh by the way my local.cf file contains required_hits 10 rewrite_header Subject [SPAM] report_safe 1 use_bayes 1 skip_rbl_checks 1 use_pyzor 1 -- View this message in context: http://www.nabble.com/Problems-running-Spam-Assassin-tf2664618.html#a7431008 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Spam Assassin Logfile analysis
Hello I can't find much info out there as to what people do in terms of analysing logfiles etc to tune their SA setup's, so I'll start with what I'm doing at the moment. I'm using the Win32 Pop3 proxy version of Spamassassin V3.1.5.1 (from (http://sourceforge.net/projects/sawin32/) It currently doesn't write a lot of info to the standard logfile, so I'm using grep (I'm using something called PowerGrep, which works on Windows) to parse several mail mbox files. Here's the Regex I'm using: ^From \- ([A-Z][a-z]{2} [A-Z][a-z]{2} [0-9]{2} [0-9]{2}\:[0-9]{2}\:[0-9]{2} [0-9]{4}).{10,1}^X\-Spam\-Status\: (Yes|No)\, score\=((?:\-)?[0-9]{1,4}\.[0-9]{0,2}) required\=[0-9]{1,3}\.[0-9]{1,2} tests\=(.{5,500})autolearn I then throw the back references into another file so I get the following Mon Aug 21 18:30:45 2006 No -92.7 AWL,DATE_IN_PAST_12_24,FVGT_u_HAS_2LETTERFLDR,HTML_MESSAGE,HTML_TAG_EXIST_TBODY,L_TITLE_MESSAGE,MK_BAD_HTML_16,MY_NUMPHP,MY_SHRT_IMG,MY_SPACER,NO_RDNS2,RM_rb_ANCHOR,RM_rb_BODY,RM_rb_BREAK,RM_rb_DIV,RM_rb_FONT,RM_rb_HTML,RM_rb_TITLE,USER_IN_WHITELIST,cust_LOCAL_TO_RCVD Mon Aug 21 18:39:32 2006 No 2.8 AWL,FCS_URI_NODOTS,FVGT_u_HAS_2LETTERFLDR,HTML_90_100,HTML_EVENT_UNSAFE,HTML_FONT_INVISIBLE,HTML_MESSAGE,ISO_7BITS,J_CHICKENPOX_54,MY_DSL,MY_SHRT_IMG,MY_SPACER,NO_RDNS2,RM_rb_ANCHOR,RM_rb_BREAK,RM_rb_HTML,RM_rb_TITLE,cust_LOCAL_TO_RCVD Mon Aug 21 18:40:23 2006 No -95.4 AWL,HTML_60_70,HTML_MESSAGE,MK_BAD_HTML_16,NO_RDNS2,NO_REAL_NAME,RM_rb_ANCHOR,RM_rb_BODY,RM_rb_BREAK,RM_rb_DIV,RM_rb_FONT,RM_rb_HTML,RM_rb_PARA,RM_rb_TITLE,USER_IN_WHITELIST,cust_LOCAL_RTNPATH_RTNPATH,cust_LOCAL_TO_RCVD I can then throw that file into excel and do some cleaing up (like turning the date string into something excel understands) and then can do some stats. So for December out of 2,400 messages I had 1.53% Ham and 98.47% Spam. Now that's depressing !! What I can't work out at the moment is to do anything usefull with the rules that were hit. I suppose I could create a list of the top ten rules for Spam and Ham... If I ever work out a better way to do it, or if someone comes up with a usefull thing to do with the rules hit, I'll let you know... Paul. -- Paul Hurley http://www.paulhurley.co.uk/ The knack of flying is learning how to throw yourself at the ground and miss. Hitchhikers Guide to the Galaxy
Re: Parallelizing Spam Assassin
hi -- turn off Bayes and AWL. On Fri, Jul 31, 2009 at 07:55, poifgh wrote: > > Hi > > I was measuring how quickly could SA [spam assassin] process spams when > several SA processes are run in parallel over separate mbox files. I used a > 8 core machine. Below are the numbers when I forked different number of > processes. > > Fork = 8; > Rate = 57 msgs/sec > > Fork = 4; > Rate = 44 msgs/sec > > Fork = 1; > Rate = 22 msgs/sec > > > I ran freshly build SA with Bayes and DNSBL turned off. Why am I not seeing > a linear increase in the throughput? Is a file locking creating the > bottleneck? If yes, which particular file is being locked? If no, what could > be the reason for this? > > thnx > -- > View this message in context: > http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24751958.html > Sent from the SpamAssassin - Users mailing list archive at Nabble.com. > > -- --j.
Re: Parallelizing Spam Assassin
On Thu, Jul 30, 2009 at 11:55:21PM -0700, poifgh wrote: > Why am I not seeing a linear increase in the throughput? > Is a file locking creating the bottleneck? Maybe the auto white list. --
Re: Parallelizing Spam Assassin
On Thu, 2009-07-30 at 23:55 -0700, poifgh wrote: > Hi > > I was measuring how quickly could SA [spam assassin] process spams when > several SA processes are run in parallel over separate mbox files. I used a > 8 core machine. Below are the numbers when I forked different number of > processes. > > Fork = 8; > Rate = 57 msgs/sec > > Fork = 4; > Rate = 44 msgs/sec > > Fork = 1; > Rate = 22 msgs/sec > > > I ran freshly build SA with Bayes and DNSBL turned off. Why am I not seeing > a linear increase in the throughput? Is a file locking creating the > bottleneck? If yes, which particular file is being locked? If no, what could > be the reason for this? > > thnx Wow! That's a real flying machine! Imagine what Barracuda Networks could do with that if they did not fill their gay little boxes with hardware rubbish from the floors of MSI and supermicro. Jesus, try and process that many messages with a $30,000 Barracuda and watch support bitch 'You are fully scanning to much mail and making our rubbish hardware wet the bed.' LOL. Well done you!
Re: Parallelizing Spam Assassin
On Fri, Jul 31, 2009 at 09:32, rich...@buzzhost.co.uk wrote: > Imagine what Barracuda Networks could do with that if they did not fill > their gay little boxes with hardware rubbish from the floors of MSI and > supermicro. Jesus, try and process that many messages with a $30,000 > Barracuda and watch support bitch 'You are fully scanning to much mail > and making our rubbish hardware wet the bed.' LOL. Richard -- please watch your language. This is a public mailing list, and offensive language here is inappropriate. -- --j.
Re: Parallelizing Spam Assassin
On Fri, Jul 31, 2009 at 09:32:42AM +0100, rich...@buzzhost.co.uk wrote: > On Thu, 2009-07-30 at 23:55 -0700, poifgh wrote: > > Hi > > > > I was measuring how quickly could SA [spam assassin] process spams when > > several SA processes are run in parallel over separate mbox files. I used a > > 8 core machine. Below are the numbers when I forked different number of > > processes. > > > > Fork = 8; > > Rate = 57 msgs/sec > > > > Fork = 4; > > Rate = 44 msgs/sec > > > > Fork = 1; > > Rate = 22 msgs/sec > > > > > > I ran freshly build SA with Bayes and DNSBL turned off. Why am I not seeing > > a linear increase in the throughput? Is a file locking creating the > > bottleneck? If yes, which particular file is being locked? If no, what could > > be the reason for this? > > > > thnx > Wow! That's a real flying machine! Yeah, given that my 4x3Ghz box masscheck peaks at 22 msgs/sec, without Net/AWL/Bayes. But that's the 3.3 SVN ruleset.. wonder what version was used and any nondefault rules/settings? Certainly sounds strange that 1 core could top out the same. Anyone else have figures? Maybe I've borked something myself..
Re: Parallelizing Spam Assassin
On Fri, 2009-07-31 at 09:53 +0100, Justin Mason wrote: > On Fri, Jul 31, 2009 at 09:32, > rich...@buzzhost.co.uk wrote: > > Imagine what Barracuda Networks could do with that if they did not fill > > their gay little boxes with hardware rubbish from the floors of MSI and > > supermicro. Jesus, try and process that many messages with a $30,000 > > Barracuda and watch support bitch 'You are fully scanning to much mail > > and making our rubbish hardware wet the bed.' LOL. > > Richard -- please watch your language. This is a public mailing > list, and offensive language here is inappropriate. > I apologise for the any language deemed offensive. Whilst 'Jesus', 'Bitch' and 'Wet the bed' are mostly acceptable, I offer no defence for openly swearing and using the filty phrase 'Barracuda Networks'. For this I apologise.
Re: Parallelizing Spam Assassin
On Thu, 2009-07-30 at 23:55 -0700, poifgh wrote: [...] > I was measuring how quickly could SA [spam assassin] process spams when > several SA processes are run in parallel over separate mbox files. I used a > 8 core machine. Below are the numbers when I forked different number of > processes. > > Fork = 8; > Rate = 57 msgs/sec > > Fork = 4; > Rate = 44 msgs/sec > > Fork = 1; > Rate = 22 msgs/sec > > > I ran freshly build SA with Bayes and DNSBL turned off. Why am I not seeing > a linear increase in the throughput? Is a file locking creating the Because the bottleneck is not (only) the CPUs? Run `vmstat 1` or similar to see (or at least get an idea;-) if the workload is I/O bound or CPU-bound or > bottleneck? If yes, which particular file is being locked? If no, what could Maybe. The default "store in files" drivers locks the DBs exclusively for each access. > be the reason for this? Switch the DB backend to some MySQL or PostgreSQL (or whatever you like using from the "supported" ones). Run that on the very same machine and compare the numbers with the above. Bernd -- Firmix Software GmbH http://www.firmix.at/ mobil: +43 664 4416156 fax: +43 1 7890849-55 Embedded Linux Development and Services
Re: Parallelizing Spam Assassin
rich...@buzzhost.co.uk wrote: > On Fri, 2009-07-31 at 09:53 +0100, Justin Mason wrote: > >> On Fri, Jul 31, 2009 at 09:32, >> rich...@buzzhost.co.uk wrote: >> >>> Imagine what Barracuda Networks could do with that if they did not fill >>> their gay little boxes with hardware rubbish from the floors of MSI and >>> supermicro. Jesus, try and process that many messages with a $30,000 >>> Barracuda and watch support bitch 'You are fully scanning to much mail >>> and making our rubbish hardware wet the bed.' LOL. >>> >> Richard -- please watch your language. This is a public mailing >> list, and offensive language here is inappropriate. >> >> > I apologise for the any language deemed offensive. Whilst 'Jesus', > 'Bitch' and 'Wet the bed' are mostly acceptable, I offer no defence for > openly swearing and using the filty phrase 'Barracuda Networks'. For > this I apologise. > > > > Richard, we are not joking. Please watch your language on this mailing list, or you will be banned from it. You have now been warned by 2 members of the Project Management Committee. You will not be warned again.
Re: Parallelizing Spam Assassin
On Fri, 2009-07-31 at 07:26 -0400, Matt Kettler wrote: > rich...@buzzhost.co.uk wrote: > > On Fri, 2009-07-31 at 09:53 +0100, Justin Mason wrote: > > > >> On Fri, Jul 31, 2009 at 09:32, > >> rich...@buzzhost.co.uk wrote: > >> > >>> Imagine what Barracuda Networks could do with that if they did not fill > >>> their gay little boxes with hardware rubbish from the floors of MSI and > >>> supermicro. Jesus, try and process that many messages with a $30,000 > >>> Barracuda and watch support bitch 'You are fully scanning to much mail > >>> and making our rubbish hardware wet the bed.' LOL. > >>> > >> Richard -- please watch your language. This is a public mailing > >> list, and offensive language here is inappropriate. > >> > >> > > I apologise for the any language deemed offensive. Whilst 'Jesus', > > 'Bitch' and 'Wet the bed' are mostly acceptable, I offer no defence for > > openly swearing and using the filty phrase 'Barracuda Networks'. For > > this I apologise. > > > > > > > > > Richard, we are not joking. Please watch your language on this mailing > list, or you will be banned from it. > > You have now been warned by 2 members of the Project Management > Committee. You will not be warned again. > > > I have already apologised. I will not use the words you appear to have found offensive again. Can I ask, is this actually about the words I used *or* because of my comments regarding Barracuda Networks? I ask because I note they made a 'monetary donation' to Apache: http://www.barracudanetworks.com/ns/company/open-source.php If you want to ban me I will understand - you need to keep the wheels greased. It would give me more time to concentrate on leaking all the Barracuda code into the public domain, along with the various 'warez' tools I've written for it. This would probably be more beneficial to Barracuda Customers than dropping in here and making jokes at such low hanging fruit. If any Barracuda Customer would like to know how to unlock their barracuda without lifting the lid, or get change the model serial number and get free e.u. email me off list as I've just been banned for upsetting a sponsor LOL
Re: Parallelizing Spam Assassin
On Fri, 31 Jul 2009, rich...@buzzhost.co.uk wrote: ... dropping in here and making jokes at such low hanging fruit. Make all the jokes at Barracuda's expense that you like, complain about them all you like, just avoid offensive language. Vitriol is more impressive if you are creative enough to avoid using profanity and vulgarity while still blasting your target to pieces. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Vista is at best mildly annoying and at worst makes you want to rush to Redmond, Wash. and rip somebody's liver out. -- Forbes --- 5 days until the 274th anniversary of John Peter Zenger's acquittal
Re: Parallelizing Spam Assassin
On Fri, 2009-07-31 at 08:25 -0700, John Hardin wrote: > On Fri, 31 Jul 2009, rich...@buzzhost.co.uk wrote: > > > ... dropping in here and making jokes at such low hanging fruit. > > Make all the jokes at Barracuda's expense that you like, complain about > them all you like, just avoid offensive language. Vitriol is more > impressive if you are creative enough to avoid using profanity and > vulgarity while still blasting your target to pieces. > Received and understood.
Re: Parallelizing Spam Assassin
Henrik K wrote: > > Yeah, given that my 4x3Ghz box masscheck peaks at 22 msgs/sec, without > Net/AWL/Bayes. But that's the 3.3 SVN ruleset.. wonder what version was > used > and any nondefault rules/settings? Certainly sounds strange that 1 core > could top out the same. Anyone else have figures? Maybe I've borked > something myself.. > The rules sets were default .. 1. Took a fresh SA download 2. Run [configured number of parallel] SA on a [different giant] mbox file without DNSBL and 'use_bayes 0' and 'bayes_auto_learn 0' -- View this message in context: http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24760106.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Parallelizing Spam Assassin
Bernd Petrovitsch wrote: > > On Thu, 2009-07-30 at 23:55 -0700, poifgh wrote: > [...] >> I ran freshly build SA with Bayes and DNSBL turned off. Why am I not >> seeing >> a linear increase in the throughput? Is a file locking creating the > Because the bottleneck is not (only) the CPUs? > Run `vmstat 1` or similar to see (or at least get an idea;-) if the > workload is I/O bound or CPU-bound or > >> bottleneck? If yes, which particular file is being locked? If no, what >> could > Maybe. The default "store in files" drivers locks the DBs exclusively > for each access. > >> be the reason for this? > Switch the DB backend to some MySQL or PostgreSQL (or whatever you like > using from the "supported" ones). Run that on the very same machine and > compare the numbers with the above. > Running 'top' with a single SA process running gives 12.5% CPU utilization which makes sense since one core is fully utilized at this point out of 8 cores. The SA process reports 100% util for that CPU When fork goes to 8, each individual CPU is utilized from 30-70% mostly staying about 30 and only a few reaching 70. I can vmstat to check out the IO which I dont think should be a problem - the disks are fast enough to deliver order of magnitudes more reads than 50 msgs/sec. Can you elaborate on 'store in files'? What are these files, what are they used for - can they be turned off? Thnx -- View this message in context: http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24760163.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Parallelizing Spam Assassin
c. r. wrote: > > On Thu, Jul 30, 2009 at 11:55:21PM -0700, poifgh wrote: >> Why am I not seeing a linear increase in the throughput? >> Is a file locking creating the bottleneck? > > Maybe the auto white list. > > -- > I can try turning off AWL and get back here.. Thnx -- View this message in context: http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24760203.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Parallelizing Spam Assassin
Henrik K wrote: > > Yeah, given that my 4x3Ghz box masscheck peaks at 22 msgs/sec, without > Net/AWL/Bayes. But that's the 3.3 SVN ruleset.. wonder what version was > used > and any nondefault rules/settings? Certainly sounds strange that 1 core > could top out the same. Anyone else have figures? Maybe I've borked > something myself.. > The problem is not with 22 being a low number, but when we have other free cores to run different SA parallely why doesnt the throughput scale linearly .. I expect for 8 cores with 8 SA running simultaneously the number to be 150+ msgs/sec but it is 1/3rd at 50 msgs/sec -- View this message in context: http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24760294.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Parallelizing Spam Assassin
I'm assuming you run a tad more messages than I, but on a quad with a failover I have never seen the failover kick in 4 years. This is not disputing your observations, just noting mine. I claim absolutely no knowledge about the core processing/stacking though I would assume (perhaps incorrectly) that the parsing would be part of the software (MTA). I freely admit I only picked up what seems the tail end of this thread but having used SA for so many years I think I have at least a handle on how it plays (hence the failover). My failover SA is in place to handle slow queries from the primary SA. Assuming (again) that mail size has been factored and any AV is running remotely? Just a few thoughts based on a very cursory read of a few posts, sadly - or happily, work make my contributions here limited. I'd be interested in the results of this though. Kind regards Nigel PS - apologies if I'm repeating prior observations. On Fri, 31 Jul 2009 10:41:47 -0700 (PDT), poifgh wrote: > > > >Henrik K wrote: >> >> Yeah, given that my 4x3Ghz box masscheck peaks at 22 msgs/sec, without >> Net/AWL/Bayes. But that's the 3.3 SVN ruleset.. wonder what version was >> used >> and any nondefault rules/settings? Certainly sounds strange that 1 core >> could top out the same. Anyone else have figures? Maybe I've borked >> something myself.. >> > >The problem is not with 22 being a low number, but when we have other free >cores to run different SA parallely why doesnt the throughput scale linearly >.. I expect for 8 cores with 8 SA running simultaneously the number to be >150+ msgs/sec but it is 1/3rd at 50 msgs/sec
Re: Parallelizing Spam Assassin
In my tests - there was not MTA. The mails/spam were collected from some server in mbox format and fed to SA using --mbox switch. The size of msgs was not altered in any fashion - just the usual size of incoming spam/mails There are no AV [you mean Anti Virus right?] running on the machine Would be back with results -- Nigel Frankcom-2 wrote: > > I'm assuming you run a tad more messages than I, but on a quad with a > failover I have never seen the failover kick in 4 years. This is not > disputing your observations, just noting mine. > > I claim absolutely no knowledge about the core processing/stacking > though I would assume (perhaps incorrectly) that the parsing would be > part of the software (MTA). > > I freely admit I only picked up what seems the tail end of this thread > but having used SA for so many years I think I have at least a handle > on how it plays (hence the failover). My failover SA is in place to > handle slow queries from the primary SA. Assuming (again) that mail > size has been factored and any AV is running remotely? > > Just a few thoughts based on a very cursory read of a few posts, sadly > - or happily, work make my contributions here limited. > > I'd be interested in the results of this though. > > Kind regards > > Nigel > > PS - apologies if I'm repeating prior observations. > > On Fri, 31 Jul 2009 10:41:47 -0700 (PDT), poifgh > wrote: > >> >> >> >>Henrik K wrote: >>> >>> Yeah, given that my 4x3Ghz box masscheck peaks at 22 msgs/sec, without >>> Net/AWL/Bayes. But that's the 3.3 SVN ruleset.. wonder what version was >>> used >>> and any nondefault rules/settings? Certainly sounds strange that 1 core >>> could top out the same. Anyone else have figures? Maybe I've borked >>> something myself.. >>> >> >>The problem is not with 22 being a low number, but when we have other free >>cores to run different SA parallely why doesnt the throughput scale linearly >>.. I expect for 8 cores with 8 SA running simultaneously the number to be >>150+ msgs/sec but it is 1/3rd at 50 msgs/sec > > -- View this message in context: http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24761236.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Parallelizing Spam Assassin
OK - I can see what metrics you are trying to ascertain - I think. I'm not sure that your test and real life are 'right'. For obvious reasons I don't want to carry this one on via list - I would suggest you ask Justin and I will be happy to give info on my local setup (this assumes Justin can grab time away from toxic nappies/daipers) There is a lot you can do to ameliorate load. On bad days my quad does 50 a second so it's doable. I will freely admit I have no clue quite how this came to be, but it is (a case of having colleagues knowing more than I do - for which I am eternally grateful; the usual culprits know who they are) Kind regards Nigel On Fri, 31 Jul 2009 11:41:14 -0700 (PDT), poifgh wrote: > >In my tests - there was not MTA. The mails/spam were collected from some >server in mbox format and fed to SA using --mbox switch. The size of msgs >was not altered in any fashion - just the usual size of incoming spam/mails > >There are no AV [you mean Anti Virus right?] running on the machine > >Would be back with results > >-- > > > > >Nigel Frankcom-2 wrote: >> >> I'm assuming you run a tad more messages than I, but on a quad with a >> failover I have never seen the failover kick in 4 years. This is not >> disputing your observations, just noting mine. >> >> I claim absolutely no knowledge about the core processing/stacking >> though I would assume (perhaps incorrectly) that the parsing would be >> part of the software (MTA). >> >> I freely admit I only picked up what seems the tail end of this thread >> but having used SA for so many years I think I have at least a handle >> on how it plays (hence the failover). My failover SA is in place to >> handle slow queries from the primary SA. Assuming (again) that mail >> size has been factored and any AV is running remotely? >> >> Just a few thoughts based on a very cursory read of a few posts, sadly >> - or happily, work make my contributions here limited. >> >> I'd be interested in the results of this though. >> >> Kind regards >> >> Nigel >> >> PS - apologies if I'm repeating prior observations. >> >> On Fri, 31 Jul 2009 10:41:47 -0700 (PDT), poifgh >> wrote: >> >>> >>> >>> >>>Henrik K wrote: Yeah, given that my 4x3Ghz box masscheck peaks at 22 msgs/sec, without Net/AWL/Bayes. But that's the 3.3 SVN ruleset.. wonder what version was used and any nondefault rules/settings? Certainly sounds strange that 1 core could top out the same. Anyone else have figures? Maybe I've borked something myself.. >>> >>>The problem is not with 22 being a low number, but when we have other free >>>cores to run different SA parallely why doesnt the throughput scale >linearly >>>.. I expect for 8 cores with 8 SA running simultaneously the number to be >>>150+ msgs/sec but it is 1/3rd at 50 msgs/sec >> >>
Re: Parallelizing Spam Assassin
> In my tests - there was not MTA. The mails/spam were collected from > some server in mbox format and fed to SA using --mbox switch. The > size of msgs was not altered in any fashion - just the usual size of > incoming spam/mails If you're interested in testing/tuning spamassassin for heavy loads you should consider using spamd daemon. Then you may use SLAMD [1] as performance evaluation platform [2]. It takes some effort to set up the environment, but SLAMD helps in repetitive testing and keeping track of the results (comparison, history, charts). [1] http://www.slamd.com [2] https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5689 -- Pawel Sasin "WIRTUALNA POLSKA" Spolka Akcyjna z siedziba w Gdansku przy ul. Traugutta 115 C, wpisana do Krajowego Rejestru Sadowego - Rejestru Przedsiebiorcow prowadzonego przez Sad Rejonowy Gdansk - Polnoc w Gdansku pod numerem KRS 068548, o kapitale zakladowym 67.980.024,00 zlotych oplaconym w calosci oraz Numerze Identyfikacji Podatkowej 957-07-51-216.
Re: Parallelizing Spam Assassin
On Jul 31, 2009, at 1:55 AM, poifgh wrote: I ran freshly build SA with Bayes and DNSBL turned off. Why am I not seeing a linear increase in the throughput? Is a file locking creating the bottleneck? If yes, which particular file is being locked? If no, what could be the reason for this? There could be many reasons, check out my talk (admittedly out of date a little but should still be mostly relevant) on High Performance Apache SpamAssassin at the following link: http://people.apache.org/~parker/presentations/index.html Keep in mind that you might also be seeing other factors like memory and disk I/O contention. You don't really spell out your testing infrastructure so its not real clear if you're even performing a valid test. Also, I wouldn't necessarily expect to see a linear increase, although you might be able to take some easy steps for increasing your overall performance. Michael
Re: Parallelizing Spam Assassin
On Jul 31, 2009, at 2:53 AM, Justin Mason wrote: On Fri, Jul 31, 2009 at 09:32, rich...@buzzhost.co.uk wrote: Imagine what Barracuda Networks could do with that if they did not fill their gay little boxes with hardware rubbish from the floors of MSI and supermicro. Jesus, try and process that many messages with a $30,000 Barracuda and watch support bitch 'You are fully scanning to much mail and making our rubbish hardware wet the bed.' LOL. Richard -- please watch your language. This is a public mailing list, and offensive language here is inappropriate. I dunno, 'gay' isn't that offensive. -- Overhead, without any fuss, the stars were going out.
Re: Parallelizing Spam Assassin
On Jul 31, 2009, at 9:25 AM, John Hardin wrote: On Fri, 31 Jul 2009, rich...@buzzhost.co.uk wrote: ... dropping in here and making jokes at such low hanging fruit. Make all the jokes at Barracuda's expense that you like, complain about them all you like, just avoid offensive language. Really? Referring to gay hardware is THAT offensive that someone would need to be banned over it? -- Is a vegetarian permitted to eat animal crackers?
Re: Parallelizing Spam Assassin
From: "Matt Kettler" Sent: Friday, 2009/July/31 04:26 rich...@buzzhost.co.uk wrote: On Fri, 2009-07-31 at 09:53 +0100, Justin Mason wrote: On Fri, Jul 31, 2009 at 09:32, rich...@buzzhost.co.uk wrote: ... Richard -- please watch your language. This is a public mailing list, and offensive language here is inappropriate. ... Richard, we are not joking. Please watch your language on this mailing list, or you will be banned from it. You have now been warned by 2 members of the Project Management Committee. You will not be warned again. Given that profanity is the effort of a small mind to express itself I have a feeling he's going to receive his third and final warning any time now, Matt. {^_-}
Re: Parallelizing Spam Assassin
On Jul 31, 2009, at 1:33 PM, jdow wrote: Given that profanity is the effort of a small mind to express itself I have a feeling he's going to receive his third and final warning any time now, Matt Given that nothing that richard said is not anything I've heard on, say, prime time TV or... a committee meeting I am really curious now as to what was considered 'obscene'. I'm quite serious. Have I stumbled into a list run by religious freaks? -- Clark's Law: Sufficiently advanced cluelessness is indistinguishable from malice Clark Slaw: Anything that has been severely damaged or destroyed by application of Clark's Law
Re: Parallelizing Spam Assassin
On Fri, Jul 31, 2009 at 12:37, LuKreme wrote: > On Jul 31, 2009, at 1:33 PM, jdow wrote: >> >> Given that profanity is the effort of a small mind to express itself >> I have a feeling he's going to receive his third and final warning any >> time now, Matt > > Given that nothing that richard said is not anything I've heard on, say, > prime time TV or... a committee meeting I am really curious now as to what > was considered 'obscene'. > > I'm quite serious. > > Have I stumbled into a list run by religious freaks? (mods: sorry if this also falls into the verboten category, I'm more trying to explore/catalog than perpetuate) Maybe it was using the word "bitch", where he could have used the word "complain". (and, religious freaks aren't the only freaks that don't like to see the word "Jesus" used in that kind of context ... saying words like "Jesus" around atheist freaks can also result in them claiming offence ... luckily religious freaks and atheist freaks aren't as common as merely religious people and merely atheist people)
Re: Parallelizing Spam Assassin
LuKreme said the following on 7/31/09 3:27 PM: >> Richard -- please watch your language. This is a public mailing >> list, and offensive language here is inappropriate. > > I dunno, 'gay' isn't that offensive. > > Gay is *not* a synonym for stupid. I do take offense to the term being used in that manner. --Glenn
Re: Parallelizing Spam Assassin
rich...@buzzhost.co.uk wrote: > email me off list as I've just been > banned for upsetting a sponsor LOL > Richard, this has nothing to do with Barracuda. They have no influence over my opinions whatsoever. I don't work for Apache or Barracuda, or any company sponsored by either.Neither Apache nor Barracuda has complained. At the time I warned you, I didn't even remember that Barracuda ever donated to Apache. I don't think any member of the PMC has any regular contact with Barracuda, although we've had occasional contact about using their RBL. Your warning is about using foul language, and then choosing to thumb your nose at the warning Justin gave you. You're behaving like an impudent and foul mouthed child, and that's unwelcome her. That said, I really don't appreciate you using this list to rant about Barracuda's products, or discuss them at all. This is the SpamAssassin list, not the Barracuda list. Barracuda may use SpamAssassin, and SpamAssassin may support the Barracuda public RBL, but beyond that, any discussion of them is, quite frankly, off-topic. I don't care how good or bad their commercial product, or its support is, because it is off-topic here. I don't welcome people praising Barracuda any more than I welcome complaints. It simply doesn't matter to SpamAssassin, so it doesn't belong here. You may as well be ranting about Ford cars for all I care, it still doesn't belongs here. This list is about SpamAssassin, nothing more, nothing less. Continue with the foul language, and you'll find the door very quickly. Keep harping on the same off-topic subject and we will eventually get tired of it. You've said your peace about Barracuda, now give it a rest, because frankly I don't care about their products, I care about our product. Is that difficult to understand? > > > >
Re: Parallelizing Spam Assassin
On Fri, Jul 31, 2009 at 10:41:47AM -0700, poifgh wrote: > > Henrik K wrote: > > > > Yeah, given that my 4x3Ghz box masscheck peaks at 22 msgs/sec, without > > Net/AWL/Bayes. But that's the 3.3 SVN ruleset.. wonder what version was > > used > > and any nondefault rules/settings? Certainly sounds strange that 1 core > > could top out the same. Anyone else have figures? Maybe I've borked > > something myself.. > > > > The problem is not with 22 being a low number, but when we have other free I did not say it was a problem. I was just wondering how fast CPU/memory you have, since my 3Ghz AMD doesn't seem to keep up. I just tested with fresh 3.2.5 install, and running 500 mail mbox with single core resulted in 11 msgs / sec. Then I used sa-compile, and it raised to 15. Did you use it also? Of course your mailbox could be a lot different, so hard to compare. > cores to run different SA parallely why doesnt the throughput scale linearly > .. I expect for 8 cores with 8 SA running simultaneously the number to be > 150+ msgs/sec but it is 1/3rd at 50 msgs/sec Anyway as people have already said here, disable AWL: use_auto_whitelist 0
Re: Parallelizing Spam Assassin
I am sorry, I did not provide any statistics of the machine involved. CPU - 8 cores with each core 2327 MHz RAM - 16GB Afair its has 7200RPM disk - 2TB. Yes, people were right in indicating AWL could be the problem. turning off AWL results in near linear scaling of SA as we increase number of processes. My input is more than a 100K [mostly] spams which allowed me to have each run last for several minutes and then take an avg to get #msgs/sec With AWL, bayes and DNSBL turned off - i get about 24 msgs/sec for 1 fork and 166 msgs/sec for 8 fork with awl on and bayes and DNSBL off, i get about 22 msgs/sec for 1 fork and 50 msgs/sec for 8 fork Thnx everyone for helping out. -- Henrik K wrote: > > On Fri, Jul 31, 2009 at 10:41:47AM -0700, poifgh wrote: > > > I did not say it was a problem. I was just wondering how fast CPU/memory > you > have, since my 3Ghz AMD doesn't seem to keep up. > > I just tested with fresh 3.2.5 install, and running 500 mail mbox with > single core resulted in 11 msgs / sec. Then I used sa-compile, and it > raised > to 15. Did you use it also? > > Of course your mailbox could be a lot different, so hard to compare. > >> cores to run different SA parallely why doesnt the throughput scale >> linearly >> .. I expect for 8 cores with 8 SA running simultaneously the number to be >> 150+ msgs/sec but it is 1/3rd at 50 msgs/sec > > Anyway as people have already said here, disable AWL: > > use_auto_whitelist 0 > > > -- View this message in context: http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24765545.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Parallelizing Spam Assassin
I havent tried with sa-compile yet - I can give it a shot -- Henrik K wrote: > > On Fri, Jul 31, 2009 at 10:41:47AM -0700, poifgh wrote: >> >> Henrik K wrote: >> > >> > Yeah, given that my 4x3Ghz box masscheck peaks at 22 msgs/sec, without >> > Net/AWL/Bayes. But that's the 3.3 SVN ruleset.. wonder what version was >> > used >> > and any nondefault rules/settings? Certainly sounds strange that 1 core >> > could top out the same. Anyone else have figures? Maybe I've borked >> > something myself.. >> > >> >> The problem is not with 22 being a low number, but when we have other >> free > > I did not say it was a problem. I was just wondering how fast CPU/memory > you > have, since my 3Ghz AMD doesn't seem to keep up. > > I just tested with fresh 3.2.5 install, and running 500 mail mbox with > single core resulted in 11 msgs / sec. Then I used sa-compile, and it > raised > to 15. Did you use it also? > > Of course your mailbox could be a lot different, so hard to compare. > >> cores to run different SA parallely why doesnt the throughput scale >> linearly >> .. I expect for 8 cores with 8 SA running simultaneously the number to be >> 150+ msgs/sec but it is 1/3rd at 50 msgs/sec > > Anyway as people have already said here, disable AWL: > > use_auto_whitelist 0 > > > -- View this message in context: http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24765570.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Parallelizing Spam Assassin
On Fri, 2009-07-31 at 17:37 -0400, Glenn Sieb wrote: > LuKreme said the following on 7/31/09 3:27 PM: > >> Richard -- please watch your language. This is a public mailing > >> list, and offensive language here is inappropriate. > > > > I dunno, 'gay' isn't that offensive. > > > > > > Gay is *not* a synonym for stupid. > > I do take offense to the term being used in that manner. > > --Glenn > I find it deeply offensive that the word 'gay' is used as a synonym for homosexual in an attempt to stop people from using 'queer' - but hey 'gays' are not the only ones with opinions that 'matter'. Gay **is** a synonym for 'stupid' (silly) as far as I am concerned. It's original meaning of 'carefree','happy','silly' and 'showy' are clearly being used with sarcasm. The fact is 'queers' hijacked the word as per this; "— USAGE Gay is now a standard term for ‘homosexual’, and is the term preferred by homosexual men to describe themselves. As a result, it is now very difficult to use gay in its earlier meanings ‘carefree’ or ‘bright and showy’ without arousing a sense of double entendre. Gay in its modern sense typically refers to men, lesbian being the standard term for homosexual women." http://www.askoxford.com/concise_oed/gay?view=uk So please *quit* with the sympathetic pink preaching and learn what the word actually means. Just because it "is the term preferred by homosexual men to describe themselves" does not mean a minority have the right to slate people who use the word properly. With regards to the dig about Barracuda - this *WAS* OT. There were some benchmark tests discussed here that were impressive. My experience of SA in daily production is on Barracuda Appliances that STRUGGLE to push 6-8 messages a second through, so it was relevant as comparison. The wording could have been chosen with more care and I apologise to Christians or dog lovers who found the use of the messiah or female form offensive. However, the use of gay in a sarcastic context clearly fits with the original origin of the word, not by that section of the society who have stolen it and made it OT and OM. For that I make ***NO*** apology. I appreciate that using 'gay' in it's real meaning may hurt the feelings of some 'homosexuals' but as I have to respect their choices and views, they should show *me* the same respect for *my* views and choices. You may not like who I am and what I do, I may not like who you are and what you do. Now do we need to continue this or throw little tin God banning threats around more or can we just *get along* knowing we are all different but frequenting this list for Spamassassin information ?
Re: Parallelizing Spam Assassin
From: "LuKreme" Sent: Friday, 2009/July/31 12:30 On Jul 31, 2009, at 9:25 AM, John Hardin wrote: On Fri, 31 Jul 2009, rich...@buzzhost.co.uk wrote: ... dropping in here and making jokes at such low hanging fruit. Make all the jokes at Barracuda's expense that you like, complain about them all you like, just avoid offensive language. Really? Referring to gay hardware is THAT offensive that someone would need to be banned over it? No, it's the word "expensive". {+_+}
Re: Parallelizing Spam Assassin
From: "LuKreme" Sent: Friday, 2009/July/31 12:37 On Jul 31, 2009, at 1:33 PM, jdow wrote: Given that profanity is the effort of a small mind to express itself I have a feeling he's going to receive his third and final warning any time now, Matt Given that nothing that richard said is not anything I've heard on, say, prime time TV or... a committee meeting I am really curious now as to what was considered 'obscene'. I'm quite serious. Have I stumbled into a list run by religious freaks? Not me. I can happily go several whole days without hearing the B word. When I hear it I get B...y. {^_^} Joanne
Re: Parallelizing Spam Assassin
From: "poifgh" Sent: Friday, 2009/July/31 19:47 I am sorry, I did not provide any statistics of the machine involved. CPU - 8 cores with each core 2327 MHz RAM - 16GB Afair its has 7200RPM disk - 2TB. One disk you might consider a striped array to get disk speed. 50 megabytes per second stresses most disks pretty hard - not to the limit. But if there is a lot of seeking involved as well as multiple copies of the files being made as they pass through the system I can see how it'd be a little rough on the disk throughput. {^_^}
Re: Parallelizing Spam Assassin
It's an American thing. Things that are normal speech for UK blokes, get Americans all disturbed. Funny, used to be the other way around...but well...times change. Justin Mason wrote: On Fri, Jul 31, 2009 at 09:32, rich...@buzzhost.co.uk wrote: Imagine what Barracuda Networks could do with that if they did not fill their gay little boxes with hardware rubbish from the floors of MSI and supermicro. Jesus, try and process that many messages with a $30,000 Barracuda and watch support bitch 'You are fully scanning to much mail and making our rubbish hardware wet the bed.' LOL. Richard -- please watch your language. This is a public mailing list, and offensive language here is inappropriate.
Re: Parallelizing Spam Assassin
* Linda Walsh : > It's an American thing. Things that are normal speech for UK blokes, get > Americans all disturbed. Sloppy language is sloppy language everywhere! I took offense in the message, too and I am neither American nor am I from the UK. But what annoys me the most is that the comments were simply off-topic. I can go and meet some friends and I can happily spend the whole night cracking one joke after another - pc or not pc. There's a place of everything. This is the place for SpamAssassin. I wish we could get back to what this thread was all about: "Parallelizing SpamAssassin". p...@rick > Funny, used to be the other way around...but well...times change. > > Justin Mason wrote: > >On Fri, Jul 31, 2009 at 09:32, > >rich...@buzzhost.co.uk wrote: > >>Imagine what Barracuda Networks could do with that if they did not fill > >>their gay little boxes with hardware rubbish from the floors of MSI and > >>supermicro. Jesus, try and process that many messages with a $30,000 > >>Barracuda and watch support bitch 'You are fully scanning to much mail > >>and making our rubbish hardware wet the bed.' LOL. > > > >Richard -- please watch your language. This is a public mailing > >list, and offensive language here is inappropriate. > > -- state of mind Digitale Kommunikation http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht MünchenPartnerschaftsregister PR 563
Re: Parallelizing Spam Assassin
May I point out, that while you may find the language crude -- it isn't language that would violate FTC standards in that in used any of the 7 or so 'unmentionable words'... People -- these standards of 'crude language' really need to be strongly held 'in check' -- the US is 'supposed' to be the society of 'free speech' unless it is obscene or threatening. I don't think his posting was either (BTW, I've never even 'heard' or seen his name before this post. All I saw was his 'uk' addr -- and I've known a few 'uk' types, and many of them sound very crude to an American ear these days. So in addition to applying strictures in a conservative manner, we must, hopefully, try to be sensitive to different cultural backgrounds. If I was talking with a black teen from downtown SF/Oakland, I'd have to translate from Eubonics -- which can sound rather crude and might contain and F-word every other sentence. I just apply my linguistic filter and attempt to get the meaning. I hardly thing this list is aimed at an young audience -- and kid 13+ is going to have heard quite an ear-full of 'colorful explicatives' from ST4:Voyage home (a family movie), to everyday peer talk. Yes -- it sounded crude...more than I, normally hear in America -- but not more than I'd hear in London. Just my 2-cents on cultural sensitivity, and the ability to be amused at cultural differences (rather than choosing to be offended by them). p.s. - Most Commercial vendor products are Bantha Poodoo -- especially for Virus/Security and Spam protection, but NOT all. Usually the highest advertised profile are the worst -- they put more budget into advertising than engineering. Yeah, I still thing SA is a bit slow, but I put much of that up to it being written in an interpretive language and it's wide flexibility and extensibility with plug-ins. Whatcha gonna do? Maybe we should rewrite it in Forth? *grin*...
Re: Parallelizing Spam Assassin
Well -- it's not just the cores -- what was the usage of the cores that were being used? were 3 out the 8 'pegged'? Are these 'real' cores, or HT cores? In the Core2 and P4 archs, HT's actually slowed down a good many workloads unless they were tightly constructed to work on the same data in cache. Else, those HT's did just enough extra work to block cache contents more than anything else. What's the disk I/O look like? I mean don't just focus on idle cores -- if the wait is on disk, maybe the cores can't get the data fast enough. If the network is involved, well, that's a drag on any message checking. I'm seeing times of .3msgs/sec, but I think that's with networking turned on. Pretty Ugly. poifgh wrote: Henrik K wrote: Yeah, given that my 4x3Ghz box masscheck peaks at 22 msgs/sec, without Net/AWL/Bayes. But that's the 3.3 SVN ruleset.. wonder what version was used and any nondefault rules/settings? Certainly sounds strange that 1 core could top out the same. Anyone else have figures? Maybe I've borked something myself.. The problem is not with 22 being a low number, but when we have other free cores to run different SA parallely why doesnt the throughput scale linearly .. I expect for 8 cores with 8 SA running simultaneously the number to be 150+ msgs/sec but it is 1/3rd at 50 msgs/sec
Re: Parallelizing Spam Assassin
On Fri, 2009-07-31 at 23:40 -0700, Linda Walsh wrote: > It's an American thing. Things that are normal speech for UK blokes, get > Americans all disturbed. I'm sure that is mostly it, Linda. They don't seem to 'get' it. Two things I observe in this whole 'barracuda-gate' posting; 1. Being 'offended' is not terminal, it does not kill, disable or have any side effects. Can you image going to a doctor and saying "You've got to treat me Doctor, I got offended, my feelings are hurt." 2. Cultural differences exist. If I am expected to respect the 'diversity' that has people jumping up and down about the use of 'gay' because *they* have a different meaning for it, it is not unreasonable to expect *them* to respect my diversity in using it in it's original context. I'm tired of being told not to offend or upset people who don't show my views and beliefs equal respect. Anyway, it's all OT and pointless in any context of processing spam - the point I made was factual love it or hate it. That was poor hardware spec used in a well known retail anti-spam appliance = 6-8 MPS 'fully scanned'.
Re: Parallelizing Spam Assassin
On Sat, Aug 01, 2009 at 12:04:08AM -0700, Linda Walsh wrote: > Well -- it's not just the cores -- what was the usage of the cores that > were being used? were 3 out the 8 'pegged'? Are these 'real' cores, or > HT cores? In the Core2 and P4 archs, HT's actually slowed down a good > many workloads unless they were tightly constructed to work on the same > data in cache. Else, those HT's did just enough extra work to block cache > contents more than anything else. I really doubt there's HT involved in a recent looking 8 core 16GB machine.. > What's the disk I/O look like? I mean don't just focus on idle cores -- > if the wait is on disk, maybe the cores can't get the data fast enough. As we already guessed, AWL (BerkeleyDB) caused disk I/O and slowness. For heavy loads you need to use SQL (or maybe the better BDB plugin in 3.3 if we get it working). > If the network is involved, well, that's a drag on any message checking. > I'm seeing times of .3msgs/sec, but I think that's with networking turned > on. Pretty Ugly. It affects single messages, but not total throughput. With network checks you just dedicate a lot more childs. Waiting for network responses takes no CPU time, thus you can process more messages simultaneously.
Re: Parallelizing Spam Assassin
Henrik K wrote: > On Sat, Aug 01, 2009 at 12:04:08AM -0700, Linda Walsh wrote: >> Well -- it's not just the cores -- what was the usage of the cores >> that >> were being used? were 3 out the 8 'pegged'? Are these 'real' cores, >> or >> HT cores? In the Core2 and P4 archs, HT's actually slowed down a >> good many workloads unless they were tightly constructed to work on >> the same >> data in cache. Else, those HT's did just enough extra work to block >> cache contents more than anything else. > > I really doubt there's HT involved in a recent looking 8 core 16GB > machine.. Why not? I have a couple of brandnew Intel Core i7 (Nehalem) systems with 8Gb RAM - they have 1 physical CPU with 4 cores and HT = 8 "cores". And they've got room for more RAM :-) /Per Jessen, Zürich
Re: Parallelizing Spam Assassin
On Sat, Aug 1, 2009 at 10:04, Henrik K wrote: > > On Sat, Aug 01, 2009 at 12:04:08AM -0700, Linda Walsh wrote: >> Well -- it's not just the cores -- what was the usage of the cores that >> were being used? were 3 out the 8 'pegged'? Are these 'real' cores, or >> HT cores? In the Core2 and P4 archs, HT's actually slowed down a good >> many workloads unless they were tightly constructed to work on the same >> data in cache. Else, those HT's did just enough extra work to block cache >> contents more than anything else. > > I really doubt there's HT involved in a recent looking 8 core 16GB machine.. > >> What's the disk I/O look like? I mean don't just focus on idle cores -- >> if the wait is on disk, maybe the cores can't get the data fast enough. > > As we already guessed, AWL (BerkeleyDB) caused disk I/O and slowness. For > heavy loads you need to use SQL (or maybe the better BDB plugin in 3.3 if we > get it working). > >> If the network is involved, well, that's a drag on any message checking. >> I'm seeing times of .3msgs/sec, but I think that's with networking turned >> on. Pretty Ugly. > > It affects single messages, but not total throughput. With network checks > you just dedicate a lot more childs. Waiting for network responses takes no > CPU time, thus you can process more messages simultaneously. although you will also need to allocate more memory, as well, to ensure that no swapping takes place. -- --j.
Re: Parallelizing Spam Assassin
On Sat, Aug 01, 2009 at 11:46:57AM +0200, Per Jessen wrote: > Henrik K wrote: > > > On Sat, Aug 01, 2009 at 12:04:08AM -0700, Linda Walsh wrote: > >> Well -- it's not just the cores -- what was the usage of the cores > >> that > >> were being used? were 3 out the 8 'pegged'? Are these 'real' cores, > >> or > >> HT cores? In the Core2 and P4 archs, HT's actually slowed down a > >> good many workloads unless they were tightly constructed to work on > >> the same > >> data in cache. Else, those HT's did just enough extra work to block > >> cache contents more than anything else. > > > > I really doubt there's HT involved in a recent looking 8 core 16GB > > machine.. > > Why not? I have a couple of brandnew Intel Core i7 (Nehalem) systems > with 8Gb RAM - they have 1 physical CPU with 4 cores and HT = > 8 "cores". And they've got room for more RAM :-) Ah a comeback.. I guess it's atleast better than the P4 stuff? That reminds me, gotta test how SA runs on a Sun T5240 with 16 core "128 cores"..
Re: Parallelizing Spam Assassin
Henrik K wrote: > On Sat, Aug 01, 2009 at 11:46:57AM +0200, Per Jessen wrote: >> Henrik K wrote: >> >> > On Sat, Aug 01, 2009 at 12:04:08AM -0700, Linda Walsh wrote: >> >> Well -- it's not just the cores -- what was the usage of the cores >> >> that >> >> were being used? were 3 out the 8 'pegged'? Are these 'real' >> >> cores, or >> >> HT cores? In the Core2 and P4 archs, HT's actually slowed down a >> >> good many workloads unless they were tightly constructed to work >> >> on the same >> >> data in cache. Else, those HT's did just enough extra work to >> >> block cache contents more than anything else. >> > >> > I really doubt there's HT involved in a recent looking 8 core 16GB >> > machine.. >> >> Why not? I have a couple of brandnew Intel Core i7 (Nehalem) systems >> with 8Gb RAM - they have 1 physical CPU with 4 cores and HT = >> 8 "cores". And they've got room for more RAM :-) > > Ah a comeback.. I guess it's atleast better than the P4 stuff? Not sure about that - AFAICT, it's exactly the same technology. (I haven't done in exhaustive tests though). /Per Jessen, Zürich
Re: Parallelizing Spam Assassin
On Fri, 2009-07-31 at 23:56 -0700, Linda Walsh wrote: > May I point out, that while you may find the language crude -- it isn't > language that would violate FTC standards in that in used any of the > 7 or so 'unmentionable words'... It's not about words on their own -- it's about how they are being used, and their meaning in context. > BTW, I've never even 'heard' or seen his name before this post. Must have been a warm and cozy place, the rock you've been hiding under. ;) You missed a 3 digit figure of posts and uncalled-for off-topic rants within a few weeks. > If I was talking with [...] I just apply my linguistic filter and > attempt to get the meaning. Sic. -- char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4"; main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Parallelizing Spam Assassin
Um, Linda.. I'm pretty positive Justin is Irish, not American. Linda Walsh wrote: > It's an American thing. Things that are normal speech for UK blokes, get > Americans all disturbed. > > Funny, used to be the other way around...but well...times change. > > > > Justin Mason wrote: >> On Fri, Jul 31, 2009 at 09:32, >> rich...@buzzhost.co.uk wrote: >>> Imagine what Barracuda Networks could do with that if they did not fill >>> their gay little boxes with hardware rubbish from the floors of MSI and >>> supermicro. Jesus, try and process that many messages with a $30,000 >>> Barracuda and watch support bitch 'You are fully scanning to much mail >>> and making our rubbish hardware wet the bed.' LOL. >> >> Richard -- please watch your language. This is a public mailing >> list, and offensive language here is inappropriate. >> > >
Re: Parallelizing Spam Assassin
This whole time I thought the subject line was "Paralyzing Spam Assassin" and the original poster was having trouble with SA locking up. Oops. ;-) -- Dan Schaefer Web Developer/Systems Analyst Performance Administration Corp.
Re: Parallelizing Spam Assassin
I would run a tcpdump on the ethernet interface while doing this, just in case there are network tests happening that you are not aware of. On Thu, Jul 30, 2009 at 11:55:21PM -0700, poifgh wrote: > > Hi > > I was measuring how quickly could SA [spam assassin] process spams when > several SA processes are run in parallel over separate mbox files. I used a > 8 core machine. Below are the numbers when I forked different number of > processes. > > Fork = 8; > Rate = 57 msgs/sec > > Fork = 4; > Rate = 44 msgs/sec > > Fork = 1; > Rate = 22 msgs/sec > > > I ran freshly build SA with Bayes and DNSBL turned off. Why am I not seeing > a linear increase in the throughput? Is a file locking creating the > bottleneck? If yes, which particular file is being locked? If no, what could > be the reason for this? > > thnx > -- > View this message in context: > http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24751958.html > Sent from the SpamAssassin - Users mailing list archive at Nabble.com. -- /* Jason Philbrook | Midcoast Internet Solutions - Wireless and DSL KB1IOJ| Broadband Internet Access, Dialup, and Hosting http://f64.nu/ | for Midcoast Mainehttp://www.midcoast.com/ */
Re: Parallelizing Spam Assassin
I did that - with DNSBL off there are no port 53 communications from SA -- Jason Philbrook wrote: > > I would run a tcpdump on the ethernet interface while doing this, just > in case there are network tests happening that you are not aware of. > > On Thu, Jul 30, 2009 at 11:55:21PM -0700, poifgh wrote: >> >> Hi >> >> I was measuring how quickly could SA [spam assassin] process spams when >> several SA processes are run in parallel over separate mbox files. I used >> a >> 8 core machine. Below are the numbers when I forked different number of >> processes. >> >> Fork = 8; >> Rate = 57 msgs/sec >> >> Fork = 4; >> Rate = 44 msgs/sec >> >> Fork = 1; >> Rate = 22 msgs/sec >> >> >> I ran freshly build SA with Bayes and DNSBL turned off. Why am I not >> seeing >> a linear increase in the throughput? Is a file locking creating the >> bottleneck? If yes, which particular file is being locked? If no, what >> could >> be the reason for this? >> >> thnx >> -- >> View this message in context: >> http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24751958.html >> Sent from the SpamAssassin - Users mailing list archive at Nabble.com. > > -- > /* > Jason Philbrook | Midcoast Internet Solutions - Wireless and DSL > KB1IOJ | Broadband Internet Access, Dialup, and Hosting > http://f64.nu/ | for Midcoast Mainehttp://www.midcoast.com/ > */ > > -- View this message in context: http://www.nabble.com/Parallelizing-Spam-Assassin-tp24751958p24796555.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Spam assassin and postfix..
Hello, I configured sapmassassin with postfix. Sapmassassin version is spamassassin-3.0.3-4.fc4 Here is my spam filter script.. ** [EMAIL PROTECTED] log]# cat /usr/local/bin/spamfilter #variables SENDMAIL="/usr/sbin/sendmail.postfix -i" EGREP=/bin/egrep # Exit codes from EX_UNAVAILABLE=69 # Number of *'s in X-Spam-level header needed to sideline message: # (Eg. Score of 5.5 = "*" ) SPAMLIMIT=5 # Clean up when done or when aborting. trap "rm -f /var/tempfs/out.$$" 0 1 2 3 15 # Pipe message to spamc cat | /usr/bin/spamc -u spamfilter > /var/tempfs/out.$$ if $EGREP -q "^X-Spam-Level: \*{$SPAMLIMIT,}" < /var/tempfs/out.$$ then ## Change the Email address where you want your spam to get fwd to $SENDMAIL -f [EMAIL PROTECTED] < /var/tempfs/out.$$ else ###$SENDMAIL "$@" < /var/tempfs/out.$$ $SENDMAIL $@ < /var/tempfs/out.$$ fi # Postfix returns the exit status of the Postfix sendmail command. exit $? * ** I made these changes in master.cf file.. Changed this line by adding "-o content_filter=spamfilter:dummy" to the default *smtp inet n - n - - smtpd -o content_filter=spamfilter:dummy * Added next two lines.. *spamfilter unix - n n - - pipe flags=Rq user=spamfilter argv=/usr/local/bin/spamfilter -f ${sender} -- ${recipient} * Once postfix reloaded I can see that mails are being processed by spamfilter. But for some mails I get these kind of error in the log file and user receives mail from MAILER-DAEMON Can some please tell me why we get these only for some mail and how to get rid of this problem. Jun 7 10:51:44 localmail spamd[14011]: spamd: identified spam (17.8/6.8) for spamfilter:7715 in 2.3 seconds, 1753 bytes. Jun 7 10:51:44 localmail spamd[14011]: spamd: result: Y 17 - MSGID_FROM_MTA_HEADER,MSGID_FROM_MTA_ID,RCVD_IN_BL_SPAMCOP_NET,UNPARSEABLE_RELAY,URIBL_AB_SURBL,URIBL_JP_SURBL,URIBL_OB_SURBL,URIBL_SBL,URIBL_SC_SURBL,URIBL_WS_SURBL scantime=2.3,size=1753,user=spamfilter,uid=7715,required_score=6.8,rhost=localhost.localdomain,raddr=127.0.0.1,rport=33304,mid=<[EMAIL PROTECTED]>,autolearn=no Jun 7 10:51:44 localmail postfix/sendmail[14909]: fatal: Recipient addresses must be specified on the command line or via the -t option Jun 7 10:51:44 localmail spamd[14009]: prefork: child states: II Jun 7 17:51:45 localmail postfix/postdrop[14910]: warning: stdin: unexpected EOF in data, record type 78 length 85 Jun 7 10:51:45 localmail postfix/postdrop[14910]: fatal: uid=7715: malformed input Jun 7 10:51:46 localmail postfix/pipe[13865]: DA97E60EB2: to=<[EMAIL PROTECTED]>, relay=spamfilter, delay=5, status=bounced (command line usage error. Command output: sendmail.postfix: fatal: Recipient addresses must be specified on the command line or via the -t option postdrop: warning: stdin: unexpected EOF in data, record type 78 length 85 postdrop: fatal: uid=7715: malformed input ) Jun 7 10:51:46 localmail postfix/cleanup[13864]: 6AF4562F46: message-id=<[EMAIL PROTECTED]> Jun 7 10:51:46 localmail postfix/qmgr[13851]: 6AF4562F46: from=<>, size=3990, nrcpt=1 (queue active) Jun 7 10:51:46 localmail postfix/qmgr[13851]: DA97E60EB2: removed Jun 7 10:51:46 localmail postfix/smtp[14867]: 6AF4562F46: to=<[EMAIL PROTECTED]>, relay=mail.aleks.com[216.34.240.136], delay=0, status=sent (250 2.0.0 k57HpkM18899 Message accepted for delivery) Jun 7 10:51:46 localmail postfix/qmgr[13851]: 6AF4562F46: removed Jun 7 10:51:48 localmail postfix/smtpd[13861]: connect from ip26.aleks.com[216.34.240.160] Jun 7 10:51:48 localmail postfix/smtpd[13861]: 5F35C60EB2: client=ip26.aleks.com[216.34.240.160] Jun 7 10:51:48 localmail postfix/cleanup[13913]: 5F35C60EB2: message-id=<[EMAIL PROTECTED]>
New Spam Assassin user
I am a new user. I am running on Windows 2003. I have several domains on my servers. I only want one domain on my server to use spam assassin. Where and what do i need to do to only filter for 1 domain on my sever. Is this possible. I would also like to setup wrongmx on this as wel.. Thanks in advance. Mike
Spam Assassin White List
I am having trouble with the standard "White List" & "BlackList" configuration in the SpamAssassin user preferences file. The "Manual White List" user guide at http://wiki.apache.org/spamassassin/ManualWhitelist states "Adding a user to your whitelist gives them a -100 score, which has the effect of always marking their mail as non-spam". I have found that when I add manually a user to the whitelist (in the SpamAssassin user preferences file) I get inconsistent results: The first example: X-Spam-Status: No, score=-100.0 X-Spam-Score: -999 X-Spam-Bar: --- The second example (for the same address): X-Spam-Status: No, score=1.4 X-Spam-Score: 14 X-Spam-Bar: + X-Spam-Flag: NO I have also found that when I manually a user to the blacklist (in the SpamAssassin user preferences file) I get the following result: X-Spam-Status: No, score=-100.0 X-Spam-Score: -999 X-Spam-Bar: --- My Questions: (i) Is there an explanation for what I have observed above in relation to the manual whitelist entries (in the SpamAssassin user preferences file)? (ii) Why would the manual blacklist entry above (in the SpamAssassin user preferences file) give the same result as a manual whitelist entry (in the SpamAssassin user preferences file)? (iii) The administrator of our server has indicated that mail matching the whitelist entries and the blacklist entries (in the SpamAssassin user preferences file) will always be processed/assessed by SpamAssassin as spam/not spam. Is this the case? If so, what is the purpose of the whitelist and the blacklist entries in the SpamAssassin user preferences file? Any assistance would be greatly appreciated. -- View this message in context: http://www.nabble.com/Spam-Assassin-White-List-tp22589650p22589650.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Frustrating spam assassin problem
I've done a lot of SA installs before, but this one is really giving me a problem. Basically, unless I put: dns_available no In my local.cf file, SA is taking 20-25 seconds to process a message. I think it is timing out somehow on something DNS related. With dns_available no in there, it is taking less than 1 second to process. I have Net::DNS installed with resolver. If I a run a lint without the "dns_available no" line, it shows this: debug: is Net::DNS::Resolver available? yes debug: Net::DNS version: 0.48 debug: trying (3) linux.org... debug: looking up NS for 'linux.org' debug: NS lookup of linux.org succeeded => Dns available (set dns_available to hardcode) debug: is DNS available? 1 But then further down it says : debug: Running tests for priority: 500 debug: RBL: success for 0 of 1 queries debug: DNS: timeout for ahbl after 15 seconds debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0xaecf2a8) implements 'check_post_dnsbl' Also when I do a lint test, I get about 30 lines of this in the output: debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0xaecf2a8) inhibited further callbacks My resolv.conf (does Net::DNS even use this) is all setup, I can do a nslookup with no problem. When I install Net::DNS module, I get the following when doing a "make test": t/00-load..ok t/00-pod...skipped all skipped: Test::Pod v0.95 required for testing POD t/00-version...ok t/01-resolver-env..ok t/01-resolver-file.ok 7/8 skipped: Could not read configuration file t/01-resolver-opt..ok t/01-resolver..ok t/02-headerok t/03-question..ok Seems kind of weird. It seems Net::DNS isn't looking at anything. I downloaded a small tool to run DNS querys with Net::DNS and none of them work. Any help please! Spent the last 2 days on this!!
Re: Spam Assassin Load Balancing
Thomas Ledbetter wrote: > With a rounb robin policy on a hardware load balancer, once the > connection is routed to a specific 'worker bee', if that machine times > out, the request will fail, and the mail wont get scanned. However, > more intelligent hardware load balancing setups can monitor the work > on each node, and take it out of service as necessary. > > What methods do other people use? We use Linux Virtual Server. > Also, when running a round-robin based cluster, is there any problem > having a mix of machines with different performance capacities? i.e. > If I have a 10 node cluster, and 3 of the servers are much slower than > the others, will it impact performance of the cluster as a whole? You could do some weighting by having the more powerful machines listed more often. /Per Jessen, Zürich
Re: Spam Assassin Load Balancing
Thomas Ledbetter wrote: First of all: we're running amavisd-new, not plain spamc/spamd anymore. We used to have N servers each running its own spamd deamons, so with separate Bayes/AWL DB. I have not understood how many machines run spamc and how many spamd. With a rounb robin policy on a hardware load balancer, once the connection is routed to a specific 'worker bee', if that machine times out, the request will fail, and the mail wont get scanned. However, more intelligent hardware load balancing setups can monitor the work on each node, and take it out of service as necessary. A load balancer sets as offline non-responding nodes, according to a different level of checks (ICMP ping, TCP ping, service check, ...). But these checks are not in real-time, so if spamd dies during analysis the connection will drop (or hang) and spamc will timeout. The load balancer won't restart the connection to another node. At least not our HLB. Been there (with LDAP), done that! Also, when running a round-robin based cluster, is there any problem having a mix of machines with different performance capacities? i.e. If I have a 10 node cluster, and 3 of the servers are much slower than the others, will it impact performance of the cluster as a whole? Even if I limit the number of spamd that run to a lower value than the higher performance machines? What do you consider as "performance"? I think the global average analysis time (what I call "performance") will obviously be affected, to an amount that depends on load distribution. With a real load balancer you can use different priorities for each node, so to keep faster machines more busy than slower ones. Anyway, I've seen spamd running on different hardware since 2004 and I wouldn't say the analysis speed has been improved significantly. Just don't let spamd nodes swap memory to disk. Good luck with the high-load spam fight, Paolo
Re: Spam Assassin Load Balancing
We're also using Linux Virtual Server for this which processed close to a billion messages for us last year. Also sounds like you need the weighted round robin feature of lvs to weight different servers accordingly which is something we do as well. On Jan 8, 2008 1:58 AM, Paolo Cravero <[EMAIL PROTECTED]> wrote: > Thomas Ledbetter wrote: > > First of all: we're running amavisd-new, not plain spamc/spamd anymore. > > We used to have N servers each running its own spamd deamons, so with > separate > Bayes/AWL DB. > > I have not understood how many machines run spamc and how many spamd. > > > With a rounb robin policy on a hardware load balancer, once the > > connection is routed to a specific 'worker bee', if that machine times > > out, the request will fail, and the mail wont get scanned. However, > > more intelligent hardware load balancing setups can monitor the work on > > each node, and take it out of service as necessary. > > A load balancer sets as offline non-responding nodes, according to a > different > level of checks (ICMP ping, TCP ping, service check, ...). But these > checks > are not in real-time, so if spamd dies during analysis the connection will > drop (or hang) and spamc will timeout. The load balancer won't restart the > connection to another node. At least not our HLB. Been there (with LDAP), > done > that! > > > Also, when running a round-robin based cluster, is there any problem > > having a mix of machines with different performance capacities? i.e. If > > I have a 10 node cluster, and 3 of the servers are much slower than the > > others, will it impact performance of the cluster as a whole? Even if I > > limit the number of spamd that run to a lower value than the higher > > performance machines? > > What do you consider as "performance"? I think the global average analysis > time (what I call "performance") will obviously be affected, to an amount > that > depends on load distribution. With a real load balancer you can use > different > priorities for each node, so to keep faster machines more busy than slower > ones. > > Anyway, I've seen spamd running on different hardware since 2004 and I > wouldn't say the analysis speed has been improved significantly. Just > don't > let spamd nodes swap memory to disk. > > Good luck with the high-load spam fight, > Paolo >
Postfix setting? or Spam Assassin?
I have a question. I'm not sure If it's a setting in Postfix or if something needs to be set with my wrapper script. Currently I have Postfix doing virtual lookups by mysql tables. From that point, I have this: Master.cf -- smtp inet n - - - - smtpd -o content_filter=spamassassin:dummy spamassassin unix - n n - - pipe flags=Rq user=spamassassin argv=/data/scripts/spamfilter.sh -f ${sender} -- ${recipient} -end Everything works except on minor detail, but I say major detail. If I send a message to "[EMAIL PROTECTED]" it goes through the spamassassin filters and assigns its value appropriately. However, if I sent a message that has a BCC or CC, or even a comma separated on the TO: line, it only works with the first value. Example: To: [EMAIL PROTECTED], [EMAIL PROTECTED] it looks up joe's SA value and assigns it to "essentially both" by not running each recipient through. This is a problem for me because cindy may have a lower score than joe. Basically one could evade a blacklist entry or a low score entry by cc'ing an additional user. Hope this makes sense. I really am frustrated. I am not sure any other topics I could google with to find help in this matter. Thanks in advance.
Re: Problems running Spam Assassin
Hi, It appears that as I was accepted to the mailing list after making my first post, my post did not hit the list. Here is my original full post below:- CosmicPerl wrote: > > Hi, > I installed the latest SpamAssassin on my server. At first all my tests > looked good, apart from load. So I setup spamc and spamd and everything > seemed great, for a short while at least. > > A day later my mqueue had about 1500 messages in it, most with the error > "local mailer (/usr/bin/procmail) exited with EX_TEMPFAIL". This seems to > be coming up if the mailbox is full or the email is to an address that > doesn't exist. > > It seemed that about every hour or so Sendmail was trying to flush out > these messages, causing 1000's of processes to be started and making the > server freeze up. Despite my Sendmail config having > define(`confMAX_DAEMON_CHILDREN', `12')dnl > > In my procmailrc file I have:- > DROPPRIVS=yes > :0fw: spamassassin.lock > * < 256000 > | spamc > > The SpamAssassin daemon was started with > /usr/bin/spamd -d -u nobody > > At some point all mail stopped coming in. When I looked at the maillog > file it had lots of lines like:- > mkdir /root/.spamassassin: Permission denied > Which I guess was causing the problem. This wasn't a problem before so I'm > not sure why it happened. Any clues? > > > Basically I need to set things up so that when sendmail trys to flush I > don't get my server falling over. > Emails that are sent to addresses that don't exist that are currently > getting the error "local mailer (/usr/bin/procmail) exited with > EX_TEMPFAIL" be delete from the queue automatically. > Ideally I'd like to give each different virtual server I have it's own > possibly spam folder. I'm using Webmin and have a 100 or so Virtual > servers so if anyone knows a good automated way of doing this that would > be great. Either way I can't have things go down again otherwise I'll > loose all my clients! > > And SpamAssassin working again. At first it was just marking emails with > [spam] in the subject. Then Yesterday It then also started changing the > message to an attachment and having "Spam detection software, running on > the system "ns.cosmicsitehosting.com", has identified this incoming email > as possible spam..." in the message text. I've no idea what was changed so > that this started happening. I didn't think I changed anything. Then last > night it stopped sending any emails. > > > Please help! > > Thanks in advance. > > Oh by the way my local.cf file contains > required_hits 10 > rewrite_header Subject [SPAM] > report_safe 1 > use_bayes 1 > skip_rbl_checks 1 > use_pyzor 1 > > > Can anyone help with this? > > -- View this message in context: http://www.nabble.com/Problems-running-Spam-Assassin-tf2664618.html#a7473120 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
RE: Problems running Spam Assassin
Actually I was replying to my forum thread and deleted my original message that it quoted as I thought it had already hit the list. Sietse van Zanen wrote: > > Probably with him being too lazy to copy and paste his original message > from the other board, or list.. > > Well, I am too lazy to follow his link... > > > > From: Theo Van Dinter > Sent: Tue 21-Nov-06 15:24 > To: CosmicPerl > Cc: users@spamassassin.apache.org > Subject: Re: Problems running Spam Assassin > > > On Tue, Nov 21, 2006 at 06:16:15AM -0800, CosmicPerl wrote: >> Can anyone help with this? > > With what? > > -- > Randomly Selected Tagline: > "... the menu is written in more elementary Spanish than a Dora the > Explorer episode ..." > - Karl Chalabala about a lunch menu at work > > -- View this message in context: http://www.nabble.com/Problems-running-Spam-Assassin-tf2664618.html#a7473175 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Problems running Spam Assassin
On Sunday 19 November 2006 18:04, CosmicPerl wrote: > Hi, > I installed the latest SpamAssassin on my server. At first all my tests > looked good, apart from load. So I setup spamc and spamd and everything > seemed great, for a short while at least. > > A day later my mqueue had about 1500 messages in it, most with the error > "local mailer (/usr/bin/procmail) exited with EX_TEMPFAIL". This seems to > be coming up if the mailbox is full or the email is to an address that > doesn't exist. > > It seemed that about every hour or so Sendmail was trying to flush out > these messages, causing 1000's of processes to be started and making the > server freeze up. Despite my Sendmail config having > define(`confMAX_DAEMON_CHILDREN', `12')dnl > > In my procmailrc file I have:- > DROPPRIVS=yes > > :0fw: spamassassin.lock > * < 256000 > | spamc > > The SpamAssassin daemon was started with > /usr/bin/spamd -d -u nobody > > At some point all mail stopped coming in. When I looked at the maillog file > it had lots of lines like:- > mkdir /root/.spamassassin: Permission denied > Which I guess was causing the problem. This wasn't a problem before so I'm > not sure why it happened. Any clues? I guess you might get some problem if you run spamd with -u nobody but without --nouser-config (either spamd will try to access the users' home directories as nobody, or it will try to access the home directory of nobody - I'm not sure, but in either case it will work badly. If you want per-user preferences together with -u you must either use -x --virtual-config-dir, make all users' .spamassassin directories readable (and writable, if you want bayes and/or AWL) by the spamd user (should be a special user - the "nobody" user isn't supposed to have any particular access to any files), or use a database. See README.spamd for security considerations if you have any untrusted users with shell access. -- Magnus Holmgren[EMAIL PROTECTED] (No Cc of list mail needed, thanks) pgpdCeXbvJVW8.pgp Description: PGP signature
RE: Problems running Spam Assassin
These mails stay there for 5 days. At least if you set up sendmail according to RFC's. that's the whole idea of SMTP store and forward. If address is unavailable, keep trying for a while before giving up. You can set the grace time to any period you like btw. -Sietse From: CosmicPerl Sent: Tue 21-Nov-06 16:48 To: users@spamassassin.apache.org Subject: Re: Problems running Spam Assassin Hi All, Ok, I've figured that having "define(`confSEPARATE_PROC', `True')" in my SendMail config was what was causing the flushing of the mail queue to create such a huge server load as it was spawning a new sendmail, procmail, and spamassassin child for each message in the mqueue. So I've disabled this, but I still cannot figure out why mail aimed at non existant uses is still staying in the message queue and not being rejected?? Any help would be very much appreciated. CosmicPerl wrote: > > Hi, > It appears that as I was accepted to the mailing list after making my > first post, my post did not hit the list. Here is my original full post > below:- > > > CosmicPerl wrote: >> >> Hi, >> I installed the latest SpamAssassin on my server. At first all my tests >> looked good, apart from load. So I setup spamc and spamd and everything >> seemed great, for a short while at least. >> >> A day later my mqueue had about 1500 messages in it, most with the error >> "local mailer (/usr/bin/procmail) exited with EX_TEMPFAIL". This seems to >> be coming up if the mailbox is full or the email is to an address that >> doesn't exist. >> >> It seemed that about every hour or so Sendmail was trying to flush out >> these messages, causing 1000's of processes to be started and making the >> server freeze up. Despite my Sendmail config having >> define(`confMAX_DAEMON_CHILDREN', `12')dnl >> >> In my procmailrc file I have:- >> DROPPRIVS=yes >> :0fw: spamassassin.lock >> * < 256000 >> | spamc >> >> The SpamAssassin daemon was started with >> /usr/bin/spamd -d -u nobody >> >> At some point all mail stopped coming in. When I looked at the maillog >> file it had lots of lines like:- >> mkdir /root/.spamassassin: Permission denied >> Which I guess was causing the problem. This wasn't a problem before so >> I'm not sure why it happened. Any clues? >> >> >> Basically I need to set things up so that when sendmail trys to flush I >> don't get my server falling over. >> Emails that are sent to addresses that don't exist that are currently >> getting the error "local mailer (/usr/bin/procmail) exited with >> EX_TEMPFAIL" be delete from the queue automatically. >> Ideally I'd like to give each different virtual server I have it's own >> possibly spam folder. I'm using Webmin and have a 100 or so Virtual >> servers so if anyone knows a good automated way of doing this that would >> be great. Either way I can't have things go down again otherwise I'll >> loose all my clients! >> >> And SpamAssassin working again. At first it was just marking emails with >> [spam] in the subject. Then Yesterday It then also started changing the >> message to an attachment and having "Spam detection software, running on >> the system "ns.cosmicsitehosting.com", has identified this incoming email >> as possible spam..." in the message text. I've no idea what was changed >> so that this started happening. I didn't think I changed anything. Then >> last night it stopped sending any emails. >> >> >> Please help! >> >> Thanks in advance. >> >> Oh by the way my local.cf file contains >> required_hits 10 >> rewrite_header Subject [SPAM] >> report_safe 1 >> use_bayes 1 >> skip_rbl_checks 1 >> use_pyzor 1 >> >> >> Can anyone help with this? >> >> > > -- View this message in context: http://www.nabble.com/Problems-running-Spam-Assassin-tf2664618.html#a7473573 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Problems running Spam Assassin
Hi All, Ok, I've figured that having "define(`confSEPARATE_PROC', `True')" in my SendMail config was what was causing the flushing of the mail queue to create such a huge server load as it was spawning a new sendmail, procmail, and spamassassin child for each message in the mqueue. So I've disabled this, but I still cannot figure out why mail aimed at non existant uses is still staying in the message queue and not being rejected?? Any help would be very much appreciated. CosmicPerl wrote: > > Hi, > It appears that as I was accepted to the mailing list after making my > first post, my post did not hit the list. Here is my original full post > below:- > > > CosmicPerl wrote: >> >> Hi, >> I installed the latest SpamAssassin on my server. At first all my tests >> looked good, apart from load. So I setup spamc and spamd and everything >> seemed great, for a short while at least. >> >> A day later my mqueue had about 1500 messages in it, most with the error >> "local mailer (/usr/bin/procmail) exited with EX_TEMPFAIL". This seems to >> be coming up if the mailbox is full or the email is to an address that >> doesn't exist. >> >> It seemed that about every hour or so Sendmail was trying to flush out >> these messages, causing 1000's of processes to be started and making the >> server freeze up. Despite my Sendmail config having >> define(`confMAX_DAEMON_CHILDREN', `12')dnl >> >> In my procmailrc file I have:- >> DROPPRIVS=yes >> :0fw: spamassassin.lock >> * < 256000 >> | spamc >> >> The SpamAssassin daemon was started with >> /usr/bin/spamd -d -u nobody >> >> At some point all mail stopped coming in. When I looked at the maillog >> file it had lots of lines like:- >> mkdir /root/.spamassassin: Permission denied >> Which I guess was causing the problem. This wasn't a problem before so >> I'm not sure why it happened. Any clues? >> >> >> Basically I need to set things up so that when sendmail trys to flush I >> don't get my server falling over. >> Emails that are sent to addresses that don't exist that are currently >> getting the error "local mailer (/usr/bin/procmail) exited with >> EX_TEMPFAIL" be delete from the queue automatically. >> Ideally I'd like to give each different virtual server I have it's own >> possibly spam folder. I'm using Webmin and have a 100 or so Virtual >> servers so if anyone knows a good automated way of doing this that would >> be great. Either way I can't have things go down again otherwise I'll >> loose all my clients! >> >> And SpamAssassin working again. At first it was just marking emails with >> [spam] in the subject. Then Yesterday It then also started changing the >> message to an attachment and having "Spam detection software, running on >> the system "ns.cosmicsitehosting.com", has identified this incoming email >> as possible spam..." in the message text. I've no idea what was changed >> so that this started happening. I didn't think I changed anything. Then >> last night it stopped sending any emails. >> >> >> Please help! >> >> Thanks in advance. >> >> Oh by the way my local.cf file contains >> required_hits 10 >> rewrite_header Subject [SPAM] >> report_safe 1 >> use_bayes 1 >> skip_rbl_checks 1 >> use_pyzor 1 >> >> >> Can anyone help with this? >> >> > > -- View this message in context: http://www.nabble.com/Problems-running-Spam-Assassin-tf2664618.html#a7473573 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Problems running Spam Assassin
Can anyone help with this? -- View this message in context: http://www.nabble.com/Problems-running-Spam-Assassin-tf2664618.html#a7471981 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Problems running Spam Assassin
On Tue, Nov 21, 2006 at 06:16:15AM -0800, CosmicPerl wrote: > Can anyone help with this? With what? -- Randomly Selected Tagline: "... the menu is written in more elementary Spanish than a Dora the Explorer episode ..." - Karl Chalabala about a lunch menu at work pgpDc6YlHHGbl.pgp Description: PGP signature
RE: Problems running Spam Assassin
Probably with him being too lazy to copy and paste his original message from the other board, or list.. Well, I am too lazy to follow his link... From: Theo Van Dinter Sent: Tue 21-Nov-06 15:24 To: CosmicPerl Cc: users@spamassassin.apache.org Subject: Re: Problems running Spam Assassin On Tue, Nov 21, 2006 at 06:16:15AM -0800, CosmicPerl wrote: > Can anyone help with this? With what? -- Randomly Selected Tagline: "... the menu is written in more elementary Spanish than a Dora the Explorer episode ..." - Karl Chalabala about a lunch menu at work
RE: Problems running Spam Assassin
Hi, I changed it to 2 days. I've also setup a script that checks for double bounce emails in the queue and deletes them automatically. The specific EX_TEMPFAIL i was getting was due to quota issues on some accounts. Seems that if the master user for a domain is over quota then any [EMAIL PROTECTED] emails get stuck in the queue rather than rejected. I'm going to make another script that checks for these emails in the queue and notifys me automatically. Now everything seems to be running smoothly. I've had the [SPAM] added to subjects for over a week now, I'm happy that it's filtering spam and not legit mail. Now I'm having trouble getting it to delete mails marked as spam rather than just add the [SPAM] to the subject. I've added the lines:- :0: * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\* almost-certainly-spam :0: * ^X-Spam-Status: Yes probably-spam To my procmailrc, files containing emails called almost-certainly-spam and probably-spam have appeared in /var/spool/mqueue/, I thought they would have gone in /var/spool/mail? Either way I'm still getting emails coming through with [SPAM] in the subject despite the lines I've added. Not sure what to do now? Please help. Sietse van Zanen wrote: > > These mails stay there for 5 days. At least if you set up sendmail > according to RFC's. > > that's the whole idea of SMTP store and forward. If address is > unavailable, keep trying for a while before giving up. You can set the > grace time to any period you like btw. > > -Sietse > > > > > From: CosmicPerl > Sent: Tue 21-Nov-06 16:48 > To: users@spamassassin.apache.org > Subject: Re: Problems running Spam Assassin > > > Hi All, > Ok, I've figured that having "define(`confSEPARATE_PROC', `True')" in my > SendMail config was what was causing the flushing of the mail queue to > create such a huge server load as it was spawning a new sendmail, > procmail, > and spamassassin child for each message in the mqueue. So I've disabled > this, but I still cannot figure out why mail aimed at non existant uses is > still staying in the message queue and not being rejected?? Any help would > be very much appreciated. > > > > CosmicPerl wrote: >> >> Hi, >> It appears that as I was accepted to the mailing list after making my >> first post, my post did not hit the list. Here is my original full post >> below:- >> >> >> CosmicPerl wrote: >>> >>> Hi, >>> I installed the latest SpamAssassin on my server. At first all my >>> tests >>> looked good, apart from load. So I setup spamc and spamd and everything >>> seemed great, for a short while at least. >>> >>> A day later my mqueue had about 1500 messages in it, most with the error >>> "local mailer (/usr/bin/procmail) exited with EX_TEMPFAIL". This seems >>> to >>> be coming up if the mailbox is full or the email is to an address that >>> doesn't exist. >>> >>> It seemed that about every hour or so Sendmail was trying to flush out >>> these messages, causing 1000's of processes to be started and making the >>> server freeze up. Despite my Sendmail config having >>> define(`confMAX_DAEMON_CHILDREN', `12')dnl >>> >>> In my procmailrc file I have:- >>> DROPPRIVS=yes >>> :0fw: spamassassin.lock >>> * < 256000 >>> | spamc >>> >>> The SpamAssassin daemon was started with >>> /usr/bin/spamd -d -u nobody >>> >>> At some point all mail stopped coming in. When I looked at the maillog >>> file it had lots of lines like:- >>> mkdir /root/.spamassassin: Permission denied >>> Which I guess was causing the problem. This wasn't a problem before so >>> I'm not sure why it happened. Any clues? >>> >>> >>> Basically I need to set things up so that when sendmail trys to flush I >>> don't get my server falling over. >>> Emails that are sent to addresses that don't exist that are currently >>> getting the error "local mailer (/usr/bin/procmail) exited with >>> EX_TEMPFAIL" be delete from the queue automatically. >>> Ideally I'd like to give each different virtual server I have it's own >>> possibly spam folder. I'm using Webmin and have a 100 or so Virtual >>> servers so if anyone knows a good automated way of doing this that would >>> be great. Either way I can't have things go down again otherwise I'll >>> loose all my clients! >>> >>> And S
Re: Problems running Spam Assassin
Hi, But I want users to be able to setup config directories. Do I need to start spamd in a different way? Do I need to run it as root, or is that to big a security hole? >You shouldn't accept mails for invalid addresses, then they won't be in the >queue at all. Sounds great, but how do I do that? I figured out the report_safe thing. Thanks in advance. Theo Van Dinter-2 wrote: > > On Tue, Nov 21, 2006 at 07:21:22AM -0800, CosmicPerl wrote: >> > The SpamAssassin daemon was started with >> > /usr/bin/spamd -d -u nobody >> > >> > At some point all mail stopped coming in. When I looked at the maillog >> > file it had lots of lines like:- >> > mkdir /root/.spamassassin: Permission denied >> > Which I guess was causing the problem. This wasn't a problem before so >> I'm >> > not sure why it happened. Any clues? > > You're telling spamd to run as user nobody, but haven't told it to stop > using > user config directories. So when mail comes in for root, root calls > spamd, > spamd can't access root's home dir, and you get the error message. > >> > Emails that are sent to addresses that don't exist that are currently >> > getting the error "local mailer (/usr/bin/procmail) exited with >> > EX_TEMPFAIL" be delete from the queue automatically. > > You shouldn't accept mails for invalid addresses, then they won't be in > the > queue at all. > >> > And SpamAssassin working again. At first it was just marking emails >> with >> > [spam] in the subject. Then Yesterday It then also started changing the >> > message to an attachment and having "Spam detection software, running >> on >> > the system "ns.cosmicsitehosting.com", has identified this incoming >> email >> > as possible spam..." in the message text. I've no idea what was changed >> so >> > that this started happening. I didn't think I changed anything. Then >> last >> > night it stopped sending any emails. > > your config enables report_safe, which causes SA to encapsulate the > original > message as an attachment. > >> > Oh by the way my local.cf file contains > [...] >> > report_safe 1 > > -- > Randomly Selected Tagline: > "She's got a mortgage on my body and a lease on my soul." > > > -- View this message in context: http://www.nabble.com/Problems-running-Spam-Assassin-tf2664618.html#a7562313 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Problems running Spam Assassin
Hi, Thanks for the reply. I want users to have the option to have their own spamassassin configuration. Is it best just to start spamd without -u? I've got over 100 virtual servers, I don't want to setup folders and permissions for each user. What's the easiest and safest way? How would you do it? Thanks Magnus Holmgren wrote: > > On Sunday 19 November 2006 18:04, CosmicPerl wrote: >> Hi, >> I installed the latest SpamAssassin on my server. At first all my tests >> looked good, apart from load. So I setup spamc and spamd and everything >> seemed great, for a short while at least. >> >> A day later my mqueue had about 1500 messages in it, most with the error >> "local mailer (/usr/bin/procmail) exited with EX_TEMPFAIL". This seems to >> be coming up if the mailbox is full or the email is to an address that >> doesn't exist. >> >> It seemed that about every hour or so Sendmail was trying to flush out >> these messages, causing 1000's of processes to be started and making the >> server freeze up. Despite my Sendmail config having >> define(`confMAX_DAEMON_CHILDREN', `12')dnl >> >> In my procmailrc file I have:- >> DROPPRIVS=yes >> >> :0fw: spamassassin.lock >> * < 256000 >> | spamc >> >> The SpamAssassin daemon was started with >> /usr/bin/spamd -d -u nobody >> >> At some point all mail stopped coming in. When I looked at the maillog >> file >> it had lots of lines like:- >> mkdir /root/.spamassassin: Permission denied >> Which I guess was causing the problem. This wasn't a problem before so >> I'm >> not sure why it happened. Any clues? > > I guess you might get some problem if you run spamd with -u nobody but > without --nouser-config (either spamd will try to access the users' home > directories as nobody, or it will try to access the home directory of > nobody - I'm not sure, but in either case it will work badly. If you want > per-user preferences together with -u you must either > use -x --virtual-config-dir, make all users' .spamassassin directories > readable (and writable, if you want bayes and/or AWL) by the spamd user > (should be a special user - the "nobody" user isn't supposed to have any > particular access to any files), or use a database. > > See README.spamd for security considerations if you have any untrusted > users > with shell access. > > -- > Magnus Holmgren[EMAIL PROTECTED] >(No Cc of list mail needed, thanks) > > > -- View this message in context: http://www.nabble.com/Problems-running-Spam-Assassin-tf2664618.html#a7572889 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Problems running Spam Assassin
Hi, Thanks for the reply. I want users to have the option to have their own spamassassin configuration. Is it best just to start spamd without -u? I've got over 100 virtual servers, I don't want to setup folders and permissions for each user. What's the easiest and safest way? How would you do it? Thanks Hi again, While I'm next to no help to you in getting it set up, Dale's suggestion of Maia Mailguard is a great one. It enables both site-wide configuration & also individual user mailboxes for classifying spam, rescuing false positives etc... all within a web interface. -- Bye for now, Terry Allen ___ hEARd Postal Address: hEARd, 26B Glenning Rd, Glenning Valley, NSW 2261, Australia Internet - WWW: http://heard.com.au http://itavservices.com EMAIL: [EMAIL PROTECTED] Phone: Australia - 02 4388 1400 / International - + 61 2 43881400 Mobile: Australia - 04 28881400 / International - 61 4 28881400 --- Non profit promotion for new music - since 1994 ---
RE: Problems running Spam Assassin
Hi, Ok, I've updated my procmailrc file to have:- MAILDIR=/var/spool/mail This has fixed procmail putting spam emails in probably-spam and almost-certainly-spam mail boxes (I also created each of those as a user, but don't think it was that which made the real difference). Now the confusing thing is that even though both these mail boxes are filling up with the spam emails, the mails are still being delivered to the users mail boxes. Looking at my procmail log that's recording with the config:- VERBOSE=yes LOGABSTRACT=all LOGFILE=/var/log/procmail.log Also I've noticed that all emails procmail logs through that are not directed to probably-spam or almost-certainly-spam are apparently put in /var/mail/root procmail: Opening "/var/mail/root" procmail: Acquiring kernel-lock procmail: Unlocking "/var/mail/root.lock" procmail: Notified comsat: "[EMAIL PROTECTED]:/var/mail/root" >From [EMAIL PROTECTED] Wed Nov 29 01:10:57 2006 Subject: Postmaster notify: see transcript for details Folder: /var/mail/root Am I to take it that some other program is at work taking emails that have been through procmail then forwarding to the mail boxes?? They must be going to procmail first to have the SPAMASSASSIN [SPAM] header tags. I'm on the last hurdle. If someone could help me with this I'd be VERY appreciative. CosmicPerl wrote: > > Hi, > I changed it to 2 days. I've also setup a script that checks for double > bounce emails in the queue and deletes them automatically. The specific > EX_TEMPFAIL i was getting was due to quota issues on some accounts. Seems > that if the master user for a domain is over quota then any > [EMAIL PROTECTED] emails get stuck in the queue rather than > rejected. I'm going to make another script that checks for these emails in > the queue and notifys me automatically. > > Now everything seems to be running smoothly. I've had the [SPAM] added to > subjects for over a week now, I'm happy that it's filtering spam and not > legit mail. Now I'm having trouble getting it to delete mails marked as > spam rather than just add the [SPAM] to the subject. > > I've added the lines:- > :0: > * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\* > almost-certainly-spam > > :0: > * ^X-Spam-Status: Yes > probably-spam > > To my procmailrc, files containing emails called almost-certainly-spam and > probably-spam have appeared in /var/spool/mqueue/, I thought they would > have gone in /var/spool/mail? > > Either way I'm still getting emails coming through with [SPAM] in the > subject despite the lines I've added. Not sure what to do now? Please > help. > > > > Sietse van Zanen wrote: >> >> These mails stay there for 5 days. At least if you set up sendmail >> according to RFC's. >> >> that's the whole idea of SMTP store and forward. If address is >> unavailable, keep trying for a while before giving up. You can set the >> grace time to any period you like btw. >> >> -Sietse >> >> >> >> >> From: CosmicPerl >> Sent: Tue 21-Nov-06 16:48 >> To: users@spamassassin.apache.org >> Subject: Re: Problems running Spam Assassin >> >> >> Hi All, >> Ok, I've figured that having "define(`confSEPARATE_PROC', `True')" in >> my >> SendMail config was what was causing the flushing of the mail queue to >> create such a huge server load as it was spawning a new sendmail, >> procmail, >> and spamassassin child for each message in the mqueue. So I've disabled >> this, but I still cannot figure out why mail aimed at non existant uses >> is >> still staying in the message queue and not being rejected?? Any help >> would >> be very much appreciated. >> >> >> >> CosmicPerl wrote: >>> >>> Hi, >>> It appears that as I was accepted to the mailing list after making my >>> first post, my post did not hit the list. Here is my original full post >>> below:- >>> >>> >>> CosmicPerl wrote: >>>> >>>> Hi, >>>> I installed the latest SpamAssassin on my server. At first all my >>>> tests >>>> looked good, apart from load. So I setup spamc and spamd and everything >>>> seemed great, for a short while at least. >>>> >>>> A day later my mqueue had about 1500 messages in it, most with the >>>> error >>>> "local mailer (/usr/bin/procmail) exited with EX_TEMPFAIL". This seems >>>> to >>>> be coming up if the mailbox is full or the em
RE: Problems running Spam Assassin
Hi, Does it have something to do with /etc/mail/virtusertable??? CosmicPerl wrote: > > Hi, > Ok, I've updated my procmailrc file to have:- > MAILDIR=/var/spool/mail > This has fixed procmail putting spam emails in probably-spam and > almost-certainly-spam mail boxes (I also created each of those as a user, > but don't think it was that which made the real difference). > > Now the confusing thing is that even though both these mail boxes are > filling up with the spam emails, the mails are still being delivered to > the users mail boxes. Looking at my procmail log that's recording with the > config:- > VERBOSE=yes > LOGABSTRACT=all > LOGFILE=/var/log/procmail.log > > Also I've noticed that all emails procmail logs through that are not > directed to probably-spam or almost-certainly-spam are apparently put in > /var/mail/root > procmail: Opening "/var/mail/root" > procmail: Acquiring kernel-lock > procmail: Unlocking "/var/mail/root.lock" > procmail: Notified comsat: "[EMAIL PROTECTED]:/var/mail/root" > From [EMAIL PROTECTED] Wed Nov 29 01:10:57 2006 > Subject: Postmaster notify: see transcript for details > Folder: /var/mail/root > > Am I to take it that some other program is at work taking emails that have > been through procmail then forwarding to the mail boxes?? They must be > going to procmail first to have the SPAMASSASSIN [SPAM] header tags. > > I'm on the last hurdle. If someone could help me with this I'd be VERY > appreciative. > > > > > CosmicPerl wrote: >> >> Hi, >> I changed it to 2 days. I've also setup a script that checks for double >> bounce emails in the queue and deletes them automatically. The specific >> EX_TEMPFAIL i was getting was due to quota issues on some accounts. Seems >> that if the master user for a domain is over quota then any >> [EMAIL PROTECTED] emails get stuck in the queue rather than >> rejected. I'm going to make another script that checks for these emails >> in the queue and notifys me automatically. >> >> Now everything seems to be running smoothly. I've had the [SPAM] added to >> subjects for over a week now, I'm happy that it's filtering spam and not >> legit mail. Now I'm having trouble getting it to delete mails marked as >> spam rather than just add the [SPAM] to the subject. >> >> I've added the lines:- >> :0: >> * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\* >> almost-certainly-spam >> >> :0: >> * ^X-Spam-Status: Yes >> probably-spam >> >> To my procmailrc, files containing emails called almost-certainly-spam >> and probably-spam have appeared in /var/spool/mqueue/, I thought they >> would have gone in /var/spool/mail? >> >> Either way I'm still getting emails coming through with [SPAM] in the >> subject despite the lines I've added. Not sure what to do now? Please >> help. >> >> >> >> Sietse van Zanen wrote: >>> >>> These mails stay there for 5 days. At least if you set up sendmail >>> according to RFC's. >>> >>> that's the whole idea of SMTP store and forward. If address is >>> unavailable, keep trying for a while before giving up. You can set the >>> grace time to any period you like btw. >>> >>> -Sietse >>> >>> >>> >>> >>> From: CosmicPerl >>> Sent: Tue 21-Nov-06 16:48 >>> To: users@spamassassin.apache.org >>> Subject: Re: Problems running Spam Assassin >>> >>> >>> Hi All, >>> Ok, I've figured that having "define(`confSEPARATE_PROC', `True')" in >>> my >>> SendMail config was what was causing the flushing of the mail queue to >>> create such a huge server load as it was spawning a new sendmail, >>> procmail, >>> and spamassassin child for each message in the mqueue. So I've disabled >>> this, but I still cannot figure out why mail aimed at non existant uses >>> is >>> still staying in the message queue and not being rejected?? Any help >>> would >>> be very much appreciated. >>> >>> >>> >>> CosmicPerl wrote: >>>> >>>> Hi, >>>> It appears that as I was accepted to the mailing list after making my >>>> first post, my post did not hit the list. Here is my original full post >>>> below:- >>>> >>>> >>>> CosmicPerl wrote: >>>>&
Re: Spam Assassin Logfile analysis
On Fri, Dec 22, 2006 at 10:00:29PM +, Paul Hurley wrote: > What I can't work out at the moment is to do anything usefull with the > rules that were hit. I suppose I could create a list of the top ten > rules for Spam and Ham... The idea behind having the log entries like that is that it could be sent through hit-frequencies (in masses/ dir) and be turned into something useful, but I don't think there has been any real work into doing things with it. fyi. -- Randomly Selected Tagline: "I am not a vegetarian because I love animals; I am a vegetarian because I hate plants." - A. Whitney Brown pgpisVJSq5nPc.pgp Description: PGP signature
New Spam Assassin Setup 3.1
Hello Everyone, I installed SA 3.1 last night from source. I followed the qmailrocks.org guide (since I use qmail) which seemed pretty easy when came to setting up SA. This setup uses qmail-queue to invoke SA and Clamav. My problem is when I monitor the mail server I see SA dominating the cpu @ 97% or more when new mail is coming into the server. I also noticed that it (SA) starts to recognize some mail and pass it right through with little delay. I'm still concerned and looked at the SA website for help on cpu utilization but no luck. Can anyone point me in the right direction for troublshooting this? I did tail the syslog and found one problem which I fixed but that did not resolve the cpu % issue. Thank you for your consideration, Tim PS: This is from the mail header of the last Spam msg I recieved: 1.25-st-qms (Clear:RC:0(63.252.150.43):SA:1(6.6/4.0):.* Processed in 149.986422* secs Process 6433) Non spam msg are processed anywhere from 2 -6 seconds.
Hiring for Spam Assassin Troubleshooting
We already have SA setup and working with Smartermail. We would like to hire someone that is familiar with SpamAssassin and a Windows 2003 server system in order to come set this thing up so that our own emails are not being detected as SPAM. Is anyone interested in this? If you are interested leave me a way to get in contact with you. Thanks, Wayne --- We already have Spam Assassin set up and working. I have done as much as I can figure out how to do and am capable of and I still cannot get these errors gone. I am done trying. (If anyone didn't see my previous post, I was interested in help with removing the NO_DNS_FOR_FROMDNS: Envelope sender has no MX or A DNS records. I have checked, we have MX and A DNS, it is set up correctly, I have our emails on whitelist received from, I have our IPs on Whitelist Froms and Trusted Networks. I still cannot get rid of SA reporting that we do not have that and tacking on 2.6 points onto our emails because of it. View this message in context: Hiring for Spam Assassin Troubleshooting Sent from the SpamAssassin - Users forum at Nabble.com.
RE: Spam assassin and postfix..
There are a few things that you can do a little differently. You can return the status of the spam check as well as the marked up message. Then just check if $? -ne 0 (or whatever logic you like). This is what we do. In the event that the message is marked as a spam we try to insert it into a database (the clean version of it). If that fails we drop the marked up version back into the postfix pipeline. -Original Message- From: J Rangi [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 07, 2006 4:44 PM To: spam mailling list Subject: Spam assassin and postfix.. Hello, I configured sapmassassin with postfix. Sapmassassin version is spamassassin-3.0.3-4.fc4 Here is my spam filter script.. ** [EMAIL PROTECTED] log]# cat /usr/local/bin/spamfilter #variables SENDMAIL="/usr/sbin/sendmail.postfix -i" EGREP=/bin/egrep # Exit codes from EX_UNAVAILABLE=69 # Number of *'s in X-Spam-level header needed to sideline message: # (Eg. Score of 5.5 = "*" ) SPAMLIMIT=5 # Clean up when done or when aborting. trap "rm -f /var/tempfs/out.$$" 0 1 2 3 15 # Pipe message to spamc cat | /usr/bin/spamc -u spamfilter > /var/tempfs/out.$$ if $EGREP -q "^X-Spam-Level: \*{$SPAMLIMIT,}" < /var/tempfs/out.$$ then ## Change the Email address where you want your spam to get fwd to $SENDMAIL -f [EMAIL PROTECTED] < /var/tempfs/out.$$ else ###$SENDMAIL "$@" < /var/tempfs/out.$$ $SENDMAIL $@ < /var/tempfs/out.$$ fi # Postfix returns the exit status of the Postfix sendmail command. exit $? * ** I made these changes in master.cf file.. Changed this line by adding "-o content_filter=spamfilter:dummy" to the default *smtp inet n - n - - smtpd -o content_filter=spamfilter:dummy * Added next two lines.. *spamfilter unix - n n - - pipe flags=Rq user=spamfilter argv=/usr/local/bin/spamfilter -f ${sender} -- ${recipient} * Once postfix reloaded I can see that mails are being processed by spamfilter. But for some mails I get these kind of error in the log file and user receives mail from MAILER-DAEMON Can some please tell me why we get these only for some mail and how to get rid of this problem. Jun 7 10:51:44 localmail spamd[14011]: spamd: identified spam (17.8/6.8) for spamfilter:7715 in 2.3 seconds, 1753 bytes. Jun 7 10:51:44 localmail spamd[14011]: spamd: result: Y 17 - MSGID_FROM_MTA_HEADER,MSGID_FROM_MTA_ID,RCVD_IN_BL_SPAMCOP_NET,UNPARSEAB LE_RELAY,URIBL_AB_SURBL,URIBL_JP_SURBL,URIBL_OB_SURBL,URIBL_SBL,URIBL_SC _SURBL,URIBL_WS_SURBL scantime=2.3,size=1753,user=spamfilter,uid=7715,required_score=6.8,rhost =localhost.localdomain,raddr=127.0.0.1,rport=33304,mid=<200606071751.k57 [EMAIL PROTECTED]>,autolearn=no Jun 7 10:51:44 localmail postfix/sendmail[14909]: fatal: Recipient addresses must be specified on the command line or via the -t option Jun 7 10:51:44 localmail spamd[14009]: prefork: child states: II Jun 7 17:51:45 localmail postfix/postdrop[14910]: warning: stdin: unexpected EOF in data, record type 78 length 85 Jun 7 10:51:45 localmail postfix/postdrop[14910]: fatal: uid=7715: malformed input Jun 7 10:51:46 localmail postfix/pipe[13865]: DA97E60EB2: to=<[EMAIL PROTECTED]>, relay=spamfilter, delay=5, status=bounced (command line usage error. Command output: sendmail.postfix: fatal: Recipient addresses must be specified on the command line or via the -t option postdrop: warning: stdin: unexpected EOF in data, record type 78 length 85 postdrop: fatal: uid=7715: malformed input ) Jun 7 10:51:46 localmail postfix/cleanup[13864]: 6AF4562F46: message-id=<[EMAIL PROTECTED]> Jun 7 10:51:46 localmail postfix/qmgr[13851]: 6AF4562F46: from=<>, size=3990, nrcpt=1 (queue active) Jun 7 10:51:46 localmail postfix/qmgr[13851]: DA97E60EB2: removed Jun 7 10:51:46 localmail postfix/smtp[14867]: 6AF4562F46: to=<[EMAIL PROTECTED]>, relay=mail.aleks.com[216.34.240.136], delay=0, status=sent (250 2.0.0 k57HpkM18899 Message accepted for delivery) Jun 7 10:51:46 localmail postfix/qmgr[13851]: 6AF4562F46: removed Jun 7 10:51:48 localmail postfix/smtpd[13861]: connect from ip26.aleks.com[216.34.240.160] Jun 7 10:51:48 localmail postfix/smtpd[13861]: 5F35C60EB2: client=ip26.aleks.com[216.34.240.160] Jun 7 10:51:48 localmail postfix/cleanup[13913]: 5F35C60EB2: message-id=<[EMAIL PROTECTED]>
Re: Spam assassin and postfix..
On Wed, 2006-06-07 at 16:43 -0700, J Rangi wrote: > Hello, > I configured sapmassassin with postfix. > Sapmassassin version is spamassassin-3.0.3-4.fc4 > Here is my spam filter script.. > > ** > [EMAIL PROTECTED] log]# cat /usr/local/bin/spamfilter > #variables > SENDMAIL="/usr/sbin/sendmail.postfix -i" > EGREP=/bin/egrep > # Exit codes from > EX_UNAVAILABLE=69 > # Number of *'s in X-Spam-level header needed to sideline message: > # (Eg. Score of 5.5 = "*" ) > SPAMLIMIT=5 > # Clean up when done or when aborting. > trap "rm -f /var/tempfs/out.$$" 0 1 2 3 15 > # Pipe message to spamc > cat | /usr/bin/spamc -u spamfilter > /var/tempfs/out.$$ > > if $EGREP -q "^X-Spam-Level: \*{$SPAMLIMIT,}" < /var/tempfs/out.$$ > then > ## Change the Email address where you want your spam to get fwd to > $SENDMAIL -f [EMAIL PROTECTED] < /var/tempfs/out.$$ > else > ###$SENDMAIL "$@" < /var/tempfs/out.$$ > $SENDMAIL $@ < /var/tempfs/out.$$ > fi > # Postfix returns the exit status of the Postfix sendmail command. > exit $? * > ** > > I made these changes in master.cf file.. > Changed this line by adding "-o content_filter=spamfilter:dummy" to the > default > *smtp inet n - n - - smtpd -o > content_filter=spamfilter:dummy * > Added next two lines.. > *spamfilter unix - n n - - pipe > flags=Rq user=spamfilter argv=/usr/local/bin/spamfilter -f ${sender} > -- ${recipient} * > > Once postfix reloaded I can see that mails are being processed by > spamfilter. > But for some mails I get these kind of error in the log file and user > receives mail from MAILER-DAEMON > Can some please tell me why we get these only for some mail and how to > get rid of this problem. > > Jun 7 10:51:44 localmail spamd[14011]: spamd: identified spam > (17.8/6.8) for spamfilter:7715 in 2.3 seconds, 1753 bytes. > Jun 7 10:51:44 localmail spamd[14011]: spamd: result: Y 17 - > MSGID_FROM_MTA_HEADER,MSGID_FROM_MTA_ID,RCVD_IN_BL_SPAMCOP_NET,UNPARSEABLE_RELAY,URIBL_AB_SURBL,URIBL_JP_SURBL,URIBL_OB_SURBL,URIBL_SBL,URIBL_SC_SURBL,URIBL_WS_SURBL > > scantime=2.3,size=1753,user=spamfilter,uid=7715,required_score=6.8,rhost=localhost.localdomain,raddr=127.0.0.1,rport=33304,mid=<[EMAIL > PROTECTED]>,autolearn=no > > Jun 7 10:51:44 localmail postfix/sendmail[14909]: fatal: Recipient > addresses must be specified on the command line or via the -t option > Jun 7 10:51:44 localmail spamd[14009]: prefork: child states: II > Jun 7 17:51:45 localmail postfix/postdrop[14910]: warning: stdin: > unexpected EOF in data, record type 78 length 85 > Jun 7 10:51:45 localmail postfix/postdrop[14910]: fatal: uid=7715: > malformed input > Jun 7 10:51:46 localmail postfix/pipe[13865]: DA97E60EB2: > to=<[EMAIL PROTECTED]>, relay=spamfilter, delay=5, > status=bounced (command line usage error. Command output: > sendmail.postfix: fatal: Recipient addresses must be specified on the > command line or via the -t option postdrop: warning: stdin: unexpected > EOF in data, record type 78 length 85 postdrop: fatal: uid=7715: > malformed input ) The clue is in the error log. It says you must use "-t" or specify the recipient on the sendmail command line. In the case the message is detected as spam, you do this: $SENDMAIL -f [EMAIL PROTECTED] < /var/tempfs/out.$$ There's no -t and no recipient. The -f option is the "from" part. I suspect you want to send spam to [EMAIL PROTECTED], in which case try something like this: $SENDMAIL -f $2 [EMAIL PROTECTED] < /var/tempfs/out.$$ $2 should be the sender, as passed into the filter script. Rather than calling spamc on each message, you might also consider a daemon solution which will reduce the overhead and startup delay time. Very helpful if processing a lot of mail. I use amavisd-new and have it run clamd before spamassassin. There are others that have been mentioned in this mailing list, as well. -Bill
Re: New Spam Assassin user
Mike L wrote: I am a new user. I am running on Windows 2003. I have several domains on my servers. I only want one domain on my server to use spam assassin. Where and what do i need to do to only filter for 1 domain on my sever. Is this possible. I would also like to setup wrongmx on this as wel.. Thanks in advance. Mike You would do this with the tool that calls SA. There are some rules that send the spam to the users, but if you want to bypass SA all together, you'd do this with the tool that is actually calling SpamAssassin (be it spamc/spamd or spamassassin directly.) As far as wrongmx, that was pretty much answered in one of your other threads. Quoting Daryl: "Yeah, put the plugin files in your local config directory... that's the same directory that has you local.cf file. If you want to only enable the WRONG_MX rule for a particular domain, you'll have to configure per user or domain scoring." I haven't seen any specs on your system, can you post the MTA and how you are calling SpamAssassin? This will help us help you. -- Thanks, JamesDR smime.p7s Description: S/MIME Cryptographic Signature
Spam Assassin If Plugin Method ...
hii, Does if plugin method of spamassassin supports else part. Eg: I have created a custom ruleset which works with SPF rulset so in my cf file i check for whether Mail::SpamAssasssin::Plugin is enable or not, if it is enable and SPF ruleset is getting hit, i assign x score for my custom custom rules. But if Mail::SpamAssassin::Plugin is disabled i wan to assign some y score to ruleset. If "ifplugin" method supports else part i can very well achieve my objective. So is it possible ? begin:vcard fn:Ashok kumar Gupta n:Gupta;Ashok kumar org:Netcore Solution Pvt Ltd;EmerGic CleanMail adr:Lower Parel (West).;;402, Peninsula Chambers,Peninsula Corporate Park,GanPat Rao Kadam Marg, ;Mumbai;;400 013;India email;internet:[EMAIL PROTECTED] title:Lead Developer tel;work:022266628174 x-mozilla-html:FALSE url:http://netcore.co.in version:2.1 end:vcard
Re: Spam Assassin White List
On Wed, 18 Mar 2009, dsh979 wrote: I have found that when I add manually a user to the whitelist (in the SpamAssassin user preferences file) I get inconsistent results: ... I have also found that when I manually a user to the blacklist (in the SpamAssassin user preferences file) I get the following result: How _exactly_ are you adding users to the whitelist and blacklist? Give us examples of what you're adding to the config file. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...in the 2nd amendment the right to arms clause means you have the right to choose how many arms you want, and the militia clause means that Congress can punish you if the answer is "none." -- David Hardy, 2nd Amendment scholar --- 1327 days until the Presidential Election
Re: Spam Assassin White List
Hello John Thanks for your reply. I am adding users to the white list and the black list (in the SpamAssassin user preferences file) as follows: blacklist_from *...@blacklist1.com blacklist_from *...@blacklist2.com blacklist_from *...@blacklist3.com required_score 100 whitelist_from *...@whitelist1.com whitelist_from *...@whitelist2.com whitelist_from *...@whitelist3.com John Hardin wrote: > > On Wed, 18 Mar 2009, dsh979 wrote: > >> I have found that when I add manually a user to the whitelist (in the >> SpamAssassin user preferences file) I get inconsistent results: > ... >> I have also found that when I manually a user to the blacklist (in the >> SpamAssassin user preferences file) I get the following result: > > How _exactly_ are you adding users to the whitelist and blacklist? Give > us examples of what you're adding to the config file. > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- >...in the 2nd amendment the right to arms clause means you have >the right to choose how many arms you want, and the militia clause >means that Congress can punish you if the answer is "none." > -- David Hardy, 2nd Amendment scholar > --- > 1327 days until the Presidential Election > > -- View this message in context: http://www.nabble.com/Spam-Assassin-White-List-tp22589650p22673278.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Spam Assassin White List
dsh979 wrote: > Hello John > > Thanks for your reply. I am adding users to the white list and the black > list (in the SpamAssassin user preferences file) as follows: > > blacklist_from *...@blacklist1.com > blacklist_from *...@blacklist2.com > blacklist_from *...@blacklist3.com > required_score 100 > whitelist_from *...@whitelist1.com > whitelist_from *...@whitelist2.com > whitelist_from *...@whitelist3.com > > Why do you have the required_score 100 in there? That could prevent your blacklists from working 100% of the time. The blacklist works by adding +100 to the message score, but if the other rules it matches come out negative, the blacklist won't be effective because the total score will be under 100.
Re: Spam Assassin White List
> Thanks for your reply. I am adding users to the white list and the black > list (in the SpamAssassin user preferences file) as follows: > whitelist_from *...@whitelist1.com whitelist_from should be used as a last resort; whitelist_from_auth and whitelist_from_rcvd are significantly safer in a world where spammers forge "From:" addresses constantly. -- Dave Pooser Cat-Herder-in-Chief, Pooserville.com "You're useless when you're high on catnip, you know that?"
Re: Spam Assassin White List
Thank you for your reply Matt. I did not realise that items listed on the white list or the black list would still be subject to the operation/analysis of the SpamAssassin Rules. You have asked why I have set the required score the 100. Lengthy explanation (sorry). I have done this to prevent SpamAssassin from inserting SpamWarnings into the header/body of the relevant email. In responding to spam I rely on the SpamAssassin Score in conjunction with other "email message indicators"), and incorporate these variables into a domain level filter (cPanel). Mail is then bounced (by the filter) without any warning in the bounced email itself, that it has been bounced because it has been identified as spam. In fact, the bounced email will have a message inserted to the effect that there is no such user/receipient. In this way, if there is a sender who receives the bounced email, hopefully they take me off their mailing list, instead of looking for a way to 'outsmart' the SpamRules. Q:How can I list items/users on a "white list" or a "black list" without the lists (and items) being the subject of further analysis by the SpamAssassin Rules (and therefore obtaining the same score for each item on the relevant list, irrespective of the operation of the SpamAssassin Rules, that is -100=white list items & +100 = black list items)? Matt Kettler-3 wrote: > > dsh979 wrote: >> Hello John >> >> Thanks for your reply. I am adding users to the white list and the black >> list (in the SpamAssassin user preferences file) as follows: >> >> blacklist_from *...@blacklist1.com >> blacklist_from *...@blacklist2.com >> blacklist_from *...@blacklist3.com >> required_score 100 >> whitelist_from *...@whitelist1.com >> whitelist_from *...@whitelist2.com >> whitelist_from *...@whitelist3.com >> >> > > Why do you have the required_score 100 in there? > > That could prevent your blacklists from working 100% of the time. > > The blacklist works by adding +100 to the message score, but if the > other rules it matches come out negative, the blacklist won't be > effective because the total score will be under 100. > > > -- View this message in context: http://www.nabble.com/Spam-Assassin-White-List-tp22589650p22674314.html Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
Re: Spam Assassin White List
On Mon, March 23, 2009 10:58 pm, dsh979 wrote: > > Thank you for your reply Matt. > > I did not realise that items listed on the white list or the black list > would still be subject to the operation/analysis of the SpamAssassin > Rules. > > You have asked why I have set the required score the 100. Lengthy > explanation (sorry). I have done this to prevent SpamAssassin from > inserting SpamWarnings into the header/body of the relevant email. In > responding to spam I rely on the SpamAssassin Score in conjunction with > other "email message indicators"), and incorporate these variables into > a > domain level filter (cPanel). Mail is then bounced (by the filter) > without > any warning in the bounced email itself, that it has been bounced > because it > has been identified as spam. In fact, the bounced email will have a > message > inserted to the effect that there is no such user/receipient. In this > way, > if there is a sender who receives the bounced email, hopefully they take > me > off their mailing list, instead of looking for a way to 'outsmart' the > SpamRules. > > Q:How can I list items/users on a "white list" or a "black list" without > the > lists (and items) being the subject of further analysis by the > SpamAssassin > Rules (and therefore obtaining the same score for each item on the > relevant > list, irrespective of the operation of the SpamAssassin Rules, that is > -100=white list items & +100 = black list items)? > A couple thoughts: 1. by returning the emails, you run the risk of false-negatives and thus creating 'email backscatter' (see wikipedia). 2. If you don't want to receive these things at all, have you considered using your MTA to block the actual IP addresses of known spammers using a couple of rules like (for sendmail): FEATURE(`dnsbl', `bl.spamcop.net',`"Rejected as Spam. See http://bl.spamcop.net?"$&{clientaddr}"; for more information"')dnl FEATURE(`dnsbl', `zen.spamhaus.org',`"Rejected as Spam. See http://spamhaus.org/query/bl?ip="$&{clientaddr}"; for more information"')dnl which rejects the email long before SA has to be bothered? When I check my logs, the spamcop rule alone blocks as many as 800-1100 email daily. Just something to consider. Karl > > > > Matt Kettler-3 wrote: >> >> dsh979 wrote: >>> Hello John >>> >>> Thanks for your reply. I am adding users to the white list and the >>> black >>> list (in the SpamAssassin user preferences file) as follows: >>> >>> blacklist_from *...@blacklist1.com >>> blacklist_from *...@blacklist2.com >>> blacklist_from *...@blacklist3.com >>> required_score 100 >>> whitelist_from *...@whitelist1.com >>> whitelist_from *...@whitelist2.com >>> whitelist_from *...@whitelist3.com >>> >>> >> >> Why do you have the required_score 100 in there? >> >> That could prevent your blacklists from working 100% of the time. >> >> The blacklist works by adding +100 to the message score, but if the >> other rules it matches come out negative, the blacklist won't be >> effective because the total score will be under 100. >> >> >> > > -- > View this message in context: > http://www.nabble.com/Spam-Assassin-White-List-tp22589650p22674314.html > Sent from the SpamAssassin - Users mailing list archive at Nabble.com. > --- Karl Pearson ka...@ourldsfamily.com Owner/Administrator of the sites at http://ourldsfamily.com --- "To mess up your Linux PC, you have to really work at it; to mess up a microsoft PC you just have to work on it." ---
Re: Spam Assassin White List
On 23.03.09 21:58, dsh979 wrote: > I did not realise that items listed on the white list or the black list > would still be subject to the operation/analysis of the SpamAssassin Rules. all rules are processed unless you play with ShortCircuit plugin. Beware of that: It may render the SA useless if you don't knwo what you are doing. > You have asked why I have set the required score the 100. Lengthy > explanation (sorry). I have done this to prevent SpamAssassin from > inserting SpamWarnings into the header/body of the relevant email. There's report_safe option to configure that. > In responding to spam I rely on the SpamAssassin Score in conjunction with > other "email message indicators"), and incorporate these variables into a > domain level filter (cPanel). cpanel? In such case you apparently should direct your questions to cpanel support (forum/list). > Mail is then bounced (by the filter) without > any warning in the bounced email itself, that it has been bounced because it > has been identified as spam. In fact, the bounced email will have a message > inserted to the effect that there is no such user/receipient. In this way, > if there is a sender who receives the bounced email, hopefully they take me > off their mailing list, instead of looking for a way to 'outsmart' the > SpamRules. bouncing sucks, bouncing spam is dangerous, since most of spam has false return address so you are bouncing to innocent third party (which may cause blogkinc your outgoing mail on blacklists). Reject unwanted the mail when it comes, don't bounce, especially when you are sure it's spam > Q:How can I list items/users on a "white list" or a "black list" without the > lists (and items) being the subject of further analysis by the SpamAssassin > Rules (and therefore obtaining the same score for each item on the relevant > list, irrespective of the operation of the SpamAssassin Rules, that is > -100=white list items & +100 = black list items)? I somehow do not understand this question. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. BSE = Mad Cow Desease ... BSA = Mad Software Producents Desease
Re: Spam Assassin White List
On Mon, 23 Mar 2009, dsh979 wrote: Q:How can I list items/users on a "white list" or a "black list" without the lists (and items) being the subject of further analysis by the SpamAssassin Rules That has to be done outside SA. Basically (modulo shortcuts, which you shouldn't be playing with) SA always checks all rules against every message it processes. If you want a hard whitelist or blacklist, then whatever is passing the messages from your MTA to SA for scoring (the glue layer) needs to implement that capability, and not give those messages to SA in the first place. As others have said, _do not_ bounce (i.e. accept and then later send a failure-to-deliver message to the sender) spams. It is a given that spam is sent with a forged From address. If you bounce spams in this way, you're simply attacking some innocent third party - and this may result in _your_ MTA getting blacklisted. If you want a hard blacklist, check the sender in your MTA _during SMTP_ and reject the message rather than accepting it. The typical way to do this is by configuring your MTA to check DNS blacklists, such as zen.spamhaus.org, and to add MTA rules rejecting specific senders or IP addresses. Also: your MTA should not be accepting messages for invalid addresses. Those need to be rejected during the SMTP phase. These particular questions are better directed at the cpanel list, as you're asking "how do I reject emails with specific from_address, to_address or sender_IP during SMTP time?" and "how do I skip SA for specific from_address, to_address, sender_IP?" Best of luck. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Liberals love sex ed because it teaches kids to be safe around their sex organs. Conservatives love gun education because it teaches kids to be safe around guns. However, both believe that the other's education goals lead to dangers too terrible to contemplate. --- 62 days since Obama's inauguration and still no unicorn!
Re: Spam Assassin White List
From: Matus UHLAR - fantomas Date: Tue, 24 Mar 2009 15:30:23 +0100 On 23.03.09 21:58, dsh979 wrote: > I did not realise that items listed on the white list or the black list > would still be subject to the operation/analysis of the SpamAssassin Rules. all rules are processed unless you play with ShortCircuit plugin. Beware of that: It may render the SA useless if you don't knwo what you are doing. > You have asked why I have set the required score the 100. Lengthy > explanation (sorry). I have done this to prevent SpamAssassin from > inserting SpamWarnings into the header/body of the relevant email. There's report_safe option to configure that. Also rewrite_header > Q:How can I list items/users on a "white list" or a "black list" without the > lists (and items) being the subject of further analysis by the SpamAssassin > Rules (and therefore obtaining the same score for each item on the relevant > list, irrespective of the operation of the SpamAssassin Rules, that is > -100=white list items & +100 = black list items)? I somehow do not understand this question. He wants the white/black lists to run first and then short circuit. So anybody in the whitelist gets a score of -100 and anybody in the blacklist gets a score of +100. This can probably be done with the ShortCircuit plugin and setting the priority of the rules so that they run first. Black lists aren't all that useful for stopping spam. The email addresses are forged in spam. -jeff
Re: Spam Assassin White List
On Tue, March 24, 2009 03:34, dsh979 wrote: > blacklist_from *...@blacklist1.com > blacklist_from *...@blacklist2.com > blacklist_from *...@blacklist3.com > required_score 100 > whitelist_from *...@whitelist1.com > whitelist_from *...@whitelist2.com > whitelist_from *...@whitelist3.com forged senders welcome :) hope *_from will be removed in next sa, its the badest check in current sa of all tests :/ change to whitelist_auth rules -- http://localhost/ 100% uptime and 100% mirrored :)
Re: Spam Assassin White List
On Wed, 25 Mar 2009 01:35:53 +0100 (CET) "Benny Pedersen" wrote: > > On Tue, March 24, 2009 03:34, dsh979 wrote: > > > whitelist_from *...@whitelist3.com > > forged senders welcome :) > > hope *_from will be removed in next sa, its the badest check in > current sa of all tests :/ > > change to whitelist_auth rules I think they all have their place. Clearly you'd want to use whitelist_auth for the likes of paypal, but whitelist_from is almost certainly a better choice when there is a low probability of forgery, e.g. one obscure company whitelisting another.
Re: Spam Assassin White List
On 24-Mar-2009, at 19:54, RW wrote: On Wed, 25 Mar 2009 01:35:53 +0100 (CET) "Benny Pedersen" wrote: On Tue, March 24, 2009 03:34, dsh979 wrote: whitelist_from *...@whitelist3.com forged senders welcome :) hope *_from will be removed in next sa, its the badest check in current sa of all tests :/ change to whitelist_auth rules I think they all have their place. Clearly you'd want to use whitelist_auth for the likes of paypal, but whitelist_from is almost certainly a better choice when there is a low probability of forgery, e.g. one obscure company whitelisting another. Is there a blacklist_noauth? Because it seems that would be far more useful for paypal. blacklist_auth *paypal* -- Say, give it up, give it up, television's taking its toll That's enough, that's enough, gimme the remote control I been nice, I been good, please don't do this to me Turn it off, turn it off, I don't want to have to see
Re: Spam Assassin White List
On Wed, 2009-03-25 at 08:19 -0600, LuKreme wrote: > On 24-Mar-2009, at 19:54, RW wrote: > > On Wed, 25 Mar 2009 01:35:53 +0100 (CET) > > "Benny Pedersen" wrote: > >> On Tue, March 24, 2009 03:34, dsh979 wrote: > Is there a blacklist_noauth? Because it seems that would be far more > useful for paypal. > > blacklist_auth *paypal* No, but I'd love one. Paypal, ebay, hotmail, yahoo, gmail. And, of course, your own domain! -- Daniel J McDonald, CCIE #2495, CISSP #78281, CNX Austin Energy http://www.austinenergy.com signature.asc Description: This is a digitally signed message part