Re: [Mimedefang] 40K+ emails a day and choking
David, David F. Skoll wrote: Stephen Ford wrote: I'm running Solaris 9 on a dual processor 220R with 2 gigs of ram and the box is having trouble keeping up with spam!?!? I hate to say this, but switch from SPARC to a commodity Intel box. Intel and AMD chips far outperform SPARC for the kind of processing MIMEDefang/SpamAssassin do. Even a mid-range dual Xeon at 2.4GHz with a couple of gigs of RAM can handle 40K emails/day with ease. Are you aware of any relevant benchmarks in this area? N ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking
On Tue, 2006-01-17 at 14:48 +, Nik Clayton wrote: David, David F. Skoll wrote: Stephen Ford wrote: I'm running Solaris 9 on a dual processor 220R with 2 gigs of ram and the box is having trouble keeping up with spam!?!? I hate to say this, but switch from SPARC to a commodity Intel box. Intel and AMD chips far outperform SPARC for the kind of processing MIMEDefang/SpamAssassin do. Even a mid-range dual Xeon at 2.4GHz with a couple of gigs of RAM can handle 40K emails/day with ease. at my last company we did with 4 machines, 3Mil/Day Messages without any problem. The machines where HP DL360 2G Ram 1 CPU ;) Are you aware of any relevant benchmarks in this area? N ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang -- Michael Lang [EMAIL PROTECTED] pgpSyljeNxerz.pgp Description: PGP signature ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking
Nik Clayton wrote: I hate to say this, but switch from SPARC to a commodity Intel box. Intel and AMD chips far outperform SPARC for the kind of processing MIMEDefang/SpamAssassin do. Even a mid-range dual Xeon at 2.4GHz with a couple of gigs of RAM can handle 40K emails/day with ease. Are you aware of any relevant benchmarks in this area? I don't have actual benchmarks, but we've had a couple of CanIt customers who tried both and switched to Intel/AMD boxes. They cited better performance/price. Regards, David. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] 40K+ emails a day and choking
From: Michael Lang Sent: Tuesday, January 17, 2006 7:50 AM [...] at my last company we did with 4 machines, 3Mil/Day Messages without any problem. The machines where HP DL360 2G Ram 1 CPU ;) With Mimedefang and SA? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] 40K+ emails a day and choking
On Tue, 2006-01-17 at 08:20 -0800, Gary Funck wrote: From: Michael Lang Sent: Tuesday, January 17, 2006 7:50 AM [...] at my last company we did with 4 machines, 3Mil/Day Messages without any problem. The machines where HP DL360 2G Ram 1 CPU ;) With Mimedefang and SA? yes, complex filter (ldap lookups ...) and clamav and kaspersky ... ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang -- Michael Lang [EMAIL PROTECTED] pgpuoLfa9DNRl.pgp Description: PGP signature ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] 40K+ emails a day and choking
At work here at 180 I am handling 200K/day easily with a Dell 2650 w/ twin 2.8 Xeon, 4Gig Ram NFS mounted maildirs running SM, MD, SA Clamd. The only bottleneck is a 1999 vintage NetApp which is about to be replaced. LA ~1 most of the time except when the NetApp gets clogged G At 09:20 AM 1/17/2006, you wrote: From: Michael Lang Sent: Tuesday, January 17, 2006 7:50 AM [...] at my last company we did with 4 machines, 3Mil/Day Messages without any problem. The machines where HP DL360 2G Ram 1 CPU ;) With Mimedefang and SA? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang John Jaeger System Administrator OneEighty Communications (888) 342-5987 (406) 294-4034 Email: [EMAIL PROTECTED] Sendmail administration is not black magic. There are legitimate technical reasons why it requires the sacrificing of a live chicken. - Unknown ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking update
-ray wrote: What if your DNS servers are in the same rack, on the same switch as mail servers. Network latency is 200 usecs. In that case is there much advantage to a caching server on the same box as mail? Probably not. In that case, a caching server on the mail box would probably hurt by using up memory for no good reason. -- David. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking
--On Tuesday, January 10, 2006 13:26 -0800 Stephen Ford [EMAIL PROTECTED] wrote: I'm running Solaris 9 on a dual processor 220R with 2 gigs of ram and the box is having trouble keeping up with spam!?!? I am using all the bells and whistles spamassassin and mimedefang can have, minus the virus protection (another box handles that). I have a min/max of 20/90 for mimedefang and I'm no longer hitting a max number of files open ceiling.. I'd say 90% of our email is spam (we are a small college with ~3000 email accounts) with scores averaging 20 and above :-/ I'm going to add a local dns (even though the dns doesn't appear to be the bottle neck) and perhaps host a dcc server. I can also setup another box and round robin them but I shouldn't *have to* should I? Oh, and yes /var/spool/MIMEDefang is swap. Our incoming pool includes a v210 with 2 gigs ram, and it does 150,000 messages a day without problems-- as comparison. MX_MINIMUM=5, MX_MAXIMUM=50. It doesn't hit 50 unless something is seriously wrong with our mail system, and in that case a maximum is a good thing. Definitely run named and nscd on mail servers. Sendmail does a lot of dns lookup and it makes a big difference. /tmp and /var/spool/MIMEDefang should be swap (tmpfs), right. That's almost twice the per cent of junk we get, and I'm counting 8 and up. 90% scores 20 and up in Spamassassin? That's incredible. I thought we had it bad, and now you've cheered me up :-) ...sorry! Joseph Brennan Columbia University Information Technology ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking
Stephen Ford wrote: I'm running Solaris 9 on a dual processor 220R with 2 gigs of ram and the box is having trouble keeping up with spam!?!? I hate to say this, but switch from SPARC to a commodity Intel box. Intel and AMD chips far outperform SPARC for the kind of processing MIMEDefang/SpamAssassin do. Even a mid-range dual Xeon at 2.4GHz with a couple of gigs of RAM can handle 40K emails/day with ease. Before you throw out your SPARC box, though, try reducing your max from 90 down to 50. If the MIMEDefang scanners start swapping, you're toast, and my new rule of thumb is around 40MB/scanner. Regards, David. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] 40K+ emails a day and choking
Stephen Ford wrote: I'm running Solaris 9 on a dual processor 220R with 2 gigs of ram and the box is having trouble keeping up with spam!?!? ... I'm going to add a local dns That's a necessity - a caching-only server is fine Oh, and yes /var/spool/MIMEDefang is swap. Good Any sug Yes * use spamd to save memory http://www.mimedefang.org/kwiki/index.cgi?SpamassassinSpamcSpamd * AV first You mentioned you scan for viruses - do you do that before or after scanning for spam? If you scan after, move it to before. Use clamav-milter on the same box and save an expensive MIMEDefang thread. (Make sure the milter order puts clamav-milter first.) * /tmp swap too SpamAssassin uses /tmp as a helper directory for all of its plugins. Make sure that's also a RAM disk (I believe that's the default for Solaris anyway?) -- Matthew.van.Eerde (at) hbinc.com 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking
If Oh, and yes /var/spool/MIMEDefang is swap. does not mean that /var/spool/MIMEDefang is on a tmpfs, then put it on one. If it is on disk you have a huge bottleneck. - Original Message - From: Stephen Ford [EMAIL PROTECTED] To: mimedefang@lists.roaringpenguin.com Sent: Tuesday, January 10, 2006 4:26 PM Subject: [Mimedefang] 40K+ emails a day and choking I'm running Solaris 9 on a dual processor 220R with 2 gigs of ram and the box is having trouble keeping up with spam!?!? I am using all the bells and whistles spamassassin and mimedefang can have, minus the virus protection (another box handles that). I have a min/max of 20/90 for mimedefang and I'm no longer hitting a max number of files open ceiling.. I'd say 90% of our email is spam (we are a small college with ~3000 email accounts) with scores averaging 20 and above :-/ I'm going to add a local dns (even though the dns doesn't appear to be the bottle neck) and perhaps host a dcc server. I can also setup another box and round robin them but I shouldn't *have to* should I? Oh, and yes /var/spool/MIMEDefang is swap. Any sug __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] 40K+ emails a day and choking
John Scully wrote: If Oh, and yes /var/spool/MIMEDefang is swap. does not mean that /var/spool/MIMEDefang is on a tmpfs, then put it on one. If it is on disk you have a huge bottleneck. In Solaris-ese, swap = tmpfs. http://sunsolve.sun.com/searchproxy/document.do?assetkey=1-30-3403-1 -- Matthew.van.Eerde (at) hbinc.com 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking
On Tue, Jan 10, 2006 at 01:26:20PM -0800, Stephen Ford wrote: hitting a max number of files open ceiling.. I'd say 90% of our email is spam (we are a small college with ~3000 email accounts) with scores averaging 20 and above :-/ I'm going to add a local dns (even though the dns doesn't appear to be the bottle neck) and perhaps host a dcc server. I can also setup another box and round robin them but I shouldn't *have to* should I? That'd help. One thing that greatly help us, which isnt mimedefang/spamassassin involved so much is, using pre-greet and RBL checks at the MTA level. That way, you stop a lot of the mail before it even hits mimedefang/spaassassin. Just a thought :) -- Champ Clark III| Vistech Communications,Inc. |850-942-0388 x 101 http://www.vistech.net GPG Key ID: 58A2A58F Key fingerprint = 7734 2A1C 007D 581E BDF7 6AD5 0F1F 655F 58A2 A58F If it wasn't for C, we'd be using BASI, PASAL and OBOL. pgp6MlHWOue1m.pgp Description: PGP signature ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking
What else is running on the box? Are you running a POP server or the UW IMAP server by any chance? If so, your performance problems could be due to excessive checking for new mail by poorly configured and/or designed email clients (eg. Outlook Express). We've been experiencing high load factors on a similar machine, and for the longest time thought it was due to MD and SA, but realized it was POP and IMAP server activity when we happened to notice that the load factor was down on weekends and at night despite the fact that there was just as much incoming mail at those times. You won't see this using top or prstat, BTW, but you should see it by analyzing the POP/IMAP server logs. - rick On Tue, 10 Jan 2006, Stephen Ford wrote: I'm running Solaris 9 on a dual processor 220R with 2 gigs of ram and the box is having trouble keeping up with spam!?!? I am using all the bells and whistles spamassassin and mimedefang can have, minus the virus protection (another box handles that). I have a min/max of 20/90 for mimedefang and I'm no longer hitting a max number of files open ceiling.. I'd say 90% of our email is spam (we are a small college with ~3000 email accounts) with scores averaging 20 and above :-/ I'm going to add a local dns (even though the dns doesn't appear to be the bottle neck) and perhaps host a dcc server. I can also setup another box and round robin them but I shouldn't *have to* should I? Oh, and yes /var/spool/MIMEDefang is swap. Any sug __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking update
Stephen Ford wrote: Ok, this is odd. At 7:30PM all of a sudden, the server started purring along. Ah. That screams network problems. DNS latencies can kill you, especially if you're using SURBL lookups inside SpamAssassin. High DNS latency causes slave processes to build up. Regards, David. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] 40K+ emails a day and choking update
From: David F. Skoll Sent: Tuesday, January 10, 2006 5:35 PM Ah. That screams network problems. DNS latencies can kill you, especially if you're using SURBL lookups inside SpamAssassin. High DNS latency causes slave processes to build up. A caching DNS server running on the same box can help? - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking update
Gary Funck wrote: A caching DNS server running on the same box can help? Maybe. A caching server still has to do the initial lookups, and if the cache miss rate is high enough, you'll still have problems. But in general, a caching server is a good idea. Regards, David. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] 40K+ emails a day and choking update
On Tue, 10 Jan 2006, David F. Skoll wrote: A caching DNS server running on the same box can help? Maybe. A caching server still has to do the initial lookups, and if the cache miss rate is high enough, you'll still have problems. But in general, a caching server is a good idea. What if your DNS servers are in the same rack, on the same switch as mail servers. Network latency is 200 usecs. In that case is there much advantage to a caching server on the same box as mail? ray -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Ray DeJean http://www.r-a-y.org Systems EngineerSoutheastern Louisiana University IBM Certified Specialist AIX Administration, AIX Support =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
