Re: spamd vs. builtin SA (was Re: [Mimedefang] Patch: adding custom headers for SpamAssassin)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 23 Nov 2006, Jan-Pieter Cornet wrote: Do you use spamc to connect to spamd or have you re-implemented the protocol in Perl? Bye, - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iQEVAwUBRWatzOgJIbZtwg6XAQIX9Qf/TrsPlVHmK4M5zYQ4cNFTISZlHEfXCyO1 ZQ1xge7WsiK7lGvbof0F0DmWU7Oyn7UOD1Ltp45Jm/cvugjnqAcvnuwPqAagh2PX tA8ji4AG10mllnWIsjeMxEtx9XLUvaOTnYWZypXknKFtntguHUO1x7JXonNx2qFR csHaPILn5+WX/N2ZIrV8cuE93NPjoOxuB72XHUdUOxfggnl9jG5f+SLvctAO5rdt HHj7GfAp9gaGyLqhdYgbxXulxdhszZxmOSDeEVNNgruPR5ckMsbaIrFzh8M0k1kX rlp9EW/y1+n1hID52PpvLjA9utBC+2RMykg3ZHNx+RYv/Z3cwes3/g== =ks7q -END PGP SIGNATURE- ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: spamd vs. builtin SA (was Re: [Mimedefang] Patch: adding custom headers for SpamAssassin)
On Fri, Nov 24, 2006 at 09:31:05AM +0100, Steffen Kaiser wrote: Do you use spamc to connect to spamd or have you re-implemented the protocol in Perl? I use spamc. It's a pretty small binary. The spamc-spamd protocol isn't defined, where the spamc interface is properly defined, so I didn't want to mess with undocumented non-forward compatibility issues. If I ever suggested that it's better to re-implement spamc in perl (I seem to recall I did), then that was based on the assumption that the spamd protocol was documented and easy. -- Jan-Pieter Cornet [EMAIL PROTECTED] !! Disclamer: The addressee of this email is not the intended recipient. !! !! This is only a test of the echelon and data retention systems. Please !! !! archive this message indefinitely to allow verification of the logs. !! ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: spamd vs. builtin SA (was Re: [Mimedefang] Patch: adding custom headers for SpamAssassin)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 24 Nov 2006, Jan-Pieter Cornet wrote: I was just wondering, because you shell-out one program per scan now. I was hoping that there is another way to connect to spamd. Bye, - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iQEVAwUBRWbe+ugJIbZtwg6XAQIdawf+MWCPNW0wnIQHZFebtsf7f0rJ6AO6HZjX qctrGOVGVWdaVN4C9pSSkSAXFTpiinHmPismJiFQ42CWOv1TtblZoyj7UDFf4E0y neKYdnBqBX8aJlEknpM7t1NtlsNFVEIYss7LlFJwTgRWO7Swfq9N6soW2PLirQ1e qQXULukr6LPcL999h9DQF+MvFuzy/mGcXtD6+lhfy2os6iXAFTmHUL/4+DqIfQtK XVE79a8NwdfCSiq606N1rYVKzG737RgM25EHMmz0fybuEGuo4zU578DBVl9YTS0M abigcUxrUiQThIW+q3q5/jFabmM2p12NfrRIhOLqgDRPn3NOyfD9OQ== =jwVl -END PGP SIGNATURE- ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: spamd vs. builtin SA (was Re: [Mimedefang] Patch: adding custom headers for SpamAssassin)
On Fri, Nov 24, 2006 at 01:00:54PM +0100, Steffen Kaiser wrote: I was just wondering, because you shell-out one program per scan now. I was hoping that there is another way to connect to spamd. Well, it's only a small C program, and it's likely in cache, not executed via the shell but directly, so the overhead is pretty minimal compared to the actual Mail::SpamAssassin scanning itself. That said, after a little looking around, there's both a libspamc.so available (without documentation?), and a protocol description in the SA distribution in spamd/PROTOCOL. So it should be possible to create a standalone perl version of spamc. However, I probably won't bother... unless we can show that the overhead is significant. Oh, I'm using IPC::Open2 to write to and read from spamc, and prevent tempfiles and executions via the shell. If you're interested, let me know. -- Jan-Pieter Cornet [EMAIL PROTECTED] !! Disclamer: The addressee of this email is not the intended recipient. !! !! This is only a test of the echelon and data retention systems. Please !! !! archive this message indefinitely to allow verification of the logs. !! ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: spamd vs. builtin SA (was Re: [Mimedefang] Patch: adding custom headers for SpamAssassin)
On Fri, Nov 24, 2006 at 05:27:08PM +0100, Jan-Pieter Cornet wrote: That said, after a little looking around, there's both a libspamc.so available (without documentation?), and a protocol description in the SA distribution in spamd/PROTOCOL. So it should be possible to create a standalone perl version of spamc. However, I probably won't bother... unless we can show that the overhead is significant. There is a Mail::SpamAssassin::Client module that ships with SA, with such confidence-building things in its documentation as: NOTE: This interface is alpha at best, and almost guaranteed to change If anyone's inclined to speak to spamd directly from within their filter, it might be a good starting point, but be prepared to fix your code in future SpamAssassin releases. Cheers, Dave -- Dave O'Neill [EMAIL PROTECTED]Roaring Penguin Software Inc. +1 (613) 231-6599http://www.roaringpenguin.com/ For CanIt technical support, please mail: [EMAIL PROTECTED] ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: spamd vs. builtin SA (was Re: [Mimedefang] Patch: adding custom headers for SpamAssassin)
On Thu, Nov 23, 2006 at 10:00:01AM -0500, David F. Skoll wrote: But, given the fact that we just switched to using spamd instead of the builtin Mail::SpamAssassin modules, we don't really need support builtin to mimedefang at the moment either... Just curious: Why the switch? Do you see better performance? Yes, but mainly due to better memory utilisation, and because of our specific setup. We don't scan everything with SpamAssassin, about 25% of the mails are scanned with SA, the rest are only virus-checked, or checked against the remote SMTP server. Loading SpamAssassin in memory requires about 25MByte, and with 50 mimedefang slaves, that adds up. Some of our servers were swapping occasionally, especially when some large map-building processes were being run, eating another 500MB of RAM. We are now using about 5 busy spamd slaves, with very occasional peaks to 15. Of course, if every email is run through spamassassin, there probably isn't a lot to gain. -- Jan-Pieter Cornet [EMAIL PROTECTED] !! Disclamer: The addressee of this email is not the intended recipient. !! !! This is only a test of the echelon and data retention systems. Please !! !! archive this message indefinitely to allow verification of the logs. !! ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: spamd vs. builtin SA (was Re: [Mimedefang] Patch: adding custom headers for SpamAssassin)
We made the same change long ago - we normally have over 100 mimedefang threads running, but average 15 spamd. This is due to so much spam being stopped by DCC, no user checks, black lists etc, all fired off from mimedefang. I had many people on this list tell me I was dead wrong, that there was no benefit to using spamd...but we sure see one. - Original Message - From: Jan-Pieter Cornet [EMAIL PROTECTED] To: mimedefang@lists.roaringpenguin.com Sent: Thursday, November 23, 2006 11:12 AM Subject: Re: spamd vs. builtin SA (was Re: [Mimedefang] Patch: adding custom headers for SpamAssassin) On Thu, Nov 23, 2006 at 10:00:01AM -0500, David F. Skoll wrote: But, given the fact that we just switched to using spamd instead of the builtin Mail::SpamAssassin modules, we don't really need support builtin to mimedefang at the moment either... Just curious: Why the switch? Do you see better performance? Yes, but mainly due to better memory utilisation, and because of our specific setup. We don't scan everything with SpamAssassin, about 25% of the mails are scanned with SA, the rest are only virus-checked, or checked against the remote SMTP server. Loading SpamAssassin in memory requires about 25MByte, and with 50 mimedefang slaves, that adds up. Some of our servers were swapping occasionally, especially when some large map-building processes were being run, eating another 500MB of RAM. We are now using about 5 busy spamd slaves, with very occasional peaks to 15. Of course, if every email is run through spamassassin, there probably isn't a lot to gain. -- Jan-Pieter Cornet [EMAIL PROTECTED] !! Disclamer: The addressee of this email is not the intended recipient. !! !! This is only a test of the echelon and data retention systems. Please !! !! archive this message indefinitely to allow verification of the logs. !! ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
