Re: multiple instances, simplification
Gary Smith wrote: I'm not worried about it exploding as we don't allow user_prefs. The machines are processed via relays. I believe the bayes_sql_override_username will solve the last piece of the puzzle. I think I will test this out this weekend on the laptop, then our test environment. 'Exploding' as in 'sucking up far more disk than expected causing I/O saturation and grotty spam detection rates' (because by default, each calling user as provided by spamc will get its own Bayes records in SQL). In switching from an environment with a sitewide BDB file-based Bayes to sitewide SQL I ended up learning about that one the hard way. :/ There's no direct relation between user_prefs and a sitewide Bayes DB. -kgd
RE: multiple instances, simplification
> If you're just trying to keep your Bayes table from exploding due to > multiple users, use the bayes_sql_override_username option. I'm not worried about it exploding as we don't allow user_prefs. The machines are processed via relays. I believe the bayes_sql_override_username will solve the last piece of the puzzle. I think I will test this out this weekend on the laptop, then our test environment. Thanks for all of the information. Gary Smith
Re: multiple instances, simplification
Gary Smith wrote: Bayes is the only real concern here, as I know I can run multiple copies (and had forgot that I could run a single copy with user_prefs). So I think this will work either way. I just needed to put a little thought into it and bounce off of people who might have already done something like this. If you're just trying to keep your Bayes table from exploding due to multiple users, use the bayes_sql_override_username option. -kgd
RE: multiple instances, simplification
> Why don't you just run 3 instances of spamd, each listening on different > ports/sockets and each with their own configuration: > > spamd --siteconfigpath=/etc/spam1 --socketpath=/tmp/spam1.sock --port=783 > spamd --siteconfigpath=/etc/spam2 --socketpath=/tmp/spam2.sock --port=784 > spamd --siteconfigpath=/etc/spam3 --socketpath=/tmp/spam3.sock --port=785 > > This way you can enable/disable different plugins for each config as > well as having totally different configurations in each instance. > Afterwards it's just a matter of calling the right instance from your > MDA by choosing the proper socket or tcp-port. > > Since you use MySql for Bayes, you can configure each instance with the > same configuracion so that they all access the same database. And > because its just for testing, don't forget to add "--min-children=1 > --max-children=1" so that each instance only runs one scanner instance, > thus conserving RAM. Jorge, This is all just a thought, based upon me try to create a development environment on a laptop, which spawned off possible configuration changes to a production environment. We currently have 6+ server running these. 3 sets of load balanced SA servers. These servers are roughly 70% idle most of the time. Running them with user preferences, instead of different instances, would allow us to remove 50% of the hardware. Running them as multiple instances on the same box, means we will still need to balance across the same number of servers. I think the virtual user angle "might" work, I just was thinking of a way to use a single consolidated mysql instance, where it doesn't care about user_name. If I can't elegantly resolve this, I could always just patch the source to use a hard coded user name in the sql statement to ensure that bayes stays consistent. Bayes is the only real concern here, as I know I can run multiple copies (and had forgot that I could run a single copy with user_prefs). So I think this will work either way. I just needed to put a little thought into it and bounce off of people who might have already done something like this. Gary Smith
Re: multiple instances, simplification
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Deugau wrote: > Gary Smith wrote: >> Instead of running multiple SA servers, it is possible to run a single consolidated SA server where only the userpref's are different for each spamc caller (given that the local config will override the global config) AND still use a single bayes DB? We use a clustered MySql instance for bayes, and I don't want to have to worry about a bayes DB per user. >> >> This big difference between the instances are mostly the required_score threshold, few score overrides and a few custom rules. >> >> Any recommendations on how to handle this? I would be really nice to use a single config for all SA instances, whereas the only difference being the user config. > > If all of the differences are in required_score, custom scores on a few rules, a few fairly trivial rules, etc, then yes, you should be able to do this. > > Either create real system users filter1, filter2, etc or read up on spamd's virtual user support. A quick read of spamd's man page shows a little clearer and more coherent set of options than I recall from ~2.x. > > -x and --virtual-config-dir are probably good places to start. > > -kgd > Why don't you just run 3 instances of spamd, each listening on different ports/sockets and each with their own configuration: spamd --siteconfigpath=/etc/spam1 --socketpath=/tmp/spam1.sock --port=783 spamd --siteconfigpath=/etc/spam2 --socketpath=/tmp/spam2.sock --port=784 spamd --siteconfigpath=/etc/spam3 --socketpath=/tmp/spam3.sock --port=785 This way you can enable/disable different plugins for each config as well as having totally different configurations in each instance. Afterwards it's just a matter of calling the right instance from your MDA by choosing the proper socket or tcp-port. Since you use MySql for Bayes, you can configure each instance with the same configuracion so that they all access the same database. And because its just for testing, don't forget to add "--min-children=1 --max-children=1" so that each instance only runs one scanner instance, thus conserving RAM. - -- Jorge Valdes jval...@intercom.com.sv -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvIwoYACgkQkGBK/EMo0qJUmQCfUNkK/hIY+Dps+bALWHzp0v8f TnAAniE39uyZUCypqlrgLoJJa7SBR0ZT =0eCa -END PGP SIGNATURE-
Re: multiple instances, simplification
Gary Smith wrote: Instead of running multiple SA servers, it is possible to run a single consolidated SA server where only the userpref's are different for each spamc caller (given that the local config will override the global config) AND still use a single bayes DB? We use a clustered MySql instance for bayes, and I don't want to have to worry about a bayes DB per user. This big difference between the instances are mostly the required_score threshold, few score overrides and a few custom rules. Any recommendations on how to handle this? I would be really nice to use a single config for all SA instances, whereas the only difference being the user config. If all of the differences are in required_score, custom scores on a few rules, a few fairly trivial rules, etc, then yes, you should be able to do this. Either create real system users filter1, filter2, etc or read up on spamd's virtual user support. A quick read of spamd's man page shows a little clearer and more coherent set of options than I recall from ~2.x. -x and --virtual-config-dir are probably good places to start. -kgd
multiple instances, simplification
Background: I've been using SA for a long time, and for a verity of reasons, we run different servers to support some minor changes in different rules. While trying to setup a multi instance version on my laptop, I copied these rules over into different directories, setup the startup/shutdown script and ran my tests and everything worked fine until I found that I didn't create the user "filter" that I run everything as (for SA). So, I created filter1, filter2, etc., for each instance that I want to run. I noticed that the log still complained that filter didn't exist. Looking into it, it appears that filter is the value being passed in via the spamc call. Now, because SA always works, I generally don't touch some of these little things, so I tend to forget things like that calling user spamc must exist on the remote spamd server, etc, as I never really need to change anything. Question: Instead of running multiple SA servers, it is possible to run a single consolidated SA server where only the userpref's are different for each spamc caller (given that the local config will override the global config) AND still use a single bayes DB? We use a clustered MySql instance for bayes, and I don't want to have to worry about a bayes DB per user. This big difference between the instances are mostly the required_score threshold, few score overrides and a few custom rules. Any recommendations on how to handle this? I would be really nice to use a single config for all SA instances, whereas the only difference being the user config. Gary
RE: multiple instances
> > I'm sure it's possible, but rather than going through all the work of > trying to script and setup chroot environments, why not use VMs? You > can then quite literally match the production setup. > > Since you are not worried about performance or memory you could give > each VM 128 MB of RAM and only be using 1 GB or so total... > Dennis, I had though about that, but the target is a mobile laptop. Our in house development we do use VM's for almost everything just for this purpose. Looking into spamd, I think I will just copy the config folder for each instance type and then run the daemon via a bash script to create it against 9 local IP's. I know for things like MySql some people already have some multi-instance scripts laying around. Anyway, I think this will suffice for now. Gary
Re: multiple instances
On Fri, 2010-04-16 at 10:08 -0700, Gary Smith wrote: > I have a need to run several different instances of SA on a single box (in > development). In production, we have 3 different SA environments (with 2+ > servers each) that have different rule sets and specific routing rules > determine which instance it gets sent to. We need to mimic this in > development. > > Ideally I would like to create all 3 instances (*2 mimicing load balancing) > on a single development box. We're not worried about the performance or > memory aspect. > > Is this possible, and if so, is there an easy way to do this. I was > thinking that I could create separate chroot environments for each one if > necessary and either bind each instance to an IP (which I'm not sure if > that's possible) or at least a different port. > > Any advice (or some sample scripts on doing this) would be greatly > appreciated. > I'm sure it's possible, but rather than going through all the work of trying to script and setup chroot environments, why not use VMs? You can then quite literally match the production setup. Since you are not worried about performance or memory you could give each VM 128 MB of RAM and only be using 1 GB or so total... --Dennis
multiple instances
I have a need to run several different instances of SA on a single box (in development). In production, we have 3 different SA environments (with 2+ servers each) that have different rule sets and specific routing rules determine which instance it gets sent to. We need to mimic this in development. Ideally I would like to create all 3 instances (*2 mimicing load balancing) on a single development box. We're not worried about the performance or memory aspect. Is this possible, and if so, is there an easy way to do this. I was thinking that I could create separate chroot environments for each one if necessary and either bind each instance to an IP (which I'm not sure if that's possible) or at least a different port. Any advice (or some sample scripts on doing this) would be greatly appreciated. Gary Smith
Re: multiple instances of SpamAssassin
> jdow writes: > Besides, "can't be done" from within SpamAssassin. If you get REALLY > imaginative you might be able to do it with procmail. But that falls > back to the "why bother?" If the employer runs an older version or an > ineffective version install a copy in a personal set of directories. I have no evidence that the central version is either old or ineffective. But I also want to be able to add spam/ham messages of my own, which can only be done on the local system. At the same time, I am not eager to give up the advantage gained due to the large number of messages processed centrally. This is a problem regardless of what version is running locally -- unless there is a way to retain the original headers. I note in passing that my original submission to this list arrived back with both an X-Spam-Status and an X-Old-Spam-Status header. No idea how or why, although my reading is that this was done at apache.org. This is exactly the behavior I'm looking for. > cilantro05 writes: > > My (large) employer runs SpamAssassin on all incoming mail. My local UNIX > > account, managed by a separate technical support group, is on a machine > > which also has spamassassin available. Running the local version removes > > the headers added by the central version. While I can see good reasons > > for this, I would really like to be able to filter on both sets of > > headers. Is there a way to tell SpamAssassin to either leave the old > > headers intact, or (ideally) to rename them, rather than deleting them? > > No version info available for the central instance. Local version is 2.55 > > -- sorry, I have no control over this. I normally run spamc from my > > .forward file, but am willing to consider other options, possibly > > including my own co py of spamassassin, although I do not have superuser > > priviliges on this machine . Thanks.
Re: multiple instances of SpamAssassin
Do not even bother with your 2.55 local installation. It's a waste of
effort and time to try to make it work.
Besides, "can't be done" from within SpamAssassin. If you get REALLY
imaginative you might be able to do it with procmail. But that falls
back to the "why bother?" If the employer runs an older version or an
ineffective version install a copy in a personal set of directories.
With some imagination I bet it can be done. You'll have to run spamd
on a different TCP address or else run straight spamassassin, though.
{^_^}
- Original Message -
From: <[EMAIL PROTECTED]>
My (large) employer runs SpamAssassin on all incoming mail. My local UNIX
account, managed by a separate technical support group, is on a machine which
also has spamassassin available. Running the local version removes the
headers added by the central version. While I can see good reasons for this,
I would really like to be able to filter on both sets of headers.
Is there a way to tell SpamAssassin to either leave the old headers intact, or
(ideally) to rename them, rather than deleting them?
No version info available for the central instance. Local version is 2.55 --
sorry, I have no control over this. I normally run spamc from my .forward
file, but am willing to consider other options, possibly including my own copy
of spamassassin, although I do not have superuser priviliges on this machine.
Thanks.
multiple instances of SpamAssassin
My (large) employer runs SpamAssassin on all incoming mail. My local UNIX account, managed by a separate technical support group, is on a machine which also has spamassassin available. Running the local version removes the headers added by the central version. While I can see good reasons for this, I would really like to be able to filter on both sets of headers. Is there a way to tell SpamAssassin to either leave the old headers intact, or (ideally) to rename them, rather than deleting them? No version info available for the central instance. Local version is 2.55 -- sorry, I have no control over this. I normally run spamc from my .forward file, but am willing to consider other options, possibly including my own copy of spamassassin, although I do not have superuser priviliges on this machine. Thanks.
