Re: Using Pzyor with high volume
On May 1, 2008, at 10:02 PM, Michael Hutchinson wrote: Anyway, just thought you ought to know about the high volume thing. You might get your end running sweet and fast, but it may cause rejected lookups when you're scanning mail. I'm pretty much putting Pyzor on the back burner for now. Even with the ReadyExec method, I don't want to call an exec over NFS constantly... it's expensive on a large scale. I could do something like create a memory disk and exec out of that, but it's just to much cobbing up. I really hoped that something could be memory resident, ie: just loaded at start time, then just work. Both dcc and razor2 both seem to be doing a good job now. -- Robert Blayzor, BOFH INOC, LLC [EMAIL PROTECTED] http://www.inoc.net/~rblayzor/ Mac OS X. Because making Unix user-friendly is easier than debugging Windows.
RE: Using Pzyor with high volume
> In regards to Pyzor. I'm wondering if anyone out there is using this > at any large scale. Unlike the razor-agent which appears to be a Perl > module that gets loaded at startup, I'm concerned about SA having to > exec the python interpreter and having that setup/teardown time for > each and every message. > > Adding salt to the wound, our SA servers run on diskless servers; so > having it have to run over NFS makes for a double whammy. > > Is there a better way to implement Pyzor or is it not even worth the > trouble? Hi Robert, It would appear you've had some success with Pyzor and ReadyExec. We use Pyzor over here at Manux, but do not use ReadyExec (we're not running out of cpu or ram any time soon..). One thing I have noticed, however, is related to high volume. Sometimes (and for several minutes at a time) we will not be able to reach the Pyzor server - it rejects our connections. This is most definitely because we're doing too many lookups. Beware of this. You probably will not see anything about it until you notice it's gone from the headers of mail that it should or has hit in the past. The fix for us would be to run our own Pyzor server, and I'll be working on that over the next while - hopefully after a discussion with the author. Anyway, just thought you ought to know about the high volume thing. You might get your end running sweet and fast, but it may cause rejected lookups when you're scanning mail. Cheers, Mike
RE: Using Pzyor with high volume
The only need Pyzor has for user accounts is to read the server list. If you set the server list to someplace all accounts can access, then it should work for you. Me, I run mine system-wide and don't use user-specific accounts. Everyone uses the same Bayes, AWL, and settings as user "spamc". - Jason -Original Message- From: Matus UHLAR - fantomas [mailto:[EMAIL PROTECTED] Sent: Thursday, May 01, 2008 12:31 PM To: users@spamassassin.apache.org Subject: Re: Using Pzyor with high volume On 30.04.08 16:39, Jason J. Ellingson wrote: > readyexec is *USER* specific. You *MUST* launch the readyexecd.py > "server" part as the SAME user as the user of readyexec "client". > > My spamd service is run as a user "spamc", while I was adding the > service as "root". > > Now, that I have the service running as "spamc", we have success! and I was just thinking about running this server-wide. My spamd runs as root so it can change uid to any user, but this probably won't help the thing... I would like to have the possibility of reading different user's pyzor config, but currently maybe server-wide config would be enough (at least for checking status of a message) -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. (R)etry, (A)bort, (C)ancer
Re: Using Pzyor with high volume
On 30.04.08 16:39, Jason J. Ellingson wrote: > readyexec is *USER* specific. You *MUST* launch the readyexecd.py > "server" part as the SAME user as the user of readyexec "client". > > My spamd service is run as a user "spamc", while I was adding the > service as "root". > > Now, that I have the service running as "spamc", we have success! and I was just thinking about running this server-wide. My spamd runs as root so it can change uid to any user, but this probably won't help the thing... I would like to have the possibility of reading different user's pyzor config, but currently maybe server-wide config would be enough (at least for checking status of a message) -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. (R)etry, (A)bort, (C)ancer
RE: Using Pzyor with high volume
If I try as "root" (but the readyexec service was launched by "spamc"), then I get: [24384] dbg: pyzor: network tests on, attempting Pyzor [24384] dbg: pyzor: pyzor is available: /usr/local/bin/readyexec [24384] dbg: pyzor: opening pipe: /usr/local/bin/readyexec /tmp/pyzor check < /tmp/.spamassassin24384QlgyvMtmp [24384] dbg: pyzor: [24389] finished: exit=0x0100 [24384] dbg: pyzor: got response: downloading servers from http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x [24384] dbg: pyzor: failure to parse response "downloading servers from http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x"; If I try as "spamc", then I get: [24756] dbg: pyzor: network tests on, attempting Pyzor [24756] dbg: pyzor: pyzor is available: /usr/local/bin/readyexec [24756] dbg: pyzor: opening pipe: /usr/local/bin/readyexec /tmp/pyzor check < /tmp/.spamassassin24756qBws8Rtmp [24756] dbg: pyzor: got response: 82.94.255.100:24441 (200, 'OK') 5 0 [24756] dbg: pyzor: listed: COUNT=5/5 WHITELIST=0 Keep in mind, I'm running on Linux (Debian Sarge - Untangle) and am using Python 2.3... So there may be some differences there. - Jason -Original Message- From: neil [mailto:[EMAIL PROTECTED] Sent: Thursday, May 01, 2008 11:00 AM To: Jason J. Ellingson Cc: users@spamassassin.apache.org Subject: Re: Using Pzyor with high volume Hi; >Since you are running the "server" portion as "nobody"... Then the spamd >user account needs to stay as "nobody". We run SA as a system filter, so no calls to multiple different users. root 20203 4.8 5.4 61144 56424 ?? S 4:11PM 2:41.23 spamd child (perl5.8.8) nobody21508 4.1 5.6 62920 58388 ?? S 4:13PM 2:27.77 spamd child (perl5.8.8) If I run it as root, it stops the error, but then it stop hitting things. Though with -D it does show that it is talking to the pyzor server via the pipe. (Note: I deleted the 66.250.40.33 from the pyzor servers list, but it is still showing up, even after a spamd restart.) # spamassassin -D pyzor < /tmp/spam.txt [42041] dbg: pyzor: network tests on, attempting Pyzor [42041] dbg: pyzor: pyzor is available: /usr/local/bin/readyexec [42041] dbg: pyzor: opening pipe: /usr/local/bin/readyexec /tmp/pyzor1 check < /tmp/.spamassassin420412jN0S2tmp [42041] dbg: pyzor: [42060] finished: exit=0x0100 [42041] dbg: pyzor: got response: 82.94.255.100:24441 (200, 'OK') 0 0\n66.250.40.33:24441 TimeoutError: [42041] dbg: pyzor: failure to parse response "66.250.40.33:24441 TimeoutError: " I'm off for a few days, so I'll have a play with it when I get back. I think, as you pointed out, it could be to do with permissions. Cheers for that. Rgds n Jason J. Ellingson wrote: > Since you are running the "server" portion as "nobody"... Then the spamd > user account needs to stay as "nobody". If you make a spamc call with a > "user" identified, then spamd will switch to that user account before > processing the message (this is for user specific needs like custom user > rules, BAYES, AWL, etc). Since the accounts now don't match, it'll give > you that error. > > - Jason > > -Original Message- > From: neil [mailto:[EMAIL PROTECTED] > Sent: Thursday, May 01, 2008 9:31 AM > To: Jason J. Ellingson > Cc: Ben Poliakoff; Robert Blayzor; users@spamassassin.apache.org > Subject: Re: Using Pzyor with high volume > > Hi; > I have it running and it works, load is down and free mem up, but not > certain if that is just because I've restarted spamd a few times. I'm > getting some weird error messages has anyone else seen this? > > I had to modify the make file > #PYTHON=python2 > PYTHON=python2.5 > > Then run the install manually > /usr/local/bin/python2.5 setup.py install > And cp readyexec /usr/local/bin > > Finally I kicked it off with > /usr/local/bin/sudo -u nobody readyexecd.py /tmp/pyzor1 pyzor.client.run > & > > local.cf > pyzor_path /usr/local/bin/readyexec > pyzor_options /tmp/pyzor1 > > I'm getting pyzor hits in the logs so all appears to be working, but the > > following are occasionally appearing on the terminal I started it from. > FreeBSD 6.2 by the way. > > > Traceback (most recent call last): > File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 385, > in process_request > self.finish_request(request, address) > File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 408, > in finish_request > super(ReadyExec, self).finish_request(request, address) > File "/usr/local/lib/python2.5/SocketServer.py", line 254, in > finish_request > self.RequestHandlerClass(request, client_address, self) > Fil
Re: Using Pzyor with high volume
Hi;
Since you are running the "server" portion as "nobody"... Then the spamd
user account needs to stay as "nobody".
We run SA as a system filter, so no calls to multiple different users.
root 20203 4.8 5.4 61144 56424 ?? S 4:11PM 2:41.23 spamd child
(perl5.8.8)
nobody21508 4.1 5.6 62920 58388 ?? S 4:13PM 2:27.77 spamd child
(perl5.8.8)
If I run it as root, it stops the error, but then it stop hitting things.
Though with -D it does show that it is talking to the pyzor server via the pipe.
(Note: I deleted the 66.250.40.33 from the pyzor servers list, but it is still
showing up, even after a spamd restart.)
# spamassassin -D pyzor < /tmp/spam.txt
[42041] dbg: pyzor: network tests on, attempting Pyzor
[42041] dbg: pyzor: pyzor is available: /usr/local/bin/readyexec
[42041] dbg: pyzor: opening pipe: /usr/local/bin/readyexec /tmp/pyzor1 check <
/tmp/.spamassassin420412jN0S2tmp
[42041] dbg: pyzor: [42060] finished: exit=0x0100
[42041] dbg: pyzor: got response: 82.94.255.100:24441 (200, 'OK') 0
0\n66.250.40.33:24441 TimeoutError:
[42041] dbg: pyzor: failure to parse response "66.250.40.33:24441 TimeoutError:
"
I'm off for a few days, so I'll have a play with it when I get back. I think,
as you pointed out, it could be to do with permissions. Cheers for that.
Rgds
n
Jason J. Ellingson wrote:
Since you are running the "server" portion as "nobody"... Then the spamd
user account needs to stay as "nobody". If you make a spamc call with a
"user" identified, then spamd will switch to that user account before
processing the message (this is for user specific needs like custom user
rules, BAYES, AWL, etc). Since the accounts now don't match, it'll give
you that error.
- Jason
-Original Message-
From: neil [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 01, 2008 9:31 AM
To: Jason J. Ellingson
Cc: Ben Poliakoff; Robert Blayzor; users@spamassassin.apache.org
Subject: Re: Using Pzyor with high volume
Hi;
I have it running and it works, load is down and free mem up, but not
certain if that is just because I've restarted spamd a few times. I'm
getting some weird error messages has anyone else seen this?
I had to modify the make file
#PYTHON=python2
PYTHON=python2.5
Then run the install manually
/usr/local/bin/python2.5 setup.py install
And cp readyexec /usr/local/bin
Finally I kicked it off with
/usr/local/bin/sudo -u nobody readyexecd.py /tmp/pyzor1 pyzor.client.run
&
local.cf
pyzor_path /usr/local/bin/readyexec
pyzor_options /tmp/pyzor1
I'm getting pyzor hits in the logs so all appears to be working, but the
following are occasionally appearing on the terminal I started it from.
FreeBSD 6.2 by the way.
Traceback (most recent call last):
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 385,
in process_request
self.finish_request(request, address)
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 408,
in finish_request
super(ReadyExec, self).finish_request(request, address)
File "/usr/local/lib/python2.5/SocketServer.py", line 254, in
finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 173,
in __init__
client_address, server)
File "/usr/local/lib/python2.5/SocketServer.py", line 522, in __init__
self.handle()
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 187,
in handle
self.handle_conduit()
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 209,
in handle_conduit
self.tell_exit(os.waitpid(pid, 0)[1] >> 8)
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 293,
in tell_exit
self.send_string("exit")
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 298,
in send_string
self.wfile.write(netstring(msg))
File "/usr/local/lib/python2.5/socket.py", line 262, in write
self.flush()
File "/usr/local/lib/python2.5/socket.py", line 249, in flush
self._sock.sendall(buffer)
error: (32, 'Broken pipe')
rgds
n
Jason J. Ellingson wrote:
Solved the problem.
readyexec is *USER* specific. You *MUST* launch the readyexecd.py
"server" part as the SAME user as the user of readyexec "client".
My spamd service is run as a user "spamc", while I was adding the
service as "root".
Now, that I have the service running as "spamc", we have success!
Thanks for your help folks!
- Jason
-Original Message-
From: Jason J. Ellingson [mailto:[EMAIL PROTECTED]
Sent: Wednesday, April 30, 2008 3:21 PM
To: Ben Poliakoff
Cc: Robert Blayzor; users@spamassassin.apache.org
Subject: RE: Using Pzyor with
RE: Using Pzyor with high volume
Since you are running the "server" portion as "nobody"... Then the spamd
user account needs to stay as "nobody". If you make a spamc call with a
"user" identified, then spamd will switch to that user account before
processing the message (this is for user specific needs like custom user
rules, BAYES, AWL, etc). Since the accounts now don't match, it'll give
you that error.
- Jason
-Original Message-
From: neil [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 01, 2008 9:31 AM
To: Jason J. Ellingson
Cc: Ben Poliakoff; Robert Blayzor; users@spamassassin.apache.org
Subject: Re: Using Pzyor with high volume
Hi;
I have it running and it works, load is down and free mem up, but not
certain if that is just because I've restarted spamd a few times. I'm
getting some weird error messages has anyone else seen this?
I had to modify the make file
#PYTHON=python2
PYTHON=python2.5
Then run the install manually
/usr/local/bin/python2.5 setup.py install
And cp readyexec /usr/local/bin
Finally I kicked it off with
/usr/local/bin/sudo -u nobody readyexecd.py /tmp/pyzor1 pyzor.client.run
&
local.cf
pyzor_path /usr/local/bin/readyexec
pyzor_options /tmp/pyzor1
I'm getting pyzor hits in the logs so all appears to be working, but the
following are occasionally appearing on the terminal I started it from.
FreeBSD 6.2 by the way.
Traceback (most recent call last):
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 385,
in process_request
self.finish_request(request, address)
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 408,
in finish_request
super(ReadyExec, self).finish_request(request, address)
File "/usr/local/lib/python2.5/SocketServer.py", line 254, in
finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 173,
in __init__
client_address, server)
File "/usr/local/lib/python2.5/SocketServer.py", line 522, in __init__
self.handle()
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 187,
in handle
self.handle_conduit()
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 209,
in handle_conduit
self.tell_exit(os.waitpid(pid, 0)[1] >> 8)
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 293,
in tell_exit
self.send_string("exit")
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 298,
in send_string
self.wfile.write(netstring(msg))
File "/usr/local/lib/python2.5/socket.py", line 262, in write
self.flush()
File "/usr/local/lib/python2.5/socket.py", line 249, in flush
self._sock.sendall(buffer)
error: (32, 'Broken pipe')
rgds
n
Jason J. Ellingson wrote:
> Solved the problem.
>
> readyexec is *USER* specific. You *MUST* launch the readyexecd.py
> "server" part as the SAME user as the user of readyexec "client".
>
> My spamd service is run as a user "spamc", while I was adding the
> service as "root".
>
> Now, that I have the service running as "spamc", we have success!
>
> Thanks for your help folks!
>
> - Jason
>
> -Original Message-
> From: Jason J. Ellingson [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, April 30, 2008 3:21 PM
> To: Ben Poliakoff
> Cc: Robert Blayzor; users@spamassassin.apache.org
> Subject: RE: Using Pzyor with high volume
>
> I am trying those settings, yet I get no Pyzor hits.
>
> I can manually do a "readyexec /tmp/pyzor ping" which works fine...
>
> Any other suggestions?
>
> Many thanks!
>
> - Jason
>
> -Original Message-
> From: Ben Poliakoff [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, April 30, 2008 1:23 PM
> To: Jason J. Ellingson
> Cc: Robert Blayzor; users@spamassassin.apache.org
> Subject: Re: Using Pzyor with high volume
>
> * Jason J. Ellingson <[EMAIL PROTECTED]> [20080430 11:07]:
>
>> Yup... I got the "server" portion running... The trick now is to get
>> SpamAssassin to use "readyexec /tmp/pyzor" instead of just "pyzor"...
>> Any suggestions? I was looking at modifying Pyzor.pm in the
>> SpamAssassin perl directory.
>>
>
> Something like this seems to work for me:
>
> use_pyzor 1
> pyzor_path /usr/local/bin/readyexec
> pyzor_options /tmp/pyzor
>
> Ben
>
>
Re: Using Pzyor with high volume
Hi;
I have it running and it works, load is down and free mem up, but not
certain if that is just because I've restarted spamd a few times. I'm
getting some weird error messages has anyone else seen this?
I had to modify the make file
#PYTHON=python2
PYTHON=python2.5
Then run the install manually
/usr/local/bin/python2.5 setup.py install
And cp readyexec /usr/local/bin
Finally I kicked it off with
/usr/local/bin/sudo -u nobody readyexecd.py /tmp/pyzor1 pyzor.client.run &
local.cf
pyzor_path /usr/local/bin/readyexec
pyzor_options /tmp/pyzor1
I'm getting pyzor hits in the logs so all appears to be working, but the
following are occasionally appearing on the terminal I started it from.
FreeBSD 6.2 by the way.
Traceback (most recent call last):
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 385,
in process_request
self.finish_request(request, address)
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 408,
in finish_request
super(ReadyExec, self).finish_request(request, address)
File "/usr/local/lib/python2.5/SocketServer.py", line 254, in
finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 173,
in __init__
client_address, server)
File "/usr/local/lib/python2.5/SocketServer.py", line 522, in __init__
self.handle()
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 187,
in handle
self.handle_conduit()
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 209,
in handle_conduit
self.tell_exit(os.waitpid(pid, 0)[1] >> 8)
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 293,
in tell_exit
self.send_string("exit")
File "/usr/local/lib/python2.5/site-packages/readyexec.py", line 298,
in send_string
self.wfile.write(netstring(msg))
File "/usr/local/lib/python2.5/socket.py", line 262, in write
self.flush()
File "/usr/local/lib/python2.5/socket.py", line 249, in flush
self._sock.sendall(buffer)
error: (32, 'Broken pipe')
rgds
n
Jason J. Ellingson wrote:
Solved the problem.
readyexec is *USER* specific. You *MUST* launch the readyexecd.py
"server" part as the SAME user as the user of readyexec "client".
My spamd service is run as a user "spamc", while I was adding the
service as "root".
Now, that I have the service running as "spamc", we have success!
Thanks for your help folks!
- Jason
-----Original Message-----
From: Jason J. Ellingson [mailto:[EMAIL PROTECTED]
Sent: Wednesday, April 30, 2008 3:21 PM
To: Ben Poliakoff
Cc: Robert Blayzor; users@spamassassin.apache.org
Subject: RE: Using Pzyor with high volume
I am trying those settings, yet I get no Pyzor hits.
I can manually do a "readyexec /tmp/pyzor ping" which works fine...
Any other suggestions?
Many thanks!
- Jason
-----Original Message-
From: Ben Poliakoff [mailto:[EMAIL PROTECTED]
Sent: Wednesday, April 30, 2008 1:23 PM
To: Jason J. Ellingson
Cc: Robert Blayzor; users@spamassassin.apache.org
Subject: Re: Using Pzyor with high volume
* Jason J. Ellingson <[EMAIL PROTECTED]> [20080430 11:07]:
Yup... I got the "server" portion running... The trick now is to get
SpamAssassin to use "readyexec /tmp/pyzor" instead of just "pyzor"...
Any suggestions? I was looking at modifying Pyzor.pm in the
SpamAssassin perl directory.
Something like this seems to work for me:
use_pyzor 1
pyzor_path /usr/local/bin/readyexec
pyzor_options /tmp/pyzor
Ben
RE: Using Pzyor with high volume
Solved the problem. readyexec is *USER* specific. You *MUST* launch the readyexecd.py "server" part as the SAME user as the user of readyexec "client". My spamd service is run as a user "spamc", while I was adding the service as "root". Now, that I have the service running as "spamc", we have success! Thanks for your help folks! - Jason -Original Message- From: Jason J. Ellingson [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 30, 2008 3:21 PM To: Ben Poliakoff Cc: Robert Blayzor; users@spamassassin.apache.org Subject: RE: Using Pzyor with high volume I am trying those settings, yet I get no Pyzor hits. I can manually do a "readyexec /tmp/pyzor ping" which works fine... Any other suggestions? Many thanks! - Jason -Original Message- From: Ben Poliakoff [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 30, 2008 1:23 PM To: Jason J. Ellingson Cc: Robert Blayzor; users@spamassassin.apache.org Subject: Re: Using Pzyor with high volume * Jason J. Ellingson <[EMAIL PROTECTED]> [20080430 11:07]: > Yup... I got the "server" portion running... The trick now is to get > SpamAssassin to use "readyexec /tmp/pyzor" instead of just "pyzor"... > Any suggestions? I was looking at modifying Pyzor.pm in the > SpamAssassin perl directory. Something like this seems to work for me: use_pyzor 1 pyzor_path /usr/local/bin/readyexec pyzor_options /tmp/pyzor Ben -- PGP fingerprint: A131 F813 7A0F C5B7 E74D C972 9118 A94D 6AF5 2019
Re: Using Pzyor with high volume
* Jason J. Ellingson <[EMAIL PROTECTED]> [20080430 13:21]: > I am trying those settings, yet I get no Pyzor hits. > > I can manually do a "readyexec /tmp/pyzor ping" which works fine... > > Any other suggestions? > Try running spamassassin with debug mode on (-D) look for pyzor related stuff. Ben -- PGP fingerprint: A131 F813 7A0F C5B7 E74D C972 9118 A94D 6AF5 2019 pgpJ18sxT0x7a.pgp Description: PGP signature
RE: Using Pzyor with high volume
I am trying those settings, yet I get no Pyzor hits. I can manually do a "readyexec /tmp/pyzor ping" which works fine... Any other suggestions? Many thanks! - Jason -Original Message- From: Ben Poliakoff [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 30, 2008 1:23 PM To: Jason J. Ellingson Cc: Robert Blayzor; users@spamassassin.apache.org Subject: Re: Using Pzyor with high volume * Jason J. Ellingson <[EMAIL PROTECTED]> [20080430 11:07]: > Yup... I got the "server" portion running... The trick now is to get > SpamAssassin to use "readyexec /tmp/pyzor" instead of just "pyzor"... > Any suggestions? I was looking at modifying Pyzor.pm in the > SpamAssassin perl directory. Something like this seems to work for me: use_pyzor 1 pyzor_path /usr/local/bin/readyexec pyzor_options /tmp/pyzor Ben -- PGP fingerprint: A131 F813 7A0F C5B7 E74D C972 9118 A94D 6AF5 2019
Re: Using Pzyor with high volume
* Jason J. Ellingson <[EMAIL PROTECTED]> [20080430 11:07]: > Yup... I got the "server" portion running... The trick now is to get > SpamAssassin to use "readyexec /tmp/pyzor" instead of just "pyzor"... > Any suggestions? I was looking at modifying Pyzor.pm in the > SpamAssassin perl directory. Something like this seems to work for me: use_pyzor 1 pyzor_path /usr/local/bin/readyexec pyzor_options /tmp/pyzor Ben -- PGP fingerprint: A131 F813 7A0F C5B7 E74D C972 9118 A94D 6AF5 2019 pgpqYXjn5Lnkm.pgp Description: PGP signature
Re: Using Pzyor with high volume
On Apr 30, 2008, at 2:04 PM, Jason J. Ellingson wrote: Yup... I got the "server" portion running... The trick now is to get SpamAssassin to use "readyexec /tmp/pyzor" instead of just "pyzor"... Any suggestions? I was looking at modifying Pyzor.pm in the SpamAssassin perl directory. My guess.. pyzor_path STRING This option tells SpamAssassin specifically where to find the "pyzor" client instead of relying on SpamAssassin to find it in the current PATH. Note that if taint mode is enabled in the Perl interpreter, you should use this, as the current PATH will have been cleared. So... pyzor_path readyexec --stop /tmp/pyzor May work... Even though ready exec is more lightweight than actually calling python each time, I'm still hoping that a non exec based plugin can appear someday. (again, if it's worth the trouble to do so). -- Robert Blayzor, BOFH INOC, LLC [EMAIL PROTECTED] http://www.inoc.net/~rblayzor/ Mac OS X. Because making Unix user-friendly is easier than debugging Windows.
RE: Using Pzyor with high volume
Yup... I got the "server" portion running... The trick now is to get SpamAssassin to use "readyexec /tmp/pyzor" instead of just "pyzor"... Any suggestions? I was looking at modifying Pyzor.pm in the SpamAssassin perl directory. - Jason -Original Message- From: Ben Poliakoff [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 30, 2008 1:02 PM To: Jason J. Ellingson Cc: Robert Blayzor; users@spamassassin.apache.org Subject: Re: Using Pzyor with high volume * Jason J. Ellingson <[EMAIL PROTECTED]> [20080430 10:59]: > I decided to look into this as well. > > I managed to get ReadyExec installed, but am having difficulty changing > the Pyzor.pm to find and use readyexec properly. Anyone else have luck? > This works for me: readyexecd.py /tmp/pyzor pyzor.client.run This stops readyexecd.py: readyexec --stop /tmp/pyzor Ben -- PGP fingerprint: A131 F813 7A0F C5B7 E74D C972 9118 A94D 6AF5 2019
Re: Using Pzyor with high volume
* Jason J. Ellingson <[EMAIL PROTECTED]> [20080430 10:59]: > I decided to look into this as well. > > I managed to get ReadyExec installed, but am having difficulty changing > the Pyzor.pm to find and use readyexec properly. Anyone else have luck? > This works for me: readyexecd.py /tmp/pyzor pyzor.client.run This stops readyexecd.py: readyexec --stop /tmp/pyzor Ben -- PGP fingerprint: A131 F813 7A0F C5B7 E74D C972 9118 A94D 6AF5 2019 pgpXgo2U89BP1.pgp Description: PGP signature
RE: Using Pzyor with high volume
I decided to look into this as well. I managed to get ReadyExec installed, but am having difficulty changing the Pyzor.pm to find and use readyexec properly. Anyone else have luck? - Jason -Original Message- From: Robert Blayzor [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 30, 2008 11:05 AM To: Ben Poliakoff Cc: users@spamassassin.apache.org Subject: Re: Using Pzyor with high volume On Apr 30, 2008, at 11:59 AM, Ben Poliakoff wrote: > Seems to be just the sort of thing to address your concern (short of > a perl implementation of the pyzor client). I should note that *I* > haven't used the ReadyExec stuff in my environment [1] (where > executing > the pyzor client hasn't been much of a resource drain), but I've > thought > about it. Yeah, I did run over this, but haven't had to much experience in installing/maintaining that. That's why I'm trying to weigh the value of Pyzor vs. having to complicate the installation any more. A Perl agent of Pyzor would be ideal. > > [1] My environment supports about 2000 users scanning roughly 45000 - > 7/day currently spread across two older linux boxes. My setup is over 10X that, which is why this is a concern! ;-) -- Robert Blayzor, BOFH INOC, LLC [EMAIL PROTECTED] http://www.inoc.net/~rblayzor/ Mac OS X. Because making Unix user-friendly is easier than debugging Windows.
Re: Using Pzyor with high volume
On Apr 30, 2008, at 11:59 AM, Ben Poliakoff wrote: Seems to be just the sort of thing to address your concern (short of a perl implementation of the pyzor client). I should note that *I* haven't used the ReadyExec stuff in my environment [1] (where executing the pyzor client hasn't been much of a resource drain), but I've thought about it. Yeah, I did run over this, but haven't had to much experience in installing/maintaining that. That's why I'm trying to weigh the value of Pyzor vs. having to complicate the installation any more. A Perl agent of Pyzor would be ideal. [1] My environment supports about 2000 users scanning roughly 45000 - 7/day currently spread across two older linux boxes. My setup is over 10X that, which is why this is a concern! ;-) -- Robert Blayzor, BOFH INOC, LLC [EMAIL PROTECTED] http://www.inoc.net/~rblayzor/ Mac OS X. Because making Unix user-friendly is easier than debugging Windows.
Re: Using Pzyor with high volume
* Robert Blayzor <[EMAIL PROTECTED]> [20080430 07:46]: > In regards to Pyzor. I'm wondering if anyone out there is using this at > any large scale. Unlike the razor-agent which appears to be a Perl module > that gets loaded at startup, I'm concerned about SA having to exec the > python interpreter and having that setup/teardown time for each and every > message. > > Adding salt to the wound, our SA servers run on diskless servers; so having > it have to run over NFS makes for a double whammy. > > Is there a better way to implement Pyzor or is it not even worth the > trouble? > Looking at the pyzor man page I've noted that pyzor can be made to run with "ReadyExec": ReadyExec is a system to eliminate the high startup-cost of executing scripts repeatedly. If you execute pyzor a lot, you might be interested in installing ReadyExec and using it with pyzor. Seems to be just the sort of thing to address your concern (short of a perl implementation of the pyzor client). I should note that *I* haven't used the ReadyExec stuff in my environment [1] (where executing the pyzor client hasn't been much of a resource drain), but I've thought about it. [1] My environment supports about 2000 users scanning roughly 45000 - 7/day currently spread across two older linux boxes. -- PGP fingerprint: A131 F813 7A0F C5B7 E74D C972 9118 A94D 6AF5 2019 pgpb5lrSS3FlU.pgp Description: PGP signature
Using Pzyor with high volume
In regards to Pyzor. I'm wondering if anyone out there is using this at any large scale. Unlike the razor-agent which appears to be a Perl module that gets loaded at startup, I'm concerned about SA having to exec the python interpreter and having that setup/teardown time for each and every message. Adding salt to the wound, our SA servers run on diskless servers; so having it have to run over NFS makes for a double whammy. Is there a better way to implement Pyzor or is it not even worth the trouble? TIA -- Robert Blayzor, BOFH INOC, LLC [EMAIL PROTECTED] http://www.inoc.net/~rblayzor/ Mac OS X. Because making Unix user-friendly is easier than debugging Windows.
