Re: spamd-setup hangup/timeout settings
Hi, sorry for reopening/closing this old case; this msg. is just for notice: Increasing kern.maxclusters has solved the problem here. :) Just in case anyone has pain with the same problem. -Florian -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Florian Fuessl Sent: Sunday, January 06, 2008 12:14 PM To: misc@openbsd.org Subject: spamd-setup hangup/timeout settings Hi, I'm running spamd-setup via regular cronjob every 20 minutes. Sometimes the spamd-setup process seems to hang and does not finish within this period, although all black- and whitelists are local files. Is there a way define timeouts for tasks of spamd-setup? What solution is recommended for this case? Killing the running spamd-setup task(s) before starting a new one? -Florian
Re: spamd-setup hangup/timeout settings
Florian Fuessl [EMAIL PROTECTED] writes: Increasing kern.maxclusters has solved the problem here. :) It would be interesting to hear a bit more about which values you have tried and to what effect. - P -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: spamd-setup hangup/timeout settings
Peter N. M. Hansteen [EMAIL PROTECTED] wrote: Florian Fuessl [EMAIL PROTECTED] writes: Increasing kern.maxclusters has solved the problem here. :) It would be interesting to hear a bit more about which values you have tried and to what effect. Doubling the default (6144) should be far enough; I did setup the value 128000 which is obviously exaggerated but seems to works fine here: ~ # netstat -m 2766 mbufs in use: 2246 mbufs allocated to data 515 mbufs allocated to packet headers 5 mbufs allocated to socket names and addresses 1675/6152/128000 mbuf clusters in use (current/peak/max) 14460 Kbytes allocated to network (27% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines
Re: spamd-setup hangup/timeout settings
Strange thing here is that spamd-setup sometimes hangs although all spamd.conf entries point to local copied traplists downloaded by a separate cron process. :-| -Flo -Original Message- Of Mike Erdely Sent: Wednesday, January 09, 2008 7:54 PM You could point to a local copy (/var/db/traplist.gz) in spamd.conf and download it in a separate cron process. -ME
Re: spamd-setup hangup/timeout settings
Jason George wrote: My spamd-setup always takes 20-30 minutes on two servers (4.1 and 4.2). This is not normal? When I run it manually; most of the time is spent downloading traplist.gz You are all connecting to beck@'s machine at the University of Alberta (www.openbsd.org) ? I use the same major ISP that the U of A uses as one of its principal peers. I get timeouts, poor throughput and generally the same behaviour as mentioned above for all transfers to that site (spamd lists, snapshots, etc). It appears somebody changed something on Saturday. I had several downloads on that day timeout after 332 minutes. Since then, they are most often in 15-35 second range; never over 2 minutes. Thanks!
Re: spamd-setup hangup/timeout settings
Jason George wrote: My spamd-setup always takes 20-30 minutes on two servers (4.1 and 4.2). This is not normal? When I run it manually; most of the time is spent downloading traplist.gz You are all connecting to beck@'s machine at the University of Alberta (www.openbsd.org) ? I use the same major ISP that the U of A uses as one of its principal peers. I get timeouts, poor throughput and generally the same behaviour as mentioned above for all transfers to that site (spamd lists, snapshots, etc). Are there any alternative? /etc/mail/spamd.conf mentions www.de.openbsd.org; but Beck's traplist.gz is not actually mirrored there. Frank
Re: spamd-setup hangup/timeout settings
On Wed, Jan 09, 2008 at 09:59:58AM -0500, Frank Bax wrote: Are there any alternative? /etc/mail/spamd.conf mentions www.de.openbsd.org; but Beck's traplist.gz is not actually mirrored there. You could point to a local copy (/var/db/traplist.gz) in spamd.conf and download it in a separate cron process. -ME
Re: spamd-setup hangup/timeout settings
Florian Fuessl wrote: Frank Bax wrote: My spamd-setup always takes 20-30 minutes on two servers (4.1 and 4.2). This is not normal? When I run it manually; most of the time is spent downloading traplist.gz This morning, I changed the crontab time /usr/libexec/spamd-setup -d 4.1 runtimes (minutes): 39, 10, 27, 32 4.2 runtimes (minutes): 23, 29, 22, 22 The runtime here is usually only a few seconds: [...] time /usr/libexec/spamd-setup -d blacklist becks 110141 entries blacklist nixspam 39962 entries blacklist sorbs-zombie 102 entries blacklist sorbs-dul 277194 entries whitelist sorbs-dul-white 357120 entries 0m13.34s real 0m5.80s user 0m2.21s system That's why it's so strange, that sometimes spamd-setup process hangs and does not quit within 20 minutes. :- Any ideas, how to handle that case correctly or how to debug that case? Sorry, I can't help you. I have determined that my problem is with download. I've timed the downloads (wget only, not spamd-setup) between 27 seconds and 45 minutes. I'm no expert here; but I don't think it's name resolution (as Peter suggested), because wget starts off downloading rather quickly, then slows/stalls during download; one time it even timed out and started over on its own. My servers are co-locate and I've asked my ISP to check into it; because I'm thinking the problem is outside my box. Frank
Re: spamd-setup hangup/timeout settings
My spamd-setup always takes 20-30 minutes on two servers (4.1 and 4.2). This is not normal? When I run it manually; most of the time is spent downloading traplist.gz This morning, I changed the crontab time /usr/libexec/spamd-setup -d 4.1 runtimes (minutes): 39, 10, 27, 32 4.2 runtimes (minutes): 23, 29, 22, 22 The runtime here is usually only a few seconds: [...] time /usr/libexec/spamd-setup -d blacklist becks 110141 entries blacklist nixspam 39962 entries blacklist sorbs-zombie 102 entries blacklist sorbs-dul 277194 entries whitelist sorbs-dul-white 357120 entries 0m13.34s real 0m5.80s user 0m2.21s system That's why it's so strange, that sometimes spamd-setup process hangs and does not quit within 20 minutes. :- Any ideas, how to handle that case correctly or how to debug that case? Sorry, I can't help you. I have determined that my problem is with download. I've timed the downloads (wget only, not spamd-setup) between 27 seconds and 45 minutes. I'm no expert here; but I don't think it's name resolution (as Peter suggested), because wget starts off downloading rather quickly, then slows/stalls during download; one time it even timed out and started over on its own. My servers are co-locate and I've asked my ISP to check into it; because I'm thinking the problem is outside my box. You are all connecting to beck@'s machine at the University of Alberta (www.openbsd.org) ? I use the same major ISP that the U of A uses as one of its principal peers. I get timeouts, poor throughput and generally the same behaviour as mentioned above for all transfers to that site (spamd lists, snapshots, etc).
Re: spamd-setup hangup/timeout settings
Frank Bax wrote: My spamd-setup always takes 20-30 minutes on two servers (4.1 and 4.2). This is not normal? When I run it manually; most of the time is spent downloading traplist.gz This morning, I changed the crontab time /usr/libexec/spamd-setup -d 4.1 runtimes (minutes): 39, 10, 27, 32 4.2 runtimes (minutes): 23, 29, 22, 22 The runtime here is usually only a few seconds: [...] time /usr/libexec/spamd-setup -d blacklist becks 110141 entries blacklist nixspam 39962 entries blacklist sorbs-zombie 102 entries blacklist sorbs-dul 277194 entries whitelist sorbs-dul-white 357120 entries 0m13.34s real 0m5.80s user 0m2.21s system That's why it's so strange, that sometimes spamd-setup process hangs and does not quit within 20 minutes. :- Any ideas, how to handle that case correctly or how to debug that case? -Florian
spamd-setup hangup/timeout settings
Hi, I'm running spamd-setup via regular cronjob every 20 minutes. Sometimes the spamd-setup process seems to hang and does not finish within this period, although all black- and whitelists are local files. Is there a way define timeouts for tasks of spamd-setup? What solution is recommended for this case? Killing the running spamd-setup task(s) before starting a new one? -Florian
Re: spamd-setup hangup/timeout settings
Florian Fuessl [EMAIL PROTECTED] writes: I'm running spamd-setup via regular cronjob every 20 minutes. Sometimes the spamd-setup process seems to hang and does not finish within this period, although all black- and whitelists are local files. I would try to figure out why the process stalls. My first hunch when something network related suddenly takes a lot longer than usual is to check that name resolution is actually working. Also, it is worth try spamd-setup with -d, then it provides a little more data on what actually happens. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.datadok.no/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: spamd-setup hangup/timeout settings
Peter N. M. Hansteen [EMAIL PROTECTED] wrote: I'm running spamd-setup via regular cronjob every 20 minutes. Sometimes the spamd-setup process seems to hang and does not finish within this period, although all black- and whitelists are local files. I would try to figure out why the process stalls. My first hunch when something network related suddenly takes a lot longer than usual is to check that name resolution is actually working. Also, it is worth try spamd-setup with -d, then it provides a little more data on what actually happens. That was also my first thought. That's why I reconfigured spamd-setup to use local files only and fetch the needed files before execution with wget and a useful timeout. The problem only occurs rarely after spamd-setup processed all black- and whitelists while trying to update the spamdb database (or maybe spamd-white pf-table). spamd-setup with -d option does not show any details after importing the last whitelist. :- Killing spamd-setup also quits the spamd process. -Florian
Re: spamd-setup hangup/timeout settings
Florian Fuessl wrote: I'm running spamd-setup via regular cronjob every 20 minutes. Sometimes the spamd-setup process seems to hang and does not finish within this period, although all black- and whitelists are local files. Is there a way define timeouts for tasks of spamd-setup? What solution is recommended for this case? Killing the running spamd-setup task(s) before starting a new one? My spamd-setup always takes 20-30 minutes on two servers (4.1 and 4.2). This is not normal? When I run it manually; most of the time is spent downloading traplist.gz This morning, I changed the crontab time /usr/libexec/spamd-setup -d 4.1 runtimes (minutes): 39, 10, 27, 32 4.2 runtimes (minutes): 23, 29, 22, 22 blacklist myblack 29 entries whitelist mywhite 107 entries Getting http://www.openbsd.org/spamd/traplist.gz blacklist uatraps 61229 entries Getting http://www.openbsd.org/spamd/nixspam.gz blacklist nixspam 4 entries Getting http://www.openbsd.org/spamd/chinacidr.txt.gz blacklist china 431 entries Getting http://www.openbsd.org/spamd/koreacidr.txt.gz blacklist korea 270 entries 22m10.16s real 0m1.66s user 0m0.37s system
Re: spamd-setup hangup/timeout settings
Frank Bax [EMAIL PROTECTED] writes: My spamd-setup always takes 20-30 minutes on two servers (4.1 and 4.2). This is not normal? When I run it manually; most of the time is spent downloading traplist.gz I guess I must stop complaining about my line speeds, then. My sole remaining 4.1 box, just now (becks here is really uatraps): $ sudo time /usr/libexec/spamd-setup -d Getting http://www.openbsd.org/spamd/traplist.gz blacklist becks 113415 entries whitelist whitelist 113458 entries 33.18 real 4.05 user 1.19 sys and on one of my 4.2s, $ sudo time /usr/libexec/spamd-setup -d Getting http://www.openbsd.org/spamd/traplist.gz blacklist becks 113415 entries whitelist whitelist 113460 entries 16.64 real 4.32 user 1.46 sys - P -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.datadok.no/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
