Re: [Clamav-users] clamav-milter children hanging, eating CPU
On Fri, Aug 13, 2004 at 04:07:47PM -0700, Jim Gaynor said: > It isn't sendmail that's borking this system, tho; it's the multiple > high-load high-memory clamav-milter processes. I've checked the sendmail > queue when those processes start to hog resources, and only had 32 items > in queue one time, 24 another. Heck, right now I have 26, and load is > still < 1.0 > > I'm not saying your approach is wrong, I'm just saying I'm not entirely > convinced it's right - spawning off more children doesn't seem the > answer when the existing ones appear to have all gone into > high-resource-consumption state... No, you're reading me backwards. It is the clamav-milter child threads killing the system. However, adding senmail processes stuck in a wait state to that only makes it worse. Get rid of the max-children in clamav-milter, and control the overall scene by reducing the number of sendmail -> milter processes spawned in sendmail. Doing it in the milter just adds choke to sendmail. This is what I have found most effective in (fairly) high-load mail systems, meaning 50-100,000 emails a day, where we do both clam and spamassassin scanning. It's better to delay the startup of a new sendmail -> clam -> whatever process, than to start up the transaction, and keep it waiting around longer because the system is resource starved. YMMV. -- -- | Stephen Gran | BOFH excuse #221: The mainframe needs | | [EMAIL PROTECTED] | to rest. It's getting old, you know. | | http://www.lobefin.net/~steve | | -- pgpzKDL6aTON2.pgp Description: PGP signature
Re: [Clamav-users] clamav-milter children hanging, eating CPU
Stephen Gran wrote: On Fri, Aug 13, 2004 at 12:14:10PM -0700, Jim Gaynor said: clamav-milter --noreject --postmaster-only --local --max-children=10 /var/run/clamav/clmilter.sock In the last two days, I've twice had my system load jump to > 10.0 (to the point where sendmail was rejecting incoming connections) due to the max number of clamav-milter children spawning and consuming memory and CPU. If I kill -9 the clamav-milter processes (doesn't respond to a graceful restart request), then restart clamav-milter, it's good to go, and doesn't immediately try to spawn the max number of children. Get rid of max-children. Without the argument, clamav-milter spawns as many child processes as it needs. With it, requests from sendmail get stuck waiting for an available child, and the load can easily go through the roof. If the load from unlimited milter processes is too much, try limitng the number of sendmail processes that are allowed at a time. MaxDaemonChildren or something - you'll have to double check, since it's been a little while. It isn't sendmail that's borking this system, tho; it's the multiple high-load high-memory clamav-milter processes. I've checked the sendmail queue when those processes start to hog resources, and only had 32 items in queue one time, 24 another. Heck, right now I have 26, and load is still < 1.0 I'm not saying your approach is wrong, I'm just saying I'm not entirely convinced it's right - spawning off more children doesn't seem the answer when the existing ones appear to have all gone into high-resource-consumption state... -jg -- Jim Gaynor, SATG - Senior Computer Specialist UW College of Engineering, Office of the Dean email: [EMAIL PROTECTED] --- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 ___ Clamav-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/clamav-users
Re: [Clamav-users] clamav-milter children hanging, eating CPU
On Fri, Aug 13, 2004 at 12:14:10PM -0700, Jim Gaynor said: > FreeBSD 4.10 > ~200 users > ClamAV version devel-20040806 > clamav-milter version 0.74a > Sendmail 8.12.11 > > clamav-milter --noreject --postmaster-only --local --max-children=10 > /var/run/clamav/clmilter.sock > > I've already checked the FAQ, searched the archive, and Googled. > > Normal system load is low, less than 1.0. Normally, one clamav-milter > process is seen in a ps. Clamav-milter doesn't even make the list in > "top", with mailman and imapd processes bumping it off the screen. > However... > > In the last two days, I've twice had my system load jump to > 10.0 (to > the point where sendmail was rejecting incoming connections) due to the > max number of clamav-milter children spawning and consuming memory and CPU. > > If I kill -9 the clamav-milter processes (doesn't respond to a graceful > restart request), then restart clamav-milter, it's good to go, and > doesn't immediately try to spawn the max number of children. > > Until the next day. > > Any thoughts or pointers, as I hunt for causes to this? Get rid of max-children. Without the argument, clamav-milter spawns as many child processes as it needs. With it, requests from sendmail get stuck waiting for an available child, and the load can easily go through the roof. If the load from unlimited milter processes is too much, try limitng the number of sendmail processes that are allowed at a time. MaxDaemonChildren or something - you'll have to double check, since it's been a little while. -- -- | Stephen Gran | Why are you so hard to ignore? | | [EMAIL PROTECTED] | | | http://www.lobefin.net/~steve | | -- pgp1wK7cKJxvw.pgp Description: PGP signature
[Clamav-users] clamav-milter children hanging, eating CPU
FreeBSD 4.10 ~200 users ClamAV version devel-20040806 clamav-milter version 0.74a Sendmail 8.12.11 clamav-milter --noreject --postmaster-only --local --max-children=10 /var/run/clamav/clmilter.sock I've already checked the FAQ, searched the archive, and Googled. Normal system load is low, less than 1.0. Normally, one clamav-milter process is seen in a ps. Clamav-milter doesn't even make the list in "top", with mailman and imapd processes bumping it off the screen. However... In the last two days, I've twice had my system load jump to > 10.0 (to the point where sendmail was rejecting incoming connections) due to the max number of clamav-milter children spawning and consuming memory and CPU. If I kill -9 the clamav-milter processes (doesn't respond to a graceful restart request), then restart clamav-milter, it's good to go, and doesn't immediately try to spawn the max number of children. Until the next day. Any thoughts or pointers, as I hunt for causes to this? -jg -- Jim Gaynor, SATG - Senior Computer Specialist UW College of Engineering, Office of the Dean email: [EMAIL PROTECTED] --- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 ___ Clamav-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/clamav-users
