Re: [Clamav-users] Freshclam stability as a daemon
Barry Gill wrote: On the subject of Cron, When I was still running Fedora Core 3, I was running my freshclam's using cron but found from time to time that when I came in to work in the morning I would have 20 to 30 cron processes still holding on chewing up resource but not finishing off anything. This caused me immense problems and was trhe primary reason for moving away from cron to freshclam in daemon mode. This is still possible and with anything run from cron you should exercise care. The first part of my wrapper for clam is: #!/bin/sh if /usr/bin/pgrep -x freshclam >/dev/null 2>&1; then echo 'Killing an instance of freshclam that is already running!' |\ /usr/bin/mailx -s '[omak] freshclam error' [EMAIL PROTECTED] /usr/bin/pkill freshclam || echo 'Cannot kill freshclam instance' fi if [ -z "$1" ]; then /usr/bin/bash -c '/usr/bin/sleep $[ RANDOM % 900 ]' fi ... I get mail from mine about 3 times a week either because a previous version is running or because of communication errors with the mirror. The rule of thumb is don't start a new freshclam process until the old process has finished or is killed. I kill it because it should certainly not still be running at the next invocation, and if it is it cannot be doing anything useful. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
RE: [Clamav-users] Freshclam stability as a daemon
On the subject of Cron, When I was still running Fedora Core 3, I was running my freshclam's using cron but found from time to time that when I came in to work in the morning I would have 20 to 30 cron processes still holding on chewing up resource but not finishing off anything. This caused me immense problems and was trhe primary reason for moving away from cron to freshclam in daemon mode. I did loads of research on the net and found sparse references to this problem and little to no resolution, since moving to daemon mode, I have had none of these issues (and still don't know the cause) so am happier with the stability I get from the daemon. B ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update
Daniel T. Staal wrote: > Has anyone tried both? What happens if you try to run freshclam as a > daemon and from cron? (Assuming you schedule them to run at different > times, of course. If they both checked at the same time I would > expect something to bork.) If they both ran at the same time, and the databases had been updated, I see significant potential for something to break. /Per Jessen, Zürich ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update
On Fri, December 29, 2006 3:54 pm, Dennis Peterson said: > This thread has become fractious (I claim some responsibility) and > unhelpful to the project. Fortunately ClamAV supports all our methods and > we can apply our own disciplines to the problems we each see and that's a > good thing. Just out of morbid curiosity at this point... Has anyone tried both? What happens if you try to run freshclam as a daemon and from cron? (Assuming you schedule them to run at different times, of course. If they both checked at the same time I would expect something to bork.) I don't see a good reason to actually run such a system, but I'm curious to know how well it works. ;) Daniel T. Staal --- This email copyright the author. Unless otherwise noted, you are expressly allowed to retransmit, quote, or otherwise use the contents for non-commercial purposes. This copyright will expire 5 years after the author's death, or in 30 years, whichever is longer, unless such a period is in excess of local copyright law. --- ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
Per Jessen wrote: Dennis Peterson wrote: And as an old school Unix admin who still believes in the mentoring responsibility of my position, I will make recommendations from time to time regarding best practices and I recommend if you run freshclam as a daemon that you monitor it and restart it if needed. Do you do that for ALL your daemon processes? As an old school mainframe sysprog, I don't monitor any of my daemon processes. (apart from *some* status-monitoring via SNMP). Throwing my 2c in, I have a cron job that runs every couple hours and just reports on the status of various daemons. It tells me if any of them are missing, basically. But, it doesn't try to restart them (bad idea, IMO; for most daemons, it's better for a human to go look at why the process isn't running and try to solve it, instead of just blindly/programatically trying to restart it). It's just warning me that something that _should_ be running is not. In the 2 years I've been running clamav, I haven't had freshclam come up missing. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update
> > Dennis Peterson wrote: > > [60 lines snipped] > > > > I can only tell you from my experience with several years and many > > versions of ClamAV that I have found no advantage in any category to > > running freshclam as a daemon, and running it in cron gives me many > > options not otherwise available - not the least of which is I can run > > it at random intervals to help break up lockstep assaults on the > > servers it polls. > > As you know, I'm running freshclam as a daemon, and I'm curious as to > what additional options (or even advantages) you get by running it > under cron? > > > And as an old school Unix admin who still believes in the mentoring > > responsibility of my position, I will make recommendations from time > > to time regarding best practices and I recommend if you run freshclam > > as a daemon that you monitor it and restart it if needed. > > Do you do that for ALL your daemon processes? As an old school > mainframe sysprog, I don't monitor any of my daemon processes. (apart > from *some* status-monitoring via SNMP). Yes, of course. This thread has become fractious (I claim some responsibility) and unhelpful to the project. Fortunately ClamAV supports all our methods and we can apply our own disciplines to the problems we each see and that's a good thing. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
Dennis Peterson wrote: [60 lines snipped] > > I can only tell you from my experience with several years and many > versions of ClamAV that I have found no advantage in any category to > running freshclam as a daemon, and running it in cron gives me many > options not otherwise available - not the least of which is I can run > it at random intervals to help break up lockstep assaults on the > servers it polls. As you know, I'm running freshclam as a daemon, and I'm curious as to what additional options (or even advantages) you get by running it under cron? > And as an old school Unix admin who still believes in the mentoring > responsibility of my position, I will make recommendations from time > to time regarding best practices and I recommend if you run freshclam > as a daemon that you monitor it and restart it if needed. Do you do that for ALL your daemon processes? As an old school mainframe sysprog, I don't monitor any of my daemon processes. (apart from *some* status-monitoring via SNMP). /Per Jessen, Zürich PS: even if you're an old school Unix admin, quoting only the relevant bits in your reply is still considered good netiquette. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon
G.W. Haywood wrote: > I'm calling for those who run freshclam as a daemon and who don't see > any problems with it to chip into this thread. How many of us are > there? We're running freshclam as a daemon - probably for about 2 years, I'm not sure. AFAIK, we have not seen any stability problems, and I do not expect any either. /Per Jessen, Zürich ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
Dennis Peterson wrote: Jim Maul wrote: Dennis Peterson wrote: This is not rocket science. Who said it was? The OP clearly asked for people who run freshclam as a daemon who have NOT had problems with it in the setup. You are not one of those people so im still trying to figure out why you felt the need to post. Cmon, this is not rocket science. As one of those who talked of freshclam dying I was offering background on why I did so and the disciplines that cause me to configure systems as I do. What was the purpose of your post? dp The purpose of my post was to point out that you did not even remotely provide what the OP was asking for. He was asking to hear from those of us who DO use freshclam as a daemon and what OUR experiences were. Instead, you chose to give a detailed explanation on why DONT use freshclam in daemon mode and what you do instead. Then you chose to throw in a little condescending "this is not rocket science" comment at the end. Classy really. Happy holidays. -Jim ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
Jim Maul wrote: Dennis Peterson wrote: This is not rocket science. Who said it was? The OP clearly asked for people who run freshclam as a daemon who have NOT had problems with it in the setup. You are not one of those people so im still trying to figure out why you felt the need to post. Cmon, this is not rocket science. As one of those who talked of freshclam dying I was offering background on why I did so and the disciplines that cause me to configure systems as I do. What was the purpose of your post? dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
On Fri, 2006-12-29 at 12:31 +, G.W. Haywood wrote: > Hi there, > > Some time ago somebody wrote, and somebody else replied: > > Why not just run freshclam as a daemon? > >>> Then you really need to have a daemon watcher to keep it going. > > Talk of freshclam dying gives me some discomfort, yet in almost two > years running freshclam as a daemon on two - not particularly busy - > servers I've never seen it fail. It uses around a megabyte of memory > on a machine with 2G of RAM and, doing hourly updates, it takes maybe > three seconds of CPU per month on a 1GHz twin-processor Pentium box. > > Naturally if freshclam dies we can expect people to mention it. I'm > calling for those who run freshclam as a daemon and who don't see any > problems with it to chip into this thread. How many of us are there? > > Here are the non-comment line in my config in case it has a bearing: > > DatabaseDirectory /var/lib/clamav > UpdateLogFile /tmp/.clam/freshclam.log > LogVerbose > LogSyslog > PidFile /var/run/clam/freshclam.pid > DNSDatabaseInfo current.cvd.clamav.net > DatabaseMirror db.uk.clamav.net > DatabaseMirror database.clamav.net > MaxAttempts 5 > Checks 24 > > Here's how I start it: > > /usr/local/bin/freshclam -d --daemon-notify=/etc/mail/clamav/clamd.conf > > Does anyone have any clues to the reasons behind freshclam's apparent > unreliability under some circumstances? Bad DB servers? Mail load? > Swap? Locking? Conflict with other processes? OS? Libraries? ... I had freshclam working from cron and after switching to the daemon mode some while back it did hang on 1 server. Freshclam daemon was up but wasn't downloading new db's. A kill and restart of freshclam daemon did the job and I have never experienced it again. The reason is unknown for now. -- With kind regards, Maurice Lucas TAOS-IT ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
Dennis Peterson wrote: G.W. Haywood wrote: Hi there, Some time ago somebody wrote, and somebody else replied: Why not just run freshclam as a daemon? Then you really need to have a daemon watcher to keep it going. Talk of freshclam dying gives me some discomfort, yet in almost two years running freshclam as a daemon on two - not particularly busy - servers I've never seen it fail. It uses around a megabyte of memory on a machine with 2G of RAM and, doing hourly updates, it takes maybe three seconds of CPU per month on a 1GHz twin-processor Pentium box. Naturally if freshclam dies we can expect people to mention it. I'm calling for those who run freshclam as a daemon and who don't see any problems with it to chip into this thread. How many of us are there? Here are the non-comment line in my config in case it has a bearing: DatabaseDirectory /var/lib/clamav UpdateLogFile /tmp/.clam/freshclam.log LogVerbose LogSyslog PidFile /var/run/clam/freshclam.pid DNSDatabaseInfo current.cvd.clamav.net DatabaseMirror db.uk.clamav.net DatabaseMirror database.clamav.net MaxAttempts 5 Checks 24 Here's how I start it: /usr/local/bin/freshclam -d --daemon-notify=/etc/mail/clamav/clamd.conf Does anyone have any clues to the reasons behind freshclam's apparent unreliability under some circumstances? Bad DB servers? Mail load? Swap? Locking? Conflict with other processes? OS? Libraries? ... -- 73, Ged. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html The operation of freshclam is unrelated to the traffic volume of the site so that is unimportant. It does only one job and it does it well. A busy site only means it is a greater liability if it should fail or if it should copy or produce flawed files, or fail to download new files. But if you run it as a daemon in a production environment then it is a simple best practice next step to monitor it and restart it should it fail. You may have a different view of what is a best practice in this regard (and it may even extend beyond freshclam) that leads you to choose to run freshclam as a daemon without monitoring and watchdog restart capability. I can only tell you from my experience with several years and many versions of ClamAV that I have found no advantage in any category to running freshclam as a daemon, and running it in cron gives me many options not otherwise available - not the least of which is I can run it at random intervals to help break up lockstep assaults on the servers it polls. And as an old school Unix admin who still believes in the mentoring responsibility of my position, I will make recommendations from time to time regarding best practices and I recommend if you run freshclam as a daemon that you monitor it and restart it if needed. Sun's SMF and other methodologies (cfengine, watchdog) can do this trivially but fail to do other checks of data integrity which must be scripted. So long as clamd can be killed and left unable to restart because of the presence of a corrupt or badly formated ndb file and since the db update process requires scripting anyway it makes sense to me to wrap the freshclam process and fetching other db's in cron driven scripts that: Run at random intervals Validate the databases that are downloaded including those that are not collected by freshclam (Sane Security, MSRBL, for examples) Move the validated files to the working directory Test the new files against known samples Retry on error or server failures Notify the admin chain and log the error This is not rocket science. Who said it was? The OP clearly asked for people who run freshclam as a daemon who have NOT had problems with it in the setup. You are not one of those people so im still trying to figure out why you felt the need to post. Cmon, this is not rocket science. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
G.W. Haywood wrote: Hi there, Some time ago somebody wrote, and somebody else replied: Why not just run freshclam as a daemon? Then you really need to have a daemon watcher to keep it going. Talk of freshclam dying gives me some discomfort, yet in almost two years running freshclam as a daemon on two - not particularly busy - servers I've never seen it fail. It uses around a megabyte of memory on a machine with 2G of RAM and, doing hourly updates, it takes maybe three seconds of CPU per month on a 1GHz twin-processor Pentium box. Naturally if freshclam dies we can expect people to mention it. I'm calling for those who run freshclam as a daemon and who don't see any problems with it to chip into this thread. How many of us are there? Here are the non-comment line in my config in case it has a bearing: DatabaseDirectory /var/lib/clamav UpdateLogFile /tmp/.clam/freshclam.log LogVerbose LogSyslog PidFile /var/run/clam/freshclam.pid DNSDatabaseInfo current.cvd.clamav.net DatabaseMirror db.uk.clamav.net DatabaseMirror database.clamav.net MaxAttempts 5 Checks 24 Here's how I start it: /usr/local/bin/freshclam -d --daemon-notify=/etc/mail/clamav/clamd.conf Does anyone have any clues to the reasons behind freshclam's apparent unreliability under some circumstances? Bad DB servers? Mail load? Swap? Locking? Conflict with other processes? OS? Libraries? ... -- 73, Ged. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html The operation of freshclam is unrelated to the traffic volume of the site so that is unimportant. It does only one job and it does it well. A busy site only means it is a greater liability if it should fail or if it should copy or produce flawed files, or fail to download new files. But if you run it as a daemon in a production environment then it is a simple best practice next step to monitor it and restart it should it fail. You may have a different view of what is a best practice in this regard (and it may even extend beyond freshclam) that leads you to choose to run freshclam as a daemon without monitoring and watchdog restart capability. I can only tell you from my experience with several years and many versions of ClamAV that I have found no advantage in any category to running freshclam as a daemon, and running it in cron gives me many options not otherwise available - not the least of which is I can run it at random intervals to help break up lockstep assaults on the servers it polls. And as an old school Unix admin who still believes in the mentoring responsibility of my position, I will make recommendations from time to time regarding best practices and I recommend if you run freshclam as a daemon that you monitor it and restart it if needed. Sun's SMF and other methodologies (cfengine, watchdog) can do this trivially but fail to do other checks of data integrity which must be scripted. So long as clamd can be killed and left unable to restart because of the presence of a corrupt or badly formated ndb file and since the db update process requires scripting anyway it makes sense to me to wrap the freshclam process and fetching other db's in cron driven scripts that: Run at random intervals Validate the databases that are downloaded including those that are not collected by freshclam (Sane Security, MSRBL, for examples) Move the validated files to the working directory Test the new files against known samples Retry on error or server failures Notify the admin chain and log the error This is not rocket science. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
Two Linux boxes here, one Generic (used to be Caldera 2.2) and one Fedora Core 5 been running it on both for close to 18 monthsnot one failure that I'm aware of. Both boxes are fairly busy as well, the caldera is an email/list/web box and the FC5 is a weather map server/forecasting tools system (lots of CPU use and disk activity at the top of every hour). ===[George R. Kasica]===+1 262 677 0766 President +1 206 374 6482 FAX Netwrx Consulting Inc. Jackson, WI USA http://www.netwrx1.com [EMAIL PROTECTED] ICQ #12862186 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
RE: [Clamav-users] Freshclam stability as a daemon
> Naturally if freshclam dies we can expect people to mention it. I'm > calling for those who run freshclam as a daemon and who don't see any > problems with it to chip into this thread. How many of us are there? I run freshclam in daemon mode for several servers processing several hundred thousand messages per day an have yet to see freshclam hang. I also run variations of 45, 47 and 49 updates... I run FC 4-6, everything in x86_64 mode, and have zero problems with freshclam. I have had problems where freshclam downloads a corrupted daily.cvd causing clamd to stopfunctioning, but that is more a mirror problem than anything else... ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
On Fri, 2006-12-29 at 12:31 +, G.W. Haywood wrote: > Hi there, > > Some time ago somebody wrote, and somebody else replied: > > Why not just run freshclam as a daemon? > >>> Then you really need to have a daemon watcher to keep it going. > Naturally if freshclam dies we can expect people to mention it. I'm > calling for those who run freshclam as a daemon and who don't see any > problems with it to chip into this thread. How many of us are there? I've been running it for several years, in daemon mode, on several servers, some higher-volumne than others. The biggest site receives about 20,000 messages a day (after greylisting. It was around 6 a day before we implemented greylisting). I do 47 checks per day with one and 49 checks per days with another (nice odd numbers so that I never hit the mirrors at precisely the same time of day). These servers are running Mandriva linux, with amavisd-new calling clamd. I have never seen freshclam hung. I ran across a Red-Hat 8 box the other day that I was requested to audit. It was running clamav .75.1-1, with freshclam in daemon mode. The configuration was untouched since two years ago when it was installed. The database was up to date, too bad the engine wasn't ;-) -- Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX Austin Energy http://www.austinenergy.com ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
On Fri, 29 Dec 2006 12:31:29 + (GMT) "G.W. Haywood" <[EMAIL PROTECTED]> wrote: > Naturally if freshclam dies we can expect people to mention it. I'm > calling for those who run freshclam as a daemon and who don't see any > problems with it to chip into this thread. How many of us are there? I run it as a daemon on a server there that has been running RH9 for some years. I have never seen the daemon fail or hang, but a long time ago I did have some problems where it would fail to resolve domain names that turned out to be having nscd active as well as bind. Stopping nscd fixed that. In short, I have never seen anything failing that is attributable to freshclam itself, and it has been perfectly stable for well over 2 years now. -- Brian Morrison bdm at fenrir dot org dot uk "Arguing with an engineer is like wrestling with a pig in the mud; after a while you realize you are muddy and the pig is enjoying it." GnuPG key ID DE32E5C5 - http://wwwkeys.uk.pgp.net/pgpnet/wwwkeys.html ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Freshclam stability as a daemon [was: DB Update email before actual update available?]
G.W. Haywood wrote: > Hi there, > > Some time ago somebody wrote, and somebody else replied: > > Why not just run freshclam as a daemon? Then you really need to have a daemon watcher to keep it going. > > Talk of freshclam dying gives me some discomfort, yet in almost two > years running freshclam as a daemon on two - not particularly busy - > servers I've never seen it fail. It uses around a megabyte of memory > on a machine with 2G of RAM and, doing hourly updates, it takes maybe > three seconds of CPU per month on a 1GHz twin-processor Pentium box. > > Naturally if freshclam dies we can expect people to mention it. I'm > calling for those who run freshclam as a daemon and who don't see any > problems with it to chip into this thread. How many of us are there? Here's my 2 cents worth... SuSE Linux 10.1 (Intel, 32 bit). Latest freshclam and clamav. Both running as daemon at all times. Low volume site (maybe 100 mails/day, 5 users) Not a single problem, ever, although we've been running it for a few months only. We're using monit (and nagios) to monitor our (vital) processes, and we haven't had a single failure. /Lars > Here are the non-comment line in my config in case it has a bearing: > > DatabaseDirectory /var/lib/clamav > UpdateLogFile /tmp/.clam/freshclam.log > LogVerbose > LogSyslog > PidFile /var/run/clam/freshclam.pid > DNSDatabaseInfo current.cvd.clamav.net > DatabaseMirror db.uk.clamav.net > DatabaseMirror database.clamav.net > MaxAttempts 5 > Checks 24 > > Here's how I start it: > > /usr/local/bin/freshclam -d --daemon-notify=/etc/mail/clamav/clamd.conf > > Does anyone have any clues to the reasons behind freshclam's apparent > unreliability under some circumstances? Bad DB servers? Mail load? > Swap? Locking? Conflict with other processes? OS? Libraries? ... > > -- > > 73, > Ged. > ___ > Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net > http://lurker.clamav.net/list/clamav-users.html > ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
