This issue was resolved once I analyzed the 'behavior' of the download method,
and the person performing the 'testing' and 'debugging' of the process ("me")...
The nightly download script, performs a WGET from "db.local.clamav.net" which I
assumed would update (only if there were a newer version of the file) the
individual DB files ('daily.cvd' being the most common one).
When the scheduled job executed, there was always an existing file to
overwrite, but when the 'tester' was debugging the process (to identify why the
downloaded file was 'corrupt') he would move the exiting file aside, to retain
it for comparison to the one manually downloaded, such that there wasn't a
file to 'overwrite'.
Once this difference was isolated, the WGET option of "--continue" became the
obvious suspect. Executing the 'strings' command on both the 'scheduled' file
and the manually downloaded version showed that the embedded 'date' values were
different, with the 'corrupt' version showing the date from a previous day (or
days). Its apparent that the "--continue" option to WGET attempts to 'append'
to the existing file, rather than replace a completed version.
The "--continue" option has since been removed from the command string within
the script, and every 'download' has since been successful...
--- On Wed, 8/8/12, Steve Brazill <yu...@sbcglobal.net> wrote:
From: Steve Brazill <yu...@sbcglobal.net>
Subject: Re: [clamav-users] Corrupt ClamAV virus DB files
To: "ClamAV users ML" <clamav-users@lists.clamav.net>
Date: Wednesday, August 8, 2012, 2:22 PM
> --- On Wed, 8/8/12, Al Varnell <alvarn...@mac.com> wrote:
>
> From: Al Varnell <alvarn...@mac.com>
> Subject: Re: [clamav-users] Corrupt ClamAV virus DB files
> To: "ClamAV users ML" <clamav-users@lists.clamav.net>
> Date: Wednesday, August 8, 2012, 1:18 PM
>
> On Aug 8, 2012, at 12:59 PM, Steve Brazill <yu...@sbcglobal.net> wrote:
>
> > Our firm has a scheduled replication of the ClamAV database files, which mor
e often than not, appear to be corrupt upon receipt.
> >
> > Though this error message has been posted by others previously, I have not
seen a definitive answer/solution, and is usually discounted as an issue with 'b
ad memory'.
> >
> > clamscan:
> > LibClamAV Error: Can't load /var/clamav/daily.cvd: Can't verify database i
ntegrity
> > ERROR: Can't verify database integrity
> > ERROR: Can't verify database integrity
> >
> > The most recent example occurred today, with the daily version 15231 (3:45AM
aprox) being corrupt, but the subsequent release of 15232 (9AM aprox) being su
ccessful.
> >
> > The replication process, which only retrieves updated files, is obtaining th
e daily file from:
> > http://db.local.clamav.net/daily.cvd
> >
> > Are the multiple releases of the daily file due to actual updates to virus i
nstances, or identification of corrupt source files on the website ?
>
> There are normally several incremental updates daily.
>
> Since there are over 140 mirror servers world-wide and you would normally be r
otating to a different one of maybe half a dozen servers in your area, it could
be that the two databases came from different servers. It's important to identif
y the IP address of any server that is consistently corrupt.
>
>
> Sent from Janet's iPad
>
> -Al-
> --
> Al Varnell
> Mountain View, CA, USA
> _______________________________________________
> Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
> http://www.clamav.net/support/ml
That's an excellent point...
Looking at our SQUID proxy access logs, I have found the following:
194.47.250.218 soyuz.df.lth.se - did not work
78.46.84.244 mx02.akxnet.de - did not work
69.163.100.14 clamav.theshell.com - worked
168.143.19.95 clamav-du.viaverio.com - did not work
209.198.147.20 clamav.io-tx.com - worked
At first glance, it appears that 'non-Americas' servers have been selected from
the mirror pool which resulted in non-functioning DB files, except for the
instance using "clamav-du" which I cannot identify its geographic location.
I'll monitor the next couple of replications, to see if there is a pattern...
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://www.clamav.net/support/ml
