Chuck Swiger wrote: > On Apr 12, 2007, at 5:02 PM, Dennis Peterson wrote: >> Tomasz Kojm <[EMAIL PROTECTED]> wrote: >>> for 3rd party databases this can be managed with a simple script, >>> no need for >>> adding a keyring manager to ClamAV >> I swear it was just 8 weeks ago or so when we last had this discussion >> and all manner of fine ideas and scripts came out of it. Me thinks >> some >> folks need to crawl the archives. > > Would these fine scripts have prevented the problems a dozen or so > people (Luigi Iotti, Shane Wise, etc) reported in the thread "Clamav > suddenly died on several boxes"? Or would it be more accurate to say > that somebody's fine script or cron job actually failed to work as > well as one would hope it should...?
They worked fine here. All I got was a lot of errors from freshclam when the main.cvd file update caused clogging of the mirrors. About the same time I noticed that the MSRBL-Images file had grown to 11 meg and that it was creating problems for clamd (it's still a problem). The scripts check the SaneSecurity and MSRBL downloads to ensure there are no invalid records prior to installing them in the working directory. This is a data integrity check, not a validity check - they're not signed. It has been true for a very long time that clamd will stop if corrupt files are found in the working directory. This is true for files that are being copied from one directory to another - if clamd tries to load a file while it is partially copied, it will stop. So that makes it very important to do atomic operations and that too was discussed. Atomic file operations are performed by rsync - at least they are nearly so. When rsync transfers a file to a directory, rsync will copy to a hidden file name, and then rename that file (and overwrite an existing file) once the transfer has completed. Technically it is still possible for clamd to error if at the instant of the renaming clamd is reading the now replaced file. But the window is drastically reduced. This action can be replicated with file moves within a partition, and scripted copies to hidden files followed by a move to the final name. Operations that happen directly inside the working directory of clamd that can cause a corruption to occur are a bad idea, and apparently that was the case with the touch command that created a "*.cvd" file. Clamd will stop and your notification script should let you know. My clamd restart script first archives all the existing working directory contents to a different directory for later investigation, then the contents of the working directory are deleted, and copies of the last known working databases are copied into the working directory. Finally, clamd is restarted. A counter prevents this from looping, and only a single instance of the restart process can run so it won't step on itself. > > [ Note that I have never encountered a problem with clamd/freshclam > from 0.88.x, but 0.90.x seems to not be as reliable thus far, and > I'll probably hold off updating to it until a month or two goes by > without significant problems being reported to the list. YMMV. ] Clamd has behaved this way with bad databases for as long as I've been using it, and that is several years. I do recall being quite surprised the first time it happened, and so have been scripting restorative actions since. > > Secondly, my point about using some form of PKI or key management was > to address this comment by Tomasz: > > "There's no perfect solution to this problem. The only good one I > could think > of is an option to clamscan/clamd that would only allow loading of > digitally > signed databases and ignore all the rest. Of course, external dbs (sane, > msrbl, etc.) would no longer be supported in such a mode." > > From what I've seen, the SaneSecurity & MSRBL databases are released > as .ndb files, and not as the signed cvd/inc files. > > The key question is why-- do these third parties not have the ability > to release signed databases by using the "sigtool --server" option > and the "ClamAV Signing Service address"? If so, then using some > form of key management like GnuPG seems to be a reasonable solution, > as the individual admins can choose which signing keys they want > ClamAV to trust. Then you should talk to the providers of those files and see if they're interested in adding more work to what they're already doing for free. Right now it is easily possible to test those files for data integrity, and if you dl them by IP rather than hostname you may have heightened confidence in the source, but if too many burdens are placed on their services, those services will probably stop. As for responsibilities, it is not ClamAV's job to validate or integrity test 3rd party databases, and it would probably create liability problems for them if they suggested they did this and failed. So that can only mean it is your responsibility to do this once you've decided to use 3rd party databases. Testing the files is as simple as: /usr/local/bin/clamscan --quiet -d scam.ndb clam.txt where clam.txt is just a one-line text file to give clamscan something brief to do. If scam.ndb is corrupt this test will return an error. And just an fyi, be cautious of the MSRBL-Images file. Rechecking it while I was typing this shows that with it in place it will cause the clamd cpu to rise to 90% and stay there. At 11M it may be too big to be practical. dp _______________________________________________ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
