The bytecode.cvd file is the original. When there is an update, we publish two things:
1. a bytecode.cdiff patch file that will update the older bytecode.cvd to the newest version. This is the "scripted update" mechanism. If using the .cdiff patch file to update, it should replace the old bytecode.cvd with a new bytecode.cld. We may issue an empty patch file (zero-bytes) to tell freshclam to download the whole bytecode.cvd instead. We do this if the patch is so big it is better to just download the whole file, or if is a bug preventing the patch file from working correctly, which there presently is for bytecode signatures (sad!). This .cdiff update mechanism would not be used in your situation because ScriptedUpdates is disabled. 2. a new bytecode.cvd. This should only be downloaded in two cases: A) If you do not have the old bytecode.cvd (or cld) and thus cannot use the patch file to update. And B) If the bytecode.cdiff patch file is empty. The issue you're facing feels to me like an issue with what the private mirror is serving. Can you please check if it is serving an empty bytecode.cvd? It feels like it may be serving both the empty bytecode.cvd and a bytecode.cld. If that's not the case, then we may have a bug in freshclam and I would love some more information on what freshclam is downloading when it runs in order to get into this strange state. Best, Micah Micah Snyder ClamAV Development Talos Cisco Systems, Inc. ________________________________ From: clamav-users <clamav-users-boun...@lists.clamav.net> on behalf of Kevin O'Connor via clamav-users <clamav-users@lists.clamav.net> Sent: Monday, February 27, 2023 11:12 AM To: ClamAV users ML <clamav-users@lists.clamav.net> Cc: Kevin O'Connor <kocon...@ampion.net> Subject: Re: [clamav-users] 0 length bytecode.cvd causing problems with clamav daemon Marc, I had a similar understanding of that document. That is; if there is no bytecode.cvd pushed by the ClamAV team, it should not exist on my local scanners. When I checked the mirror and there was no bytecode.cvd file, yet it appeared on my scanner machines with 0 length, I figured that the new release had highlighted a misconfiguration in my freshclam.conf that the earlier version was more forgiving of. However I have not found what that might be. Your idea of removing all the files in the /var/lib/clamav directory is what I found worked initially, but that seems like a poor workaround as I need this running all the time. I don't know when our clients will drop files on us that need a scan. Thanks for looking at it. Kevin On Mon, Feb 27, 2023 at 1:11 PM Marc via clamav-users <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> wrote: i would suggest, to delete alle libraries in /var/lib/clamav and download all complete new. CLD Files comes not regularly, normally we have CVD only. If i understand this well, CLD Files comes only when error occures while updating. https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html Von / From: Kevin O'connor <mailto:kocon...@ampion.net<mailto:kocon...@ampion.net>> An / To: Newcomer01 <mailto:newcome...@posteo.de<mailto:newcome...@posteo.de>> Gesendet / Sent: Montag, Februar 27, 2023 um 18:38 (at 06:38 PM) +0100 Betreff / Subject: Re: [clamav-users] 0 length bytecode.cvd causing problems with clamav daemon > Heh, good question. Just checked again, and it looks like that was a > copy-paste error. There is only one PrivateMirror line. > Kevin > > On Mon, Feb 27, 2023 at 12:02 PM newcomer01 via clamav-users > <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> wrote: > > why you have set two times the "PrivateMirror" with identically IP's? > Can't believe that this happens with the automated PostInst 😉 > > > Von / From: Clamav User Mailinglist > <mailto:clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> > An / To: Newcomer01 <mailto:newcome...@posteo.de<mailto:newcome...@posteo.de>> > CC / CC: Kevin O'connor > <mailto:kocon...@ampion.net<mailto:kocon...@ampion.net>> > Gesendet / Sent: Montag, Februar 27, 2023 um 16:58 (at 04:58 PM) +0100 > Betreff / Subject: [clamav-users] 0 length bytecode.cvd causing problems with > clamav daemon > > I am having an issue with 0 length bytecode.cvd files on my scanner > > instances. This seems to have started sometime on 22 Feb, I'm afraid I > > don't have an exact time. The clamav daemon produces logs like the > > following: > > > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: cli_cvdverify: > > Can't read CVD header > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: Can't load > > /var/lib/clamav/bytecode.cld: Broken or not a CVD file > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: > > cli_loaddbdir(): error loading database /var/lib/clamav/bytecode.cld > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: Mon Feb 27 14:39:11 2023 -> > > !Broken or not a CVD file > > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Main > > process exited, code=exited, status=1/FAILURE > > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Failed with > > result 'exit-code'. > > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Consumed > > 8.679s CPU time. > > > > > > I feel like I have narrowed the problem down to a 0 length 'bytecode.cvd' > > file. Here is a listing of the definitions directory: > > > > $ ls -l /var/lib/clamav > > total 226168 > > -rw-r--r-- 1 clamav clamav 314802 Feb 27 14:06 bytecode.cld > > -rw-r--r-- 1 clamav clamav 0 Feb 27 02:00 bytecode.cvd > > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 10:01 daily.cld > > -rw-r--r-- 1 clamav clamav 69 Feb 23 15:33 freshclam.dat > > -rw-r--r-- 1 clamav clamav 170479789 Feb 27 02:00 main.cvd > > > > > > My initial fix (before narrowing the problem down to bytecode.cvd) was to > > > > 1. stop freshclam > > 2. clean this directory > > 3. restart freshclam > > 4. give it time to get the definitions (from a private mirror) > > 5. start clamav daemon > > > > This would work for maybe 1/2 day then the empty bytecode.cvd file would > > reappear and the daemon would fail. > > > > This morning I was able to spend some more time and find that it was just > > the one file that needed to be removed. > > > > I have a local mirror because there are several instances of this scanner > > in use (at least 2 instances for several environments). I have checked the > > mirror and it appears to be working fine and keeping the definitions up to > > date inside our environment. In addition, the scanner instances appear to > > be keeping the local set of definitions up to date with the mirror. > > > > The mirror does not have a bytecode.cvd file on it (here is a listing of > > its definitions directory) > > > > $ ls -l /var/lib/clamav > > total 226172 > > -rw-r--r-- 1 clamav clamav 314802 Feb 22 22:02 bytecode.cld > > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 09:06 daily.cld > > -rw-r--r-- 1 clamav clamav 69 Jan 29 2022 freshclam.dat > > -rw-r--r-- 1 clamav clamav 170479789 Jan 29 2022 main.cvd > > -rw-r--r-- 1 clamav clamav 87 Jan 29 2022 test.html > > > > > > To the best of my knowledge, the software is up to date: > > > > $ sudo freshclam -V > > ClamAV 0.103.8/26825/Mon Feb 27 08:24:38 2023 > > > > > > Here is the freshclam.conf used on all the local sanner instances > > > > $ cat /etc/clamav/freshclam.conf > > # Automatically created by the clamav-freshclam postinst > > # Comments will get lost when you reconfigure the clamav-freshclam package > > > > DatabaseOwner clamav > > UpdateLogFile /var/log/clamav/freshclam.log > > LogVerbose false > > LogSyslog false > > LogFacility LOG_LOCAL6 > > LogFileMaxSize 0 > > LogRotate true > > LogTime true > > Foreground false > > Debug false > > MaxAttempts 5 > > DatabaseDirectory /var/lib/clamav > > DNSDatabaseInfo current.cvd.clamav.net<http://current.cvd.clamav.net> > > <http://current.cvd.clamav.net> <http://current.cvd.clamav.net> > > ConnectTimeout 30 > > ReceiveTimeout 0 > > TestDatabases yes > > CompressLocalDatabase no > > Bytecode true > > NotifyClamd /etc/clamav/clamd.conf > > # Check for new database 24 times a day > > Checks 24 > > PrivateMirror http://10.50.0.2 > > ScriptedUpdates no > > PrivateMirror http://10.50.0.2 > > > > > > The scanner has been working fine for about 12 months, keeping the software > > and the definitions up to date. The only configuration item that seems to > > relate is "Bytecode true", but the description seems to discuss just the > > downloading of the file, not whether it is created on the local instance. > > > > Does anyone have any pointers? > > > > Thanks > > Kevin > > -- > > > > *Kevin O'Connor* > > Principal DevOps Engineer > > M: 617-834-1291 > > > > email-footer-logos.jpg (1000×120) > > > > STATEMENT OF CONFIDENTIALITY: The information contained in this message and > > any attachments are intended solely for the addressee(s) and may contain > > confidential or privileged information. If you are not the intended > > recipient, or responsible for delivering the e-mail to the intended > > recipient, you have received this message in error. Any use, dissemination, > > forwarding, printing, or copying is strictly prohibited. Please notify > > Ampion immediately at secur...@ampion.net<mailto:secur...@ampion.net> and > > destroy all copies of this message and any attachments. > > > > > > _______________________________________________ > > > > Manage your clamav-users mailing list subscription / unsubscribe: > > https://lists.clamav.net/mailman/listinfo/clamav-users > > > > > > Help us build a comprehensive ClamAV guide: > > https://github.com/Cisco-Talos/clamav-documentation > > > > https://docs.clamav.net/#mailing-lists-and-chat > > _______________________________________________ > > Manage your clamav-users mailing list subscription / unsubscribe: > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/Cisco-Talos/clamav-documentation > > https://docs.clamav.net/#mailing-lists-and-chat > _______________________________________________ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat
_______________________________________________ Manage your clamav-users mailing list subscription / unsubscribe: https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/Cisco-Talos/clamav-documentation https://docs.clamav.net/#mailing-lists-and-chat
