Hello, thank you.
I reported a bug : https://bugzilla.clamav.net/show_bug.cgi?id=12009 If needed I can provide more informations. Best regards On 18 January 2018 at 23:23, Tom Judge (tomjudge) <tomju...@cisco.com> wrote: > It sounds like at some point the freshclam process being used to populate > the private mirror failed to delete the CVD file after applying a CDIFF and > generating the CLD. > > The client in private mirror mode will look for both a CVD and a CLD and > download the file with the correct version number after doing a HEAD for > the metadata of the file. > > I'd start by deleting all the data from the mirror and re-initializing it > with freshclam. After 4 hours or so run freshclam again on the master to > make sure it downloads and apples the CDIFF correctly and generates the CLD > and removes the CLD. > > It's possible there is/was a permissions issue not allowing freshclam to > delete the original CVD. > > > Tom > > > On Jan 18, 2018, at 3:45 PM, Al Varnell <alvarn...@mac.com> wrote: > > > > As I mentioned before, I don't use a private mirror, so I've dropped out > of the conversation. Someone who does use option 2 needs to double-check > your settings here. > > > > -Al- > > > > On Thu, Jan 18, 2018 at 10:07 AM, Hugo Deprez wrote: > >> Hello, > >> > >> I don't think I'm the only one having this issue. It seems that I used > the > >> official configuration from http://www.clamav.net/doc/mirrors-faq.html > <http://www.clamav.net/doc/mirrors-faq.html> > >> Should I report the issue somewhere ? > >> > >> The only way I see to avoid this issue is to create a shell script and > to > >> push the updates my self on all servers, but this is not the proper way > to > >> do this... > >> > >> Hugo > >> > >> On 15 January 2018 at 11:44, Hugo Deprez <hugo.dep...@gmail.com > <mailto:hugo.dep...@gmail.com>> wrote: > >> > >>> Hello, > >>> > >>> by using this freshclam.conf : > >>> > >>> AllowSupplementaryGroups false > >>> Bytecode true > >>> Checks 24 > >>> CompressLocalDatabase no > >>> ConnectTimeout 30 > >>> DNSDatabaseInfo current.cvd.clamav.net <http://current.cvd.clamav.net/ > > > >>> DatabaseDirectory /var/lib/clamav > >>> DatabaseMirror mirror.recette.local > >>> DatabaseOwner clamav > >>> Debug false > >>> Foreground false > >>> LogFacility LOG_LOCAL6 > >>> LogFileMaxSize 0 > >>> LogRotate true > >>> LogSyslog true > >>> LogTime yes > >>> LogVerbose false > >>> MaxAttempts 5 > >>> PidFile /var/run/clamav/freshclam.pid > >>> #PrivateMirror mirror.recette.local > >>> ReceiveTimeout 30 > >>> ScriptedUpdates no > >>> TestDatabases yes > >>> UpdateLogFile /var/log/clamav/freshclam.log > >>> > >>> I got the old issue : > >>> > >>> root@server:/var/lib/clamav# rm -f * > >>> root@server:/var/lib/clamav# freshclam > >>> ClamAV update process started at Mon Jan 15 11:38:48 2018 > >>> Downloading main.cvd [100%] > >>> main.cvd updated (version: 58, sigs: 4566249, f-level: 60, builder: > sigmgr) > >>> Downloading daily.cvd [100%] > >>> daily.cvd updated (version: 24223, sigs: 1825336, f-level: 63, builder: > >>> neo) > >>> Downloading bytecode.cvd [100%] > >>> bytecode.cvd updated (version: 315, sigs: 75, f-level: 63, builder: > >>> raynman) > >>> Database updated (6391660 signatures) from mirror.recette.local (IP: > >>> 192.168.1.15) > >>> root@raadmin:/var/lib/clamav# freshclam > >>> ClamAV update process started at Mon Jan 15 11:40:00 2018 > >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, > builder: > >>> sigmgr) > >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, > >>> builder: neo) > >>> Downloading bytecode.cvd [100%] > >>> WARNING: Mirror 192.168.8.15 is not synchronized. > >>> Trying again in 5 secs... > >>> ClamAV update process started at Mon Jan 15 11:40:05 2018 > >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, > builder: > >>> sigmgr) > >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, > >>> builder: neo) > >>> WARNING: Can't download bytecode.cvd from mirror.recette.local > >>> Trying again in 5 secs... > >>> ClamAV update process started at Mon Jan 15 11:40:10 2018 > >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, > builder: > >>> sigmgr) > >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, > >>> builder: neo) > >>> WARNING: Can't download bytecode.cvd from mirror.recette.local > >>> Trying again in 5 secs... > >>> ClamAV update process started at Mon Jan 15 11:40:15 2018 > >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, > builder: > >>> sigmgr) > >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, > >>> builder: neo) > >>> WARNING: Can't download bytecode.cvd from mirror.recette.local > >>> Trying again in 5 secs... > >>> ClamAV update process started at Mon Jan 15 11:40:20 2018 > >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, > builder: > >>> sigmgr) > >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, > >>> builder: neo) > >>> ERROR: Can't download bytecode.cvd from mirror.recette.local > >>> Giving up on mirror.recette.local ... > >>> Update failed. Your network may be down or none of the mirrors listed > in > >>> /etc/clamav/freshclam.conf is working. Check > http://www.clamav.net/doc/ <http://www.clamav.net/doc/> > >>> mirrors-faq.html for possible reasons. > >>> > >>> > >>> > >>> On 14 January 2018 at 10:36, Al Varnell <alvarn...@mac.com <mailto: > alvarn...@mac.com>> wrote: > >>> > >>>> > >>>> On Sat, Jan 13, 2018 at 04:12 AM, Hugo Deprez wrote: > >>>>> > >>>>> hello, > >>>>> > >>>>> sorry I wasn't clear : > >>>>> > >>>>> - I used solution number 3 or more than one year, but freshclam on > >>>> clients > >>>>> was randomly reporting error on downloading cvd files. I never found > the > >>>>> solution : > >>>>> > >>>>> *Tue Dec 1 17:54:34 2015 -> WARNING: Can't download daily.cvd from * > >>>>> > >>>>> *mirror* > >>>>> > >>>>> - So now I switch to solution number 2 from > >>>>> https://www.clamav.net/documents/private-local-mirrors < > https://www.clamav.net/documents/private-local-mirrors> > >>>>> > >>>>> My "/etc/clamav/freshclam.conf" is : > >>>>> > >>>>> *AllowSupplementaryGroups falseBytecode trueChecks > >>>>> 24CompressLocalDatabase noConnectTimeout 30DNSDatabaseInfo > >>>>> current.cvd.clamav.net > >>>>> <http://current.cvd.clamav.net>DatabaseDirectory > >>>>> /var/lib/clamavDatabaseMirror mirror.recette.localDatabaseOwner > >>>>> clamavDebug falseForeground falseLogFacility LOG_LOCAL6LogFileMaxSize > >>>>> 0LogRotate trueLogSyslog trueLogTime yesLogVerbose falseMaxAttempts > >>>>> 5PidFile /var/run/clamav/freshclam.pidPrivateMirror > >>>>> mirror.recette.localReceiveTimeout 30ScriptedUpdates noTestDatabases > >>>>> yesUpdateLogFile /var/log/clamav/freshclam.log* > >>>>> > >>>>> But when I run freshclam, it still downloading the cld files : > >>>> > >>>> Because you changed PrivateMirror to mirror.recette.local. See > >>>> freshclam.conf: > >>>> > >>>> # This option allows you to easily point freshclam to private mirrors. > >>>> # If PrivateMirror is set, freshclam does not attempt to use DNS > >>>> # to determine whether its databases are out-of-date, instead it will > >>>> # use the If-Modified-Since request or directly check the headers of > the > >>>> # remote database files. For each database, freshclam first attempts > >>>> # to download the CLD file. If that fails, it tries to download the > >>>> # CVD file. This option overrides DatabaseMirror, DNSDatabaseInfo > >>>> # and ScriptedUpdates. It can be used multiple times to provide > >>>> # fall-back mirrors. > >>>> # Default: disabled > >>>> #PrivateMirror mirror1.mynetwork.com <http://mirror1.mynetwork.com/> > >>>> > >>>> Try commenting that line out. > >>>> > >>>>> *# freshclam* > >>>>> > >>>>> *ClamAV update process started at Sat Jan 13 13:07:47 2018Reading CVD > >>>>> header (main.cld): OK (IMS)Reading CVD header (main.cvd): OK > >>>>> (IMS)main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, > >>>>> builder: sigmgr)Reading CVD header (daily.cld): OKDownloading > >>>>> daily.cld [100%]daily.cld updated (version: 24217, sigs: 1823727, > >>>>> f-level: 63, builder: neo)Reading CVD header (bytecode.cld): > >>>>> OKbytecode.cvd is up to date (version: 319, sigs: 75, f-level: 63, > >>>>> builder: neo)[LibClamAV] Detected duplicate databases > >>>>> /var/lib/clamav/daily.cvd and /var/lib/clamav/daily.cld. The > >>>>> /var/lib/clamav/daily.cvd database is older and will not be loaded, > >>>>> you should manually remove it from the database directory.Database > >>>>> updated (6390051 signatures) from mirror.recette.local (IP: > >>>>> 192.168.1.15)*If I remove cld files from my private mirror : > >>>>> > >>>>> *# freshclamClamAV update process started at Sat Jan 13 13:10:18 > >>>>> 2018Downloading main.cld [100%]main.cld updated (version: 58, sigs: > >>>>> 4566249, f-level: 60, builder: sigmgr)Downloading daily.cld > >>>>> [100%]daily.cld updated (version: 24217, sigs: 1823727, f-level: 63, > >>>>> builder: neo)WARNING: getfile: bytecode.cld not found on > >>>>> mirror.recette.local (IP: 192.168.1.15)WARNING: Can't download > >>>>> bytecode.cld from mirror.recette.localDownloading bytecode.cvd > >>>>> [100%]bytecode.cvd updated (version: 319, sigs: 75, f-level: 63, > >>>>> builder: neo)Database updated (6390051 signatures) from > >>>>> mirror.recette.local (IP: 192.168.1.15)* > >>>>> > >>>>> I just want to run freshclam without any error. > >>>>> > >>>>> * Any idea ? * > >>>>> > >>>>> On 11 January 2018 at 10:37, Al Varnell <alvarn...@mac.com <mailto: > alvarn...@mac.com>> wrote: > >>>>> > >>>>>> On Thu, Jan 11, 2018 at 01:15 AM, Hugo Deprez wrote: > >>>>>>> Hello, > >>>>>>> > >>>>>>> thank you for the answer. > >>>>>>> I don't think my freshclam is trying to download the cdiff file > >>>> because I > >>>>>>> use : ScriptedUpdates no > >>>>>>> > >>>>>>> Is this parameter you are talking about incremental updates ? > >>>>>> > >>>>>> Yes, that's what I was referring to. > >>>>>> > >>>>>>> I know this is not bandwith efficient, but I had to many issues > with > >>>>>>> freshclam and my local repository made by clamdownloader.pl > >>>>>> > >>>>>> So you are using an option 3 private local mirror as described in < > >>>>>> https://www.clamav.net/documents/private-local-mirrors < > https://www.clamav.net/documents/private-local-mirrors>>. Make sure you > >>>>>> also change freshclam.conf "DatabaseMirror machine1.mylan" where > >>>>>> machine1.mylan is the name of your mirror server. > >>>>>> > >>>>>> Also make sure you are using the latest version of the > >>>> clamavdownloader.pl > >>>>>> <https://github.com/akissa/clamav-faq/blob/master/ < > https://github.com/akissa/clamav-faq/blob/master/> > >>>>>> mirrors/clamdownloader.pl> > >>>>>> > >>>>>> The existing clamdownloader.pl script does not have any error > >>>> correction > >>>>>> it simply bails out if a downloaded file is not valid and is unable > to > >>>>>> retry different mirrors if one fails. That is the most likely reason > >>>> for > >>>>>> those 404 errors. > >>>>>> > >>>>>> I have no experience with private local mirrors, but you might have > >>>> better > >>>>>> luck with the clamavmirror script <https://pypi.python.org/pypi/ < > https://pypi.python.org/pypi/> > >>>>>> clamavmirror/0.0.3>. > >>>>>> > >>>>>> -Al- > >>>>>> > >>>>>>> Best regards, > >>>>>>> > >>>>>>> On 10 January 2018 at 10:20, Al Varnell <alvarn...@mac.com > <mailto:alvarn...@mac.com> <mailto: > >>>>>> alvarn...@mac.com <mailto:alvarn...@mac.com>>> wrote: > >>>>>>> > >>>>>>>> The first time freshclam pulls down a daily.cdiff file, your > >>>> daily.cvd > >>>>>>>> file will be decompressed to daily.cld and the .cdiff file added > to > >>>> it. > >>>>>>>> From that point on you should only have the daily.cld file. One > >>>>>> exception > >>>>>>>> is that if for some reason freshclam is unable to find needed > .cdiff > >>>>>> files > >>>>>>>> on a mirror, it will download a new .cvd file which will start the > >>>>>> process > >>>>>>>> again. > >>>>>>>> > >>>>>>>> It is possible to reconfigure freshclam to download .cvd files by > >>>>>>>> disabling incremental updates, but that would be very inefficient > >>>> use of > >>>>>>>> bandwidth and mirror server time, so unless you have a overriding > >>>> need > >>>>>> for > >>>>>>>> that, I don't recommend it. > >>>>>>>> > >>>>>>>> -Al- > >>>>>>>> > >>>>>>>> On Wed, Jan 10, 2018 at 01:12 AM, Hugo Deprez wrote: > >>>>>>>>> Hello, > >>>>>>>>> > >>>>>>>>> I have a question about daily.cvd and daily.cld files. If I > >>>> understood > >>>>>>>>> correctly, those two files are almost the same : one is > compressed, > >>>> the > >>>>>>>>> other is not. > >>>>>>>>> Still I have an issue in my setup : > >>>>>>>>> > >>>>>>>>> If I put both filtes in apache2 server (which act as > PrivateMirror) > >>>> : > >>>>>>>>> > >>>>>>>>> -rw-r--r-- 1 www-data www-data 117892267 07.06.2017 23:38 > main.cvd > >>>>>>>>> -rw-r--r-- 1 www-data www-data 153228 07.12.2017 03:17 > >>>> bytecode.cvd > >>>>>>>>> lrwxrwxrwx 1 root root 8 05.01.2018 10:14 > main.cld > >>>> -> > >>>>>>>>> main.cvd > >>>>>>>>> lrwxrwxrwx 1 root root 9 05.01.2018 10:14 > daily.cld > >>>> -> > >>>>>>>>> daily.cvd > >>>>>>>>> lrwxrwxrwx 1 root root 12 05.01.2018 10:14 > >>>> bytecode.cld > >>>>>> -> > >>>>>>>>> bytecode.cvd > >>>>>>>>> -rw-r--r-- 1 www-data www-data 43804052 10.01.2018 06:17 > daily.cvd > >>>>>>>>> > >>>>>>>>> On the client side I have this Warning : > >>>>>>>>> > >>>>>>>>> LibClamAV Warning: Detected duplicate databases > >>>>>> /var/lib/clamav/daily.cvd > >>>>>>>>> and /var/lib/clamav/daily.cld, please manually remove one of them > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> But If I remove the *.cld files on my PrivateMirror I got > multiples > >>>> 404 > >>>>>>>>> errors from the Freshclam clients. > >>>>>>>>> > >>>>>>>>> Is there a way to configure Freshclam in order to grabe only the > cvd > >>>>>>>> files > >>>>>>>>> ? > >>>>>>>>> > >>>>>>>>> Best regards, > > _______________________________________________ > > clamav-users mailing list > > clamav-users@lists.clamav.net > > http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users > > > > > > Help us build a comprehensive ClamAV guide: > > https://github.com/vrtadmin/clamav-faq > > > > http://www.clamav.net/contact.html#ml > > > _______________________________________________ > clamav-users mailing list > clamav-users@lists.clamav.net > http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml > > _______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
