As long as you are using fresh clam, which handles incremental delta updates, and the instances are running correctly (not stuck in a loop or something silly), the best place to get updates is from database.clamav.net <http://database.clamav.net/>
> On Jan 27, 2021, at 5:45 AM, Vangelis Katsikaros via clamav-users > <clamav-users@lists.clamav.net> wrote: > > Thanks for the actions Joel, my IPs are now unblocked. > > I'd be interested to continue the discussion on > a) how to find the main & daily .inc and .cld files > b) why in ubuntu 18.04 there is a systemd condition for the clamav-daemon to > run, but the files are not available for download > I am afraid that in the long run my service could again DoS > database.clamav.net <http://database.clamav.net/> and it would be prudent for > me to proceed with the private mirror solution. > > Regards > Vangelis > > On Tue, Jan 26, 2021 at 7:20 PM Joel Esler (jesler) <jes...@cisco.com > <mailto:jes...@cisco.com>> wrote: > Let me remove the block for your IP and see what we see. > >> On Jan 26, 2021, at 12:17 PM, Vangelis Katsikaros <vkatsika...@gmail.com >> <mailto:vkatsika...@gmail.com>> wrote: >> >> Hi Joel, thanks for the quick response. We already download once every hour >> (the default ubuntu 18.04 behavior). However, we are using auto scaling and >> we might be running a large number of EC2 instances (a few hundreds), that >> could try to download simultaneously. To further scale without issues it >> seemed quite easy (and it was easy in practice) to use a private mirror. The >> system conditions and the cld & inc files is the only blocker to make this >> work :/ >> >> On Tue, Jan 26, 2021 at 7:00 PM Joel Esler (jesler) <jes...@cisco.com >> <mailto:jes...@cisco.com>> wrote: >> Why wouldn’t you just download less often? Instead of doing all of this? >> We publish updates once a day, there’s no need to download the entire >> package once a second. >> >> -- >> Joel Esler >> Manager, Communities Division >> Cisco Talos Intelligence Group >> http://www.talosintelligence.com <http://www.talosintelligence.com/> | >> https://www.snort.org <https://www.snort.org/> >> >>> On Jan 26, 2021, at 11:46 AM, Vangelis Katsikaros via clamav-users >>> <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>> >>> wrote: >>> >>> Hi >>> >>> I am using Ubuntu 18.04 and recently found out that downloaind clamav files >>> was blocked by cloudflare. Based on >>> https://lists.clamav.net/pipermail/clamav-users/2020-April/009482.html >>> <https://lists.clamav.net/pipermail/clamav-users/2020-April/009482.html> I >>> contacted Joel Esler (many thanks for the quick reply) and inform me that >>> we were downloading too often. >>> >>> I tried to fix the situation by introducing a private mirror, but I ran >>> into problems with cld and inc files and systemd conditions for the >>> clamav-daemon service. I'd like to ask what are the "cld" and "inc" files, >>> and how to download them in order to meet the systemd condition that >>> requires them to be on disk? >>> >>> I did the following steps: >>> 1) Created a private mirror as described in >>> https://www.clamav.net/documents/private-local-mirrors >>> <https://www.clamav.net/documents/private-local-mirrors> and mirrored all >>> "cvd" files. >>> >>> 2) Cloud-init ensures that on boot the files are downloaded from the >>> private mirror and are always available locally. Tthen cloud-init restarts >>> the clamav daemon. >>> >>> 3) Configured freshclam with >>> PrivateMirror https://my-clamav-mirror/clamav >>> <https://my-clamav-mirror/clamav> >>> ScriptedUpdates no >>> Checks 3 >>> >>> 4) Now freshclam works fine, however the clamav daemon does not, even if >>> the cvd are available locally. >>> $ sudo systemctl status clamav-daemon.service >>> ● clamav-daemon.service - Clam AntiVirus userspace daemon >>> Loaded: loaded (/lib/systemd/system/clamav-daemon.service; enabled; >>> vendor preset: enabled) >>> Drop-In: /etc/systemd/system/clamav-daemon.service.d >>> └─extend.conf >>> Active: inactive (dead) >>> Condition: start condition failed at Tue 2021-01-26 12:46:17 UTC; 3h 9min >>> ago >>> └─ ConditionPathExistsGlob=/var/lib/clamav/daily.{c[vl]d,inc} >>> was not met >>> Docs: man:clamd(8) >>> man:clamd.conf(5) >>> https://www.clamav.net/documents/ >>> <https://www.clamav.net/documents/> >>> >>> The systemd unit file has 2 ConditionPathExistsGlob that require the >>> additional files "main.cld", "main.inc", "daily.cld", "daily.inc" to be >>> available, for the service to start. >>> $ cat /lib/systemd/system/clamav-daemon.service >>> [Unit] >>> Description=Clam AntiVirus userspace daemon >>> Documentation=man:clamd(8) man:clamd.conf(5) >>> https://www.clamav.net/documents/ <https://www.clamav.net/documents/> >>> # Check for database existence >>> ConditionPathExistsGlob=/var/lib/clamav/main.{c[vl]d,inc} >>> ConditionPathExistsGlob=/var/lib/clamav/daily.{c[vl]d,inc} >>> >>> [Service] >>> ExecStart=/usr/sbin/clamd --foreground=true >>> # Reload the database >>> ExecReload=/bin/kill -USR2 $MAINPID >>> StandardOutput=syslog >>> TimeoutStartSec=420 >>> >>> [Install] >>> WantedBy=multi-user.target >>> >>> >>> 5) The "main.cld", "main.inc", "daily.cld", "daily.inc" files are not >>> available to download from the main download site >>> >>> $ curl --head -s https://database.clamav.net/daily.cld >>> <https://database.clamav.net/daily.cld> | grep HTTP >>> HTTP/2 404 >>> $ curl --head -s https://database.clamav.net/daily.inc >>> <https://database.clamav.net/daily.inc> | grep HTTP >>> HTTP/2 404 >>> $ curl --head -s https://database.clamav.net/main.cld >>> <https://database.clamav.net/main.cld> | grep HTTP >>> HTTP/2 404 >>> $ curl --head -s https://database.clamav.net/main.inc >>> <https://database.clamav.net/main.inc> | grep HTTP >>> HTTP/2 404 >>> >>> >>> 6) I ran the suggested clamdownloader.pl <http://clamdownloader.pl/> (again >>> from https://www.clamav.net/documents/private-local-mirrors >>> <https://www.clamav.net/documents/private-local-mirrors>) but I don't see >>> any >>> $ ls -al /tmp/clam >>> total 222848 >>> drwxrwxr-x 3 ubuntu ubuntu 4096 Jan 26 16:27 . >>> drwxrwxrwt 93 root root 12288 Jan 26 16:27 .. >>> -rw-rw-r-- 1 ubuntu ubuntu 296388 Sep 19 2019 bytecode.cvd >>> -rw-rw-r-- 1 ubuntu ubuntu 110005306 Jan 25 12:28 daily.cvd >>> -rw-rw-r-- 1 ubuntu ubuntu 42 Jan 26 16:27 dns.txt >>> -rw-rw-r-- 1 ubuntu ubuntu 117859675 Nov 25 2019 main.cvd >>> drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 26 16:27 temp >>> >>> but I don't see any "main.cld", "main.inc", "daily.cld", "daily.inc" files. >>> >>> Repeating the question in the beginning: I'd like to ask what are the "cld" >>> and "inc" files, and how to download in my private mirror in order to meet >>> the systemd condition that requires them to be on disk? >>> >>> Regards >>> Vangelis Katsikaros >>> >>> _______________________________________________ >>> >>> clamav-users mailing list >>> clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net> >>> https://lists.clamav.net/mailman/listinfo/clamav-users >>> <https://lists.clamav.net/mailman/listinfo/clamav-users> >>> >>> >>> Help us build a comprehensive ClamAV guide: >>> https://github.com/vrtadmin/clamav-faq >>> <https://github.com/vrtadmin/clamav-faq> >>> >>> http://www.clamav.net/contact.html#ml >>> <http://www.clamav.net/contact.html#ml> >> > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
