Let me remove the block for your IP and see what we see. > On Jan 26, 2021, at 12:17 PM, Vangelis Katsikaros <[email protected]> > wrote: > > Hi Joel, thanks for the quick response. We already download once every hour > (the default ubuntu 18.04 behavior). However, we are using auto scaling and > we might be running a large number of EC2 instances (a few hundreds), that > could try to download simultaneously. To further scale without issues it > seemed quite easy (and it was easy in practice) to use a private mirror. The > system conditions and the cld & inc files is the only blocker to make this > work :/ > > On Tue, Jan 26, 2021 at 7:00 PM Joel Esler (jesler) <[email protected] > <mailto:[email protected]>> wrote: > Why wouldn’t you just download less often? Instead of doing all of this? We > publish updates once a day, there’s no need to download the entire package > once a second. > > -- > Joel Esler > Manager, Communities Division > Cisco Talos Intelligence Group > http://www.talosintelligence.com <http://www.talosintelligence.com/> | > https://www.snort.org <https://www.snort.org/> > >> On Jan 26, 2021, at 11:46 AM, Vangelis Katsikaros via clamav-users >> <[email protected] <mailto:[email protected]>> wrote: >> >> Hi >> >> I am using Ubuntu 18.04 and recently found out that downloaind clamav files >> was blocked by cloudflare. Based on >> https://lists.clamav.net/pipermail/clamav-users/2020-April/009482.html >> <https://lists.clamav.net/pipermail/clamav-users/2020-April/009482.html> I >> contacted Joel Esler (many thanks for the quick reply) and inform me that we >> were downloading too often. >> >> I tried to fix the situation by introducing a private mirror, but I ran into >> problems with cld and inc files and systemd conditions for the clamav-daemon >> service. I'd like to ask what are the "cld" and "inc" files, and how to >> download them in order to meet the systemd condition that requires them to >> be on disk? >> >> I did the following steps: >> 1) Created a private mirror as described in >> https://www.clamav.net/documents/private-local-mirrors >> <https://www.clamav.net/documents/private-local-mirrors> and mirrored all >> "cvd" files. >> >> 2) Cloud-init ensures that on boot the files are downloaded from the private >> mirror and are always available locally. Tthen cloud-init restarts the >> clamav daemon. >> >> 3) Configured freshclam with >> PrivateMirror https://my-clamav-mirror/clamav >> <https://my-clamav-mirror/clamav> >> ScriptedUpdates no >> Checks 3 >> >> 4) Now freshclam works fine, however the clamav daemon does not, even if the >> cvd are available locally. >> $ sudo systemctl status clamav-daemon.service >> ● clamav-daemon.service - Clam AntiVirus userspace daemon >> Loaded: loaded (/lib/systemd/system/clamav-daemon.service; enabled; >> vendor preset: enabled) >> Drop-In: /etc/systemd/system/clamav-daemon.service.d >> └─extend.conf >> Active: inactive (dead) >> Condition: start condition failed at Tue 2021-01-26 12:46:17 UTC; 3h 9min ago >> └─ ConditionPathExistsGlob=/var/lib/clamav/daily.{c[vl]d,inc} was >> not met >> Docs: man:clamd(8) >> man:clamd.conf(5) >> https://www.clamav.net/documents/ >> <https://www.clamav.net/documents/> >> >> The systemd unit file has 2 ConditionPathExistsGlob that require the >> additional files "main.cld", "main.inc", "daily.cld", "daily.inc" to be >> available, for the service to start. >> $ cat /lib/systemd/system/clamav-daemon.service >> [Unit] >> Description=Clam AntiVirus userspace daemon >> Documentation=man:clamd(8) man:clamd.conf(5) >> https://www.clamav.net/documents/ <https://www.clamav.net/documents/> >> # Check for database existence >> ConditionPathExistsGlob=/var/lib/clamav/main.{c[vl]d,inc} >> ConditionPathExistsGlob=/var/lib/clamav/daily.{c[vl]d,inc} >> >> [Service] >> ExecStart=/usr/sbin/clamd --foreground=true >> # Reload the database >> ExecReload=/bin/kill -USR2 $MAINPID >> StandardOutput=syslog >> TimeoutStartSec=420 >> >> [Install] >> WantedBy=multi-user.target >> >> >> 5) The "main.cld", "main.inc", "daily.cld", "daily.inc" files are not >> available to download from the main download site >> >> $ curl --head -s https://database.clamav.net/daily.cld >> <https://database.clamav.net/daily.cld> | grep HTTP >> HTTP/2 404 >> $ curl --head -s https://database.clamav.net/daily.inc >> <https://database.clamav.net/daily.inc> | grep HTTP >> HTTP/2 404 >> $ curl --head -s https://database.clamav.net/main.cld >> <https://database.clamav.net/main.cld> | grep HTTP >> HTTP/2 404 >> $ curl --head -s https://database.clamav.net/main.inc >> <https://database.clamav.net/main.inc> | grep HTTP >> HTTP/2 404 >> >> >> 6) I ran the suggested clamdownloader.pl <http://clamdownloader.pl/> (again >> from https://www.clamav.net/documents/private-local-mirrors >> <https://www.clamav.net/documents/private-local-mirrors>) but I don't see >> any >> $ ls -al /tmp/clam >> total 222848 >> drwxrwxr-x 3 ubuntu ubuntu 4096 Jan 26 16:27 . >> drwxrwxrwt 93 root root 12288 Jan 26 16:27 .. >> -rw-rw-r-- 1 ubuntu ubuntu 296388 Sep 19 2019 bytecode.cvd >> -rw-rw-r-- 1 ubuntu ubuntu 110005306 Jan 25 12:28 daily.cvd >> -rw-rw-r-- 1 ubuntu ubuntu 42 Jan 26 16:27 dns.txt >> -rw-rw-r-- 1 ubuntu ubuntu 117859675 Nov 25 2019 main.cvd >> drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 26 16:27 temp >> >> but I don't see any "main.cld", "main.inc", "daily.cld", "daily.inc" files. >> >> Repeating the question in the beginning: I'd like to ask what are the "cld" >> and "inc" files, and how to download in my private mirror in order to meet >> the systemd condition that requires them to be on disk? >> >> Regards >> Vangelis Katsikaros >> >> _______________________________________________ >> >> clamav-users mailing list >> [email protected] <mailto:[email protected]> >> https://lists.clamav.net/mailman/listinfo/clamav-users >> <https://lists.clamav.net/mailman/listinfo/clamav-users> >> >> >> Help us build a comprehensive ClamAV guide: >> https://github.com/vrtadmin/clamav-faq >> <https://github.com/vrtadmin/clamav-faq> >> >> http://www.clamav.net/contact.html#ml <http://www.clamav.net/contact.html#ml> >
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ clamav-users mailing list [email protected] https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
