Hi Mark, I'm not sure how you got a config with the default set to "ReceiveTimeout 30". I just tested with ubunte 20.04 a moment ago and a fresh `apt install clamav` (0.103.2). The config I found in /etc/clamav/freshclam.conf has "ReceiveTimeout 0" which means it is disabled. ClamAV's built-in default (if you don't specify) is also "0". So I'm not really sure what went wrong for you.
So we do have a minor problem with the mirrors.dat in 0.103.2. It will be owned by root instead of by the "clamav" user if you run "sudo freshclam --daemon". Then if you try running freshclam a different way, you may run into permissions issues. We'll have to fix this in the next patch version ☹. -Micah > -----Original Message----- > From: clamav-users <clamav-users-boun...@lists.clamav.net> On Behalf Of > clamav.mbou...@spamgourmet.com > Sent: Thursday, May 6, 2021 1:12 PM > To: clamav-users@lists.clamav.net > Subject: Re: [clamav-users] Update on rate limits and downloading > > Joel Esler jesler via clamav-users wrote: > > Overall — we’re doing much better. > > > > We’ve reduced the amount of bandwidth we’re serving by 4x, so we’ve > > made significant progress. > > > > /However, /we still have over 700 individual systems downloading the > > full daily.cvd over 200x a day. (This should be once a day, /if > > that/.) > > > > If you are not using 0.103.2 and it’s accompanying FreshClam to > > download these updates, and when you do create a NEW FreshClam.conf > > file and move your settings to that. We’re going to have to start > > blocking these atrocious abusers, as the rate limits are hurting > > everyone else at this point. > > I'm new to installing ClamAV, so there may be something I haven't done > quite right here. A couple of weeks ago, I installed ClamAV 0.103.2 from the > Ubuntu repositories (clamav, clamav-freshclam, clamav-daemon, clamav- > docs, clamtk and libclamunrar9 packages). > > By default, FreshClam seems to use too short a download timeout and retry > too frequently, triggering the rate limiting. After installing, the FreshClam > service would repeatedly attempt to download the daily.cvd file, time out > after 30 seconds, and wait 5 seconds before trying again. > After a few attempts, it then gets blocked by the CDN (if that's what "you are > on cool-down" in the log means?) for 4 hours. By the time I'd realised this > was happening following the initial install, I was already blocked. > > Perhaps this might, if left in a default configuration, be seen to attempt to > download daily.cvd over 100 times a day, but without ever actually getting > the whole file. From what I'd seen here and in documentation / FAQs, I > thought FreshClam was supposed to avoid retrying so frequently that it > triggers the rate limiting? > > I don't know if the default configuration is provided by ClamAV or the > Ubuntu packaging (either way, it seems FreshClam shouldn't just keep > retrying so quickly?) In my case, freshclam.conf originally had > "ReceiveTimeout 30". Increasing it to 60 wasn't enough. I then went to 600, > which was successful. Somewhere in between would probably have been > fine, but incrementing more gradually would have been a long process, > having to wait at least 4 hours between attempts (particularly as restarting > FreshClam after setting a new timeout seems to get blocked for a further 4 > hours - not just the remainder of the original block). > > In case it's of any use (and if this list allows it), I've attached my > freshclam.log > from those initial attempts. > > All seems to be working OK now, but posting here in case the information is > useful. > > > Please help us, stay diligent, keep going keep upgrading. Upgrade to > > 0.103.2, and keep your mirrors.dat file around, this file contains a > > snapshot of where you are in your update progression so that the next > > time that FreshClam run, it can start where it left off. > > Interesting you should mention mirrors.dat... Aside from the downloads > timing out, there are also some errors in my freshclam.log about not being > able to create mirrors.dat. That's a bit odd, since the /var/lib/clamav/ > directory is owned and writeable by the correct user, but the mirrors.dat file > within it is owned by root. Deleting that file and restarting the freshclam > service, the mirrors.dat file gets recreated, again owned by root. That error > hasn't appeared in the logs since, although mirrors.dat is still dated 25th > April, > so I'm not sure if there's still a problem with that. > > -- > Mark. _______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
