Re: [clamav-users] ClamAV signature update sync errors have gotten worse
On Aug 21, 2018, at 12:32 PM, G.W. Haywood mailto:cla...@jubileegroup.co.uk>> wrote: Hi there, On Tue, 21 Aug 2018, Joel Esler wrote: The amount of people using ClamAV version 0.90 and below is surprising as well. That's not really surprising to me. Most of them probably don't even know that they're running it, and those who do could easily be lying as it's trivial to forge a User-Agent string. Especially given what's happened in the past to users of old versions, if there is any surprise it's that you're still serving files to them. In my view it would be perfectly reasonable to block them. It might even save you some money. We have blocked people that are 0.80 and below, to see if anyone brings it up (to which, I think this list would violently react with something akin to "You are running 13 year old AV?"). No one has, publicly or privately. We'll probably proceed with a blog post stating that we're blocking everyone below the version that introduced diff'ing (0.93.3). Also rate limiting people that are attempting to download the main.cvd every 1 minute has helped. The good news is, the top ten successful download versions (by User-Agent) are within the last 4 or 5 releases. (0.99.4 is our largest deployed version, followed by 0.100.1, for those of you that are curious) -- Joel Esler Sr. Manager Community, Branding, and Open Source Talos Group http://www.talosintelligence.com ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ClamAV signature update sync errors have gotten worse
Hi there, On Tue, 21 Aug 2018, Joel Esler wrote: The amount of people using ClamAV version 0.90 and below is surprising as well. That's not really surprising to me. Most of them probably don't even know that they're running it, and those who do could easily be lying as it's trivial to forge a User-Agent string. Especially given what's happened in the past to users of old versions, if there is any surprise it's that you're still serving files to them. In my view it would be perfectly reasonable to block them. It might even save you some money. -- 73, Ged. ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ClamAV signature update sync errors have gotten worse
CC'ing your comments over to Micah. We have a heavy freshclam rewrite in the pipeline. The amount of people using ClamAV version 0.90 and below is surprising as well. None of those versions support .diff files on the daily file. So, those versions are downloading the whole daily.cvd (sometimes hundreds of times a day) even though those versions of ClamAV won't work with the current options in daily.cvd, so they can't even start up. But Freshclam is still updating! > On Aug 20, 2018, at 10:25 PM, Paul Kosinski wrote: > > It's good to save so much (5 PB) Internet traffic. > > What we were seeing from our end was that there were a lot of full-size > downloads of daily.cvd that were useless because they were the old > version rather than the new version advertised by the DNS TXT record. > > Besides being annoying because of lots of extra logging by freshclam, > it kept killing off the mirror IP addresses due to the update failures, > and thus eventually blocked all downloads. > > Since we already had a wrapper around freshclam to do some extra stuff > in our environment, I decided to write the extra code to only invoke > freshclam if the prefix of the cvd file(s) showed the correct version. > After that, it was easy to log the delay to separate file. > > I guess my question at this point is: how many other users of freshclam > are seeing the problem we had? The behavior we were seeing not only > wasted bandwidth, it also caused semi-permanent blockage of future > updates. Users who don't monitor their logs (like many desktop users?) > could be far out of date with their ClamAV signatures. > > P.S. It shouldn't be too hard to modify freshclam itself to deal with > this problem in a similar fashion. But I didn't want to fork a fairly > complicated program which mainly does stuff that has nothing to do with > this particular problem. > > > > On Mon, 20 Aug 2018 15:43:14 + > "Joel Esler (jesler)" wrote: > >> Thank you. We have to make adjustments very slowly to not disrupt >> anyone. >> >> Cloudflare has helped us save 2 PB in the last month, delivering >> updates an average of 39% faster. We are seeing excellent results. >> >>> On Aug 18, 2018, at 1:09 AM, Paul Kosinski >>> wrote: >>> >>> Joel, >>> >>> Still lots of delays since "2018-08-11 13:18:02 No delay", but none >>> quite as long as the previous batch: >>> >>> 2018-08-11 21:33:02 00:15:00 delay >>> 2018-08-12 05:48:02 01:00:00 delay >>> 2018-08-12 14:33:01 01:15:00 delay >>> 2018-08-12 22:48:02 01:00:00 delay >>> 2018-08-13 05:18:01 No delay >>> 2018-08-13 13:18:02 No delay >>> 2018-08-13 21:33:01 00:14:59 delay >>> 2018-08-14 05:18:01 No delay >>> 2018-08-14 13:18:02 No delay >>> 2018-08-14 21:33:02 00:30:01 delay >>> 2018-08-15 05:03:02 No delay >>> 2018-08-15 13:48:01 00:45:00 delay >>> 2018-08-15 22:03:01 No delay >>> 2018-08-16 05:03:02 No delay >>> 2018-08-16 14:03:02 01:00:01 delay >>> 2018-08-16 21:18:01 00:14:59 delay >>> 2018-08-17 06:03:01 No delay >>> 2018-08-17 13:33:02 00:30:01 delay >>> 2018-08-17 21:03:02 No delay >>> >>> >>> On Thu, 16 Aug 2018 22:13:48 + >>> "Joel Esler (jesler)" wrote: >>> Paul, how are things looking from your side? > >> > ___ > clamav-users mailing list > clamav-users@lists.clamav.net > http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ClamAV signature update sync errors have gotten worse
It's good to save so much (5 PB) Internet traffic. What we were seeing from our end was that there were a lot of full-size downloads of daily.cvd that were useless because they were the old version rather than the new version advertised by the DNS TXT record. Besides being annoying because of lots of extra logging by freshclam, it kept killing off the mirror IP addresses due to the update failures, and thus eventually blocked all downloads. Since we already had a wrapper around freshclam to do some extra stuff in our environment, I decided to write the extra code to only invoke freshclam if the prefix of the cvd file(s) showed the correct version. After that, it was easy to log the delay to separate file. I guess my question at this point is: how many other users of freshclam are seeing the problem we had? The behavior we were seeing not only wasted bandwidth, it also caused semi-permanent blockage of future updates. Users who don't monitor their logs (like many desktop users?) could be far out of date with their ClamAV signatures. P.S. It shouldn't be too hard to modify freshclam itself to deal with this problem in a similar fashion. But I didn't want to fork a fairly complicated program which mainly does stuff that has nothing to do with this particular problem. On Mon, 20 Aug 2018 15:43:14 + "Joel Esler (jesler)" wrote: > Thank you. We have to make adjustments very slowly to not disrupt > anyone. > > Cloudflare has helped us save 2 PB in the last month, delivering > updates an average of 39% faster. We are seeing excellent results. > > > On Aug 18, 2018, at 1:09 AM, Paul Kosinski > > wrote: > > > > Joel, > > > > Still lots of delays since "2018-08-11 13:18:02 No delay", but none > > quite as long as the previous batch: > > > > 2018-08-11 21:33:02 00:15:00 delay > > 2018-08-12 05:48:02 01:00:00 delay > > 2018-08-12 14:33:01 01:15:00 delay > > 2018-08-12 22:48:02 01:00:00 delay > > 2018-08-13 05:18:01 No delay > > 2018-08-13 13:18:02 No delay > > 2018-08-13 21:33:01 00:14:59 delay > > 2018-08-14 05:18:01 No delay > > 2018-08-14 13:18:02 No delay > > 2018-08-14 21:33:02 00:30:01 delay > > 2018-08-15 05:03:02 No delay > > 2018-08-15 13:48:01 00:45:00 delay > > 2018-08-15 22:03:01 No delay > > 2018-08-16 05:03:02 No delay > > 2018-08-16 14:03:02 01:00:01 delay > > 2018-08-16 21:18:01 00:14:59 delay > > 2018-08-17 06:03:01 No delay > > 2018-08-17 13:33:02 00:30:01 delay > > 2018-08-17 21:03:02 No delay > > > > > > On Thu, 16 Aug 2018 22:13:48 + > > "Joel Esler (jesler)" wrote: > > > >> Paul, how are things looking from your side? > ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ClamAV signature update sync errors have gotten worse
Thank you. We have to make adjustments very slowly to not disrupt anyone. Cloudflare has helped us save 2 PB in the last month, delivering updates an average of 39% faster. We are seeing excellent results. > On Aug 18, 2018, at 1:09 AM, Paul Kosinski wrote: > > Joel, > > Still lots of delays since "2018-08-11 13:18:02 No delay", but none > quite as long as the previous batch: > > 2018-08-11 21:33:02 00:15:00 delay > 2018-08-12 05:48:02 01:00:00 delay > 2018-08-12 14:33:01 01:15:00 delay > 2018-08-12 22:48:02 01:00:00 delay > 2018-08-13 05:18:01 No delay > 2018-08-13 13:18:02 No delay > 2018-08-13 21:33:01 00:14:59 delay > 2018-08-14 05:18:01 No delay > 2018-08-14 13:18:02 No delay > 2018-08-14 21:33:02 00:30:01 delay > 2018-08-15 05:03:02 No delay > 2018-08-15 13:48:01 00:45:00 delay > 2018-08-15 22:03:01 No delay > 2018-08-16 05:03:02 No delay > 2018-08-16 14:03:02 01:00:01 delay > 2018-08-16 21:18:01 00:14:59 delay > 2018-08-17 06:03:01 No delay > 2018-08-17 13:33:02 00:30:01 delay > 2018-08-17 21:03:02 No delay > > > On Thu, 16 Aug 2018 22:13:48 + > "Joel Esler (jesler)" wrote: > >> Paul, how are things looking from your side? >> >> -- >> Joel Esler >> Sr. Manager >> Community, Branding, and Open Source >> Talos Group >> http://www.talosintelligence.com >> >> On Aug 11, 2018, at 6:12 PM, Joel Esler (jesler) >> mailto:jes...@cisco.com>> wrote: >> >> I actually just made an adjustment today to see if that will resolve >> the issues. Please keep these coming?! >> >> Sent from my iPad >> >> On Aug 11, 2018, at 2:10 PM, Paul Kosinski >> mailto:clamav-us...@iment.com>> wrote: >> >> Here is the latest report for ClamAV virus update mirror delays since >> the end of July. DNS TXT vs actual file availability has gotten >> worse! Now, over 57% of updates are DNS-announced prematurely. (Last >> time, only about 1/3 were announced prematurely.) >> >> 2018-07-31 21:33:01 No delay >> 2018-08-01 05:03:01 00:14:59 delay >> 2018-08-01 13:18:01 No delay >> 2018-08-01 21:03:02 00:15:00 delay >> 2018-08-02 05:18:02 No delay >> 2018-08-02 13:48:02 00:30:00 delay >> 2018-08-02 21:48:01 00:15:00 delay >> 2018-08-03 05:18:02 No delay >> 2018-08-03 14:18:02 00:30:01 delay >> 2018-08-03 21:48:02 00:30:01 delay >> 2018-08-04 05:18:01 No delay >> 2018-08-04 13:18:02 No delay >> 2018-08-04 21:33:01 00:14:59 delay >> 2018-08-05 05:48:02 01:00:00 delay >> 2018-08-05 13:03:02 00:15:00 delay >> 2018-08-05 21:03:02 00:15:00 delay >> 2018-08-06 05:18:02 No delay >> 2018-08-06 13:18:02 No delay >> 2018-08-07 00:03:01 02:45:00 delay >> 2018-08-07 05:18:02 No delay >> 2018-08-07 13:18:02 No delay >> 2018-08-07 21:48:01 00:29:59 delay >> 2018-08-08 05:48:01 01:00:00 delay >> 2018-08-08 13:48:02 00:30:01 delay >> 2018-08-08 21:18:01 No delay >> 2018-08-09 05:33:02 No delay >> 2018-08-09 14:33:01 01:15:00 delay >> 2018-08-09 22:33:01 00:44:59 delay >> 2018-08-10 05:48:01 01:00:00 delay >> 2018-08-10 13:48:02 00:30:00 delay >> 2018-08-11 00:48:02 03:30:00 delay >> 2018-08-11 05:33:02 No delay >> 2018-08-11 13:18:02 No delay > ___ > clamav-users mailing list > clamav-users@lists.clamav.net > http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ClamAV signature update sync errors have gotten worse
Joel, Still lots of delays since "2018-08-11 13:18:02 No delay", but none quite as long as the previous batch: 2018-08-11 21:33:02 00:15:00 delay 2018-08-12 05:48:02 01:00:00 delay 2018-08-12 14:33:01 01:15:00 delay 2018-08-12 22:48:02 01:00:00 delay 2018-08-13 05:18:01 No delay 2018-08-13 13:18:02 No delay 2018-08-13 21:33:01 00:14:59 delay 2018-08-14 05:18:01 No delay 2018-08-14 13:18:02 No delay 2018-08-14 21:33:02 00:30:01 delay 2018-08-15 05:03:02 No delay 2018-08-15 13:48:01 00:45:00 delay 2018-08-15 22:03:01 No delay 2018-08-16 05:03:02 No delay 2018-08-16 14:03:02 01:00:01 delay 2018-08-16 21:18:01 00:14:59 delay 2018-08-17 06:03:01 No delay 2018-08-17 13:33:02 00:30:01 delay 2018-08-17 21:03:02 No delay On Thu, 16 Aug 2018 22:13:48 + "Joel Esler (jesler)" wrote: > Paul, how are things looking from your side? > > -- > Joel Esler > Sr. Manager > Community, Branding, and Open Source > Talos Group > http://www.talosintelligence.com > > On Aug 11, 2018, at 6:12 PM, Joel Esler (jesler) > mailto:jes...@cisco.com>> wrote: > > I actually just made an adjustment today to see if that will resolve > the issues. Please keep these coming?! > > Sent from my iPad > > On Aug 11, 2018, at 2:10 PM, Paul Kosinski > mailto:clamav-us...@iment.com>> wrote: > > Here is the latest report for ClamAV virus update mirror delays since > the end of July. DNS TXT vs actual file availability has gotten > worse! Now, over 57% of updates are DNS-announced prematurely. (Last > time, only about 1/3 were announced prematurely.) > > 2018-07-31 21:33:01 No delay > 2018-08-01 05:03:01 00:14:59 delay > 2018-08-01 13:18:01 No delay > 2018-08-01 21:03:02 00:15:00 delay > 2018-08-02 05:18:02 No delay > 2018-08-02 13:48:02 00:30:00 delay > 2018-08-02 21:48:01 00:15:00 delay > 2018-08-03 05:18:02 No delay > 2018-08-03 14:18:02 00:30:01 delay > 2018-08-03 21:48:02 00:30:01 delay > 2018-08-04 05:18:01 No delay > 2018-08-04 13:18:02 No delay > 2018-08-04 21:33:01 00:14:59 delay > 2018-08-05 05:48:02 01:00:00 delay > 2018-08-05 13:03:02 00:15:00 delay > 2018-08-05 21:03:02 00:15:00 delay > 2018-08-06 05:18:02 No delay > 2018-08-06 13:18:02 No delay > 2018-08-07 00:03:01 02:45:00 delay > 2018-08-07 05:18:02 No delay > 2018-08-07 13:18:02 No delay > 2018-08-07 21:48:01 00:29:59 delay > 2018-08-08 05:48:01 01:00:00 delay > 2018-08-08 13:48:02 00:30:01 delay > 2018-08-08 21:18:01 No delay > 2018-08-09 05:33:02 No delay > 2018-08-09 14:33:01 01:15:00 delay > 2018-08-09 22:33:01 00:44:59 delay > 2018-08-10 05:48:01 01:00:00 delay > 2018-08-10 13:48:02 00:30:00 delay > 2018-08-11 00:48:02 03:30:00 delay > 2018-08-11 05:33:02 No delay > 2018-08-11 13:18:02 No delay ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ClamAV signature update sync errors have gotten worse
I actually just made an adjustment today to see if that will resolve the issues. Please keep these coming?! Sent from my iPad > On Aug 11, 2018, at 2:10 PM, Paul Kosinski wrote: > > Here is the latest report for ClamAV virus update mirror delays since > the end of July. DNS TXT vs actual file availability has gotten worse! > Now, over 57% of updates are DNS-announced prematurely. (Last time, only > about 1/3 were announced prematurely.) > > 2018-07-31 21:33:01 No delay > 2018-08-01 05:03:01 00:14:59 delay > 2018-08-01 13:18:01 No delay > 2018-08-01 21:03:02 00:15:00 delay > 2018-08-02 05:18:02 No delay > 2018-08-02 13:48:02 00:30:00 delay > 2018-08-02 21:48:01 00:15:00 delay > 2018-08-03 05:18:02 No delay > 2018-08-03 14:18:02 00:30:01 delay > 2018-08-03 21:48:02 00:30:01 delay > 2018-08-04 05:18:01 No delay > 2018-08-04 13:18:02 No delay > 2018-08-04 21:33:01 00:14:59 delay > 2018-08-05 05:48:02 01:00:00 delay > 2018-08-05 13:03:02 00:15:00 delay > 2018-08-05 21:03:02 00:15:00 delay > 2018-08-06 05:18:02 No delay > 2018-08-06 13:18:02 No delay > 2018-08-07 00:03:01 02:45:00 delay > 2018-08-07 05:18:02 No delay > 2018-08-07 13:18:02 No delay > 2018-08-07 21:48:01 00:29:59 delay > 2018-08-08 05:48:01 01:00:00 delay > 2018-08-08 13:48:02 00:30:01 delay > 2018-08-08 21:18:01 No delay > 2018-08-09 05:33:02 No delay > 2018-08-09 14:33:01 01:15:00 delay > 2018-08-09 22:33:01 00:44:59 delay > 2018-08-10 05:48:01 01:00:00 delay > 2018-08-10 13:48:02 00:30:00 delay > 2018-08-11 00:48:02 03:30:00 delay > 2018-08-11 05:33:02 No delay > 2018-08-11 13:18:02 No delay > > > > > On Tue, 31 Jul 2018 13:47:39 -0400 > Paul Kosinski wrote: > >> There are still over 1/3 signature update sync errors with the new >> ClamAV mirrors. >> >> You may remember that I previously added code to our ClamAV update >> protocol to verify that the actually available daily.cvd etc. matched >> the version number reported by the DNS TXT record. (This is done by >> using curl to retrieve a short prefix of the daily.cvd etc. files.) >> >> Some days ago I also added code to summarize any synchronization >> problems. This code logs how much, if any, delay obtains between the >> DNS TXT record reporting a new version and curl agreeing that the new >> version is actually available from a mirror. This protocol operates >> every 15 minutes (giving rise to a rounding error of up to 15 >> minutes). >> >> Recent results are as follows: >> >> 2018-07-26 05:18:02 No delay >> 2018-07-26 13:18:02 No delay >> 2018-07-26 13:33:01 No delay >> 2018-07-26 13:48:01 No delay >> 2018-07-26 14:03:01 No delay >> 2018-07-26 14:18:02 No delay >> 2018-07-26 21:33:02 00:45:00 delay >> 2018-07-27 05:03:02 No delay >> 2018-07-27 13:18:02 No delay >> 2018-07-27 13:18:02 No delay >> 2018-07-27 21:34:05 No delay >> 2018-07-27 21:34:05 No delay >> 2018-07-28 05:48:02 00:30:01 delay >> 2018-07-28 13:18:02 No delay >> 2018-07-28 21:18:02 00:30:01 delay >> 2018-07-29 05:33:01 00:29:59 delay >> 2018-07-29 14:18:02 01:30:00 delay >> 2018-07-29 21:18:02 00:30:00 delay >> 2018-07-30 04:48:02 No delay >> 2018-07-30 13:33:01 00:44:59 delay >> 2018-07-30 21:48:02 01:00:00 delay >> 2018-07-31 05:18:02 No delay >> 2018-07-31 13:33:02 No delay >> >> Note: The ClamAV update protocol uses the URL "database.clamav.net", >> which in turn resolves to 1 of 5 IP addresses, which in turn route to >> one of many geographically separated systems. Thus, the delays may not >> correspond to problems with particular paths and/or mirror instances. >> >> >> P.S. I can provide a more detailed freshclam log for this if desired. > >> > ___ > clamav-users mailing list > clamav-users@lists.clamav.net > http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml ___ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
