As some of you may be aware, ever since ClamAV began using Cloudflare, we have seen many occasions when files like daily.cvd were not available to our LAN until well after the DNS TXT record implied they should be.
However, we discovered that these same files *are* available to our Web/email server right away. So what is the difference? The first difference is that our Web server (a VM) is offsite, and is served by the "IAD" Cloudflare complex, whereas our local setup is served by the "BOS" Cloudflare complex. The second, and likely explanatory difference, is that our local setup is connected via Comcast (a dynamic IP and all that), while our Web server (with its static IP etc.) is almost certainly more directly connected to the Internet as a whole. The workaround we have adopted is as follows: we installed a "tinyproxy" server on our offsite VM. To ensure it only proxys for us, it listens on the encrypted OpenVPN tunnel we already had in place for FTP uploads etc. Then, instead of directly accessing database.clamav.net, freshclam uses our remote VM as a proxy,so that the cvd files are downloaded indirectly from Cloudflare's IAD server complex (via tinyproxy) rather than directly from Cloudflare's BOS server complex. Since switching to this workaround a few days ago, we haven't observed any delays: the cvd files are available right away when the DNS TXT query says they should be. I strongly suspect that Comcast is the culprit in the delays that had plagued us. This is especially suggested by the fact that Cloudflare returns a "Cache-Control:" header similar to: Cache-Control: public, max-age=13672 where the max-age value varies, but is often several hours. In my opinion, for data like ClamAV virus updates, the "Cache-Control:" should specify "no-cache". Can Cloudflare do this for ClamAV? --------------------------------------------------------------------- Below is a pair of recent (pre-workaround) log excerpts. They show a delay of over 2.5 hours experienced from BOS (via Comcast) vs no delay from IAD. Note that the BOS "Date:" timestamp of 16:49:01 GMT *still* shows a "Last-Modified:" timestamp of 06:15:18 GMT, while IAD already shows the up-to-date "Last-Modified:" timestamp of 14:14:30 GMT at the much earlier "Date:" of 14:29:01 GMT! IAD Date: Sun, 02 Dec 2018 14:09:01 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 14:29:01 GMT Last-Modified: Sun, 02 Dec 2018 14:14:30 GMT ClamAV-VDB:02 Dec 2018 09-13 -0500:25173:2167842:63:ba557f61737b9d4b66acc96f7044b524:3nBAOxo97ssSNZb BOS Date: Sun, 02 Dec 2018 14:09:01 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 14:29:01 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 14:49:01 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 15:09:01 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 15:29:02 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 15:49:02 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 16:09:01 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 16:29:01 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 16:49:01 GMT Last-Modified: Sun, 02 Dec 2018 06:15:18 GMT ClamAV-VDB:02 Dec 2018 01-14 -0500:25172:2167574:63:13c670e3a525c4fd17bf65524ff05fcd:nwPmlNwUbKmexgT Date: Sun, 02 Dec 2018 17:09:01 GMT Last-Modified: Sun, 02 Dec 2018 14:14:30 GMT ClamAV-VDB:02 Dec 2018 09-13 -0500:25173:2167842:63:ba557f61737b9d4b66acc96f7044b524:3nBAOxo97ssSNZb _______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml