Re: [squid-users] Sudden but sustained high bandwidth usage

Tue, 08 Mar 2016 14:39:18 -0800 


While debugging, found this:
2016/03/08 18:22:49.212 kid2| 88,3| client_side_reply.cc(463) cacheHit: clientCacheHit: http://au.v4.download.windowsupdate.com/c/msdownload/update/software/uprl/2016/03/windows-kb890830-x64-v5.34_e0074d1fa34d00f8b35e6d5c7be867292222c263.exe, 0 bytes 2016/03/08 18:22:49.212 kid2| 88,3| client_side_reply.cc(470) cacheHit: clientCacheHit: swapin failure for http://au.v4.download.windowsupdate.com/c/msdownload/update/software/uprl/2016/03/windows-kb890830-x64-v5.34_e0074d1fa34d00f8b35e6d5c7be867292222c263.exe
This is what generating TCP_SWAPFAIL_MISS for a lot of files, and it redownloads them over and over 



Em 07/03/2016 19:08, Heiler Bemerguy escreveu:
I know what is happening. I just don't know how to fix it without breaking windows updates caching. 
The "extra" traffic is coming from windows updates mirrors.
/acl wupdatecachable url_regex -i (microsoft|windowsupdate)\.com.*\.(cab|exe|ms[i|u|f]|dat|zip|psf|appx|appxbundle|esd)// 
//range_offset_limit none wupdatecachable/
Commenting this second line is enough to put an end to the insane downloads. But then all the RANGE requests of windows updates won't be cached.
I don't know how to explain these FD numbers. I'm using EXT4 and I don't know what are vmware cache disks.
Oh, and it seems not only windows updates' range requests are giving trouble:
1457381845.725 241 10.101.1.23 TCP_SWAPFAIL_MISS/206 33123 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381846.036 293 10.101.1.23 TCP_SWAPFAIL_MISS/206 151352 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381846.120 66 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381846.310 66 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381846.397 60 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381846.514 63 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381846.633 65 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381846.747 63 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381846.839 63 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381846.949 81 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381847.041 61 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381847.192 118 10.101.1.23 TCP_SWAPFAIL_MISS/206 65897 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381847.447 207 10.101.1.23 TCP_SWAPFAIL_MISS/206 121720 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381862.824 639 10.101.1.23 TCP_SWAPFAIL_MISS/206 33122 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381865.720 2879 10.101.1.23 TCP_SWAPFAIL_MISS/206 221234 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381866.078 343 10.101.1.23 TCP_SWAPFAIL_MISS/206 33130 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381867.107 995 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381867.523 400 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381867.830 291 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381868.082 238 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381868.318 219 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381868.503 171 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381868.695 174 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381868.880 170 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381869.179 286 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381869.424 231 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381869.761 322 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381870.008 232 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381870.236 214 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381870.429 178 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381870.609 174 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381870.805 180 10.101.1.23 TCP_SWAPFAIL_MISS/206 65897 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 1457381871.008 189 10.101.1.23 TCP_SWAPFAIL_MISS/206 65898 GET http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf - HIER_DIRECT/201.49.144.135 application/pdf 

Best Regards,

--
Heiler Bemerguy - (91) 98151-4894
Assessor Técnico - CINBESA (91) 3184-1751

Em 07/03/2016 18:00, Eliezer Croitoru escreveu:

On 07/03/2016 22:08, Yuri Voinov wrote:

90 Gb first, 300 Gb second.

Thanks but...
Wouldn't it be much simpler and cheaper to just use WSUS instead all of the hassle??(if it's a closed business environment) 
And when does the TCP_SWAPFAIL_MISS happens? always?
And a little tweak for the squid.conf
read_ahead_gap 4096 KB

The above doesn't match your environment bandwidth.
You are just spending too much bandwidth on requests that might not be fulfilled. 
Try changing the settings to:
read_ahead_gap 128 KB

and see if it helps with something.
Also since your issue is bandwidth and users are not allowed to the Internet without the proxy I would try to dump the pfsense pf states to see what happens in the network layer, what src IP is consuming all this bandwidth.(or get a wider picture) Also squid would not be the answer for a fully reasonable network usage, it only comes to help with couple specific things and not to mirror the whole Internet or even just MS as it is.(not saying that your case is a trial such as this) MS has more then one API that can be mistaken as a download and it can consume more then actually required caching. 

I do not know exactly what this means from the info page:
        Maximum number of file descriptors:   81920
        Largest file desc currently in use:   6157
        Number of file desc currently in use: 8216
If the number of FD currently in use is 8216 then the largest file desc currently in use doesn't match. This specific question might be a bug or expected result but I do not know and Amos or Alex or others might know the answer for this specific info page issue.
Another question I have which might be related(I have experienced such issues with GlusterFS in the past), how is the VM cache disk\s are connected? Is it connected directly to the VM or in the hypervisor level? How do you mount them(fstab)? are these on the root disk or do you have couple disks mounted? Did you had the chance to try to use other FS then EXT4? reiserFS? XFS? other? 

The above questions are related to the TCP_SWAPFAIL_MISS.
Since there is an issue and you are probably only on the "buffering" testing stage of the cache_dir I would try to somehow reproduce the issue but it's not clear to me what is the exact way to replicate the issue. 

Thanks,
Eliezer




_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


--
Heiler Bemerguy - (91) 98151-4894
Assessor Técnico - CINBESA (91) 3184-1751


_______________________________________________
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Reply via email to