Re: [squid-users] Slowness in Squid [squid-users Digest, Vol 26, Issue 82]
On Sunday 23 October 2016 at 14:42:02, Krishna Kulkarni wrote: > Hi Antony, > Thanks for the reply. I have made changes in squid.conf as per your > suggestion and have allocated 20 GB of Hard disk space. Have you made any measurements at all (either before making the disk cache bigger, or since) of what percentage of content Squid is actually caching for you? In other words, how much bandwidth is Squid saving you, compared to simply not using Squid at all and getting the content directly? Also, what made you believe that your disk cache was too small and needed to be 20Gbytes instead? > Squid server at my location handles http/https requests for more than 500 > hosts. What's more important is the number of requests per second going through Squid - it doesn't matter how many hosts are generating them. > But at peak hours squid usually performs very slow and browser takes > 1-2 minutes just to serve google home page and more time than that for > heavy web page. Have you compared this side-by-side with a browser configured to use Squid and a browser configured to go direct? > I have verified network link utilization & found it consumes not more than > 15 mb whereas link bandwidth is of 45mb So, why are you using Squid? > but still squid serves web pages very slow to client hosts. What hardware are you running Squid on? Which operating system / version are you running it under? What load is Squid generating on the machine? > Any suggestions in squid configuration to overcome this issue would be > highly appreciated. Have you made any measurements of the type of traffic your users are generating (for eaxmple, HTTP vs. HTTPS) and how much of this is cacheable at all? Squid won't help you if the content they're fetching can't be cached (either encrypted, or dynamically-generated etc.). Regards, Antony. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Slowness in Squid
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 In the final, I do not think Squid architecture is designed for fast access to huge amounts of memory. It came from a time when computers were young, memory cost like Boeing and hardly Squid itself seriously reworked in this part since then. 23.10.2016 18:15, Yuri Voinov пишет: > > > > 23.10.2016 18:09, Matus UHLAR - fantomas пишет: > >> 23.10.2016 17:40, Yuri Voinov пишет: > >>> This effect is good known to all who have worked with relational > >>> databases. In fact, it is typical in general for all caches except > >>> purpose-built highly scalable systems. > > >>> 23.10.2016 17:37, Matus UHLAR - fantomas пишет: > doesn't that imply kind of effectiveness? > > > On 23.10.16 17:53, Yuri Voinov wrote: > >> In fact, the explanation is very simple. At some point soon will get the > >> content from the disc using an index of any kind than consequentially > >> and fully scan the giant structure in RAM. > >> > >> Performance indicator is expressed in the data access time. But this is > >> from the category of personal experience. Everyone can choose their own > >> road to hell. :) > > > saying this you could say that huge in-memory cache for OSes is > useless > This is not me talking, and tuning specialists. > > > iirc in some squid versions itwas caused by linear searching for memory > > objects. > Only up to a point and is highly dependent on the server hardware > architecture and software process architecture. > > > using indexes should speed up saerching and bigger probability to find and > > higher probability to find object in memory could outweight searching > time. > Certainly. As I said above, it depends on the software architecture > access to a cache memory. If there is an effective index structure (like > a balanced B-tree), the effect disappears. > > > databases are much faster when using indexes properly, aren't they? > Absolutely yes. But: Most database index structures exist for disk > objects and do not exist for the memory structures. There's a completely > different mechanism. Indices disk objects (which are the metadata) are > loaded into memory and used to access the on-disk data. But access to > the memory is carried out mostly through a simple list structures. > > Which leads to performance degradation in case of huge caches. Again we > come back to the importance of software architecture of the memory accesses. > > -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJYDKrmAAoJENNXIZxhPexGMPEIALfRoGL7EDTi4lS1rmItes6k VJATrwChT8uZR+nexYTusVRaiYc2OnbQthTLYSOTYWqXl43l+Haj0+YAAc4edS1J 8ajAY0FzmGZsLynTsRLq526QsIXBUcuTAnbXbZb16g8sHbWZ/cnZzeR2SBP86qyC b7EqAsQV1preTlqXo4WpfaZFDZldjwTaemjb91Rl9HsdBaKxcru35wzZbdvefbng y4I9QPAv6xVFvEjw/IsUdpSe9vRPHFaLXF7WyYF2rM9+1mmCWd9YhFK0NASJ8Jxs REhMi/WMZHFCh3SK1Dj5H+bH4xONN0AHuf0YogpbgvmjxrJkQhRB88RkH2xNiRE= =/YWN -END PGP SIGNATURE- 0x613DEC46.asc Description: application/pgp-keys ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Slowness in Squid
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 23.10.2016 18:09, Matus UHLAR - fantomas пишет: >> 23.10.2016 17:40, Yuri Voinov пишет: >>> This effect is good known to all who have worked with relational >>> databases. In fact, it is typical in general for all caches except >>> purpose-built highly scalable systems. > >>> 23.10.2016 17:37, Matus UHLAR - fantomas пишет: >>> > doesn't that imply kind of effectiveness? > > On 23.10.16 17:53, Yuri Voinov wrote: >> In fact, the explanation is very simple. At some point soon will get the >> content from the disc using an index of any kind than consequentially >> and fully scan the giant structure in RAM. >> >> Performance indicator is expressed in the data access time. But this is >> from the category of personal experience. Everyone can choose their own >> road to hell. :) > > saying this you could say that huge in-memory cache for OSes is useless This is not me talking, and tuning specialists. > > iirc in some squid versions itwas caused by linear searching for memory > objects. Only up to a point and is highly dependent on the server hardware architecture and software process architecture. > > using indexes should speed up saerching and bigger probability to find and > higher probability to find object in memory could outweight searching time. Certainly. As I said above, it depends on the software architecture access to a cache memory. If there is an effective index structure (like a balanced B-tree), the effect disappears. > > databases are much faster when using indexes properly, aren't they? Absolutely yes. But: Most database index structures exist for disk objects and do not exist for the memory structures. There's a completely different mechanism. Indices disk objects (which are the metadata) are loaded into memory and used to access the on-disk data. But access to the memory is carried out mostly through a simple list structures. Which leads to performance degradation in case of huge caches. Again we come back to the importance of software architecture of the memory accesses. -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJYDKn1AAoJENNXIZxhPexGoloH/2LRFaUccVw1lJHdPW0AhB4b al73ryEEuXo7y0H42T661Xs2rIReOdyIz68qvOUUq8dJBuY48SIAktd5DDfVEU8z FyGTzxub4LPzU6xKO+LCPd8Mp2SXLayE7Gb3MuMKq++XzubARKfxHwzft+cvAMAN GTCa+2WIgtQ3WowN7gaOZmKqW7GVSb0rz2yXSOw1sJMQ+VKvAv+vbWgiRi9osiAR VDgCYWMPX6aQLQGFweLhDVU84xkvxMnCcUisK+DnNXO9DoLwRBbwDbvXuTutRnik Wk+k1LB4A2OWdHsvAbNsPORqSaeUzjBLHkWROu4H3A0b5Kd+yDRXYThFSA85AsA= =Khvr -END PGP SIGNATURE- 0x613DEC46.asc Description: application/pgp-keys ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Slowness in Squid
23.10.2016 17:40, Yuri Voinov пишет: This effect is good known to all who have worked with relational databases. In fact, it is typical in general for all caches except purpose-built highly scalable systems. 23.10.2016 17:37, Matus UHLAR - fantomas пишет: > doesn't that imply kind of effectiveness? On 23.10.16 17:53, Yuri Voinov wrote: In fact, the explanation is very simple. At some point soon will get the content from the disc using an index of any kind than consequentially and fully scan the giant structure in RAM. Performance indicator is expressed in the data access time. But this is from the category of personal experience. Everyone can choose their own road to hell. :) saying this you could say that huge in-memory cache for OSes is useless iirc in some squid versions itwas caused by linear searching for memory objects. using indexes should speed up saerching and bigger probability to find and higher probability to find object in memory could outweight searching time. databases are much faster when using indexes properly, aren't they? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Support bacteria - they're the only culture some people have. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Slowness in Squid
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 In fact, the explanation is very simple. At some point soon will get the content from the disc using an index of any kind than consequentially and fully scan the giant structure in RAM. Performance indicator is expressed in the data access time. But this is from the category of personal experience. Everyone can choose their own road to hell. :) 23.10.2016 17:40, Yuri Voinov пишет: > > This effect is good known to all who have worked with relational > databases. In fact, it is typical in general for all caches except > purpose-built highly scalable systems. > > > 23.10.2016 17:37, Matus UHLAR - fantomas пишет: > > doesn't that imply kind of effectiveness? > > -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJYDKS6AAoJENNXIZxhPexGZPgH/0zb2g9Q35y7FOs5twsdtaYV D/4KC5XarOmK6Ki9Nok3J6LVo4fa6gtIJwoZiItIy+Gh3ByvbZePhjfwNcmJmQn/ p3mt/AelOjaCcI/0ZDxg28PxdNE2Scso6TXE+ZtlHRwpbhZQ5WsXF/bYS+VaZjsM cAW9BAR0/pOTPbOaRC7iFO2Jp7POLHhOEIP/al75aMhedAMz1UIymGa/Wxj4ZMpA UpG76xAe78UkBg7mPJAXJmAddnnynlC1VazmlZwSs9YJqFVSnSQ3SvfPxXUW4cN4 3E2iCq3EnO0rDn6TqLmG6w7ZqrZLNsl/JJTwecJ+Ai19xfvODYciAPjz2jB/Kuw= =8j7T -END PGP SIGNATURE- 0x613DEC46.asc Description: application/pgp-keys ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Slowness in Squid
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 This effect is good known to all who have worked with relational databases. In fact, it is typical in general for all caches except purpose-built highly scalable systems. 23.10.2016 17:37, Matus UHLAR - fantomas пишет: > doesn't that imply kind of effectiveness? -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJYDKG+AAoJENNXIZxhPexGv9IH/A+T/eimJHaIkzY4oyDjbM9Z fiDZyJPKoQz3uEDCS1qu+xxm33aol+IXFfE6L9ksuTUDwhzVnfw+m1aeEKv0NbBR ODUL6dUhHnSmKWQ4TjDG/nXaQHK7QQDy+uqeIdZe1P9WHdGJ0qNDKwtCV2mD9hHH QHg9Ukx7ycYDNAxxBvFfieOIhRfemuh8L+Z7Ne7nrmE6yGW+UCZze1/HfipYdFHH IAiMO87icjbdd//EcLvhupMAQIBvSBMdehHsri+eHS/hwId2aWGtE/eqRovi8ITe pPS2xKRC4H1IpzuMMNy0xCGsZ4UZ/vVN8fL0OhMX+VSX48RRN0C2GA1giFcNUAM= =wnN6 -END PGP SIGNATURE- 0x613DEC46.asc Description: application/pgp-keys ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Slowness in Squid
On 23.10.16 17:15, Yuri Voinov wrote: Keep in mind - a huge in-memory cache does not always give the acceleration. Moreover, in most cases you can get the opposite effect expected. It is a common misconception - that the giant memory cache will give a giant performance gain. doesn't that imply kind of effectiveness? 23.10.2016 9:36, Krishna Kulkarni пишет: I am new to squid.. I have installed squid 3.5 on CentOS 6.7. As a configuration part, I have kept most of the things default. Please advice on how to allocate cache memory of 20 GB to squid. I got to know that, more cache memory would increase performance of squid.. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. The only substitute for good manners is fast reflexes. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Slowness in Squid
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Keep in mind - a huge in-memory cache does not always give the acceleration. Moreover, in most cases you can get the opposite effect expected. It is a common misconception - that the giant memory cache will give a giant performance gain. 23.10.2016 9:36, Krishna Kulkarni пишет: > > Dear Team > I am new to squid.. I have installed squid 3.5 on CentOS 6.7. As a configuration part, I have kept most of the things default. Please advice on how to allocate cache memory of 20 GB to squid. I got to know that, more cache memory would increase performance of squid.. > > Thanks, > Krishna > > > > ___ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJYDJvoAAoJENNXIZxhPexGyFsH/3or29Reesz9IzZVim7aNk7h 8MYCw8n7l4oaWtZFXdnj6f7sCMKTBVYoxNXqI4O172/XPgkg6y3onFqgU3a5eNIT r1f09FLi7MAdBAl0YtuwbRXpcKe9SyjhA/hzOhC1UiN3nUcxSR/hyc0oKpjw3Oy1 LwrBQGq7ZjZlNnKZh/uatxyKZolizof9uvKufVJqJdGtJRwkfjc6ELBpC/Lp7chz yVe2JA7Qi0NHzMoQwS7HblZ/o60E7rdoGTPhBdozxoOMiteW3ZVbm97Mol+t1BM2 s0IsYw9IJSoWPUJ8V+AUryjrzComGnXR4wl20pFrZ+eO/v+tZff6OZVb5A5PDCE= =6ozO -END PGP SIGNATURE- 0x613DEC46.asc Description: application/pgp-keys ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Slowness in Squid
On Sunday 23 October 2016 at 05:36:22, Krishna Kulkarni wrote: > I am new to squid.. I have installed squid 3.5 on CentOS 6.7. As a > configuration part, I have kept most of the things default. Please advice > on how to allocate cache memory of 20 GB to squid. Do you mean cache memory, or disk cache? If you mean memory (RAM) and you have enough of this in your system (eg: 32 Gbytes or more), then find the section in squid.conf which starts with: # MEMORY CACHE OPTIONS And read about the tag "cache_mem". To set this value (normally 256 Mbytes) to 20 Gbytes, set: cache_mem 20 GB If, on the other hand, you do not mean memory, but you mean disk cache, then find the section in squid.conf with starts with: # TAG: cache_dir and read that section. Pay particular attention to the line which says: # cache_dir ufs Directory-Name Mbytes L1 L2 [options] And then later there is an example: # cache_dir ufs /var/spool/squid3 100 16 256 Uncomment that line and change the 100 (Megabytes) in that line to 2 (for 20 Gigabytes) and perhaps also adjust the 16 to something like 64 or even 256 - for a large cache you don't want a few directories with lots of entries each, so it's worthwhile creating lots of directories to keep the number of files in each down. > I got to know that, more cache memory would increase performance of squid.. What do you mean by "performance"? Antony. -- Just when you think you're done, a cat floats by with buttered toast strapped to its back. - Steve Krug, "Don't make me think" Please reply to the list; please *don't* CC me. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
[squid-users] Slowness in Squid
Dear Team I am new to squid.. I have installed squid 3.5 on CentOS 6.7. As a configuration part, I have kept most of the things default. Please advice on how to allocate cache memory of 20 GB to squid. I got to know that, more cache memory would increase performance of squid.. Thanks, Krishna ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
