[squid-users] squidclient mgr:info squid performance
Hi Amos, Sir, Thanks for your kind response.As always you share your great guidance. > On 16/12/2011 12:51 a.m., Benjamin wrote: >> Hi, >> >> what is the meaning for "Select loop called: 730321522 times, 0.117 >> ms avg" >> >> squidclient mgr:info >> >> Connection information for squid: >> Number of clients accessing cache:0 >> Number of HTTP requests received:32626996 >> Number of ICP messages received:0 >> Number of ICP messages sent:0 >> Number of queued ICP replies:0 >> Number of HTCP messages received:0 >> Number of HTCP messages sent:0 >> Request failure ratio: 0.00 >> Average HTTP requests per minute since start:22877.7 >> Average ICP messages per minute since start:0.0 >> Select loop called: 730321522 times, 0.117 ms avg <- >> >> Is it showing that , squid does in/out for objects with this >> (730321522) times. is it right perception? > Squid has performed 730321522 tests to check for I/O events, an > average of every 0.117ms. That is all. > Several read/write/connect/close/DNS handling events may have happened > on each of those checks. > > >> My full output of squidclient mgr:info. Please tell me that my squid >> is performing well or do i need to do more tuning or something? >> >> >> >> squidclient mgr:info >> HTTP/1.0 200 OK >> Server: squid >> Mime-Version: 1.0 >> Date: Thu, 15 Dec 2011 11:42:25 GMT >> Content-Type: text/plain >> Expires: Thu, 15 Dec 2011 11:42:25 GMT >> Last-Modified: Thu, 15 Dec 2011 11:42:25 GMT >> X-Cache: MISS from CACHE-Engine >> X-Cache-Lookup: MISS from CACHE-Engine:3128 >> Connection: close >> >> Squid Object Cache: Version 3.1.10 >> Start Time:Wed, 14 Dec 2011 11:56:16 GMT >> Current Time:Thu, 15 Dec 2011 11:42:25 GMT >> Connection information for squid: >> Number of clients accessing cache:0 >> Number of HTTP requests received:32626996 >> Number of ICP messages received:0 >> Number of ICP messages sent:0 >> Number of queued ICP replies:0 >> Number of HTCP messages received:0 >> Number of HTCP messages sent:0 >> Request failure ratio: 0.00 >> Average HTTP requests per minute since start:22877.7 > Okay. 381rps aint too shabby. Highest-performance should be able to > reach at least 950rps with a lot of fine tweaking and tuning, but of > course that depends on the clients actually being there making request > and relatively fast themselves. What are the fine tweaking and tunning required from squid side and os side to achieve more performance to get 950rps. >> Average ICP messages per minute since start:0.0 >> Select loop called: 730321522 times, 0.117 ms avg >> Cache information for squid: >> Hits as % of all requests:5min: 33.2%, 60min: 32.1% > Fairly good for a forward proxy. You might be able to tweak it up into > the 40+% range. But that depends on the clients traffic to determine > what you can do and what will work. What do u suggest to achieve more cache hit? >> Hits as % of bytes sent:5min: 21.0%, 60min: 17.7% >> Memory hits as % of hit requests:5min: 16.0%, 60min: 16.0% > Memory hits is where you get the most speed. You might want to see > what is causing this to be low and see if you can tune it up a bit. > But again, that depends on the clients traffic. > As we have total memory 12 GB, and firstly i assign 8gb to squid and 4 gb for OS.But many times in high load , i see that in free -mto there was remaining free memory is very less. so then i set 2. gb for squid and rest of for OS but in that case also at high load i can see same less memory remaining in free -mto. i can see much ram in cached tab in free -mto output. Kindly suggest me to what to do with get best performance from memory side by OS and squid way >> Disk hits as % of hit requests:5min: 64.4%, 60min: 64.1% >> Storage Swap size:304802412 KB >> Storage Swap capacity:77.7% used, 22.3% free >> Storage Mem size:2535920 KB >> Storage Mem capacity:100.0% used, 0.0% free > Um. 2.5GB of RAM caching going on and ony giving you 16% hit ratio. > This does not seem great. > >> Mean Object Size:28.43 KB >> Requests given to unlinkd:0 >> Median Service Times (seconds) 5 min60 min: >> HTTP Requests (All): 0.24524 0.25890 >> Cache Misses: 0.35832 0.35832 >> Cache Hits:0.00919 0.00919 >> Near Hits: 0.15048 0.15048 >> Not-Modified Replies: 0.0 0.0 >> DNS Lookups: 0.13638 0.13638 > DNS appears to be slow. ~130ms. I would hope for at least one more > zero after the decimal point, making it under 100ms on average. But > this may be limited by the amount of time already spent processing > requests, so "it depends". I configured dnsmasq for dns caching.What else i need to do to tune dns to get more performance? > Amos My complete squid.conf acl manager
Re: [squid-users] squidclient mgr:info squid performance
On 16/12/2011 12:51 a.m., Benjamin wrote: Hi, what is the meaning for "Select loop called: 730321522 times, 0.117 ms avg" squidclient mgr:info Connection information for squid: Number of clients accessing cache:0 Number of HTTP requests received:32626996 Number of ICP messages received:0 Number of ICP messages sent:0 Number of queued ICP replies:0 Number of HTCP messages received:0 Number of HTCP messages sent:0 Request failure ratio: 0.00 Average HTTP requests per minute since start:22877.7 Average ICP messages per minute since start:0.0 Select loop called: 730321522 times, 0.117 ms avg <- Is it showing that , squid does in/out for objects with this (730321522) times. is it right perception? Squid has performed 730321522 tests to check for I/O events, an average of every 0.117ms. That is all. Several read/write/connect/close/DNS handling events may have happened on each of those checks. My full output of squidclient mgr:info. Please tell me that my squid is performing well or do i need to do more tuning or something? squidclient mgr:info HTTP/1.0 200 OK Server: squid Mime-Version: 1.0 Date: Thu, 15 Dec 2011 11:42:25 GMT Content-Type: text/plain Expires: Thu, 15 Dec 2011 11:42:25 GMT Last-Modified: Thu, 15 Dec 2011 11:42:25 GMT X-Cache: MISS from CACHE-Engine X-Cache-Lookup: MISS from CACHE-Engine:3128 Connection: close Squid Object Cache: Version 3.1.10 Start Time:Wed, 14 Dec 2011 11:56:16 GMT Current Time:Thu, 15 Dec 2011 11:42:25 GMT Connection information for squid: Number of clients accessing cache:0 Number of HTTP requests received:32626996 Number of ICP messages received:0 Number of ICP messages sent:0 Number of queued ICP replies:0 Number of HTCP messages received:0 Number of HTCP messages sent:0 Request failure ratio: 0.00 Average HTTP requests per minute since start:22877.7 Okay. 381rps aint too shabby. Highest-performance should be able to reach at least 950rps with a lot of fine tweaking and tuning, but of course that depends on the clients actually being there making request and relatively fast themselves. Average ICP messages per minute since start:0.0 Select loop called: 730321522 times, 0.117 ms avg Cache information for squid: Hits as % of all requests:5min: 33.2%, 60min: 32.1% Fairly good for a forward proxy. You might be able to tweak it up into the 40+% range. But that depends on the clients traffic to determine what you can do and what will work. Hits as % of bytes sent:5min: 21.0%, 60min: 17.7% Memory hits as % of hit requests:5min: 16.0%, 60min: 16.0% Memory hits is where you get the most speed. You might want to see what is causing this to be low and see if you can tune it up a bit. But again, that depends on the clients traffic. Disk hits as % of hit requests:5min: 64.4%, 60min: 64.1% Storage Swap size:304802412 KB Storage Swap capacity:77.7% used, 22.3% free Storage Mem size:2535920 KB Storage Mem capacity:100.0% used, 0.0% free Um. 2.5GB of RAM caching going on and ony giving you 16% hit ratio. This does not seem great. Mean Object Size:28.43 KB Requests given to unlinkd:0 Median Service Times (seconds) 5 min60 min: HTTP Requests (All): 0.24524 0.25890 Cache Misses: 0.35832 0.35832 Cache Hits:0.00919 0.00919 Near Hits: 0.15048 0.15048 Not-Modified Replies: 0.0 0.0 DNS Lookups: 0.13638 0.13638 DNS appears to be slow. ~130ms. I would hope for at least one more zero after the decimal point, making it under 100ms on average. But this may be limited by the amount of time already spent processing requests, so "it depends". Amos
[squid-users] squidclient mgr:info squid performance
Hi, what is the meaning for "Select loop called: 730321522 times, 0.117 ms avg" squidclient mgr:info Connection information for squid: Number of clients accessing cache:0 Number of HTTP requests received:32626996 Number of ICP messages received:0 Number of ICP messages sent:0 Number of queued ICP replies:0 Number of HTCP messages received:0 Number of HTCP messages sent:0 Request failure ratio: 0.00 Average HTTP requests per minute since start:22877.7 Average ICP messages per minute since start:0.0 Select loop called: 730321522 times, 0.117 ms avg <- Is it showing that , squid does in/out for objects with this (730321522) times. is it right perception? My full output of squidclient mgr:info. Please tell me that my squid is performing well or do i need to do more tuning or something? squidclient mgr:info HTTP/1.0 200 OK Server: squid Mime-Version: 1.0 Date: Thu, 15 Dec 2011 11:42:25 GMT Content-Type: text/plain Expires: Thu, 15 Dec 2011 11:42:25 GMT Last-Modified: Thu, 15 Dec 2011 11:42:25 GMT X-Cache: MISS from CACHE-Engine X-Cache-Lookup: MISS from CACHE-Engine:3128 Connection: close Squid Object Cache: Version 3.1.10 Start Time:Wed, 14 Dec 2011 11:56:16 GMT Current Time:Thu, 15 Dec 2011 11:42:25 GMT Connection information for squid: Number of clients accessing cache:0 Number of HTTP requests received:32626996 Number of ICP messages received:0 Number of ICP messages sent:0 Number of queued ICP replies:0 Number of HTCP messages received:0 Number of HTCP messages sent:0 Request failure ratio: 0.00 Average HTTP requests per minute since start:22877.7 Average ICP messages per minute since start:0.0 Select loop called: 730321522 times, 0.117 ms avg Cache information for squid: Hits as % of all requests:5min: 33.2%, 60min: 32.1% Hits as % of bytes sent:5min: 21.0%, 60min: 17.7% Memory hits as % of hit requests:5min: 16.0%, 60min: 16.0% Disk hits as % of hit requests:5min: 64.4%, 60min: 64.1% Storage Swap size:304802412 KB Storage Swap capacity:77.7% used, 22.3% free Storage Mem size:2535920 KB Storage Mem capacity:100.0% used, 0.0% free Mean Object Size:28.43 KB Requests given to unlinkd:0 Median Service Times (seconds) 5 min60 min: HTTP Requests (All): 0.24524 0.25890 Cache Misses: 0.35832 0.35832 Cache Hits:0.00919 0.00919 Near Hits: 0.15048 0.15048 Not-Modified Replies: 0.0 0.0 DNS Lookups: 0.13638 0.13638 ICP Queries: 0.0 0.0 Resource usage for squid: UP Time:85568.728 seconds CPU Time:39677.180 seconds CPU Usage:46.37% CPU Usage, 5 minute avg:62.85% CPU Usage, 60 minute avg:65.45% Process Data Segment Size via sbrk(): 5418384 KB Maximum Resident Size: 21795440 KB Page faults with physical i/o: 2960 Memory usage for squid via mallinfo(): Total space in arena: 1224212 KB Ordinary blocks: 1224087 KB415 blks Small blocks: 0 KB 0 blks Holding blocks: 51136 KB 10 blks Free Small blocks: 0 KB Free Ordinary blocks: 124 KB Total in use: 1275223 KB 100% Total free: 124 KB 0% Total size:1275348 KB Memory accounted for: Total accounted: 535912 KB 42% memPool accounted: 4730216 KB 42% memPool unaccounted: -3454869 KB 0% memPoolAlloc calls: 8209544357 memPoolFree calls: 8240665554 File descriptor usage for squid: Maximum number of file descriptors: 65000 Largest file desc currently in use: 20009 Number of file desc currently in use: 18391 Files queued for open: 0 Available number of file descriptors: 46609 Reserved number of file descriptors: 100 Store Disk files open: 654 Internal Data Structures: 10721767 StoreEntries 288699 StoreEntries with MemObjects 286763 Hot Object Cache Items 10720018 on-disk objects Regards, Benjamin
Re: [squid-users] squidclient mgr:info
On 03/04/11 07:22, sq...@sourcesystemsonline.com wrote: Good day, Special thanks to Hasanen AL-Bana. Thank you for swift response. Do you mean ICP or HTCP is useful only if i have another cache server WITHIN my network? No. Location does not matter. If you are using another proxy as cache_peer or someone else is using yours as their cache_peer, then the proxy<->proxy protocols matter. Please, what information do want me to supply you so that you can help fine tune my squid performance because am already having "headache" with squid rebuilding store most of the time (see my post "Cache log errors"). Rebuild usually occurs under two scenarios: * on restart following a crash * on restart following a shutdown/restart which was aborted early before the cache meta data was saved to disk. From your other thread it appears that you have hit some complications which lead to this second case. (continued in that other thread). Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.11 Beta testers wanted for 3.2.0.5
Re: [squid-users] squidclient mgr:info
Good day, Special thanks to Hasanen AL-Bana. Thank you for swift response. Do you mean ICP or HTCP is useful only if i have another cache server WITHIN my network? Please, what information do want me to supply you so that you can help fine tune my squid performance because am already having "headache" with squid rebuilding store most of the time (see my post "Cache log errors"). Regards Yomi > Hi Yomi, > > > ICP & HTCP are protocols used between 2 or more cache servers to > communicate and share their data , if you only have one single cache > server then you don't have to worry about ICP or HTCP. > > from you squidclient info I can see some bad things like your cache > miss time :Cache Misses: 51.56862 which is extremely high. > > Another thing is the Page faults with physical i/o: 25098 , the > lower this number , the better performance you get...this indicates > that you are using too much RAM and your process is using swap > partition and decreasing performance. > > > > On Sat, Apr 2, 2011 at 6:20 PM, wrote: >> >> Good day, >> After checking the activities of my squid-cache proxy, i discover that >> there >> is no activities on ICP and HTCP, why? >> What is the implication of this? >> If they are important to squid performance, how can i implement them? >> Regards, >> Yomi >> >> >> Microsoft Windows [Version 6.1.7601] >> Copyright (c) 2009 Microsoft Corporation. All rights reserved. >> >> C:\squid\bin>squidclient mgr:info >> HTTP/1.0 200 OK >> Server: squid/2.7.STABLE8 >> Date: Sat, 02 Apr 2011 13:40:27 GMT >> Content-Type: text/plain >> Expires: Sat, 02 Apr 2011 13:40:27 GMT >> X-Cache: MISS from ADMIN >> X-Cache-Lookup: MISS from ADMIN:3128 >> Via: 1.0 ADMIN:3128 (squid/2.7.STABLE8) >> Connection: close >> >> Squid Object Cache: Version 2.7.STABLE8 >> >> Running as squid Windows System Service on Windows Vista >> Service command line is: >> Start Time: Sat, 02 Apr 2011 10:38:44 GMT >> Current Time: Sat, 02 Apr 2011 13:40:27 GMT >> Connection information for squid: >> Number of clients accessing cache: 7 >> Number of HTTP requests received: 15437 >> Number of ICP messages received: 0 >> Number of ICP messages sent: 0 >> Number of queued ICP replies: 0 >> Number of HTCP messages received: 0 >> Number of HTCP messages sent: 0 >> Request failure ratio: 0.00 >> Average HTTP requests per minute since start: 85.0 >> Average ICP messages per minute since start: 0.0 >> Select loop called: 14516733 times, 0.751 ms avg >> Cache information for squid: >> Request Hit Ratios: 5min: 5.6%, 60min: 13.6% >> Byte Hit Ratios: 5min: 47.5%, 60min: 11.2% >> Request Memory Hit Ratios: 5min: 100.0%, 60min: 67.8% >> Request Disk Hit Ratios: 5min: 0.0%, 60min: 0.0% >> Storage Swap size: 91072 KB >> Storage Mem size: 54680 KB >> Mean Object Size: 10.76 KB >> Requests given to unlinkd: 0 >> Median Service Times (seconds) 5 min 60 min: >> HTTP Requests (All): 2.02066 1.05672 >> Cache Misses: 51.56862 1.24267 >> Cache Hits: 0.0 0.0 >> Near Hits: 0.0 0.61549 >> Not-Modified Replies: 0.0 0.0 >> DNS Lookups: 0.09117 0.26619 >> ICP Queries: 0.0 0.0 >> Resource usage for squid: >> UP Time: 10902.873 seconds >> CPU Time: 10220.328 seconds >> CPU Usage: 93.74% >> CPU Usage, 5 minute avg: 93.70% >> CPU Usage, 60 minute avg: 94.06% >> Maximum Resident Size: 91776 KB >> Page faults with physical i/o: 25098 >> Memory accounted for: >> Total accounted: 63592 KB >> memPoolAlloc calls: 2900352005 >> memPoolFree calls: 2900133281 >> File descriptor usage for squid: >> Maximum number of file descriptors: 2048 >> Largest file desc currently in use: 120 >> Number of file desc currently in use: 22 >> Files queued for open: 0 >> Available number of file descriptors: 2026 >> Reserved number of file descriptors: 100 >> Store Disk files open: 0 >> IO loop method: select >> Internal Data Structures: >> 20387 StoreEntries >> 6133 StoreEntries with MemObjects >> 6131 Hot Object Cache Items >> 8461 on-disk objects >> >> C:\squid\bin> >
Re: [squid-users] squidclient mgr:info
Hi Yomi, ICP & HTCP are protocols used between 2 or more cache servers to communicate and share their data , if you only have one single cache server then you don't have to worry about ICP or HTCP. from you squidclient info I can see some bad things like your cache miss time :Cache Misses: 51.56862 which is extremely high. Another thing is the Page faults with physical i/o: 25098 , the lower this number , the better performance you get...this indicates that you are using too much RAM and your process is using swap partition and decreasing performance. On Sat, Apr 2, 2011 at 6:20 PM, wrote: > > Good day, > After checking the activities of my squid-cache proxy, i discover that there > is no activities on ICP and HTCP, why? > What is the implication of this? > If they are important to squid performance, how can i implement them? > Regards, > Yomi > > > Microsoft Windows [Version 6.1.7601] > Copyright (c) 2009 Microsoft Corporation. All rights reserved. > > C:\squid\bin>squidclient mgr:info > HTTP/1.0 200 OK > Server: squid/2.7.STABLE8 > Date: Sat, 02 Apr 2011 13:40:27 GMT > Content-Type: text/plain > Expires: Sat, 02 Apr 2011 13:40:27 GMT > X-Cache: MISS from ADMIN > X-Cache-Lookup: MISS from ADMIN:3128 > Via: 1.0 ADMIN:3128 (squid/2.7.STABLE8) > Connection: close > > Squid Object Cache: Version 2.7.STABLE8 > > Running as squid Windows System Service on Windows Vista > Service command line is: > Start Time: Sat, 02 Apr 2011 10:38:44 GMT > Current Time: Sat, 02 Apr 2011 13:40:27 GMT > Connection information for squid: > Number of clients accessing cache: 7 > Number of HTTP requests received: 15437 > Number of ICP messages received: 0 > Number of ICP messages sent: 0 > Number of queued ICP replies: 0 > Number of HTCP messages received: 0 > Number of HTCP messages sent: 0 > Request failure ratio: 0.00 > Average HTTP requests per minute since start: 85.0 > Average ICP messages per minute since start: 0.0 > Select loop called: 14516733 times, 0.751 ms avg > Cache information for squid: > Request Hit Ratios: 5min: 5.6%, 60min: 13.6% > Byte Hit Ratios: 5min: 47.5%, 60min: 11.2% > Request Memory Hit Ratios: 5min: 100.0%, 60min: 67.8% > Request Disk Hit Ratios: 5min: 0.0%, 60min: 0.0% > Storage Swap size: 91072 KB > Storage Mem size: 54680 KB > Mean Object Size: 10.76 KB > Requests given to unlinkd: 0 > Median Service Times (seconds) 5 min 60 min: > HTTP Requests (All): 2.02066 1.05672 > Cache Misses: 51.56862 1.24267 > Cache Hits: 0.0 0.0 > Near Hits: 0.0 0.61549 > Not-Modified Replies: 0.0 0.0 > DNS Lookups: 0.09117 0.26619 > ICP Queries: 0.0 0.0 > Resource usage for squid: > UP Time: 10902.873 seconds > CPU Time: 10220.328 seconds > CPU Usage: 93.74% > CPU Usage, 5 minute avg: 93.70% > CPU Usage, 60 minute avg: 94.06% > Maximum Resident Size: 91776 KB > Page faults with physical i/o: 25098 > Memory accounted for: > Total accounted: 63592 KB > memPoolAlloc calls: 2900352005 > memPoolFree calls: 2900133281 > File descriptor usage for squid: > Maximum number of file descriptors: 2048 > Largest file desc currently in use: 120 > Number of file desc currently in use: 22 > Files queued for open: 0 > Available number of file descriptors: 2026 > Reserved number of file descriptors: 100 > Store Disk files open: 0 > IO loop method: select > Internal Data Structures: > 20387 StoreEntries > 6133 StoreEntries with MemObjects > 6131 Hot Object Cache Items > 8461 on-disk objects > > C:\squid\bin>
[squid-users] squidclient mgr:info
Good day, After checking the activities of my squid-cache proxy, i discover that there is no activities on ICP and HTCP, why? What is the implication of this? If they are important to squid performance, how can i implement them? Regards, Yomi Microsoft Windows [Version 6.1.7601] Copyright (c) 2009 Microsoft Corporation. All rights reserved. C:\squid\bin>squidclient mgr:info HTTP/1.0 200 OK Server: squid/2.7.STABLE8 Date: Sat, 02 Apr 2011 13:40:27 GMT Content-Type: text/plain Expires: Sat, 02 Apr 2011 13:40:27 GMT X-Cache: MISS from ADMIN X-Cache-Lookup: MISS from ADMIN:3128 Via: 1.0 ADMIN:3128 (squid/2.7.STABLE8) Connection: close Squid Object Cache: Version 2.7.STABLE8 Running as squid Windows System Service on Windows Vista Service command line is: Start Time: Sat, 02 Apr 2011 10:38:44 GMT Current Time: Sat, 02 Apr 2011 13:40:27 GMT Connection information for squid: Number of clients accessing cache: 7 Number of HTTP requests received: 15437 Number of ICP messages received:0 Number of ICP messages sent:0 Number of queued ICP replies: 0 Number of HTCP messages received: 0 Number of HTCP messages sent: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 85.0 Average ICP messages per minute since start:0.0 Select loop called: 14516733 times, 0.751 ms avg Cache information for squid: Request Hit Ratios: 5min: 5.6%, 60min: 13.6% Byte Hit Ratios:5min: 47.5%, 60min: 11.2% Request Memory Hit Ratios: 5min: 100.0%, 60min: 67.8% Request Disk Hit Ratios:5min: 0.0%, 60min: 0.0% Storage Swap size: 91072 KB Storage Mem size: 54680 KB Mean Object Size: 10.76 KB Requests given to unlinkd: 0 Median Service Times (seconds) 5 min60 min: HTTP Requests (All): 2.02066 1.05672 Cache Misses: 51.56862 1.24267 Cache Hits:0.0 0.0 Near Hits: 0.0 0.61549 Not-Modified Replies: 0.0 0.0 DNS Lookups: 0.09117 0.26619 ICP Queries: 0.0 0.0 Resource usage for squid: UP Time:10902.873 seconds CPU Time: 10220.328 seconds CPU Usage: 93.74% CPU Usage, 5 minute avg:93.70% CPU Usage, 60 minute avg: 94.06% Maximum Resident Size: 91776 KB Page faults with physical i/o: 25098 Memory accounted for: Total accounted:63592 KB memPoolAlloc calls: 2900352005 memPoolFree calls: 2900133281 File descriptor usage for squid: Maximum number of file descriptors: 2048 Largest file desc currently in use:120 Number of file desc currently in use: 22 Files queued for open: 0 Available number of file descriptors: 2026 Reserved number of file descriptors: 100 Store Disk files open: 0 IO loop method: select Internal Data Structures: 20387 StoreEntries 6133 StoreEntries with MemObjects 6131 Hot Object Cache Items 8461 on-disk objects C:\squid\bin>
