Hello Today i have seen some issue with Disk I/O performance issue can any tell me how to improve that !
# squidinfo Connection information for squid: Number of clients accessing cache: 0 Number of HTTP requests received: 422259 Number of ICP messages received: 0 Number of ICP messages sent: 0 Number of queued ICP replies: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 12157.7 Average ICP messages per minute since start: 0.0 Select loop called: 4180430 times, 0.498 ms avg Cache information for squid: Request Hit Ratios: 5min: 19.1%, 60min: 20.1% Byte Hit Ratios: 5min: 9.9%, 60min: 9.6% Request Memory Hit Ratios: 5min: 19.3%, 60min: 20.8% Request Disk Hit Ratios: 5min: 61.6%, 60min: 53.3% Storage Swap size: 169917856 KB Storage Mem size: 126032 KB Mean Object Size: 49.83 KB Requests given to unlinkd: 0 Median Service Times (seconds) 5 min 60 min: HTTP Requests (All): 0.42149 0.39928 Cache Misses: 0.42149 0.39928 Cache Hits: 0.46965 0.52331 Near Hits: 0.94847 0.89858 Not-Modified Replies: 0.12106 0.00286 DNS Lookups: 0.00000 0.00000 ICP Queries: 0.00000 0.00000 Resource usage for squid: UP Time: 2083.915 seconds CPU Time: 455.501 seconds CPU Usage: 21.86% CPU Usage, 5 minute avg: 20.97% CPU Usage, 60 minute avg: 21.88% Process Data Segment Size via sbrk(): 728820 KB Maximum Resident Size: 0 KB Page faults with physical i/o: 9 Memory usage for squid via mallinfo(): Total space in arena: 728952 KB Ordinary blocks: 703156 KB 15798 blks Small blocks: 0 KB 0 blks Holding blocks: 96620 KB 8 blks Free Small blocks: 0 KB Free Ordinary blocks: 25795 KB Total in use: 799776 KB 97% Total free: 25795 KB 3% Total size: 825572 KB Memory accounted for: Total accounted: 501426 KB memPoolAlloc calls: 76354783 memPoolFree calls: 69149858 File descriptor usage for squid: Maximum number of file descriptors: 64000 Largest file desc currently in use: 3491 Number of file desc currently in use: 2700 Files queued for open: 25 Available number of file descriptors: 61275 Reserved number of file descriptors: 100 Store Disk files open: 254 IO loop method: epoll Internal Data Structures: 3418559 StoreEntries 14446 StoreEntries with MemObjects 13569 Hot Object Cache Items 3410021 on-disk objects Store Directory Statistics: Store Entries : 3418561 Maximum Swap Size : 184320000 KB Current Store Swap Size: 169917840 KB Current Capacity : 92% used, 8% free Store Directory #0 (aufs): /cache1 FS Block Size 4096 Bytes First level subdirectories: 64 Second level subdirectories: 256 Maximum Size: 61440000 KB Current Size: 55189460 KB Percent Used: 89.83% Current load metric: 1045 / 1000 Filemap bits in use: 1011127 of 2097152 (48%) Filesystem Space in use: 71357340/76920416 KB (93%) Filesystem Inodes in use: 2113612/4890624 (43%) Flags: Accepted object sizes: 0 - (unlimited) bytes Removal policy: heap Store Directory #1 (aufs): /cache2 FS Block Size 4096 Bytes First level subdirectories: 128 Second level subdirectories: 256 Maximum Size: 122880000 KB Current Size: 114728380 KB Percent Used: 93.37% Current load metric: 1045 / 1000 Filemap bits in use: 2399082 of 4194304 (57%) Filesystem Space in use: 116823500/153834852 KB (76%) Filesystem Inodes in use: 4227212/9773056 (43%) Flags: Accepted object sizes: 0 - (unlimited) bytes Removal policy: heap # iotop Total DISK READ: 2.20 M/s | Total DISK WRITE: 157.73 K/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 1687 rt/0 squid 7.51 K/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1673 rt/0 squid 548.30 K/s 15.02 K/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1684 rt/0 squid 0.00 B/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1681 rt/0 squid 37.56 K/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1665 rt/0 squid 3.76 K/s 450.66 K/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1683 rt/0 squid 3.76 K/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1674 rt/0 squid 401.84 K/s 3.76 K/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1686 rt/0 squid 146.46 K/s 435.64 K/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1670 rt/0 squid 184.02 K/s 75.11 K/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1669 rt/0 squid 0.00 B/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1688 rt/0 squid 0.00 B/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1679 rt/0 squid 259.13 K/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1672 rt/0 squid 0.00 B/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1671 rt/0 squid 3.76 K/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1682 rt/0 squid 120.18 K/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1685 rt/0 squid 0.00 B/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1680 rt/0 squid 11.27 K/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf 1668 rt/0 squid 60.09 K/s 0.00 B/s 0.00 % 99.99 % (squid) -D -f /etc/squid/squid.conf #iostat Linux 2.6.29.6-217.2.8.fc11.x86_64 (localhost.localdomain) 09/04/2009 avg-cpu: %user %nice %system %iowait %steal %idle 4.26 0.00 20.63 38.61 0.00 36.50 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sda 14.22 224.17 200.88 650114 582570 sda1 0.03 1.14 0.00 3304 10 sda2 0.01 0.61 0.00 1760 0 sda3 14.15 222.13 200.88 644170 582560 sdb 15.49 630.73 97.01 1829122 281336 sdb1 15.47 630.44 97.01 1828306 281336 sdc 5.43 388.19 0.01 1125754 16 sdc1 5.41 387.91 0.01 1124938 16 sdd 95.29 2576.46 4429.77 7471802 12846464 sdd1 95.28 2576.18 4429.77 7470986 12846464 # free -m [r...@localhost ~]# free -m total used free shared buffers cached Mem: 3913 2290 1623 0 461 618 -/+ buffers/cache: 1210 2703 Swap: 8128 0 8128 # mount options /dev/sdb1 /cache1 ext4 defaults,noatime,nodiratime,commit=30 1 2 /dev/sdd1 /cache2 ext4 defaults,noatime,nodiratime,commit=30 1 2 # utlization [r...@localhost ~]# squidclient -p 3129 mgr:utilization Cache Utilisation: Last 5 minutes: sample_start_time = 1252077945.295770 (Fri, 04 Sep 2009 15:25:45 GMT) sample_end_time = 1252078245.298118 (Fri, 04 Sep 2009 15:30:45 GMT) client_http.requests = 192.955156/sec client_http.hits = 35.936385/sec client_http.errors = 0.000000/sec client_http.kbytes_in = 160.075414/sec client_http.kbytes_out = 4297.156368/sec client_http.all_median_svc_time = 0.378247 seconds client_http.miss_median_svc_time = 0.358317 seconds client_http.nm_median_svc_time = 0.232302 seconds client_http.nh_median_svc_time = 0.806511 seconds client_http.hit_median_svc_time = 0.649683 seconds server.all.requests = 151.812145/sec server.all.errors = 0.000000/sec server.all.kbytes_in = 3876.292995/sec server.all.kbytes_out = 146.765518/sec server.http.requests = 151.812145/sec server.http.errors = 0.000000/sec server.http.kbytes_in = 3876.296328/sec server.http.kbytes_out = 146.765518/sec server.ftp.requests = 0.000000/sec server.ftp.errors = 0.000000/sec server.ftp.kbytes_in = 0.000000/sec server.ftp.kbytes_out = 0.000000/sec server.other.requests = 0.000000/sec server.other.errors = 0.000000/sec server.other.kbytes_in = 0.000000/sec server.other.kbytes_out = 0.000000/sec icp.pkts_sent = 0.000000/sec icp.pkts_recv = 0.000000/sec icp.queries_sent = 0.000000/sec icp.replies_sent = 0.000000/sec icp.queries_recv = 0.000000/sec icp.replies_recv = 0.000000/sec icp.replies_queued = 0.000000/sec icp.query_timeouts = 0.000000/sec icp.kbytes_sent = 0.000000/sec icp.kbytes_recv = 0.000000/sec icp.q_kbytes_sent = 0.000000/sec icp.r_kbytes_sent = 0.000000/sec icp.q_kbytes_recv = 0.000000/sec icp.r_kbytes_recv = 0.000000/sec icp.query_median_svc_time = 0.000000 seconds icp.reply_median_svc_time = 0.000000 seconds dns.median_svc_time = 0.000000 seconds unlink.requests = 0.000000/sec page_faults = 0.000000/sec swap.outs = 30.246430/sec swap.ins = 48.952950/sec swap.files_cleaned = 0.333331/sec aborted_requests = 5.413291/sec cpu_time = 62.780456 seconds wall_time = 300.002348 seconds cpu_usage = 20.926655% Last 15 minutes: sample_start_time = 1252077345.259287 (Fri, 04 Sep 2009 15:15:45 GMT) sample_end_time = 1252078245.298118 (Fri, 04 Sep 2009 15:30:45 GMT) client_http.requests = 192.605023/sec client_http.hits = 36.387319/sec client_http.errors = 0.000000/sec client_http.kbytes_in = 215.286267/sec client_http.kbytes_out = 4463.820739/sec client_http.all_median_svc_time = 0.399283 seconds client_http.miss_median_svc_time = 0.399283 seconds client_http.nm_median_svc_time = 0.134979 seconds client_http.nh_median_svc_time = 0.898576 seconds client_http.hit_median_svc_time = 0.495759 seconds server.all.requests = 151.836782/sec server.all.errors = 0.000000/sec server.all.kbytes_in = 4026.657379/sec server.all.kbytes_out = 202.620147/sec server.http.requests = 151.824561/sec server.http.errors = 0.000000/sec server.http.kbytes_in = 4026.599603/sec server.http.kbytes_out = 202.605703/sec server.ftp.requests = 0.000000/sec server.ftp.errors = 0.000000/sec server.ftp.kbytes_in = 0.000000/sec server.ftp.kbytes_out = 0.000000/sec server.other.requests = 0.012222/sec server.other.errors = 0.000000/sec server.other.kbytes_in = 0.057775/sec server.other.kbytes_out = 0.015555/sec icp.pkts_sent = 0.000000/sec icp.pkts_recv = 0.000000/sec icp.queries_sent = 0.000000/sec icp.replies_sent = 0.000000/sec icp.queries_recv = 0.000000/sec icp.replies_recv = 0.000000/sec icp.replies_queued = 0.000000/sec icp.query_timeouts = 0.000000/sec icp.kbytes_sent = 0.000000/sec icp.kbytes_recv = 0.000000/sec icp.q_kbytes_sent = 0.000000/sec icp.r_kbytes_sent = 0.000000/sec icp.q_kbytes_recv = 0.000000/sec icp.r_kbytes_recv = 0.000000/sec icp.query_median_svc_time = 0.000000 seconds icp.reply_median_svc_time = 0.000000 seconds dns.median_svc_time = 0.000000 seconds unlink.requests = 0.000000/sec page_faults = 0.000000/sec swap.outs = 30.466463/sec swap.ins = 47.375734/sec swap.files_cleaned = 0.491090/sec aborted_requests = 5.685310/sec cpu_time = 186.805601 seconds wall_time = 900.038831 seconds cpu_usage = 20.755282% Last hour: (no values recorded yet) Last 8 hours: (no values recorded yet) Last day: (no values recorded yet) Last 3 days: (no values recorded yet) Totals since cache startup: sample_time = 1252078245.298118 (Fri, 04 Sep 2009 15:30:45 GMT) client_http.requests = 497585 client_http.hits = 98806 client_http.errors = 0 client_http.kbytes_in = 486898 client_http.kbytes_out = 10448627 client_http.hit_kbytes_out = 1022894 server.all.requests = 391925 server.all.errors = 0 server.all.kbytes_in = 9445666 server.all.kbytes_out = 453239 server.http.requests = 391907 server.http.errors = 0 server.http.kbytes_in = 9445583 server.http.kbytes_out = 453216 server.ftp.requests = 0 server.ftp.errors = 0 server.ftp.kbytes_in = 0 server.ftp.kbytes_out = 0 server.other.requests = 18 server.other.errors = 0 server.other.kbytes_in = 83 server.other.kbytes_out = 23 icp.pkts_sent = 0 icp.pkts_recv = 0 icp.queries_sent = 0 icp.replies_sent = 0 icp.queries_recv = 0 icp.replies_recv = 0 icp.query_timeouts = 0 icp.replies_queued = 0 icp.kbytes_sent = 0 icp.kbytes_recv = 0 icp.q_kbytes_sent = 0 icp.r_kbytes_sent = 0 icp.q_kbytes_recv = 0 icp.r_kbytes_recv = 0 unlink.requests = 0 page_faults = 9 cpu_time = 536.720405 wall_time = 7.108411 swap.outs = 73321 swap.ins = 103874 swap.files_cleaned = 1268 aborted_requests = 14161 #squid I/O [r...@localhost ~]# squidclient -p 3129 mgr:squidaio_counts ASYNC IO Counters: Operation # Requests open 138875 close 138624 cancel 138624 write 1892913 read 312034 stat 0 unlink 113294 check_callback 621451 queue 262 Threads Status: # ID # Requests 1 0x7f126111d910 97898 2 0x7f126115e910 93939 3 0x7f126119f910 101793 4 0x7f12611e0910 100535 5 0x7f1261221910 102801 6 0x7f1261262910 92777 7 0x7f12612a3910 104368 8 0x7f12612e4910 97976 9 0x7f1261325910 98596 10 0x7f1261366910 103603 11 0x7f12613a7910 112119 12 0x7f12613e8910 100345 13 0x7f1261429910 97499 14 0x7f126146a910 98678 15 0x7f12614ab910 102280 16 0x7f12614ec910 93246 17 0x7f126152d910 93879 18 0x7f126156e910 106020 19 0x7f12615af910 94895 20 0x7f12615f0910 112291 21 0x7f1261631910 110356 22 0x7f1261672910 97180 23 0x7f12616b3910 102370 24 0x7f12616f4910 100078 25 0x7f1261735910 91527 26 0x7f1261776910 88429 # squid.conf minimum_expiry_time 3600 seconds http_port 3129 transparent cache_mem 128 MB pipeline_prefetch off shutdown_lifetime 2 seconds coredump_dir /var/log/squid acl squidlocal src 127.0.0.1 acl all src 0.0.0.0/0.0.0.0 acl ourusers src xxx.xxx.xxx.0/24 hierarchy_stoplist cgi-bin ? acl nocache url_regex "/extra/nocache" cache deny nocache maximum_object_size 30720 MB minimum_object_size 1 KB maximum_object_size_in_memory 32 KB cache_replacement_policy heap LFUDA memory_replacement_policy heap GDSF cache_dir aufs /cache1 60000 64 256 cache_dir aufs /cache2 120000 128 256 range_offset_limit 0 KB cache_access_log /var/log/squid/access.log squid cache_log /var/log/squid/cache.log cache_store_log none allow_underscore on dns_timeout 1 minutes refresh_pattern ^ftp: 1440 60% 22160 refresh_pattern ^gopher: 1440 20% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 480 50% 22160 reload-into-ims negative_ttl 1 seconds positive_dns_ttl 60 hours persistent_request_timeout 30 seconds acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 563 acl Safe_ports port 1195 1107 1174 1212 1000 acl Safe_ports port 80 # http acl Safe_ports port 82 # http acl Safe_ports port 81 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 563 # https, snews acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT acl purge method PURGE acl apache rep_header Server ^Apache broken_vary_encoding allow apache http_access allow manager localhost http_access allow manager all http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow ourusers http_access allow squidlocal http_access deny all http_reply_access allow all cache allow all http_access allow purge localhost always_direct allow all never_direct deny all icp_access allow ourusers icp_access deny all cache_mgr info@ visible_hostname proxy dns_testnames localhost reload_into_ims on quick_abort_min 0 KB quick_abort_max 0 KB log_fqdn off half_closed_clients off client_db off ipcache_size 2048 ipcache_low 90 ipcache_high 95 fqdncache_size 8129 log_icp_queries off strip_query_terms off store_dir_select_algorithm least-load client_persistent_connections off server_persistent_connections on collapsed_forwarding on refresh_stale_hit 180 seconds forwarded_for off memory_pools off via off cache_effective_user squid cache_effective_group squid vary_ignore_expire on ignore_unknown_nameservers on uri_whitespace allow all request_header_max_size 120 KB update_headers off pid_filename /var/run/squid.pid -- View this message in context: http://www.nabble.com/I-O-Performance-Tuning-tp25307139p25307139.html Sent from the Squid - Users mailing list archive at Nabble.com.