Re: [squid-users] Squid slow down after awhile
On 15/10/17 05:20, masoud mazarei wrote: i sent you an pcap file which shows the problem. my client ip is 172.22.127.1 and target host is 94.182.227.21. squid machine mac is e4:11:5b:ea:30:c2. filter pcap file in wireshark by filter "ip.host==94.182.227.21" you will see that the first SYN packet arrived in No.304 and relative time 6.637173 but first packet which goes out from cache machine as client by squid happend in No.371 and relative time 45.013691 That kind of indicates the problem is either in how long the client takes to deliver the HTTP request to Squid, or DNS lookups to find the destination(s). what is happend in (6.637173 - 45.013691) duration.? For a transparent proxy these things have happened between SYN on client<->Squid and SYN on Squid<->server: * NAT/TPROXY record lookups for client connection state * wait for the client to send its HTTP request. - with happy eyeballs there may be a large wait between the SYN and first data sent by client for ~50% of connections. * parsing of that HTTP request message. * DNS lookup(s) for Host header verification * http_access checks * Adaptation hooks (ICAP / eCAP), if any * URL re-writer lookups, if any * HTTP 'cache' directive ACL checks * HTTP cache lookup * DNS lookups to find destination, if any - this should be very fast since the Host verify results should be cached. But if any of the above took longer than DNS TTL new lookups may be required - naturally increasing the delay further. * Destination selection * TCP server connection(s) setup - if you are only looking at IPv4 packets you may be missing multiple SYN packets for IPv6 servers before the first IPv4 SYN packet appears. The points above with sub-notes are the ones most likely to be delayed for seconds. You may be seeing one particular source of the problem, or multiple adding together. 45 sec seems an unusual number. Most of the timeouts in Squid and networking are multiples of 30 sec. which debug level will help me to know what is happend in background? i enabled "debug_options 5,3 6,3 46,3 11,3 19,3 55,3 58,3" BUT there is no valuable data to solve this problem. You may need an ALL,6 trace then to see if there are any clues in odd places. As verbose as it is the debugging in Squid is far from complete so for some of these delay issues there no specific lines to look for mention and we have to go by relative timing of things. The durations between actions on the list of points above should narrow down a bit better what to look at. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid slow down after awhile
TCP ESTABLISHED=54 TCP SYNC/WAIT/FIN=0 FD limitation=64000 net.core.rmem_max = 33554432 net.core.wmem_max = 33554432 net.ipv4.tcp_rmem = 4096 87380 33554432 net.ipv4.tcp_wmem = 4096 65536 33554432 net.ipv4.tcp_window_scaling=1 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_mem = 50576 64768 98152 net.core.netdev_max_backlog = 2500 net.ipv4.netfilter.ip_conntrack_max = 1048576 net.ipv4.ip_local_port_range = 1800065535 CISCO TCAM 10 mac learned(cache server mac,gateway mac ...) no forward loop no error in cache.log and syslog iptables totaly have 10 rows in all chains. and no mem usage which affects os and machine and i have only 1 user behind cache server. also i have checked with many users behind cache but no differ in result. i have same problem with squid 3.5 and squid 3.1 On Sat, Oct 14, 2017 at 12:36 PM, Amos Jeffrieswrote: > On 13/10/17 22:09, masoud mazarei wrote: > >> I setup squid as traspsrent proxy with wccp configuration(layer 2). >> All things work propery but after awhile it slows down. >> I try to findout whats going to happen using tcpdump >> And i know packets have delay to go out from squid machine. And I guess >> it may happened cause of tproxy config of linux kernel then i check linux >> kernel tproxy with >> https://github.com/LiamHaworth/go-tproxy >> And there is not delay to send packet out. All things work correctly with >> no delay. but when I stsrt to use squid after awhile I have delay in packet >> sending. >> Whats your opinion? >> >> > Insufficient data: > > * Check you WCCP TCAM table size. > > * Check your iptables/netfilter memory usage and table capacities. > > * Check your available TCP ports. > > * Check your Squid machines per-process FD limitations. As viewed by the > OS and what Squid thinks are available. > > * Check for forwarding loops (set 'via on' in squid.conf). > > * Check your logs for any issues that may be reported. Especially squid > cache.log and OS system log. > > > Amos > ___ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users > -- Yours Sincerely Masoud Mazarei ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Squid slow down after awhile
On 13/10/17 22:09, masoud mazarei wrote: I setup squid as traspsrent proxy with wccp configuration(layer 2). All things work propery but after awhile it slows down. I try to findout whats going to happen using tcpdump And i know packets have delay to go out from squid machine. And I guess it may happened cause of tproxy config of linux kernel then i check linux kernel tproxy with https://github.com/LiamHaworth/go-tproxy And there is not delay to send packet out. All things work correctly with no delay. but when I stsrt to use squid after awhile I have delay in packet sending. Whats your opinion? Insufficient data: * Check you WCCP TCAM table size. * Check your iptables/netfilter memory usage and table capacities. * Check your available TCP ports. * Check your Squid machines per-process FD limitations. As viewed by the OS and what Squid thinks are available. * Check for forwarding loops (set 'via on' in squid.conf). * Check your logs for any issues that may be reported. Especially squid cache.log and OS system log. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
[squid-users] Squid slow down after awhile
I setup squid as traspsrent proxy with wccp configuration(layer 2). All things work propery but after awhile it slows down. I try to findout whats going to happen using tcpdump And i know packets have delay to go out from squid machine. And I guess it may happened cause of tproxy config of linux kernel then i check linux kernel tproxy with https://github.com/LiamHaworth/go-tproxy And there is not delay to send packet out. All things work correctly with no delay. but when I stsrt to use squid after awhile I have delay in packet sending. Whats your opinion? -- Yours Sincerely Masoud Mazarei ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
[squid-users] Squid - slow download speeds....
Running CentOS 6.3 with Squid 3.1.10 and getting very slow download times - 20-35 k/sec. When I bypass the Squid server, I get 335 k/sec download times. I have tried bypassing caching the website I am testing to get my download numbers but still get about the same result thru the Squid. # Internet tester site (still get ~27 k/sec) acl tst dstdomain .testmy.net always_direct allow tst There are a bunch of threads out there but no real solution that I can find so far, anyone have this same problem and solution? __ NIPRO GLASS __ CAUTION - This message may contain privileged and confidential information intended only for the use of the addressee named above. If you are not the intended recipient of this message you are hereby notified that any use, dissemination, distribution or reproduction of this message is prohibited. If you have received this message in error please notify NIPRO GLASS immediately. Any views expressed in this message are those of the individual sender and may not necessarily reflect the views of NIPRO GLASS.
Re: [squid-users] Squid - slow download speeds....
On 10/9/2012 9:44 PM, Luderitz Bob wrote: Running CentOS 6.3 with Squid 3.1.10 and getting very slow download times - 20-35 k/sec. When I bypass the Squid server, I get 335 k/sec download times. I have tried bypassing caching the website I am testing to get my download numbers but still get about the same result thru the Squid. # Internet tester site (still get ~27 k/sec) acl tst dstdomain .testmy.net always_direct allow tst There are a bunch of threads out there but no real solution that I can find so far, anyone have this same problem and solution? The first thing is to use an updated version which suppose to be 3.1.21 to get most patches etc. Tests you should do anyhow are: test without any cache_dir but only with mem_cache. and if you will give some more data such as squid.conf and hardware will make things clearer. Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer at ngtech.co.il
Re: [squid-users] Squid slow performance
Are you able to perform a wget from the machine on a known fast source? Like a mirror to test the Internet connection... On Tue, Jul 24, 2012 at 10:38 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 24/07/2012 7:21 p.m., Alamgir Shamim wrote: Hello, We are using squid version squid-2.6.STABLE21-6.el5. But getting very poor performance. Our total Internet user are almost 750. At a time 500 to 600 user browse internet. all of them are getting slow response. it takes 10 to 30 second to load a page. Here I am giving you some out put. top out put... = top - 12:54:09 up 31 days, 5:59, 2 users, load average: 0.18, 0.24, 0.24 Tasks: 80 total, 1 running, 79 sleeping, 0 stopped, 0 zombie Cpu(s): 1.5%us, 0.2%sy, 0.0%ni, 93.5%id, 4.2%wa, 0.0%hi, 0.6%si, 0.0%st Mem: 2075016k total, 2020432k used,54584k free, 233784k buffers Swap: 2031608k total, 96k used, 2031512k free, 1538140k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 12866 squid 15 0 100m 70m 2508 S3 3.5 4:56.96 squid 6815 root 18 0 10936 4280 428 S0 0.2 0:00.00 squid free -m output [root@proxy253 squid]# free -m total used free sharedbuffers cached Mem: 2026 1973 53 0228 1502 -/+ buffers/cache:242 1783 Swap: 1983 0 1983 Processor infor == processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.20GHz stepping: 3 cpu MHz : 3200.343 cache size : 2048 KB physical id : 0 siblings: 2 processor : 1 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.20GHz stepping: 3 cpu MHz : 3200.343 cache size : 2048 KB physical id : 0 siblings: 2 - Could you please let me know what might be the reason for getting slow performance. We have 14 mbps of Internet bandwidth. We are doing some content filtering also with squid. There are several possible reasons: * you have configured squid in a inefficient way - I'm happy to do free performance audits here if you want to paste your squid.conf to the list (obscure the cachemgr_passwd detail though please) * you are content filtering - this is a major slowdown for Squid no matter what type of filtering is being done * overloaded or slow disks I/O speeds on disk cache - only relevant if you are disk caching * 2.6 will be adding to the slowdown - 2.7 series had a performance focus on development and is a good 20% faster just in the code. - 2.6 has low HTTP/1.1 compliance, meaning you loose out on many HTTP/1.1 performance features available in later releases. Amos -- Regards, Jason Leschnik. [m] 0432 35 4224 [w@] jason dot leschnik at ansto dot gov dot au [U@] jml...@uow.edu.au
[squid-users] Squid slow performance
Hello, We are using squid version squid-2.6.STABLE21-6.el5. But getting very poor performance. Our total Internet user are almost 750. At a time 500 to 600 user browse internet. all of them are getting slow response. it takes 10 to 30 second to load a page. Here I am giving you some out put. top out put... = top - 12:54:09 up 31 days, 5:59, 2 users, load average: 0.18, 0.24, 0.24 Tasks: 80 total, 1 running, 79 sleeping, 0 stopped, 0 zombie Cpu(s): 1.5%us, 0.2%sy, 0.0%ni, 93.5%id, 4.2%wa, 0.0%hi, 0.6%si, 0.0%st Mem: 2075016k total, 2020432k used,54584k free, 233784k buffers Swap: 2031608k total, 96k used, 2031512k free, 1538140k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 12866 squid 15 0 100m 70m 2508 S3 3.5 4:56.96 squid 2531 root 15 0 26744 5656 3328 S0 0.3 1:34.82 snmpd 2583 ntp 15 0 4448 4448 3440 S0 0.2 6:11.39 ntpd 6815 root 18 0 10936 4280 428 S0 0.2 0:00.00 squid 2781 haldaemo 19 0 5748 3728 1604 S0 0.2 0:00.78 hald 12716 root 15 0 10116 2808 2252 S0 0.1 0:02.98 sshd 12746 root 16 0 10116 2808 2252 S0 0.1 0:00.24 sshd 2565 root 18 0 9848 1972 1364 S0 0.1 0:00.00 cupsd 12718 root 15 0 4764 1512 1200 S0 0.1 0:00.25 bash 12750 root 15 0 4764 1512 1200 S0 0.1 0:00.05 bash 2756 avahi 18 0 2672 1328 1152 S0 0.1 0:00.16 avahi-daemon free -m output [root@proxy253 squid]# free -m total used free sharedbuffers cached Mem: 2026 1973 53 0228 1502 -/+ buffers/cache:242 1783 Swap: 1983 0 1983 Processor infor == processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.20GHz stepping: 3 cpu MHz : 3200.343 cache size : 2048 KB physical id : 0 siblings: 2 processor : 1 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.20GHz stepping: 3 cpu MHz : 3200.343 cache size : 2048 KB physical id : 0 siblings: 2 - Could you please let me know what might be the reason for getting slow performance. We have 14 mbps of Internet bandwidth. We are doing some content filtering also with squid. Please help us. Regards, --- Md. Alamgir Shamim (Tuhin) Senior Manager || Enterprise Network Technology Operations BRAC Bank Limited | Anik Tower 220/B, Tejgaon Industrial Area, Dhaka 1208, Bangladesh Tel : +88-02-8836302-5 Ext. 5368 | Fax : +88-02-9898910 GSM - 01713481059 URL - http://www.bracbank.com
Re: [squid-users] Squid slow performance
On 24/07/2012 7:21 p.m., Alamgir Shamim wrote: Hello, We are using squid version squid-2.6.STABLE21-6.el5. But getting very poor performance. Our total Internet user are almost 750. At a time 500 to 600 user browse internet. all of them are getting slow response. it takes 10 to 30 second to load a page. Here I am giving you some out put. top out put... = top - 12:54:09 up 31 days, 5:59, 2 users, load average: 0.18, 0.24, 0.24 Tasks: 80 total, 1 running, 79 sleeping, 0 stopped, 0 zombie Cpu(s): 1.5%us, 0.2%sy, 0.0%ni, 93.5%id, 4.2%wa, 0.0%hi, 0.6%si, 0.0%st Mem: 2075016k total, 2020432k used,54584k free, 233784k buffers Swap: 2031608k total, 96k used, 2031512k free, 1538140k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 12866 squid 15 0 100m 70m 2508 S3 3.5 4:56.96 squid 6815 root 18 0 10936 4280 428 S0 0.2 0:00.00 squid free -m output [root@proxy253 squid]# free -m total used free sharedbuffers cached Mem: 2026 1973 53 0228 1502 -/+ buffers/cache:242 1783 Swap: 1983 0 1983 Processor infor == processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.20GHz stepping: 3 cpu MHz : 3200.343 cache size : 2048 KB physical id : 0 siblings: 2 processor : 1 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.20GHz stepping: 3 cpu MHz : 3200.343 cache size : 2048 KB physical id : 0 siblings: 2 - Could you please let me know what might be the reason for getting slow performance. We have 14 mbps of Internet bandwidth. We are doing some content filtering also with squid. There are several possible reasons: * you have configured squid in a inefficient way - I'm happy to do free performance audits here if you want to paste your squid.conf to the list (obscure the cachemgr_passwd detail though please) * you are content filtering - this is a major slowdown for Squid no matter what type of filtering is being done * overloaded or slow disks I/O speeds on disk cache - only relevant if you are disk caching * 2.6 will be adding to the slowdown - 2.7 series had a performance focus on development and is a good 20% faster just in the code. - 2.6 has low HTTP/1.1 compliance, meaning you loose out on many HTTP/1.1 performance features available in later releases. Amos
Fw: [squid-users] squid slow response time
hello dear Amos and list finally i found the problem of slowing squid on linux the problem was not related to the squid. it was related to the File descriptor in linux operating system. as you know file descriptor in linux has a maximum value that indicates the maximum open files and sockets by the process. you can see the OS max file descriptor with the following command ulimit –n in my boxes ( 5 vps with centos 5.8 ) the maximum file descriptor was 1024. so the squid could not to open more files and sockets when reached the maximum open files and sockets. with increasing this value to 16384as shown echo fs.file-max = 64000 /etc/sysctl.conf echo * soft nofile 16384 /etc/security/limits.conf echo * hard nofile 16384 /etc/security/limits.conf echo ulimit -n 16384 /etc/profile and adding the line in squid config files max_filedesc 16384 and also changing the ephemeral port count as shown echo 1024 65000 /proc/sys/net/ipv4/ip_local_port_range and restarting the machine the problem of slow response time and slow squid was solved and squid is working so fast. before changing the above parameters squid could open about 570 connections. but now it could open more than 1800 connections. i suggest to mention above point in squid installation help for avoiding slow squid on servers with heavy traffic. thanks all for helping. From: Amos Jeffries squ...@treenet.co.nz To: squid-users@squid-cache.org Sent: Sunday, 27 May 2012, 15:45 Subject: Re: [squid-users] squid slow response time On 24/05/2012 10:05 a.m., Ali Esf wrote: hello list and hello dear Amos thanks for your help. some of my problems with squid are solved but some of them not. i compared squid on Linux Centos 5.8 with cc proxy on Microsoft windows server 2003 and understood that the ccproxy is more fast than squid on the same specification machine and supports more users. i captured the screen of the cc proxy and squid. http://up98.org/upload/server1/02/j/bpufq054uyf1qeamraj.jpg the above picture shows cc proxy on windows.as you see it supports 64 users and 1264 connections and even more. http://up98.org/upload/server1/02/j/kqlr5fcr2fvk1jafqva4.jpg the above picture shows port 9090 that is configed for http proxy by squid by netstat command. it shows there are 574 connections through port 9090 and squid. http://up98.org/upload/server1/02/j/hprnte4gldvsylb19xf.jpgthe above picture shows the number of users to port 9090 that are 37 users. Ah, I see. You are confusing users with TCP connections. There is no relationship in HTTP between number of users supported and number of connections supported. The number of TCP connections as measured by netstat has only one limit: 65535 TCP connections per receiving IP:port on the box. This will be true for both proxies I'm sure. What will be different is the HTTP keep-alive support. Which determines how and when connections are closed, and how many requests happen inside each before closure. Pipelining of requests also determines whether any requests are aborted and have to be retried. What you are looking at depends entirely on what those 1264/64 means to ccproxy. Is that 1264 authenticated users using 64 concurrent TCP connections? or 1264 TCP connections and 64 currently alive? or is it 1264 requests received over 64 TCP connections? Squid has a similar confusion looking solely at netstat numbers. One user can open 1 or more TCP connections, and any or none of them can be kept alive. The amount of speed slowdown you can expect from one or both depends entirely on the amount of requests sent over the TCP connection. Which is where the questions above come in very imporant: * Authenticating requires minimum 1-2 requests per user. * HTTP keep-alive feature permits one single TCP connection (netstat == 1) to handle many thousands or millions of requests. This will be different for each of the proxies, and depend on the type of requests being sent by the clients. Say for example, you have 1500 users. 1480 connect at once and both proxies handle them fast. those clients disconnect. *only* 1 of them connects later but this one has a virus. The infected user can turn on the PC, not even open the browser, and the virus open a TCP connection and fills it with 10,000,000 small HTTP requests. How long is it going to process and reject 10 million requests? netstat shows 1 connection total. For that period all other users will see degraded service to some degree. HTTP software (any type) is measured in requests-per-second as a simple consistant measure to avoid all this fuzzy boundaries and calculation issues. when the number of users increases the response time of squid become too slowly that sometimes takes 11 - 15 seconds to load the google web page. but i tested that the speed of download files through squid is great and the problem is when loading the pages when users
Re: [squid-users] squid slow response time
On 24/05/2012 10:05 a.m., Ali Esf wrote: hello list and hello dear Amos thanks for your help. some of my problems with squid are solved but some of them not. i compared squid on Linux Centos 5.8 with cc proxy on Microsoft windows server 2003 and understood that the ccproxy is more fast than squid on the same specification machine and supports more users. i captured the screen of the cc proxy and squid. http://up98.org/upload/server1/02/j/bpufq054uyf1qeamraj.jpg the above picture shows cc proxy on windows.as you see it supports 64 users and 1264 connections and even more. http://up98.org/upload/server1/02/j/kqlr5fcr2fvk1jafqva4.jpg the above picture shows port 9090 that is configed for http proxy by squid by netstat command. it shows there are 574 connections through port 9090 and squid. http://up98.org/upload/server1/02/j/hprnte4gldvsylb19xf.jpgthe above picture shows the number of users to port 9090 that are 37 users. Ah, I see. You are confusing users with TCP connections. There is no relationship in HTTP between number of users supported and number of connections supported. The number of TCP connections as measured by netstat has only one limit: 65535 TCP connections per receiving IP:port on the box. This will be true for both proxies I'm sure. What will be different is the HTTP keep-alive support. Which determines how and when connections are closed, and how many requests happen inside each before closure. Pipelining of requests also determines whether any requests are aborted and have to be retried. What you are looking at depends entirely on what those 1264/64 means to ccproxy. Is that 1264 authenticated users using 64 concurrent TCP connections? or 1264 TCP connections and 64 currently alive? or is it 1264 requests received over 64 TCP connections? Squid has a similar confusion looking solely at netstat numbers. One user can open 1 or more TCP connections, and any or none of them can be kept alive. The amount of speed slowdown you can expect from one or both depends entirely on the amount of requests sent over the TCP connection. Which is where the questions above come in very imporant: * Authenticating requires minimum 1-2 requests per user. * HTTP keep-alive feature permits one single TCP connection (netstat == 1) to handle many thousands or millions of requests. This will be different for each of the proxies, and depend on the type of requests being sent by the clients. Say for example, you have 1500 users. 1480 connect at once and both proxies handle them fast. those clients disconnect. *only* 1 of them connects later but this one has a virus. The infected user can turn on the PC, not even open the browser, and the virus open a TCP connection and fills it with 10,000,000 small HTTP requests. How long is it going to process and reject 10 million requests? netstat shows 1 connection total. For that period all other users will see degraded service to some degree. HTTP software (any type) is measured in requests-per-second as a simple consistant measure to avoid all this fuzzy boundaries and calculation issues. when the number of users increases the response time of squid become too slowly that sometimes takes 11 - 15 seconds to load the google web page. but i tested that the speed of download files through squid is great and the problem is when loading the pages when users get around 40. Hmm, 40 (clients) * 2 (FD per client) * 65536 (buffer bytes per connection) == 5242880 (bytes of buffer) + size of objects requested == ? how many MB/GB of RAM do you have free? DO NOT count swap and virtual memory, if Squid swaps the first thing to start thrashing I/O speed is the VMem pages used for memory cache and its index. and also in cc proxy with even 64 users and more the speed of loading pages is great.it is as like as there is no any proxy. the machines specification is the same and are : ram = 1 GB port = 1 Gbps cpu = Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, 2 cores The current stable release of Squid are single-core software. ccproxy has multi-core support. What versions you test is *important* when comparing these things. os = CentOS Linux 5.8 hard disk space = 30 GB we use squid just for proxy and not for catching. and need authentication just by user name and password through mysql database. here is the configuration:: cache deny all snip ... cache_mem 800 MB Um, you have caused Squid to allocate itself 800 MB of your 1024 MB on the box. Just for memory cache ... when caching there is disabled (deny all). Either remove the huge cache_mem allocation (non-caching proxy), or re-enable caching (cachign proxy) to see what Squid can actually do when sufficient RAM is available. Amos
[squid-users] squid slow response time
hello list and hello dear Amos thanks for your help. some of my problems with squid are solved but some of them not. i compared squid on Linux Centos 5.8 with cc proxy on Microsoft windows server 2003 and understood that the ccproxy is more fast than squid on the same specification machine and supports more users. i captured the screen of the cc proxy and squid. http://up98.org/upload/server1/02/j/bpufq054uyf1qeamraj.jpg the above picture shows cc proxy on windows.as you see it supports 64 users and 1264 connections and even more. http://up98.org/upload/server1/02/j/kqlr5fcr2fvk1jafqva4.jpg the above picture shows port 9090 that is configed for http proxy by squid by netstat command. it shows there are 574 connections through port 9090 and squid. http://up98.org/upload/server1/02/j/hprnte4gldvsylb19xf.jpgthe above picture shows the number of users to port 9090 that are 37 users. when the number of users increases the response time of squid become too slowly that sometimes takes 11 - 15 seconds to load the google web page. but i tested that the speed of download files through squid is great and the problem is when loading the pages when users get around 40. and also in cc proxy with even 64 users and more the speed of loading pages is great.it is as like as there is no any proxy. the machines specification is the same and are : ram = 1 GB port = 1 Gbps cpu = Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, 2 cores os = CentOS Linux 5.8 hard disk space = 30 GB we use squid just for proxy and not for catching. and need authentication just by user name and password through mysql database. here is the configuration:: cache deny all # # Recommended minimum configuration: # auth_param basic program /usr/local/squid/libexec/squid_db_auth --user squid_user --password user_password --plaintext --persist acl manager proto cache_object acl localhost src 127.0.0.1/32 ::1 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1 # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT acl user_pass_auth proxy_auth REQUIRED # replace 10.0.0.1 with your webserver IP # # Recommended minimum Access Permission configuration: # # Only allow cachemgr access from localhost http_access allow manager localhost http_access deny manager # Deny requests to certain unsafe ports http_access deny !Safe_ports #http_access deny CONNECT !SSL_Ports # Deny CONNECT to other than secure SSL ports # We strongly recommend the following be uncommented to protect innocent # web applications running on the proxy server who think the only # one who can access services on localhost is a local user #http_access deny to_localhost # # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS # # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from where browsing should be allowed http_access allow user_pass_auth http_access deny all access_log none cache_store_log none cache_log /dev/null dns_nameservers 4.2.2.4 8.8.8.8 # And finally deny all other access to this proxy # Squid normally listens to port 3128 http_port 9090 # Uncomment and adjust the following to add a disk cache directory. #cache_dir ufs /usr/local/squid/var/cache 100 16 256 # Leave coredumps in the first cache dir #coredump_dir /usr/local/squid/var/cache visible_hostname www.amirvpn.in # Add any of your own refresh_pattern entries above these. refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 cache_effective_user squid cache_effective_group squid cache_mem 800 MB regards ali
[squid-users] squid slow
i notice squid is slowing down..browsing goes very slow. real slow. anyway to boost it? how can i tweak the settings? thanks
Re: [squid-users] squid slow
Ivan Matala ivanmat...@gmail.com wrote on 2011-06-21 07:11: i notice squid is slowing down..browsing goes very slow. real slow. anyway to boost it? how can i tweak the settings? thanks Which version? What is very slow? Details of your network? --- Have a nice day. Joachim (Germany)
Re: [squid-users] squid slow
On 21/06/11 17:37, Joachim Wiedorn wrote: Ivan Matalaivanmat...@gmail.com wrote on 2011-06-21 07:11: i notice squid is slowing down..browsing goes very slow. real slow. anyway to boost it? how can i tweak the settings? thanks Which version? What is very slow? Details of your network? --- Have a nice day. Joachim (Germany) Ivan, I think the list needs some more detail than what you have provided. It's a bit like someone posting on a car forum with just my car is too slow, why? ;-) Most importantly: details of your hardware (especially memory and disk config), squid version, OS version, and the squid.conf of your server. There are many people that may offer help then. Cheers Alex
Re: [squid-users] squid slow
On Tue, 21 Jun 2011 07:11:17 -0700, Ivan Matala wrote: i notice squid is slowing down..browsing goes very slow. real slow. anyway to boost it? how can i tweak the settings? thanks Yes. Find out why. There are a number of different things it could be, including some DoS attacks starting to become popular against old or insecurely configured Squid. Amos
[squid-users] Squid slow on one site
Hello all, I'm using squid 2.6. My company using Microsoft CRM web service, the name crm.xyz.com I'm connect from the web address crm.xyz.com http://crm.xyz.com/ squid was recalled over 2.6 also comes as 14sn-15sn. however is stopped squid proxy and direct connect to MS CRM web site from adsl comes in 3sec - 5sec. Should I do to expedite this page? Thanks for your relations.
[squid-users] SQuid Slow
Dear all, Is there any way to fast the simple html pages. Currently everything working fine, but user complaint that it appear all html page at once which cause delay. So my question is it possible that as squid receive object it should display to user's, instead of display whole website at once. That’s how I guess user will be not boar if there browser's displaying object's. Require your help Regards Adnan
Re: [squid-users] SQuid Slow
tis 2010-03-16 klockan 17:22 +0500 skrev Adnan Shahzad: Is there any way to fast the simple html pages. Currently everything working fine, but user complaint that it appear all html page at once which cause delay. Squid relays data to the requesting browser as soon as it receives it. It does not buffer. The symptoms you describe has been reported before, but it's unclear what is causing them. But a guess is that it's related to the number of concurrent connections opened by the browser an possible presence of one or two slow objects early in the page delaying the browser from downloading the data it needs to start rendering the page. Regards Henrik
Re: [squid-users] SQuid Slow
On Tue, 16 Mar 2010 20:56:58 +0100, Henrik Nordström hen...@henriknordstrom.net wrote: tis 2010-03-16 klockan 17:22 +0500 skrev Adnan Shahzad: Is there any way to fast the simple html pages. Currently everything working fine, but user complaint that it appear all html page at once which cause delay. Squid relays data to the requesting browser as soon as it receives it. It does not buffer. The symptoms you describe has been reported before, but it's unclear what is causing them. But a guess is that it's related to the number of concurrent connections opened by the browser an possible presence of one or two slow objects early in the page delaying the browser from downloading the data it needs to start rendering the page. Regards Henrik I know some add-ons to Firefox which alter page content cause delays while they buffer the entire thing re-write it before display. IE6 also has something which can cause it to act the same, holding a page HTML from display until all objects have been received (preventing display if some fail!). In Squid the chunked decoder needs to buffer the reply to decode it for clients, and for some very large objects can be delayed. Mostly seen with large images such as google maps/earth/streetview. But may also occur if the server is slow and HTML page very long. Amos
Re: [squid-users] SQuid Slow
tis 2010-03-16 klockan 21:12 + skrev Amos Jeffries: In Squid the chunked decoder needs to buffer the reply to decode it for clients It should not, just the chunk header needs buffering if split across reads, not the chunk data. Regards Henrik
Re: [squid-users] squid slow
Muhammad Sharfuddin wrote: On Mon, 2009-08-24 at 17:05 +1200, Amos Jeffries wrote: On Mon, 24 Aug 2009 10:24:41 +0600, Muhammad Sharfuddin m.sharfud...@nds.com.pk wrote: Note: the netfilter guys recommend using the iptables-restore tool for firewall setup. It's much faster and much more secure than an incremental build of the rules like this. Ok, I will try. NOTE: The following rules only apply to external people attempting to connect to your internal LAN machines. ... Or to people using your proxy as a free gateway to elsewhere on the Internet. They can do that to your proxy by simply sending an HTTP request to any one of your internal LAN IPs with a forged HTTP header and URL. I think only the following rule is for anyone(internal/external) acl allowed_for_all url_regex -i /etc/squid/allowed_for_all.txt http_access allow allowed_for_all acl ftp_site url_regex -i ftp://ftp.sight-board.de http_access allow ftp_site all the other rules are *only* for specific machines/IPes e.g acl hod_ip src /etc/squid/ipes/hod_ip.txt http_access allow hod_ip acl cad_ip src /etc/squid/ipes/cad_ip.txt http_access deny cad_ip acl hod_tl_ip src /etc/squid/ipes/hod_and_tl_ip.txt http_access allow hod_tl_ip So I really dont understand why you said/wrote 'The following rules *ONLY* apply to external people' Because you allow localnet (AKA unrestricted access to all internal client) before doing those rules. Regardless of what they are they will only be tested against the requests coming from outside your localnet defined local network ranges. cache_dir diskd /var/cache/squid 5 16 256 diskd is probably your problem. From the use of iptables as a firewall I would guess that this is a linux box. On linux you should try AUFS storage for fastest speed. If that label is the only change on the config line you can test it with a simple re-config. well same results with aufs. you are recommending aufs over diskd, and the following url suggest 'diskd' as the the store type of choice for the Cache-off's http://www.linuxsa.org.au/pipermail/linuxsa/2004-June/070228.html Written in 2004. Server CPU threading has come a long way since then. diskd is a single-threaded helper application, with a processing IO _upper_ limit of 1 file read at a time. Squid itself does not block, but the helper reads/writes are blocking each other _within the helper_. AUFS is a multi-threaded component utilizing the kernel and all available CPU for non-blocking read/write to as many files as needed simultaneously. Limits are defined by the available FD in Squid and system CPU capabilities. diskd is only recommended for use on *BSD systems where AUFS support is not available (yet). Also with ~50GB of storage you are probably wanting to use something like 32 or 64 for the Level-1 value (currently 16). Changing that requires a cache delete and rebuild with 'squid -z' though. whats the rule/formula for Level-1 and Level-2 value ? is it related with storage size ? Yes. OS used to have an upper limit on the number of files stored in a single directory. I think most still do for the common filing systems. Between them these numbers define how many folders are used in the cache. Smaller caches only need a few folders, bigger caches need a lot more to keep the OS happy. The default squid.conf comes tuned for a 200MB cache. Quite small for any real use. When you are heading into tens of GB its a good idea to start upping these numbers. How much depends on your OS filesystem and avg. object size in the cache. Big and huge objects obviously reduce the pressure for extra folders. These days I'm advising people terminate their file extension patterns with (\?.*)?$ instead of just $ to catch all the sites using dynamic parts in their URLs. you mean the following ? (\?.swf)?$ (\?.mdi)?$ e.g refresh_pattern -i (\?.swf)?$ 43200 100% 43200 override-lastmod override-expire No, no. This: refresh_pattern -i \.swf(\?.*)?$ Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE18 Current Beta Squid 3.1.0.13
Re: [squid-users] squid slow
On Mon, 2009-08-24 at 18:30 +1200, Amos Jeffries wrote: Muhammad Sharfuddin wrote: On Mon, 2009-08-24 at 17:05 +1200, Amos Jeffries wrote: On Mon, 24 Aug 2009 10:24:41 +0600, Muhammad Sharfuddin m.sharfud...@nds.com.pk wrote: So I really dont understand why you said/wrote 'The following rules *ONLY* apply to external people' Because you allow localnet (AKA unrestricted access to all internal client) before doing those rules. Regardless of what they are they will only be tested against the requests coming from outside your localnet defined local network ranges. thanks in squid.conf #http_access allow localnet then # squid -k reconfigure you are recommending aufs over diskd, and the following url suggest 'diskd' as the the store type of choice for the Cache-off's http://www.linuxsa.org.au/pipermail/linuxsa/2004-June/070228.html Written in 2004. Server CPU threading has come a long way since then. diskd is a single-threaded helper application, with a processing IO _upper_ limit of 1 file read at a time. Squid itself does not block, but the helper reads/writes are blocking each other _within the helper_. AUFS is a multi-threaded component utilizing the kernel and all available CPU for non-blocking read/write to as many files as needed simultaneously. Limits are defined by the available FD in Squid and system CPU capabilities. diskd is only recommended for use on *BSD systems where AUFS support is not available (yet). Ok, nice and thanks Also with ~50GB of storage you are probably wanting to use something like 32 or 64 for the Level-1 value (currently 16). Changing that requires a cache delete and rebuild with 'squid -z' though. whats the rule/formula for Level-1 and Level-2 value ? is it related with storage size ? Yes. OS used to have an upper limit on the number of files stored in a single directory. I think most still do for the common filing systems. Between them these numbers define how many folders are used in the cache. Smaller caches only need a few folders, bigger caches need a lot more to keep the OS happy. The default squid.conf comes tuned for a 200MB cache. Quite small for any real use. When you are heading into tens of GB its a good idea to start upping these numbers. How much depends on your OS filesystem and avg. object size in the cache. Big and huge objects obviously reduce the pressure for extra folders. cache_dir aufs /var/cache/squid 5 32 256 so the above one is OK, or should I also need to increase the Level-2 value(256) ? By the way whats the best file system for caching reiserfs, ext3, or XFS ( I am using reiserfs with 'notail,noatime,noacl' options) These days I'm advising people terminate their file extension patterns with (\?.*)?$ instead of just $ to catch all the sites using dynamic parts in their URLs. you mean the following ? (\?.swf)?$ (\?.mdi)?$ e.g refresh_pattern -i (\?.swf)?$ 43200 100% 43200 override-lastmod override-expire No, no. This: refresh_pattern -i \.swf(\?.*)?$ Thanks Regards --ms
[squid-users] squid slow
Squid Version 2.7.STABLE5 SUSE Enterprise 11 i586 browsing is slow if use squid as a proxy server. on a MS Win 2003 SP2 x864, we installed a small proxy software(cc proxy), and then did the browsing, the result was good/excellent. please help/suggest details: two nics, one connected with DSL modem(eth1), and one connected with our local lan(eth0) runlevel script for routing and firewalling: # cat /etc/init.d/routing #!/bin/bash # /etc/init.d/routing ### BEGIN INIT INFO # Provides: routing # Required-Start: $network # Required-Stop: # Default-Start: 3 5 # Default-Stop: # Description:set up routing and Firewalling ### END INIT INFO # squid server IP SQUID_SERVER=192.168.0.1 # Interface connected to Internet INTERNET=eth1 # Interface connected to LAN LAN_IN=eth0 # Squid port SQUID_PORT=8080 # Load IPTABLES modules for NAT and IP conntrack support modprobe ip_conntrack modprobe ip_conntrack_ftp # For win xp ftp client modprobe ip_nat_ftp . /etc/rc.status # First reset status of this service rc_reset case $1 in stop) echo -n Stoping Firewall # disaable routing echo 0 /proc/sys/net/ipv4/ip_forward # DO NOT MODIFY BELOW # Clean old firewall iptables -F iptables -X iptables -t nat -F iptables -t nat -X # Remember status and be verbose rc_status -v ;; start) echo -n Starting Firewall and router # enable routing echo 1 /proc/sys/net/ipv4/ip_forward # Setting default filter policy iptables -P INPUT DROP iptables -P OUTPUT ACCEPT # Unlimited access to loop back iptables -A INPUT -i lo -j ACCEPT iptables -A OUTPUT -o lo -j ACCEPT # To block Msn Messenger iptables -A INPUT -p tcp --dport 1863 -j DROP iptables -A FORWARD -o $INTERNET -i $LAN_IN -d 0/0 -p tcp --dport 1863 -j DROP iptables -A FORWARD -o $INTERNET -i $LAN_IN -d 0/0 -p udp --dport 1863 -j DROP # Allow unlimited access to $INTERNET from our local lan iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT # set this system as a router for Rest of LAN iptables --table nat --append POSTROUTING --out-interface $INTERNET -j MASQUERADE iptables --append FORWARD --in-interface $LAN_IN -j ACCEPT # unlimited access to LAN iptables -A INPUT -i $LAN_IN -j ACCEPT iptables -A OUTPUT -o $LAN_IN -j ACCEPT ### for Win2k3 SP1 virtual machine iptables -A INPUT -s 192.168.1.3 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT ### # DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy iptables -t nat -A PREROUTING -i $LAN_IN -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT # if it is same system iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT # DROP everything and Log it iptables -A INPUT -j LOG iptables -A INPUT -j DROP # Remember status and be verbose rc_status -v ;; restart) ## Stop the service and regardless of whether it was ## running or not, start it again. $0 stop $0 start # Remember status and be quiet rc_status ;; esac rc_exit Name Resolution: -- # cat /etc/resolv.conf nameserver 127.0.0.1 # grep forward* /etc/named.conf forward first; forwarders { 202.163.96.3; 202.163.96.4; }; squid.conf -- # grep -v ^# /etc/squid/squid.conf acl all src all acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT acl CON_SKYPE method CONNECT acl skype url_regex ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9] http_access allow CON_SKYPE skype http_access allow manager localhost http_access deny manager http_access deny !Safe_ports # * acl ftp_software dstdomain .sight-board.de http_access deny CONNECT !SSL_ports !ftp_software # * http_access allow localnet http_access allow localhost acl porn_web url_regex -i /etc/squid/blacklists/porn/domains.txt http_access deny porn_web acl hod_ip src /etc/squid/ipes/hod_ip.txt
Re: [squid-users] squid slow
On Mon, 24 Aug 2009 10:24:41 +0600, Muhammad Sharfuddin m.sharfud...@nds.com.pk wrote: Squid Version 2.7.STABLE5 SUSE Enterprise 11 i586 browsing is slow if use squid as a proxy server. on a MS Win 2003 SP2 x864, we installed a small proxy software(cc proxy), and then did the browsing, the result was good/excellent. please help/suggest details: two nics, one connected with DSL modem(eth1), and one connected with our local lan(eth0) runlevel script for routing and firewalling: # cat /etc/init.d/routing #!/bin/bash # /etc/init.d/routing ### BEGIN INIT INFO # Provides: routing # Required-Start: $network # Required-Stop: # Default-Start: 3 5 # Default-Stop: # Description:set up routing and Firewalling ### END INIT INFO # squid server IP SQUID_SERVER=192.168.0.1 # Interface connected to Internet INTERNET=eth1 # Interface connected to LAN LAN_IN=eth0 # Squid port SQUID_PORT=8080 # Load IPTABLES modules for NAT and IP conntrack support modprobe ip_conntrack modprobe ip_conntrack_ftp # For win xp ftp client modprobe ip_nat_ftp . /etc/rc.status # First reset status of this service rc_reset case $1 in stop) echo -n Stoping Firewall # disaable routing echo 0 /proc/sys/net/ipv4/ip_forward # DO NOT MODIFY BELOW # Clean old firewall iptables -F iptables -X iptables -t nat -F iptables -t nat -X # Remember status and be verbose rc_status -v ;; start) echo -n Starting Firewall and router Note: the netfilter guys recommend using the iptables-restore tool for firewall setup. It's much faster and much more secure than an incremental build of the rules like this. # enable routing echo 1 /proc/sys/net/ipv4/ip_forward # Setting default filter policy iptables -P INPUT DROP iptables -P OUTPUT ACCEPT # Unlimited access to loop back iptables -A INPUT -i lo -j ACCEPT iptables -A OUTPUT -o lo -j ACCEPT # To block Msn Messenger iptables -A INPUT -p tcp --dport 1863 -j DROP iptables -A FORWARD -o $INTERNET -i $LAN_IN -d 0/0 -p tcp --dport 1863 -j DROP iptables -A FORWARD -o $INTERNET -i $LAN_IN -d 0/0 -p udp --dport 1863 -j DROP # Allow unlimited access to $INTERNET from our local lan iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT # set this system as a router for Rest of LAN iptables --table nat --append POSTROUTING --out-interface $INTERNET -j MASQUERADE iptables --append FORWARD --in-interface $LAN_IN -j ACCEPT # unlimited access to LAN iptables -A INPUT -i $LAN_IN -j ACCEPT iptables -A OUTPUT -o $LAN_IN -j ACCEPT ### for Win2k3 SP1 virtual machine iptables -A INPUT -s 192.168.1.3 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT ### # DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy iptables -t nat -A PREROUTING -i $LAN_IN -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT # if it is same system iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT # DROP everything and Log it iptables -A INPUT -j LOG iptables -A INPUT -j DROP # Remember status and be verbose rc_status -v ;; restart) ## Stop the service and regardless of whether it was ## running or not, start it again. $0 stop $0 start # Remember status and be quiet rc_status ;; esac rc_exit Name Resolution: -- # cat /etc/resolv.conf nameserver 127.0.0.1 # grep forward* /etc/named.conf forward first; forwarders { 202.163.96.3; 202.163.96.4; }; squid.conf -- # grep -v ^# /etc/squid/squid.conf acl all src all acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12# RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80# http acl Safe_ports port 21# ftp acl Safe_ports port 443 # https acl Safe_ports port 70# gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535# unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT acl CON_SKYPE
Re: [squid-users] squid slow
On Mon, 2009-08-24 at 17:05 +1200, Amos Jeffries wrote: On Mon, 24 Aug 2009 10:24:41 +0600, Muhammad Sharfuddin m.sharfud...@nds.com.pk wrote: Note: the netfilter guys recommend using the iptables-restore tool for firewall setup. It's much faster and much more secure than an incremental build of the rules like this. Ok, I will try. NOTE: The following rules only apply to external people attempting to connect to your internal LAN machines. ... Or to people using your proxy as a free gateway to elsewhere on the Internet. They can do that to your proxy by simply sending an HTTP request to any one of your internal LAN IPs with a forged HTTP header and URL. I think only the following rule is for anyone(internal/external) acl allowed_for_all url_regex -i /etc/squid/allowed_for_all.txt http_access allow allowed_for_all acl ftp_site url_regex -i ftp://ftp.sight-board.de http_access allow ftp_site all the other rules are *only* for specific machines/IPes e.g acl hod_ip src /etc/squid/ipes/hod_ip.txt http_access allow hod_ip acl cad_ip src /etc/squid/ipes/cad_ip.txt http_access deny cad_ip acl hod_tl_ip src /etc/squid/ipes/hod_and_tl_ip.txt http_access allow hod_tl_ip So I really dont understand why you said/wrote 'The following rules *ONLY* apply to external people' cache_dir diskd /var/cache/squid 5 16 256 diskd is probably your problem. From the use of iptables as a firewall I would guess that this is a linux box. On linux you should try AUFS storage for fastest speed. If that label is the only change on the config line you can test it with a simple re-config. well same results with aufs. you are recommending aufs over diskd, and the following url suggest 'diskd' as the the store type of choice for the Cache-off's http://www.linuxsa.org.au/pipermail/linuxsa/2004-June/070228.html Also with ~50GB of storage you are probably wanting to use something like 32 or 64 for the Level-1 value (currently 16). Changing that requires a cache delete and rebuild with 'squid -z' though. whats the rule/formula for Level-1 and Level-2 value ? is it related with storage size ? These days I'm advising people terminate their file extension patterns with (\?.*)?$ instead of just $ to catch all the sites using dynamic parts in their URLs. you mean the following ? (\?.swf)?$ (\?.mdi)?$ e.g refresh_pattern -i (\?.swf)?$ 43200 100% 43200 override-lastmod override-expire Regards --ms
[squid-users] Squid Slow Downloads problem--large files
Help ! Anyone got ideas ? Best Regards, Adam --邮件原件- -发件人: adam.cheng [mailto:[EMAIL PROTECTED] -发送时间: 2006年7月18日 11:54 -收件人: squid-users@squid-cache.org -主题: [squid-users] Squid Slow Downloads problem--large files - -Hi, squid-user - -In my test , Apache is much more faster than squid when there are about -30Mbps load. - -brief result: - ---large file download testing (10M and 40M) ---Current load: 15Mbps (squid service) ---IOwait: 6% ---test result: (same box, same environment) - Apache: 60~70Mbytes/s - Squid: 700~900Kbytes/s (HIT from squid cache) - -Detail information listed below. - --- -- - --Hi, squid-users: -- --I have met a slow download problem of squid , would anybody like to tell -me what’s --the matter with my squid Or is there any way to resolve this problem ? -- --Squid info: -- --[EMAIL PROTECTED] ~]# squid -v --Squid Cache: Version 2.5.STABLE12 --configure options: --prefix=/usr/local/squid --enable-epoll ---disable-ident-lookups enable-async-io=160 --enable-storeio=ufs,aufs,diskd --enable-snmp enable-cache-digests --enable-useragent-log --enable-referer-log enable-kill-parent-hack --enable--internal-dns -- -- --squid.conf: -- --http_port 80 --icp_port 0 --acl httpmp3 url_regex -i ^http://.*\.mp3$ --no_cache deny httpmp3 --acl httpwmv url_regex -i ^http://.*\.wmv$ --no_cache deny httpwmv --acl httprm url_regex -i ^http://.*\.rm$ --no_cache deny httprm -- cache_mem 1768 MB -- cache_swap_low 70 -- cache_swap_high 80 --maximum_object_size 204800 KB --minimum_object_size 0 KB --maximum_object_size_in_memory 102400 KB -- cache_replacement_policy lru -- memory_replacement_policy lru --cache_dir diskd /data/cache1 28000 16 256 --cache_dir diskd /data/cache2 28000 16 256 --logformat squid_custom_log %ts.%03tu %6tr %a %Ss/%03Hs %st %rm -%ru --%un %Sh/%A %mt %{Referer}h %{User-Agent}h %{Cookie}h --cache_access_log /data/proclog/log/squid/access.log squid_custom_log --cache_log /data/proclog/log/squid/cache.log --cache_store_log none --pid_filename /var/run/squid.pid -- hosts_file /etc/hosts -- diskd_program /usr/local/squid/libexec/diskd -- unlinkd_program /usr/local/squid/libexec/unlinkd -- -- --refresh_pattern -i ^http://player.toodou.com.*2073600 100% -2073600 --ignore-reload --refresh_pattern -i ^http://www.blogcn.com.*1440 50% 1440 --refresh_pattern -i ^http://images.blogcn.com.*1440 50% 1440 --refresh_pattern -i ^http://female.blogcn.com.*1440 50% 1440 --refresh_pattern -i ^http://img.365ren.com.* 720 100% 720 --refresh_pattern -i ^http://cfs1.365ren.com.*720 100% 720 --refresh_pattern -i ^http://cafe-img.365ren.com.*720 100% 720 --refresh_pattern -i ^http://cafe-cfs1.365ren.com.*720 100% 720 --refresh_pattern -i ^http60 0% 60 ignore-reload --collapsed_forwarding on --refresh_stale_hit 0 minute --request_timeout 30 seconds -- persistent_request_timeout 3 seconds -- pconn_timeout 60 seconds --acl all src 0.0.0.0/0.0.0.0 --acl manager proto cache_object --acl localhost src 127.0.0.1/255.255.255.255 --acl to_localhost dst 127.0.0.0/8 --acl SSL_ports port 443 563 --acl Safe_ports port 80 # http --acl Safe_ports port 21 # ftp --acl Safe_ports port 443 563 # https, snews --acl Safe_ports port 70 # gopher --acl Safe_ports port 210 # wais --acl Safe_ports port 1025-65535 # unregistered ports --acl Safe_ports port 280 # http-mgmt --acl Safe_ports port 488 # gss-http --acl Safe_ports port 591 # filemaker --acl Safe_ports port 777 # multiling http --acl CONNECT method CONNECT --acl monitor src 192.168.1.0/255.255.255.0 --http_access allow manager --http_access allow manager monitor --http_access deny manager --acl PURGE method PURGE --http_access allow PURGE localhost --http_access deny purge --acl snmppublic snmp_community public --snmp_access allow snmppublic localhost --http_access deny !Safe_ports --http_access deny CONNECT !SSL_ports --http_access allow all --http_reply_access allow all -- cache_mgr [EMAIL PROTECTED] -- cache_effective_user squid -- cache_effective_group squid --visible_hostname CHN-SH-3-341 --httpd_accel_host virtual --httpd_accel_port 80 --httpd_accel_single_host off --httpd_accel_with_proxy off --httpd_accel_uses_host_header on --dns_testnames original1.chinacache.com original2.chinacache.com -- logfile_rotate 0 -- --cachemgr_passwd test4squid config -- store_avg_object_size 20 KB --client_db off --header_access X-Cache-Lookup deny all --snmp_port 3401 --acl snmppublic snmp_community public -- client_persistent_connections off -- server_persistent_connections off --vary_ignore_expire on --strip_query_terms off --negative_ttl 0 minute --dns_retransmit_interval 10 seconds --store_dir_select_algorithm round-robin --dns_timeout 2 minute --negative_dns_ttl 1 minute
[squid-users] Squid Slow Downloads problem--large files
Hi, squid-user In my test , Apache is much more faster than squid when there are about 30Mbps load. brief result: --large file download testing (10M and 40M) --Current load: 15Mbps (squid service) --IOwait: 6% --test result: (same box, same environment) Apache: 60~70Mbytes/s Squid: 700~900Kbytes/s (HIT from squid cache) Detail information listed below. -Hi, squid-users: - -I have met a slow download problem of squid , would anybody like to tell me what’s -the matter with my squid Or is there any way to resolve this problem ? - -Squid info: - -[EMAIL PROTECTED] ~]# squid -v -Squid Cache: Version 2.5.STABLE12 -configure options: --prefix=/usr/local/squid --enable-epoll --disable-ident-lookups ---enable-async-io=160 --enable-storeio=ufs,aufs,diskd --enable-snmp ---enable-cache-digests --enable-useragent-log --enable-referer-log ---enable-kill-parent-hack --enable--internal-dns --- - - -squid.conf: - -http_port 80 -icp_port 0 -acl httpmp3 url_regex -i ^http://.*\.mp3$ -no_cache deny httpmp3 -acl httpwmv url_regex -i ^http://.*\.wmv$ -no_cache deny httpwmv -acl httprm url_regex -i ^http://.*\.rm$ -no_cache deny httprm - cache_mem 1768 MB - cache_swap_low 70 - cache_swap_high 80 -maximum_object_size 204800 KB -minimum_object_size 0 KB -maximum_object_size_in_memory 102400 KB - cache_replacement_policy lru - memory_replacement_policy lru -cache_dir diskd /data/cache1 28000 16 256 -cache_dir diskd /data/cache2 28000 16 256 -logformat squid_custom_log %ts.%03tu %6tr %a %Ss/%03Hs %st %rm %ru -%un %Sh/%A %mt %{Referer}h %{User-Agent}h %{Cookie}h -cache_access_log /data/proclog/log/squid/access.log squid_custom_log -cache_log /data/proclog/log/squid/cache.log -cache_store_log none -pid_filename /var/run/squid.pid - hosts_file /etc/hosts - diskd_program /usr/local/squid/libexec/diskd - unlinkd_program /usr/local/squid/libexec/unlinkd - - -refresh_pattern -i ^http://player.toodou.com.*2073600 100% 2073600 -ignore-reload -refresh_pattern -i ^http://www.blogcn.com.*1440 50% 1440 -refresh_pattern -i ^http://images.blogcn.com.*1440 50% 1440 -refresh_pattern -i ^http://female.blogcn.com.*1440 50% 1440 -refresh_pattern -i ^http://img.365ren.com.* 720 100% 720 -refresh_pattern -i ^http://cfs1.365ren.com.*720 100% 720 -refresh_pattern -i ^http://cafe-img.365ren.com.*720 100% 720 -refresh_pattern -i ^http://cafe-cfs1.365ren.com.*720 100% 720 -refresh_pattern -i ^http60 0% 60 ignore-reload -collapsed_forwarding on -refresh_stale_hit 0 minute -request_timeout 30 seconds - persistent_request_timeout 3 seconds - pconn_timeout 60 seconds -acl all src 0.0.0.0/0.0.0.0 -acl manager proto cache_object -acl localhost src 127.0.0.1/255.255.255.255 -acl to_localhost dst 127.0.0.0/8 -acl SSL_ports port 443 563 -acl Safe_ports port 80 # http -acl Safe_ports port 21 # ftp -acl Safe_ports port 443 563 # https, snews -acl Safe_ports port 70 # gopher -acl Safe_ports port 210 # wais -acl Safe_ports port 1025-65535 # unregistered ports -acl Safe_ports port 280 # http-mgmt -acl Safe_ports port 488 # gss-http -acl Safe_ports port 591 # filemaker -acl Safe_ports port 777 # multiling http -acl CONNECT method CONNECT -acl monitor src 192.168.1.0/255.255.255.0 -http_access allow manager -http_access allow manager monitor -http_access deny manager -acl PURGE method PURGE -http_access allow PURGE localhost -http_access deny purge -acl snmppublic snmp_community public -snmp_access allow snmppublic localhost -http_access deny !Safe_ports -http_access deny CONNECT !SSL_ports -http_access allow all -http_reply_access allow all - cache_mgr [EMAIL PROTECTED] - cache_effective_user squid - cache_effective_group squid -visible_hostname CHN-SH-3-341 -httpd_accel_host virtual -httpd_accel_port 80 -httpd_accel_single_host off -httpd_accel_with_proxy off -httpd_accel_uses_host_header on -dns_testnames original1.chinacache.com original2.chinacache.com - logfile_rotate 0 - -cachemgr_passwd test4squid config - store_avg_object_size 20 KB -client_db off -header_access X-Cache-Lookup deny all -snmp_port 3401 -acl snmppublic snmp_community public - client_persistent_connections off - server_persistent_connections off -vary_ignore_expire on -strip_query_terms off -negative_ttl 0 minute -dns_retransmit_interval 10 seconds -store_dir_select_algorithm round-robin -dns_timeout 2 minute -negative_dns_ttl 1 minute -connect_timeout 30 seconds -read_timeout 15 minutes - --- - - -Test information: (all test was done to the same box, ) - -Squid with 80 port: - -[EMAIL PROTECTED] ~]# wget -SO test -http://player.toodou.com/flv/001/126/216/1126216.flv ---18:56:54--
[squid-users] squid slow for one particular web site
Hi All, I have a squid proxy (Squid Cache: Version 2.5.STABLE10) running on a Open BSD server. I am having a problem with one particular web site http://62.17.163.240/. When users try to access this weeb site it takes 6 to 8 seconds to download each page. Where as if we try to access any other web site using the proxy it takes only a second to download the web page. If we by pass the proxy and access the web site http://62.17.163.240/ it's pretty quick. I am not able to figure out why only this particular web site. It should cache the web site and when you try to access this weeb site 2nd or 3rd time it should get it from the cache instead of going to the origin server. What can I do to fix the problem? Below are the logs: 1141028847.893772 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/common.js - DIRECT/62.17.163.240 - 1141028848.412 3382 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/interface_images/generic/arrowr_r.gif - DIRECT/62.17.163.240 - 1141028851.208 3315 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/browserfuncs.js - DIRECT/62.17.163.240 - 1141028852.022814 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/page.js - DIRECT/62.17.163.240 - 1141028852.848821 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/spi.js - DIRECT/62.17.163.240 - 1141028853.690842 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/generic_oce.js - DIRECT/62.17.163.240 - 1141028854.513805 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/communicate.js - DIRECT/62.17.163.240 - 1141028855.486972 172.26.11.50 TCP_MISS/304 286 GET http://62.17.163.240/lo_0668/jscripts/scofind.js - DIRECT/62.17.163.240 - 1141028856.273788 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/interface_images/generic/arrowl.gif - DIRECT/62.17.163.240 - 1141028856.458946 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/interface_images/generic/arrowr.gif - DIRECT/62.17.163.240 - 1141028857.266793 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/interface_images/generic/pix_transp.gif - DIRECT/62.17.163.240 - 1141028857.268791 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/interface_images/generic/42.gif - DIRECT/62.17.163.240 - 1141028857.450992 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/interface_images/generic/pix_transp.gif - DIRECT/62.17.163.240 - 1141028858.074767 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/interface_images/generic/arrowl_r.gif - DIRECT/62.17.163.240 - 1141028858.243975 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/interface_images/generic/43.gif - DIRECT/62.17.163.240 - 1141028860.174 3701 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/interface_images/generic/pix_transp.gif - DIRECT/62.17.163.240 - 1141028860.475 35 172.26.11.50 TCP_IMS_HIT/304 216 GET http://62.17.163.240/lo_0668/interface_images/generic/arrowr_r.gif - NONE/- image/gif 1141028861.448767 172.26.11.50 TCP_REFRESH_HIT/304 286 GET http://62.17.163.240/lo_0668/lo_0668/lo_0668page018.html - DIRECT/62.17.163.240 - 1141028861.713 3926 172.26.11.50 TCP_MISS/404 1843 GET http://62.17.163.240/favicon.ico - DIRECT/62.17.163.240 text/html 1141028862.121752 172.26.11.50 TCP_MISS/404 1843 GET http://62.17.163.240/favicon.ico - DIRECT/62.17.163.240 text/html 1141028862.390942 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/common.js - DIRECT/62.17.163.240 - 1141028863.315925 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/browserfuncs.js - DIRECT/62.17.163.240 - 1141028864.245929 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/page.js - DIRECT/62.17.163.240 - 1141028865.181936 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/spi.js - DIRECT/62.17.163.240 - 1141028866.105924 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/generic_oce.js - DIRECT/62.17.163.240 - 1141028866.909803 172.26.11.50 TCP_MISS/304 287 GET http://62.17.163.240/lo_0668/jscripts/communicate.js - DIRECT/62.17.163.240 - 1141028867.724802 172.26.11.50 TCP_MISS/304 286 GET http://62.17.163.240/lo_0668/jscripts/scofind.js - DIRECT/62.17.163.240 - Thanks
Re: [squid-users] squid slow for one particular web site
On 2/27/06, Raj [EMAIL PROTECTED] wrote: Hi All, I have a squid proxy (Squid Cache: Version 2.5.STABLE10) running on a Open BSD server. I am having a problem with one particular web site http://62.17.163.240/. When users try to access this weeb site it takes 6 to 8 seconds to download each page. Where as if we try to access any other web site using the proxy it takes only a second to download the web page. If we by pass the proxy and access the web site http://62.17.163.240/ it's pretty quick. I am not able to figure out why only this particular web site. It should cache the web site and when you try to access this weeb site 2nd or 3rd time it should get it from the cache instead of going to the origin server. What can I do to fix the problem? Hmm, as far as I could experience now at home, on direct-DSL the site is pretty slow too. It contains at the beginning a line as in ; META HTTP-EQUIV=refresh CONTENT=0;URL=/portal/welcome.asp I am under the vast impression, that the underlying portal is rather very slow. M.
Re: [squid-users] squid slow for one particular web site
The first page is a bit slow. But other links are okay if I bypass proxy. When you first access that web site using proxy it should go to the origin server. But for subsequent requests it should get it from the cache. But that is not happening for this site. Thanks On 2/28/06, Mark Elsen [EMAIL PROTECTED] wrote: On 2/27/06, Raj [EMAIL PROTECTED] wrote: Hi All, I have a squid proxy (Squid Cache: Version 2.5.STABLE10) running on a Open BSD server. I am having a problem with one particular web site http://62.17.163.240/. When users try to access this weeb site it takes 6 to 8 seconds to download each page. Where as if we try to access any other web site using the proxy it takes only a second to download the web page. If we by pass the proxy and access the web site http://62.17.163.240/ it's pretty quick. I am not able to figure out why only this particular web site. It should cache the web site and when you try to access this weeb site 2nd or 3rd time it should get it from the cache instead of going to the origin server. What can I do to fix the problem? Hmm, as far as I could experience now at home, on direct-DSL the site is pretty slow too. It contains at the beginning a line as in ; META HTTP-EQUIV=refresh CONTENT=0;URL=/portal/welcome.asp I am under the vast impression, that the underlying portal is rather very slow. M.
Re: [squid-users] squid slow for one particular web site
The first page is a bit slow. But other links are okay if I bypass proxy. When you first access that web site using proxy it should go to the origin server. But for subsequent requests it should get it from the cache. But that is not happening for this site. Should it ? Tryout the relevant objects with : http://www.ircache.net/cgi-bin/cacheability.py M.
Re: [squid-users] Squid Slow Downloads problem
So: You 're downloading on the Squid box, without using the Squid proxy ..and that's also slow? Correct? If that's the case, it has nothing to do with Squid but with the box itself. (Network cable, Duplex settings) On 16.02 14:52, Hesham Shakil wrote: No, all I am saying is that downloads are slower when using Squid proxy and fast when not using squid proxy, on both the Squid box and other machines on the network. So this is a Squid problem :). As you can read in the original email, I used apache+mod_proxy+mod_cache on the Squid box (thats what i am using for the time being till the problem solves) and its working fine, its only using squid that slows down downloads to almost half the available bandwidth. the squid proxy seems to be using proxy.saudi.net.sa as parent cache. did you try to turn this parent off? -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. There's a long-standing bug relating to the x86 architecture that allows you to install Windows. -- Matthew D. Fuller
[squid-users] Squid Slow Downloads problem
Downloading files via squid (on Linux ONLY.. read below) is slower than without squid. The normal download speed bypassing squid is 28+KB/s while through squid it reduces to 16-17KB/s. Browsing seems fine, and the internet bandwidth tests also give 28+KB/s when run through squid, so the problems seems only with files 1MB+ in size. I am running Squid 2.5.STABLE12 on Linux 2.6.12 kernel on: P4 3GHz HT processor with 1GB RAM SATA hard disk using ext3 filesystem Here is what I have tried so far: - Tried apache+mod_proxy+mod_cache on the same linux machine, worked perfectly with 28+KB/s transfer rate - Tried Squid-2.5.STABLE12 compiled for windows on a Windows XP machine running on the same Internet connection and it worked fine at speeds of 28+KB/s - Tried ufs/aufs/diskd but none improved the speed - Tried recompiling squid with NONE but the very basic options - Tried recompiling squid with and without pthreads/aio etc. Here is squid build configuration: Squid Cache: Version 2.5.STABLE12 configure options: i586-mandriva-linux-gnu --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/sbin --sbindir=/usr/sbin --sysconfdir=/etc/squid --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/lib/squid --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --enable-icap-support --with-maxfd=1024 --enable-poll --enable-snmp --enable-removal-policies=heap,lru --enable-storeio=aufs,coss,diskd,ufs,null --enable-useragent-log --enable-referer-log --enable-cachemgr-hostname=localhost --enable-truncate --enable-underscores --enable-carp --enable-async-io --enable-htcp --enable-delay-pools --enable-linux-netfilter --enable-ssl --enable-arp-acl --enable-auth=basic,digest,ntlm --enable-basic-auth-helpers=winbind,multi-domain-NTLM,getpwnam,YP,SMB,PAM,NCSA,MSNT,LDAP --enable-ntlm-auth-helpers=SMB,fakeauth,no_check,winbind --enable-digest-auth-helpers=password --enable-external-acl-helpers=ip_user,ldap_group,unix_group,wbinfo_group,winbind_group --disable-follow-x-forwarded-for --with-pthreads --with-winbind-auth-challenge --disable-dependency-tracking --disable-ident-lookups and finally my squid.conf: http_port 192.168.0.1:8080 icp_port 0 cache_peer proxy.saudi.net.saparent80807 no-query default hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? no_cache deny QUERY cache_dir aufs /var/spool/squid 1024 16 256 dns_nameservers 192.168.0.1 auth_param basic children 5 auth_param basic realm Squid proxy-caching web server auth_param basic credentialsttl 2 hours auth_param basic casesensitive off refresh_pattern ^ftp:144020%10080 refresh_pattern ^gopher:14400%1440 refresh_pattern .020%4320 acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 563 acl Safe_ports port 80# http acl Safe_ports port 21# ftp acl Safe_ports port 443 563# https, snews acl Safe_ports port 70# gopher acl Safe_ports port 210# wais acl Safe_ports port 1025-65535# unregistered ports acl Safe_ports port 280# http-mgmt acl Safe_ports port 488# gss-http acl Safe_ports port 591# filemaker acl Safe_ports port 777# multiling http acl CONNECT method CONNECT acl localnet src 192.168.0.0/255.255.0.0 acl to_localnet dst 192.168.0.0/255.255.0.0 http_access allow manager localhost http_access deny manager http_access deny !Safe_ports http_access allow CONNECT Safe_ports http_access allow localnet http_reply_access allow all icp_access allow all visible_hostname linserver never_direct allow all coredump_dir /var/spool/squid Any ideas what the problem could be? Regards Hesham S. Ahmed
Re: [squid-users] Squid Slow Downloads problem
On Thu, 2006-02-16 at 12:14 -0600, Hesham Shakil wrote: Downloading files via squid (on Linux ONLY.. read below) is slower than without squid. The normal download speed bypassing squid is 28+KB/s while through squid it reduces to 16-17KB/s. Browsing seems fine, and the internet bandwidth tests also give 28+KB/s when run through squid, so the problems seems only with files 1MB+ in size. Hi, You said that the download is faster without Squid. Is that on the same box? So are you downloading from the Squid box? B -- Schelstraete Bart http://www.schelstraete.org [EMAIL PROTECTED] 21:17:00 up 5 days, 3:28, 3 users, load average: 0.18, 0.51, 0.86
Re: [squid-users] Squid Slow Downloads problem
On Thu, 2006-02-16 at 12:14 -0600, Hesham Shakil wrote: Downloading files via squid (on Linux ONLY.. read below) is slower than without squid. The normal download speed bypassing squid is 28+KB/s while through squid it reduces to 16-17KB/s. Browsing seems fine, and the internet bandwidth tests also give 28+KB/s when run through squid, so the problems seems only with files 1MB+ in size. Hi, You said that the download is faster without Squid. Is that on the same box? So are you downloading from the Squid box? Downloads on the Squid box itself or on other machines on the network are all slow when connecting through squid proxy. Regards Hesham S. Ahmed
Re: [squid-users] Squid Slow Downloads problem
On Thu, 2006-02-16 at 14:38 -0600, Hesham Shakil wrote: On Thu, 2006-02-16 at 12:14 -0600, Hesham Shakil wrote: Downloading files via squid (on Linux ONLY.. read below) is slower than without squid. The normal download speed bypassing squid is 28+KB/s while through squid it reduces to 16-17KB/s. Browsing seems fine, and the internet bandwidth tests also give 28+KB/s when run through squid, so the problems seems only with files 1MB+ in size. Hi, You said that the download is faster without Squid. Is that on the same box? So are you downloading from the Squid box? Downloads on the Squid box itself or on other machines on the network are all slow when connecting through squid proxy. So: You 're downloading on the Squid box, without using the Squid proxy ..and that's also slow? Correct? If that's the case, it has nothing to do with Squid but with the box itself. (Network cable, Duplex settings) Bart -- Schelstraete Bart http://www.schelstraete.org [EMAIL PROTECTED] 21:44:54 up 5 days, 3:56, 3 users, load average: 0.69, 0.93, 0.92
Re: [squid-users] Squid Slow Downloads problem
On Thu, 2006-02-16 at 14:38 -0600, Hesham Shakil wrote: On Thu, 2006-02-16 at 12:14 -0600, Hesham Shakil wrote: Downloading files via squid (on Linux ONLY.. read below) is slower than without squid. The normal download speed bypassing squid is 28+KB/s while through squid it reduces to 16-17KB/s. Browsing seems fine, and the internet bandwidth tests also give 28+KB/s when run through squid, so the problems seems only with files 1MB+ in size. Hi, You said that the download is faster without Squid. Is that on the same box? So are you downloading from the Squid box? Downloads on the Squid box itself or on other machines on the network are all slow when connecting through squid proxy. So: You 're downloading on the Squid box, without using the Squid proxy ..and that's also slow? Correct? If that's the case, it has nothing to do with Squid but with the box itself. (Network cable, Duplex settings) No, all I am saying is that downloads are slower when using Squid proxy and fast when not using squid proxy, on both the Squid box and other machines on the network. So this is a Squid problem :). As you can read in the original email, I used apache+mod_proxy+mod_cache on the Squid box (thats what i am using for the time being till the problem solves) and its working fine, its only using squid that slows down downloads to almost half the available bandwidth. Hesham S. Ahmed
Re: [squid-users] Squid Slow Downloads problem
On Thu, 2006-02-16 at 12:14 -0600, Hesham Shakil wrote: - Tried apache+mod_proxy+mod_cache on the same linux machine, worked perfectly with 28+KB/s transfer rate - Tried Squid-2.5.STABLE12 compiled for windows on a Windows XP machine running on the same Internet connection and it worked fine at speeds of 28+KB/s - Tried ufs/aufs/diskd but none improved the speed - Tried recompiling squid with NONE but the very basic options - Tried recompiling squid with and without pthreads/aio etc. For the moment I don't see anything wrong. What you can maybe try, just for a test , is disabling the caching for a certain site/filetype. And then try to download such a file via the proxy. If that works faster, then you will have a problem with the disks (cache) or something similar. If not...then you will have another problem (deuh :) . Maybe increase loglevel then, maybe something can be discovered there. Note: Do you have an antivirus running on that box? Because that will slow down the downloads. Bart -- Schelstraete Bart http://www.schelstraete.org [EMAIL PROTECTED] 22:21:18 up 5 days, 4:32, 3 users, load average: 0.42, 0.83, 1.11
Re: [squid-users] Squid Slow Downloads problem
For the moment I don't see anything wrong. What you can maybe try, just for a test , is disabling the caching for a certain site/filetype. And then try to download such a file via the proxy. If that works faster, then you will have a problem with the disks (cache) or something similar. If not...then you will have another problem (deuh :) . Maybe increase loglevel then, maybe something can be discovered there. Note: Do you have an antivirus running on that box? Because that will slow down the downloads. I have tried looking at everything, no AV is running, moved the cache to a different hard disk, reduced maximum_object_size to 512 KB and tried large downloads but still the same. I have looked into the logfiles, nothing noticable. I will try it on a vanilla kernel, somehow I feel its mandriva's kernel optimizations. HSA
[squid-users] Squid slow shutdown
Hello, I am using Squid on a home desktop system (Linux/Gentoo/AMD64), which is typically shutdown several times a day. During system shutdown it takes quite a long time (~30 seconds) to stop Squid. (This time is about the same when I stop Squid manually by running init scripts or calling squid -k shutdown). I am aware that Squids needs to shutdown cleanly, but it appears it does nothing (minimal cpu/disc activity). Is it normal, or is there something wrong with my setup? Thanks, Michal
RE: [squid-users] Squid slow shutdown
Sounds about normal to me. Mine takes about 2min. -Original Message- From: Michal W. [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 23, 2005 4:11 PM To: squid-users@squid-cache.org Subject: [squid-users] Squid slow shutdown Hello, I am using Squid on a home desktop system (Linux/Gentoo/AMD64), which is typically shutdown several times a day. During system shutdown it takes quite a long time (~30 seconds) to stop Squid. (This time is about the same when I stop Squid manually by running init scripts or calling squid -k shutdown). I am aware that Squids needs to shutdown cleanly, but it appears it does nothing (minimal cpu/disc activity). Is it normal, or is there something wrong with my setup? Thanks, Michal
RE: [squid-users] Squid slow shutdown
-Original Message- From: Michal W. [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 23, 2005 12:11 PM To: squid-users@squid-cache.org Subject: [squid-users] Squid slow shutdown Hello, I am using Squid on a home desktop system (Linux/Gentoo/AMD64), which is typically shutdown several times a day. During system shutdown it takes quite a long time (~30 seconds) to stop Squid. (This time is about the same when I stop Squid manually by running init scripts or calling squid -k shutdown). I am aware that Squids needs to shutdown cleanly, but it appears it does nothing (minimal cpu/disc activity). Is it normal, or is there something wrong with my setup? Thanks, Michal From squid.conf.default: # TAG: shutdown_lifetime time-units # When SIGTERM or SIGHUP is received, the cache is put into # shutdown pending mode until all active sockets are closed. # This value is the lifetime to set for all open descriptors # during shutdown mode. Any active clients after this many # seconds will receive a 'timeout' message. # #Default: # shutdown_lifetime 30 seconds Likely you have a process that is accessing a site through the cache. The cache, trying to be a good neighbor, is waiting for that access to finish before shutting down. After the shutdown_lifetime, it just closes the connection. Either make sure nothing is accessing the internet through Squid, or lower this value. Then again, I could be completely off base. Chris
Re: [squid-users] Squid slow...
Squid Cache: Version 2.5.STABLE9 configure options: --prefix=/usr --exec_prefix=/usr --bindir=/usr/sbin --sbindir=/usr/sbin --libexecdir=/usr/lib/squid --sysconfdir=/etc/squid --localstatedir=/var/spool/squid --datadir=/usr/share/squid --enable-async-io --with-pthreads --enable-storeio=ufs,aufs,diskd,null --enable-linux-netfilter --enable-arp-acl --enable-removal-policies=lru,heap --enable-snmp --enable-delay-pools --enable-htcp --enable-poll --enable-cache-digests --enable-underscores --enable-referer-log --enable-useragent-log --enable-auth=basic,digest,ntlm --enable-carp --with-large-files i386-debian-linux running on debian sarge, using a diskd backend. Transfers are only going at around 350KB/sec for cached files, but when I strace squid, this speeds up to 4.5MB/sec ... What the heck is going on? Is there an easy fix for this? Heisenbug? Debian and or Redhat rpm builds usually have all options enabled on configure. First, I would recommend to install SQUID ´manually´ and use only those configure options which you need. Check whether the problem persists. M.
[squid-users] Squid slow...
Squid Cache: Version 2.5.STABLE9 configure options: --prefix=/usr --exec_prefix=/usr --bindir=/usr/sbin --sbindir=/usr/sbin --libexecdir=/usr/lib/squid --sysconfdir=/etc/squid --localstatedir=/var/spool/squid --datadir=/usr/share/squid --enable-async-io --with-pthreads --enable-storeio=ufs,aufs,diskd,null --enable-linux-netfilter --enable-arp-acl --enable-removal-policies=lru,heap --enable-snmp --enable-delay-pools --enable-htcp --enable-poll --enable-cache-digests --enable-underscores --enable-referer-log --enable-useragent-log --enable-auth=basic,digest,ntlm --enable-carp --with-large-files i386-debian-linux running on debian sarge, using a diskd backend. Transfers are only going at around 350KB/sec for cached files, but when I strace squid, this speeds up to 4.5MB/sec ... What the heck is going on? Is there an easy fix for this? Heisenbug? -- Ryan Castellucci http://ryanc.org/
RE: [squid-users] squid slow --- don't know why
-- Original Message -- From: Angela Burrell [EMAIL PROTECTED] Date: Fri, 10 Sep 2004 09:22:17 -0400 Joseph, I have never used delay pools, but I did incorporate HTTPd_accel into my squid, which helped my speed. I do not host a web server, but it seemed to help. I'm assuming you are using a transparent proxy. Yes I'm using a transparent proxy Try adding the following lines to your squid.conf, then restart squid. # HTTP_ACCEL options httpd_accel_host virtual httpd_accel_port 80 httpd_accel_with_proxy on httpd_accel_uses_host_header on See if that helps. OK, I tried adding the following options and restarted squid, but no difference. Still the same results. With the delay pools off, things are faster, but not as fast as if I bypass the proxy all together. Is that how it is suppose to be? If so, then I think I'm all set. But if the speed results from that website should be faster with a proxy, then I suppose there is still something else going on. Also, I don't see a cache_effective_user or cache_effective_group directive. Not sure if this is necessary, but when troubleshooting I find it's best not to leave any option out, no matter how insignificant it may seem. I agree, I will add these too and try it again. Joseph Angela
[squid-users] squid slow --- don't know why
Hello List, I am new to this list and new to squid. I am having some trouble with my proxy (squid) running really slow. I have searched the Internet and I have also looked in the archives of this list. I found that some threads in the archives suggest not enough memory and squid having to use swap. I've also found some treads on dns quieries slowing things down. I don't think either of these are reasons why my squid is running slow. Here are some details: Computer Specs: --- - dual 400Mhz CPU - 1 G RAM - 31G SCSI hard drive - running debian linux (sarge) When I ps -aux and look at how much memory squid is using it only uses about 32M. It looks as though I have plenty of memory. I installed a caching-only version of bind, and everything seems to be resolving fast. If I browse the Internet without the proxy everything goes really fast (T1 connection). Once I have my browser use the proxy though, it still works, but really slow 5-30 sec/page. Oh, one more detail. I have squid setup to authenticate via ldap_auth. Below is my cache.log file. I do not know what else you might need me to give you to help me, so if there is something else please let me know. cache.log: 2004/09/09 12:01:01| Starting Squid Cache version 2.5.STABLE6 for i386-debian-linux-gnu... 2004/09/09 12:01:01| Process ID 10378 2004/09/09 12:01:01| With 1024 file descriptors available 2004/09/09 12:01:01| DNS Socket created at 0.0.0.0, port 32934, FD 5 2004/09/09 12:01:01| Adding nameserver 127.0.0.1 from /etc/resolv.conf 2004/09/09 12:01:01| helperOpenServers: Starting 10 'ldap_auth' processes 2004/09/09 12:01:01| User-Agent logging is disabled. 2004/09/09 12:01:01| Referer logging is disabled. 2004/09/09 12:01:01| Unlinkd pipe opened on FD 20 2004/09/09 12:01:01| Swap maxSize 2560 KB, estimated 1969230 objects 2004/09/09 12:01:01| Target number of buckets: 98461 2004/09/09 12:01:01| Using 131072 Store buckets 2004/09/09 12:01:01| Max Mem size: 262144 KB 2004/09/09 12:01:01| Max Swap size: 2560 KB 2004/09/09 12:01:01| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2004/09/09 12:01:01| Rebuilding storage in /cache (CLEAN) 2004/09/09 12:01:01| Using Least Load store dir selection 2004/09/09 12:01:01| Set Current Directory to /var/spool/squid 2004/09/09 12:01:01| Loaded Icons. 2004/09/09 12:01:01| Accepting HTTP connections at 0.0.0.0, port 3128, FD 22. 2004/09/09 12:01:01| Accepting ICP messages at 0.0.0.0, port 3130, FD 23. 2004/09/09 12:01:01| HTCP Disabled. 2004/09/09 12:01:01| WCCP Disabled. 2004/09/09 12:01:01| Ready to serve requests. 2004/09/09 12:01:02| Store rebuilding is 3.0% complete 2004/09/09 12:01:06| Done reading /cache swaplog (135414 entries) 2004/09/09 12:01:06| Finished rebuilding storage from disk. 2004/09/09 12:01:06|135414 Entries scanned 2004/09/09 12:01:06| 0 Invalid entries. 2004/09/09 12:01:06| 0 With invalid flags. 2004/09/09 12:01:06|135414 Objects loaded. 2004/09/09 12:01:06| 0 Objects expired. 2004/09/09 12:01:06| 0 Objects cancelled. 2004/09/09 12:01:06| 0 Duplicate URLs purged. 2004/09/09 12:01:06| 0 Swapfile clashes avoided. 2004/09/09 12:01:06| Took 4.8 seconds (28048.1 objects/sec). 2004/09/09 12:01:06| Beginning Validation Procedure 2004/09/09 12:01:06| Completed Validation Procedure 2004/09/09 12:01:06| Validated 135414 Entries 2004/09/09 12:01:06| store_swap_size = 1076212k 2004/09/09 12:01:06| storeLateRelease: released 0 objects Thank You for your help, Joseph Rinckey
RE: [squid-users] squid slow --- don't know why
Hi Joseph Here is an interesting link that a list member posted a while back with the same problem. This is a link to a website where you can do speed tests. http://www.numion.com/yourspeed/ Please try the test with your squid on, then again with your squid off. and post the results. Thanks AB -Original Message- From: Joseph Rinckey [mailto:[EMAIL PROTECTED] Sent: September 9, 2004 12:25 PM To: [EMAIL PROTECTED] Subject: [squid-users] squid slow --- don't know why Hello List, I am new to this list and new to squid. I am having some trouble with my proxy (squid) running really slow. I have searched the Internet and I have also looked in the archives of this list. I found that some threads in the archives suggest not enough memory and squid having to use swap. I've also found some treads on dns quieries slowing things down. I don't think either of these are reasons why my squid is running slow. Here are some details: Computer Specs: --- - dual 400Mhz CPU - 1 G RAM - 31G SCSI hard drive - running debian linux (sarge) When I ps -aux and look at how much memory squid is using it only uses about 32M. It looks as though I have plenty of memory. I installed a caching-only version of bind, and everything seems to be resolving fast. If I browse the Internet without the proxy everything goes really fast (T1 connection). Once I have my browser use the proxy though, it still works, but really slow 5-30 sec/page. Oh, one more detail. I have squid setup to authenticate via ldap_auth. Below is my cache.log file. I do not know what else you might need me to give you to help me, so if there is something else please let me know. cache.log: 2004/09/09 12:01:01| Starting Squid Cache version 2.5.STABLE6 for i386-debian-linux-gnu... 2004/09/09 12:01:01| Process ID 10378 2004/09/09 12:01:01| With 1024 file descriptors available 2004/09/09 12:01:01| DNS Socket created at 0.0.0.0, port 32934, FD 5 2004/09/09 12:01:01| Adding nameserver 127.0.0.1 from /etc/resolv.conf 2004/09/09 12:01:01| helperOpenServers: Starting 10 'ldap_auth' processes 2004/09/09 12:01:01| User-Agent logging is disabled. 2004/09/09 12:01:01| Referer logging is disabled. 2004/09/09 12:01:01| Unlinkd pipe opened on FD 20 2004/09/09 12:01:01| Swap maxSize 2560 KB, estimated 1969230 objects 2004/09/09 12:01:01| Target number of buckets: 98461 2004/09/09 12:01:01| Using 131072 Store buckets 2004/09/09 12:01:01| Max Mem size: 262144 KB 2004/09/09 12:01:01| Max Swap size: 2560 KB 2004/09/09 12:01:01| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2004/09/09 12:01:01| Rebuilding storage in /cache (CLEAN) 2004/09/09 12:01:01| Using Least Load store dir selection 2004/09/09 12:01:01| Set Current Directory to /var/spool/squid 2004/09/09 12:01:01| Loaded Icons. 2004/09/09 12:01:01| Accepting HTTP connections at 0.0.0.0, port 3128, FD 22. 2004/09/09 12:01:01| Accepting ICP messages at 0.0.0.0, port 3130, FD 23. 2004/09/09 12:01:01| HTCP Disabled. 2004/09/09 12:01:01| WCCP Disabled. 2004/09/09 12:01:01| Ready to serve requests. 2004/09/09 12:01:02| Store rebuilding is 3.0% complete 2004/09/09 12:01:06| Done reading /cache swaplog (135414 entries) 2004/09/09 12:01:06| Finished rebuilding storage from disk. 2004/09/09 12:01:06|135414 Entries scanned 2004/09/09 12:01:06| 0 Invalid entries. 2004/09/09 12:01:06| 0 With invalid flags. 2004/09/09 12:01:06|135414 Objects loaded. 2004/09/09 12:01:06| 0 Objects expired. 2004/09/09 12:01:06| 0 Objects cancelled. 2004/09/09 12:01:06| 0 Duplicate URLs purged. 2004/09/09 12:01:06| 0 Swapfile clashes avoided. 2004/09/09 12:01:06| Took 4.8 seconds (28048.1 objects/sec). 2004/09/09 12:01:06| Beginning Validation Procedure 2004/09/09 12:01:06| Completed Validation Procedure 2004/09/09 12:01:06| Validated 135414 Entries 2004/09/09 12:01:06| store_swap_size = 1076212k 2004/09/09 12:01:06| storeLateRelease: released 0 objects Thank You for your help, Joseph Rinckey
Re: [squid-users] squid slow --- don't know why
Wow! I didn't know it was that bad. I ran each test twice. I didn't know if you wantted all the stats, or just the speeds. Below are the speeds: Without the proxy - 482 Kbits (I have a T1, I was shocked. But that's not a squid issue) 499 Kbits With the proxy - 98 Kbits 16 Kbits I kept an eye on top while doing the tests with the proxy set, and squid did not take anymore than 2% of the CPU. The load average is 0.04 right now. Anything else? Joseph Rinckey Angela Burrell wrote: Hi Joseph Here is an interesting link that a list member posted a while back with the same problem. This is a link to a website where you can do speed tests. http://www.numion.com/yourspeed/ Please try the test with your squid on, then again with your squid off. and post the results. Thanks AB -Original Message- From: Joseph Rinckey [mailto:[EMAIL PROTECTED] Sent: September 9, 2004 12:25 PM To: [EMAIL PROTECTED] Subject: [squid-users] squid slow --- don't know why Hello List, I am new to this list and new to squid. I am having some trouble with my proxy (squid) running really slow. I have searched the Internet and I have also looked in the archives of this list. I found that some threads in the archives suggest not enough memory and squid having to use swap. I've also found some treads on dns quieries slowing things down. I don't think either of these are reasons why my squid is running slow. Here are some details: Computer Specs: --- - dual 400Mhz CPU - 1 G RAM - 31G SCSI hard drive - running debian linux (sarge) When I ps -aux and look at how much memory squid is using it only uses about 32M. It looks as though I have plenty of memory. I installed a caching-only version of bind, and everything seems to be resolving fast. If I browse the Internet without the proxy everything goes really fast (T1 connection). Once I have my browser use the proxy though, it still works, but really slow 5-30 sec/page. Oh, one more detail. I have squid setup to authenticate via ldap_auth. Below is my cache.log file. I do not know what else you might need me to give you to help me, so if there is something else please let me know. cache.log: 2004/09/09 12:01:01| Starting Squid Cache version 2.5.STABLE6 for i386-debian-linux-gnu... 2004/09/09 12:01:01| Process ID 10378 2004/09/09 12:01:01| With 1024 file descriptors available 2004/09/09 12:01:01| DNS Socket created at 0.0.0.0, port 32934, FD 5 2004/09/09 12:01:01| Adding nameserver 127.0.0.1 from /etc/resolv.conf 2004/09/09 12:01:01| helperOpenServers: Starting 10 'ldap_auth' processes 2004/09/09 12:01:01| User-Agent logging is disabled. 2004/09/09 12:01:01| Referer logging is disabled. 2004/09/09 12:01:01| Unlinkd pipe opened on FD 20 2004/09/09 12:01:01| Swap maxSize 2560 KB, estimated 1969230 objects 2004/09/09 12:01:01| Target number of buckets: 98461 2004/09/09 12:01:01| Using 131072 Store buckets 2004/09/09 12:01:01| Max Mem size: 262144 KB 2004/09/09 12:01:01| Max Swap size: 2560 KB 2004/09/09 12:01:01| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2004/09/09 12:01:01| Rebuilding storage in /cache (CLEAN) 2004/09/09 12:01:01| Using Least Load store dir selection 2004/09/09 12:01:01| Set Current Directory to /var/spool/squid 2004/09/09 12:01:01| Loaded Icons. 2004/09/09 12:01:01| Accepting HTTP connections at 0.0.0.0, port 3128, FD 22. 2004/09/09 12:01:01| Accepting ICP messages at 0.0.0.0, port 3130, FD 23. 2004/09/09 12:01:01| HTCP Disabled. 2004/09/09 12:01:01| WCCP Disabled. 2004/09/09 12:01:01| Ready to serve requests. 2004/09/09 12:01:02| Store rebuilding is 3.0% complete 2004/09/09 12:01:06| Done reading /cache swaplog (135414 entries) 2004/09/09 12:01:06| Finished rebuilding storage from disk. 2004/09/09 12:01:06|135414 Entries scanned 2004/09/09 12:01:06| 0 Invalid entries. 2004/09/09 12:01:06| 0 With invalid flags. 2004/09/09 12:01:06|135414 Objects loaded. 2004/09/09 12:01:06| 0 Objects expired. 2004/09/09 12:01:06| 0 Objects cancelled. 2004/09/09 12:01:06| 0 Duplicate URLs purged. 2004/09/09 12:01:06| 0 Swapfile clashes avoided. 2004/09/09 12:01:06| Took 4.8 seconds (28048.1 objects/sec). 2004/09/09 12:01:06| Beginning Validation Procedure 2004/09/09 12:01:06| Completed Validation Procedure 2004/09/09 12:01:06| Validated 135414 Entries 2004/09/09 12:01:06| store_swap_size = 1076212k 2004/09/09 12:01:06| storeLateRelease: released 0 objects Thank You for your help, Joseph Rinckey
RE: [squid-users] squid slow --- don't know why
Here are my results, measuring my speed to servers within my own country, using an ADSL connection via LAN on two clients. These two clients are the only two active on the LAN right now. With squid: (Windows client) 1. 327 Kbps 2. 370 Kbps Without squid: (Windows client) 1. 286 Kbps 2. 338 Kbps 3. 324 Kbps Without squid: (Linux client) 1. 61 Kbps 2. 271 Kbps 3. 293 Kbps With squid: (Linux client) 1. 289 Kbps 2. 291 Kbps 3. 296 Kbps As you can see, with squid configuration, there should not be a slowdown to your Internet access. Yours is quite drastic. There might be a configuration problem: could you please post your squid.conf file? Use this command: grep ^[a-z] squid.conf squid.conf.org to remove comments. -Original Message- From: Joseph Rinckey [mailto:[EMAIL PROTECTED] Sent: September 9, 2004 2:45 PM To: Angela Burrell Cc: [EMAIL PROTECTED] Subject: Re: [squid-users] squid slow --- don't know why Wow! I didn't know it was that bad. I ran each test twice. I didn't know if you wantted all the stats, or just the speeds. Below are the speeds: Without the proxy - 482 Kbits (I have a T1, I was shocked. But that's not a squid issue) 499 Kbits With the proxy - 98 Kbits 16 Kbits I kept an eye on top while doing the tests with the proxy set, and squid did not take anymore than 2% of the CPU. The load average is 0.04 right now. Anything else? Joseph Rinckey Angela Burrell wrote: Hi Joseph Here is an interesting link that a list member posted a while back with the same problem. This is a link to a website where you can do speed tests. http://www.numion.com/yourspeed/ Please try the test with your squid on, then again with your squid off. and post the results. Thanks AB -Original Message- From: Joseph Rinckey [mailto:[EMAIL PROTECTED] Sent: September 9, 2004 12:25 PM To: [EMAIL PROTECTED] Subject: [squid-users] squid slow --- don't know why Hello List, I am new to this list and new to squid. I am having some trouble with my proxy (squid) running really slow. I have searched the Internet and I have also looked in the archives of this list. I found that some threads in the archives suggest not enough memory and squid having to use swap. I've also found some treads on dns quieries slowing things down. I don't think either of these are reasons why my squid is running slow. Here are some details: Computer Specs: --- - dual 400Mhz CPU - 1 G RAM - 31G SCSI hard drive - running debian linux (sarge) When I ps -aux and look at how much memory squid is using it only uses about 32M. It looks as though I have plenty of memory. I installed a caching-only version of bind, and everything seems to be resolving fast. If I browse the Internet without the proxy everything goes really fast (T1 connection). Once I have my browser use the proxy though, it still works, but really slow 5-30 sec/page. Oh, one more detail. I have squid setup to authenticate via ldap_auth. Below is my cache.log file. I do not know what else you might need me to give you to help me, so if there is something else please let me know. cache.log: 2004/09/09 12:01:01| Starting Squid Cache version 2.5.STABLE6 for i386-debian-linux-gnu... 2004/09/09 12:01:01| Process ID 10378 2004/09/09 12:01:01| With 1024 file descriptors available 2004/09/09 12:01:01| DNS Socket created at 0.0.0.0, port 32934, FD 5 2004/09/09 12:01:01| Adding nameserver 127.0.0.1 from /etc/resolv.conf 2004/09/09 12:01:01| helperOpenServers: Starting 10 'ldap_auth' processes 2004/09/09 12:01:01| User-Agent logging is disabled. 2004/09/09 12:01:01| Referer logging is disabled. 2004/09/09 12:01:01| Unlinkd pipe opened on FD 20 2004/09/09 12:01:01| Swap maxSize 2560 KB, estimated 1969230 objects 2004/09/09 12:01:01| Target number of buckets: 98461 2004/09/09 12:01:01| Using 131072 Store buckets 2004/09/09 12:01:01| Max Mem size: 262144 KB 2004/09/09 12:01:01| Max Swap size: 2560 KB 2004/09/09 12:01:01| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2004/09/09 12:01:01| Rebuilding storage in /cache (CLEAN) 2004/09/09 12:01:01| Using Least Load store dir selection 2004/09/09 12:01:01| Set Current Directory to /var/spool/squid 2004/09/09 12:01:01| Loaded Icons. 2004/09/09 12:01:01| Accepting HTTP connections at 0.0.0.0, port 3128, FD 22. 2004/09/09 12:01:01| Accepting ICP messages at 0.0.0.0, port 3130, FD 23. 2004/09/09 12:01:01| HTCP Disabled. 2004/09/09 12:01:01| WCCP Disabled. 2004/09/09 12:01:01| Ready to serve requests. 2004/09/09 12:01:02| Store rebuilding is 3.0% complete 2004/09/09 12:01:06| Done reading /cache swaplog (135414 entries) 2004/09/09 12:01:06| Finished rebuilding storage from disk. 2004/09/09 12:01:06|135414 Entries scanned 2004/09/09 12:01:06| 0 Invalid entries. 2004/09/09 12:01:06| 0 With invalid flags. 2004/09/09 12:01:06|135414 Objects loaded. 2004/09/09 12:01:06
Re: [squid-users] squid slow --- don't know why
My squid.conf file is below. I also commented out the delay pool lines and restarted squid to see what kind of speeds I got. They were faster than with the delay pools, but still slower than without the proxy all together. The speeds were around the 315 Kbps. Much better than 16Kbps:) If the delay pools are in fact my problem I guess I don't know how to set them up. I just want to slow everyone down who downloads files bigger than 2Mb. But I do not want to slow everything down. squid.conf: http_port 3128 hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? no_cache deny QUERY cache_mem 256 MB cache_swap_low 95 cache_swap_high 98 cache_dir ufs /cache 25000 32 512 auth_param basic program /usr/lib/squid/ldap_auth -b ou=people,dc=pineriver,dc=k12,dc=mi,dc=us -s sub -h 192.168.0.13 -p 389 -v 3 -f uid=%s auth_param basic children 10 auth_param basic realm Pine River Area Schools (proxy/cache server) auth_param basic credentialsttl 1 hours refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern . 0 20% 4320 acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 563 # https, snews acl SSL_ports port 873 # rsync acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 563 # https, snews acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl Safe_ports port 631 # cups acl Safe_ports port 873 # rsync acl Safe_ports port 901 # SWAT acl purge method PURGE acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access deny !Safe_ports http_access deny CONNECT !SSL_portshttp_access deny to_localhost acl ldapauth proxy_auth REQUIRED acl mail-server dstdomain .pineriver.k12.mi.us acl tech-laptop src 172.20.20.254 no_cache deny mail-server always_direct allow mail-server http_access allow ldapauth http_access allow localhost http_access allow tech-laptop http_access deny all http_reply_access allow all icp_access allow all cache_mgr [EMAIL PROTECTED] visible_hostname pr-cache.pineriver.k12.mi.us delay_pools 1 delay_class 1 1 delay_parameters 1 12500/24000 delay_access 1 allow all coredump_dir /var/spool/squid -- Joseph Rinckey Angela Burrell wrote: Here are my results, measuring my speed to servers within my own country, using an ADSL connection via LAN on two clients. These two clients are the only two active on the LAN right now. With squid: (Windows client) 1. 327 Kbps 2. 370 Kbps Without squid: (Windows client) 1. 286 Kbps 2. 338 Kbps 3. 324 Kbps Without squid: (Linux client) 1. 61 Kbps 2. 271 Kbps 3. 293 Kbps With squid: (Linux client) 1. 289 Kbps 2. 291 Kbps 3. 296 Kbps As you can see, with squid configuration, there should not be a slowdown to your Internet access. Yours is quite drastic. There might be a configuration problem: could you please post your squid.conf file? Use this command: grep ^[a-z] squid.conf squid.conf.org to remove comments. -Original Message- From: Joseph Rinckey [mailto:[EMAIL PROTECTED] Sent: September 9, 2004 2:45 PM To: Angela Burrell Cc: [EMAIL PROTECTED] Subject: Re: [squid-users] squid slow --- don't know why Wow! I didn't know it was that bad. I ran each test twice. I didn't know if you wantted all the stats, or just the speeds. Below are the speeds: Without the proxy - 482 Kbits (I have a T1, I was shocked. But that's not a squid issue) 499 Kbits With the proxy - 98 Kbits 16 Kbits I kept an eye on top while doing the tests with the proxy set, and squid did not take anymore than 2% of the CPU. The load average is 0.04 right now. Anything else? Joseph Rinckey Angela Burrell wrote: Hi Joseph Here is an interesting link that a list member posted a while back with the same problem. This is a link to a website where you can do speed tests. http://www.numion.com/yourspeed/ Please try the test with your squid on, then again with your squid off. and post the results. Thanks AB -Original Message- From: Joseph Rinckey [mailto:[EMAIL PROTECTED] Sent: September 9, 2004 12:25 PM To: [EMAIL PROTECTED] Subject: [squid-users] squid slow --- don't know why Hello List, I am new to this list and new to squid. I am having some trouble with my proxy (squid) running really slow. I have searched the Internet and I have also looked in the archives of this list. I found that some threads in the archives suggest not enough memory and squid having to use swap. I've also found some treads on dns quieries
[squid-users] Squid Slow
Hi everybody I still have the same problem with squid and squidguard because when I use urlblacklist.com `s file squid works very slow If I change my blacklist for squidguard`s blacklist everything goes good. I have slackware 10 and squid2-5STABLE6 with squidguard 1.2.0 Thanks
RE: [squid-users] Squid slow down
Dear Leandro How much memory do you have allocated for Squid ? Regards Babar Kazmi. From: Leandro Coelho Söndahl [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: [squid-users] Squid slow down Date: 19 Jul 2004 17:44:48 - First of all sorry for my english. Hello everybody I need a help. I compiled the quid-cache with tranparent proxy on my network-server. The first times it's working looks good. But days after (2 weeks accurately) he slow down the conections. Please help me. My e-mail is [EMAIL PROTECTED] Leandro C. Sondahl
[squid-users] Squid slow down
First of all sorry for my english. Hello everybody I need a help. I compiled the quid-cache with tranparent proxy on my network-server. The first times it's working looks good. But days after (2 weeks accurately) he slow down the conections. Please help me. My e-mail is [EMAIL PROTECTED] Leandro C. Sondahl
[squid-users] squid slow!! pls help!
Hi all I'm running Squid Cache: Version 2.4.STABLE6 on :- Linux version 2.4.20-28.7smp ([EMAIL PROTECTED]) (gcc version 2.9 6 2731 (Red Hat Linux 7.3 2.96-126)) #1 SMP Thu Dec 18 11:18:31 EST 2003 Server is PIII 866 MHz, with 1Gb RAM. 3 x 18 Gb SCSI drives. Internet browsing was fine until Monday...when things slowed down. I suspecte dthat the new Netsky virus slowed down my network, but i just ran a test now. If I bypass my proxy and go directly through my firewall, my browsing is fast. If i go via my proxy, things are extremely slow. My proxy services about 4000 usersso you can understand my desperation. Cache.log does not have any errors that i haven't seen before. /var/log/messages is similar. Can anyone assist? TIA Rgds, HG
RE: [squid-users] squid slow!! pls help!
Hi all I'm running Squid Cache: Version 2.4.STABLE6 on :- Linux version 2.4.20-28.7smp ([EMAIL PROTECTED]) (gcc version 2.9 6 2731 (Red Hat Linux 7.3 2.96-126)) #1 SMP Thu Dec 18 11:18:31 EST 2003 Server is PIII 866 MHz, with 1Gb RAM. 3 x 18 Gb SCSI drives. Internet browsing was fine until Monday...when things slowed down. I suspecte dthat the new Netsky virus slowed down my network, but i just ran a test now. If I bypass my proxy and go directly through my firewall, my browsing is fast. If i go via my proxy, things are extremely slow. My proxy services about 4000 usersso you can understand my desperation. Cache.log does not have any errors that i haven't seen before. /var/log/messages is similar. Can anyone assist? http://www.squid-cache.org/Doc/FAQ/FAQ-11.html#ss11.17 M.
[squid-users] squid slow / non-responsive
Hi all: having a problem with squid for last 12 hours or so. Running Squid 2.5 Stable 3 Redhat 9, std install, on Intel Xeon 2.4GHz with 1 gig ram 1 36Gig Ultra 320 SCSI drive, (only caching 8 gig of content) Transparent with Cisco wccpV1 Actually 2 squid boxes exactly the same. They have been running fine for months. (sept for the port scanning worms that keep messing with us) Anyway, all of a suddon today, I've had customers complain they can't surf to web sites we host (on some sun Ultra boxes on our IP space) Squid isn't suppose to be caching anything from our cidr. don't think this is specifically related to the issue. I've discovered a couple pages from another ISP that does not work either. What seems to be happening, is customer tried to surf to a page couple items get returned from the page (and show in squid access.log) but then it appears to freeze there. If you don't touch anything, 30-60 sec later, the rest of the page completes. (or most of it anyway) ie a page with simple html, 20-30 jpg images, and a background image some of the images don;t come up, sometimes the background does not come up. I've tried not going transparent, and directly configuring a brouser to use one of the squid boxes. same result. Of course turning off both squid boxes, and everything just screames along. Any suggestions greatly appreciated. BG
Re: [squid-users] squid slow / non-responsive
On Thu, 6 Nov 2003, Brad Groshok wrote: What seems to be happening, is customer tried to surf to a page couple items get returned from the page (and show in squid access.log) but then it appears to freeze there. If you don't touch anything, 30-60 sec later, the rest of the page completes. (or most of it anyway) Could be a number of things. The most likely: a) A networking problem such as incorrect half/full duplex negotiation with the switch or bad cable. Check the interface settings on both the server and the switch. b) Some serious resource shortage on the server. Use normal system monitoring tools to see if there is anything overloading your server. c) IP conflict where something else is trying to use the IP address of your Squid server. Regards Henrik