Re: [squid-users] Squid slow performance
Are you able to perform a wget from the machine on a known fast source? Like a mirror to test the Internet connection... On Tue, Jul 24, 2012 at 10:38 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 24/07/2012 7:21 p.m., Alamgir Shamim wrote: Hello, We are using squid version squid-2.6.STABLE21-6.el5. But getting very poor performance. Our total Internet user are almost 750. At a time 500 to 600 user browse internet. all of them are getting slow response. it takes 10 to 30 second to load a page. Here I am giving you some out put. top out put... = top - 12:54:09 up 31 days, 5:59, 2 users, load average: 0.18, 0.24, 0.24 Tasks: 80 total, 1 running, 79 sleeping, 0 stopped, 0 zombie Cpu(s): 1.5%us, 0.2%sy, 0.0%ni, 93.5%id, 4.2%wa, 0.0%hi, 0.6%si, 0.0%st Mem: 2075016k total, 2020432k used,54584k free, 233784k buffers Swap: 2031608k total, 96k used, 2031512k free, 1538140k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 12866 squid 15 0 100m 70m 2508 S3 3.5 4:56.96 squid 6815 root 18 0 10936 4280 428 S0 0.2 0:00.00 squid free -m output [root@proxy253 squid]# free -m total used free sharedbuffers cached Mem: 2026 1973 53 0228 1502 -/+ buffers/cache:242 1783 Swap: 1983 0 1983 Processor infor == processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.20GHz stepping: 3 cpu MHz : 3200.343 cache size : 2048 KB physical id : 0 siblings: 2 processor : 1 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.20GHz stepping: 3 cpu MHz : 3200.343 cache size : 2048 KB physical id : 0 siblings: 2 - Could you please let me know what might be the reason for getting slow performance. We have 14 mbps of Internet bandwidth. We are doing some content filtering also with squid. There are several possible reasons: * you have configured squid in a inefficient way - I'm happy to do free performance audits here if you want to paste your squid.conf to the list (obscure the cachemgr_passwd detail though please) * you are content filtering - this is a major slowdown for Squid no matter what type of filtering is being done * overloaded or slow disks I/O speeds on disk cache - only relevant if you are disk caching * 2.6 will be adding to the slowdown - 2.7 series had a performance focus on development and is a good 20% faster just in the code. - 2.6 has low HTTP/1.1 compliance, meaning you loose out on many HTTP/1.1 performance features available in later releases. Amos -- Regards, Jason Leschnik. [m] 0432 35 4224 [w@] jason dot leschnik at ansto dot gov dot au [U@] jml...@uow.edu.au
RE: [squid-users] Squid + Cisco 4500 + WCCP2
Very sorry for bothering you again although i get the redirection from the router to squid, using tcpdump (10.72.192.61 test internal address) 11:38:37.956330 IP 199.47.218.151.80 10.72.192.61.50690: Flags [S.], seq 1048613649, ack 1347334415, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:38.399796 IP 10.72.192.61.50697 199.47.218.151.80: Flags [S], seq 3043000771, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 11:38:38.399880 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:39.756353 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.356350 IP 176.9.44.80.80 10.72.192.61.50693: Flags [S.], seq 326259738, ack 1299448389, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.409101 IP 10.72.192.61.50697 199.47.218.151.80: Flags [S], seq 3043000771, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 11:38:41.409164 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.556343 IP 176.9.44.80.80 10.72.192.61.50694: Flags [S.], seq 2634200113, ack 3423797704, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.756336 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.756362 IP 209.85.148.139.80 10.72.192.61.50695: Flags [S.], seq 2040290141, ack 953271924, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:42.356340 IP 209.85.148.139.80 10.72.192.61.50696: Flags [S.], seq 69242255, ack 3941278742, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 i still can't get linux to redirect to squid (port 8080), access.log is empty i use the following iptables - # Generated by iptables-save v1.4.12.1 on Wed Jul 25 11:36:37 2012 *filter :INPUT ACCEPT [105007:140596865] :FORWARD ACCEPT [3:120] :OUTPUT ACCEPT [212743:136992211] -A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT COMMIT # Completed on Wed Jul 25 11:36:37 2012 # Generated by iptables-save v1.4.12.1 on Wed Jul 25 11:36:37 2012 *nat :PREROUTING ACCEPT [0:0] :INPUT ACCEPT [1254:65132] :OUTPUT ACCEPT [118:7345] :POSTROUTING ACCEPT [0:0] -A PREROUTING -d $SQUID_IP -i eth0 -p tcp -j ACCEPT -A PREROUTING -s $NETWORK_SPACE -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080 -A POSTROUTING -j MASQUERADE COMMIT # Completed on Wed Jul 25 11:36:37 2012 --- Catch is that i use l2 redirection, so source and destination is eth0, no gre tunnel. Can it be done or should a create a virtual device and redirect input from there? Thank you in advance John
Re: [squid-users] Squid + Cisco 4500 + WCCP2
On Wed, Jul 25, 2012 at 3:04 PM, Indunil Jayasooriya induni...@gmail.com wrote: Can your squid box to go to internet ? ( Pls check /etc/resolv.conf file ) How many interfaces does your squid box have? 1 or 2 ? in /etc/sysctl.conf file , pls check net.ipv4.ip_forward parameter? try to make it to one in following manner. net.ipv4.ip_forward = 1 On Wed, Jul 25, 2012 at 2:13 PM, Ioannis Pliatsikas gpli...@ee.duth.gr wrote: Very sorry for bothering you again although i get the redirection from the router to squid, using tcpdump (10.72.192.61 test internal address) 11:38:37.956330 IP 199.47.218.151.80 10.72.192.61.50690: Flags [S.], seq 1048613649, ack 1347334415, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:38.399796 IP 10.72.192.61.50697 199.47.218.151.80: Flags [S], seq 3043000771, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 11:38:38.399880 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:39.756353 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.356350 IP 176.9.44.80.80 10.72.192.61.50693: Flags [S.], seq 326259738, ack 1299448389, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.409101 IP 10.72.192.61.50697 199.47.218.151.80: Flags [S], seq 3043000771, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 11:38:41.409164 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.556343 IP 176.9.44.80.80 10.72.192.61.50694: Flags [S.], seq 2634200113, ack 3423797704, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.756336 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.756362 IP 209.85.148.139.80 10.72.192.61.50695: Flags [S.], seq 2040290141, ack 953271924, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:42.356340 IP 209.85.148.139.80 10.72.192.61.50696: Flags [S.], seq 69242255, ack 3941278742, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 i still can't get linux to redirect to squid (port 8080), access.log is empty i use the following iptables - # Generated by iptables-save v1.4.12.1 on Wed Jul 25 11:36:37 2012 *filter :INPUT ACCEPT [105007:140596865] :FORWARD ACCEPT [3:120] :OUTPUT ACCEPT [212743:136992211] -A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT COMMIT # Completed on Wed Jul 25 11:36:37 2012 # Generated by iptables-save v1.4.12.1 on Wed Jul 25 11:36:37 2012 *nat :PREROUTING ACCEPT [0:0] :INPUT ACCEPT [1254:65132] :OUTPUT ACCEPT [118:7345] :POSTROUTING ACCEPT [0:0] -A PREROUTING -d $SQUID_IP -i eth0 -p tcp -j ACCEPT -A PREROUTING -s $NETWORK_SPACE -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080 -A POSTROUTING -j MASQUERADE COMMIT # Completed on Wed Jul 25 11:36:37 2012 --- Catch is that i use l2 redirection, so source and destination is eth0, no gre tunnel. Can it be done or should a create a virtual device and redirect input from there? Thank you in advance John -- Thank you Indunil Jayasooriya -- Thank you Indunil Jayasooriya
Re: [squid-users] Squid + Cisco 4500 + WCCP2
Only 1 interface is available Sorry. forgot to add that i have also configured that echo 0 /proc/sys/net/ipv4/conf/default/rp_filter echo 0 /proc/sys/net/ipv4/conf/eth0/rp_filter echo 1 /proc/sys/net/ipv4/ip_forward Thank you John On Wed, 25 Jul 2012 15:05:33 +0530, Indunil Jayasooriya wrote: On Wed, Jul 25, 2012 at 3:04 PM, Indunil Jayasooriya induni...@gmail.com wrote: Can your squid box to go to internet ? ( Pls check /etc/resolv.conf file ) How many interfaces does your squid box have? 1 or 2 ? in /etc/sysctl.conf file , pls check net.ipv4.ip_forward parameter? try to make it to one in following manner. net.ipv4.ip_forward = 1 On Wed, Jul 25, 2012 at 2:13 PM, Ioannis Pliatsikas gpli...@ee.duth.gr wrote: Very sorry for bothering you again although i get the redirection from the router to squid, using tcpdump (10.72.192.61 test internal address) 11:38:37.956330 IP 199.47.218.151.80 10.72.192.61.50690: Flags [S.], seq 1048613649, ack 1347334415, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:38.399796 IP 10.72.192.61.50697 199.47.218.151.80: Flags [S], seq 3043000771, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 11:38:38.399880 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:39.756353 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.356350 IP 176.9.44.80.80 10.72.192.61.50693: Flags [S.], seq 326259738, ack 1299448389, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.409101 IP 10.72.192.61.50697 199.47.218.151.80: Flags [S], seq 3043000771, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 11:38:41.409164 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.556343 IP 176.9.44.80.80 10.72.192.61.50694: Flags [S.], seq 2634200113, ack 3423797704, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.756336 IP 199.47.218.151.80 10.72.192.61.50697: Flags [S.], seq 3389808826, ack 3043000772, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:41.756362 IP 209.85.148.139.80 10.72.192.61.50695: Flags [S.], seq 2040290141, ack 953271924, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 11:38:42.356340 IP 209.85.148.139.80 10.72.192.61.50696: Flags [S.], seq 69242255, ack 3941278742, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 4], length 0 i still can't get linux to redirect to squid (port 8080), access.log is empty i use the following iptables - # Generated by iptables-save v1.4.12.1 on Wed Jul 25 11:36:37 2012 *filter :INPUT ACCEPT [105007:140596865] :FORWARD ACCEPT [3:120] :OUTPUT ACCEPT [212743:136992211] -A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT COMMIT # Completed on Wed Jul 25 11:36:37 2012 # Generated by iptables-save v1.4.12.1 on Wed Jul 25 11:36:37 2012 *nat :PREROUTING ACCEPT [0:0] :INPUT ACCEPT [1254:65132] :OUTPUT ACCEPT [118:7345] :POSTROUTING ACCEPT [0:0] -A PREROUTING -d $SQUID_IP -i eth0 -p tcp -j ACCEPT -A PREROUTING -s $NETWORK_SPACE -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080 -A POSTROUTING -j MASQUERADE COMMIT # Completed on Wed Jul 25 11:36:37 2012 --- Catch is that i use l2 redirection, so source and destination is eth0, no gre tunnel. Can it be done or should a create a virtual device and redirect input from there? Thank you in advance John -- Thank you Indunil Jayasooriya -- Thank you Indunil Jayasooriya
Re: [squid-users] Squid + Cisco 4500 + WCCP2
On 25/07/2012 10:29 p.m., Ioannis Pliatsikas wrote: Only 1 interface is available Sorry. forgot to add that i have also configured that echo 0 /proc/sys/net/ipv4/conf/default/rp_filter echo 0 /proc/sys/net/ipv4/conf/eth0/rp_filter echo 1 /proc/sys/net/ipv4/ip_forward There is another rp_filter setting which can get in the way: echo 0 /proc/sys/net/ipv4/conf/all/rp_filter Amos
Re: [squid-users] Squid AD login problem
On 25/07/2012 2:01 p.m., Eliezer Croitoru wrote: On 7/24/2012 4:13 PM, Nicole Hähnel wrote: Hi, recently we are using Squid 3.1.20 on SLES11 SP1 to control the webaccess in our Microsoft AD network. There are some internal microsoft based websites like Sharepoint for instance. Without squid we can open these websites without renewed authentication to the browser. With squid (wpad file) we get a login box, but in spite of the right credentials we won't be logged in. All computers are authenticated to the AD, so squid has to pass through the kerberos certificate. Are there any hints on that? Thanks! Kind regards, Nicole what is the content of the WPAD script? the access to the sharepoint and other internal server are through the squid server at all? do you see anything logged in the access.log file when you are trying to access the sharepoint page? Eliezer Sharepoint is on my little list of MS software which breaks HTTP when faced with Squid-3.1 slightly unique combination of HTTP/1.0 to clients and HTTP/1.1 to servers. It seems to have some keep-alive issues with Squid. The workaround is to use Squid-3.2 where more of HTTP/1.1 is supported. Meanwhile, I'm looking for someone to help design and run a permutation series of test cases on Squid to ensure it emits the correct keep-alive/close value in both directions. Amos
[squid-users] Traffic redirection
Hi all, I am trying to setup a topology like the one shown below where Squid will be a transparent proxy. I have a restriction so that I cannot use iptables to redirect traffic to Squid. So, there is a daemon in Box that captures http traffic from Client and re-writes its Destination IP to point to Squid and destination port to 3128. All boxes can access each other. The problem is, I ran tcpdump on all boxes and I do see traffic arriving at Squid, but Squid does not register a MISS or HIT. The actual data still comes from Apache. Do I need to re-write any HTTP header or some other configuration for this? Client --- Box --- Squid - Apache Thanks
[squid-users] Re: Traffic redirection
Hi all, I observed two more things: 1. I ran wireshark on the Squid box and observed that the client is looking for a service called ndl-aas on port 3128. But no such service is running on the system. 2. netstat shows that Squid listens on IPV6 addresses (shows tcp6 for port 3128). Are these normal and expected? Thanks On Wed, Jul 25, 2012 at 5:26 PM, Abhishek Chanda abhishek.li...@gmail.com wrote: Hi all, I am trying to setup a topology like the one shown below where Squid will be a transparent proxy. I have a restriction so that I cannot use iptables to redirect traffic to Squid. So, there is a daemon in Box that captures http traffic from Client and re-writes its Destination IP to point to Squid and destination port to 3128. All boxes can access each other. The problem is, I ran tcpdump on all boxes and I do see traffic arriving at Squid, but Squid does not register a MISS or HIT. The actual data still comes from Apache. Do I need to re-write any HTTP header or some other configuration for this? Client --- Box --- Squid - Apache Thanks
Re: [squid-users] Re: Traffic redirection
On 26.07.2012 13:54, Abhishek Chanda wrote: Hi all, I observed two more things: 1. I ran wireshark on the Squid box and observed that the client is looking for a service called ndl-aas on port 3128. But no such service is running on the system. Normal if your /etc/services is listing the IANA registrations instead of the SANS registrations. You can change the port 3128 entry in that file to http-proxy to make it show Squid clearer. 2. netstat shows that Squid listens on IPV6 addresses (shows tcp6 for port 3128). Are these normal and expected? Normal for IPv6-enabled Squid. Thanks On Wed, Jul 25, 2012 at 5:26 PM, Abhishek Chanda wrote: Hi all, I am trying to setup a topology like the one shown below where Squid will be a transparent proxy. I have a restriction so that I cannot use iptables to redirect traffic to Squid. So, there is a daemon in Box that captures http traffic from Client and re-writes its Destination IP to point to Squid and destination port to 3128. All boxes can access each other. The problem is, I ran tcpdump on all boxes and I do see traffic arriving at Squid, but Squid does not register a MISS or HIT. The actual data still comes from Apache. Do I need to re-write any HTTP header or some other configuration for this? Client --- Box --- Squid - Apache Thanks Squid version? Squid requires some way to determine that the mapping has taken place, and to identify what the original details were. The standard NAT functionality on your box usually provides this for DNAT via socket options. Question is why you can't use the built-in software? Amos
Re: [squid-users] Re: Traffic redirection
Hi Amos, Thanks for the reply. My Squid is 3.1.19. I am trying to use OpenFlow to automate the deployment of Squid in my organization. When the OpenFlow controller sees a new HTTP packet, it modifies it's destination IP and port to that of Squid and sends it back. Thus, I expected I will not need iptable rules here. I am a bit confused about how Squid does DNAT. Can you point me to some document? Thanks On Wed, Jul 25, 2012 at 8:11 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 26.07.2012 13:54, Abhishek Chanda wrote: Hi all, I observed two more things: 1. I ran wireshark on the Squid box and observed that the client is looking for a service called ndl-aas on port 3128. But no such service is running on the system. Normal if your /etc/services is listing the IANA registrations instead of the SANS registrations. You can change the port 3128 entry in that file to http-proxy to make it show Squid clearer. 2. netstat shows that Squid listens on IPV6 addresses (shows tcp6 for port 3128). Are these normal and expected? Normal for IPv6-enabled Squid. Thanks On Wed, Jul 25, 2012 at 5:26 PM, Abhishek Chanda wrote: Hi all, I am trying to setup a topology like the one shown below where Squid will be a transparent proxy. I have a restriction so that I cannot use iptables to redirect traffic to Squid. So, there is a daemon in Box that captures http traffic from Client and re-writes its Destination IP to point to Squid and destination port to 3128. All boxes can access each other. The problem is, I ran tcpdump on all boxes and I do see traffic arriving at Squid, but Squid does not register a MISS or HIT. The actual data still comes from Apache. Do I need to re-write any HTTP header or some other configuration for this? Client --- Box --- Squid - Apache Thanks Squid version? Squid requires some way to determine that the mapping has taken place, and to identify what the original details were. The standard NAT functionality on your box usually provides this for DNAT via socket options. Question is why you can't use the built-in software? Amos
