Re: [squid-users] Transparent Squid Stalls For Up To Two Minutes
I appreciate your response. I don't believe it's a file system issue, I've tried troubleshooting that for several weeks. Originally, I was using 16 256 (the default) as directory layout. I've tried using ext4, reiser (my favorite filesystem) and now it's on btrfs. I also have the filesystem mounted with noatime. When I was using reiser, I had disabled tail packing as well. As you can see, I'm using aufs, but I've also tried diskd. The IP tables NAT/DNAT stuff happens at my router. See this DD-WRT wiki article for how it's done (http://www.dd-wrt.com/wiki/index.php/Transparent_Proxy), I actually wrote the section on multiple hosts can bypass the proxy. Either way, it's not a router issue. If I set my browser to the use the proxy directly, the delays still happen 99% of the time. Originally,I was using dans with antivirus. But the delays have gotten to be horrible. I went back to a standard squid setup to try to resolve the problem. At this point, I simply want to get squid working because a lot of the sites we visit continously may benefit from cacheing (news sites with lots of graphics, etc). Once I get this problem resolved, I'll go back to using dans w/ antivirus. 10.0.0.254 (the squid host) is excluded from the IP tables rules on DD-WRT, along with my Xbox 360, my BluRay player, my HD-DVD player and my DirecTV receiver. The three DNS servers specified in the squid.conf all resolve names properly and are open to the squid host. Thanks Doug Eubanks ad...@dougware.net 919-201-8750 Strange. What is the output of squid -v and squidclient mgr:info (AKA info cachmgr page)? Amos _ From: Amos Jeffries [mailto:squ...@treenet.co.nz] To: ad...@dougware.net Cc: squid-users@squid-cache.org Sent: Mon, 18 May 2009 14:55:39 + Subject: Re: [squid-users] Transparent Squid Stalls For Up To Two Minutes Doug Eubanks wrote: I'm having an intermittent squid issue. It's plagued me with CentOS 5.x, Fedora 6, and now Fedora 11 (all using the RPM build that came with the OS). My DD-WRT router forwards all of my outgoing port 80 requests to my transparent proxy using IP tables. For some reason, squid will hang when opening a URL for up to two minutes. It doesn't always happen and sometimes restarting squid will correct the problem (for a while). The system is pretty hefty 3ghz P4 with 2G of RAM with a SATA II drive. That should be plenty for a small home network of about 10 clients. When I test DNS lookups from the host, requests are returned within less than a second. I'm pretty sure that's not the problem. Here is my squid.conf, any input would be greatly appreciated! acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access allow localnet http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localnet http_access allow localhost http_access deny all htcp_access allow localnet htcp_access deny all http_port 3128 transparent Is the NAT / REDIRECT/DNAT happening on the Squid box? It needs to. hierarchy_stoplist cgi-bin ? cache_mem 32 MB maximum_object_size_in_memory 128 KB cache_replacement_policy heap LRU cache_dir aufs /var/spool/squid 4096 8 16 4GB of objects under 512KB small (avg set at 64KB later), using only an 8x16 inode array. You may have a FS overload problem. Also, Squid 'pulses' cache garbage collection one directory at a time. Very large amounts of files in any one directory can slow things down a lot at random times. It's generally better to increase the L1/L2 numbers from default as the cache gets bigger. max_open_disk_fds 0 minimum_object_size 0 KB maximum_object_size 512 KB access_log /var/log/squid/access.log squid refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern (cgi-bin|\?)0 0% 0 refresh_pattern . 0 20% 4320 visible_hostname doug-linux.dougware.net unique_hostname doug-linux.dougware.net coredump_dir /var/spool/squid cache_mgr ad...@dougware.net dns_nameservers 10.0.0.254 10.0.0.253 69.197.163.239 store_avg_object_size 64 KB
[squid-users] Transparent Squid Stalls For Up To Two Minutes
I'm having an intermittent squid issue. It's plagued me with CentOS 5.x, Fedora 6, and now Fedora 11 (all using the RPM build that came with the OS). My DD-WRT router forwards all of my outgoing port 80 requests to my transparent proxy using IP tables. For some reason, squid will hang when opening a URL for up to two minutes. It doesn't always happen and sometimes restarting squid will correct the problem (for a while). The system is pretty hefty 3ghz P4 with 2G of RAM with a SATA II drive. That should be plenty for a small home network of about 10 clients. When I test DNS lookups from the host, requests are returned within less than a second. I'm pretty sure that's not the problem. Here is my squid.conf, any input would be greatly appreciated! acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access allow localnet http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localnet http_access allow localhost http_access deny all htcp_access allow localnet htcp_access deny all http_port 3128 transparent hierarchy_stoplist cgi-bin ? cache_mem 32 MB maximum_object_size_in_memory 128 KB cache_replacement_policy heap LRU cache_dir aufs /var/spool/squid 4096 8 16 max_open_disk_fds 0 minimum_object_size 0 KB maximum_object_size 512 KB access_log /var/log/squid/access.log squid refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern (cgi-bin|\?)0 0% 0 refresh_pattern . 0 20% 4320 visible_hostname doug-linux.dougware.net unique_hostname doug-linux.dougware.net coredump_dir /var/spool/squid cache_mgr ad...@dougware.net dns_nameservers 10.0.0.254 10.0.0.253 69.197.163.239 store_avg_object_size 64 KB memory_replacement_policy heap LRU tcp_outgoing_address 10.0.0.254 udp_outgoing_address 10.0.0.254 Thanks Doug Eubanks ad...@dougware.net 919-201-8750
Re: [squid-users] Transparent Squid Stalls For Up To Two Minutes
Doug Eubanks wrote: I'm having an intermittent squid issue. It's plagued me with CentOS 5.x, Fedora 6, and now Fedora 11 (all using the RPM build that came with the OS). My DD-WRT router forwards all of my outgoing port 80 requests to my transparent proxy using IP tables. For some reason, squid will hang when opening a URL for up to two minutes. It doesn't always happen and sometimes restarting squid will correct the problem (for a while). The system is pretty hefty 3ghz P4 with 2G of RAM with a SATA II drive. That should be plenty for a small home network of about 10 clients. When I test DNS lookups from the host, requests are returned within less than a second. I'm pretty sure that's not the problem. Here is my squid.conf, any input would be greatly appreciated! acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access allow localnet http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localnet http_access allow localhost http_access deny all htcp_access allow localnet htcp_access deny all http_port 3128 transparent Is the NAT / REDIRECT/DNAT happening on the Squid box? It needs to. hierarchy_stoplist cgi-bin ? cache_mem 32 MB maximum_object_size_in_memory 128 KB cache_replacement_policy heap LRU cache_dir aufs /var/spool/squid 4096 8 16 4GB of objects under 512KB small (avg set at 64KB later), using only an 8x16 inode array. You may have a FS overload problem. Also, Squid 'pulses' cache garbage collection one directory at a time. Very large amounts of files in any one directory can slow things down a lot at random times. It's generally better to increase the L1/L2 numbers from default as the cache gets bigger. max_open_disk_fds 0 minimum_object_size 0 KB maximum_object_size 512 KB access_log /var/log/squid/access.log squid refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern (cgi-bin|\?)0 0% 0 refresh_pattern . 0 20% 4320 visible_hostname doug-linux.dougware.net unique_hostname doug-linux.dougware.net coredump_dir /var/spool/squid cache_mgr ad...@dougware.net dns_nameservers 10.0.0.254 10.0.0.253 69.197.163.239 store_avg_object_size 64 KB memory_replacement_policy heap LRU tcp_outgoing_address 10.0.0.254 udp_outgoing_address 10.0.0.254 Does 10.0.0.254 port 53 have access to ALL the DNS servers: 10.0.0.254 10.0.0.253 69.197.163.239 Are you excluding 10.0.0.254 from the interception at the DD-WRT? Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15 Current Beta Squid 3.1.0.7
Re: [squid-users] Transparent Squid Stalls For Up To Two Minutes
I appreciate your response. I don't believe it's a file system issue, I've tried troubleshooting that for several weeks. Originally, I was using 16 256 (the default) as directory layout. I've tried using ext4, reiser (my favorite filesystem) and now it's on btrfs. I also have the filesystem mounted with noatime. When I was using reiser, I had disabled tail packing as well. As you can see, I'm using aufs, but I've also tried diskd. The IP tables NAT/DNAT stuff happens at my router. See this DD-WRT wiki article for how it's done (http://www.dd-wrt.com/wiki/index.php/Transparent_Proxy), I actually wrote the section on multiple hosts can bypass the proxy. Either way, it's not a router issue. If I set my browser to the use the proxy directly, the delays still happen 99% of the time. Originally,I was using dans with antivirus. But the delays have gotten to be horrible. I went back to a standard squid setup to try to resolve the problem. At this point, I simply want to get squid working because a lot of the sites we visit continously may benefit from cacheing (news sites with lots of graphics, etc). Once I get this problem resolved, I'll go back to using dans w/ antivirus. 10.0.0.254 (the squid host) is excluded from the IP tables rules on DD-WRT, along with my Xbox 360, my BluRay player, my HD-DVD player and my DirecTV receiver. The three DNS servers specified in the squid.conf all resolve names properly and are open to the squid host. Thanks Doug Eubanks ad...@dougware.net 919-201-8750 _ From: Amos Jeffries [mailto:squ...@treenet.co.nz] To: ad...@dougware.net Cc: squid-users@squid-cache.org Sent: Mon, 18 May 2009 14:55:39 + Subject: Re: [squid-users] Transparent Squid Stalls For Up To Two Minutes Doug Eubanks wrote: I'm having an intermittent squid issue. It's plagued me with CentOS 5.x, Fedora 6, and now Fedora 11 (all using the RPM build that came with the OS). My DD-WRT router forwards all of my outgoing port 80 requests to my transparent proxy using IP tables. For some reason, squid will hang when opening a URL for up to two minutes. It doesn't always happen and sometimes restarting squid will correct the problem (for a while). The system is pretty hefty 3ghz P4 with 2G of RAM with a SATA II drive. That should be plenty for a small home network of about 10 clients. When I test DNS lookups from the host, requests are returned within less than a second. I'm pretty sure that's not the problem. Here is my squid.conf, any input would be greatly appreciated! acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access allow localnet http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localnet http_access allow localhost http_access deny all htcp_access allow localnet htcp_access deny all http_port 3128 transparent Is the NAT / REDIRECT/DNAT happening on the Squid box? It needs to. hierarchy_stoplist cgi-bin ? cache_mem 32 MB maximum_object_size_in_memory 128 KB cache_replacement_policy heap LRU cache_dir aufs /var/spool/squid 4096 8 16 4GB of objects under 512KB small (avg set at 64KB later), using only an 8x16 inode array. You may have a FS overload problem. Also, Squid 'pulses' cache garbage collection one directory at a time. Very large amounts of files in any one directory can slow things down a lot at random times. It's generally better to increase the L1/L2 numbers from default as the cache gets bigger. max_open_disk_fds 0 minimum_object_size 0 KB maximum_object_size 512 KB access_log /var/log/squid/access.log squid refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern (cgi-bin|\?)0 0% 0 refresh_pattern . 0 20% 4320 visible_hostname doug-linux.dougware.net unique_hostname doug-linux.dougware.net coredump_dir /var/spool/squid cache_mgr ad...@dougware.net dns_nameservers 10.0.0.254 10.0.0.253 69.197.163.239 store_avg_object_size 64 KB memory_replacement_policy heap LRU tcp_outgoing_address 10.0.0.254 udp_outgoing_address 10.0.0.254 Does 10.0.0.254 port 53 have
