[squid-users] Squid 3.0 and Linux 2.6 kernel tweaks
Hi All, I searched through the mail archives and squid-cache site, but I’ve been unable to find anything that mentions any kernel tweaks that may or may not be necessary with a 2.6 kernel. I am mostly concerned with running high volume reverse proxy setups. For example, back in the Squid v 2.6 days with 2.2 or 2.4 linux kernels I see lots of recommendations for raising the file descriptor limits to 8192 or 16384. With the 2.6 kernel it seems the default kernel params (at least in RHEL5) far exceed the fd kernel tweaks people used for 2.2 or 2.4 kernels. It would seem adding the ulimit –HSn 8192 would actually be decreasing the file descriptor limits from the 2.6 kernel defaults. Also, what is the default value for –with-filedescriptors that Squid 3.0 STABLE24 supports? I don’t see that in the output of ./compile –help I’d be happy to add a wiki page addressing the following scalability topics if someone points me to the correct location. - Checking/Increasing the ephemeral port range - Checking/increasing file descriptor limits - Checking/decreasing TCP TIME_WAIT Regards, Andy Andy Litzinger ▪ Sr. Network Engineer o. 206.436.8086 ▪ f. 206.213.0606 ▪ http://www.theplatform.com
RE: [squid-users] Squid 3.0 and Linux 2.6 kernel tweaks
fre 2010-02-19 klockan 10:10 -0800 skrev Andy Litzinger: For example, back in the Squid v 2.6 days with 2.2 or 2.4 linux kernels I see lots of recommendations for raising the file descriptor limits to 8192 or 16384. With the 2.6 kernel it seems the default kernel params (at least in RHEL5) far exceed the fd kernel tweaks people used for 2.2 or 2.4 kernels. It would seem adding the ulimit – HSn 8192 would actually be decreasing the file descriptor limits from the 2.6 kernel defaults. 2.6 kernel default ulimit is 1024. We run with stock kernels from CentOS/RHEL so I guess I meant in those the kernel and shell fd limits are way higher. Also, what is the default value for –with-filedescriptors that Squid 3.0 STABLE24 supports? I don’t see that in the output of ./compile – help On must systems the default is whatever the ulimit is set to when you run configure. Great, thanks. Is there any way to confirm this on a compiled squid, or is it best practice to define the value upon compilation? I’d be happy to add a wiki page addressing the following scalability topics if someone points me to the correct location. - Checking/Increasing the ephemeral port range usually not needed unless you have many hundreds/s forwarded requests. - Checking/increasing file descriptor limits Squid tells at startup what limit it is running under. I'm not sure I understand what you mean here. How/where does squid get this value? And I suppose I should have said checking/increasing the kernel file descriptor limits (/proc/sys/fs/file-max) and the shell file descriptor limits (ulimit -n). - Checking/decreasing TCP TIME_WAIT Usually not needed. Closely connected to the ephemeral port range issue mentioned above. I understand that TIME_WAIT and ephemeral port increases are not usually needed, but I am concerned with the case of reverse proxying thousands of very short lived requests per second. I suppose it's likely for the service to die long before I exhaust available resources, but at least I'll know I won't be bottlenecking anything. I appreciate your feedback! I do think it would be valuable for this type of qualified information to make it into the wiki somewhere. I'll look for the process to do so, but if you have any hints as to where this info should live I would love to hear them. Cheers, Andy
[squid-users] RE: Advisory SQUID-2010:2 - Remote Denial of Service issue in HCTP
Does the HTCP port have to be open towards the attacker or can the attacker exploit the bug through a squid listening port? i.e. If I have a firewall in front of squid (reverse proxy) that only allows port 80/443 in from the web and HTCP is bound to some other port am I at risk from attackers outside my firewall? -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Friday, February 12, 2010 6:30 AM To: squid-annou...@squid-cache.org; Squid Subject: Advisory SQUID-2010:2 - Remote Denial of Service issue in HCTP __ Squid Proxy Cache Security Update Advisory SQUID-2010:2 __ Advisory ID:SQUID-2010:2 Date: February 12, 2010 Summary:Remote Denial of Service issue in HCTP Affected versions: Squid 2.x, Squid 3.0 - 3.0.STABLE23 Fixed in version: Squid 3.0.STABLE24 __ http://www.squid-cache.org/Advisories/SQUID-2010_2.txt __ Problem Description: Due to incorrect processing Squid is vulnerable to a denial of service attack when receiving specially crafted HTCP packets. __ Severity: This problem allows any machine to perform a denial of service attack on the Squid service when its HTCP port is open. __ Updated Packages: This bug is fixed by Squid versions 3.0.STABLE24 In addition, patches addressing these problems can be found In our patch archives. Squid 2.7: http://www.squid-cache.org/Versions/v2/2.7/changesets/12600.patch Squid 3.0: http://www.squid-cache.org/Versions/v3/3.0/changesets/3.0-ADV-2010_2.patch If you are using a prepackaged version of Squid then please refer to the package vendor for availability information on updated packages. __ Determining if your version is vulnerable: All Squid-3.0 releases without htcp_port in their configuration file (the default) are not vulnerable. Squid-3.1 releases are not vulnerable. For unpatched Squid-2.x and Squid-3.0 releases; if your cache.log contains a line with Accepting HTCP messages on port when run with debug level 1 (debug_options ALL,1). Your Squid is vulnerable. Alternatively; for unpatched Squid-2.x and Squid-3.0 releases. If the command squidclient mgr:config | grep htcp_port displays a non-zero HTCP port your Squid is vulnerable. __ Workarounds: For Squid-2.x: * Configuring htcp_port 0 explicitly For Squid-3.0: * Ensuring that any unnecessary htcp_port setting left in squid.conf after upgrading to 3.0 are removed. __ Contact details for the Squid project: For installation / upgrade support on binary packaged versions of Squid: Your first point of contact should be your binary package vendor. If your install and build Squid from the original Squid sources then the squid-users@squid-cache.org mailing list is your primary support point. For subscription details see http://www.squid-cache.org/Support/mailing-lists.html. For reporting of non-security bugs in the latest STABLE release the squid bugzilla database should be used http://www.squid-cache.org/bugs/. For reporting of security sensitive bugs send an email to the squid-b...@squid-cache.org mailing list. It's a closed list (though anyone can post) and security related bug reports are treated in confidence until the impact has been established. __ Credits: The vulnerability was discovered by Kieran Whitbread. __ Revision history: 2010-02-12 14:11 GMT Initial Release __ END
[squid-users] Reverse Proxy that listens and forwards to multiple ports to the same backend server
Hi all, I'm banging my head on what I think should be a simple config. I want squid to receive requests on port 80 and forward them on to the origin server on port 80. I also want squid to receive requests on port 8081 and forward requests to the same origin server on port 8081. I have a Load Balancer (BigIP) sitting in front of my Squid server and the origin server Squid points to is also actually a VIP on the LB that sits in front of a pool of real origin servers. The goal is simple proxy- I'm not caching anything (that is working fine). Clients connect to http/https://my.test.com This resolves in my DNS to 192.168.94.225, a VIP hosted on the LB that forwards traffic on to Squid. The origin server VIP for the content is 192.168.94.226 This is what the flows should look like focusing only on the destination TCP port as it goes through each device: Desired HTTP request flow: Request port 80 --- LB --- request port 80 --- Squid --- request port 80 --- origin VIP on LB request port 8080 --- server listening on port 8080 Desired HTTPS request flow: Request port 443 --- LB (SSL offload) --- request port 8081 --- Squid --- request port 8081 --- Origin VIP on LB request port 8081 --- server listening on port 8081 What I see happening for the HTTPS requests is that the request arrives properly at the squid server on port 8081, but squid forwards the request to the Origin VIP on port 80 instead of 8081. Here is the config I'm trying: http_port 80 accel defaultsite=my.test.com http_port 8081 accel defaultsite=my.test.com icp_port 0 htcp_port 0 snmp_port 3401 debug_options ALL,1 33,2 cache_peer 192.168.94.226 parent 80 0 no-query no-digest originserver name=my_test cache_peer 192.168.94.226 parent 8081 0 no-query no-digest originserver name=my_test_ssl acl our_http_port port 80 acl our_ssl_port port 8081 acl my_test_dom dstdomain my.test.com cache_peer_access my_test_ssl allow our_ssl_port my_test_dom cache_peer_access my_test_ssl deny all cache_peer_access my_test allow our_http_port my_test_dom cache_peer_access my_test deny all # acl to block caching acl our_sites dstdomain .test.com # acl listing the IP of each vip acl vips dst 192.168.94.225 acl acceleratedPort port 80 8081 # we do NOT want the responses to # any requests to be cached. cache deny our_sites # Allow requests to make it through to the VIPs # but only on the expected ports http_access allow vips acceleratedPort http_access deny all http_reply_access allow all cache_effective_user squid cache_effective_group squid visible_hostname testproxy.test.com unique_hostname testsquid01 client_db off uri_whitespace allow strip_query_terms off relaxed_header_parser on minimum_expiry_time 30 seconds request_header_access Accept-Encoding deny all any suggestions? Thanks! Andy
[squid-users] RE: Reverse Proxy that listens and forwards to multiple ports to the same backend server
I should have mentioned I am running Squid3.0 Stable 18 -Original Message- From: Andy Litzinger Sent: Wednesday, August 12, 2009 10:03 AM To: 'squid-users@squid-cache.org' Subject: Reverse Proxy that listens and forwards to multiple ports to the same backend server Hi all, I'm banging my head on what I think should be a simple config. I want squid to receive requests on port 80 and forward them on to the origin server on port 80. I also want squid to receive requests on port 8081 and forward requests to the same origin server on port 8081. I have a Load Balancer (BigIP) sitting in front of my Squid server and the origin server Squid points to is also actually a VIP on the LB that sits in front of a pool of real origin servers. The goal is simple proxy- I'm not caching anything (that is working fine). Clients connect to http/https://my.test.com This resolves in my DNS to 192.168.94.225, a VIP hosted on the LB that forwards traffic on to Squid. The origin server VIP for the content is 192.168.94.226 This is what the flows should look like focusing only on the destination TCP port as it goes through each device: Desired HTTP request flow: Request port 80 --- LB --- request port 80 --- Squid --- request port 80 --- origin VIP on LB request port 8080 --- server listening on port 8080 Desired HTTPS request flow: Request port 443 --- LB (SSL offload) --- request port 8081 --- Squid --- request port 8081 --- Origin VIP on LB request port 8081 --- server listening on port 8081 What I see happening for the HTTPS requests is that the request arrives properly at the squid server on port 8081, but squid forwards the request to the Origin VIP on port 80 instead of 8081. Here is the config I'm trying: http_port 80 accel defaultsite=my.test.com http_port 8081 accel defaultsite=my.test.com icp_port 0 htcp_port 0 snmp_port 3401 debug_options ALL,1 33,2 cache_peer 192.168.94.226 parent 80 0 no-query no-digest originserver name=my_test cache_peer 192.168.94.226 parent 8081 0 no-query no-digest originserver name=my_test_ssl acl our_http_port port 80 acl our_ssl_port port 8081 acl my_test_dom dstdomain my.test.com cache_peer_access my_test_ssl allow our_ssl_port my_test_dom cache_peer_access my_test_ssl deny all cache_peer_access my_test allow our_http_port my_test_dom cache_peer_access my_test deny all # acl to block caching acl our_sites dstdomain .test.com # acl listing the IP of each vip acl vips dst 192.168.94.225 acl acceleratedPort port 80 8081 # we do NOT want the responses to # any requests to be cached. cache deny our_sites # Allow requests to make it through to the VIPs # but only on the expected ports http_access allow vips acceleratedPort http_access deny all http_reply_access allow all cache_effective_user squid cache_effective_group squid visible_hostname testproxy.test.com unique_hostname testsquid01 client_db off uri_whitespace allow strip_query_terms off relaxed_header_parser on minimum_expiry_time 30 seconds request_header_access Accept-Encoding deny all any suggestions? Thanks! Andy
[squid-users] RE: Reverse Proxy that listens and forwards to multiple ports to the same backend server
I may have solved my own issue. It looks like my acl should use 'myport' instead of 'port' e.g. acl our_http_port port 80 should be: acl our_http_port myport 80 I'm not sure I understand the difference or why this works so I'd be happy to hear an explanation from anyone who knows. -andy -Original Message- From: Andy Litzinger Sent: Wednesday, August 12, 2009 10:30 AM To: Andy Litzinger; squid-users@squid-cache.org Subject: RE: Reverse Proxy that listens and forwards to multiple ports to the same backend server I should have mentioned I am running Squid3.0 Stable 18 -Original Message- From: Andy Litzinger Sent: Wednesday, August 12, 2009 10:03 AM To: 'squid-users@squid-cache.org' Subject: Reverse Proxy that listens and forwards to multiple ports to the same backend server Hi all, I'm banging my head on what I think should be a simple config. I want squid to receive requests on port 80 and forward them on to the origin server on port 80. I also want squid to receive requests on port 8081 and forward requests to the same origin server on port 8081. I have a Load Balancer (BigIP) sitting in front of my Squid server and the origin server Squid points to is also actually a VIP on the LB that sits in front of a pool of real origin servers. The goal is simple proxy- I'm not caching anything (that is working fine). Clients connect to http/https://my.test.com This resolves in my DNS to 192.168.94.225, a VIP hosted on the LB that forwards traffic on to Squid. The origin server VIP for the content is 192.168.94.226 This is what the flows should look like focusing only on the destination TCP port as it goes through each device: Desired HTTP request flow: Request port 80 --- LB --- request port 80 --- Squid --- request port 80 --- origin VIP on LB request port 8080 --- server listening on port 8080 Desired HTTPS request flow: Request port 443 --- LB (SSL offload) --- request port 8081 --- Squid --- request port 8081 --- Origin VIP on LB request port 8081 --- server listening on port 8081 What I see happening for the HTTPS requests is that the request arrives properly at the squid server on port 8081, but squid forwards the request to the Origin VIP on port 80 instead of 8081. Here is the config I'm trying: http_port 80 accel defaultsite=my.test.com http_port 8081 accel defaultsite=my.test.com icp_port 0 htcp_port 0 snmp_port 3401 debug_options ALL,1 33,2 cache_peer 192.168.94.226 parent 80 0 no-query no-digest originserver name=my_test cache_peer 192.168.94.226 parent 8081 0 no-query no-digest originserver name=my_test_ssl acl our_http_port port 80 acl our_ssl_port port 8081 acl my_test_dom dstdomain my.test.com cache_peer_access my_test_ssl allow our_ssl_port my_test_dom cache_peer_access my_test_ssl deny all cache_peer_access my_test allow our_http_port my_test_dom cache_peer_access my_test deny all # acl to block caching acl our_sites dstdomain .test.com # acl listing the IP of each vip acl vips dst 192.168.94.225 acl acceleratedPort port 80 8081 # we do NOT want the responses to # any requests to be cached. cache deny our_sites # Allow requests to make it through to the VIPs # but only on the expected ports http_access allow vips acceleratedPort http_access deny all http_reply_access allow all cache_effective_user squid cache_effective_group squid visible_hostname testproxy.test.com unique_hostname testsquid01 client_db off uri_whitespace allow strip_query_terms off relaxed_header_parser on minimum_expiry_time 30 seconds request_header_access Accept-Encoding deny all any suggestions? Thanks! Andy
[squid-users] RE: TCP_MISS/200 with squid-2.7.STABLE6 Reverse proxy config
Have you tried bumping up the logging level and seeing what the cache.log tells you? http://www.squid-cache.org/Versions/v2/2.7/cfgman/debug_options.html You could try bumping up the debug level for all components, or you could focus on particular section. If I were you I might start with bumping up debug for all components to level 3 (of 9): debug_options ALL,3 you can keep bumping that up, but if the log gets too verbose for you to parse you might try debugging only the sections that are likely culprits. The list of debug sections can be found in the source tarball under doc/debug-sections.txt . also in Ch. 16 of the O'reilly squid book if you have it (thought the list there is not as up to date) so for an example, if you were wanted to up the debug level to 3 on just access controls you could try the following: debug_options ALL,1 28,3 You'll find it's easier to parse these debug messages if you do this in a test or low request volume environment. Also, do you have any example requests that you expect to cache, but aren't? -andy -Original Message- From: Quin Guin [mailto:quing...@yahoo.com] Sent: Thursday, April 16, 2009 1:19 PM To: squid-users@squid-cache.org Subject: TCP_MISS/200 with squid-2.7.STABLE6 Reverse proxy config Hi, I have been using squid for many years as a forward proxy and now I need to setup a reverse. I have read and study many different email threads and FAQ on this topic but I can't seem to get past TCP_MISS/200s. Please see my most basic config below and I know there is a lot more that can be done to make it more secure but I am just trying to get a TCP_MISS/200 then a TCP_HIT!!! I am open to trying things and I tried installing 3.1 on RHELL4-U6 64 bit but it has its keeps giving this error: configure: error: pthread library required but cannot be found. I will work on that later. http_port 81 accel defaultsite=f99.net cache_peer 10.20.20.39 parent 88 0 no-query originserver login=PASS name=dtvAccel ##ACL# acl ALL dstdomain f99.net http_access allow ALL cache_peer_access dtvAccel allow All cache_peer_access dtvAccel deny all ##Headers## via on header_access Via allow all header_access Age deny all header_access X-Cache deny all ##Cache Config## collapsed_forwarding on minimum_expiry_time 120 seconds cache_mem 256 MB maximum_object_size 40960 KB maximum_object_size_in_memory 50 KB ipcache_size 40960 # dc setting changed - orig first - new second # cache_dir aufs /usr/local/squid-2.7/var/cache 5 16 256 cache_dir ufs /usr/local/squid/var/cache 5000 16 256 access_log /usr/local/squid/var/logs/access.log squid cache_store_log /usr/local/squid/var/logs/squid-store.log #refresh_pattern ^ftp: 144020% 10080 #refresh_pattern ^gopher:14400% 1440 #refresh_pattern (/cgi-bin/|\?) 0 20% 720 refresh_pattern -i \.jpg$ 10 90% 10 override-expire override-lastmod ignore-reload reload-into-ims refresh_pattern -i \.jpeg$ 10 90% 10 override-expire override-lastmod ignore-reload reload-into-ims refresh_pattern -i \.gif$ 10 90% 10 override-expire override-lastmod ignore-reload reload-into-ims refresh_pattern -i \.png$ 10 90% 10 override-expire override-lastmod ignore-reload reload-into-ims refresh_pattern -i \.swf$ 10 90% 10 override-expire override-lastmod ignore-reload reload-into-ims refresh_pattern -i \.flv$ 10 90% 10 override-expire override-lastmod ignore-reload reload-into-ims refresh_pattern -i \.js$ 2 90% 2 override-expire override-lastmod ignore-reload reload-into-ims refresh_pattern -i \.css$ 2 90% 2 override-expire override-lastmod ignore-reload reload-into-ims refresh_pattern -i \.htm$10 90% 10 refresh_pattern -i \.html$ 10 90% 10 #icp_access allow all cache_mgr quing...@yahoo.com visible_hostname diuqs logfile_rotate 12 coredump_dir /usr/local/squid/var/cache Thank you very much, Quin
