RE: [squid-users] Reverse Proxy Cluster Issues
I know it's terrible responding to my own thread, but this is still an issue. Does anyone have an idea what could be the problem? Squids I have set up do not contact its siblings for stale resources, but instead retrieve it from the origin server. There are 2 squid boxes that use each other as siblings. I believe all the http request and response headers are correct. Here is one such example from access.log file that represents request for stale resource that was retrieved directly from origin server without contacting squid sibling prior (mime headers and query terms are logged as well): 1272472882.323 2 10.208.26.217 TCP_REFRESH_UNMODIFIED/200 297 GET http://rss_site:8080/feedservice/RSSServlet?test=184 - FIRST_UP_PARENT/RSS_server - [Host: rss_site:8080\r\nCache-Control: max-age=1000, s-maxage=1000\r\n] [HTTP/1.1 200 OK\r\nServer: Apache-Coyote/1.1\r\nLast-Modified: Wed, 28 Apr 2010 16:37:48 GMT\r\nCache-Control: Public, max-age=10, s-maxage=10\r\nContent-Length: 9\r\nDate: Wed, 28 Apr 2010 16:41:09 GMT\r\n\r] Thank you. -Original Message- From: Cimic, Senad (Legal) Sent: Tuesday, April 20, 2010 4:22 PM To: squ...@treenet.co.nz; squid-users@squid-cache.org Subject: RE: [squid-users] Reverse Proxy Cluster Issues -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Thursday, April 15, 2010 8:55 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Reverse Proxy Cluster Issues senad.ci...@thomsonreuters.com wrote: Thanks Amos, removing hierarchy_stoplist solved my query-string issue. However, I'm not sure what you meant by removing cache/no_cache controls. I can't see any such operators in my squif.config file. Can you please elaborate more? Good. It's just a little bit of trash left over from very old configs which might have also been causing you issues. Amos Thanks again. -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Wednesday, April 14, 2010 6:17 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Reverse Proxy Cluster Issues On Wed, 14 Apr 2010 08:13:01 -0500, senad.ci...@thomsonreuters.com wrote: Hi, I am first time squid user and was wondering if could get some help. I tried to find answers to these questions on-line, but unsuccessfully... I have 2 squid boxes setup as reverse proxies in a cluster (they're using each other as siblings). On the backend I'm using single tomcat server that both squid boxes use to retrieve content. Squid version I'm using is 3.0. I'm running into couple issues: Issue #1: Whenever squid box receives request for url that contains querystring (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact sibling cache for that resource, but it retrieves it from the backend server right away. What's odd is that it works (sometimes...) when query string is not present (e.g. http://site1:8080/RSSSource/rss/feed). Issue #2: Let's say squidA receives request for some resource (e.g. http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its cache, it will check if it's available from squidB. However, if squidA has expired version of that resource, it doesn't contact squidB but retrieves it directly from the backend server, which should not be the case (it should check if squidB had valid copy available), correct? Here are relevant squid.conf lines for one of the squids (everything else is unchanged, config for the second squid is the same except for sibling references): Nope. The relevant lines are hierarchy_stoplist (prevent peers being asked for query-string URLs). and cache/no_cache controls (prevent QUERY ACL matches being stored locally.) Both of which need to be removed from your config. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.1 I switched to squid 3.1.1 and the issue still remains: instead of contacting sibling for the expired resources, squid goes to the origin server right away. I have cluster of only 2 squids, here is the whole squid.config file for one of them (squid.config for the other one is the same except for sibling references). Is there anything I missed in the config that could be causing this issue? # # Recommended minimum configuration: # acl manager proto cache_object acl localhost src 127.0.0.1/32 acl localhost src ::1/128 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 acl to_localhost dst ::1/128 # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl
RE: [squid-users] Reverse Proxy Cluster Issues
-Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Thursday, April 15, 2010 8:55 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Reverse Proxy Cluster Issues senad.ci...@thomsonreuters.com wrote: Thanks Amos, removing hierarchy_stoplist solved my query-string issue. However, I'm not sure what you meant by removing cache/no_cache controls. I can't see any such operators in my squif.config file. Can you please elaborate more? Good. It's just a little bit of trash left over from very old configs which might have also been causing you issues. Amos Thanks again. -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Wednesday, April 14, 2010 6:17 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Reverse Proxy Cluster Issues On Wed, 14 Apr 2010 08:13:01 -0500, senad.ci...@thomsonreuters.com wrote: Hi, I am first time squid user and was wondering if could get some help. I tried to find answers to these questions on-line, but unsuccessfully... I have 2 squid boxes setup as reverse proxies in a cluster (they're using each other as siblings). On the backend I'm using single tomcat server that both squid boxes use to retrieve content. Squid version I'm using is 3.0. I'm running into couple issues: Issue #1: Whenever squid box receives request for url that contains querystring (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact sibling cache for that resource, but it retrieves it from the backend server right away. What's odd is that it works (sometimes...) when query string is not present (e.g. http://site1:8080/RSSSource/rss/feed). Issue #2: Let's say squidA receives request for some resource (e.g. http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its cache, it will check if it's available from squidB. However, if squidA has expired version of that resource, it doesn't contact squidB but retrieves it directly from the backend server, which should not be the case (it should check if squidB had valid copy available), correct? Here are relevant squid.conf lines for one of the squids (everything else is unchanged, config for the second squid is the same except for sibling references): Nope. The relevant lines are hierarchy_stoplist (prevent peers being asked for query-string URLs). and cache/no_cache controls (prevent QUERY ACL matches being stored locally.) Both of which need to be removed from your config. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.1 I switched to squid 3.1.1 and the issue still remains: instead of contacting sibling for the expired resources, squid goes to the origin server right away. I have cluster of only 2 squids, here is the whole squid.config file for one of them (squid.config for the other one is the same except for sibling references). Is there anything I missed in the config that could be causing this issue? # # Recommended minimum configuration: # acl manager proto cache_object acl localhost src 127.0.0.1/32 acl localhost src ::1/128 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 acl to_localhost dst ::1/128 # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT # # Recommended minimum Access Permission configuration: # # Only allow cachemgr access from localhost http_access allow manager localhost http_access deny manager # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # We strongly recommend the following be uncommented to protect innocent # web applications running on the proxy server who think the only # one who can access services on localhost is a local user #http_access deny to_localhost # # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS # # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from where browsing should be allowed http_access allow
RE: [squid-users] Reverse Proxy Cluster Issues
Thanks Amos, removing hierarchy_stoplist solved my query-string issue. However, I'm not sure what you meant by removing cache/no_cache controls. I can't see any such operators in my squif.config file. Can you please elaborate more? Thanks again. -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Wednesday, April 14, 2010 6:17 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Reverse Proxy Cluster Issues On Wed, 14 Apr 2010 08:13:01 -0500, senad.ci...@thomsonreuters.com wrote: Hi, I am first time squid user and was wondering if could get some help. I tried to find answers to these questions on-line, but unsuccessfully... I have 2 squid boxes setup as reverse proxies in a cluster (they're using each other as siblings). On the backend I'm using single tomcat server that both squid boxes use to retrieve content. Squid version I'm using is 3.0. I'm running into couple issues: Issue #1: Whenever squid box receives request for url that contains querystring (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact sibling cache for that resource, but it retrieves it from the backend server right away. What's odd is that it works (sometimes...) when query string is not present (e.g. http://site1:8080/RSSSource/rss/feed). Issue #2: Let's say squidA receives request for some resource (e.g. http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its cache, it will check if it's available from squidB. However, if squidA has expired version of that resource, it doesn't contact squidB but retrieves it directly from the backend server, which should not be the case (it should check if squidB had valid copy available), correct? Here are relevant squid.conf lines for one of the squids (everything else is unchanged, config for the second squid is the same except for sibling references): Nope. The relevant lines are hierarchy_stoplist (prevent peers being asked for query-string URLs). and cache/no_cache controls (prevent QUERY ACL matches being stored locally.) Both of which need to be removed from your config. Amos
Re: [squid-users] Reverse Proxy Cluster Issues
senad.ci...@thomsonreuters.com wrote: Thanks Amos, removing hierarchy_stoplist solved my query-string issue. However, I'm not sure what you meant by removing cache/no_cache controls. I can't see any such operators in my squif.config file. Can you please elaborate more? Good. It's just a little bit of trash left over from very old configs which might have also been causing you issues. Amos Thanks again. -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Wednesday, April 14, 2010 6:17 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Reverse Proxy Cluster Issues On Wed, 14 Apr 2010 08:13:01 -0500, senad.ci...@thomsonreuters.com wrote: Hi, I am first time squid user and was wondering if could get some help. I tried to find answers to these questions on-line, but unsuccessfully... I have 2 squid boxes setup as reverse proxies in a cluster (they're using each other as siblings). On the backend I'm using single tomcat server that both squid boxes use to retrieve content. Squid version I'm using is 3.0. I'm running into couple issues: Issue #1: Whenever squid box receives request for url that contains querystring (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact sibling cache for that resource, but it retrieves it from the backend server right away. What's odd is that it works (sometimes...) when query string is not present (e.g. http://site1:8080/RSSSource/rss/feed). Issue #2: Let's say squidA receives request for some resource (e.g. http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its cache, it will check if it's available from squidB. However, if squidA has expired version of that resource, it doesn't contact squidB but retrieves it directly from the backend server, which should not be the case (it should check if squidB had valid copy available), correct? Here are relevant squid.conf lines for one of the squids (everything else is unchanged, config for the second squid is the same except for sibling references): Nope. The relevant lines are hierarchy_stoplist (prevent peers being asked for query-string URLs). and cache/no_cache controls (prevent QUERY ACL matches being stored locally.) Both of which need to be removed from your config. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.1
Re: [squid-users] Reverse Proxy Cluster Issues
On 14/04/2010 11:34 AM, senad.ci...@thomsonreuters.com wrote: Hi Ron, Thank you for the quick response. I'm still not clear on these unfortunately: Issue 1: I believe squid should use query parameters as well when evaluating cached objects. Let's say I request object from squidA http://site1:8080/RSSSource/rss/feed?max=1. Let's say squidA doesn't have it in cache, so it will get it from the backend server. If I request the same object while it is fresh in cache, it will get it from cache. Similarly, if I request object with different query parameter http://site1:8080/RSSSource/rss/feed?max=2 it will recognize that it is a request for a different object (which is not cached) and it will retrieve it from the backend server. The issue in these scenarios is that it never checks if its sibling has those objects. It is impossible for squid to know what Tomcat will do with ?max=1 It needs to let Tomcat generate a new page. Issue 2: I'm using ICP to communicate between siblings. If cacheA receives request for an object that is in its cache but expired, it should send ICP query to squidB, correct? Depending if it is available in squidB, squidB will respond with either UDP_HIT or UDP_MISS response, so there shouldn't be any loops, correct? Beyond my pay scale. Perhaps someone with a more detailed knowledge of peer caches will explain what has to be done to make this work if it is possible. Thanks again, Senad -Original Message- From: Ron Wheeler [mailto:rwhee...@artifact-software.com] Sent: Wednesday, April 14, 2010 9:11 AM To: Cimic, Senad (Legal) Subject: Re: [squid-users] Reverse Proxy Cluster Issues On 14/04/2010 9:13 AM, senad.ci...@thomsonreuters.com wrote: Hi, I am first time squid user and was wondering if could get some help. I tried to find answers to these questions on-line, but unsuccessfully... I have 2 squid boxes setup as reverse proxies in a cluster (they're using each other as siblings). On the backend I'm using single tomcat server that both squid boxes use to retrieve content. Squid version I'm using is 3.0. I'm running into couple issues: Issue #1: Whenever squid box receives request for url that contains querystring (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact sibling cache for that resource, but it retrieves it from the backend server right away. What's odd is that it works (sometimes...) when query string is not present (e.g. http://site1:8080/RSSSource/rss/feed). Issue #2: Let's say squidA receives request for some resource (e.g. http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its cache, it will check if it's available from squidB. However, if squidA has expired version of that resource, it doesn't contact squidB but retrieves it directly from the backend server, which should not be the case (it should check if squidB had valid copy available), correct? Here are relevant squid.conf lines for one of the squids (everything else is unchanged, config for the second squid is the same except for sibling references): ## http_port 80 accel vhost icp_port 3130 acl sites_server1 dstdomain site1 acl siblings src address.of.squidA.com cache_peer address.of.backend.server.com parent 8080 0 no-query no-digest originserver name=server1 cache_peer address.of.squidA.com sibling 80 3130 name=sibling1 no-digest allow-miss weight=5 cache_peer_access server1 allow sites_server1 cache_peer_access server1 allow siblings cache_peer_access sibling1 allow sites_server1 http_access allow sites_server1 http_access allow siblings http_access deny all icp_access allow siblings icp_access deny all miss_access deny siblings ### I tried using HTCP instead of ICP, but I got same results... Does anyone know solution to these 2 problems? One thing I didn't mention is that tomcat backend server is including conditional get headers in responses, however I don't think it matters... Let me know if more info needed. Thanks in advance! Issue # 1 is the correct behaviour. Squid has no way of predicting what you might want to do with those arguments. Issue #2 looks like a good decision. What is both have expired versions? They could get into a loop going back and forth forever unless someone is really careful. Ron
Re: [squid-users] Reverse Proxy Cluster Issues
On Wed, 14 Apr 2010 08:13:01 -0500, senad.ci...@thomsonreuters.com wrote: Hi, I am first time squid user and was wondering if could get some help. I tried to find answers to these questions on-line, but unsuccessfully... I have 2 squid boxes setup as reverse proxies in a cluster (they're using each other as siblings). On the backend I'm using single tomcat server that both squid boxes use to retrieve content. Squid version I'm using is 3.0. I'm running into couple issues: Issue #1: Whenever squid box receives request for url that contains querystring (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact sibling cache for that resource, but it retrieves it from the backend server right away. What's odd is that it works (sometimes...) when query string is not present (e.g. http://site1:8080/RSSSource/rss/feed). Issue #2: Let's say squidA receives request for some resource (e.g. http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its cache, it will check if it's available from squidB. However, if squidA has expired version of that resource, it doesn't contact squidB but retrieves it directly from the backend server, which should not be the case (it should check if squidB had valid copy available), correct? Here are relevant squid.conf lines for one of the squids (everything else is unchanged, config for the second squid is the same except for sibling references): Nope. The relevant lines are hierarchy_stoplist (prevent peers being asked for query-string URLs). and cache/no_cache controls (prevent QUERY ACL matches being stored locally.) Both of which need to be removed from your config. Amos
