RE: [squid-users] Reverse Proxy Cluster Issues
I know it's terrible responding to my own thread, but this is still an issue. Does anyone have an idea what could be the problem? Squids I have set up do not contact its siblings for stale resources, but instead retrieve it from the origin server. There are 2 squid boxes that use each other as siblings. I believe all the http request and response headers are correct. Here is one such example from access.log file that represents request for stale resource that was retrieved directly from origin server without contacting squid sibling prior (mime headers and query terms are logged as well): 1272472882.323 2 10.208.26.217 TCP_REFRESH_UNMODIFIED/200 297 GET http://rss_site:8080/feedservice/RSSServlet?test=184 - FIRST_UP_PARENT/RSS_server - [Host: rss_site:8080\r\nCache-Control: max-age=1000, s-maxage=1000\r\n] [HTTP/1.1 200 OK\r\nServer: Apache-Coyote/1.1\r\nLast-Modified: Wed, 28 Apr 2010 16:37:48 GMT\r\nCache-Control: Public, max-age=10, s-maxage=10\r\nContent-Length: 9\r\nDate: Wed, 28 Apr 2010 16:41:09 GMT\r\n\r] Thank you. -Original Message- From: Cimic, Senad (Legal) Sent: Tuesday, April 20, 2010 4:22 PM To: squ...@treenet.co.nz; squid-users@squid-cache.org Subject: RE: [squid-users] Reverse Proxy Cluster Issues -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Thursday, April 15, 2010 8:55 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Reverse Proxy Cluster Issues senad.ci...@thomsonreuters.com wrote: > Thanks Amos, removing hierarchy_stoplist solved my query-string issue. > > However, I'm not sure what you meant by removing cache/no_cache controls. I > can't see any such operators in my squif.config file. Can you please > elaborate more? > Good. It's just a little bit of trash left over from very old configs which might have also been causing you issues. Amos > Thanks again. > > -Original Message- > From: Amos Jeffries [mailto:squ...@treenet.co.nz] > Sent: Wednesday, April 14, 2010 6:17 PM > To: squid-users@squid-cache.org > Subject: Re: [squid-users] Reverse Proxy Cluster Issues > > On Wed, 14 Apr 2010 08:13:01 -0500, > wrote: >> Hi, >> >> I am first time squid user and was wondering if could get some help. I >> tried to find answers to these questions on-line, but unsuccessfully... >> >> I have 2 squid boxes setup as reverse proxies in a cluster (they're >> using each other as siblings). On the backend I'm using single tomcat >> server that both squid boxes use to retrieve content. Squid version I'm >> using is 3.0. I'm running into couple issues: >> >> Issue #1: >> Whenever squid box receives request for url that contains querystring >> (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact >> sibling cache for that resource, but it retrieves it from the backend >> server right away. What's odd is that it works (sometimes...) when query >> string is not present (e.g. http://site1:8080/RSSSource/rss/feed). >> >> Issue #2: >> Let's say squidA receives request for some resource (e.g. >> http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its >> cache, it will check if it's available from squidB. However, if squidA >> has expired version of that resource, it doesn't contact squidB but >> retrieves it directly from the backend server, which should not be the >> case (it should check if squidB had valid copy available), correct? >> >> Here are relevant squid.conf lines for one of the squids (everything >> else is unchanged, config for the second squid is the same except for >> sibling references): > > Nope. > > The relevant lines are hierarchy_stoplist (prevent peers being asked for > query-string URLs). > and cache/no_cache controls (prevent "QUERY" ACL matches being stored > locally.) > > Both of which need to be removed from your config. > > Amos > -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.1 I switched to squid 3.1.1 and the issue still remains: instead of contacting sibling for the expired resources, squid goes to the origin server right away. I have cluster of only 2 squids, here is the whole squid.config file for one of them (squid.config for the other one is the same except for sibling references). Is there anything I missed in the config that could be causing this issue? # # Recommended minimum configuration: # acl manager proto cache_object acl localhost src 127.0.0.1/32 acl localhost src ::1/128 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 acl to_localhost dst ::1/128 # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (di
RE: [squid-users] Reverse Proxy Cluster Issues
-Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Thursday, April 15, 2010 8:55 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Reverse Proxy Cluster Issues senad.ci...@thomsonreuters.com wrote: > Thanks Amos, removing hierarchy_stoplist solved my query-string issue. > > However, I'm not sure what you meant by removing cache/no_cache controls. I > can't see any such operators in my squif.config file. Can you please > elaborate more? > Good. It's just a little bit of trash left over from very old configs which might have also been causing you issues. Amos > Thanks again. > > -Original Message- > From: Amos Jeffries [mailto:squ...@treenet.co.nz] > Sent: Wednesday, April 14, 2010 6:17 PM > To: squid-users@squid-cache.org > Subject: Re: [squid-users] Reverse Proxy Cluster Issues > > On Wed, 14 Apr 2010 08:13:01 -0500, > wrote: >> Hi, >> >> I am first time squid user and was wondering if could get some help. I >> tried to find answers to these questions on-line, but unsuccessfully... >> >> I have 2 squid boxes setup as reverse proxies in a cluster (they're >> using each other as siblings). On the backend I'm using single tomcat >> server that both squid boxes use to retrieve content. Squid version I'm >> using is 3.0. I'm running into couple issues: >> >> Issue #1: >> Whenever squid box receives request for url that contains querystring >> (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact >> sibling cache for that resource, but it retrieves it from the backend >> server right away. What's odd is that it works (sometimes...) when query >> string is not present (e.g. http://site1:8080/RSSSource/rss/feed). >> >> Issue #2: >> Let's say squidA receives request for some resource (e.g. >> http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its >> cache, it will check if it's available from squidB. However, if squidA >> has expired version of that resource, it doesn't contact squidB but >> retrieves it directly from the backend server, which should not be the >> case (it should check if squidB had valid copy available), correct? >> >> Here are relevant squid.conf lines for one of the squids (everything >> else is unchanged, config for the second squid is the same except for >> sibling references): > > Nope. > > The relevant lines are hierarchy_stoplist (prevent peers being asked for > query-string URLs). > and cache/no_cache controls (prevent "QUERY" ACL matches being stored > locally.) > > Both of which need to be removed from your config. > > Amos > -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.1 I switched to squid 3.1.1 and the issue still remains: instead of contacting sibling for the expired resources, squid goes to the origin server right away. I have cluster of only 2 squids, here is the whole squid.config file for one of them (squid.config for the other one is the same except for sibling references). Is there anything I missed in the config that could be causing this issue? # # Recommended minimum configuration: # acl manager proto cache_object acl localhost src 127.0.0.1/32 acl localhost src ::1/128 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 acl to_localhost dst ::1/128 # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT # # Recommended minimum Access Permission configuration: # # Only allow cachemgr access from localhost http_access allow manager localhost http_access deny manager # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # We strongly recommend the following be uncommented to protect innocent # web applications running on the proxy server who think the only # one who can access services on "localhost" is a local user #http_access deny to_localhost # # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS # # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from w
RE: [squid-users] Reverse Proxy Cluster Issues
Thanks Amos, removing hierarchy_stoplist solved my query-string issue. However, I'm not sure what you meant by removing cache/no_cache controls. I can't see any such operators in my squif.config file. Can you please elaborate more? Thanks again. -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Wednesday, April 14, 2010 6:17 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Reverse Proxy Cluster Issues On Wed, 14 Apr 2010 08:13:01 -0500, wrote: > Hi, > > I am first time squid user and was wondering if could get some help. I > tried to find answers to these questions on-line, but unsuccessfully... > > I have 2 squid boxes setup as reverse proxies in a cluster (they're > using each other as siblings). On the backend I'm using single tomcat > server that both squid boxes use to retrieve content. Squid version I'm > using is 3.0. I'm running into couple issues: > > Issue #1: > Whenever squid box receives request for url that contains querystring > (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact > sibling cache for that resource, but it retrieves it from the backend > server right away. What's odd is that it works (sometimes...) when query > string is not present (e.g. http://site1:8080/RSSSource/rss/feed). > > Issue #2: > Let's say squidA receives request for some resource (e.g. > http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its > cache, it will check if it's available from squidB. However, if squidA > has expired version of that resource, it doesn't contact squidB but > retrieves it directly from the backend server, which should not be the > case (it should check if squidB had valid copy available), correct? > > Here are relevant squid.conf lines for one of the squids (everything > else is unchanged, config for the second squid is the same except for > sibling references): Nope. The relevant lines are hierarchy_stoplist (prevent peers being asked for query-string URLs). and cache/no_cache controls (prevent "QUERY" ACL matches being stored locally.) Both of which need to be removed from your config. Amos
[squid-users] Reverse Proxy Cluster Issues
Hi, I am first time squid user and was wondering if could get some help. I tried to find answers to these questions on-line, but unsuccessfully... I have 2 squid boxes setup as reverse proxies in a cluster (they're using each other as siblings). On the backend I'm using single tomcat server that both squid boxes use to retrieve content. Squid version I'm using is 3.0. I'm running into couple issues: Issue #1: Whenever squid box receives request for url that contains querystring (e.g. - http://site1:8080/RSSSource/rss/feed?max=1) it does not contact sibling cache for that resource, but it retrieves it from the backend server right away. What's odd is that it works (sometimes...) when query string is not present (e.g. http://site1:8080/RSSSource/rss/feed). Issue #2: Let's say squidA receives request for some resource (e.g. http://site1:8080/RSSSource/rss/feed). If squidA doesn't have it in its cache, it will check if it's available from squidB. However, if squidA has expired version of that resource, it doesn't contact squidB but retrieves it directly from the backend server, which should not be the case (it should check if squidB had valid copy available), correct? Here are relevant squid.conf lines for one of the squids (everything else is unchanged, config for the second squid is the same except for sibling references): ## http_port 80 accel vhost icp_port 3130 acl sites_server1 dstdomain site1 acl siblings src address.of.squidA.com cache_peer address.of.backend.server.com parent 8080 0 no-query no-digest originserver name=server1 cache_peer address.of.squidA.com sibling 80 3130 name=sibling1 no-digest allow-miss weight=5 cache_peer_access server1 allow sites_server1 cache_peer_access server1 allow siblings cache_peer_access sibling1 allow sites_server1 http_access allow sites_server1 http_access allow siblings http_access deny all icp_access allow siblings icp_access deny all miss_access deny siblings ### I tried using HTCP instead of ICP, but I got same results... Does anyone know solution to these 2 problems? One thing I didn't mention is that tomcat backend server is including conditional get headers in responses, however I don't think it matters... Let me know if more info needed. Thanks in advance!
