Re: [squid-users] problem using squid as proxy server to load balance reverse-proxies
On Fri, Feb 11, 2011 at 1:14 PM, Amos Jeffries squ...@treenet.co.nz wrote:
On 12/02/11 06:37, Sri Rao wrote:
Hi Amos,
I am trying to setup squid as a ssl proxy to load balance btwn
reverse-proxies. I believe the config is right but what is happening
What you have setup is a forward proxy load balancer which only permits
management and binary-over-HTTP tunneled traffic from its localhost
machine
IP.
That is actually what I want. I want to do binary-over-HTTP from the
localhost to the reverse-proxy servers. When the forward proxy tries
to connect to the origin server directly it does a tunnelConnect but
even though I have set originserver for the cache_peers it seems to
just forward the CONNECT instead of doing a tunnelConnect. I thought
originserver should force squid to treat the cache_peers as if they
were web servers?
It should. You seem to have found a bug there. I've added a fix for that
now.
Where can I grab the fix?
It will be here when the mirrors next update:
http://www.squid-cache.org/Versions/v3/3.1/changesets/squid-3.1-10230.patch
A secondary problem in your config was never_direct allow sp_test -
since
sp_test always matches direct tunnel setup (tunnelConnect) is not
permitted.
yeah I never want it to go direct to the origin. I want it to connect
to the peers but as the originserver which should still be
tunnelConnect right?
Hmm, I think I finally get what you are trying to do. :)
And no Squid's handling of CONNECT is not smart enough to do CONNECT
properly to origins when the origin is a cache_peer without direct TCP
access from Squid.
tunnelConnect is Squid being a gateway and converting the CONNECT into a
TCP tunnel directly CONNECTed from to the destination server. Similar to the
way SSH would for example. Bytes are shuffled but squid sees none of them.
Like so:
client--(CONNECT)--Squid --(direct TCP)--some host
using cache_peer is Squid passing an HTTP requests (just happens to be
CONNECT) on unchanged for another proxy cache_peer to process. The tunnel
data is just a regular HTTP body entity to Squid, same as a POST with data
going both ways to the client and cache_peer.
Like so:
client--(CONNECT)--Squid--(CONNECT)--Other proxy--(direct TCP)--some
host
inside the tunnel:
client --(binary)-- some host
In your case you have the peer origins hostname in the CONNECT destination
yes? so allowing CONNECT to go direct will go there.
I think you should be doing never_direct deny of everything *except*
CONNECT requests to your internal origins.
Okay let's say I am trying to loadbalance using squid to 2 origin
servers. The 2 origin servers would be setup as cache_peers applying
the originserver directive no? Right now that wouldn't happen. It
would return not allowed for cache_peers right? The patch below would
allow for cache_peers if set as originserver to do the passthru you
are talking about above.
I thought a possible patch could be:
diff -Naur squid-3.1.11/src/tunnel.cc squid-3.1.11-cf/src/tunnel.cc
--- squid-3.1.11/src/tunnel.cc 2011-02-07 20:05:51.0 -0800
+++ squid-3.1.11-cf/src/tunnel.cc 2011-02-11 11:08:34.256181949 -0800
@@ -589,10 +589,10 @@
err-callback_data = tunnelState;
errorSend(tunnelState-client.fd(), err);
} else {
-if (tunnelState-servers-_peer)
-tunnelProxyConnected(tunnelState-server.fd(), tunnelState);
-else {
+if (!tunnelState-servers-_peer ||
tunnelState-servers-_peer-options.originserver)
tunnelConnected(tunnelState-server.fd(), tunnelState);
+else {
+tunnelProxyConnected(tunnelState-server.fd(), tunnelState);
}
commSetTimeout(tunnelState-server.fd(),
Wondering if there are reasons that this shouldn't be done?
Sri
Re: [squid-users] problem using squid as proxy server to load balance reverse-proxies
On 12/02/11 22:05, Sri Rao wrote:
snip
Okay let's say I am trying to loadbalance using squid to 2 origin
servers. The 2 origin servers would be setup as cache_peers applying
the originserver directive no? Right now that wouldn't happen. It
would return not allowed for cache_peers right? The patch below would
allow for cache_peers if set as originserver to do the passthru you
are talking about above.
I thought a possible patch could be:
diff -Naur squid-3.1.11/src/tunnel.cc squid-3.1.11-cf/src/tunnel.cc
--- squid-3.1.11/src/tunnel.cc 2011-02-07 20:05:51.0 -0800
+++ squid-3.1.11-cf/src/tunnel.cc 2011-02-11 11:08:34.256181949 -0800
@@ -589,10 +589,10 @@
err-callback_data = tunnelState;
errorSend(tunnelState-client.fd(), err);
} else {
-if (tunnelState-servers-_peer)
-tunnelProxyConnected(tunnelState-server.fd(), tunnelState);
-else {
+if (!tunnelState-servers-_peer ||
tunnelState-servers-_peer-options.originserver)
tunnelConnected(tunnelState-server.fd(), tunnelState);
+else {
+tunnelProxyConnected(tunnelState-server.fd(), tunnelState);
}
commSetTimeout(tunnelState-server.fd(),
Wondering if there are reasons that this shouldn't be done?
Hmm, my brain seems not to have been working much. :(
Yes that appears a correct and useful solution. Thank you.
If you can test this and verify that it produces the right operation for
your needs I'll replace the earlier patch with this one.
Amos
--
Please be using
Current Stable Squid 2.7.STABLE9 or 3.1.11
Beta testers wanted for 3.2.0.4
Re: [squid-users] problem using squid as proxy server to load balance reverse-proxies
Yep. The patch seems to be working correctly.
Thanks,
Sri
On Sat, Feb 12, 2011 at 1:21 PM, Amos Jeffries squ...@treenet.co.nz wrote:
On 12/02/11 22:05, Sri Rao wrote:
snip
Okay let's say I am trying to loadbalance using squid to 2 origin
servers. The 2 origin servers would be setup as cache_peers applying
the originserver directive no? Right now that wouldn't happen. It
would return not allowed for cache_peers right? The patch below would
allow for cache_peers if set as originserver to do the passthru you
are talking about above.
I thought a possible patch could be:
diff -Naur squid-3.1.11/src/tunnel.cc squid-3.1.11-cf/src/tunnel.cc
--- squid-3.1.11/src/tunnel.cc 2011-02-07 20:05:51.0 -0800
+++ squid-3.1.11-cf/src/tunnel.cc 2011-02-11 11:08:34.256181949
-0800
@@ -589,10 +589,10 @@
err-callback_data = tunnelState;
errorSend(tunnelState-client.fd(), err);
} else {
- if (tunnelState-servers-_peer)
- tunnelProxyConnected(tunnelState-server.fd(), tunnelState);
- else {
+ if (!tunnelState-servers-_peer ||
tunnelState-servers-_peer-options.originserver)
tunnelConnected(tunnelState-server.fd(), tunnelState);
+ else {
+ tunnelProxyConnected(tunnelState-server.fd(), tunnelState);
}
commSetTimeout(tunnelState-server.fd(),
Wondering if there are reasons that this shouldn't be done?
Hmm, my brain seems not to have been working much. :(
Yes that appears a correct and useful solution. Thank you.
If you can test this and verify that it produces the right operation for
your needs I'll replace the earlier patch with this one.
Amos
--
Please be using
Current Stable Squid 2.7.STABLE9 or 3.1.11
Beta testers wanted for 3.2.0.4
Re: [squid-users] problem using squid as proxy server to load balance reverse-proxies
On 11/02/11 19:25, Sri Rao wrote: Hi Amos, Thanks for the quick reply! I am trying to setup squid as a ssl proxy to load balance btwn reverse-proxies. I believe the config is right but what is happening What you have setup is a forward proxy load balancer which only permits management and binary-over-HTTP tunneled traffic from its localhost machine IP. That is actually what I want. I want to do binary-over-HTTP from the localhost to the reverse-proxy servers. When the forward proxy tries to connect to the origin server directly it does a tunnelConnect but even though I have set originserver for the cache_peers it seems to just forward the CONNECT instead of doing a tunnelConnect. I thought originserver should force squid to treat the cache_peers as if they were web servers? It should. You seem to have found a bug there. I've added a fix for that now. A secondary problem in your config was never_direct allow sp_test - since sp_test always matches direct tunnel setup (tunnelConnect) is not permitted. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.11 Beta testers wanted for 3.2.0.4
Re: [squid-users] problem using squid as proxy server to load balance reverse-proxies
Hi Amos, I am trying to setup squid as a ssl proxy to load balance btwn reverse-proxies. I believe the config is right but what is happening What you have setup is a forward proxy load balancer which only permits management and binary-over-HTTP tunneled traffic from its localhost machine IP. That is actually what I want. I want to do binary-over-HTTP from the localhost to the reverse-proxy servers. When the forward proxy tries to connect to the origin server directly it does a tunnelConnect but even though I have set originserver for the cache_peers it seems to just forward the CONNECT instead of doing a tunnelConnect. I thought originserver should force squid to treat the cache_peers as if they were web servers? It should. You seem to have found a bug there. I've added a fix for that now. Where can I grab the fix? A secondary problem in your config was never_direct allow sp_test - since sp_test always matches direct tunnel setup (tunnelConnect) is not permitted. yeah I never want it to go direct to the origin. I want it to connect to the peers but as the originserver which should still be tunnelConnect right? Thanks, Sri
Re: [squid-users] problem using squid as proxy server to load balance reverse-proxies
On 12/02/11 06:37, Sri Rao wrote: Hi Amos, I am trying to setup squid as a ssl proxy to load balance btwn reverse-proxies. I believe the config is right but what is happening What you have setup is a forward proxy load balancer which only permits management and binary-over-HTTP tunneled traffic from its localhost machine IP. That is actually what I want. I want to do binary-over-HTTP from the localhost to the reverse-proxy servers. When the forward proxy tries to connect to the origin server directly it does a tunnelConnect but even though I have set originserver for the cache_peers it seems to just forward the CONNECT instead of doing a tunnelConnect. I thought originserver should force squid to treat the cache_peers as if they were web servers? It should. You seem to have found a bug there. I've added a fix for that now. Where can I grab the fix? It will be here when the mirrors next update: http://www.squid-cache.org/Versions/v3/3.1/changesets/squid-3.1-10230.patch A secondary problem in your config was never_direct allow sp_test - since sp_test always matches direct tunnel setup (tunnelConnect) is not permitted. yeah I never want it to go direct to the origin. I want it to connect to the peers but as the originserver which should still be tunnelConnect right? Hmm, I think I finally get what you are trying to do. :) And no Squid's handling of CONNECT is not smart enough to do CONNECT properly to origins when the origin is a cache_peer without direct TCP access from Squid. tunnelConnect is Squid being a gateway and converting the CONNECT into a TCP tunnel directly CONNECTed from to the destination server. Similar to the way SSH would for example. Bytes are shuffled but squid sees none of them. Like so: client--(CONNECT)--Squid --(direct TCP)--some host using cache_peer is Squid passing an HTTP requests (just happens to be CONNECT) on unchanged for another proxy cache_peer to process. The tunnel data is just a regular HTTP body entity to Squid, same as a POST with data going both ways to the client and cache_peer. Like so: client--(CONNECT)--Squid--(CONNECT)--Other proxy--(direct TCP)--some host inside the tunnel: client --(binary)-- some host In your case you have the peer origins hostname in the CONNECT destination yes? so allowing CONNECT to go direct will go there. I think you should be doing never_direct deny of everything *except* CONNECT requests to your internal origins. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.11 Beta testers wanted for 3.2.0.4
[squid-users] problem using squid as proxy server to load balance reverse-proxies
Hi, I am trying to setup squid as a ssl proxy to load balance btwn reverse-proxies. I believe the config is right but what is happening is that squid gets the CONNECT request and connects to the reverse servers on the right port but forwards the CONNECT request instead of connecting to them as the originserver. I am pasting the config as it is right now. I am using localhost as test reverse proxies just for testing. It Also doesn't seem to be failing to the next peer when the first one it selects either returns an error(http error code or connection failure) and I have retry_on_error. Thanks for your help! Sri pid_filename /var/run/squid_sptest.pid debug_options ALL,1 44,9 26,9 17,9 3,9 5,9 15,9 33,9 39,9 61,9 21,5 http_port 127.0.0.1:7174 hierarchy_stoplist cgi-bin ? retry_on_error on refresh_pattern . 0 0% 0 refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 acl sp_test myport 7174 acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 10.0.0.0/8 # RFC 1918 possible internal network acl localnet src 172.16.0.0/12 # RFC 1918 possible internal network acl localnet src 192.168.0.0/16 # RFC 1918 possible internal network acl SSL_ports port 443 acl CONNECT method CONNECT http_access allow sp_test localhost CONNECT http_access allow manager localhost http_access deny manager http_access deny CONNECT !SSL_ports http_access deny all cache_peer 127.0.0.1 parent 8174 0 originserver proxy-only no-query round-robin weight=2 default cache_peer 127.0.0.12 parent 8174 0 originserver proxy-only no-query round-robin weight=1 cache_peer_access 127.0.0.1 allow sp_test cache_peer_access 127.0.0.12 allow sp_test cache_peer_access 127.0.0.1 deny all cache_peer_access 127.0.0.12 deny all never_direct allow sp_test cache deny all
Re: [squid-users] problem using squid as proxy server to load balance reverse-proxies
On 11/02/11 09:00, Sri Rao wrote: Hi, I am trying to setup squid as a ssl proxy to load balance btwn reverse-proxies. I believe the config is right but what is happening What you have setup is a forward proxy load balancer which only permits management and binary-over-HTTP tunneled traffic from its localhost machine IP. is that squid gets the CONNECT request and connects to the reverse servers on the right port but forwards the CONNECT request instead of connecting to them as the originserver. I am pasting the config as it is right now. I am using localhost as test reverse proxies just for testing. It Also doesn't seem to be failing to the next peer when the first one it selects either returns an error(http error code or connection failure) and I have retry_on_error. This would be an artifact of the special handling CONNECT requests have. Your goal of having an SSL proxy directly opposes the use of CONNECT. Since CONNECT is a binary-over-HTTP tunnel. I suggest going back to your first stated criteria setup squid as a ssl proxy and getting that going. This means using the https_port directive (NOT the http_port!!). With a server SSL certificate. Squid will then be an SSL proxy. * Problem 2 is then how to get browsers etc to send traffic to it. Since your third criteria is to pass traffic to reverse proxies it implies that this is to be a front-end reverse-proxy itself. If that is correct, then setup the https_port with the reverse-proxy accel options. And do a standard reverse-proxy to two backends configuration. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.11 Beta testers wanted for 3.2.0.4
Re: [squid-users] problem using squid as proxy server to load balance reverse-proxies
Hi Amos, Thanks for the quick reply! I am trying to setup squid as a ssl proxy to load balance btwn reverse-proxies. I believe the config is right but what is happening What you have setup is a forward proxy load balancer which only permits management and binary-over-HTTP tunneled traffic from its localhost machine IP. That is actually what I want. I want to do binary-over-HTTP from the localhost to the reverse-proxy servers. When the forward proxy tries to connect to the origin server directly it does a tunnelConnect but even though I have set originserver for the cache_peers it seems to just forward the CONNECT instead of doing a tunnelConnect. I thought originserver should force squid to treat the cache_peers as if they were web servers? is that squid gets the CONNECT request and connects to the reverse servers on the right port but forwards the CONNECT request instead of connecting to them as the originserver. I am pasting the config as it is right now. I am using localhost as test reverse proxies just for testing. It Also doesn't seem to be failing to the next peer when the first one it selects either returns an error(http error code or connection failure) and I have retry_on_error. This would be an artifact of the special handling CONNECT requests have. Your goal of having an SSL proxy directly opposes the use of CONNECT. Since CONNECT is a binary-over-HTTP tunnel. I suggest going back to your first stated criteria setup squid as a ssl proxy and getting that going. I would rather not have to maintain certs as I will have several of these squid proxies. This means using the https_port directive (NOT the http_port!!). With a server SSL certificate. Squid will then be an SSL proxy. * Problem 2 is then how to get browsers etc to send traffic to it. Since your third criteria is to pass traffic to reverse proxies it implies that this is to be a front-end reverse-proxy itself. If that is correct, then setup the https_port with the reverse-proxy accel options. And do a standard reverse-proxy to two backends configuration. Thanks for the info...will definitely keep this in mind. Sri
