Re: [squid-users] reverse proxies and Host request header

2015-06-12 Thread Amos Jeffries
On 13/06/2015 2:08 a.m., Julianne Bielski wrote:
 I have a general question about the use of the http Host request header by
 reverse proxies.
 
 As I understand it, the Host request header is used by transparent forward
 proxies as a way to route a request to the correct
 origin server since, unlike with an explicit proxy, the host is not
 included in the URI portion of the http method line.
 
 However, reverse proxies are always transparent from the perspective of
 the client and the Host header is often used by the proxy
 to map to the correct back end origin server.

The term transparent has been overloaded so much its meaningless by
itself.

Beyond what you are asking and Anthony already answered well. There is
the key difference of DNS involvement between reverse and interception
proxy.

A DNS lookup is used by the client to find and explicitly contact the
reverse proxy. The proxy is able to use that as a guarantee that if the
Host header does not contain a name its pre-configured for handling,
that it can/must reject the request entirely.

The same guarnatee for reverse proxies allows it make free use of all
the origin server features of HTTP without causing security problems to
the client/user. Thus the long list of problems at
http://wiki.squid-cache.org/SquidFaq/InterceptionProxy#Concepts_of_Interception_Caching
does not apply to reverse proxies.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


Re: [squid-users] reverse proxies and Host request header

2015-06-12 Thread Antony Stone
On Friday 12 June 2015 at 17:49:38 (EU time), Julianne Bielski wrote:

 With virtual hosting, the client is asking for a virtual origin server's
 host and DNS enables the ip address for the physical host to respond. The
 virtual host still sees its hostname in the host header, not the physical
 host's.

There is no virtual host to see its hostname in the header.  There is a 
single real web server (which might be a virtual machine, but that's a 
different use of the term virtual, and doesn't matter here one way or the 
other), which can happily handle requests for multiple websites (which you 
might want to call hostnames, domains, etc).

The single web server never sees its own hostname in the header (assuming that 
its real hostname is not one of the virtual names it serves web pages for), 
and the web service is quite happy with this.

 With a reverse proxy, the backend origin server doesn't see its Host name,
 it sees the proxy's, even though the proxy is an http client with respect
 to the origin server.

The backend server sees the hostname (if you want to call it that) of the 
website for which it is supposed to serve pages.

An example might help:

www.example.com
www.example.net
www.example.org
downloads.example.org

may all have DNS entries pointing to a single IP address.

The machine at that IP address (it doesn't matter what its own hostname is, so 
I won't even suggest what it might be) has a web server configured to respond 
to requests for any of those web sites, therefore clients get the expected 
answers, without necessarily realising that they're all served by the same 
machine at a single IP address.

Now put a reverse proxy in the way - the above hostnames now resolve to the IP 
address of the reverse proxy, it's configured to pass the requests on to the 
appropriate backend server/s, and if that's just a single machine, it is 
configured exactly the same way as the original web server was - it responds to 
requests for any of the virtual site names.


I hope that helps;

Regards,


Antony.

-- 
A user interface is like a joke.
If you have to explain it, it didn't work.

   Please reply to the list;
 please *don't* CC me.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


[squid-users] reverse proxies and Host request header

2015-06-12 Thread Julianne Bielski
I have a general question about the use of the http Host request header by
reverse proxies.

As I understand it, the Host request header is used by transparent forward
proxies as a way to route a request to the correct
origin server since, unlike with an explicit proxy, the host is not
included in the URI portion of the http method line.

However, reverse proxies are always transparent from the perspective of
the client and the Host header is often used by the proxy
to map to the correct back end origin server.

I also think they usually pass the Host header as-is to the origin server.
This last
piece puzzles me because it means that the origin server is being given a
different host name than itself in the header. Is this
behavior correct? Does it ever cause problems?

J. Bielski
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


Re: [squid-users] reverse proxies and Host request header

2015-06-12 Thread Julianne Bielski

With virtual hosting, the client is asking for a virtual origin server's
host and DNS enables the ip address for the physical host to respond. The
virtual host still sees its hostname in the host header, not the physical
host's.

With a reverse proxy, the backend origin server doesn't see its Host name,
it sees the proxy's, even though the proxy is an http client with respect
to the origin server.



From:   Antony Stone antony.st...@squid.open.source.it
To: squid-users@lists.squid-cache.org
Date:   06/12/2015 10:14 AM
Subject:Re: [squid-users] reverse proxies and Host request header
Sent by:squid-users squid-users-boun...@lists.squid-cache.org



On Friday 12 June 2015 at 16:08:59 (EU time), Julianne Bielski wrote:

 reverse proxies are always transparent from the perspective of
 the client and the Host header is often used by the proxy
 to map to the correct back end origin server.

 I also think they usually pass the Host header as-is to the origin
server.
 This last piece puzzles me because it means that the origin server is
being
 given a different host name than itself in the header. Is this behavior
 correct? Does it ever cause problems?

How is this different from a normal web server serving multiple virtual
host
sites?

The web server doesn't care who it is, it just cares which virtual host
it's
being asked to serve pages for.  A reverse proxy in the way basically makes
no
difference.


Regards,


Antony.

--
All generalisations are inaccurate.

   Please reply to the
list;
 please *don't* CC
me.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


Re: [squid-users] reverse proxies and Host request header

2015-06-12 Thread Antony Stone
On Friday 12 June 2015 at 16:08:59 (EU time), Julianne Bielski wrote:

 reverse proxies are always transparent from the perspective of
 the client and the Host header is often used by the proxy
 to map to the correct back end origin server.
 
 I also think they usually pass the Host header as-is to the origin server.
 This last piece puzzles me because it means that the origin server is being
 given a different host name than itself in the header. Is this behavior
 correct? Does it ever cause problems?

How is this different from a normal web server serving multiple virtual host 
sites?

The web server doesn't care who it is, it just cares which virtual host it's 
being asked to serve pages for.  A reverse proxy in the way basically makes no 
difference.


Regards,


Antony.

-- 
All generalisations are inaccurate.

   Please reply to the list;
 please *don't* CC me.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


Re: [squid-users] Reverse proxies...

2008-06-30 Thread John Doe
 I still have the GET internal://pc-03/squid-internal-periodic/store_digest 
 problem though..
 
 What problem? It's cache digest exchanges between the Squids..

Not really a problem but pc-03 (or it's corresponding IP) are nowhere in the 
conf files, and so I was just wondering how the same (internal://pc-03/) did 
end up in the 3 squids logs...
If the 3 squids were really on 3 different servers, there would be 3 different 
hostnames/IPs in the logs, right?
But no big deal as long as it works  ^_^

Thx,
JD


  



Re: [squid-users] Reverse proxies...

2008-06-30 Thread Henrik Nordstrom
On mån, 2008-06-30 at 03:08 -0700, John Doe wrote:
  I still have the GET internal://pc-03/squid-internal-periodic/store_digest 
  problem though..
  
  What problem? It's cache digest exchanges between the Squids..
 
 Not really a problem but pc-03 (or it's corresponding IP) are nowhere in the 
 conf files, and so I was just wondering how the same (internal://pc-03/) did 
 end up in the 3 squids logs...

Odd..

 If the 3 squids were really on 3 different servers, there would be 3 
 different hostnames/IPs in the logs, right?

IP yes. Hostname is what they request..

IP is from where the request was sent.

The hostname in the URL is what that IP requested.. or how it was
rebuilt using the http_port options in accelerator mode..

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Reverse proxies...

2008-06-27 Thread John Doe
Indeed, my unique server has 3 IP aliases.

Apaches:
 127.0.0.1:8081
 127.0.0.1:8082
 127.0.0.1:8083
Squids:
 192.168.17.11:80
 192.168.17.12:80
 192.168.17.13:80

I added udp_incoming_address 192.168.17.[11|12|13] respectively in each squid 
confs
I don't need to change udp_outgoing_address, do I?  (See netstat at the end of 
this post)

squid1:
 2008/06/27 10:39:20| Accepting accelerated HTTP connections at 192.168.17.11, 
port 80, FD 12.
 2008/06/27 10:39:20| Accepting ICP messages at 192.168.17.11, port 3130, FD 13.
 2008/06/27 10:39:20| Configuring Parent 127.0.0.1/8081/0
 2008/06/27 10:39:20| Configuring Sibling 192.168.17.12/80/3130
 2008/06/27 10:39:20| Configuring Sibling 192.168.17.13/80/3130

squid2:
 2008/06/27 10:39:23| Accepting accelerated HTTP connections at 192.168.17.12, 
port 80, FD 12.
 2008/06/27 10:39:23| Accepting ICP messages at 192.168.17.12, port 3130, FD 13.
 2008/06/27 10:39:23| Configuring Parent 127.0.0.1/8082/0
 2008/06/27 10:39:23| Configuring Sibling 192.168.17.11/80/3130
 2008/06/27 10:39:23| Configuring Sibling 192.168.17.13/80/3130

squid3:
 2008/06/27 10:39:27| Accepting accelerated HTTP connections at 192.168.17.13, 
port 80, FD 12.
 2008/06/27 10:39:27| Accepting ICP messages at 192.168.17.13, port 3130, FD 13.
 2008/06/27 10:39:27| Configuring Parent 127.0.0.1/8083/0
 2008/06/27 10:39:27| Configuring Sibling 192.168.17.11/80/3130
 2008/06/27 10:39:27| Configuring Sibling 192.168.17.12/80/3130

###
Purge everything on all squids and browse squid1...
###

squid1:
 1214557685.718  2 192.168.17.11 TCP_MISS/200 2329 GET 
http://192.168.17.11/ - FIRST_UP_PARENT/127.0.0.1 text/html
 1214557685.891  2 192.168.17.11 TCP_MISS/200 4378 GET 
http://192.168.17.11/img/apache_header.gif - FIRST_UP_PARENT/127.0.0.1 image/gif

 1214557685.716 RELEASE 00 0029 32AC333767667E106447F39486BD2C7F0   
 -1-1-1 unknown -1/0 NONE http://192.168.17.12/squid- 
internal-periodic/store_digest
 1214557685.716 SWAPOUT 00  1BDEF453F2C0A866DD905F6C79CF1002  200 
1214557685 1214555963 1214559563 application/cache-digest 147/147 GET  
http://192.168.17.12/squid-internal-periodic/store_digest
 1214557685.717 SWAPOUT 00 0001 8A4E6E4ABE750B9AFCBBE4277ADA20FA  200 
1214557685 1214468758 1214557985 text/html 2020/2020 GET http://192.168.17.11/
 1214557685.889 SWAPOUT 00 0002 8BC2233ECEF0C976FDD190FF1A652998  200 
1214557685 1214468039 1214557985 image/gif 4084/4084 GET http://192.168.17.11/ 
img/apache_header.gif

squid2:
 1214557685.187  0 192.168.17.11 UDP_MISS/000 42 ICP_QUERY 
http://192.168.17.11/ - NONE/- -
 1214557685.716  0 192.168.17.12 TCP_MEM_HIT/200 374 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest
 1214557685.716  0 192.168.17.11 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.11/img/apache_header.gif - NONE/- -

squid3:
 1214557685.331  0 192.168.17.11 UDP_MISS/000 42 ICP_QUERY 
http://192.168.17.11/ - NONE/- -
 1214557685.716  0 192.168.17.11 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.11/img/apache_header.gif - NONE/- -
 1214557745.716  0 192.168.17.13 TCP_MEM_HIT/200 374 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest

apache1:
 127.0.0.1 - - [27/Jun/2008:11:08:05 +0200] GET / HTTP/1.0 200 2020
 127.0.0.1 - - [27/Jun/2008:11:08:05 +0200] GET /img/apache_header.gif 
HTTP/1.0 200 4084

###
Browse squid1 again...
###

squid1:
 1214557857.361  0 192.168.17.11 TCP_IMS_HIT/304 254 GET 
http://192.168.17.11/ - NONE/- text/html
 1214557857.484122 192.168.17.11 TCP_IMS_HIT/304 239 GET 
http://192.168.17.11/img/apache_header.gif - NONE/- image/gif

 1214557857.361 RELEASE -1  F935846DBCC888FE053A2D9A6E8B529B  304 
1214557685 1214468758 1214561285 text/html -1/0 GET http://192.168.17.11/
 1214557857.484 RELEASE -1  8262DA350A386275C1F0FFE25DEDFD1F  304 
1214557685 1214468039 1214561285 image/gif -1/0 GET 
http://192.168.17.11/img/apache_header.gif

###
Browse squid2...
###

squid1:
 1214558005.099  0 192.168.17.12 UDP_MISS/000 42 ICP_QUERY 
http://192.168.17.12/ - NONE/- -
 1214558005.774  0 192.168.17.11 TCP_MEM_HIT/200 374 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest
 1214558005.860  0 192.168.17.12 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.12/img/apache_header.gif - NONE/- -

squid2:
 1214558005.955181 192.168.17.12 TCP_MISS/200 2329 GET 
http://192.168.17.12/ - TIMEOUT_FIRST_UP_PARENT/127.0.0.1 

Re: [squid-users] Reverse proxies...

2008-06-27 Thread John Doe
 Also, even if I have the following:
 2008/06/27 10:39:23| Local cache digest enabled; rebuild/rewrite every 
 600/3600 
 The digests don't seem to be rebuild every 10 minutes (or is it silent?).
 And they seem to exchange digests after the ICP_QUERY...

In fact, after 1 hour, I got:

squid1:
 1214563796.846 RELEASE -1  B3E1F4FCF08D694B1FD5E848C7533251  200 
1214560196 1214560196 1214563796 application/cache-digest 145/145 GET 
internal://pc-03.iper/squid-internal-periodic/store_digest
 1214564100.709 RELEASE 00  E2F7000207B9A6BD38C373925811D656  200 
1214560261 1214560200 1214563800 application/cache-digest 146/146 GET 
http://192.168.17.12/squid-internal-periodic/store_digest
 1214564100.709 SWAPOUT 00 0004 1BDEF453F2C0A866DD905F6C79CF1002  200 
1214564100 1214563800 1214567400 application/cache-digest 146/146 GET 
http://192.168.17.12/squid-internal-periodic/store_digest
 1214564160.709 RELEASE 00 0003 1B1CAAE973E8AF74C183BDC81300FD3B  200 
1214560321 1214560203 1214563803 application/cache-digest 146/146 GET 
http://192.168.17.13/squid-internal-periodic/store_digest
 1214564160.709 SWAPOUT 00 0005 6BB5F118D5D548296562668556E2FF74  200 
1214564160 1214563803 1214567403 application/cache-digest 146/146 GET 
http://192.168.17.13/squid-internal-periodic/store_digest

squid2:
 1214564100.709  0 192.168.17.12 TCP_MEM_HIT/200 373 GET 
internal://pc-03.iper/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest
 1214563800.002 RELEASE -1  072835FF91677E6ECD367A054F6744BD  200 
1214560200 1214560200 1214563800 application/cache-digest 146/146 GET 
internal://pc-03.iper/squid-internal-periodic/store_digest

squid3:
 1214564160.709  0 192.168.17.13 TCP_MEM_HIT/200 373 GET 
internal://pc-03.iper/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest
 1214563803.117 RELEASE -1  072835FF91677E6ECD367A054F6744BD  200 
1214560203 1214560203 1214563803 application/cache-digest 146/146 GET 
internal://pc-03.iper/squid-internal-periodic/store_digest

But still same problem...

JD


  



Re: [squid-users] Reverse proxies...

2008-06-27 Thread Amos Jeffries

John Doe wrote:

Indeed, my unique server has 3 IP aliases.

Apaches:
 127.0.0.1:8081
 127.0.0.1:8082
 127.0.0.1:8083
Squids:
 192.168.17.11:80
 192.168.17.12:80
 192.168.17.13:80

I added udp_incoming_address 192.168.17.[11|12|13] respectively in each squid 
confs
I don't need to change udp_outgoing_address, do I?  (See netstat at the end of 
this post)


Maybe you do. Definitely tcp_outgoing_address might be a good idea too.

Amos



squid1:
 2008/06/27 10:39:20| Accepting accelerated HTTP connections at 192.168.17.11, 
port 80, FD 12.
 2008/06/27 10:39:20| Accepting ICP messages at 192.168.17.11, port 3130, FD 13.
 2008/06/27 10:39:20| Configuring Parent 127.0.0.1/8081/0
 2008/06/27 10:39:20| Configuring Sibling 192.168.17.12/80/3130
 2008/06/27 10:39:20| Configuring Sibling 192.168.17.13/80/3130

squid2:
 2008/06/27 10:39:23| Accepting accelerated HTTP connections at 192.168.17.12, 
port 80, FD 12.
 2008/06/27 10:39:23| Accepting ICP messages at 192.168.17.12, port 3130, FD 13.
 2008/06/27 10:39:23| Configuring Parent 127.0.0.1/8082/0
 2008/06/27 10:39:23| Configuring Sibling 192.168.17.11/80/3130
 2008/06/27 10:39:23| Configuring Sibling 192.168.17.13/80/3130

squid3:
 2008/06/27 10:39:27| Accepting accelerated HTTP connections at 192.168.17.13, 
port 80, FD 12.
 2008/06/27 10:39:27| Accepting ICP messages at 192.168.17.13, port 3130, FD 13.
 2008/06/27 10:39:27| Configuring Parent 127.0.0.1/8083/0
 2008/06/27 10:39:27| Configuring Sibling 192.168.17.11/80/3130
 2008/06/27 10:39:27| Configuring Sibling 192.168.17.12/80/3130

###
Purge everything on all squids and browse squid1...
###

squid1:
 1214557685.718  2 192.168.17.11 TCP_MISS/200 2329 GET 
http://192.168.17.11/ - FIRST_UP_PARENT/127.0.0.1 text/html
 1214557685.891  2 192.168.17.11 TCP_MISS/200 4378 GET 
http://192.168.17.11/img/apache_header.gif - FIRST_UP_PARENT/127.0.0.1 image/gif

 1214557685.716 RELEASE 00 0029 32AC333767667E106447F39486BD2C7F0   
 -1-1-1 unknown -1/0 NONE http://192.168.17.12/squid- 
internal-periodic/store_digest
 1214557685.716 SWAPOUT 00  1BDEF453F2C0A866DD905F6C79CF1002  200 
1214557685 1214555963 1214559563 application/cache-digest 147/147 GET  
http://192.168.17.12/squid-internal-periodic/store_digest
 1214557685.717 SWAPOUT 00 0001 8A4E6E4ABE750B9AFCBBE4277ADA20FA  200 
1214557685 1214468758 1214557985 text/html 2020/2020 GET http://192.168.17.11/
 1214557685.889 SWAPOUT 00 0002 8BC2233ECEF0C976FDD190FF1A652998  200 
1214557685 1214468039 1214557985 image/gif 4084/4084 GET http://192.168.17.11/ 
img/apache_header.gif

squid2:
 1214557685.187  0 192.168.17.11 UDP_MISS/000 42 ICP_QUERY 
http://192.168.17.11/ - NONE/- -
 1214557685.716  0 192.168.17.12 TCP_MEM_HIT/200 374 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest
 1214557685.716  0 192.168.17.11 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.11/img/apache_header.gif - NONE/- -

squid3:
 1214557685.331  0 192.168.17.11 UDP_MISS/000 42 ICP_QUERY 
http://192.168.17.11/ - NONE/- -
 1214557685.716  0 192.168.17.11 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.11/img/apache_header.gif - NONE/- -
 1214557745.716  0 192.168.17.13 TCP_MEM_HIT/200 374 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest

apache1:
 127.0.0.1 - - [27/Jun/2008:11:08:05 +0200] GET / HTTP/1.0 200 2020
 127.0.0.1 - - [27/Jun/2008:11:08:05 +0200] GET /img/apache_header.gif 
HTTP/1.0 200 4084

###
Browse squid1 again...
###

squid1:
 1214557857.361  0 192.168.17.11 TCP_IMS_HIT/304 254 GET 
http://192.168.17.11/ - NONE/- text/html
 1214557857.484122 192.168.17.11 TCP_IMS_HIT/304 239 GET 
http://192.168.17.11/img/apache_header.gif - NONE/- image/gif

 1214557857.361 RELEASE -1  F935846DBCC888FE053A2D9A6E8B529B  304 
1214557685 1214468758 1214561285 text/html -1/0 GET http://192.168.17.11/
 1214557857.484 RELEASE -1  8262DA350A386275C1F0FFE25DEDFD1F  304 
1214557685 1214468039 1214561285 image/gif -1/0 GET 
http://192.168.17.11/img/apache_header.gif

###
Browse squid2...
###

squid1:
 1214558005.099  0 192.168.17.12 UDP_MISS/000 42 ICP_QUERY 
http://192.168.17.12/ - NONE/- -
 1214558005.774  0 192.168.17.11 TCP_MEM_HIT/200 374 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest
 1214558005.860  0 192.168.17.12 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.12/img/apache_header.gif - NONE/- -

squid2:
 1214558005.955

Re: [squid-users] Reverse proxies...

2008-06-27 Thread John Doe
- Original Message 
  Indeed, my unique server has 3 IP aliases.
  
  Apaches:
   127.0.0.1:8081
   127.0.0.1:8082
   127.0.0.1:8083
  Squids:
   192.168.17.11:80
   192.168.17.12:80
   192.168.17.13:80
  
  I added udp_incoming_address 192.168.17.[11|12|13] respectively in each 
  squid 
 confs
  I don't need to change udp_outgoing_address, do I?  (See netstat at the end 
  of 
 this post)
 
 Maybe you do. Definitely tcp_outgoing_address might be a good idea too.
 
 Amos

Ok, I try that.  But, the doc says: If udp_outgoing_address is set to 
255.255.255.255
(the default) it will use the same socket as udp_incoming_address.  So I 
thought I could leave the default...  And my netstat did show:

 udp0  0 192.168.17.13:3130  0.0.0.0:*  

 udp0  0 192.168.17.12:3130  0.0.0.0:*  

 udp0  0 192.168.17.11:3130  0.0.0.0:*  

 udp0  0 192.168.17.11:33134 0.0.0.0:*  

 udp0  0 192.168.17.12:33136 0.0.0.0:*  

 udp0  0 192.168.17.13:33137 0.0.0.0:*

==

So I did add 3 new IP aliases for the outgoings (outgoing ICP IPs do not show 
up in logs but they do in netstat):

2008/06/27 15:50:25| Accepting accelerated HTTP connections at 192.168.17.11, 
port 80, FD 14.
2008/06/27 15:50:25| Accepting ICP messages at 192.168.17.11, port 3130, FD 15.
2008/06/27 15:50:25| Outgoing ICP messages on port 3130, FD 16.
2008/06/27 15:50:25| Configuring Parent 127.0.0.1/8081/0
2008/06/27 15:50:25| Configuring Sibling 192.168.17.12/80/3130
2008/06/27 15:50:25| Configuring Sibling 192.168.17.13/80/3130

2008/06/27 15:50:28| Accepting accelerated HTTP connections at 192.168.17.12, 
port 80, FD 14.
2008/06/27 15:50:28| Accepting ICP messages at 192.168.17.12, port 3130, FD 15.
2008/06/27 15:50:28| Outgoing ICP messages on port 3130, FD 16.
2008/06/27 15:50:28| Configuring Parent 127.0.0.1/8082/0
2008/06/27 15:50:28| Configuring Sibling 192.168.17.11/80/3130
2008/06/27 15:50:28| Configuring Sibling 192.168.17.13/80/3130

2008/06/27 15:50:31| Accepting accelerated HTTP connections at 192.168.17.13, 
port 80, FD 14.
2008/06/27 15:50:31| Accepting ICP messages at 192.168.17.13, port 3130, FD 15.
2008/06/27 15:50:31| Outgoing ICP messages on port 3130, FD 16.
2008/06/27 15:50:31| Configuring Parent 127.0.0.1/8083/0
2008/06/27 15:50:31| Configuring Sibling 192.168.17.11/80/3130
2008/06/27 15:50:31| Configuring Sibling 192.168.17.12/80/3130

udp0  0 192.168.17.16:3130  0.0.0.0:*
udp0  0 192.168.17.13:3130  0.0.0.0:*
udp0  0 192.168.17.15:3130  0.0.0.0:*
udp0  0 192.168.17.12:3130  0.0.0.0:*
udp0  0 192.168.17.14:3130  0.0.0.0:*
udp0  0 192.168.17.11:3130  0.0.0.0:*
udp0  0 192.168.17.14:33213 0.0.0.0:*
udp0  0 192.168.17.15:33214 0.0.0.0:*
udp0  0 192.168.17.16:33215 0.0.0.0:*

==

Same test (fill the 3 squids then delete 1 object from squid1 and browse 
squid1) and same result (squid1 asks siblings but end up asking apache1):

squid1:
1214575092.895  0 192.168.17.11 TCP_MEM_HIT/200 2338 GET 
http://192.168.17.11/ - NONE/- text/html
1214575093.043148 192.168.17.11 TCP_MISS/200 4378 GET 
http://192.168.17.11/img/apache_header.gif - FIRST_UP_PARENT/127.0.0.1 image/gif

1214575093.042 SWAPOUT 00 0004 8BC2233ECEF0C976FDD190FF1A652998  200 
1214575093 1214468039 1214575393 image/gif 4084/4084 GET 
http://192.168.17.11/img/apache_header.gif

squid2:
1214575093.033  0 192.168.17.14 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.11/img/apache_header.gif - NONE/- -
1214575099.628  0 192.168.17.12 TCP_MEM_HIT/200 374 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest

1214575094.267 RELEASE 00 0004 F560EF12A0FB4963759C32554341CB170
-1-1-1 unknown -1/0 NONE 
http://192.168.17.13/squid-internal-periodic/store_digest
1214575094.267 SWAPOUT 00 0004 6BB5F118D5D548296562668556E2FF74  200 
1214575094 1214574631 1214578231 application/cache-digest 147/147 GET 
http://192.168.17.13/squid-internal-periodic/store_digest

squid3:
1214575092.501  0 192.168.17.14 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.11/img/apache_header.gif - NONE/- -
1214575094.267  0 192.168.17.13 TCP_MEM_HIT/200 374 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest

1214575099.628 RELEASE 00 0004 D03BA4AEAA228DAF8EA08C9197F055220
-1-1-1 unknown -1/0 NONE 
http://192.168.17.12/squid-internal-periodic/store_digest
1214575099.628 SWAPOUT 00 0004 1BDEF453F2C0A866DD905F6C79CF1002  200 
1214575099 1214574628 1214578228 application/cache-digest 147/147 GET 

Re: [squid-users] Reverse proxies...

2008-06-27 Thread John Doe
My bad...

I did not realize that the ICP_QUERY URL should have been the apache IPs and 
not the squids IPs...
I was accessing directly my squids instead of using them as transparent proxies.
Once setup as proxies in my browser, I get:
ICP_QUERY http://127.0.0.1/img/apache_header.gif
which will match for all squids.

I still have the GET internal://pc-03/squid-internal-periodic/store_digest 
problem though..

JD


  



Re: [squid-users] Reverse proxies...

2008-06-27 Thread Henrik Nordstrom
On lör, 2008-06-28 at 01:02 +1200, Amos Jeffries wrote:

 Maybe you do. Definitely tcp_outgoing_address might be a good idea too.

It helps sanity.

But only udp_incoming_address, not udp_outgoing_address. See the
description of udp_outgoing_address for why..

  ###
  Browse squid2...
  ###
  
  squid1:
   1214558005.099  0 192.168.17.12 UDP_MISS/000 42 ICP_QUERY 
  http://192.168.17.12/ - NONE/- -

I would expect an UDP_HIT here. The object is in squid1..

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Reverse proxies...

2008-06-27 Thread Henrik Nordstrom

On fre, 2008-06-27 at 03:25 -0700, John Doe wrote:

 squid1:
  1214557685.718  2 192.168.17.11 TCP_MISS/200 2329 GET 
 http://192.168.17.11/ - FIRST_UP_PARENT/127.0.0.1 text/html

 ###
 Browse squid2...
 ###
 
 squid1:
  1214558005.099  0 192.168.17.12 UDP_MISS/000 42 ICP_QUERY 
 http://192.168.17.12/ - NONE/- -

Now I see the error. The URL you use is server specific, and it fails
because of this reason alone.

You need to make a more real test, sending the same request, with the
same Host header. I.e. if you use a browser you need to use the exact
same URL in the url-bar, and direct it to the right server by other
means.

Alternatively, if you only host a single site, you can configure Squid
with the defaultsite= http_port option telling your Squids what the
actual site name is (not the per-server IP, that should NEVER be
specified here...).

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Reverse proxies...

2008-06-27 Thread Henrik Nordstrom
On fre, 2008-06-27 at 08:06 -0700, John Doe wrote:

 I still have the GET internal://pc-03/squid-internal-periodic/store_digest 
 problem though..

What problem? It's cache digest exchanges between the Squids..

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Reverse proxies...

2008-06-26 Thread John Doe
Thanks again for your answers.
On my test server, I did setup 3 apaches on 127.0.0.1:[81|82|83] and 3 squids 
on 192.168.16.[11|12|13]
Sorry, the post is a bit long...

==
Delete browser cache and start 3 squids with brand new cache...
==

== /var/log/squid/cache.log ==

2008/06/26 11:10:54| Accepting accelerated HTTP connections at 192.168.17.11, 
port 80, FD 11.
2008/06/26 11:10:54| Accepting ICP messages at 0.0.0.0, port 3130, FD 12.
2008/06/26 11:10:54| WCCP Disabled.
2008/06/26 11:10:54| Configuring Parent 127.0.0.1/8081/0
2008/06/26 11:10:54| Configuring Sibling 192.168.17.12/80/3130
2008/06/26 11:10:54| Configuring Sibling 192.168.17.13/80/3130
2008/06/26 11:10:54| Ready to serve requests.


== /var/log/squid2/cache.log ==

2008/06/26 11:10:56| Accepting accelerated HTTP connections at 192.168.17.12, 
port 80, FD 11.
2008/06/26 11:10:56| Accepting ICP messages at 0.0.0.0, port 3130, FD 12.
2008/06/26 11:10:56| WCCP Disabled.
2008/06/26 11:10:56| Configuring Parent 127.0.0.1/8082/0
2008/06/26 11:10:56| Configuring Sibling 192.168.17.11/80/3130
2008/06/26 11:10:56| Configuring Sibling 192.168.17.13/80/3130
2008/06/26 11:10:56| Ready to serve requests.


== /var/log/squid3/cache.log ==

2008/06/26 11:10:57| Accepting accelerated HTTP connections at 192.168.17.13, 
port 80, FD 11.
2008/06/26 11:10:57| Accepting ICP messages at 0.0.0.0, port 3130, FD 12.
2008/06/26 11:10:57| WCCP Disabled.
2008/06/26 11:10:57| Configuring Parent 127.0.0.1/8083/0
2008/06/26 11:10:57| Configuring Sibling 192.168.17.11/80/3130
2008/06/26 11:10:57| Configuring Sibling 192.168.17.12/80/3130
2008/06/26 11:10:57| Ready to serve requests.


==
Access squid1...
==

== /var/log/squid1/access.log ==
1214491088.291   1001 192.168.17.11 TCP_MISS/200 2329 GET http://192.168.17.11/ 
- TIMEOUT_FIRST_UP_PARENT/127.0.0.1 text/html
1214491089.292   1000 192.168.17.11 TCP_MISS/200 4378 GET 
http://192.168.17.11/img/apache_header.gif - TIMEOUT_FIRST_UP_PARENT/127.0.0.1 
image/gif

== /var/log/httpd/apache-8081.access_log ==
127.0.0.1 - - [26/Jun/2008:16:38:08 +0200] GET / HTTP/1.0 200 2020
127.0.0.1 - - [26/Jun/2008:16:38:09 +0200] GET /img/apache_header.gif 
HTTP/1.0 200 4084

== /var/log/squid2/access.log ==
1214491087.293  0 192.168.17.12 TCP_MEM_HIT/200 373 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest

== /var/log/squid3/access.log ==
1214491086.740  0 192.168.17.12 UDP_MISS/000 42 ICP_QUERY 
http://192.168.17.11/ - NONE/- -
1214491086.740  0 192.168.17.13 UDP_MISS/000 42 ICP_QUERY 
http://192.168.17.11/ - NONE/- -
1214491087.740  0 192.168.17.12 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.11/img/apache_header.gif - NONE/- -
1214491087.740  0 192.168.17.13 UDP_MISS/000 63 ICP_QUERY 
http://192.168.17.11/img/apache_header.gif - NONE/- -
1214491147.291  0 192.168.17.13 TCP_MEM_HIT/200 373 GET 
internal://pc-03/squid-internal-periodic/store_digest - NONE/- 
application/cache-digest

==
Access squid1 again...
==

== /var/log/squid1/access.log ==
1214491262.555  0 192.168.17.11 TCP_REFRESH_HIT/304 189 GET 
http://192.168.17.11/ - FIRST_UP_PARENT/127.0.0.1 -
1214491262.936380 192.168.17.11 TCP_REFRESH_HIT/304 189 GET 
http://192.168.17.11/img/apache_header.gif - FIRST_UP_PARENT/127.0.0.1 -

== /var/log/httpd/apache-8081.access_log ==
127.0.0.1 - - [26/Jun/2008:16:41:02 +0200] GET / HTTP/1.0 304 -
127.0.0.1 - - [26/Jun/2008:16:41:02 +0200] GET /img/apache_header.gif 
HTTP/1.0 304 -

==
Access squid1 several times
==

== /var/log/squid1/access.log ==
1214491330.658  0 192.168.17.11 TCP_IMS_HIT/304 253 GET 
http://192.168.17.11/ - NONE/- text/html
1214491331.150492 192.168.17.11 TCP_IMS_HIT/304 238 GET 
http://192.168.17.11/img/apache_header.gif - NONE/- image/gif
1214491393.377749 192.168.17.11 TCP_IMS_HIT/304 244 GET 
http://192.168.17.11/ - NONE/- text/html
1214491393.556 17 192.168.17.11 TCP_IMS_HIT/304 237 GET 
http://192.168.17.11/img/apache_header.gif - NONE/- image/gif
1214491409.228683 192.168.17.11 TCP_IMS_HIT/304 253 GET 
http://192.168.17.11/ - NONE/- text/html
1214491409.545317 192.168.17.11 TCP_IMS_HIT/304 238 GET 
http://192.168.17.11/img/apache_header.gif - NONE/- image/gif

==
Access squid1 again with shift-reload
==

== 

Re: [squid-users] Reverse proxies...

2008-06-26 Thread Henrik Nordstrom
On tor, 2008-06-26 at 09:05 -0700, John Doe wrote:
 Thanks again for your answers.
 On my test server, I did setup 3 apaches on 127.0.0.1:[81|82|83] and 3 squids 
 on 192.168.16.[11|12|13]

On the same server, or different servers?

I ask because from your logs it looks like you have not assigned an
address for ICP, which means the ICP port listens on all addresses at
once...

 2008/06/26 11:10:54| Accepting ICP messages at 0.0.0.0, port 3130, FD 12.

See the udp_incoming_port directive for how to control the address ICP
listens on.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Reverse proxies...

2008-06-24 Thread John Doe
Hi again,

I must admit I am still a bit uncomfortable with access rules and other things 
in reverse proxy...

Here's my conf again:

+--- Squid1 --- Apache1
|   |
Internet ---+--- Squid2 --- Apache3
|   .
|   .
+--- SquidN --- ApacheN

Each squid is a reverse proxy for one apache listening on an external IP.
Squid (port 80) and apache (port 8080) are both on the same server for now.
All squids are proxy-only siblings on an internal IP.

Basicaly, I want the following: client ask squid.  If it is a miss, squid ask 
siblings in proxy-only (in order not to cache the same objects N times through 
the N squids).

Here are the confusing parts of the confs:

Common to all confs:
 acl from_localnet src 192.168.0.0/16# Internal IPs
 icp_access allow from_localnet
 icp_access deny all
#miss_access deny from_localnet  # Do I need this miss?
 acl to_examplecom dstdomain .example.com
 http_access allow to_examplecom

squid1.conf:
 http_port EXTIP1:80 accel defaultsite=www.example.com vhost
#https_port EXTIP1:3129 defaultsite=www.example.com cert=squid.pem 
key=squid.key
 cache_peer EXTIP1 parent 8080 0 no-query originserver max-conn=64 
name=apache1 front-end-https=auto
 cache_peer INTIP2 sibling 80 3130 proxy-only name=squid2
 ...
 cache_peer INTIPN sibling 80 3130 proxy-only name=squidN
 cache_peer_access apache1 allow to_examplecom
 cache_peer_access apache1 deny all

squid2.conf:
 http_port EXTIP2:80 accel defaultsite=www.example.com vhost
#https_port EXTIP2:3129 defaultsite=www.example.com cert=squid.pem 
key=squid.key
 cache_peer EXTIP2 parent 8080 0 no-query originserver max-conn=64 
name=apache2 front-end-https=auto
 cache_peer INTIP1 sibling 80 3130 proxy-only name=squid1
 ...
 cache_peer INTIPN sibling 80 3130 proxy-only name=squidN
 cache_peer_access apache2 allow to_examplecom
 cache_peer_access apache2 deny all



squidN.conf:
 http_port EXTIP3:80 accel defaultsite=www.example.com vhost
#https_port EXTIPN:3129 defaultsite=www.example.com cert=squid.pem 
key=squid.key
 cache_peer EXTIPN parent 8080 0 no-query originserver max-conn=64 
name=apacheN front-end-https=auto
 cache_peer INTIP1 sibling 80 3130 proxy-only name=squid1
 cache_peer INTIP2 sibling 80 3130 proxy-only name=squid2
 ...
 cache_peer INTIP[N-1] sibling 80 3130 proxy-only name=squid[N-1]
 cache_peer_access apacheN allow to_examplecom
 cache_peer_access apacheN deny all


Questions:
1. Anything wrong in the confs above?  Do I need some cache_peer_access for the 
siblings communication?
3. If squids sits right next to the apaches (on a gigabit switch), can I not 
use digests?
4. If I want to handle leeches (using wrong referer) with a redirector 
script, would the following work?

 acl examplecom_referer referer_regex example.com
 url_rewrite_program handleleech.php
 url_rewrite_children 5
 url_rewrite_concurrency 0
 url_rewrite_host_header off  # Do I need this for a 
reverse proxy?
 url_rewrite_access allow !examplecom_referer
 url_rewrite_access deny all

 The redirector, based on some internal rules, will either send back a 302 or 
the document URL...

5. If I want the apaches to tell squid to cache for 1 week and squids to tell 
clients to cache for 1 month, can I just use header_replace like these?

 header_replace Expires Sun Jan  1 00:00:00 CET 2012
 header_replace Cache-Control max-age=2678400

 Or is it refresh_pattern with override-expire and override-lastmod?

6. Is there a way to prevent shift-reload from making squid ask apache when 
the object is still in its cache...?  Something like that?

 refresh_pattern 0 0% 0 ignore-reload

Thank you,
JD


  



Re: [squid-users] Reverse proxies...

2008-06-24 Thread Henrik Nordstrom
tis 2008-06-24 klockan 11:30 -0700 skrev John Doe:
 #miss_access deny from_localnet  # Do I need this miss?

No, sibling relations automatically enforce this by using a
Cache-Control header sent by the requesting Squid.

 Questions:
 1. Anything wrong in the confs above?  Do I need some cache_peer_access for 
 the siblings communication?

Default for cache_peer_access is to consider the peer a candidate for
all requests.

 3. If squids sits right next to the apaches (on a gigabit switch), can I not 
 use digests?

cache digests is between the squids, and may be quite beneficial to you
as ICP can then be skipped on known cache hits.

 4. If I want to handle leeches (using wrong referer) with a redirector 
 script, would the following work?

Probably, but you may want to consider implementing this using an
external acl instead of a redirector..

 5. If I want the apaches to tell squid to cache for 1 week and squids to tell 
 clients to cache for 1 month, can I just useheader_replace like these?
 
  header_replace Expires Sun Jan  1 00:00:00 CET 2012
  header_replace Cache-Control max-age=2678400

Better to use a combination of max-age and s-maxage. What they are
intended for..

 6. Is there a way to prevent shift-reload from making squid ask apache when 
 the object is still in its cache...?  Something like that?
 
  refresh_pattern 0 0% 0 ignore-reload

Yes.

Regards
Henrik



[squid-users] Reverse proxies...

2008-06-18 Thread John Doe
Hi everybody,

I need to setup something like this:

+--- Squid1 --- Apache1
|   |
|   proxyonly
|   |
Internet ---+--- Squid2 --- Apache2
|   |
|   proxyonly
|   |
+--- Squid3 --- Apache3

All squids are siblings and each one only cache one server.
Squids (port 80) and apaches (port 8080) are on the same server for now.
Would these confs work?

squid1.conf:
 http_port IP1:80 accel defaultsite=www.example.com vhost
 cache_peer IP1 parent 8080 0 no-query originserver name=ap ache 
front-end-https=auto
 cache_peer IP1 sibling 80 3130 proxy-only name=squid1
 cache_peer IP2 sibling 80 3130 proxy-only name=squid2
 cache_peer IP3 sibling 80 3130 proxy-only name=squid3

squid2.conf:
 http_port IP2:80 accel defaultsite=www.example.com vhost
 cache_peer IP2 parent 8080 0 no-query originserver name=apache 
front-end-https=auto
 cache_peer IP1 sibling 80 3130 proxy-only name=squid1
 cache_peer IP2 sibling 80 3130 proxy-only name=squid2
 cache_peer IP3 sibling 80 3130 proxy-only name=squid3

squid3.conf:
 http_port IP3:80 accel defaultsite=www.example.com vhost
 cache_peer IP3 parent 8080 0 no-query originserver name=apache 
front-end-https=auto
 cache_peer IP1 sibling 80 3130 proxy-only name=squid1
 cache_peer IP2 sibling 80 3130 proxy-only name=squid2
 cache_peer IP3 sibling 80 3130 proxy-only name=squid3

Do I have to remove the self sibling line?
Do I miss any classic cache_peer options for a regular website (with https 
login page)?

Thank you,
JD


  



Re: [squid-users] Reverse proxies...

2008-06-18 Thread Henrik Nordstrom
On ons, 2008-06-18 at 06:55 -0700, John Doe wrote:

 Do I have to remove the self sibling line?

Preferably yes. It will probably still surive with it, but there will
quite likely be some forwarding loops detected and additionally unneeded
load on your servers.

 Do I miss any classic cache_peer options for a regular website (with https 
 login page)?

login=PASS perhaps? Needed if you are using proper HTTP authentication,
but not needed if you are using forms based authentication + cookies..

Regards
Henrik


signature.asc
Description: This is a digitally signed message part