Re: [squid-users] reverse proxy configuration still MISSes some pages which should be a HIT....

2011-11-01 Thread Einar Indridason
On Tue, Nov 01, 2011 at 11:33:28AM +1300, Amos Jeffries wrote:
 
 'delta' (time difference) between the two requests is 120 seconds (2
 minutes).
 
  + Server indicates 'must-revalidate'. Always contact backend server.
 
  + max-age is 10 seconds. Always fetch new content if current is
 older than 10 seconds.
 
  + origin servers object was modified 60 seconds after request (1).
 
 
 So this is correct. The cached object was stale, backend had an
 updated copy which got returned in full using status 200.
 
 If-None-Match and If-Modified-Since are both true conditions for
 these tests. Either one alone is enough to make a 200 happen.

Amos, thank you very, very much for that detailed answer :-)
Not quite what I was hoping for, me wanting to get more HITs even if one
or both of the above headers were to be included.
(The goal is to reduce MISS-es and increase HIT-s, obviously.)

But I think you pointed me to another direction... I need to cross-check
and even tripple check how the actual back-end is configured.

Amos, if you ever visit Iceland, I'll buy you a beer :-)

Thanks again,
Cheers,
--
EinarI


[squid-users] reverse proxy configuration still MISSes some pages which should be a HIT....

2011-10-31 Thread Einar Indridason
Hi.

I'm using squid 3.1.16, compiled from source with:
./configure --prefix=/usr/local/squid-3.1.16/ --enable-useragent-log
--enable-referer-log --disable-ident-lookups --with-large-files

Running on a 64bit Debian 6 box.


If I send a request:  Sent by doing:  cat file | nc proxy.example.com 80 
==
HEAD / HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Cookie: eplicaWebVisitor=-926431977; fptab=skjalftar; 
JSESSIONID=C44066454BC7A2C8A052BC0C69D44620
DNT: 1
Connection: keep-alive
If-Modified-Since: Sat, 30 Oct 2011 16:42:36 GMT
Cache-Control: max-age=0
If-None-Match: S-is-94659-1319906578198


==

I get back:

==
HTTP/1.0 200 OK
Date: Mon, 31 Oct 2011 18:22:45 GMT
Set-Cookie: JSESSIONID=05358DBC68CE264A981D34FB8322CADC; Path=/
Powered-By: Eplica WMS 2.0 (2.0-SNAPSHOT)
Last-Modified: Mon, 31 Oct 2011 18:22:21 GMT
Expires: Mon, 31 Oct 2011 18:22:55 GMT
Cache-Control: public, must-revalidate, max-age=10
ETag: S-is-94983-1320085375761
Content-Type: text/html;charset=UTF-8
Content-Language: is-IS
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 18425
X-Cache: MISS from proxy.example.com
Via: 1.0 proxy.example.com (squid/3.1.16)
Connection: keep-alive

==

If I send the same request, but leave out the If-None-Match, I get:
HTTP/1.0 200 OK
Date: Mon, 31 Oct 2011 18:24:10 GMT
Powered-By: Eplica WMS 2.0 (2.0-SNAPSHOT)
Last-Modified: Mon, 31 Oct 2011 18:23:22 GMT
Expires: Mon, 31 Oct 2011 18:24:20 GMT
Cache-Control: public, must-revalidate, max-age=10
ETag: S-is-94983-1320085460159
Content-Type: text/html;charset=UTF-8
Content-Language: is-IS
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 18425
Age: 3
X-Cache: HIT from proxy.example.com
Via: 1.0 proxy.example.com (squid/3.1.16)
Connection: keep-alive

==

Hmm... I *think* the needed lines from squid.conf would look like, 
but please correct me if this is not enough to determine the cause:

http_port 1.2.3.4:80 accel defaultsite=www.example.com vhost ignore-cc

cache_peer 1.2.3.99 parent 80 0 no-query originserver name=myAccel


Now, is there a simple(ish) way of throwing away / ignoring that 
If-None-Match header, or configure squid in other ways, to go to the
cache, and create a HIT?

Thanks,
--
EinarI


Re: [squid-users] reverse proxy configuration still MISSes some pages which should be a HIT....

2011-10-31 Thread Amos Jeffries

On Mon, 31 Oct 2011 18:56:00 +, Einar Indridason wrote:

Hi.

I'm using squid 3.1.16, compiled from source with:
./configure --prefix=/usr/local/squid-3.1.16/ --enable-useragent-log
--enable-referer-log --disable-ident-lookups --with-large-files

Running on a 64bit Debian 6 box.


If I send a request:  Sent by doing:  cat file | nc proxy.example.com 
80


==
HEAD / HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101
Firefox/7.0.1
Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Cookie: eplicaWebVisitor=-926431977; fptab=skjalftar;
JSESSIONID=C44066454BC7A2C8A052BC0C69D44620
DNT: 1
Connection: keep-alive
If-Modified-Since: Sat, 30 Oct 2011 16:42:36 GMT
Cache-Control: max-age=0
If-None-Match: S-is-94659-1319906578198



==

I get back:



Calling this (1) ...



==
HTTP/1.0 200 OK
Date: Mon, 31 Oct 2011 18:22:45 GMT
Set-Cookie: JSESSIONID=05358DBC68CE264A981D34FB8322CADC; Path=/
Powered-By: Eplica WMS 2.0 (2.0-SNAPSHOT)
Last-Modified: Mon, 31 Oct 2011 18:22:21 GMT
Expires: Mon, 31 Oct 2011 18:22:55 GMT
Cache-Control: public, must-revalidate, max-age=10
ETag: S-is-94983-1320085375761
Content-Type: text/html;charset=UTF-8
Content-Language: is-IS
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 18425
X-Cache: MISS from proxy.example.com
Via: 1.0 proxy.example.com (squid/3.1.16)
Connection: keep-alive


==



Calling this (2) ...


If I send the same request, but leave out the If-None-Match, I get:
HTTP/1.0 200 OK
Date: Mon, 31 Oct 2011 18:24:10 GMT
Powered-By: Eplica WMS 2.0 (2.0-SNAPSHOT)
Last-Modified: Mon, 31 Oct 2011 18:23:22 GMT
Expires: Mon, 31 Oct 2011 18:24:20 GMT
Cache-Control: public, must-revalidate, max-age=10
ETag: S-is-94983-1320085460159
Content-Type: text/html;charset=UTF-8
Content-Language: is-IS
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 18425
Age: 3
X-Cache: HIT from proxy.example.com
Via: 1.0 proxy.example.com (squid/3.1.16)
Connection: keep-alive


==


'delta' (time difference) between the two requests is 120 seconds (2 
minutes).


 + Server indicates 'must-revalidate'. Always contact backend server.

 + max-age is 10 seconds. Always fetch new content if current is older 
than 10 seconds.


 + origin servers object was modified 60 seconds after request (1).


So this is correct. The cached object was stale, backend had an updated 
copy which got returned in full using status 200.


If-None-Match and If-Modified-Since are both true conditions for 
these tests. Either one alone is enough to make a 200 happen.




Hmm... I *think* the needed lines from squid.conf would look like,
but please correct me if this is not enough to determine the cause:

http_port 1.2.3.4:80 accel defaultsite=www.example.com vhost 
ignore-cc



The ignore-cc directive is there to ignore the client when it tries 
to override the server Cache-Crontrol. In the above your server is 
saying max-age=10 (give clients things up to 10 seconds old). But the 
client is attempting to override and says max-age=0 (nothing 1 second or 
older may be sent to me).
 Since this is a reverse-proxy and your Squid is one of the servers for 
this domain it is able to safely ignore that client max-age, and say 
here is object X, its valid right now (despite being 1-10 seconds old).



In the case you detailed above, it will make Squid ignore the max-age=0 
(force a reload) from the client. BUT, the server is still indicating 10 
second max-age and must-revalidate. So the revalidate conditions will 
still happen and possibly produce a 200.




cache_peer 1.2.3.99 parent 80 0 no-query originserver name=myAccel


Now, is there a simple(ish) way of throwing away / ignoring that
If-None-Match header, or configure squid in other ways, to go to 
the

cache, and create a HIT?


That is up to your server to respond with 304 instead of 200. When 
testing conditional requests a 304 message is equivalent to a HIT in 
older traffic.


As or ignoring the If-* headers. This is a very bad idea(tm)...

Consider a login script which presents exactly two variants. One says 
Successful login. The other says Successful logout.


The If-* values and ETag encodes which of these the client is 
attempting to display so Squid and the server can override with 200 and 
essentially say 'no display this instead'.


In the login example, the server would check its login/out state for 
the client and allow the display or replace it. Overriding these details 
and making Squid HIT would