Persistent Storage
I was wondering if there's a time line for when persistent storage will be available. Right now I'm considering running squid with a disk based cache which will query the origin servers and having varnish in front of that since squid is unable to handle our load it seems. (And Varnish makes a whole lot more sense to me) Also on some test instances I get error messages such as these: Dec 2 08:37:00 ip-2 varnishd[76185]: Child (138) not responding to ping, killing it. Dec 2 08:37:00 ip-2 varnishd[76185]: child (8721) Started Dec 2 08:37:01 ip-2 varnishd[76185]: Child (8721) said Closed fds: 4 5 6 7 11 12 14 15 Dec 2 08:37:01 ip-2 varnishd[76185]: Child (8721) said Child starts Dec 2 08:37:01 ip-2 varnishd[76185]: Child (8721) said managed to mmap 10737418240 bytes of 10737418240 Dec 2 08:37:01 ip-2 varnishd[76185]: Child (8721) said Ready Dec 2 14:25:15 ip-2 varnishd[76185]: Child (8721) not responding to ping, killing it. Dec 2 09:25:17 ip-2 kernel: pid 8721 (varnishd), uid 65534: exited on signal 3 Dec 2 14:25:17 ip-2 varnishd[76185]: Child (8721) not responding to ping, killing it. Dec 2 14:25:17 ip-2 varnishd[76185]: child (26870) Started Dec 2 14:25:18 ip-2 varnishd[76185]: Child (26870) said Closed fds: 4 5 6 7 11 12 14 15 Dec 2 14:25:18 ip-2 varnishd[76185]: Child (26870) said Child starts Dec 2 14:25:18 ip-2 varnishd[76185]: Child (26870) said managed to mmap 10737418240 bytes of 10737418240 Dec 2 14:25:18 ip-2 varnishd[76185]: Child (26870) said Ready Is that indicative of a problem or should I just up the timeout value for pings? This is using either revision 3459 or 2.0.2 out of ports on FreeBSD 6.3. I'm going to be trying another test on FreeBSD 7.0 with either 2.0.2 or the latest checkout. Thanks ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Question about threads
Im running 32bit. But I think that I have succeded creating more then 238 threads before on another system with the same setup. Anyway, 64bit might be the thing to have... If I want to have Debian, is it AMD64 version that I should go for? (OT) / E Poul-Henning Kamp skrev: In message [EMAIL PROTECTED], Erik Torlen writes: I still have the same problem :( The threads are created up to 238 where they are stopped, even if I set threads_max = 1000 and threads_pools = 2 (or 3). I also tested the tips and decreased the stack sixe to 512 and increased overflow_max to 1% . Any idea what could be wrong? Are you running on a 32bit or 64bit system ? On a 32bit system you may simply be running out of address-space... ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Strange Opera error in POST
Ok, I have gathered some more info about the problem.
What I have is a form login that has 3 input fields, username, password
and security code (from captcha).
When opera is making the POST it receives a 200 OK and NOT a 302 Moved
Temp. as expected.
This is the request that Opera is making:
13 RxRequestc POST
13 RxURLc /takelogin.php
13 RxProtocol c HTTP/1.1
13 RxHeader c User-Agent: Opera/9.27 (Windows NT 5.1; U; sv)
13 RxHeader c Host: mydomain.com:8080
13 RxHeader c Accept: text/html, application/xml;q=0.9,
application/xhtml+xml, image/png, image/jpeg, image/gif,
image/x-xbitmap, */*;q=0.1
13 RxHeader c Accept-Language: sv-SE,sv;q=0.9,en;q=0.8
13 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
13 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
13 RxHeader c Referer: http://mydomain.com:8080/login.php
13 RxHeader c Cookie: PHPSESSID=f3a60f178adede632c761dca745054cf
13 RxHeader c Cookie2: $Version=1
13 RxHeader c Connection: Keep-Alive, TE
13 RxHeader c TE: deflate, gzip, chunked, identity, trailers
13 RxHeader c Content-Length: 63
13 RxHeader c Content-Type: application/x-www-form-urlencoded
My vcl has this code when POST are received:
if(req.request != GET req.request != HEAD) {
set req.http.Connection = close;
pass;
}
This code usually works with FF and IE but NOT with Opera.
If I remove set req.http.Connection = close;
the login process works with no problem.
I have had problems with POSTs before, thats why I've been using
Connection = close on POSTs.
Here is the whole error log:
0 CLI - Rd ping
0 CLI - Wr 0 200 PONG 1213029670
13 SessionClose - timeout
13 StatSess - 82.182.xx.xx 1306 0 1 2 0 0 2 744 6750
0 CLI - Rd ping
0 CLI - Wr 0 200 PONG 1213029673
13 SessionOpen c 82.182.xx.xx 1307 0.0.0.0:8080
13 ReqStart c 82.182.xx.xx 1307 803913450
13 RxRequestc POST
13 RxURLc /takelogin.php
13 RxProtocol c HTTP/1.1
13 RxHeader c User-Agent: Opera/9.27 (Windows NT 5.1; U; sv)
13 RxHeader c Host: mydomain.com:8080
13 RxHeader c Accept: text/html, application/xml;q=0.9,
application/xhtml+xml, image/png, image/jpeg, image/gif,
image/x-xbitmap, */*;q=0.1
13 RxHeader c Accept-Language: sv-SE,sv;q=0.9,en;q=0.8
13 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
13 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
13 RxHeader c Referer: http://mydomain.com:8080/login.php
13 RxHeader c Cookie: PHPSESSID=f3a60f178adede632c761dca745054cf
13 RxHeader c Cookie2: $Version=1
13 RxHeader c Connection: Keep-Alive, TE
13 RxHeader c TE: deflate, gzip, chunked, identity, trailers
13 RxHeader c Content-Length: 63
13 RxHeader c Content-Type: application/x-www-form-urlencoded
13 VCL_call c recv
13 VCL_return c pass
13 VCL_call c pass
13 VCL_return c pass
14 TxRequest- POST
14 TxURL- /takelogin.php
14 TxProtocol - HTTP/1.1
14 TxHeader - User-Agent: Opera/9.27 (Windows NT 5.1; U; sv)
14 TxHeader - Host: mydomain.com:8080
14 TxHeader - Accept: text/html, application/xml;q=0.9,
application/xhtml+xml, image/png, image/jpeg, image/gif,
image/x-xbitmap, */*;q=0.1
14 TxHeader - Accept-Language: sv-SE,sv;q=0.9,en;q=0.8
14 TxHeader - Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
14 TxHeader - Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
14 TxHeader - Referer: http://mydomain.com:8080/login.php
14 TxHeader - Cookie: PHPSESSID=f3a60f178adede632c761dca745054cf
14 TxHeader - Cookie2: $Version=1
14 TxHeader - Content-Type: application/x-www-form-urlencoded
14 TxHeader - X-Varnish: 803913450
14 TxHeader - X-Forwarded-For: 82.182.xx.xx
14 RxProtocol - HTTP/1.1
14 RxStatus - 200
14 RxResponse - OK
14 RxHeader - Date: Mon, 09 Jun 2008 16:41:14 GMT
14 RxHeader - Server: Apache
14 RxHeader - X-Powered-By: PHP/5.2.0-8+etch10
14 RxHeader - Content-Length: 23
14 RxHeader - Content-Type: text/html; charset=UTF-8
13 ObjProtocol c HTTP/1.1
13 ObjStatusc 200
13 ObjResponse c OK
13 ObjHeaderc Date: Mon, 09 Jun 2008 16:41:14 GMT
13 ObjHeaderc Server: Apache
13 ObjHeaderc X-Powered-By: PHP/5.2.0-8+etch10
13 ObjHeaderc Content-Type: text/html; charset=UTF-8
14 BackendReuse - mydomain
13 TTL c 803913450 RFC 120 1213029674 1213029674 0 0 0
13 VCL_call c fetch
13 VCL_return c pass
13 Length c 23
13 VCL_call c deliver
13 VCL_return c deliver
13 TxProtocol c HTTP/1.1
13 TxStatus c 200
13 TxResponse c OK
13 TxHeader c Server: Apache
Re: Strange Opera error in POST
Yes, Im aware of that. But I did have some problem in an earlier version
of trunk (and especially 1.1.2), but with the latest it seems better.
I have seen alot of updates on trunk the latest days, what's the status
of it, is it stable for production use?
/ Duja
Poul-Henning Kamp skrev:
In message [EMAIL PROTECTED], Erik Torlen writes:
My vcl has this code when POST are received:
if(req.request != GET req.request != HEAD) {
set req.http.Connection = close;
pass;
}
This code usually works with FF and IE but NOT with Opera.
If I remove set req.http.Connection = close;
the login process works with no problem.
I have had problems with POSTs before, thats why I've been using
Connection = close on POSTs.
I'm not sure I can say much here...
The close trick is mostly, if not only, relevant for pipe mode,
where it prevents more than the first request from the client from
being piped. It doesn't really do anything positive for pass.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Strange Opera error in POST
What I should mention is that the problem can be solved by another method.
If I remove the Cookie2 AND TE header it all works out fine:
remove req.http.Cookie2;
remove req.http.TE;
if(req.request != GET req.request != HEAD) {
set req.http.Connection = close;
pass;
}
This works!
But I must remove BOTH Cookie2 and TE header, else it wont work.
/ Duja
Poul-Henning Kamp skrev:
In message [EMAIL PROTECTED], Erik Torlen writes:
My vcl has this code when POST are received:
if(req.request != GET req.request != HEAD) {
set req.http.Connection = close;
pass;
}
This code usually works with FF and IE but NOT with Opera.
If I remove set req.http.Connection = close;
the login process works with no problem.
I have had problems with POSTs before, thats why I've been using
Connection = close on POSTs.
I'm not sure I can say much here...
The close trick is mostly, if not only, relevant for pipe mode,
where it prevents more than the first request from the client from
being piped. It doesn't really do anything positive for pass.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish/Swap usage and network timeouts
Hello,
I have 2 machines running varnish which handles around 1000 requests per
second each and sometimes the machines will just stop responding to any
network communication. A coworker used wireshark to check the connection
and saw a whole lot of tcp retransmissions. Normally I would suspect
some sort of network buffer issue but there is nothing in the logs at
all. This is on FreeBSD 6.3 with a default 64 bit kernel with bge
network cards. They are both running varnish-trunk revision 2635.
Attached is the VCL file that is currently running. Varnish is started
with the following options:
file,/vol/data1/varnish.cache,90% -t 259200 -h classic,59 -p
lru_interval=3600
The only suspicious things I've found are: Varnish says it is using 372G
of virtual memory when I told it to use a 58gb file and there is only
2gb of physical memory in the machine and there is pretty constant
swapping going on, mostly under 500K but it seems to be pretty constant
no matter the number of connections and such. The amount of swap space
in the machine is 4gb and things are using 1.1gb.
Threads don't seem to be a problem, as the highest I've seen it is at
500 and that was after an extended period of time around 250 or so. So I
don't think I'm running into thread issues
So I guess I'm asking if anyone has any other suggestions other then
reinstalling to FreeBSD 7.0 which seems to be the best solution I can
see. If that is the best solution, what is the optimal FreeBSD
configuration for Varnish? Lots of swap space? No swap space? Currently
the mmap-ed file is on an SSD so one alternative I thought of was to
make the entire SSD part of swap and use the malloc option and get rid
of any regular disk basked swap.
Thanks for any help,
Erik
backend default {
.host = xzy.com;
.port = 80;
}
sub vcl_recv {
if (req.url ~ \.js|css$) {
if (req.http.Accept-Encoding ~ gzip) {
set req.http.Accept-Encoding = gzip;
} elsif (req.http.Accept-Encoding ~ deflate) {
set req.http.Accept-Encoding = deflate;
} else {
unset req.http.accept-encoding;
}
} else {
unset req.http.accept-encoding;
}
unset req.http.Cookie;
set req.grace = 10m;
}
sub vcl_fetch {
set obj.http.P3P = CP='NOI CURa ADMa DEVa TAIa OUR BUS IND UNI COM
NAV INT', policyref='http://www.okcupid.com/w3c/p3p.xml';
set obj.grace = 10m;
if (obj.ttl 259200s) {
set obj.ttl = 259200s;
}
}
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Directors user sessions
But the loadbalancing is already implenteed. I dont see why it shouldn't be used as loadbalancer if the functionality exists? I don't want to use one place for all sessions, like a file share or something in that direction. Im thinking about creating a Header that is called X-Backend: (a|b|c|d). This could be used to check what backend the user should use. Its really not a nice way to do it but its a way of doing it. Is there any idea that I could create a ticket for a feature like this? / Erik Cherife Li skrev: On 2008-3-28 19:05, [EMAIL PROTECTED] wrote: Hi, I got a question regarding the Directors in varnish vcl. If user A is logging in to http://mywebsite.com and the website is using varnish (with directors) in front of 4 backend servers. The 4 backend servers is identical. User A is logging in and hits server 1. He then goes to his profile and hits server 2. The server 2 doesn't know that user A is logged and redirect him to some Not logged in-page. Is there any way for varnish to lookup which server that user A should be directed to? Some kind of Sticky Session function? IMHO, Varnish is for caching, rather than for redirecting. Maybe you could consider HAProxy, or pound, or IPVS, or similar implementation. Besides, I know that sessions can be shared. / Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnishlogging
Well C then :P Dag-Erling Smørgrav skrev: Erik [EMAIL PROTECTED] writes: Ah ok, I see :( Unfortunately i'm not a c++ guru so I can just cross my fingers and hope that some handy guy writes a patch. Actually, there isn't a single line of C++ code in Varnish. You must be thinking of Squid... DES ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnishlogging
Is it possible to log many different tags and match just one of them against a regex? For example. I want to include TxUrl, TxResponse, ReqStart, ReqEnd and match TxResponse against regex Service Unavailable. / Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: A few users see a blank page
Hi,
I have seen the same behaviour on my Apache2 server with php5 installed.
Many users got blank pages
when navigating around the website. I couldn't figure out what the
problem was so I simply removed varnish
and the problem disappeared.
Another problem I had was that php's X-FORWARDED-FOR was empty at random
times. I looked in the
apache2 log and saw some empty fields their too. Couldn't find a
solution for it :(
/ Erik
Marco Molinari skrev:
I can now reproduce it and I found a lot of info.
I wrote a simple script that keeps on reloading itself, adding some
chars each time; I disabled compression both in Apache and in PHP and
I connect my browser through a 2.6 Squid proxy; and now I can't see
PHP pages larger than 8000 bytes. I also found an interesting
connection with cookies.
Here are the log lines for Varnish requesting two pages to Apache:
Apache:
127.0.0.1, 10.0.2.2 - - [12/Feb/2008:14:24:26 +0100] GET
/test_varnish2.php?n=10t=1202822659 HTTP/1.0 200 8000 -
{MY-USER-AGENT}
127.0.0.1, 10.0.2.2 - - [12/Feb/2008:14:24:29 +0100] GET
/test_varnish2.php?n=11t=1202822659 HTTP/1.0 200 8001 -
{MY-USER-AGENT}
Varnish:
10.0.2.2 - - [12/Feb/2008:14:24:26 +0100] GET
http://dev.{MY-SITE}:2280/test_varnish2.php?n=10t=1202822659
HTTP/1.0 200 8000 - {MY-USER-AGENT}
10.0.2.2 - - [12/Feb/2008:14:24:29 +0100] GET
http://dev.{MY-SITE}:2280/test_varnish2.php?n=11t=1202822659
HTTP/1.0 200 0 - {MY-USER-AGENT}
As you can see Varnish logs a 0 size for the 8001 bytes request.
Here is the extended log (as per varnishlog) for those requests (sorry
for the length):
15 ReqStart c 10.0.2.2 43859 67671826
15 RxRequestc GET
15 RxURLc /test_varnish2.php?n=10t=1202822659
15 RxProtocol c HTTP/1.0
15 RxHeader c Host: dev.{MY-SITE}:2280
15 RxHeader c User-Agent: {MY-USER-AGENT}
15 RxHeader c Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
15 RxHeader c Accept-Language: en-us,en;q=0.5
15 RxHeader c Accept-Encoding: gzip,deflate
15 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
15 RxHeader c Cookie:
__utma=.xx.xx.xx.xx.23;
__utmz=.xx.21.5.utmcsr=mail.google.com|utmccn=(referral)|utmcmd=referral|utmcct=%2Fmail%2F;
__utmz=.xx.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(non
15 RxHeader c Via: 1.1 virtual:3129 (squid/2.6.STABLE14)
15 RxHeader c X-Forwarded-For: 127.0.0.1
15 RxHeader c Cache-Control: max-age=259200
15 RxHeader c Connection: keep-alive
15 VCL_call c recv
15 VCL_return c pass
15 VCL_call c pass
15 VCL_return c pass
18 BackendOpen b default 127.0.0.1 32988 127.0.0.1 8080
18 BackendXID b 67671826
15 Backend c 18 default
18 TxRequestb GET
18 TxURLb /test_varnish2.php?n=10t=1202822659
18 TxProtocol b HTTP/1.0
18 TxHeader b Host: dev.{MY-SITE}:2280
18 TxHeader b User-Agent: {MY-USER-AGENT}
18 TxHeader b Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
18 TxHeader b Accept-Language: en-us,en;q=0.5
18 TxHeader b Accept-Encoding: gzip,deflate
18 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
18 TxHeader b Cookie:
__utma=.xx.xx.xx.xx.23;
__utmz=.xx.21.5.utmcsr=mail.google.com|utmccn=(referral)|utmcmd=referral|utmcct=%2Fmail%2F;
__utmz=xx.xx.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(non
18 TxHeader b Via: 1.1 virtual:3129 (squid/2.6.STABLE14)
18 TxHeader b X-Forwarded-For: 127.0.0.1
18 TxHeader b X-Varnish: 67671826
18 TxHeader b X-Forwarded-for: 10.0.2.2
18 RxProtocol b HTTP/1.1
18 RxStatus b 200
18 RxResponse b OK
18 RxHeader b Date: Tue, 12 Feb 2008 13:24:26 GMT
18 RxHeader b Server: Apache/2.0.52 (CentOS)
18 RxHeader b X-Powered-By: PHP/4.3.9
18 RxHeader b Content-Length: 8000
18 RxHeader b Connection: close
18 RxHeader b Content-Type: text/html; charset=UTF-8
15 ObjProtocol c HTTP/1.1
15 ObjStatusc 200
15 ObjResponse c OK
15 ObjHeaderc Date: Tue, 12 Feb 2008 13:24:26 GMT
15 ObjHeaderc Server: Apache/2.0.52 (CentOS)
15 ObjHeaderc X-Powered-By: PHP/4.3.9
15 ObjHeaderc Content-Type: text/html; charset=UTF-8
18 BackendClose b default
15 TTL c 67671826 RFC 120 1202822666 1202822666 0 0 0
15 VCL_call c fetch
15 VCL_return c insert
15 Length c 8000
15 VCL_call c deliver
15 VCL_return c deliver
15 TxProtocol c HTTP/1.1
15 TxStatus c 200
15 TxResponse c OK
15 TxHeader c Server: Apache/2.0.52 (CentOS)
15 TxHeader c X-Powered
IP X-Forward-for
Hi, Im experiencing a strange problem when using varnish in front of apache2. I m running a php website where I log the client IP to the usertable in the database. I first check the Client IP to see if it is a valid IP, if not, I check the x-forward-for for a valid ip (I check for multiple IPs in x-forward-for). If neither client ip or x-forward-for contained a valid ip, it returns the client ip (in most cases 127.0.0.1). The strange thing is that I get alot of IPs that comes from 127.0.0.1 which means that x-forward-for is NOT set. This is the log from varnish: 81.224.31.xxx - - [31/Jan/2008:20:52:09 +0100] GET 127.0.0.1 - - [31/Jan/2008:20:52:09 +0100] (null) (null) (null) (null) (null) - - backendserver - - [31/Jan/2008:19:52:09 +0100] GET 81.227.81.xxx - - [31/Jan/2008:20:52:09 +0100] GET 127.0.0.1 - - [31/Jan/2008:19:52:11 +0100] GET 217.197.62.xxx - - [31/Jan/2008:20:52:11 +0100] GET 127.0.0.1 - - [31/Jan/2008:19:52:11 +0100] GET 85.229.6.xxx - - [31/Jan/2008:20:52:11 +0100] GET 127.0.0.1 - - [31/Jan/2008:19:52:11 +0100] GET 85.229.6.xxx - - [31/Jan/2008:20:52:11 +0100] GET 127.0.0.1 - - [31/Jan/2008:19:52:12 +0100] GET 83.233.193.xxx - - [31/Jan/2008:20:52:12 +0100] GET Why does it have so many connections from 127.0.0.1 and why is the date an hour late? It also got a connection from backendserver which is the name of my backendserver in the vcl config. The logfile from apache2 shows me this whit the same connections: 81.224.31.xxx - - [31/Jan/2008:20:52:08 +0100] GET 81.227.81.xxx - - [31/Jan/2008:20:52:09 +0100] GET 217.197.62.xxx - - [31/Jan/2008:20:52:11 +0100] GET 85.229.6.xxx - - [31/Jan/2008:20:52:11 +0100] GET 85.229.6.xxx - - [31/Jan/2008:20:52:11 +0100] GET 83.233.193.xxx - - [31/Jan/2008:20:52:12 +0100] GET No 127.0.0.1 connections in this logfile. Is the 127.0.0.1 connections from apache2 when varnish fetches the object from back their? Im not sure if these logentries say enough about my problem. I will try to find some more entries that could show some more about the issue. I appreciate all help I can get on this one. / Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
RE: Configure varnish to pipe a subdomain
Sending another one, the previous got messed up (using a webmail client)
Neither am I a regex expert, but I dont think that the regex is the problem.
You are using http.url instead of using http.host.
And if im not totally wrong, you dont need the ~ to check the host, you can use
==.
Change this:
if (req.http.url ~ admin.example.com$) {
pipe;
}
To this:
if (req.http.host == admin.example.com) {
pipe;
}
And this:
if (req.http.Cookie req.http.url ~ admin.example.com$) {
pipe;
}
To this:
if (req.http.Cookie req.http.host == admin.example.com) {
pipe;
}
Althought I do not know if you should pipe the cookies. I think its pass that
would be used when cookies is set but Im not an expert, maybe im wrong.
Cheers
Erik
Original Message ---
Hello list,
I installed varnish on a staging system today and I am totally impressed
by the performance.
Unfortunately I'm no regex expert, so I will need your help with this
config issue.
I want to use varnish for a domain - lets call it example.com
The domain has got a subdomain admin.example.com - any GET / POST to
this subdomain and any pages underneath should not be cached but should
directly passed to the backend.
This is my config - which does not work for as the requests to
admin.example.com as they are also being cached :-(
Thanks so much for your help!
Philipp
(comments are in german - please dont bother!):
/* Definition Default Backend */
backend default {
set backend.host = 127.0.0.1;
set backend.port = 1234;
}
sub vcl_recv {
/* Falls etwas an den Server geschickt wird, geben wir das durch */
if (req.request == POST) {
pipe;
}
/* Anfragen an die admin Seite werden direkt durchgereicht /*
if (req.http.url ~ admin.example.com$) {
pipe;
}
if (req.http.Cookie req.http.url ~ admin.example.com$) {
pipe;
}
if (req.http.Expect) {
pipe;
}
/* Wir cachen Bilder, Dokumente, Audio und FLV Filme*/
if (req.url ~
\.(jpg|jpeg|gif|png|tiff|tif|svg|swf|flv|ico|doc|ppt|xls|pdf|mp3|mp4|mp4a|ogg)$)
{
lookup;
}
/* Wir cachen auch CSS und javascript */
if (req.url ~ \.(css|js)$) {
lookup;
}
/* Und wir cachen HTML Code */
if (req.url ~ \.(html|htm)$) {
lookup;
}
/* Filme cachen wir auch - testweise */
if (req.url ~ \.(mov|avi|wmv)$) {
lookup;
}
/* ZIP Dateien cachen wir experimentell auch */
if (req.url ~ \.(zip)$) {
lookup;
}
/* Cookie Requests an die Hauptseite cachen wir weg */
if (req.http.Cookie) {
lookup;
}
/* Kein Caching fuer Content hinter http authentification */
if (req.http.Authenticate || req.http.Authorization) {
pipe;
}
/* Oh und wir cachen alles andere */
lookup;
}
sub vcl_fetch {
/* Wir setzen die TTL fuer jedes Objekt das eine TTL von unter
60sec hat auf 60sec */
if (obj.ttl 60s) {
}
if (!obj.valid) {
error;
}
if (!obj.cacheable) {
pass;
}
if (obj.http.Set-Cookie) {
pass;
}
/* Wir cachen HTML maximal 30 Minuten */
if (req.url ~ \.(htm|html)$) {
set obj.ttl = 1800s;
}
/* Wir cachen jedes Objekt maximal fuer drei Stunden */
set obj.ttl = 10800s;
}
sub vcl_hit {
if (req.request == PURGE) {
set obj.ttl = 0s;
error 200 Purged.;
}
}
sub vcl_miss {
if (req.request == PURGE) {
error 404 Not in cache.;
}
}
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
RE: Re: Find timestamp in log
Im using 1.1.2 from SF. I noticed in my log that the POST
request isn't piped, it is passed. Probably thats why the
request takes such a long time to finish.
I thought that all POST were piped auto like this?
if (req.request != GET req.request != HEAD) {
pipe;
}
Could you explain the values in the ReqEnd:
12 ReqEnd c 100929047 1200397302.365636110 1200397902.687728643
0.016155005 600.287691593 0.034400940
Thanks
Erik
--- Poul-Henning Kamp wrote
Which version of varnish are you using ?
We can't really do much logging for a piped request, since all we
do is pass bytes back and forth.
For normal requests timestamps are in the ReqEnd record.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
RE: Re: What's the meaning of req.http.Expect?
The expect header:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.20
Original Message ---
In message [EMAIL PROTECTED], MontyRee writes:
I can see req.http.Expect like below from the lots of vcl example.
if (req.http.Expect) { pipe; }
Then, what's the meaning of the req.http.Expect?
The test above is If the request has a Expect: HTTP header
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnishtop hangs
Its not near an overload when it hangs. I do some requests against it and *poff*, varnishtop hangs and I can't even close it with ctrl+c. If you have any requests on some more info am more then glad to help. Cheers Erik Poul-Henning Kamp skrev: In message [EMAIL PROTECTED], Erik writes: I'm making alot of requests and the varnishtop shows the correct= stats but after a while with some certain tags (e.g TxURL) it= just hangs. I cannot make e Ctrl+C or anything, simply dead. Hmm, I wonder if your shmlog segment is overrun too fast. In that case all the shmlog tailing programs can get confused. How much traffic does your server take ? ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
varnishtop hangs
I do the following in varnishtop: varnishtop -i TxURL
after some request it hangs and from strace I get:
rt_sigaction(SIGTSTP, {SIG_IGN}, {0xb7f86fe0, [], SA_RESTART}, 8) = 0
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
rt_sigaction(SIGTSTP, {0xb7f86fe0, [], SA_RESTART}, NULL, 8) = 0
gettimeofday({1199697921, 208862}, NULL) = 0
select(1, [0], NULL, NULL, {1, 0}) = 0 (Timeout)
gettimeofday({1199697922, 208654}, NULL) = 0
time(NULL) = 1199697922
rt_sigaction(SIGTSTP, {SIG_IGN}, {0xb7f86fe0, [], SA_RESTART}, 8) = 0
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
rt_sigaction(SIGTSTP, {0xb7f86fe0, [], SA_RESTART}, NULL, 8) = 0
gettimeofday({1199697922, 208821}, NULL) = 0
select(1, [0], NULL, NULL, {1, 0}) = 0 (Timeout)
gettimeofday({1199697923, 208618}, NULL) = 0
time(NULL) = 1199697923
rt_sigaction(SIGTSTP, {SIG_IGN}, {0xb7f86fe0, [], SA_RESTART}, 8) = 0
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
rt_sigaction(SIGTSTP, {0xb7f86fe0, [], SA_RESTART}, NULL, 8) = 0
gettimeofday({1199697923, 208781}, NULL) = 0
select(1, [0], NULL, NULL, {1, 0}) = 0 (Timeout)
gettimeofday({1199697924, 208580}, NULL) = 0
time(NULL) = 1199697924
rt_sigaction(SIGTSTP, {SIG_IGN}, {0xb7f86fe0, [], SA_RESTART}, 8) = 0
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
rt_sigaction(SIGTSTP, {0xb7f86fe0, [], SA_RESTART}, NULL, 8) = 0
gettimeofday({1199697924, 208748}, NULL) = 0
select(1, [0], NULL, NULL, {1, 0}) = 0 (Timeout)
gettimeofday({1199697925, 208553}, NULL) = 0
time(NULL) = 1199697925
rt_sigaction(SIGTSTP, {SIG_IGN}, {0xb7f86fe0, [], SA_RESTART}, 8) = 0
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
write(1, \01013\33[3;6H1.00 TxURL /\33[4;6H1..., 584) = 584
rt_sigaction(SIGTSTP, {0xb7f86fe0, [], SA_RESTART}, NULL, 8) = 0
gettimeofday({1199697925, 209013}, NULL) = 0
select(1, [0], NULL, NULL, {1, 0}) = 0 (Timeout)
gettimeofday({1199697926, 224518}, NULL) = 0
futex(0x804af08, FUTEX_WAIT, 2, NULL
# varnishd -V
varnishd (varnish-1.1.2)
Copyright (c) 2006-2007 Linpro AS / Verdens Gang AS
Running ubuntu
# uname -a
Linux web1 2.6.20-16-generic #2 SMP Sun Sep 23 19:50:39 UTC 2007 i686 GNU/Linux
Im using varnish from the .tar.gz package on SF.
Thank you
Erik
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
hash.url missing in varnish 1.1.2
Hi, Downloaded and installed 1.1.2 from SF and noticed that hash.url has been removed from varnishadm. Didn't find anything about that on http://varnish.projects.linpro.no/wiki/Releases/1.1.2 . Why has it been removed? Cheers Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
purge_url
I will stop bothering the helpful guys at the irc for a minute and ask a
question here ;)
My VCL conf looks like this:
sub vcl_recv {
#Change the host header to www.mysite.com
if(req.http.host == 10.1.1.54 || req.http.host == test.mysite.com) {
set req.http.host = www.mysite.com;
}
#Purge specified files from acl purge.
#Purge = Delete the specified url from the cache
if(req.request == PURGE) {
if(client.ip ~ purge) {
purge_url(req.url);
}
}
sub vcl_miss {
#set http version 1.1
set bereq.proto = HTTP/1.1;
if (req.request == PURGE) {
error 404 The url could not be found in the cache.;
}
}
sub vcl_hit {
if (req.request == PURGE) {
set obj.ttl = 0s;
error 200 Purged successfully!;
}
}
}
# End of VCL Conf
I receive a 404 error The url could not be found in the cache. when I try to
PURGE the url.
PURGE http://www.mysite.com/data/hello.gif HTTP/1.1
This is the varnishlog when I try to purge:
12 RxRequestc PURGE
12 RxURLc /data/hello.gif
12 RxProtocol c HTTP/1.1
12 VCL_call c recv
12 VCL_acl c MATCH purge 84.xx.xx.xx
12 VCL_return c lookup
12 VCL_call c hash
12 VCL_return c hash
12 Debugc Hash: /data/hello.gif#10.1.1.54:80#
12 VCL_call c miss
0 Debug VCL_error(404, The url could not be found in the cache.)
12 VCL_return c error
12 Length c 453
12 TxProtocol c HTTP/1.0
12 TxStatus c 404
12 TxResponse c Not Found
12 TxHeader c Server: Varnish
12 TxHeader c Retry-After: 0
12 TxHeader c Content-Type: text/html; charset=utf-8
12 TxHeader c Content-Length: 453
12 TxHeader c Date: Thu, 13 Dec 2007 11:34:50 GMT
12 TxHeader c X-Varnish: 1188471893
12 TxHeader c Age: nan
12 TxHeader c Via: 1.1 varnish
12 TxHeader c Connection: keep-alive
When I try to request it with GET i receive:
12 ReqStart c 84.xx.xx.xx 32162 1188471894
12 RxRequestc GET
12 RxURLc /data/hello.gif
12 RxProtocol c HTTP/1.1
12 RxHeader c Host: www.mysite.com
12 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1;
sv-SE; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
12 RxHeader c Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
12 RxHeader c Accept-Language: sv,en-us;q=0.7,en;q=0.3
12 RxHeader c Accept-Encoding: gzip,deflate
12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
12 RxHeader c Keep-Alive: 300
12 RxHeader c Connection: keep-alive
12 RxHeader c
Cookie:__utma=75765975.864378540.1197378455.1197453843.1197535348.6;
12 VCL_call c recv
12 VCL_return c lookup
12 VCL_call c hash
12 VCL_return c hash
12 Debugc Hash Match: /data/hello.gif#www.mysite.com#
12 Hit c 1188471893
12 VCL_call c hit
12 VCL_return c deliver
12 Length c 905
12 VCL_call c deliver
12 VCL_return c deliver
12 TxProtocol c HTTP/1.1
12 TxStatus c 200
12 TxResponse c OK
12 TxHeader c Server: Zeus/4.3
12 TxHeader c Content-Type: image/gif
12 TxHeader c Content-Language: sv-SE
12 TxHeader c Content-Length: 905
12 TxHeader c Date: Thu, 13 Dec 2007 11:37:26 GMT
12 TxHeader c X-Varnish: 1188471894 1188471893
12 TxHeader c Age: 43
12 TxHeader c Via: 1.1 varnish
12 TxHeader c Connection: keep-alive
The strange thing is that the host is different in PURGE Match and GET Match. I
have used the same host on both requests. Does it have anything to do with the
vcl code where Im changing the hostheader?
/ Erik
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish Hash
Just to make this clear, does varnish identify an object like this in vcl_hash?
sub vcl_hash {
set req.hash += req.url;
set req.hash += req.http.host;
hash;
}
/ Erik
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish and HTTPS
Hi, I know that Varnish doesn't handle https. But what if the request is in https? Would varnish pass it to the backend or just drop it? / Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
RE: Re: Varnish and HTTPS
Thanks for the info but I did just wondered if varnish pass it to the backend or if it drops the request. I did some tests and it seems like it passes the request to the backend. BTW, what program may I use to have https2http? Isn't https encrypted? / Erik Original Message --- Erik [EMAIL PROTECTED] writes: I know that Varnish doesn't handle https. But what if the request is in https? Would varnish pass it to the backend or just drop it? It would not be a normal HTTP request, so Varnish would not try to handle it. You could place a https to http gateway between your users and varnish, so varnish can accelerate the http traffic. The pipeline would look like this: Duh Internet - https2http gw - varnish - http server -- Stig Sandbeck Mathisen, Linpro ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
RE: Re: Varnish crashes everyday and same time
Hi, This is what I do and what I get: /etc/init.d/varnish start Starting Varnish: varnishUsing old SHMFILE rolling(2)... It seems to me that the varnish is running? But when trying to connect it doesn't work! Althought when I run without -d -d or -d it works! I would really like to commit some logdata from varnishd but since I cant get the debug to work it has to wait :( / Erik Original Message --- you need to write start after starting varnishd in debug mode. janis a satisfied Varnishd user :) On Thursday 22 November 2007 12:18, Erik wrote: Hi again, I made some logging of the memory and it looks fine. I also turned of VCL Trace but that didn't solved it. The crash happened again today but a few hours later then usual. I tried to start varnishd in debug mode but I cant get it to work. When I set it to -d or -d -d it starts but no connection can be made against it. Any ideas? I forgot to mention that Im running varnish on a Virtual Server 2005 with 512 MB RAM (150 MB free) and 10 GB diskspace. / Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish crashes everyday and same time
Hi, Varnish crashes everyday, same time. This is what I got from the log files: 12 SessionOpen c xx.xx.xx.xx 31989 0 Debug Acceptor is epoll 0 Error CLI read 0 (errno=32) I also found this thread in the mailing archive from July: http://projects.linpro.no/pipermail/varnish-misc/2007-July/000670html That is the last post on that subject, no answer was posted to Anup Shukla. I dont know if that is the same problem but it has the same error in one of the posts. Im running varnish compiled from source on Debian 4.0 Etch. Im gonna start a logjob of the memory to see if thats its the problem. / Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Debug: On Waiting list?
Hi,
Im struggling with a couple of high response times when using
varnish on our site. I have configured varnish so it will
cache only static content (jpg,gif,png,css,js etc) even if a
cookie is set. But all our jpeg's are getting a high response
time when Im running some test against the site.
One of the jpeg's are pic_mini.jpg, it has a response time of 357ms. I turned
on logging and vcl_trace and found the line:
14 Debugc Hash Match:
/Portals/0/Picture/pic_mini.jpg#www3.mydomain.com#
14 Debugc on waiting list on obj 1767384108
Does on waiting list has something to do with the high response time?
The rest fo the log for pic_mini.jpg is:
14 ReqStart c xx.xx.xx.187 41506 1767384112
14 RxRequestc GET
14 RxURLc /Portals/0/Picture/pic_mini.jpg
14 RxProtocol c HTTP/1.1
14 RxHeader c Host: www3.mydomain.com
14 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows
NT 5.1; sv-SE; rv:1.8.1.8) Gecko/20071008 Firefox/2.0.0.8
14 RxHeader c Accept: */*
14 RxHeader c Accept-Language: sv,en-us;q=3D0.7,en;q=3D0.3
14 RxHeader c Accept-Encoding: gzip,deflate
14 RxHeader c Accept-Charset:=
ISO-8859-1,utf-8;q=3D0.7,*;q=3D0.7
14 RxHeader c Keep-Alive: 300
14 RxHeader c Connection: Keep-Alive
14 RxHeader c Cookie: language=3Den-US;
.ASPXANONYMOUS=3DnBeDLiReyAEkYTIwNmNiYjYtNmRmMS00MmQ4LThlZGItY=
mZlNThjYjhjZjky0
14 VCL_call c recv
14 VCL_tracec 1 84.14
14 VCL_tracec 2 87.13
14 VCL_tracec 3 87.51
14 VCL_return c lookup
14 VCL_call c hash
14 VCL_tracec 47 22.14
14 VCL_tracec 48 24.9
14 VCL_tracec 49 24.24
14 VCL_return c hash
14 Debugc Hash Match:
/Portals/0/Picture/pic_mini.jpg#www3.mydomain.com#
14 Debugc on waiting list on obj 1767384108
In vcl_recv im fetching static content using:
sub vcl_recv {
# *.jpg , *.jpeg , *.gif , *.png , *.ico
if (req.url ~ .(jpg|gif|png|jpeg|ico)$) {
lookup;
}
# *.js , *.css , *.swf
if (req.url ~ .(js|css|swf)$) {
lookup;
}
if (req.http.Cookie) {
# *.jpg , *.jpeg , *.gif , *.png , *.ico
if (req.url ~ .(jpg|gif|png|jpeg|ico)$) {
lookup;
}
# *.js , *.css , *.swf
if (req.url ~ .(js|css|swf)$) {
lookup;
}
}
pass;
} #RECV
and my vcl_fetch is using:
sub vcl_fetch {
# *.jpg , *.jpeg , *.gif , *.png , *.ico
if (req.url ~ .(jpg|gif|png|jpeg|ico)$) {
insert;
}
# *.js , *.css , *.swf
if (req.url ~ .(js|css|swf)$) {
insert;
}
if (obj.http.Set-Cookie) {
# *.jpg , *.jpeg , *.gif , *.png , *.ico
if (req.url ~ .(jpg|gif|png|jpeg|ico)$) {
insert;
}
# *.js , *.css , *.swf
if (req.url ~ .(js|css|swf)$) {
insert;
}
pass;
}
#
pass;
} #FETCH
// Erik
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnishstat vs. telnet stats
Ok. Thanks for all your help mate ;) Erik - Original Message - From: Peter Oomen [EMAIL PROTECTED] To: Erik [EMAIL PROTECTED] Sent: Thursday, September 20, 2007 4:34 PM Subject: Re: varnishstat vs. telnet stats yes, cache hits mean hits from the cache.. On 9/20/07, Erik [EMAIL PROTECTED] wrote: Thank you very much. It is up n running ;) A few question about the cache tho. The stats are saying that it has fetched like 40 items and have had about 650 cache hits. Is this enough to tell if the cache is running, I mean, cache hits? that must mean that its serving from the cache? // Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.5.487 / Virus Database: 269.13.25/1018 - Release Date: 2007-09-19 15:59 ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: varnishstat vs. telnet stats
Ok. Thanks for all your help mate ;) Erik yes, cache hits mean hits from the cache.. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
varnishstat vs. telnet stats
Thank you very much. It is up n running ;) A few question about the cache tho. The stats are saying that it has fetched like 40 items and have had about 650 cache hits. Is this enough to tell if the cache is running, I mean, cache hits? that must mean that its serving from the cache? // Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Problems with the cache
Im trying to get varnish running smoothly on my server with apache on it. Unforunately Im having some problems to get the caching thing working. I have set varnishd to use default.vcl on a Debian 4.0 etch OS. it starts up without any problem but doesn't cache anything. Also the varnishlog doesn't seem to log anything. I'm having it running as a daemon, it's writing the log to /var/log/varnish/varnish.log but that file is empty. Varnishstat doesn't update itself when using vcl, only when using -b localhost:8080 as a parameter to varnishd. Im using all default config files, except that I have edited the backend in default.vcl and in /etc/default/varnish . However, if I use the -b parameter woith varnishd, I could get varnishd to work but nothing is cached. Alot of objects are fetched but nothing is parsed out to the client. It just fetches it from the backend server over and over again. As you see, i have alot of problems. It became a big mess after some tweaking and now am stuck with alooot of problems :( / Erik ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
