Re: Survey; how do you use Varnish?
Tollef Fog Heen napsal(a): > ]] Václav Bílek > > | 5)gracefull restart( not in meaning of persistent cache, but in the > | meaning of not flushing all clients and letting them wait for tens of > | seconds till enough threads appear.) > > You can tune the thread_pool_add_delay parameter to at least bring the > time down somewhat. > alredy set to "1" bud for 4000 threads it takes around 20seconds till fully working state ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: GRSEC and Varnish
]] Bernardf FRIT | Then the parent varnishd process starts immediately a new child process | which lasts some time. | | Is there any way to fix this. Remocve the GRSEC kernel ? Upgrade the | kernel ? Varnish ? or whatever ? Work out why it thinks that varnishd is doing something wrong? It doesn't seem to say so in the log. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
]] Václav Bílek | 5)gracefull restart( not in meaning of persistent cache, but in the | meaning of not flushing all clients and letting them wait for tens of | seconds till enough threads appear.) You can tune the thread_pool_add_delay parameter to at least bring the time down somewhat. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
RE: Survey; how do you use Varnish?
1) How many servers do you have running Varnish? 8 servers (2 sites x 4 servers), load balanced behind F5 GTM. We aim to be able to lose a site AND suffer a hardware failure and keep on truckin'. We could probably run on one or two servers at a push, but our backend would most likely explode before Varnish broke a sweat. Each server is a quad core Xeon w/ 16G RAM. We have a fairly large working set. (+1 Varnish server for Dev / test, which is a VM) 2) What sort of total load are you having? Mbit/s or hits per second are preferred metrics. ~900 req/sec at peak per Prod server / 60Mbps 3) What sort of site is it? *) Retail <= online auctions 4) Do you use ESI? No. 5) What features are you missing from Varnish. Varnishlog filtering language (or other enhancements in this area) Dynamic stats counters Large dataset performance improvements -Original Message- From: varnish-misc-boun...@projects.linpro.no [mailto:varnish-misc-boun...@projects.linpro.no] On Behalf Of Martin Boer Sent: Tuesday, 2 February 2010 10:23 p.m. To: Per Andreas Buer Cc: varnish-misc@projects.linpro.no Subject: Re: Survey; how do you use Varnish? 1) One active server. We have another one as hot standby. 2) 50Mbit, 200 requests/second max. Most of the time it's 10Mbit, 40 requests/second which isn't much. 3) Internet touroperator. 4) Nope 5) Automatic refreshing of data without having the endusers have to wait for the response. The reason we use varnish most is because our website has complex, timeconsuming queries to backend systems. The answers to these queries do vary several times per day but are still cachable. Of course varnish also helps te bring down the load on those backend systems but the main use is that varnish gives the endusers a lightning fast prerendered interactive experience which is a paradox. We like working paradoxes. Something like 'refresh pages after object.prefetch seconds if at least someone requested that object the last object.ttl seconds' where object.ttl is larger then object.refresh. So an object might be prefetched even a couple of times without anyone being interested but will be removed from the cache eventually after object.ttl has expired. Regards, Martin Boer Per Andreas Buer wrote: > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Fwd: Varnish caching 503 error pages
You can use the obj.cacheable property, which is a suggestion from Varnish
about the response should be cacheable. By default Error response codes
aren't cached, so test the following:
sub vcl_fetch {
if (!obj.cacheable){
return (pass);
}
}
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish caching 503 error pages
If your default_ttl is not 0, then this may be the expected behavior. I'm not
sure if Varnish should really ever cache >=500 responses?
But in VCL you could do something like:
sub vcl_fetch {
if ( obj.status >= 500 ) {
set obj.ttl = 0s;
set obj.cacheable = false;
}
}
Adjusting timeouts for ttl for 404s is also handy. Hope it helps,
--
Ken
On Feb 2, 2010, at 1:37 PM, pub crawler wrote:
> Thought I'd ask the list before I went on a voyage with this one.
>
> Sometime our backend app servers gets overloaded and go into
> protection mode whereby they sends out 503 errors until they recover.
>
> Varnish is in front of the app servers and when this happens the 503
> ends up as a cached item in Varnish.
>
> Here's output to show such:
>
> HTTP/1.1 503 Service Unavailable
> Keep-Alive: timeout=30
> Content-Type: text/html
> Content-Language: en-US
> Content-Length: 122
> Date: Tue, 02 Feb 2010 21:34:15 GMT
> X-Varnish: 655791101 655790499
> Age: 97
> Via: 1.1 varnish
> Connection: keep-alive
> X-Served-By: atom3302
> X-Cache: HIT
> X-Cache-Hits: 13
>
> What is the best way to explicitly refuse caching of 503 errors by Varnish?
>
> Thanks!
> ___
> varnish-misc mailing list
> varnish-misc@projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish caching 503 error pages
Thought I'd ask the list before I went on a voyage with this one. Sometime our backend app servers gets overloaded and go into protection mode whereby they sends out 503 errors until they recover. Varnish is in front of the app servers and when this happens the 503 ends up as a cached item in Varnish. Here's output to show such: HTTP/1.1 503 Service Unavailable Keep-Alive: timeout=30 Content-Type: text/html Content-Language: en-US Content-Length: 122 Date: Tue, 02 Feb 2010 21:34:15 GMT X-Varnish: 655791101 655790499 Age: 97 Via: 1.1 varnish Connection: keep-alive X-Served-By: atom3302 X-Cache: HIT X-Cache-Hits: 13 What is the best way to explicitly refuse caching of 503 errors by Varnish? Thanks! ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
GRSEC and Varnish
Hi, I'am running : - varnishd (varnish-2.0.4) - linux kernel 2.6.27.10-grsec--grs-ipv4-64 and it appears that the grsec Kernel repeatedly and unexpectedly sends signal 11 to the varnishd child. .../... Feb 2 12:01:02 XX varnishd[17111]: segfault at 1000 ip 0043abf0 sp 47d89ae0 error 4 in varnishd[40+5] Feb 2 12:01:02 XX grsec: From 82.67.39.69: signal 11 sent to /usr/sbin/varnishd[varnishd:17111] uid/euid:65534/65534 gid/egid:65534/65534, parent /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 13:45:44 XX varnishd[22187]: segfault at f5000 ip 0043abf0 sp 48538ae0 error 4 in varnishd[40+5] Feb 2 13:45:44 XX grsec: From 80.13.19.228: signal 11 sent to /usr/sbin/varnishd[varnishd:22187] uid/euid:65534/65534 gid/egid:65534/65534, paren t /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 13:54:57 XX varnishd[22236]: segfault at 1000 ip 0043abf0 sp 45445ae0 error 4 in varnishd[40+5] Feb 2 13:54:57 XX grsec: From 80.13.19.228: signal 11 sent to /usr/sbin/varnishd[varnishd:22236] uid/euid:65534/65534 gid/egid:65534/65534, paren t /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 14:13:41 XX varnishd[22595]: segfault at ae000 ip 0043abf0 sp 40ff4ae0 error 4 in varnishd[40+5] Feb 2 14:13:41 XX grsec: From 83.145.80.130: signal 11 sent to /usr/sbin/varnishd[varnishd:22595] uid/euid:65534/65534 gid/egid:65534/65534, pare nt /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 14:31:08 XX varnishd[23547]: segfault at 1000 ip 0043abf0 sp 45b40ae0 error 4 in varnishd[40+5] Feb 2 14:31:08 XX grsec: From 81.49.118.48: signal 11 sent to /usr/sbin/varnishd[varnishd:23547] uid/euid:65534/65534 gid/egid:65534/65534, paren t /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Feb 2 16:19:05 XX varnishd[24256]: segfault at f7000 ip 0043abf0 sp 473bcae0 error 4 in varnishd[40+5] Feb 2 16:19:05 XX grsec: From 192.196.142.20: signal 11 sent to /usr/sbin/varnishd[varnishd:24256] uid/euid:65534/65534 gid/egid:65534/65534, par ent /usr/sbin/varnishd[varnishd:28927] uid/euid:0/0 gid/egid:0/0 Then the parent varnishd process starts immediately a new child process which lasts some time. Is there any way to fix this. Remocve the GRSEC kernel ? Upgrade the kernel ? Varnish ? or whatever ? Thanks in advance. -- Bernard FRIT ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
1)8 2)600-800Mbit/s 60Kreq/s ( there is still big margin on that 8 servers) 3)sport realtime results 4)no 5)gracefull restart( not in meaning of persistent cache, but in the meaning of not flushing all clients and letting them wait for tens of seconds till enough threads appear.) Per Andreas Buer napsal(a): > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Memory usage
In message <24a219a51001261923n40146083yb221aac2cadbc...@mail.gmail.com>, Marti n Goldman writes: >1. How can you tell whether your Varnish objects fit in RAM? If you start seeing disk-activity, they do not fit. >2. If I have objects residing in virtual memory, to what extent will my >performance be adversely affected? If I want my site to be fast, do I >basically need to go out and buy as much RAM as it will take so that virtual >memory isn't needed? Well, the impact is the necessary disk-I/O to bring the object into RAM. Getting more RAM is one solution, but if your working set is much larger than 4G, getting a SSD disk instead might be a better investment. >3. I noticed tonight that my machine was using a few hundred megs of swap >space, Yes, varnish will force inactive programs (inetd, getty, sendmail etc) out to swap so it can get at the RAM. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
1) ~20 (Mixture of production and development environments, spread across London, Tokyo and New York) 2) Low figures - 300-500 hits/s at peak. A lot of our backends have very high latency though (~200ms just to connect to the server), and are predominantly static content, so caching speeds up application loading considerably. 3) Global corporate intranet (Finance sector) 4) No, but we have looked at it. The lack of user/entitlements awareness put us off it, and we moved the logic to another layer. 5) #1 - GZIP compression #2 - SSL support (I know, I know...) - both for accepting incoming connections, and for outgoing connections to backends Thanks, Sam On 29 January 2010 14:48, Per Andreas Buer wrote: > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > > -- > Per Andreas Buer > Redpill Linpro Group - Changing the Game > > Mobile +47 958 39 117 / Phone: +47 21 54 41 21 > ___ > varnish-misc mailing list > varnish-misc@projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
1) 15 2) Arround 10k/s results in 500-600Mbit. 3) Social Network website 4) No 5) Large dataset performance improvements, High traffic performance improvements rr ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Survey; how do you use Varnish?
1) One active server. We have another one as hot standby. 2) 50Mbit, 200 requests/second max. Most of the time it's 10Mbit, 40 requests/second which isn't much. 3) Internet touroperator. 4) Nope 5) Automatic refreshing of data without having the endusers have to wait for the response. The reason we use varnish most is because our website has complex, timeconsuming queries to backend systems. The answers to these queries do vary several times per day but are still cachable. Of course varnish also helps te bring down the load on those backend systems but the main use is that varnish gives the endusers a lightning fast prerendered interactive experience which is a paradox. We like working paradoxes. Something like 'refresh pages after object.prefetch seconds if at least someone requested that object the last object.ttl seconds' where object.ttl is larger then object.refresh. So an object might be prefetched even a couple of times without anyone being interested but will be removed from the cache eventually after object.ttl has expired. Regards, Martin Boer Per Andreas Buer wrote: > Hi list. > > I'm working for Redpill Linpro, you might have heard of us - we're the main > sponsor of Varnish development. We're a bit curious about how Varnish is > used, what features are used and what is missing. What does a typical > installation look like? The information you would choose to reveal to me > would be aggregated and deleted and I promise you I won't use it for any > sales activities or harass you in any way. We will pubish the result on this > list if the feedback is significant. If you have the time and would like to > help us please take some time and answer the questions in a direct mail to > me. Thanks. > > 1) How many servers do you have running Varnish? > > 2) What sort of total load are you having? Mbit/s or hits per second are > preferred metrics. > > 3) What sort of site is it? > *) Online media > *) Cooperate website (ibm.com or similar) > *) Retail > *) Educational > *) Social website > > 4) Do you use ESI? > > 5) What features are you missing from Varnish. Max three features, > prioritized. Please refer to > http://varnish-cache.org/wiki/PostTwoShoppingList for features. > > > > ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
