Re: varnishstat vs. telnet stats
Ok. Thanks for all your help mate ;) Erik yes, cache hits mean hits from the cache.. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Description of varnishstat-items
Hi. We're trying to understand some of the numbers from varnishstat. In particular, we're wondering when: Client connections accepted Client requests received are increased. Are they increased when the connection is accepted and when the request is received, or will Varnish wait until the request is handled? The reason I ask, is that we have some backends that are unable to handle the traffic if there are a lot of cache misses. This will result in a lot of hanging requests. At the moment, we're rejecting all the requests to these backends. However, if we turn it back on, connections and requests decreases to less than 1/5 of what it was before. So basically, I'm wondering if this indicates that there's a congestion in our network, or if it's just a natural consequence of the requests taking longer to handle? -- Trond Michelsen ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Varnish mini status...
This is just an update to all our patient (and in some cases less patient :-) users out there. I'm back on Varnish now, after having spent a week on EuroBSDcon2007 and I thought it would be a good idea to outline what is going to happen from now on. The two big things are ESI:include, which we have promised our sponsors will be finished Real Soon Now, and bugfixes. Since the ESI:include stuff will be pretty drastic, seen from a source-code point of view, I want to make sure we have a usable -trunk version before I pull it apart again. With respect to bugs, here is a little taxonomy: functional or Varnish does something wrong These are usually easy to reproduce and quick to fix, once we know them. I will hit these right away once I can see what is going on. stability or Varnish crashes after some time Some of these will, upon examination, transpire to be in the functional category, but digging the evidence out of the logfiles to find this out can take considerable time. The remainder are often very time consuming to nail down because they are hard to reproduce. I keep looking for new clues to what might be causing these and as I work the source code for different purposes I look out for things that could be causing problems. (memory leaks etc). However, there is little point in sorting the entire haystack, hoping to find a needle, given that I'm not even sure if it is a needle I'm looking for, so it is not productive to spend my full attention on these, until some kind of clue appears. I know it is frustrating to you to have a known bug of this class, but trust me, I am trying to nail it. nice to have or I wish varnish could do this Some of these are simple and will happen right away, others are complex and will be put in the ever-expanding wish-list and some are so big that we need sponsors to get through them. About the ESI:include stuff. One of the things which will fall out of this is the ability to prefetch, it's essentially the same code that will be needed and it is my plan to implement this first. After that, there is a pretty major shift in memory allocation, because we need to be able to stack requests on a session to implement the nesting of ESI:include because the included document can also contain an ESI:include, and so on. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Description of varnishstat-items
On Fri, Sep 21, 2007 at 08:22:12AM +, Poul-Henning Kamp wrote: In message [EMAIL PROTECTED], Trond Michelsen writes: In particular, we're wondering when: Client connections accepted This is increased when we have accepted the connection. On systems with accept-filters this does usually not happen until we have also received the first request. Client requests received This is increased whenever we have complete request and starts to service it. Thank you. That's pretty much what we expected. The reason I ask, is that we have some backends that are unable to handle the traffic if there are a lot of cache misses. This will result in a lot of hanging requests. At the moment, we're rejecting all the requests to these backends. However, if we turn it back on, connections and requests decreases to less than 1/5 of what it was before. So basically, I'm wondering if this indicates that there's a congestion in our network, or if it's just a natural consequence of the requests taking longer to handle? I'm not sure I can answer that based on the information you have provided... Yeah, I know I'm a bit vague here. Sorry about that. Here's a quick overview of our setup: We have two Varnish mashines. Varnish1 is a frontend for a webservice that generates images. Varnish2 is a frontent for a webservice that generates XML-content. These services runs on separate servers. The Webserver fetches XML-data from Varnish2 to generate HTML-pages, which contains links to the images that are served by Varnish1. So, basically all traffic to Varnish2 comes from a single machine, while the traffic to Varnish1 comes from everyone. The Webserver is located offsite. With a single FTP-connection, we get a transfer rate of 250-300Mbps between the webserver and Varnish2. Both varnish1 and 2 have a gigabit-connection to NIX. Both Varnish machines are 2xdual-core Xeon servers with 8GB memory. The backends for Varnish1 and 2 have NFS-mounted disks from the same NFS-server. Right now, we have added a rule to Varnish1, that will reject any request for the images that are most time consuming to generate for the backends. Whenever we remove this restriction, and allow all requests to go through to the backend - the number of incoming requests to Varnish1 decreases dramatically. This isn't necessarily surprising, since the clients will have to wait for the first images to come through before requesting the next. The thing that does surprise us is that whenever we try to allow all images from Varnish1, after about 30s the webserver becomes unable to connect to Varnish2. Connects from the webserver to Varnish2 piles up, and without the XML-data, it cannot generate any webpages at all. So far we have not been able to determine if Varnish2 is refusing connections, or if the connections never get to the varnish-machines at all. -- Trond Michelsen ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
