How useful various stats will be can certainly be discussed. I believe being able do show the rate of connections, requests, bytes transferred (and possibly also number of active connections and requests) will be very helpful when trying to diagnose load issues on a server with 10s of thousands of active connections.
In order to show the rate of connections being made you have to take multiple samples of the total number of connections (and take the delta and divide by time). I don't think there is an easy way to collect the total number of HTTP connections, outside of the library itself. It should be a relatively trivial to add a counter or two to the library. Creating/destroying connections within the library is already race-free. I don't mind adding a simple patch to the library. I just thought there might be others who possibly wanted something similar, or maybe already have added some statistics to libmicrohttpd. I will be happy to share with others any such changes/patches that I may make. As for adding epoll to libmicrohttpd, I don't see it as being all that useful. If you run the daemon threaded mode, each thread will be polling a single fd. I have run oprofile on the system while serving over 10k connections and close to 10 gbps of network load. The kernel polling code is not significant at all. -eivind ________________________________________ From: [email protected] [[email protected]] On Behalf Of Christian Grothoff [[email protected]] Sent: Saturday, February 11, 2012 3:20 AM To: [email protected] Subject: Re: [libmicrohttpd] connection statistics The real issue with this kind of performance monitoring is that for multi-threaded applications it is very hard to give you a "sound" number. Also, I don't see how this would be helpful for the application that is running at the time. If the system operator wants this information, it is pretty easy to get a snapshot: lsof and netstat will show you how many connections the application has open and what the overall TCP state looks like for the OS. So for server status monitoring, why not use these standard tools? Now, if you actually have a very large number of mostly keep-alive connections, there are two possible answers: shorter timeout (so that they go away), or you're yet another person who'd like to see ePoll support for MHD (which is not that easy to add, hence it is not likely that I'll find the time to do this anytime soon). Looking at how many select/poll calls you're doing and how long they take (for example, using 'strace -c -e trace=select,poll') might be a good idea here. In summary, I'm not sure I see a need for this kind of performance-monitoring support to be integrated with MHD, as the OS already gives you this and more high-quality information easily. My 2 cents! Happy hacking! Christian On 02/11/2012 12:40 AM, Eivind Sarto wrote: > The completion callback handler is called whenever an HTTP request has > completed. > At that point there is no way of knowing if the connection remains open. > > -eivind > ________________________________________ > From: [email protected] > [[email protected]] On Behalf Of Keith Mendoza > [[email protected]] > Sent: Friday, February 10, 2012 3:32 PM > To: libmicrohttpd development and user mailinglist > Subject: Re: [libmicrohttpd] connection statistics > > If you're intent is to see the number of active connections at the > moment I would say that this is something that you should be able to > do from within your application. This might even prove useful to you > at some future time in handling streaming of the video data. I can't > provide any use cases, but I'm getting the "it might be useful" itch. > > If I understand it correctly, the completion handler is called when > the HTTP connected between your application and the client is closed. > So, if the keep-alive is still there wouldn't that mean that the > connection is still open and the completion callback should not be > called? > > On Fri, Feb 10, 2012 at 3:17 PM, Eivind Sarto<[email protected]> wrote: >> I am using libmicrohttpd for a video streaming project. >> The HTTP server load can be quite high with a large number of requests per >> seconds and a high network bandwidth, >> but the HTTP connections are mostly keep-alives. >> >> In order to troubleshoot load related problems, it can be difficult to get >> an idea of what is going on a given server. >> I want to add some kind of server status monitoring and I would like to be >> able to display the number of active >> connections/requests and total connections/requests (among other things). >> >> I can collect total and active number of requests from counters in the >> callback to the default handler and the completion handler. >> However, I cannot find a way to get the total and active number of >> connections. The only one who can keep an accurate count of >> the number of connections is the internals of the library. >> >> Does anyone see any value to adding some statistics/counters to >> libmicrohttpd that can returned via an API? >> >> Or, is there something I am overlooking that I could do to display the >> number of active connections and total connections created? >> >> -eivind >> >> > >
