Re: [Libevent-users] [UPDATE] Using the sourceforge.net bugtracker.
On Tue, Nov 06, 2007 at 02:05:30AM -0500, Nick Mathewson wrote: > Update: I have added 2 patches, 10 feature requests, and 12 bug > reports to the tracker. I re-read or skimmed about 500 emails to do > this, so it is likely I missed something, or added a bug that has > already been fixed. My apologies in advance. Hey, thanks Nick. The work is appreciated. -cl ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] [UPDATE] Using the sourceforge.net bugtracker.
On Mon, Nov 05, 2007 at 02:58:27PM -0500, Nick Mathewson wrote: > Hi, all! > > I've noticed that we're getting a lot more bug reports and patches > than can easily be handled by the "mailbox method" of organization. > Clearly, we need to do a better job. Fortunately, we have a > sourceforge.net project page for Libevent: > http://sourceforge.net/projects/levent > This page has trackers for bugs, patches, and feature requests. Let's > use it! Update: I have added 2 patches, 10 feature requests, and 12 bug reports to the tracker. I re-read or skimmed about 500 emails to do this, so it is likely I missed something, or added a bug that has already been fixed. My apologies in advance. I. What to do with the stuff on the tracker list If you have a fix for a bug or an implementation for a feature, add a new patch for it. Maybe discuss your approach a little first, if it's tricky. Niels -- I haven't done any triage on the lists yet. Some of the things listed should get fixed in 1.3; some are probably 1.4 showstoppers; and some can probably wait till 1.5 or later. II. Adding new stuff to the list. Every bug or feature request or patch that I know about is on the tracker now. If you have a bug or feature request or patch that *isn't* there, then please do the following: - Double-check that I didn't add it to one of the lists. - Confirm that the bug still exists. - Verify that the bug is really a bug - Come up with some test code to elicit the buggy behavior, if you possibly can. (If you are sure that the code is wrong, but you can't actually write code to provoke the bad behavior that you're sure is happening, then maybe you haven't really understood the code. Ask on the list maybe?) - Submit it to the tracker. [If this sounds like too much work, just bounce me an email saying "hey, you missed this one!"] II. How to find out about new tracker entries There's a mailing list at [EMAIL PROTECTED]; you can subscribe here: https://lists.sourceforge.net/lists/listinfo/levent-tracker III. Other stuff you should know, and which should probably go on the webpage. The commits list is here: https://lists.sourceforge.net/lists/listinfo/levent-commits The SVN repository is here: https://levent.svn.sourceforge.net/svnroot/levent Things that you might want to check out of it are: https://levent.svn.sourceforge.net/svnroot/levent/trunk/libevent https://levent.svn.sourceforge.net/svnroot/levent/branches/patches-1.3/libevent Now, it's really time for me to sleep. peace, all. -- Nick Mathewson pgpXoRbZPOWro.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] web link
On 11/5/07, Niels Provos <[EMAIL PROTECTED]> wrote: > > Should be fixed now. > > On 11/5/07, arthur <[EMAIL PROTECTED]> wrote: > > > > > > Looks the web link in the end of each email is broken. > > > > http://monkey.org/mailman/listinfo/libevent-users > > __ It could be nice to keep the link to svn repo at some public place web page. -- Sincerely, Robert Iakobashvili, coroberti %x40 gmail %x2e com ... http://curl-loader.sourceforge.net A web testing and traffic generation tool. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] TAILQ_ENTRY missing in evhttp.h on linux
On Tue, Oct 30, 2007 at 10:30:26AM -0500, Paul Fisher wrote:
> In using the latest 1.3e on linux, evhttp.h fails to compile because
> of a missing definition of TAILQ_ENTRY in evhttp.h. This is due to
> the fact that the workaround in event.h is #define'd and #undef'd
> within event.h and not available to evhttp.h when defining "struct
> evhttp_request". This patch obviously fixes it:
>
> --- libevent-1.3e/evhttp.h2007-08-25 13:49:22.0 -0500
> +++ libevent-1.3e.002/evhttp.h2007-10-29 22:32:07.0 -0500
> @@ -108,7 +108,14 @@
> * reasonable accessors.
> */
> struct evhttp_request {
> +#if defined(TAILQ_ENTRY)
> TAILQ_ENTRY(evhttp_request) next;
> +#else
> +struct { \
> + struct type *tqe_next; /* next element */ \
> + struct type **tqe_prev; /* address of previous next element */ \
> +} next;
> +#endif
I think you mean "struct evhttp_request" instead of "struct type".
I've checked this in; it's not the most elegant fix (as noted in the
rest of the thread), but an inelegant fix beats no fix at all.
--
Nick Mathewson
pgplwJtbB6uU5.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] Using libev with evbuffer & interpreted C++
I have been working on a project to make libevent, eventxx, and bufferevent work together with the CINT C++ interpreter, and have sample tests working normal events and buffered events. The libev announcement inspired me to try it out, and it is working so far quite well for me. The project is at: http://code.google.com/p/event-hpp/, with the documentation at: http://www.larcpublishing.com/doc/event-hpp/ My bufferevent sample test is at: http://www.larcpublishing.com/doc/event-hpp/test-buffer-eof_8cpp-example.html The evbuffer part is split out, according to some discussion in the recent past. The evbuffer part is built from evbuffer.c, buffer.c, and event_tagging.c. I hope to get to the HTTP and RPC part in the near future. This project is designed to be built to run on a C/C++ interpreter (http://root.cern.ch/twiki/bin/view/ROOT/CINT). I am primarily using this as a personal research project, but I hope people can find the modularization and my rudimentary EvBufferEvent class useful. Chris ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Mon, Nov 05, 2007 at 02:13:20PM -0800, Scott Lamb <[EMAIL PROTECTED]> wrote: > >Its higher than necessary, and more complex than neccessary. > > Higher than necessary? I don't care; it's worth it. > > More complex than necessary? Not really. Well, I gave an example that is less complex and more efficient, the best of all worlds. And fully binary-compatible. > >responsible. > > Erm, no. In the ENOMEM case, I know which fd you decided to kill, though > not actually that you've killed it. ENOMEM is not a per-file descriptor > problem. The word "kill" is misleading, the fd is still there, alive and kicking. But the event handler has been stopped. > I'm looking at the Linux kernel source code now (core_sys_select), and > you are wrong on Linux as well: Maybe differeing linux versions? I definitely get ENOMEM with gigabytes of free memory whne exceeding the configured limit. Of course, linux might *also* return ENOMEM for transient conditions. > might be useful, but it's nothing the caller can't do on its own by > retrying the failed function. The problem is that *all* callers will need to do this, and this is not something I will ever require of all callers. As a callback will solve your problem (the special case) and will make it much easier for the general case, it definitely seems the better solution, leading to better programs in general. > which allowed attributes specified at eventloop creation time > (extensible without further API changes). One of them might have been > > libname_eventloop_attr_setenomem(libname_eventloop_attr_t *, void > (*enomem_callback)(/*undecided*/), void *enomem_callback_arg); Thats interesting, but I guess i will go for a simpler set-callback function. The question still remains under what conditions exactly to call it - *alloc is obvious, but should there be a separate handler > >>* any X11 application is screwed if you close its connection to the server > > > >Uhm, no? :) > > Uhm, yes. Repeating untruths doesn't make them any truer. You confuse the default action of terminating your program with "being screwed", but that is simply not true (even if libX11 makes this hard, other X libs such as the more modern XCB don't). Most X11 apps just exit, but thats not a law. > Not necessarily. Servers do occasionally reload their listen sockets > (SIGHUP handlers); they just don't have logic to do so on demand from a > crazy library. They still have to handle accept errors, though. > >(There are userspace nfs daemons, but even if they use libevent, they are > >far more broken and fragile than the standard nfs daemons in linux). > > Please don't assume I'm a moron. I don't, but you keep making untrue statements that I correct. If you think I assume you are a moron by correcting you, you could just stop making false generic statements such as above and we will not run into any problems :) However, let me repeat that I do not assume you are a moron, and it probably isn't too nice to assime I did. Its certainly not healthy for our otherwise very productive discussion that I would hate to lose :( > would not have said this if I hadn't actually seen libevent-based NFS > code that ships on real systems. > Specifically, I'm talking about the version of rpc.idmapd running by > default on RHEL 5.0. which actually is not a nfs daemon (just like portmap is not an nfs daemon, its a specific service that cna optionally be used by the nfs daemons),. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
Marc Lehmann wrote: On Sun, Nov 04, 2007 at 09:37:52PM -0800, Scott Lamb <[EMAIL PROTECTED]> wrote: There is actually a lot of code that relies on this just working, and the only other event loop I know that has a problem with this is Tk. Ugh, I'd argue that idiom is broken. But if the support's free, I guess it doesn't matter. Well, the idiom is how it works in most languages nowadays, so wether broken or not its important to support. The main problem is that with the current libevent implementation, failures are completely silent. I've used garbage-collected languages without this problem. It's not the language; it's the idiom. This is true, but its an optional feature you don't have to use. In case you wonder, EV, the perl interface to libev, uses this feature. It makes most sense when embedding, of course (not all the world is an .so :). Hmm, in your Perl example, I wouldn't rule out you wanting to share the event loop with some C-based library and being unable to do so. If you want to share it, you of course cannot do that unless that c-based library uses the same version of the code. But this is in no way difefrent to libevent: if one program links it statically and other shared, or if one program uses 1.3d and another 1.3e (different sonames), then you can't share it either. The former is of course the fault of the linking program. The latter is fixed (recently); libevent is now versioning properly. There's definitely nothing you can't do with a void*, so this is all a question of efficiency. Its also a question of coding overhead, which is probably more important than efficiency in this case. Especially as efficieny is a red herring. in the libev proposed api, callbacks get the watcher object passed, which a) is often what you want anyways b) is allocated by user code, so you cna just make the struct larger by appending your data. this solves the efficiency issue and also makes for a simpler API. In fact, I pondered long on wether I even add a void *arg to each event watcher, but I thought the overhead of havign to "subclass" is large in the common case where a single pointer indeed is enough. In any case, let me repeat that this is an optional feature, not a limitation, in the API. It is an invitation for abuse (losing compatibility). I assert that the cost of a sizeof(void*) to point to the relevant part of your structure (which can be nearby...still reasonable cache locality) is not too high. Its higher than necessary, and more complex than neccessary. Higher than necessary? I don't care; it's worth it. More complex than necessary? Not really. All callbacks will be called with EV_ERROR when an error occurs. And yes, if you don't do error handlign and endlessly retry the same operation in a loop, you run into problems. But as that is an obvious programming bug, I don't see any problem here. Hmm. Let me introduce a use case: an event-driven program which must not fail. init or similar. I worked on such a program recently. If it were unreliable, you would have to send the system back to the factory for repair (i.e., flashing new software). On ENOMEM, it would basically sleep and retry. This was quite successful (memory could be temporarily consumed by network buffers, etc, which cleared itself up after a while). This could be solved easier with having a generic callback thats called in such special conditions. The alternative to have every event_add etc. call surrounded by checking code is not realistic, especially if some of the code is not easily changable (such as the libevent code itself which doesn't check for such errors!). For this program, it's important to know more than that an error has occurred. EV_ERROR is totally inadequate. You already know more: you know which watcher caused it, or which watcher is responsible. Erm, no. In the ENOMEM case, I know which fd you decided to kill, though not actually that you've killed it. ENOMEM is not a per-file descriptor problem. What is my program supposed to do? It can't distinguish them, and the correct behavior in each of these conditions is totally different. Also, in the program I'm thinking of, "libev chose to kill this file descriptor" probably means a network link just went down. Ergh. killing the file descriptor might indeed be harsh. all that libev does now is stop the event watcher and signal an error. That's better...in the sense that "we've stopped murdering people; now we only beat them severely" is better. Besides, if you cannot malloc the few bytes ev_once requires you need a *lot* of good error handlign code to continue sensibly. Yes, as I've mentioned above, there are programs for which this level of error handling is necessary. I agree, but libevent already doesn't provide this, so its moot to compare this case to libevent. However, it would be very interetsing to actualyl talk about an API that solves this problem. For example, on ENOMEM, one could have a callbac
Re: Heap-based timer implementation (was Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent)
> also, it seems libevent is exporting all those heap functions (they are > all external symbols). it probably shouldn't. or maybe there is some macro > trick that fixes this, but at leats in my .so all min_ functions are > exported. easier, they are declared static, and I need to study my nm output closer, ignore me, sorry (but I caught it in time :) -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: Heap-based timer implementation (was Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent)
On Mon, Nov 05, 2007 at 01:46:19PM -0500, Nick Mathewson <[EMAIL PROTECTED]> wrote: > In case anybody here isn't watching the commits list [1], Niels > applied a patch from Maxim Yegorushkin to make the timeout > implementation based on a heap, rather than a RB-tree. Thanks, Maxim! also, it seems libevent is exporting all those heap functions (they are all external symbols). it probably shouldn't. or maybe there is some macro trick that fixes this, but at leats in my .so all min_ functions are exported. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] Re: Heap-based timer implementation (benchmark)
On Mon, Nov 05, 2007 at 01:46:19PM -0500, Nick Mathewson <[EMAIL PROTECTED]> wrote: > In case anybody here isn't watching the commits list [1], Niels > applied a patch from Maxim Yegorushkin to make the timeout > implementation based on a heap, rather than a RB-tree. Thanks, Maxim! Thats great news! I just ran a course benchmark against libevent trunk (which has the changes entry for the min_heap code and also references to it in the code). Its the same benchmark, except that I used less repeats and less x-axis sampling points. timers 1.3e vs. libev: http://libev.schmorp.de/dat.t1.png trunk vs. libev: http://data.plan9.de/trunk.t1.png The libev results are virtually the same (ignoring some spikes). It looks as if the libevent growth went down, however: libevevent now has similar growth characteristics as libev (probably due to the heap). Unfortunately, absolute timings are much worse now for libevent, the 1.3e version is definitely faster. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Two questions...
On Mon, Nov 05, 2007 at 02:52:08PM -0500, arthur <[EMAIL PROTECTED]> wrote: > 2. one process (I didn't use thread) can only have a limited number of > fd's -> fork multiple processes to more accept client connection I am not aware of any such limitation. Both per-process (ulimit) and per-system settings (file-max) are easily configurable. > 4. sometime crashing after fork -> move event_init after fork (in other > word, each process needs to event_init itself) Yes :) Easy enough to do if you don't need event management in the parent process. Greetings, -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Sun, Nov 04, 2007 at 09:37:52PM -0800, Scott Lamb <[EMAIL PROTECTED]> wrote: > > There is actually a lot of code that relies on this just working, and the > > only other event loop I know that has a problem with this is Tk. > > Ugh, I'd argue that idiom is broken. But if the support's free, I guess > it doesn't matter. Well, the idiom is how it works in most languages nowadays, so wether broken or not its important to support. The main problem is that with the current libevent implementation, failures are completely silent. > >>> as is required for integration of other event-based libraries, without > >>> having to force the use of some construct around event_loop. > > > > I don't udnerstand that, there is no construct around event_loop, its > > handled > > completely seperately. > > My question was in response to your "I added idle watchers, pid watchers Ah, now I get it :) Well, it requires a construct around event_loop because there is no way to hook into event_loop. Of course, if you have those watcher type you don't need anything around your loop call. Sorry if that was worded confusingly. > > This is true, but its an optional feature you don't have to use. In case > > you wonder, EV, the perl interface to libev, uses this feature. > > > > It makes most sense when embedding, of course (not all the world is an .so > > :). > > Hmm, in your Perl example, I wouldn't rule out you wanting to share the > event loop with some C-based library and being unable to do so. If you want to share it, you of course cannot do that unless that c-based library uses the same version of the code. But this is in no way difefrent to libevent: if one program links it statically and other shared, or if one program uses 1.3d and another 1.3e (different sonames), then you can't share it either. > There's definitely nothing you can't do with a void*, so this is all a > question of efficiency. Its also a question of coding overhead, which is probably more important than efficiency in this case. Especially as efficieny is a red herring. in the libev proposed api, callbacks get the watcher object passed, which a) is often what you want anyways b) is allocated by user code, so you cna just make the struct larger by appending your data. this solves the efficiency issue and also makes for a simpler API. In fact, I pondered long on wether I even add a void *arg to each event watcher, but I thought the overhead of havign to "subclass" is large in the common case where a single pointer indeed is enough. In any case, let me repeat that this is an optional feature, not a limitation, in the API. > I assert that the cost of a sizeof(void*) to > point to the relevant part of your structure (which can be > nearby...still reasonable cache locality) is not too high. Its higher than necessary, and more complex than neccessary. > > All callbacks will be called with EV_ERROR when an error occurs. And yes, > > if you don't do error handlign and endlessly retry the same operation in a > > loop, you run into problems. > > > > But as that is an obvious programming bug, I don't see any problem here. > > Hmm. Let me introduce a use case: an event-driven program which must not > fail. init or similar. > > I worked on such a program recently. If it were unreliable, you would > have to send the system back to the factory for repair (i.e., flashing > new software). On ENOMEM, it would basically sleep and retry. This was > quite successful (memory could be temporarily consumed by network > buffers, etc, which cleared itself up after a while). This could be solved easier with having a generic callback thats called in such special conditions. The alternative to have every event_add etc. call surrounded by checking code is not realistic, especially if some of the code is not easily changable (such as the libevent code itself which doesn't check for such errors!). > For this program, it's important to know more than that an error has > occurred. EV_ERROR is totally inadequate. You already know more: you know which watcher caused it, or which watcher is responsible. > What is my program supposed to do? It can't distinguish them, and the > correct behavior in each of these conditions is totally different. Also, > in the program I'm thinking of, "libev chose to kill this file > descriptor" probably means a network link just went down. Ergh. killing the file descriptor might indeed be harsh. all that libev does now is stop the event watcher and signal an error. > > Besides, if you cannot malloc the few bytes ev_once requires you need a > > *lot* of good error handlign code to continue sensibly. > > Yes, as I've mentioned above, there are programs for which this level of > error handling is necessary. I agree, but libevent already doesn't provide this, so its moot to compare this case to libevent. However, it would be very interetsing to actualyl talk about an API that solves this problem. For example, on ENOMEM, one coul
Re: [Libevent-users] web link
Should be fixed now. On 11/5/07, arthur <[EMAIL PROTECTED]> wrote: > > > Looks the web link in the end of each email is broken. > > http://monkey.org/mailman/listinfo/libevent-users > ___ > Libevent-users mailing list > Libevent-users@monkey.org > http://monkey.org/mailman/listinfo/libevent-users > > ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Mon, Nov 05, 2007 at 09:50:54AM -0800, Scott Lamb <[EMAIL PROTECTED]> wrote: > >leaves me unclear on what happened and no reasonable way to recover (e.g. > >in the case of EBADF in select). > > No, as I said before, libevent retains errno. I've attached a > demonstration program. If it doesn't return 0 when run against your > libevent emulation layer, your code is broken. Libev actually pinpoints the faulty fd and tells the callback, which belongs to the part of the code that actually created the event watcher in the first place. libevent doesn't allow you to do that. so even when errno tells you ebadf, you can do nothing about it, because you cnanot identify the event watcher responsible for it. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Using the sourceforge.net bugtracker.
Hi, all! I've noticed that we're getting a lot more bug reports and patches than can easily be handled by the "mailbox method" of organization. Clearly, we need to do a better job. Fortunately, we have a sourceforge.net project page for Libevent: http://sourceforge.net/projects/levent This page has trackers for bugs, patches, and feature requests. Let's use it! What does this mean in practice? - If you have a bug report, feature request, or patch, and you enter it into the appropriate tracker, that'd be great! It will help make sure that nothing gets forgotten. - If you prefer not to use the sourceforge interface, you don't have to; if you send stuff to me or to the mailing list, I'll try to get it added to the tracker there as soon as possible. - With any luck, this should make libevent development more organized. Some time today or tomorrow, I'm going to go through my backlog in libevent-users, my personal notes, and my "to-do" mailbox, and enter in all the pending libevent bugs, patches, and feature requests that I can find. Please send me an email soon if: - You'd rather enter your bugs/patches/feature ideas yourself. - You don't want me to enter your bugs/patches/feature ideas. I'll send another email when I'm done. I'm also creating a mailing list to get announcements of new tracker entries; I'll include that info in my mail too. many thanks, -- Nick Mathewson pgpr7gY6B7hdJ.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] web link
Looks the web link in the end of each email is broken. http://monkey.org/mailman/listinfo/libevent-users___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Sun, Nov 04, 2007 at 09:47:59PM -0800, Christopher Layne <[EMAIL PROTECTED]> wrote: > > What is my program supposed to do? It can't distinguish them, and the > > correct behavior in each of these conditions is totally different. Also, > > in the program I'm thinking of, "libev chose to kill this file > > descriptor" probably means a network link just went down. Ergh. > > Great point. It should back away and leave things alone - notifying > to the caller (or whoever else is listening) "this is not acceptable to > me - I suggest you fix it - because I won't" (aka unix way). The "caller" in this case is the callback, however, because its the continuation of whatever code requested to watch e.g. a bad fd in the first place. Also, there is no way around an error status for the callback, one simply *must* provide a sensible status to the callback when something goes wrong, because that might be a long time after the watcher was added. So instead of notifying both the caller of the start funktion and the callback later, I don't see why notifying the callback in all cases would be worse, in fact, it lets you have errors easier. And yes, if you don't check for errors in your callback you are doomed. Returning errors to the caller of the event only requires additional checking code, and I have yet to see any code that actively checks for problems while calling event_once or event_add. But most callbacks do the sensible thing when called with EV_ERROR even when they don't care, because in the case of I/O errors they will try to read or write and see it doesn't work. > > No, on error your library may have muddied the water already by screwing > > with the file descriptors. libevent also makes errors clearer by simply > > returning error from the failed function (I'm thinking of event_once() > > vs ev_once() here.) Yes, closing is not a good idea, reporting an error and removing the event form the fd set is enough. However, I still maintain concentrating error handlign in the one place where its likely to be present already as opposed to reporting errors to places where nobody cares (show me the code that catches errors after making event_(once|del|add) calls) is the right thing to do. Wether one reports more detailed errors is then another question. And might be solved as easily as giving errno a defined value to use. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Two questions...
Hi All, Just to share what I did when testing with another project of mine ;-) I am working on a socket proxy/filter on Solaris and here are what my issues and how they are addressed: 1. can't build libevent on Win32 box -> switch to Ubuntu to develop 2. one process (I didn't use thread) can only have a limited number of fd's -> fork multiple processes to more accept client connection 3. need synchronize multiple processes to accept -> create semaphore in main process and accept only after grabbing the semaphore 4. sometime crashing after fork -> move event_init after fork (in other word, each process needs to event_init itself) 5. sometime crashing on Solaris 10 (in evport.c) -> static link to libevent 1.3e (instead of 1.2a so as system installed) Arthur - Original Message - From: "Niels Provos" <[EMAIL PROTECTED]> To: "Marco Bambini" <[EMAIL PROTECTED]> Cc: Sent: Monday, November 05, 2007 10:37 AM Subject: Re: [Libevent-users] Two questions... > On 11/5/07, Marco Bambini <[EMAIL PROTECTED]> wrote: > > - I am interested in using it in a high load server I am writing, > > ideally it should handle some thousands of TCP/IP concurrent > > connections ... actual implementation is one thread per connection > > architecture with a select statement for each thread ... what is the > > best way to develop and high load server with libevent? (just one big > > event list and one thread?) > > That really depends on your model and what kind of architecture your > deploy your system on. Libevent allows you to run one event loop per > thread. > > Niels. > ___ > Libevent-users mailing list > Libevent-users@monkey.org > http://monkey.org/mailman/listinfo/libevent-users ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: Heap-based timer implementation (was Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent)
Nick Mathewson wrote: In case anybody here isn't watching the commits list [1], Niels applied a patch from Maxim Yegorushkin to make the timeout implementation based on a heap, rather than a RB-tree. Thanks, Maxim! This will probably need some testing; if anybody wants to play around with it, just check out the SVN trunk [2] and send in any bugs you run into, or any improvements that we should make. Very nice. I hope to see any other internal improvements from libev make it into the libevent codebase one at a time in patch form. And then, perhaps, API changes considered? I have to admit, I have a few complaints with the libevent API myself. Maybe all API changes should be discussed and you/Niels can determine if it's worth finding a way to make a (hopefully backwards compatible) transition? Best regards, Scott ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Heap-based timer implementation (was Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent)
On Sun, Nov 04, 2007 at 05:19:36PM -0800, Christopher Layne wrote: > On Sun, Nov 04, 2007 at 05:04:25PM -0800, Scott Lamb wrote: > > > * timers are managed by a priority queue (O(1) for important operations > > > as opposed to O(log n) in libevent, also resulting in much simpler > > > code). > > > > In terms of concrete data types, you appear to have used a binary heap? > > So by "important operations" you mean removal, correct? Insertion is > > still O(log n)? The asymptotic behavior is no different, then, as > > insertion happens at least as often as removal. > > Not to put on my O-face, but binary heap insert is *average* > O(1). There should be a performance win for libevent, when it comes > to timer checking, as using a heap will also be O(1) for heap_min() > - which will benefit pending timer calculation code. However, early > delete w/ pending timer will need some rigging or tombstone games. I > wouldn't be surprised that, in a case where one is consistently > resetting timers (think read/write before x time passes) and > re-adding said event, that in the end it will be the same amount of > CPU time. In case anybody here isn't watching the commits list [1], Niels applied a patch from Maxim Yegorushkin to make the timeout implementation based on a heap, rather than a RB-tree. Thanks, Maxim! This will probably need some testing; if anybody wants to play around with it, just check out the SVN trunk [2] and send in any bugs you run into, or any improvements that we should make. [1] The list is [EMAIL PROTECTED] ; go to https://lists.sourceforge.net/lists/listinfo/levent-commits if you want to subscribe. [2] The subversion repository is at https://levent.svn.sourceforge.net/svnroot/levent To check out trunk, make sure you have subversion installed, and type svn co https://levent.svn.sourceforge.net/svnroot/levent/trunk libevent-trunk yrs, -- Nick Mathewson pgph6MPLjfKJi.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.3e srcdir(VPATH) patch
On Mon, Nov 05, 2007 at 10:17:55AM -0800, Niels Provos wrote: > This has been fixed in trunk and is going to make it into the next release. > > Niels. Fixed also in the 1.3 branch now; 1.3f should have the fix. Thanks! -- Nick Mathewson pgpsTQz4cZuPl.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.3e srcdir(VPATH) patch
This has been fixed in trunk and is going to make it into the next release. Niels. On 11/5/07, Kelly Anderson <[EMAIL PROTECTED]> wrote: > Hello, > > I submitted a patch for 1.3b that fixed this. As it didn't get applied, > here it is again for 1.3e. > > Regards > > --- ./Makefile.am.orig 2007-07-30 21:57:55.0 -0600 > +++ ./Makefile.am 2007-11-05 10:46:35.869698466 -0700 > @@ -47,11 +47,11 @@ libevent_la_LDFLAGS = -release @VERSION@ > > include_HEADERS = event.h evhttp.h evdns.h > > -INCLUDES = -Icompat $(SYS_INCLUDES) > +INCLUDES = -I$(srcdir)/compat $(SYS_INCLUDES) > > man_MANS = event.3 evdns.3 > > verify: libevent.la > - cd $(srcdir)/test && make verify > + cd test && make verify > > DISTCLEANFILES = *~ > --- ./sample/Makefile.am.orig 2007-07-30 21:41:03.0 -0600 > +++ ./sample/Makefile.am2007-11-05 10:42:18.248696983 -0700 > @@ -1,7 +1,7 @@ > AUTOMAKE_OPTIONS = foreign no-dependencies > > LDADD = ../libevent.la > -AM_CFLAGS = -I../compat > +AM_CFLAGS = -I$(top_srcdir)/compat > > noinst_PROGRAMS = event-test time-test signal-test > > --- ./test/Makefile.am.orig 2007-08-14 18:24:20.0 -0600 > +++ ./test/Makefile.am 2007-11-05 10:42:18.254697974 -0700 > @@ -1,7 +1,7 @@ > AUTOMAKE_OPTIONS = foreign no-dependencies > > LDADD = ../libevent.la > -CPPFLAGS = -I.. -I../compat > +CPPFLAGS = -I$(top_srcdir) -I$(top_srcdir)/compat > > EXTRA_DIST = regress.rpc > > @@ -17,7 +17,7 @@ regress_SOURCES = regress.c regress.h re > bench_SOURCES = bench.c > > regress.gen.c regress.gen.h: regress.rpc > - ../event_rpcgen.py regress.rpc || echo "No Python installed" > + $(top_srcdir)/event_rpcgen.py regress.rpc || echo "No Python > installed" > > DISTCLEANFILES = *~ > CLEANFILES = regress.gen.h regress.gen.c > > ___ > Libevent-users mailing list > Libevent-users@monkey.org > http://monkey.org/mailman/listinfo/libevent-users > > ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] libevent 1.3e srcdir(VPATH) patch
Hello, I submitted a patch for 1.3b that fixed this. As it didn't get applied, here it is again for 1.3e. Regards --- ./Makefile.am.orig 2007-07-30 21:57:55.0 -0600 +++ ./Makefile.am 2007-11-05 10:46:35.869698466 -0700 @@ -47,11 +47,11 @@ libevent_la_LDFLAGS = -release @VERSION@ include_HEADERS = event.h evhttp.h evdns.h -INCLUDES = -Icompat $(SYS_INCLUDES) +INCLUDES = -I$(srcdir)/compat $(SYS_INCLUDES) man_MANS = event.3 evdns.3 verify: libevent.la - cd $(srcdir)/test && make verify + cd test && make verify DISTCLEANFILES = *~ --- ./sample/Makefile.am.orig 2007-07-30 21:41:03.0 -0600 +++ ./sample/Makefile.am2007-11-05 10:42:18.248696983 -0700 @@ -1,7 +1,7 @@ AUTOMAKE_OPTIONS = foreign no-dependencies LDADD = ../libevent.la -AM_CFLAGS = -I../compat +AM_CFLAGS = -I$(top_srcdir)/compat noinst_PROGRAMS = event-test time-test signal-test --- ./test/Makefile.am.orig 2007-08-14 18:24:20.0 -0600 +++ ./test/Makefile.am 2007-11-05 10:42:18.254697974 -0700 @@ -1,7 +1,7 @@ AUTOMAKE_OPTIONS = foreign no-dependencies LDADD = ../libevent.la -CPPFLAGS = -I.. -I../compat +CPPFLAGS = -I$(top_srcdir) -I$(top_srcdir)/compat EXTRA_DIST = regress.rpc @@ -17,7 +17,7 @@ regress_SOURCES = regress.c regress.h re bench_SOURCES = bench.c regress.gen.c regress.gen.h: regress.rpc - ../event_rpcgen.py regress.rpc || echo "No Python installed" + $(top_srcdir)/event_rpcgen.py regress.rpc || echo "No Python installed" DISTCLEANFILES = *~ CLEANFILES = regress.gen.h regress.gen.c ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
Marc Lehmann wrote:
On Sun, Nov 04, 2007 at 09:56:44PM -0800, Scott Lamb <[EMAIL PROTECTED]> wrote:
returning from event_loop, leaving the app unclear on what has happened
and what to do.
In any case, you can get the same behaviour as libevent by calling unloop
in case of an error, so the interface is strictly more powerful.
Another reason this is undesirable: to get the libevent behavior, I'd
have to have this logic in *every callback* to get the same behavior.
Hmm, one could actually roll this into the libevent compatibility
layer. I'll think about this (didn't occur to me before).
Note, however, that I think the libevent interface is undesirable, as it
leaves me unclear on what happened and no reasonable way to recover (e.g.
in the case of EBADF in select).
No, as I said before, libevent retains errno. I've attached a
demonstration program. If it doesn't return 0 when run against your
libevent emulation layer, your code is broken.
Also, with libevent, you would need similar logic around each call to
event_add/del and so on, because those can all fail in libevent, so its just
a different place, really.
It's not just a different place - it's where the failure happened.
(And I do think I will provide an oom-error handler for this specific
case, as it is about the only generic error that isn't specific to some
watcher).
Thanks for these ideas,
#include
#include
#include
#include
#include
#include
const int BADFD = 64;
static void
cb(int fd, short events, void *cbarg)
{
}
int
main(int argc, char **argv)
{
struct event ev;
int sts;
const char *method;
/* Initialize, hopefully with select method */
putenv("EVENT_NOKQUEUE=yes");
putenv("EVENT_NODEVPOLL=yes");
putenv("EVENT_NOPOLL=yes");
putenv("EVENT_NOEPOLL=yes");
putenv("EVENT_NORTSIG=yes");
putenv("EVENT_NODEVPORT=yes");
event_init();
method = event_get_method();
if (strcmp(method, "select") != 0)
return EXIT_FAILURE;
/* add a bad descriptor */
close(BADFD); /* just to be sure */
event_set(&ev, BADFD, EV_READ|EV_PERSIST, cb, NULL);
sts = event_add(&ev, NULL);
if (sts != 0)
return EXIT_FAILURE;
/* try dispatching */
sts = event_dispatch();
if (sts != 0) {
perror("event_dispatch");
return (errno == EBADF) ? EXIT_SUCCESS : EXIT_FAILURE;
}
return EXIT_FAILURE;
}
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Sun, Nov 04, 2007 at 10:04:05PM -0800, Scott Lamb <[EMAIL PROTECTED]> wrote: > Realistically, I think unit tests and bug reports/workarounds are about > the only reason to blacklist specific event dispatchers. select() sucks, > but well, that's why it's at the bottom of the list, right? There is a hardcoded "priority" list that only incidentally has the same order as the constants, but yes, select comes last, even if its usually faster than poll :) > If you are really concerned about syscall overhead (you mentioned it in > another context, though I don't think it should be a major concern), you > might want to prefer poll() to epoll() or kqueue() if you're going to Yes, I thought about dynamically using select for a small (~50 or so) fd sets as select is faster than epoll in many common small scenarios, but that is mostly an epoll vs. poll issue. For kqueue, I can't quite see the overhead, as kqueue has the same number of syscalls per iteration as select/poll (namely one). the sysclal is more complicated, but is likely faster than poll in all cases (I have zero benchmark data on that, maybe the bsd people fucked it up, but likely, they didn't :) > But that's a situation where you'd want a little more coarseness: low > startup overhead methods vs. low per-socket overhead ones. Resorting to > naming specific event mechanisms seems undesirable in terms of older > programs taking advantage of newer event facilities. It is meant strictly as a benchmark, bug workaround, tuning mechanism. Less being able to programmatically decide which is best but more offering the user a mechanism to influence based on e.g. a config file, so one can configure your mythical big app to use select because it performs better in practise. Choice isn't evil, as long as there is an obvious default choice if you don't care (which is 0/EV_METHOD_AUTO). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Sun, Nov 04, 2007 at 09:56:44PM -0800, Scott Lamb <[EMAIL PROTECTED]> wrote: > > returning from event_loop, leaving the app unclear on what has happened > > and what to do. > > > > In any case, you can get the same behaviour as libevent by calling unloop > > in case of an error, so the interface is strictly more powerful. > > Another reason this is undesirable: to get the libevent behavior, I'd > have to have this logic in *every callback* to get the same behavior. Hmm, one could actually roll this into the libevent compatibility layer. I'll think about this (didn't occur to me before). Note, however, that I think the libevent interface is undesirable, as it leaves me unclear on what happened and no reasonable way to recover (e.g. in the case of EBADF in select). Also, with libevent, you would need similar logic around each call to event_add/del and so on, because those can all fail in libevent, so its just a different place, really. (And I do think I will provide an oom-error handler for this specific case, as it is about the only generic error that isn't specific to some watcher). Thanks for these ideas, -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Two questions...
On 11/5/07, Marco Bambini <[EMAIL PROTECTED]> wrote: > - I am interested in using it in a high load server I am writing, > ideally it should handle some thousands of TCP/IP concurrent > connections ... actual implementation is one thread per connection > architecture with a select statement for each thread ... what is the > best way to develop and high load server with libevent? (just one big > event list and one thread?) That really depends on your model and what kind of architecture your deploy your system on. Libevent allows you to run one event loop per thread. Niels. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Two questions...
On Mon, Nov 05, 2007 at 03:49:40PM +0100, Marco Bambini wrote: > Hi guys, > > I just examined the libevent library and I found it very interesting. > I have two questions > > - what about its license? can I use it in a commercial (not open > source) project? The license is the so-called "3-clause BSD" or "modified BSD" license. You can find a copy at the head of every file. It does not prohibit you from using libevent in commercial projects. All you need to do is follow the terms of the license: basically, this means that you need to include the license and it its accompanying disclaimer as a part of all source and binary distributions, and all documentation accompanying binary distributions. (I am not a lawyer; the summary above is not a substitute for reading the terms of the license.) yrs, -- Nick Mathewson pgpUde8Z376p4.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Two questions...
Hi guys, I just examined the libevent library and I found it very interesting. I have two questions - what about its license? can I use it in a commercial (not open source) project? - I am interested in using it in a high load server I am writing, ideally it should handle some thousands of TCP/IP concurrent connections ... actual implementation is one thread per connection architecture with a select statement for each thread ... what is the best way to develop and high load server with libevent? (just one big event list and one thread?) Thanks a lot in advance for your kind replies. Best regards, --- Marco Bambini http://www.sqlabs.net http://www.sqlabs.net/blog/ http://www.sqlabs.net/realsqlserver/ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
