Re: [Libevent-users] libevent 1.2
From: Christopher Baus [EMAIL PROTECTED] Has there been much interest in an IOCP port for libevent? I would certainly be interested. I even started working on a rough port, but as mentioned IOCP is proactive rather than reactive, (ie you tell it how much to read/send and the event tells you that) and it got rather awkward pretty quickly. In libdemux, the event library used by felix (felix.sf.net), I modelled the problem with control blocks (accept, connect, read, write, AcceptEx, ReadFileEx, TransmitFile, ecc) that contained their args, a start method and a finish callback. The control blocks interacted with the event source (select, epoll, iocp, evtports, kq, no signal stuff, setting sig masks is a pain and no aio_* interface yet cos it's not that compelling), associating themselves either via start (unix) or the constructor itself (win32/iocp). It works well, despite the above annoying difference which is probably a wrinkle inherent in the difference between reactive and pro-active. There are threadsafe and non-threadsafe versions of the reactors (which I call demuxers, echoed by asio, so I guess I wasn't completely on drugs when I made that name up) and self pipe classes for those who like to exit multithreaded apps or get attention in a polite manner. I'm not happy with the win32 self pipe as it's not anonymous (anon pipes don't work with IOCPs, socketpair doesn't exist, AF_UNIX is unimplemented, inventing a pair by connecting to localhost raises the hackles of firewall software and so on), hence I need a way of uniquely creating named pipes or to find some flag that limits the scope of the named pipe. Unfortunately, I don't know much about windows. The license is FFAU (free for any use) so if anyone's interested, steal away! Of course I'd be interested in any bug fixes (including understanding the kq workaround) and grand demuxer unifying theorems. Suggestions on how to improve the timer events are also welcome (timer queues on win32 and timed cond var on unix, I think the interface I used in the latter renders it a bit naf). Be aware that the srcs have been made literate and so exist in a single file: lpsrc/flx_demux.pak and require one or two config style macros (existence of various event APIs, sock_addr_len type and other stuff that's easy to guess). The slickest implementation I've seen that works with *nix non-blocking I/O and Windows IOCP is boost ASIO http://asio.sourceforge.net/. Instead of modeling reactive behavior with IOCP, it mimics proactive behavior on *nix. It works out surprisingly well. Wow, that looks very complete and mature, even if the name does strike fear into the hearts of australians. There's even a self pipe class for unix... but I don't see a win32 version. From: William Ahern [EMAIL PROTECTED] I'm privy to my own [C] library which does this and more. Still, I don't think this necessarily belongs in libevent. libevent is popular because it does cross-platform event _polling_ very well, in a clean and simple interface. It polls? I thought the whole point of event interfaces was to avoid polling readiness, preferring instead knowing, however unix style evt interfaces fall down a little in this respect, and offer only suspecting, still, saves CPU. Integrating compound (higher-level) asynchronous actions like this is still a very fluid area. Take the recent DNS support in 1.2. It's probably going to follow the same tired path that c-ares, ADNS and UDNS have... slow and abortive patches for IPv6, SRV, CNAME, et al (granted UDNS has probably done the best out of all of them). Sure, once I'd covered basic socket readiness, file io and so on, I looked to DNS thinking there would be a similar readiness interface. There wasn't, instead existing various blocking implementations where you can either wait or spawn a thread to wait for you. That kind of wrapping seemed trivial and for similar reasons I didn't wrap aio_*, because the impls that I saw seem to be pthread+blocking read/write... In any case, most overlapped=IOCPable stuff in windows seems quite low-level, lower than DNS, which is basically a socket application rather than a fundamental io operation. libevent fills in its niche almost perfectly. If it starts to spill out all over the place things will only get messier; libevent originally made things cleaner. Maybe it's inevitable... I suppose that's how fluid areas become well trodden ground. Covering IOCPs isn't so hard nor polluting with the following interface: unix: demuxer d; control_block_x cb(args); // override finish to customise behaviour d.start(cb); d.wait(); win32/iocp: demuxer d; control_block_y cb(d, args); // override finish to customise behaviour d.start(); d.wait(); RF ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
On Wed, Oct 18, 2006 at 12:04:11AM -0600, Jason Ish wrote: I ran across at least one criticism of libevent gaining too much other than pure event library: http://varnish.projects.linpro.no/wiki/EventLibrary All of these criticisms are partially or wholly invalid. The DNS and HTTP code is effectively only distributed in the tarball; it's not part of the library itself. I don't understand the issue with struct event_base. Obviously it evolved, but I don't understand how the current API is any less useful to a threaded application. The timestamp issue sounds like, it doesn't have my particular pet feature, so the library is misdesigned. First of all, why is anybody depending on timestamps in such a pervasive manner as to want it built into the libevent API, knowing or not knowing that the timestamp may or may not be the particular timestamp the application desires (timestamp on a signal delivered through a pipe flushed at the end of a loop... how is that remotely useful for something seriously depending on a timestamp?). I *have* written code to give back a fixed timestamp per loop, and maybe it would be convenient if I could access libevent's copy, but I wouldn't criticize libevent for that. And where are these bugs and misfeatures in libevent? Personally I don't like the model used for the evbuffer framework, but I wouldn't call it a bug or misfeature. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
On Oct 18, 2006, at 11:49 AM, William Ahern wrote: On Wed, Oct 18, 2006 at 12:04:11AM -0600, Jason Ish wrote: I ran across at least one criticism of libevent gaining too much other than pure event library: http://varnish.projects.linpro.no/wiki/EventLibrary All of these criticisms are partially or wholly invalid. The DNS and HTTP code is effectively only distributed in the tarball; it's not part of the library itself. Presumably they _will_ be in the library, but I don't see the problem, either. I'm using libevent in a resource-constrained system, and typing p4 delete foo.c didn't take much of my time. (By the way, Niels, if you at some point get a 2wire iNID, look for your name in the user manual. :) On a less-constrained system, you can simply not use those parts of the library. I don't understand the issue with struct event_base. Obviously it evolved, but I don't understand how the current API is any less useful to a threaded application. I agree with this part - I don't like the default base concept either. My complaint is simple: it's error-prone. If you forget event_base_set on an event, it's associated with the latest base created. This will probably work *most* of the time. It'd be much less confusing if it consistently broke. But I understand libevent has to keep backward compatibility in mind, and I still haven't decided what I'd actually like in a threaded event library, so I'm not proposing any changes, at least right now. The timestamp issue sounds like, it doesn't have my particular pet feature, so the library is misdesigned. First of all, why is anybody depending on timestamps in such a pervasive manner as to want it built into the libevent API, knowing or not knowing that the timestamp may or may not be the particular timestamp the application desires (timestamp on a signal delivered through a pipe flushed at the end of a loop... how is that remotely useful for something seriously depending on a timestamp?). I *have* written code to give back a fixed timestamp per loop, and maybe it would be convenient if I could access libevent's copy, but I wouldn't criticize libevent for that. I don't know what their complaint is, but I had to modify libevent's timer stuff in my copy to achieve two purposes: 1) abrupt clock sets. (Our clock starts at epoch on boot, then gets set to a real time via SNTP.) There's code in there to handle this, but I didn't understand/trust it, so I removed it in favor of just using CLOCK_MONOTONIC. Unfortunately, it's not that portable - notably, stupid OS X still doesn't support it. Otherwise I'd send in a patch. 2) recurring timers without drift. I made my copy just take absolute uptimes, but you could probably achieve most of the same thing by implementing EV_PERIODIC for timers. -- Scott Lamb http://www.slamb.org/ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
Hi There! Completely agree with you, William. If you need a feature, spend time to add it. cheers Youns William Ahern a crit: On Wed, Oct 18, 2006 at 12:04:11AM -0600, Jason Ish wrote: I ran across at least one criticism of libevent gaining too much other than pure event library: http://varnish.projects.linpro.no/wiki/EventLibrary All of these criticisms are partially or wholly invalid. The DNS and HTTP code is effectively only distributed in the tarball; it's not part of the library itself. I don't understand the issue with struct event_base. Obviously it evolved, but I don't understand how the current API is any less useful to a threaded application. The timestamp issue sounds like, "it doesn't have my particular pet feature, so the library is misdesigned". First of all, why is anybody depending on timestamps in such a pervasive manner as to want it built into the libevent API, knowing or not knowing that the timestamp may or may not be the particular timestamp the application desires (timestamp on a signal delivered through a pipe flushed at the end of a loop... how is that remotely useful for something seriously depending on a timestamp?). I *have* written code to give back a fixed timestamp per loop, and maybe it would be convenient if I could access libevent's copy, but I wouldn't criticize libevent for that. And where are these bugs and misfeatures in libevent? Personally I don't like the model used for the evbuffer framework, but I wouldn't call it a bug or misfeature. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
On Wed, Oct 18, 2006 at 12:11:20PM -0700, Scott Lamb wrote: On Oct 18, 2006, at 11:49 AM, William Ahern wrote: On Wed, Oct 18, 2006 at 12:04:11AM -0600, Jason Ish wrote: I ran across at least one criticism of libevent gaining too much other than pure event library: http://varnish.projects.linpro.no/wiki/EventLibrary All of these criticisms are partially or wholly invalid. The DNS and HTTP code is effectively only distributed in the tarball; it's not part of the library itself. Presumably they _will_ be in the library, but I don't see the problem, either. I'm using libevent in a resource-constrained system, and typing p4 delete foo.c didn't take much of my time. (By the way, Niels, if you at some point get a 2wire iNID, look for your name in the user manual. :) On a less-constrained system, you can simply not use those parts of the library. I've simply wrapped many of the libevent functions to take an event_base as the first parameter. Well, its not really an event base, but my wrapper around one. Then I don't have to worry about setting the base, and more importantly, the people who I have to write code with don't have to worry about it. My original post was not really to bring up the misfeatures that are brought up in the wiki post, but more to query about more 'framework' type features being incorporated into libevent. Albeit a little application specific, I've built up many 'framework' type things around libevent that more or less match a lot of the features found in Twisted (www.twistedmatrix.com) and I find these very useful, but I don't know if they belong in libevent. -- Jason ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
From: William Ahern [EMAIL PROTECTED] On Wed, Oct 18, 2006 at 12:21:17AM -, Christopher Baus wrote: I'm not recommending that this should be in libevent, but if you are looking for something that abstracts out both IOCP and non-blocking sockets, ASIO is a good place to start. I wouldn't write a project in C++ if you paid me a million dollars. ;) Hmm, I suppose, but have you considered that that's a lot of money? RF ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Patches and complaints
Niels, I sent a few patches to the list and got no reply from you - I think you were on vacation. Here they are again, slightly cleaned up. One- line summary here; full description at the top of the attached files. * event-once-leak.patch - fixes event_once() leak on event_add() failure. * event-poll-leak.patch - fixes realloc() leak in poll() code * event-signal.patch - fixes a SIGCHLD problem reported by Anton Povarov [EMAIL PROTECTED] test/regress as of libevent 1.2 segfaults for me (Linux/x86_64) in the evbuffer stuff. I haven't done much to diagnose, since I don't use it. But here's something. After turning off -O and turning on USE_DEBUG: $ LD_LIBRARY_PATH=.libs gdb test/.libs/regress ... (gdb) run ... Evbuffer: Program received signal SIGSEGV, Segmentation fault. 0x003f12173c90 in ?? () from /lib64/libc.so.6 (gdb) bt #0 0x003f12173c90 in ?? () from /lib64/libc.so.6 #1 0x003f1214488b in _IO_vfprintf_internal (s=0x7fff44567930, format=Variable format is not available. ) at vfprintf.c:1559 #2 0x003f121652fa in _IO_vsnprintf (string=0x511310 , maxlen=Variable maxlen is not available. ) at vsnprintf.c:120 #3 0x2aab4aaf in evbuffer_add_vprintf (buf=0x511160, fmt=0x406347 %s/%d, ap=0x7fff44567b00) at buffer.c:144 #4 0x2aab4c33 in evbuffer_add_printf (buf=0x511160, fmt=0x406347 %s/%d) at buffer.c:168 #5 0x00402f15 in test_evbuffer () at regress.c:492 #6 0x00403c05 in main (argc=1, argv=0x7fff44567d18) at regress.c:905 (gdb) A couple other complaints: * CVS seems out of date - it doesn't have evdns.c, and the http tests fail. (They work for me in libevent 1.2) * evhttp.c doesn't compile for me with USE_DEBUG in either current CVS or libevent 1.2. Looks like some data structures have changed since the event_debug()s were written. Best regards, Scott -- Scott Lamb http://www.slamb.org/ event-once-leak.patch Description: Binary data event-poll-leak.patch Description: Binary data event-signal.patch Description: Binary data ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users