[Libevent-users] libevent 1.2 varargs bug
Hello. i've been trying to 'make verify' libevent-1.2 on x86-64 SLES9 (gcc 3.3.3, glibc 2.3.5 (20050720) ) system. it just segfaults on regress evbuffer test, i've tracked the issue down to the loop calling vsnprintf() inside evbuffer_add_vprintf() the problem is that you can't call vsnprintf() many times with the same va_list it just segfaults on the second call, the glibc documentation states the following: http://www.gnu.org/software/libc/manual/html_node/Variable-Arguments-Output.html#Variable-Arguments-Output In some other systems, the va_list pointer may become invalid after the call to vprintf, so you must not use va_arg after you call vprintf. Instead, you should call va_end to retire the pointer from service. However, you can safely call va_start on another pointer variable and begin fetching the arguments again through that pointer. Calling vprintf does not destroy the argument list of your function, merely the particular pointer that you passed to it. i suggest using va_copy()-ing ap every time you enter the loop. proposed patch is attached, successfuly tested on x86, x86-64 SLES9. -- Anton Povarov [ [EMAIL PROTECTED] ] [ ICQ: 85431470 ] vprintf_va_args.patch Description: Binary data ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
From: Christopher Baus [EMAIL PROTECTED] Has there been much interest in an IOCP port for libevent? I would certainly be interested. I even started working on a rough port, but as mentioned IOCP is proactive rather than reactive, (ie you tell it how much to read/send and the event tells you that) and it got rather awkward pretty quickly. In libdemux, the event library used by felix (felix.sf.net), I modelled the problem with control blocks (accept, connect, read, write, AcceptEx, ReadFileEx, TransmitFile, ecc) that contained their args, a start method and a finish callback. The control blocks interacted with the event source (select, epoll, iocp, evtports, kq, no signal stuff, setting sig masks is a pain and no aio_* interface yet cos it's not that compelling), associating themselves either via start (unix) or the constructor itself (win32/iocp). It works well, despite the above annoying difference which is probably a wrinkle inherent in the difference between reactive and pro-active. There are threadsafe and non-threadsafe versions of the reactors (which I call demuxers, echoed by asio, so I guess I wasn't completely on drugs when I made that name up) and self pipe classes for those who like to exit multithreaded apps or get attention in a polite manner. I'm not happy with the win32 self pipe as it's not anonymous (anon pipes don't work with IOCPs, socketpair doesn't exist, AF_UNIX is unimplemented, inventing a pair by connecting to localhost raises the hackles of firewall software and so on), hence I need a way of uniquely creating named pipes or to find some flag that limits the scope of the named pipe. Unfortunately, I don't know much about windows. The license is FFAU (free for any use) so if anyone's interested, steal away! Of course I'd be interested in any bug fixes (including understanding the kq workaround) and grand demuxer unifying theorems. Suggestions on how to improve the timer events are also welcome (timer queues on win32 and timed cond var on unix, I think the interface I used in the latter renders it a bit naf). Be aware that the srcs have been made literate and so exist in a single file: lpsrc/flx_demux.pak and require one or two config style macros (existence of various event APIs, sock_addr_len type and other stuff that's easy to guess). The slickest implementation I've seen that works with *nix non-blocking I/O and Windows IOCP is boost ASIO http://asio.sourceforge.net/. Instead of modeling reactive behavior with IOCP, it mimics proactive behavior on *nix. It works out surprisingly well. Wow, that looks very complete and mature, even if the name does strike fear into the hearts of australians. There's even a self pipe class for unix... but I don't see a win32 version. From: William Ahern [EMAIL PROTECTED] I'm privy to my own [C] library which does this and more. Still, I don't think this necessarily belongs in libevent. libevent is popular because it does cross-platform event _polling_ very well, in a clean and simple interface. It polls? I thought the whole point of event interfaces was to avoid polling readiness, preferring instead knowing, however unix style evt interfaces fall down a little in this respect, and offer only suspecting, still, saves CPU. Integrating compound (higher-level) asynchronous actions like this is still a very fluid area. Take the recent DNS support in 1.2. It's probably going to follow the same tired path that c-ares, ADNS and UDNS have... slow and abortive patches for IPv6, SRV, CNAME, et al (granted UDNS has probably done the best out of all of them). Sure, once I'd covered basic socket readiness, file io and so on, I looked to DNS thinking there would be a similar readiness interface. There wasn't, instead existing various blocking implementations where you can either wait or spawn a thread to wait for you. That kind of wrapping seemed trivial and for similar reasons I didn't wrap aio_*, because the impls that I saw seem to be pthread+blocking read/write... In any case, most overlapped=IOCPable stuff in windows seems quite low-level, lower than DNS, which is basically a socket application rather than a fundamental io operation. libevent fills in its niche almost perfectly. If it starts to spill out all over the place things will only get messier; libevent originally made things cleaner. Maybe it's inevitable... I suppose that's how fluid areas become well trodden ground. Covering IOCPs isn't so hard nor polluting with the following interface: unix: demuxer d; control_block_x cb(args); // override finish to customise behaviour d.start(cb); d.wait(); win32/iocp: demuxer d; control_block_y cb(d, args); // override finish to customise behaviour d.start(); d.wait(); RF ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
On Wed, Oct 18, 2006 at 12:04:11AM -0600, Jason Ish wrote: I ran across at least one criticism of libevent gaining too much other than pure event library: http://varnish.projects.linpro.no/wiki/EventLibrary All of these criticisms are partially or wholly invalid. The DNS and HTTP code is effectively only distributed in the tarball; it's not part of the library itself. I don't understand the issue with struct event_base. Obviously it evolved, but I don't understand how the current API is any less useful to a threaded application. The timestamp issue sounds like, it doesn't have my particular pet feature, so the library is misdesigned. First of all, why is anybody depending on timestamps in such a pervasive manner as to want it built into the libevent API, knowing or not knowing that the timestamp may or may not be the particular timestamp the application desires (timestamp on a signal delivered through a pipe flushed at the end of a loop... how is that remotely useful for something seriously depending on a timestamp?). I *have* written code to give back a fixed timestamp per loop, and maybe it would be convenient if I could access libevent's copy, but I wouldn't criticize libevent for that. And where are these bugs and misfeatures in libevent? Personally I don't like the model used for the evbuffer framework, but I wouldn't call it a bug or misfeature. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
On Oct 18, 2006, at 11:49 AM, William Ahern wrote: On Wed, Oct 18, 2006 at 12:04:11AM -0600, Jason Ish wrote: I ran across at least one criticism of libevent gaining too much other than pure event library: http://varnish.projects.linpro.no/wiki/EventLibrary All of these criticisms are partially or wholly invalid. The DNS and HTTP code is effectively only distributed in the tarball; it's not part of the library itself. Presumably they _will_ be in the library, but I don't see the problem, either. I'm using libevent in a resource-constrained system, and typing p4 delete foo.c didn't take much of my time. (By the way, Niels, if you at some point get a 2wire iNID, look for your name in the user manual. :) On a less-constrained system, you can simply not use those parts of the library. I don't understand the issue with struct event_base. Obviously it evolved, but I don't understand how the current API is any less useful to a threaded application. I agree with this part - I don't like the default base concept either. My complaint is simple: it's error-prone. If you forget event_base_set on an event, it's associated with the latest base created. This will probably work *most* of the time. It'd be much less confusing if it consistently broke. But I understand libevent has to keep backward compatibility in mind, and I still haven't decided what I'd actually like in a threaded event library, so I'm not proposing any changes, at least right now. The timestamp issue sounds like, it doesn't have my particular pet feature, so the library is misdesigned. First of all, why is anybody depending on timestamps in such a pervasive manner as to want it built into the libevent API, knowing or not knowing that the timestamp may or may not be the particular timestamp the application desires (timestamp on a signal delivered through a pipe flushed at the end of a loop... how is that remotely useful for something seriously depending on a timestamp?). I *have* written code to give back a fixed timestamp per loop, and maybe it would be convenient if I could access libevent's copy, but I wouldn't criticize libevent for that. I don't know what their complaint is, but I had to modify libevent's timer stuff in my copy to achieve two purposes: 1) abrupt clock sets. (Our clock starts at epoch on boot, then gets set to a real time via SNTP.) There's code in there to handle this, but I didn't understand/trust it, so I removed it in favor of just using CLOCK_MONOTONIC. Unfortunately, it's not that portable - notably, stupid OS X still doesn't support it. Otherwise I'd send in a patch. 2) recurring timers without drift. I made my copy just take absolute uptimes, but you could probably achieve most of the same thing by implementing EV_PERIODIC for timers. -- Scott Lamb http://www.slamb.org/ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
Hi There! Completely agree with you, William. If you need a feature, spend time to add it. cheers Youns William Ahern a crit: On Wed, Oct 18, 2006 at 12:04:11AM -0600, Jason Ish wrote: I ran across at least one criticism of libevent gaining too much other than pure event library: http://varnish.projects.linpro.no/wiki/EventLibrary All of these criticisms are partially or wholly invalid. The DNS and HTTP code is effectively only distributed in the tarball; it's not part of the library itself. I don't understand the issue with struct event_base. Obviously it evolved, but I don't understand how the current API is any less useful to a threaded application. The timestamp issue sounds like, "it doesn't have my particular pet feature, so the library is misdesigned". First of all, why is anybody depending on timestamps in such a pervasive manner as to want it built into the libevent API, knowing or not knowing that the timestamp may or may not be the particular timestamp the application desires (timestamp on a signal delivered through a pipe flushed at the end of a loop... how is that remotely useful for something seriously depending on a timestamp?). I *have* written code to give back a fixed timestamp per loop, and maybe it would be convenient if I could access libevent's copy, but I wouldn't criticize libevent for that. And where are these bugs and misfeatures in libevent? Personally I don't like the model used for the evbuffer framework, but I wouldn't call it a bug or misfeature. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
On Wed, Oct 18, 2006 at 12:11:20PM -0700, Scott Lamb wrote: On Oct 18, 2006, at 11:49 AM, William Ahern wrote: On Wed, Oct 18, 2006 at 12:04:11AM -0600, Jason Ish wrote: I ran across at least one criticism of libevent gaining too much other than pure event library: http://varnish.projects.linpro.no/wiki/EventLibrary All of these criticisms are partially or wholly invalid. The DNS and HTTP code is effectively only distributed in the tarball; it's not part of the library itself. Presumably they _will_ be in the library, but I don't see the problem, either. I'm using libevent in a resource-constrained system, and typing p4 delete foo.c didn't take much of my time. (By the way, Niels, if you at some point get a 2wire iNID, look for your name in the user manual. :) On a less-constrained system, you can simply not use those parts of the library. I've simply wrapped many of the libevent functions to take an event_base as the first parameter. Well, its not really an event base, but my wrapper around one. Then I don't have to worry about setting the base, and more importantly, the people who I have to write code with don't have to worry about it. My original post was not really to bring up the misfeatures that are brought up in the wiki post, but more to query about more 'framework' type features being incorporated into libevent. Albeit a little application specific, I've built up many 'framework' type things around libevent that more or less match a lot of the features found in Twisted (www.twistedmatrix.com) and I find these very useful, but I don't know if they belong in libevent. -- Jason ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
From: William Ahern [EMAIL PROTECTED] On Wed, Oct 18, 2006 at 12:21:17AM -, Christopher Baus wrote: I'm not recommending that this should be in libevent, but if you are looking for something that abstracts out both IOCP and non-blocking sockets, ASIO is a good place to start. I wouldn't write a project in C++ if you paid me a million dollars. ;) Hmm, I suppose, but have you considered that that's a lot of money? RF ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
On 10/17/06, Scott Lamb [EMAIL PROTECTED] wrote: On Oct 17, 2006, at 12:19 PM, Rhythmic Fistman wrote: On 10/17/06, Scott Lamb [EMAIL PROTECTED] wrote: On Oct 17, 2006, at 11:08 AM, Rhythmic Fistman wrote: That's funny, there doesn't seem to be a self-pipe-trick implementation in lib-evt... how do you get by without it? There is, although it uses a socketpair() instead of a pipe(). Ah, I was grepping trick. It looks like it's used internally in the signal style implementations - so how does a multithreaded app get thread_dispatch to return when it's time to exit? libevent doesn't have any interthread communication. But you canlayer stuff on top of it - look through the archives for a coupleposts by Andrew Danforth.Message-ID: [EMAIL PROTECTED]Date: Thu, 5 Oct 2006 21:42:35 -0400From: Andrew Danforth [EMAIL PROTECTED]References: [EMAIL PROTECTED]Subject: [Libevent-users] Re: Passing data between event loops inmultithreaded apps and his code at http://www.weirdness.net/code/libeventI've been thinking that a fancier threading system might performbetter, but I still haven't found the time to write and benchmark different approaches.--Scott Lamb http://www.slamb.org/___Libevent-users mailing list Libevent-users@monkey.orghttp://monkey.org/mailman/listinfo/libevent-users ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
Neato! Seems you've subsumed most OS event waiting interfaces.Does that include win32 IOCPs (io completion ports) yet?If not, they're definitely worth a look. However, they're not very selecty, in that they scale well and they tell you when somethinghas actually happened instead of when something might happen withoutblocking, maybe. I don't know how well that would fit in with libevt. Has there been much interest in an IOCP port for libevent? I would certainly be interested.I even started working on a rough port, but as mentioned IOCP is proactive rather than reactive,(ie you tell it how much to read/send and the event tells you that) and it got rather awkward pretty quickly. It might be worth finding some way to wrap the IOCP structures in a module/class that behaves a little more like standard sockets so it would be easier to integrate. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent 1.2
The slickest implementation I've seen that works with *nix non-blocking I/O and Windows IOCP is boost ASIO http://asio.sourceforge.net/. Instead of modeling reactive behavior with IOCP, it mimics proactive behavior on *nix. It works out surprisingly well. I'm privy to my own [C] library which does this and more. Still, I don't think this necessarily belongs in libevent. I'm not recommending that this should be in libevent, but if you are looking for something that abstracts out both IOCP and non-blocking sockets, ASIO is a good place to start. http://baus.net/ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
