Re: libev's child reaping breaks system() function
> > It works fine without multithreading, so how is it libev? > > > I also considered this - as I already mentioned. I didn't get any I explaiend the facts to you, and you can look them up easily online too. If you insist on your non-fatc based opinion, that is your right. > The only solution I'm aware of is using a custom SIGCHLD signal Solution to *what*? > handler and activating an async watcher when the signal handler > gets invoked. This is like simulating an ev_signal. > Is this the way I'm supposed to go under these circumstances? Special requirements that rely on unportable behaviour might require special implementations, yes. This is primarily your problem, the best that libev can do is provide you the ability to do it, which is clearly does. > That's true, there are no segfaults. But imagine libev or some other > library would always "ignore" SIGCHLDs, maybe repeatedly to prevent > the user from fixing the handler. waitpid() wouldn't be able to > reap children anymore. This would be documented waitpid() > behaviour, too. If libev gave you the option to do it either way, that would be fine, no? I fail to see your point. It's not as if libev forced this behaviour on you. > I would think that a library with such side-conditions, effectively > breaking a documented system call is poorly designed. You know that this is a lie, so why do you repeat it? All you do here now is trolling: neither waitpid nor system are broken by libev in any way whatsoever, you know that, and repeatedly claiming the opposite is simply not acceptable. If you want to troll, go elsewhere. > > > Making the mechanism asynchronous/non-blocking, would > > > require major code refactoring and make conceptually > > > simple code dis-proportionally harder to read. > > > > Seems like an empty claim - can you back it up? > > Yes. Usually you're using system() for programs that [...] libev doesn't force you to do that, you can also rovide your own signal handler, _as you well know_. Since you can't back up your claim, I'll consider it dropped. > Let's not talk about the problems you would have when > using that mechanism from a thread that hasn't got the > default loop (the only one supporting signal/child > watchers). There are many well known problems with multihreading - you can't fork and do soemthing nontrivial anymore, system, read, close etc. all suddenly have race conditions, signal handlign is order of magnitudes more complex etc. All you do is whine about that and want libev to somehow solve the problem for you. But the problem is multihreading, and that simply requires harder and more code. Whining doesn't help you, you need to sit down and face the reality: if you use threads, your code gets more complicated. libev can't magically fix that for you, and trolling here by making repeated wrong claims is not going to win me over in any way either. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
- Original Message - > From: "Marc Lehmann" > Since you can install your own sigchld handler already, this doesn't > seem > to be so helpful? > I admit, it wouldn't be that useful, but it would be cleaner to use an ev_signal instead of an ev_async+custom signal handler. > > Also system() isn't broken by multithreading - it works fine as > > long as only one thread's executing system() at a time. > > It's the child reaping that breaks it. > > It works fine without multithreading, so how is it libev? > I also considered this - as I already mentioned. I didn't get any feedback from the uClibc people though. But since libev's default child-reaping side-conditions also break waitpid() calls in multithreaded environments, I see the responsibility more on your side. Let's summarize the problem *again*: libev's internally reaps any child process (see childcb() in ev.c). This results in child-reaping race conditions with any concurrent waitpid() call, including system() calls that cannot mask out the signals correctly. If you don't need event-based child termination handling, you can reset SIGCHLD signal handler (presumably to "default" since when "ignoring" the signal, the kernel won't keep zombie-children). But if you still want to use libev for child reaping you can't simply use an ev_signal and do waitpid()s in its handler, e.g. don't reap any child but only children that don't intersect with children reaped concurrently to prevent race conditions. The only solution I'm aware of is using a custom SIGCHLD signal handler and activating an async watcher when the signal handler gets invoked. This is like simulating an ev_signal. Is this the way I'm supposed to go under these circumstances? > So far, you have not explained how libev "breaks" system, btw. On a > naive > implementation, you'd simply get ECHILD, which is documented by POSIX > for > this case, so apparently libev doesn't break anything at all. > > > *any* other waitpid() occurrence by resulting in race > > conditions. waitpid() is even defined as thread-safe. > > The (documented) behaviour of waitpid is not changed by libev though. > That's true, there are no segfaults. But imagine libev or some other library would always "ignore" SIGCHLDs, maybe repeatedly to prevent the user from fixing the handler. waitpid() wouldn't be able to reap children anymore. This would be documented waitpid() behaviour, too. I would think that a library with such side-conditions, effectively breaking a documented system call is poorly designed. > > Making the mechanism asynchronous/non-blocking, would > > require major code refactoring and make conceptually > > simple code dis-proportionally harder to read. > > Seems like an empty claim - can you back it up? Yes. Usually you're using system() for programs that aren't expected to run long enough to cause any trouble. That's why it may well be used in libev watcher handlers. Now if you replace these calls with some implementation that forks and registers an ev_child (you're still interested in the termination status!) you'll get your result in the watcher handler. Since your code might depend on the status, you have to continue in that handler. This complicates otherwise simple code because it cannot be written sequentially anymore. Let's not talk about the problems you would have when using that mechanism from a thread that hasn't got the default loop (the only one supporting signal/child watchers). cheers, Robin -- -- -- managed broadband access -- Travelping GmbH phone: +49-391-8190990 Roentgenstr. 13 fax: +49-391-819099299 D-39108 Magdeburg email: i...@travelping.com GERMANY web: http://www.travelping.com Company Registration: Amtsgericht Stendal Reg No.: HRB 10578 Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780 -- ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
On Fri, Aug 27, 2010 at 12:35:46PM +0200, Robin Haberkorn wrote: > libev can easily do something about it: make child reaping more > flexible by allowing SIGCHLD ev_signals. In addition to doing your own sigchld handler, btw., you can install a handler thats called everytime sigchld gets invoked, and/or a handler that gets called for every child (pid=0). I really think all options are there... -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
On Fri, Aug 27, 2010 at 12:35:46PM +0200, Robin Haberkorn wrote: > libev can easily do something about it: make child reaping more > flexible by allowing SIGCHLD ev_signals. Since you can install your own sigchld handler already, this doesn't seem to be so helpful? > Also system() isn't broken by multithreading - it works fine as > long as only one thread's executing system() at a time. > It's the child reaping that breaks it. It works fine without multithreading, so how is it libev? So far, you have not explained how libev "breaks" system, btw. On a naive implementation, you'd simply get ECHILD, which is documented by POSIX for this case, so apparently libev doesn't break anything at all. > *any* other waitpid() occurrence by resulting in race > conditions. waitpid() is even defined as thread-safe. The (documented) behaviour of waitpid is not changed by libev though. libev doesn't force you to use it's facilities for child reaping. If they are a problem for you, libev fully supports not using them, in a variety of ways - you can even implement your own. > ok, suppose I won't use system() anymore. The thing is if I > would rewrite it on my own, I would have to do it very > similar to the libc implementation. As a result, I would > experience the same kind of race condition. Nobody said multithreading is easy - it "breaks" a lot of very standard functions in subtle ways (for example, using libeio from perl will make theoretically impossible to call fork/system/open etc., but thats a problem with POSIX and/or the implementation, not with libeio). If you look closely, you will find that the "breakage" is a documented feature of POSIX however, so it's not really breaking, just changing the rules. Most systems try to implement POSIX, and some stop there. The problems you experience are caused by the way POSIX works. It might be unexpected, but thats life? > Using an event-based child termination handling while keeping > the system()-like function synchronous/blocking would > introduce an unnecessary synchronization overhead > (like adding ev_childs to the default loop belonging to > another thread and using condition variables) as well Does it? I am not sure. Even if, what could libev do more than it does right now? If you want to use a less efficient reaping mechanism because you think it's overhead is less than whatever other method, then you cna fully do so, without breaking libev. If you don't know how, please read the documentation and ask about anything thats left unclear. > Making the mechanism asynchronous/non-blocking, would > require major code refactoring and make conceptually > simple code dis-proportionally harder to read. Seems like an empty claim - can you back it up? > I don't see why this shouldn't be possible (and as I > pointed out, it is already using a custom signal handler and > async watcher). Well, how? > Is there a design decision that says: either be content > with libev's default child reaping or don't use any event-based > child reaping approach at all? No, not at all, libev allows you to implement any child reaping mechanism you like. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
> > multi-threading itself breaks a lot of stuff (fork for example) in > subtle > ways. this issue is not really something that libev can do something > about. > libev can easily do something about it: make child reaping more flexible by allowing SIGCHLD ev_signals. Also system() isn't broken by multithreading - it works fine as long as only one thread's executing system() at a time. It's the child reaping that breaks it. It will also break *any* other waitpid() occurrence by resulting in race conditions. waitpid() is even defined as thread-safe. > > the solution is to not use system then, just as any other funciton > that > causes problems when threading. > ok, suppose I won't use system() anymore. The thing is if I would rewrite it on my own, I would have to do it very similar to the libc implementation. As a result, I would experience the same kind of race condition. Using an event-based child termination handling while keeping the system()-like function synchronous/blocking would introduce an unnecessary synchronization overhead (like adding ev_childs to the default loop belonging to another thread and using condition variables) as well as other problems. Making the mechanism asynchronous/non-blocking, would require major code refactoring and make conceptually simple code dis-proportionally harder to read. > > That doesn't seem to be pretty much straight-forward. > > IMHO, I could live without the ev_child watcher at all but > > I would be happy with a clean way to use ev_signal to process > > SIGCHLDs. > > Don't use system or multihreading - posix is harsh, and multithreading > imposes rather draconic limitations on a program. not much that can be > doen about it. it's a pain in the ass, but libev can only use what > posix > offers. I don't see why this shouldn't be possible (and as I pointed out, it is already using a custom signal handler and async watcher). Is there a design decision that says: either be content with libev's default child reaping or don't use any event-based child reaping approach at all? cheers, Robin -- -- -- managed broadband access -- Travelping GmbH phone: +49-391-8190990 Roentgenstr. 13 fax: +49-391-819099299 D-39108 Magdeburg email: i...@travelping.com GERMANY web: http://www.travelping.com Company Registration: Amtsgericht Stendal Reg No.: HRB 10578 Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780 -- ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
On Thu, Aug 26, 2010 at 08:02:17PM +0200, Robin Haberkorn wrote: > However in multi-threaded environments, system() cannot multi-threading itself breaks a lot of stuff (fork for example) in subtle ways. this issue is not really something that libev can do something about. the solution is to not use system then, just as any other funciton that causes problems when threading. > That doesn't seem to be pretty much straight-forward. > IMHO, I could live without the ev_child watcher at all but > I would be happy with a clean way to use ev_signal to process > SIGCHLDs. Don't use system or multihreading - posix is harsh, and multithreading imposes rather draconic limitations on a program. not much that can be doen about it. it's a pain in the ass, but libev can only use what posix offers. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
libev's child reaping breaks system() function
Hi! I don't know if that has been brought up before (I couldn't find anything). libev reaps all child processess synchronously in the default event loop. This behaviour can result in child reaping race conditions, generally if waitpid() and in particular if system() (declared in stdlib.h) is also used in the program. In single-threaded programs, this shouldn't cause any problem because of the synchronous reaping. Even if children would be reaped asynchronously, system() would block (mask out) SIGCHLD temporarily. However in multi-threaded environments, system() cannot mask out SIGCHLD for every thread. I'm currently investigating whether this is ok or a bug in itself. At least it happens with both uClibc and glibc which use a similar system() implementation (which call sigprocmask() for masking out SIGCHLD...). Of course it would be possible to reset the signal handler after default loop initialization, but what if you still want to use events for child reaping purposes? It would still be possible to reap only the children you're actually interested in or synchronize the waitpid() and system() calls with a mutex. Unfortunately it doesn't seem to be possible to use an ev_signal for SIGCHLDs, maybe because libev uses an ev_signal for child reaping internally!? So the only way to not break the system() (and waitpid()) calls in a multi-threaded environment seems to be using an async watcher. In pseudo-code: ev_default_init(0); ev_async_init(&async, async_cb); ev_async_start(EV_DEFAULT_UC_ &async); signal(SIGCHLD, chld_hnd); ... void chld_hnd(int s) { ev_async_send(EV_DEFAULT_UC_ &async); } ... void async_cb(EV_P_ ev_async *w, int revents) { // do whatever we would have done in an ev_signal handler } That doesn't seem to be pretty much straight-forward. IMHO, I could live without the ev_child watcher at all but I would be happy with a clean way to use ev_signal to process SIGCHLDs. cheers, Robin -- -- -- managed broadband access -- Travelping GmbH phone: +49-391-8190990 Roentgenstr. 13 fax: +49-391-819099299 D-39108 Magdeburg email: i...@travelping.com GERMANY web: http://www.travelping.com Company Registration: Amtsgericht Stendal Reg No.: HRB 10578 Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780 -- ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev