Re: libev's child reaping breaks system() function
- Original Message - From: Marc Lehmann schm...@schmorp.de Since you can install your own sigchld handler already, this doesn't seem to be so helpful? I admit, it wouldn't be that useful, but it would be cleaner to use an ev_signal instead of an ev_async+custom signal handler. Also system() isn't broken by multithreading - it works fine as long as only one thread's executing system() at a time. It's the child reaping that breaks it. It works fine without multithreading, so how is it libev? I also considered this - as I already mentioned. I didn't get any feedback from the uClibc people though. But since libev's default child-reaping side-conditions also break waitpid() calls in multithreaded environments, I see the responsibility more on your side. Let's summarize the problem *again*: libev's internally reaps any child process (see childcb() in ev.c). This results in child-reaping race conditions with any concurrent waitpid() call, including system() calls that cannot mask out the signals correctly. If you don't need event-based child termination handling, you can reset SIGCHLD signal handler (presumably to default since when ignoring the signal, the kernel won't keep zombie-children). But if you still want to use libev for child reaping you can't simply use an ev_signal and do waitpid()s in its handler, e.g. don't reap any child but only children that don't intersect with children reaped concurrently to prevent race conditions. The only solution I'm aware of is using a custom SIGCHLD signal handler and activating an async watcher when the signal handler gets invoked. This is like simulating an ev_signal. Is this the way I'm supposed to go under these circumstances? So far, you have not explained how libev breaks system, btw. On a naive implementation, you'd simply get ECHILD, which is documented by POSIX for this case, so apparently libev doesn't break anything at all. *any* other waitpid() occurrence by resulting in race conditions. waitpid() is even defined as thread-safe. The (documented) behaviour of waitpid is not changed by libev though. That's true, there are no segfaults. But imagine libev or some other library would always ignore SIGCHLDs, maybe repeatedly to prevent the user from fixing the handler. waitpid() wouldn't be able to reap children anymore. This would be documented waitpid() behaviour, too. I would think that a library with such side-conditions, effectively breaking a documented system call is poorly designed. Making the mechanism asynchronous/non-blocking, would require major code refactoring and make conceptually simple code dis-proportionally harder to read. Seems like an empty claim - can you back it up? Yes. Usually you're using system() for programs that aren't expected to run long enough to cause any trouble. That's why it may well be used in libev watcher handlers. Now if you replace these calls with some implementation that forks and registers an ev_child (you're still interested in the termination status!) you'll get your result in the watcher handler. Since your code might depend on the status, you have to continue in that handler. This complicates otherwise simple code because it cannot be written sequentially anymore. Let's not talk about the problems you would have when using that mechanism from a thread that hasn't got the default loop (the only one supporting signal/child watchers). cheers, Robin -- -- -- managed broadband access -- Travelping GmbH phone: +49-391-8190990 Roentgenstr. 13 fax: +49-391-819099299 D-39108 Magdeburg email: i...@travelping.com GERMANY web: http://www.travelping.com Company Registration: Amtsgericht Stendal Reg No.: HRB 10578 Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780 -- ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
It works fine without multithreading, so how is it libev? I also considered this - as I already mentioned. I didn't get any I explaiend the facts to you, and you can look them up easily online too. If you insist on your non-fatc based opinion, that is your right. The only solution I'm aware of is using a custom SIGCHLD signal Solution to *what*? handler and activating an async watcher when the signal handler gets invoked. This is like simulating an ev_signal. Is this the way I'm supposed to go under these circumstances? Special requirements that rely on unportable behaviour might require special implementations, yes. This is primarily your problem, the best that libev can do is provide you the ability to do it, which is clearly does. That's true, there are no segfaults. But imagine libev or some other library would always ignore SIGCHLDs, maybe repeatedly to prevent the user from fixing the handler. waitpid() wouldn't be able to reap children anymore. This would be documented waitpid() behaviour, too. If libev gave you the option to do it either way, that would be fine, no? I fail to see your point. It's not as if libev forced this behaviour on you. I would think that a library with such side-conditions, effectively breaking a documented system call is poorly designed. You know that this is a lie, so why do you repeat it? All you do here now is trolling: neither waitpid nor system are broken by libev in any way whatsoever, you know that, and repeatedly claiming the opposite is simply not acceptable. If you want to troll, go elsewhere. Making the mechanism asynchronous/non-blocking, would require major code refactoring and make conceptually simple code dis-proportionally harder to read. Seems like an empty claim - can you back it up? Yes. Usually you're using system() for programs that [...] libev doesn't force you to do that, you can also rovide your own signal handler, _as you well know_. Since you can't back up your claim, I'll consider it dropped. Let's not talk about the problems you would have when using that mechanism from a thread that hasn't got the default loop (the only one supporting signal/child watchers). There are many well known problems with multihreading - you can't fork and do soemthing nontrivial anymore, system, read, close etc. all suddenly have race conditions, signal handlign is order of magnitudes more complex etc. All you do is whine about that and want libev to somehow solve the problem for you. But the problem is multihreading, and that simply requires harder and more code. Whining doesn't help you, you need to sit down and face the reality: if you use threads, your code gets more complicated. libev can't magically fix that for you, and trolling here by making repeated wrong claims is not going to win me over in any way either. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
multi-threading itself breaks a lot of stuff (fork for example) in subtle ways. this issue is not really something that libev can do something about. libev can easily do something about it: make child reaping more flexible by allowing SIGCHLD ev_signals. Also system() isn't broken by multithreading - it works fine as long as only one thread's executing system() at a time. It's the child reaping that breaks it. It will also break *any* other waitpid() occurrence by resulting in race conditions. waitpid() is even defined as thread-safe. the solution is to not use system then, just as any other funciton that causes problems when threading. ok, suppose I won't use system() anymore. The thing is if I would rewrite it on my own, I would have to do it very similar to the libc implementation. As a result, I would experience the same kind of race condition. Using an event-based child termination handling while keeping the system()-like function synchronous/blocking would introduce an unnecessary synchronization overhead (like adding ev_childs to the default loop belonging to another thread and using condition variables) as well as other problems. Making the mechanism asynchronous/non-blocking, would require major code refactoring and make conceptually simple code dis-proportionally harder to read. That doesn't seem to be pretty much straight-forward. IMHO, I could live without the ev_child watcher at all but I would be happy with a clean way to use ev_signal to process SIGCHLDs. Don't use system or multihreading - posix is harsh, and multithreading imposes rather draconic limitations on a program. not much that can be doen about it. it's a pain in the ass, but libev can only use what posix offers. I don't see why this shouldn't be possible (and as I pointed out, it is already using a custom signal handler and async watcher). Is there a design decision that says: either be content with libev's default child reaping or don't use any event-based child reaping approach at all? cheers, Robin -- -- -- managed broadband access -- Travelping GmbH phone: +49-391-8190990 Roentgenstr. 13 fax: +49-391-819099299 D-39108 Magdeburg email: i...@travelping.com GERMANY web: http://www.travelping.com Company Registration: Amtsgericht Stendal Reg No.: HRB 10578 Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780 -- ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
On Fri, Aug 27, 2010 at 12:35:46PM +0200, Robin Haberkorn r...@travelping.com wrote: libev can easily do something about it: make child reaping more flexible by allowing SIGCHLD ev_signals. Since you can install your own sigchld handler already, this doesn't seem to be so helpful? Also system() isn't broken by multithreading - it works fine as long as only one thread's executing system() at a time. It's the child reaping that breaks it. It works fine without multithreading, so how is it libev? So far, you have not explained how libev breaks system, btw. On a naive implementation, you'd simply get ECHILD, which is documented by POSIX for this case, so apparently libev doesn't break anything at all. *any* other waitpid() occurrence by resulting in race conditions. waitpid() is even defined as thread-safe. The (documented) behaviour of waitpid is not changed by libev though. libev doesn't force you to use it's facilities for child reaping. If they are a problem for you, libev fully supports not using them, in a variety of ways - you can even implement your own. ok, suppose I won't use system() anymore. The thing is if I would rewrite it on my own, I would have to do it very similar to the libc implementation. As a result, I would experience the same kind of race condition. Nobody said multithreading is easy - it breaks a lot of very standard functions in subtle ways (for example, using libeio from perl will make theoretically impossible to call fork/system/open etc., but thats a problem with POSIX and/or the implementation, not with libeio). If you look closely, you will find that the breakage is a documented feature of POSIX however, so it's not really breaking, just changing the rules. Most systems try to implement POSIX, and some stop there. The problems you experience are caused by the way POSIX works. It might be unexpected, but thats life? Using an event-based child termination handling while keeping the system()-like function synchronous/blocking would introduce an unnecessary synchronization overhead (like adding ev_childs to the default loop belonging to another thread and using condition variables) as well Does it? I am not sure. Even if, what could libev do more than it does right now? If you want to use a less efficient reaping mechanism because you think it's overhead is less than whatever other method, then you cna fully do so, without breaking libev. If you don't know how, please read the documentation and ask about anything thats left unclear. Making the mechanism asynchronous/non-blocking, would require major code refactoring and make conceptually simple code dis-proportionally harder to read. Seems like an empty claim - can you back it up? I don't see why this shouldn't be possible (and as I pointed out, it is already using a custom signal handler and async watcher). Well, how? Is there a design decision that says: either be content with libev's default child reaping or don't use any event-based child reaping approach at all? No, not at all, libev allows you to implement any child reaping mechanism you like. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
On Fri, Aug 27, 2010 at 12:35:46PM +0200, Robin Haberkorn r...@travelping.com wrote: libev can easily do something about it: make child reaping more flexible by allowing SIGCHLD ev_signals. In addition to doing your own sigchld handler, btw., you can install a handler thats called everytime sigchld gets invoked, and/or a handler that gets called for every child (pid=0). I really think all options are there... -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
libev's child reaping breaks system() function
Hi!
I don't know if that has been brought up before (I couldn't
find anything). libev reaps all child processess synchronously
in the default event loop. This behaviour can result in child
reaping race conditions, generally if waitpid() and in
particular if system() (declared in stdlib.h) is also used in
the program.
In single-threaded programs, this shouldn't cause any problem
because of the synchronous reaping. Even if children would
be reaped asynchronously, system() would block (mask out)
SIGCHLD temporarily.
However in multi-threaded environments, system() cannot
mask out SIGCHLD for every thread. I'm currently investigating
whether this is ok or a bug in itself. At least it happens with
both uClibc and glibc which use a similar system()
implementation (which call sigprocmask() for masking out
SIGCHLD...).
Of course it would be possible to reset the signal handler
after default loop initialization, but what if you still want
to use events for child reaping purposes? It would still be
possible to reap only the children you're actually interested
in or synchronize the waitpid() and system() calls with a
mutex. Unfortunately it doesn't seem to be possible to
use an ev_signal for SIGCHLDs, maybe because libev uses an
ev_signal for child reaping internally!?
So the only way to not break the system() (and waitpid())
calls in a multi-threaded environment seems to be using
an async watcher. In pseudo-code:
ev_default_init(0);
ev_async_init(async, async_cb);
ev_async_start(EV_DEFAULT_UC_ async);
signal(SIGCHLD, chld_hnd);
...
void chld_hnd(int s) {
ev_async_send(EV_DEFAULT_UC_ async);
}
...
void async_cb(EV_P_ ev_async *w, int revents) {
// do whatever we would have done in an ev_signal handler
}
That doesn't seem to be pretty much straight-forward.
IMHO, I could live without the ev_child watcher at all but
I would be happy with a clean way to use ev_signal to process
SIGCHLDs.
cheers,
Robin
--
--
-- managed broadband access --
Travelping GmbH phone: +49-391-8190990
Roentgenstr. 13 fax: +49-391-819099299
D-39108 Magdeburg email: i...@travelping.com
GERMANY web: http://www.travelping.com
Company Registration: Amtsgericht Stendal Reg No.: HRB 10578
Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780
--
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Re: libev's child reaping breaks system() function
On Thu, Aug 26, 2010 at 08:02:17PM +0200, Robin Haberkorn r...@travelping.com wrote: However in multi-threaded environments, system() cannot multi-threading itself breaks a lot of stuff (fork for example) in subtle ways. this issue is not really something that libev can do something about. the solution is to not use system then, just as any other funciton that causes problems when threading. That doesn't seem to be pretty much straight-forward. IMHO, I could live without the ev_child watcher at all but I would be happy with a clean way to use ev_signal to process SIGCHLDs. Don't use system or multihreading - posix is harsh, and multithreading imposes rather draconic limitations on a program. not much that can be doen about it. it's a pain in the ass, but libev can only use what posix offers. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
