Re: Bug in PoCo::Child when PID wraps round
Hi Erick,
I work with Tony. I have a new patch now which addresses this issue,
as well as another leak: calling $self-wheel($id) will initialize a
key in $self-{$PKG}{wheels}{$id}, causing a leak if the wheel had
already finished or if it never existed.
Cheers,
Ricky
=== BEGIN PATCH ===
*** Child.pm2009-06-02 16:10:53.120660674 +0200
--- Child.pm.patched2009-05-28 16:38:08.687685140 +0200
***
*** 265,279
# clean up
! delete $self-{$PKG}{wheels}{$id};
! delete $self-{$PKG}{pids}{$id};
# all expiring children should issue a done except when
# the return code is non-zero which indicates a failure
# if the caller asked we quit, fire a done regardless
# of the child's return code value (we might have hard killed)
! my $event = ($self-{$PKG}{wheels}{$id}{quit} || $rc == 0)
? done : died
;
$self-callback($event, { wheel = $id, rc = $rc });
--- 265,281
# clean up
! delete $self-{$PKG}{pids}{$self-{$PKG}{wheels}{$id}{ref}-PID};
! my $wheel = delete $self-{$PKG}{wheels}{$id};
! delete $self-{$PKG}{SIGCHLD}{$id};
! delete $self-{$PKG}{CLOSED}{$id};
# all expiring children should issue a done except when
# the return code is non-zero which indicates a failure
# if the caller asked we quit, fire a done regardless
# of the child's return code value (we might have hard killed)
! my $event = ($wheel-{quit} || $rc == 0)
? done : died
;
$self-callback($event, { wheel = $id, rc = $rc });
***
*** 306,311
--- 308,314
sub wheel {
my $self = shift;
my $id = shift || $self-{$PKG}{wheels}{current};
+ return undef unless exists $self-{$PKG}{wheels}{$id};
$self-{$PKG}{wheels}{$id}{ref};
}
=== END PATCH ===
On Jun 4, 2009, at 7:30 PM, Erick Calder wrote:
sorry, I still own it. I've just been disconnected from the world
a long
time. it will take me a while to get back to you on this but I will
From: Tony Wildish wild...@mail.cern.ch
Date: Thu, 7 May 2009 10:58:12 +0200
To: poe@perl.org
Subject: Re: Bug in PoCo::Child when PID wraps round (fwd)
Hi,
just checking one more time: is anyone looking after PoCo::Child,
or is
it orphaned?
TIA,
Tony.
On Wed, 15 Apr 2009, Tony Wildish wrote:
Hi,
I sent this to the bug-poe-component-ch...@rt.cpan.org list some
time ago
but it looks like that list is dead. Is there someone here managing
PoCo::Child?
Cheers,
Tony.
-- Forwarded message --
Date: Tue, 24 Mar 2009 12:56:33 +0100 (CET)
From: Tony Wildish wild...@mail.cern.ch
To: bug-poe-component-ch...@rt.cpan.org
Subject: Bug in PoCo::Child when PID wraps round
Hi,
we have been using PoCo::Child in a project for a while now and
have just
uncovered a bug. The internal bookkeeping for the association of
wheelIDs
and PIDs is wrong, with the result that when the OS PID wraps
round, the
wrong wheelID will be given to the 'done' event.
I'm using PoCo::Child version 1.39, perl 5.8.5, kernel
2.6.9-78.0.8.EL.cernsmp.
You can see the bug by inspection of the code. At line 192 we have:
my $id = $wheel-ID;
$self-debug(qq/run(): @$cmd, wheel=$id, pid=/ . $wheel-
PID);
$self-{$PKG}{pids}{$wheel-PID} = $id;
so the {pids} are inserted keyed on $wheel-PID. But at line
268, the
clean up deletes as follows:
delete $self-{$PKG}{wheels}{$id};
delete $self-{$PKG}{pids}{$id};
so it's deleting keyed on wheel-ID, not $wheel-PID.
When the PIDs wrap round, the protection in sig_child is broken
by this:
sub sig_child {
my ($kernel, $self, $pid, $rc) = @_[KERNEL, OBJECT, ARG1, ARG2];
my $id = $self-{$PKG}{pids}{$pid} || ;
# child death signals are issued by the OS and sent to all
# sessions; we want to handle only our own children
return unless $id;
Because the {pids} hash had the wrong entry deleted in the
cleanup, when
the PIDs wrap, an old $id will be found where none should be. So
sig_child
will not return here, and will cause 'done' to be fired with the old
wheelID, which is plainly wrong.
Also, the cleanup around line 268/269 should remove {CLOSED} and
{SIGCHLD} entries for the wheel too. That's less serious because
it won't
cause a problem until wheel-IDs wrap round, which will take a lot
longer,
but it's still a leak.
Cheers,
Tony.
smime.p7s
Description: S/MIME cryptographic signature
Re: Bug in PoCo::Child when PID wraps round
sorry, I still own it. I've just been disconnected from the world a long
time. it will take me a while to get back to you on this but I will
From: Tony Wildish wild...@mail.cern.ch
Date: Thu, 7 May 2009 10:58:12 +0200
To: poe@perl.org
Subject: Re: Bug in PoCo::Child when PID wraps round (fwd)
Hi,
just checking one more time: is anyone looking after PoCo::Child, or is
it orphaned?
TIA,
Tony.
On Wed, 15 Apr 2009, Tony Wildish wrote:
Hi,
I sent this to the bug-poe-component-ch...@rt.cpan.org list some time ago
but it looks like that list is dead. Is there someone here managing
PoCo::Child?
Cheers,
Tony.
-- Forwarded message --
Date: Tue, 24 Mar 2009 12:56:33 +0100 (CET)
From: Tony Wildish wild...@mail.cern.ch
To: bug-poe-component-ch...@rt.cpan.org
Subject: Bug in PoCo::Child when PID wraps round
Hi,
we have been using PoCo::Child in a project for a while now and have just
uncovered a bug. The internal bookkeeping for the association of wheelIDs
and PIDs is wrong, with the result that when the OS PID wraps round, the
wrong wheelID will be given to the 'done' event.
I'm using PoCo::Child version 1.39, perl 5.8.5, kernel
2.6.9-78.0.8.EL.cernsmp.
You can see the bug by inspection of the code. At line 192 we have:
my $id = $wheel-ID;
$self-debug(qq/run(): @$cmd, wheel=$id, pid=/ . $wheel-PID);
$self-{$PKG}{pids}{$wheel-PID} = $id;
so the {pids} are inserted keyed on $wheel-PID. But at line 268, the
clean up deletes as follows:
delete $self-{$PKG}{wheels}{$id};
delete $self-{$PKG}{pids}{$id};
so it's deleting keyed on wheel-ID, not $wheel-PID.
When the PIDs wrap round, the protection in sig_child is broken by this:
sub sig_child {
my ($kernel, $self, $pid, $rc) = @_[KERNEL, OBJECT, ARG1, ARG2];
my $id = $self-{$PKG}{pids}{$pid} || ;
# child death signals are issued by the OS and sent to all
# sessions; we want to handle only our own children
return unless $id;
Because the {pids} hash had the wrong entry deleted in the cleanup, when
the PIDs wrap, an old $id will be found where none should be. So sig_child
will not return here, and will cause 'done' to be fired with the old
wheelID, which is plainly wrong.
Also, the cleanup around line 268/269 should remove {CLOSED} and
{SIGCHLD} entries for the wheel too. That's less serious because it won't
cause a problem until wheel-IDs wrap round, which will take a lot longer,
but it's still a leak.
Cheers,
Tony.
Re: Bug in PoCo::Child when PID wraps round (fwd)
Hi,
just checking one more time: is anyone looking after PoCo::Child, or is
it orphaned?
TIA,
Tony.
On Wed, 15 Apr 2009, Tony Wildish wrote:
Hi,
I sent this to the bug-poe-component-ch...@rt.cpan.org list some time ago
but it looks like that list is dead. Is there someone here managing
PoCo::Child?
Cheers,
Tony.
-- Forwarded message --
Date: Tue, 24 Mar 2009 12:56:33 +0100 (CET)
From: Tony Wildish wild...@mail.cern.ch
To: bug-poe-component-ch...@rt.cpan.org
Subject: Bug in PoCo::Child when PID wraps round
Hi,
we have been using PoCo::Child in a project for a while now and have just
uncovered a bug. The internal bookkeeping for the association of wheelIDs
and PIDs is wrong, with the result that when the OS PID wraps round, the
wrong wheelID will be given to the 'done' event.
I'm using PoCo::Child version 1.39, perl 5.8.5, kernel
2.6.9-78.0.8.EL.cernsmp.
You can see the bug by inspection of the code. At line 192 we have:
my $id = $wheel-ID;
$self-debug(qq/run(): @$cmd, wheel=$id, pid=/ . $wheel-PID);
$self-{$PKG}{pids}{$wheel-PID} = $id;
so the {pids} are inserted keyed on $wheel-PID. But at line 268, the
clean up deletes as follows:
delete $self-{$PKG}{wheels}{$id};
delete $self-{$PKG}{pids}{$id};
so it's deleting keyed on wheel-ID, not $wheel-PID.
When the PIDs wrap round, the protection in sig_child is broken by this:
sub sig_child {
my ($kernel, $self, $pid, $rc) = @_[KERNEL, OBJECT, ARG1, ARG2];
my $id = $self-{$PKG}{pids}{$pid} || ;
# child death signals are issued by the OS and sent to all
# sessions; we want to handle only our own children
return unless $id;
Because the {pids} hash had the wrong entry deleted in the cleanup, when
the PIDs wrap, an old $id will be found where none should be. So sig_child
will not return here, and will cause 'done' to be fired with the old
wheelID, which is plainly wrong.
Also, the cleanup around line 268/269 should remove {CLOSED} and
{SIGCHLD} entries for the wheel too. That's less serious because it won't
cause a problem until wheel-IDs wrap round, which will take a lot longer,
but it's still a leak.
Cheers,
Tony.
Re: Buffering in PoCo::Child
You'll need to patch it. POE::Wheel::ReadWrite wasn't originally designed for unbuffered writes. There's a patch to add it, but I haven't included it because it fails one of POE's tests. You can find the patch in POE's queue at rt.cpan.org. -- Rocco Caputo - [EMAIL PROTECTED] put() calls aren't buffered in filters. They are however buffered in the driver, and they're flushed On Dec 14, 2006, at 10:26, Mike Schroeder wrote: We have a POE app that reads multiple input files, reformats them and the posts them to various binaries wrapped with PoCo::Child that in turn load the data into our underlying system. The problem we are running into is that PoCo::Child seems to be buffering two or three records at a time. Sometimes we have interdependencies between records going into various loaders, so we need these to be unbuffered. In looking at PoCo::Child, I see how it uses POE::Wheel::Run, which in turn uses POE::Driver::SysRW and that Driver::SysRW has a flush () method. Do I need to patch PoCo::Child's write() to flush() after each write ()? Or is there some setting or a better way to get unbuffered writes to PoCo::Child? Thanks in advance for any advice.
Buffering in PoCo::Child
We have a POE app that reads multiple input files, reformats them and the posts them to various binaries wrapped with PoCo::Child that in turn load the data into our underlying system. The problem we are running into is that PoCo::Child seems to be buffering two or three records at a time. Sometimes we have interdependencies between records going into various loaders, so we need these to be unbuffered. In looking at PoCo::Child, I see how it uses POE::Wheel::Run, which in turn uses POE::Driver::SysRW and that Driver::SysRW has a flush() method. Do I need to patch PoCo::Child's write() to flush() after each write()? Or is there some setting or a better way to get unbuffered writes to PoCo::Child? Thanks in advance for any advice.
Re: Buffering in PoCo::Child
It really depends on how you are processing the records and sending them upstream. Could it be your Filter is doing the buffering? On 12/14/06, Mike Schroeder [EMAIL PROTECTED] wrote: We have a POE app that reads multiple input files, reformats them and the posts them to various binaries wrapped with PoCo::Child that in turn load the data into our underlying system. The problem we are running into is that PoCo::Child seems to be buffering two or three records at a time. Sometimes we have interdependencies between records going into various loaders, so we need these to be unbuffered. In looking at PoCo::Child, I see how it uses POE::Wheel::Run, which in turn uses POE::Driver::SysRW and that Driver::SysRW has a flush() method. Do I need to patch PoCo::Child's write() to flush() after each write()? Or is there some setting or a better way to get unbuffered writes to PoCo::Child? Thanks in advance for any advice.
PoCo::Child
with my apologies to everyone who has waited so long for this release, version 1.39 of this module has just been posted to CPAN. it fixes a number of issues, see the Changes file. in the coming year I hope to have more time to devote to POE (at present I'm working 60+ hrs/wk which leaves little time for anything else), specifically to release a series of modules that implement a p2p network I built a while back. until then, Happy New Year's everyone! - ekkis
Re: PoCo::Child
From: Mike Schroeder [EMAIL PROTECTED] Organization: DonorWare LLC Date: Tue, 14 Dec 2004 16:12:31 -0700 To: [EMAIL PROTECTED] Subject: PoCo::Child The PoCo::Child examples all use ls as the child program. Mike, for examples of how to read/write, see the PoCo::Player components e.g. Mp3, Enc, Slideshow. However, I'm not sure how to get rid of the thing! When I am done sending info to the STDIN, I send a $c-quit(), (and checking the process list the process is indeed gone), but something seems to be hanging on -- POE won't wrap up and clean up after itself -- it just hangs. I tried using PoCo::DebugShell, but PoCo::DebugShell and PoCo::Child seem to fight over STDOUT: this likely has to do with the fact that ::Child is not closing the pipe... Philip Gwyn sent me a patch months ago but only now am I getting around to applying it. I should have something ready for release soon. will post to this list then. - ekkis
Re: PoCo::Child
Erick Calder wrote: Mike, for examples of how to read/write, see the PoCo::Player components e.g. Mp3, Enc, Slideshow. Thanks Erick -- that's exactly what I did and got it working for the most part. However, I'm not sure how to get rid of the thing! When I am done sending info to the STDIN, I send a $c-quit(), (and checking the process list the process is indeed gone), but something seems to be hanging on -- POE won't wrap up and clean up after itself -- it just hangs. I tried using PoCo::DebugShell, but PoCo::DebugShell and PoCo::Child seem to fight over STDOUT: this likely has to do with the fact that ::Child is not closing the pipe... Philip Gwyn sent me a patch months ago but only now am I getting around to applying it. I should have something ready for release soon. will post to this list then. Great to hear -- this is the only remaining issue for me with the great component. Regards, Mike.
PoCo::Child
The PoCo::Child examples all use ls as the child program. This is fine for showing how to work with a child that wants some command line arguments to generate STDOUT, but I need a solution that lets me write to STDIN and respond to STDOUT. So I reworked the example to use nl to accept some $c-write() commands and add a line number -- with a little tweaking (had to use conduit = 'pty' instead of pipe), this works quite well. However, I'm not sure how to get rid of the thing! When I am done sending info to the STDIN, I send a $c-quit(), (and checking the process list the process is indeed gone), but something seems to be hanging on -- POE won't wrap up and clean up after itself -- it just hangs. I tried using PoCo::DebugShell, but PoCo::DebugShell and PoCo::Child seem to fight over STDOUT: Can't locate object method OPEN via package POE::Component::DebugShell::Output (perhaps you forgot to load POE::Component::DebugShell::Output?) at /usr/lib/perl5/site_perl/5.6.1/POE/Wheel/Run.pm line 379. Any suggestions on where to go next?
RE: PoCo::Child
It didn't install for me: ok 1 - use PoCo::Child ok 2 - session created ok 3 - read-only client started ok 4 - got stdout ok 5 - second instance ok 6 - got stderr ok 7 - done ok 8 - hard callbacks ok 9 - interactive client started ok 10 - client write ok 11 - quit tested ok 12 - client restarted not ok 13 - client died! []: # Failed test (test.pl at line 128) not ok 14 - client died! []: # Failed test (test.pl at line 128) ok 15 - killed ok 16 - all tests successful # Looks like you planned 14 tests but ran 2 extra. make: *** [test_dynamic] Error 4 /usr/bin/make test -- NOT OK Running make install make test had returned bad status, won't install without force What can I check? It's Linux, Redhat 7.1. L8r, Rob -Original Message- From: Erick Calder [mailto:[EMAIL PROTECTED]] Sent: Wednesday, September 18, 2002 9:49 PM To: [EMAIL PROTECTED] Subject: PoCo::Child Rocco thought it'd be useful to have a component made out of POE::Wheel::Run so as to eliminate having to create an interim session just to receive events from the wheel... so I've put one together under the name in the subject header. It should be available on CPAN or you can pick up a copy from: http://perl.arix.com hope you find it useful. any patches can be sent to [EMAIL PROTECTED] I can be found on EFNet's #poe as ekkis. - e the pain of living life this way, must take its toll on you someday... - Paradise Lost
RE: PoCo::Child
FYI, the link on perl.arix.com seems to be broken: http://perl.arix.com/POE-Component-Child-1.7.tar.gz thx. I've fixed it. -Original Message- From: Peter Chen [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 19, 2002 7:11 AM To: Erick Calder Cc: [EMAIL PROTECTED] Subject: Re: PoCo::Child On Thu, 2002-09-19 at 00:49, Erick Calder wrote: Rocco thought it'd be useful to have a component made out of POE::Wheel::Run so as to eliminate having to create an interim session just to receive events from the wheel... so I've put one together under the name in the subject header. It should be available on CPAN or you can pick up a copy from: http://perl.arix.com hope you find it useful. That's great. I found it on CPAN http://search.cpan.org/author/ECALDER/POE-Component-Child-1.7/Child.pm FYI, the link on perl.arix.com seems to be broken: http://perl.arix.com/POE-Component-Child-1.7.tar.gz Pete
