Poe Design Question
I have built a POE app which forks children, executes some work, returns status back up the parent. All works fine. Currently this is all running from one Session. I now have a requirement to execute jobs which connect to the same modem bank together. I was thinking I might structure it in the following way:- PARENT - Create Session + POE::Wheel::Run (PWR) one process per all jobs which need to connect to modem bank CHILD 1 - Create Session + PWR one process per job CHILD n - Create Session + PWR one process per job This way I can use the heap of each session to manage the child jobs which are connected to each modem bank When using this mechanism to control how many jobs are passed to PWR its dependent on the heap content which in the original approach would contain all jobs :- while (keys(%{$heap->{task}}) < $PARALLELISM ) { Is there a better/simpler way someone can think of to use PWR and the Session Heap to continue to execute jobs against different devices whilst always maintaining 4 simultaneous jobs in each PWR (until complete) Simon Currently start and manage starting tasks this way:- sub StartTasks { =Head3 _start Function called when POE wants to start tasks Shouldnt start any more than $self->GetBSHParalleism($SDU::BatchSectionsHandler::SectionName) When it does it spawns children Event handlers allow information to be passed back to the parent process Mapping between state and function is done in the object states line in the POE::Session->create in _IterateChildrenParallel =cut my ($kernel, $self, $heap) = @_[KERNEL, OBJECT]; my ($child, $task); $self->{_logger}->debug("Enter/Reentry"); while (keys(%{$heap->{task}}) < $PARALLELISM ) { if (ref($self->{_children}) eq "ARRAY"){ $child = shift @{$self->{_children}}; last unless defined $child; $self->{_logger}->debug("ARRAY "."Child = $child"); $task = POE::Wheel::Run->new( Program => sub {local *__ANON__ = "_StartTasksProgramSub"; $self->_CreateChild($child); }, StdoutEvent => "TaskResult", StdoutFilter => POE::Filter::Reference->new(), StderrEvent => "TaskError", CloseEvent => "TaskDone", ); $heap->{task}->{$task->ID} = $task; $self->{_logger}->info("Created child Task/PID ".$task->ID."/".$task->PID); $_[KERNEL]->sig_child($task->PID, 'sig_child'); # Wheel events include the wheel's ID. $_[HEAP]{children_by_wid}{$task->ID} = $task; # Signal events include the process ID. $_[HEAP]{children_by_pid}{$task->PID} = $task; } elsif (ref($self->{_children}) eq "HASH") { $self->{_logger}->debug("HASH "."Child = $child"); $self->_CreateChild($child); } else { $self->{_logger}->debug("SCALAR "."Child = $child"); $self->_CreateChild($child); } # If end } # While end } Simon Taylor Development Team Lead NMS Applications Network Managed Services Nortel Networks (e) s...@nortel.com (p) +44 1279 402291 ESN 6 742 2291 (m) +44 7740 533743 ESN 6 748 3743 (yim) setuk_nortel
Poe Design Question
(updated) I have built a POE app which forks children, executes some work, returns status back up the parent. All works fine. Currently this is all running from one Session. I now have a requirement to execute jobs which connect to the same modem bank together. I was thinking I might structure it in the following way. Assume 4 modem banks with 20 jobs per modem bank. PARENT - Create Session + POE::Wheel::Run (PWR) one process per all jobs which need to connect to modem bank CHILD 1 - Create Session + PWR CHILD 1.1 one process per job .. CHILD 1.20 one process per job . CHILD 4 - Create Session + PWR one process per job This way I can use the heap of each session to manage the child jobs which are connected to each modem bank When using this mechanism to control how many jobs are passed to PWR its dependent on the heap content which in the original approach would contain all jobs :- while (keys(%{$heap->{task}}) < $PARALLELISM ) { Is there a better/simpler way someone can think of to use PWR and the Session Heap to continue to execute jobs against different devices whilst always maintaining 4 simultaneous jobs in each PWR (until complete) Simon Currently start and manage starting tasks this way:- sub StartTasks { =Head3 _start Function called when POE wants to start tasks Shouldnt start any more than $self->GetBSHParalleism($SDU::BatchSectionsHandler::SectionName) When it does it spawns children Event handlers allow information to be passed back to the parent process Mapping between state and function is done in the object states line in the POE::Session->create in _IterateChildrenParallel =cut my ($kernel, $self, $heap) = @_[KERNEL, OBJECT]; my ($child, $task); $self->{_logger}->debug("Enter/Reentry"); while (keys(%{$heap->{task}}) < $PARALLELISM ) { if (ref($self->{_children}) eq "ARRAY"){ $child = shift @{$self->{_children}}; last unless defined $child; $self->{_logger}->debug("ARRAY "."Child = $child"); $task = POE::Wheel::Run->new( Program => sub {local *__ANON__ = "_StartTasksProgramSub"; $self->_CreateChild($child); }, StdoutEvent => "TaskResult", StdoutFilter => POE::Filter::Reference->new(), StderrEvent => "TaskError", CloseEvent => "TaskDone", ); $heap->{task}->{$task->ID} = $task; $self->{_logger}->info("Created child Task/PID ".$task->ID."/".$task->PID); $_[KERNEL]->sig_child($task->PID, 'sig_child'); # Wheel events include the wheel's ID. $_[HEAP]{children_by_wid}{$task->ID} = $task; # Signal events include the process ID. $_[HEAP]{children_by_pid}{$task->PID} = $task; } elsif (ref($self->{_children}) eq "HASH") { $self->{_logger}->debug("HASH "."Child = $child"); $self->_CreateChild($child); } else { $self->{_logger}->debug("SCALAR "."Child = $child"); $self->_CreateChild($child); } # If end } # While end } Simon Taylor Development Team Lead NMS Applications Network Managed Services Nortel Networks (e) s...@nortel.com (p) +44 1279 402291 ESN 6 742 2291 (m) +44 7740 533743 ESN 6 748 3743 (yim) setuk_nortel
Re: Poe Design Question
What's your bottleneck? If it's just speaking with modems, POE can do that asynchronously. You may be able to simplify the design by pulling the jobs back into the main process. -- Rocco Caputo - rcap...@pobox.com On Nov 24, 2009, at 15:00, Simon Taylor wrote: (updated) I have built a POE app which forks children, executes some work, returns status back up the parent. All works fine. Currently this is all running from one Session. I now have a requirement to execute jobs which connect to the same modem bank together. I was thinking I might structure it in the following way. Assume 4 modem banks with 20 jobs per modem bank. PARENT - Create Session + POE::Wheel::Run (PWR) one process per all jobs which need to connect to modem bank CHILD 1 - Create Session + PWR CHILD 1.1 one process per job .. CHILD 1.20 one process per job . CHILD 4 - Create Session + PWR one process per job This way I can use the heap of each session to manage the child jobs which are connected to each modem bank When using this mechanism to control how many jobs are passed to PWR its dependent on the heap content which in the original approach would contain all jobs :- while (keys(%{$heap->{task}}) < $PARALLELISM ) { Is there a better/simpler way someone can think of to use PWR and the Session Heap to continue to execute jobs against different devices whilst always maintaining 4 simultaneous jobs in each PWR (until complete) Simon Currently start and manage starting tasks this way:- sub StartTasks { =Head3 _start Function called when POE wants to start tasks Shouldnt start any more than $self->GetBSHParalleism($SDU::BatchSectionsHandler::SectionName) When it does it spawns children Event handlers allow information to be passed back to the parent process Mapping between state and function is done in the object states line in the POE::Session->create in _IterateChildrenParallel =cut my ($kernel, $self, $heap) = @_[KERNEL, OBJECT]; my ($child, $task); $self->{_logger}->debug("Enter/Reentry"); while (keys(%{$heap->{task}}) < $PARALLELISM ) { if (ref($self->{_children}) eq "ARRAY"){ $child = shift @{$self->{_children}}; last unless defined $child; $self->{_logger}->debug("ARRAY "."Child = $child"); $task = POE::Wheel::Run->new( Program => sub {local *__ANON__ = "_StartTasksProgramSub"; $self->_CreateChild($child); }, StdoutEvent => "TaskResult", StdoutFilter => POE::Filter::Reference->new(), StderrEvent => "TaskError", CloseEvent => "TaskDone", ); $heap->{task}->{$task->ID} = $task; $self->{_logger}->info("Created child Task/PID ".$task->ID."/".$task->PID); $_[KERNEL]->sig_child($task->PID, 'sig_child'); # Wheel events include the wheel's ID. $_[HEAP]{children_by_wid}{$task->ID} = $task; # Signal events include the process ID. $_[HEAP]{children_by_pid}{$task->PID} = $task; } elsif (ref($self->{_children}) eq "HASH") { $self->{_logger}->debug("HASH "."Child = $child"); $self->_CreateChild($child); } else { $self->{_logger}->debug("SCALAR "."Child = $child"); $self->_CreateChild($child); } # If end } # While end } Simon Taylor Development Team Lead NMS Applications Network Managed Services Nortel Networks (e) s...@nortel.com (p) +44 1279 402291 ESN 6 742 2291 (m) +44 7740 533743 ESN 6 748 3743 (yim) setuk_nortel
RE: Poe Design Question
Its not so much speaking with the modems its speaking with the modem banks - each of which only allows 4 concurrent connections and im never sure how many total connections i will need to each modem bank. I was thinking of having a POE::Session per modem bank and then i can use the HEAP of that session to manage how many current connections i have to a modem bank. Using the following mechanism for managing jobs (while (keys(%{$heap->{task}}) < $PARALLELISM ) {) I cant see how i might use the heap of one session to manage connections to multiple modem banks. Simon Simon Taylor Development Team Lead NMS Applications Network Managed Services Nortel Networks (e) s...@nortel.com (p) +44 1279 402291 ESN 6 742 2291 (m) +44 7740 533743 ESN 6 748 3743 (yim) setuk_nortel -Original Message- From: Rocco Caputo [mailto:rcap...@pobox.com] Sent: 25 November 2009 19:37 To: POE Mailing List Subject: Re: Poe Design Question What's your bottleneck? If it's just speaking with modems, POE can do that asynchronously. You may be able to simplify the design by pulling the jobs back into the main process. -- Rocco Caputo - rcap...@pobox.com On Nov 24, 2009, at 15:00, Simon Taylor wrote: > (updated) > > I have built a POE app which forks children, executes some work, > returns > status back up the parent. All works fine. > > Currently this is all running from one Session. > > I now have a requirement to execute jobs which connect to the same > modem > bank together. > > I was thinking I might structure it in the following way. > > Assume 4 modem banks with 20 jobs per modem bank. > > > > PARENT - Create Session + POE::Wheel::Run (PWR) one process per all > jobs > which need to connect to modem bank > > CHILD 1 - Create Session + PWR > > CHILD 1.1 one process per job > >.. > > CHILD 1.20 one process per job > > . > > CHILD 4 - Create Session + PWR one process per job > > > > This way I can use the heap of each session to manage the child jobs > which are connected to each modem bank > > > > When using this mechanism to control how many jobs are passed to PWR > its > dependent on the heap content which in the original approach would > contain all jobs :- > > while (keys(%{$heap->{task}}) < $PARALLELISM ) { > > > > Is there a better/simpler way someone can think of to use PWR and the > Session Heap to continue to execute jobs against different devices > whilst always maintaining 4 simultaneous jobs in each PWR (until > complete) > > > > Simon > > > > > > Currently start and manage starting tasks this way:- > > > > sub StartTasks { > > =Head3 _start > > Function called when POE wants to start tasks > > Shouldnt start any more than > $self->GetBSHParalleism($SDU::BatchSectionsHandler::SectionName) > > When it does it spawns children > > Event handlers allow information to be passed back to the parent > process > > Mapping between state and function is done in the object states line > in > the POE::Session->create in _IterateChildrenParallel > > > > =cut > > my ($kernel, $self, $heap) = @_[KERNEL, OBJECT]; > > my ($child, $task); > > > > $self->{_logger}->debug("Enter/Reentry"); > > while (keys(%{$heap->{task}}) < $PARALLELISM ) { > > if (ref($self->{_children}) eq "ARRAY"){ > > $child = shift @{$self->{_children}}; > > last unless defined $child; > > $self->{_logger}->debug("ARRAY "."Child = $child"); > > $task = POE::Wheel::Run->new( > > Program => sub {local *__ANON__ = > "_StartTasksProgramSub"; > > $self->_CreateChild($child); > > }, > > StdoutEvent => "TaskResult", > > StdoutFilter => POE::Filter::Reference->new(), > > StderrEvent => "TaskError", > > CloseEvent => "TaskDone", > >); > >$heap->{task}->{$task->ID} = $task; > >$self->{_logger}->info("Created child Task/PID > ".$task->ID."/".$task->PID); > >$_[KERNEL]->sig_child($task->PID, 'sig_child'); > > > ># Wheel events include the wheel's ID. > >$_[HEAP]{children_by_wid}{$task->ID} = $task; > > > ># S