Poe Design Question
I have built a POE app which forks children, executes some work, returns
status back up the parent. All works fine.
Currently this is all running from one Session.
I now have a requirement to execute jobs which connect to the same modem
bank together.
I was thinking I might structure it in the following way:-
PARENT - Create Session + POE::Wheel::Run (PWR) one process per all jobs
which need to connect to modem bank
CHILD 1 - Create Session + PWR one process per job
CHILD n - Create Session + PWR one process per job
This way I can use the heap of each session to manage the child jobs
which are connected to each modem bank
When using this mechanism to control how many jobs are passed to PWR its
dependent on the heap content which in the original approach would
contain all jobs :-
while (keys(%{$heap->{task}}) < $PARALLELISM ) {
Is there a better/simpler way someone can think of to use PWR and the
Session Heap to continue to execute jobs against different devices
whilst always maintaining 4 simultaneous jobs in each PWR (until
complete)
Simon
Currently start and manage starting tasks this way:-
sub StartTasks {
=Head3 _start
Function called when POE wants to start tasks
Shouldnt start any more than
$self->GetBSHParalleism($SDU::BatchSectionsHandler::SectionName)
When it does it spawns children
Event handlers allow information to be passed back to the parent process
Mapping between state and function is done in the object states line in
the POE::Session->create in _IterateChildrenParallel
=cut
my ($kernel, $self, $heap) = @_[KERNEL, OBJECT];
my ($child, $task);
$self->{_logger}->debug("Enter/Reentry");
while (keys(%{$heap->{task}}) < $PARALLELISM ) {
if (ref($self->{_children}) eq "ARRAY"){
$child = shift @{$self->{_children}};
last unless defined $child;
$self->{_logger}->debug("ARRAY "."Child = $child");
$task = POE::Wheel::Run->new(
Program => sub {local *__ANON__ =
"_StartTasksProgramSub";
$self->_CreateChild($child);
},
StdoutEvent => "TaskResult",
StdoutFilter => POE::Filter::Reference->new(),
StderrEvent => "TaskError",
CloseEvent => "TaskDone",
);
$heap->{task}->{$task->ID} = $task;
$self->{_logger}->info("Created child Task/PID
".$task->ID."/".$task->PID);
$_[KERNEL]->sig_child($task->PID, 'sig_child');
# Wheel events include the wheel's ID.
$_[HEAP]{children_by_wid}{$task->ID} = $task;
# Signal events include the process ID.
$_[HEAP]{children_by_pid}{$task->PID} = $task;
} elsif (ref($self->{_children}) eq "HASH") {
$self->{_logger}->debug("HASH "."Child = $child");
$self->_CreateChild($child);
} else {
$self->{_logger}->debug("SCALAR "."Child = $child");
$self->_CreateChild($child);
} # If end
} # While end
}
Simon Taylor
Development Team Lead NMS Applications
Network Managed Services
Nortel Networks
(e) s...@nortel.com
(p) +44 1279 402291 ESN 6 742 2291
(m) +44 7740 533743 ESN 6 748 3743
(yim) setuk_nortel
Poe Design Question
(updated)
I have built a POE app which forks children, executes some work, returns
status back up the parent. All works fine.
Currently this is all running from one Session.
I now have a requirement to execute jobs which connect to the same modem
bank together.
I was thinking I might structure it in the following way.
Assume 4 modem banks with 20 jobs per modem bank.
PARENT - Create Session + POE::Wheel::Run (PWR) one process per all jobs
which need to connect to modem bank
CHILD 1 - Create Session + PWR
CHILD 1.1 one process per job
..
CHILD 1.20 one process per job
.
CHILD 4 - Create Session + PWR one process per job
This way I can use the heap of each session to manage the child jobs
which are connected to each modem bank
When using this mechanism to control how many jobs are passed to PWR its
dependent on the heap content which in the original approach would
contain all jobs :-
while (keys(%{$heap->{task}}) < $PARALLELISM ) {
Is there a better/simpler way someone can think of to use PWR and the
Session Heap to continue to execute jobs against different devices
whilst always maintaining 4 simultaneous jobs in each PWR (until
complete)
Simon
Currently start and manage starting tasks this way:-
sub StartTasks {
=Head3 _start
Function called when POE wants to start tasks
Shouldnt start any more than
$self->GetBSHParalleism($SDU::BatchSectionsHandler::SectionName)
When it does it spawns children
Event handlers allow information to be passed back to the parent process
Mapping between state and function is done in the object states line in
the POE::Session->create in _IterateChildrenParallel
=cut
my ($kernel, $self, $heap) = @_[KERNEL, OBJECT];
my ($child, $task);
$self->{_logger}->debug("Enter/Reentry");
while (keys(%{$heap->{task}}) < $PARALLELISM ) {
if (ref($self->{_children}) eq "ARRAY"){
$child = shift @{$self->{_children}};
last unless defined $child;
$self->{_logger}->debug("ARRAY "."Child = $child");
$task = POE::Wheel::Run->new(
Program => sub {local *__ANON__ =
"_StartTasksProgramSub";
$self->_CreateChild($child);
},
StdoutEvent => "TaskResult",
StdoutFilter => POE::Filter::Reference->new(),
StderrEvent => "TaskError",
CloseEvent => "TaskDone",
);
$heap->{task}->{$task->ID} = $task;
$self->{_logger}->info("Created child Task/PID
".$task->ID."/".$task->PID);
$_[KERNEL]->sig_child($task->PID, 'sig_child');
# Wheel events include the wheel's ID.
$_[HEAP]{children_by_wid}{$task->ID} = $task;
# Signal events include the process ID.
$_[HEAP]{children_by_pid}{$task->PID} = $task;
} elsif (ref($self->{_children}) eq "HASH") {
$self->{_logger}->debug("HASH "."Child = $child");
$self->_CreateChild($child);
} else {
$self->{_logger}->debug("SCALAR "."Child = $child");
$self->_CreateChild($child);
} # If end
} # While end
}
Simon Taylor
Development Team Lead NMS Applications
Network Managed Services
Nortel Networks
(e) s...@nortel.com
(p) +44 1279 402291 ESN 6 742 2291
(m) +44 7740 533743 ESN 6 748 3743
(yim) setuk_nortel
Re: Poe Design Question
What's your bottleneck? If it's just speaking with modems, POE can do
that asynchronously. You may be able to simplify the design by
pulling the jobs back into the main process.
--
Rocco Caputo - rcap...@pobox.com
On Nov 24, 2009, at 15:00, Simon Taylor wrote:
(updated)
I have built a POE app which forks children, executes some work,
returns
status back up the parent. All works fine.
Currently this is all running from one Session.
I now have a requirement to execute jobs which connect to the same
modem
bank together.
I was thinking I might structure it in the following way.
Assume 4 modem banks with 20 jobs per modem bank.
PARENT - Create Session + POE::Wheel::Run (PWR) one process per all
jobs
which need to connect to modem bank
CHILD 1 - Create Session + PWR
CHILD 1.1 one process per job
..
CHILD 1.20 one process per job
.
CHILD 4 - Create Session + PWR one process per job
This way I can use the heap of each session to manage the child jobs
which are connected to each modem bank
When using this mechanism to control how many jobs are passed to PWR
its
dependent on the heap content which in the original approach would
contain all jobs :-
while (keys(%{$heap->{task}}) < $PARALLELISM ) {
Is there a better/simpler way someone can think of to use PWR and the
Session Heap to continue to execute jobs against different devices
whilst always maintaining 4 simultaneous jobs in each PWR (until
complete)
Simon
Currently start and manage starting tasks this way:-
sub StartTasks {
=Head3 _start
Function called when POE wants to start tasks
Shouldnt start any more than
$self->GetBSHParalleism($SDU::BatchSectionsHandler::SectionName)
When it does it spawns children
Event handlers allow information to be passed back to the parent
process
Mapping between state and function is done in the object states line
in
the POE::Session->create in _IterateChildrenParallel
=cut
my ($kernel, $self, $heap) = @_[KERNEL, OBJECT];
my ($child, $task);
$self->{_logger}->debug("Enter/Reentry");
while (keys(%{$heap->{task}}) < $PARALLELISM ) {
if (ref($self->{_children}) eq "ARRAY"){
$child = shift @{$self->{_children}};
last unless defined $child;
$self->{_logger}->debug("ARRAY "."Child = $child");
$task = POE::Wheel::Run->new(
Program => sub {local *__ANON__ =
"_StartTasksProgramSub";
$self->_CreateChild($child);
},
StdoutEvent => "TaskResult",
StdoutFilter => POE::Filter::Reference->new(),
StderrEvent => "TaskError",
CloseEvent => "TaskDone",
);
$heap->{task}->{$task->ID} = $task;
$self->{_logger}->info("Created child Task/PID
".$task->ID."/".$task->PID);
$_[KERNEL]->sig_child($task->PID, 'sig_child');
# Wheel events include the wheel's ID.
$_[HEAP]{children_by_wid}{$task->ID} = $task;
# Signal events include the process ID.
$_[HEAP]{children_by_pid}{$task->PID} = $task;
} elsif (ref($self->{_children}) eq "HASH") {
$self->{_logger}->debug("HASH "."Child = $child");
$self->_CreateChild($child);
} else {
$self->{_logger}->debug("SCALAR "."Child = $child");
$self->_CreateChild($child);
} # If end
} # While end
}
Simon Taylor
Development Team Lead NMS Applications
Network Managed Services
Nortel Networks
(e) s...@nortel.com
(p) +44 1279 402291 ESN 6 742 2291
(m) +44 7740 533743 ESN 6 748 3743
(yim) setuk_nortel
RE: Poe Design Question
Its not so much speaking with the modems its speaking with the modem
banks - each of which only allows 4 concurrent connections and im never
sure how many total connections i will need to each modem bank.
I was thinking of having a POE::Session per modem bank and then i can
use the HEAP of that session to manage how many current connections i
have to a modem bank.
Using the following mechanism for managing jobs (while
(keys(%{$heap->{task}}) < $PARALLELISM ) {) I cant see how i might use
the heap of one session to manage connections to multiple modem banks.
Simon
Simon Taylor
Development Team Lead NMS Applications
Network Managed Services
Nortel Networks
(e) s...@nortel.com
(p) +44 1279 402291 ESN 6 742 2291
(m) +44 7740 533743 ESN 6 748 3743
(yim) setuk_nortel
-Original Message-
From: Rocco Caputo [mailto:rcap...@pobox.com]
Sent: 25 November 2009 19:37
To: POE Mailing List
Subject: Re: Poe Design Question
What's your bottleneck? If it's just speaking with modems, POE can do
that asynchronously. You may be able to simplify the design by
pulling the jobs back into the main process.
--
Rocco Caputo - rcap...@pobox.com
On Nov 24, 2009, at 15:00, Simon Taylor wrote:
> (updated)
>
> I have built a POE app which forks children, executes some work,
> returns
> status back up the parent. All works fine.
>
> Currently this is all running from one Session.
>
> I now have a requirement to execute jobs which connect to the same
> modem
> bank together.
>
> I was thinking I might structure it in the following way.
>
> Assume 4 modem banks with 20 jobs per modem bank.
>
>
>
> PARENT - Create Session + POE::Wheel::Run (PWR) one process per all
> jobs
> which need to connect to modem bank
>
> CHILD 1 - Create Session + PWR
>
> CHILD 1.1 one process per job
>
>..
>
> CHILD 1.20 one process per job
>
> .
>
> CHILD 4 - Create Session + PWR one process per job
>
>
>
> This way I can use the heap of each session to manage the child jobs
> which are connected to each modem bank
>
>
>
> When using this mechanism to control how many jobs are passed to PWR
> its
> dependent on the heap content which in the original approach would
> contain all jobs :-
>
> while (keys(%{$heap->{task}}) < $PARALLELISM ) {
>
>
>
> Is there a better/simpler way someone can think of to use PWR and the
> Session Heap to continue to execute jobs against different devices
> whilst always maintaining 4 simultaneous jobs in each PWR (until
> complete)
>
>
>
> Simon
>
>
>
>
>
> Currently start and manage starting tasks this way:-
>
>
>
> sub StartTasks {
>
> =Head3 _start
>
> Function called when POE wants to start tasks
>
> Shouldnt start any more than
> $self->GetBSHParalleism($SDU::BatchSectionsHandler::SectionName)
>
> When it does it spawns children
>
> Event handlers allow information to be passed back to the parent
> process
>
> Mapping between state and function is done in the object states line
> in
> the POE::Session->create in _IterateChildrenParallel
>
>
>
> =cut
>
> my ($kernel, $self, $heap) = @_[KERNEL, OBJECT];
>
> my ($child, $task);
>
>
>
> $self->{_logger}->debug("Enter/Reentry");
>
> while (keys(%{$heap->{task}}) < $PARALLELISM ) {
>
> if (ref($self->{_children}) eq "ARRAY"){
>
> $child = shift @{$self->{_children}};
>
> last unless defined $child;
>
> $self->{_logger}->debug("ARRAY "."Child = $child");
>
> $task = POE::Wheel::Run->new(
>
> Program => sub {local *__ANON__ =
> "_StartTasksProgramSub";
>
> $self->_CreateChild($child);
>
> },
>
> StdoutEvent => "TaskResult",
>
> StdoutFilter => POE::Filter::Reference->new(),
>
> StderrEvent => "TaskError",
>
> CloseEvent => "TaskDone",
>
>);
>
>$heap->{task}->{$task->ID} = $task;
>
>$self->{_logger}->info("Created child Task/PID
> ".$task->ID."/".$task->PID);
>
>$_[KERNEL]->sig_child($task->PID, 'sig_child');
>
>
>
># Wheel events include the wheel's ID.
>
>$_[HEAP]{children_by_wid}{$task->ID} = $task;
>
>
>
># S
