Re: fork, parallel and global values
On Fri, May 14, 2010 at 3:07 PM, Dr.Ruud rvtol+use...@isolution.nlrvtol%2buse...@isolution.nl wrote: raphael() wrote: I want to do work on all elements of an array simultaneously. What kind of data/reality does your array represent? I tried Parallel::ForkManager. It works well when you don't need to update global value like downloading multiple files at the same time. I want something like above that forks while updating global values. Are threads/fork what I am looking for? If you have a list of file names that need to be processed, and that can be handled independently, then always go for fork-join, don't even consider threading. The parent will detect when the child is finished, and can look in a predefined location (I normally use a database) for the result. If your audience is not a bunch of twitchers, then stay away from threading. -- Ruud -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/ Hi, thanks for replying. No twitching audience :) This is just a rough project of mine trying to get a feel of parallelism. Your advice on using a database as a place to use shared info is more useful after I read http://www.perlmonks.org/index.pl?node_id=288022 *thanks* let see what I can gain from this.
Re: fork, parallel and global values
raphael() wrote: I want to do work on all elements of an array simultaneously. What kind of data/reality does your array represent? I tried Parallel::ForkManager. It works well when you don't need to update global value like downloading multiple files at the same time. I want something like above that forks while updating global values. Are threads/fork what I am looking for? If you have a list of file names that need to be processed, and that can be handled independently, then always go for fork-join, don't even consider threading. The parent will detect when the child is finished, and can look in a predefined location (I normally use a database) for the result. If your audience is not a bunch of twitchers, then stay away from threading. -- Ruud -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/
Re: fork, parallel and global values
On Tue, May 11, 2010 at 1:23 AM, C.DeRykus dery...@gmail.com wrote:
On May 10, 7:07 am, raphael.j...@gmail.com (raphael()) wrote:
Hello,
-- CODE --
#!/usr/bin/env perl
use strict;
use warnings;
use Parallel::ForkManager;
# Parallel::ForkManager
my $pfm = Parallel::ForkManager-new(5);
my %hash;
for my $num ( qw/ 1 2 3 4 5 1 / ) {
$pfm-start and next;
$hash{$num}++;
$pfm-finish;}
$pfm-wait_all_children;
# doesn't print because %hash is undef
print $_ = $hash{$_}\n for sort keys %hash;
print hash is undef\n unless %hash;
--- END ---
I want to do work on all elements of an array simultaneously.
I tried Parallel::ForkManager. It works well when you don't need to
update
global value like downloading
multiple files at the same time. I want something like above that forks
while updating global values.
Are threads/fork what I am looking for?
Any help would be *appreciated.*
If your host supports Sys V IPC, one possible solution:
IPC::Shareable
--
Charles DeRykus
--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/
No IPC::Shareable. But I can go in for Win32::MMF::Shareable.
However I went in for threads::shared which is working great so far.
You know this threads `thing` is great. So much work can be done within
short time.
--- CODE --
use strict;
use warnings;
use threads;
use threads::shared;
use File::Basename;
use WWW::Mechanize;
use Parallel::ForkManager;
my ( %hash, @missing ) :shared;
# ---BLAH BLAH ~ CODE HERE
if ( $response-is_success ) {
my $ref = []; share( $ref );# THIS IS SO COOL! LOL
push @{$ref}, ( $INFO, $url );
$hash{$base} = $ref; # $hash{$base} = [] ~ ERROR.
Only scalar references?
} else {
--- END ---
That it. No questions for now.
Re: fork, parallel and global values
On Tue, May 11, 2010 at 1:59 AM, Eric Veith1 eric.ve...@de.ibm.com wrote: raphael() raphael.j...@gmail.com wrote on 05/10/2010 04:07:58 PM: I want to do work on all elements of an array simultaneously. To clarify: You want to access an hash defined in the parent process from all N child processes? I'm sorry to tell you, but this won't work so easily. When forking, data is *copied*. Which means that every child process gets a copy of all the parent's variables, but whenever you modify them, the modification has only an effect within the child process, because it's just a copy. There's a whole bag full of inter process communication (IPC) algorithms, ranging from files over sockets to posix message queues. You might to have a look at perldoc perlipc and the whole lot of IPC modules on cpan. HTH. Eric -- Eric MSP Veith eric.ve...@de.ibm.com Hechtsheimer Str. 2 DE-55131 Mainz Germany IBM Deutschland GmbH Vorsitzender des Aufsichtsrats: Erich Clementi Geschäftsführung: Martin Jetter (Vorsitzender), Reinhard Reschke, Christoph Grandpierre, Matthias Hartmann, Michael Diemer, Martina Koederitz Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart, HRB 14562 WEEE-Reg.-Nr. DE 99369940 Hi Eric. I knew I wouldn't get to eat the cake so easily. Let see what other have to say on this. *Thanks* for your reply. PS - perldoc perlipc seems complicated :(
Re: fork, parallel and global values
On May 10, 7:07 am, raphael.j...@gmail.com (raphael()) wrote:
Hello,
-- CODE --
#!/usr/bin/env perl
use strict;
use warnings;
use Parallel::ForkManager;
# Parallel::ForkManager
my $pfm = Parallel::ForkManager-new(5);
my %hash;
for my $num ( qw/ 1 2 3 4 5 1 / ) {
$pfm-start and next;
$hash{$num}++;
$pfm-finish;}
$pfm-wait_all_children;
# doesn't print because %hash is undef
print $_ = $hash{$_}\n for sort keys %hash;
print hash is undef\n unless %hash;
--- END ---
I want to do work on all elements of an array simultaneously.
I tried Parallel::ForkManager. It works well when you don't need to update
global value like downloading
multiple files at the same time. I want something like above that forks
while updating global values.
Are threads/fork what I am looking for?
Any help would be *appreciated.*
If your host supports Sys V IPC, one possible solution:
IPC::Shareable
--
Charles DeRykus
--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/
Re: fork, parallel and global values
raphael() raphael.j...@gmail.com wrote on 05/10/2010 04:07:58 PM: I want to do work on all elements of an array simultaneously. To clarify: You want to access an hash defined in the parent process from all N child processes? I'm sorry to tell you, but this won't work so easily. When forking, data is *copied*. Which means that every child process gets a copy of all the parent's variables, but whenever you modify them, the modification has only an effect within the child process, because it's just a copy. There's a whole bag full of inter process communication (IPC) algorithms, ranging from files over sockets to posix message queues. You might to have a look at perldoc perlipc and the whole lot of IPC modules on cpan. HTH. Eric -- Eric MSP Veith eric.ve...@de.ibm.com Hechtsheimer Str. 2 DE-55131 Mainz Germany IBM Deutschland GmbH Vorsitzender des Aufsichtsrats: Erich Clementi Geschäftsführung: Martin Jetter (Vorsitzender), Reinhard Reschke, Christoph Grandpierre, Matthias Hartmann, Michael Diemer, Martina Koederitz Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart, HRB 14562 WEEE-Reg.-Nr. DE 99369940 -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/
