Okay, I have started to work with PDL::ParallelCPU on the side. I'm having a little trouble, but I still need to struggle with it a little before I ask any questions about it.
In the meantime, I was wondering if anyone could try running the original code I provided and if they could perhaps explain why it fails in the way I described. Thanks, Perldough 2012/10/10 Derek Lamb <[email protected]> > There is PDL 2.4.11 (see pdl.perl.org) but ParallelCPU was introduced in > 2.4.10 so you should be OK there. As Mark mentioned, you do not 'use > PDL::ParallelCPU;' (it is just a documentation module, the real work > happens in PDL::Core), but rather just set the number of target threads and > the minimum size of data that you want auto-pthreaded. See > http://pdl.perl.org/?docs=ParallelCPU&title=PDL::ParallelCPU for details > and an example. > > If you try to allocate a piddle > 1GB then the shell will complain and > advise you to set $PDL::BIGPDL=1. I don't think that is your problem here. > Sorry I can't run your example, 'use threads' complains on my machine. > > Derek > > > On Oct 10, 2012, at 4:21 PM, Grasswistle wrote: > > Hello Chris, > > I am using PDL v2.4.10, as reported by *perl -MPDL -e "print > \"$PDL::VERSION\";"** * The command use PDL::parallelCPU* *fails. for > me though. Is there another more recent version of PDL I am not aware of? > > Also, I monitored the system memory while I was running the program and > didn't find that the memory usage got to be too high. Is there a maximum > on the memory that Perl/PDL can consume? How do I go about changing that? > > Thanks, > Perldough > > 2012/10/10 Chris Marshall <[email protected]> > >> Hi Grasswistle- >> >> Maybe there is a possible ithread-safety issue going >> on but my guess would be you are running out of >> memory which will make perl exit. >> >> Also, you don't say what version of PDL you are using, >> but the current release has support for automatic threadloop >> parallelization across processors. Take a look at the >> docs for PDL::ParallelCPU. >> >> --Chris >> >> On Wed, Oct 10, 2012 at 3:33 PM, Grasswistle <[email protected]> >> wrote: >> > Hello, >> > >> > I need to process a bunch of images in very little time. As a result, I >> > have decided to try and parallelise multiple processes. I knew this >> was not >> > going to be trivial, but now I've narrowed down the problem and I'm >> > completely stuck. The problem begins when I try to do convolution: the >> > importation of the image into a piddle. >> > >> > I have written a small test program pasted at the foot of this email. >> In >> > the code there are two clusters of commented out code. The first >> convolves >> > an image with PDL::ImageND::convolveND while the second convolves an >> image >> > with PDL::Image2D::conv2d. You may try uncommenting one cluster of >> code at >> > a time. >> > >> > When I try to import a small png (300x255), all four threads return >> within >> > the allotted 30 seconds regardless of the import method used. However, >> when >> > a lager image is used (2740x1818, about the size needed for my >> application), >> > the problems occur. Using conv2D, the programs exhibits no problems and >> > behaves as before. Using convolveND, the program exits well before the >> 30 >> > seconds are up and before all 4 threads have returned. This seems to >> > indicate some sort of problem, but without any error messages, it is >> > difficult to tell what is going on... >> > >> > Here are links to the images I used: >> > small image: http://imageshack.us/a/img23/9039/ballr.png >> > large image (yes, it is blank): >> > http://imageshack.us/a/img138/3980/atestblank.png >> > >> > I am using Perl 5.12.2 on a virtualised installation of XP. >> > >> > See code below. >> > >> > Thanks, >> > Perldough >> > >> > <code> >> > use PDL; >> > use threads; >> > >> > my $thread1 = threads->create(\&analysis_A); # start thread running >> > my $thread2 = threads->create(\&analysis_A); >> > my $thread3 = threads->create(\&analysis_A); >> > my $thread4 = threads->create(\&analysis_A); >> > >> > sleep 30; >> > >> > sub analysis_A >> > { >> > require PDL::IO::GD; >> > my $im = PDL::IO::GD->new( {filename => 'A_test-blank.png'} ); >> > my $piddle = $im->to_pdl(); >> > >> > #importation method 1 >> > #require PDL::ImageND; >> > #my $new = PDL::ImageND::convolveND( $piddle, ones(5,5)); # Works >> in 1 >> > thread, fails in multiple >> > >> > #importation method 2 >> > #require PDL::Image2D; >> > #my $new = PDL::Image2D::conv2d( $piddle, ones(5,5)); # Works in >> > multiple threads >> > >> > my $tid = threads->tid(); >> > print "thread $tid: Done \n"; >> > } >> > </code> >> > >> > _______________________________________________ >> > Perldl mailing list >> > [email protected] >> > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl >> > >> > > _______________________________________________ > Perldl mailing list > [email protected] > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl > > >
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
