perldoc PDL::ParallelCPU gives the documentation on auto-parallel threads it is not a module. PDL uses 32bit indexing so the maximum size of a piddle is <2**32.
Also, consider using profiling and/or benchmarking to measure where things need to be sped up in your code. With the example of your code, speeding up file IO could be more important than speeding up the convolutions. --Chris On 10/10/12, Grasswistle <[email protected]> wrote: > Hello Chris, > > I am using PDL v2.4.10, as reported by *perl -MPDL -e "print > \"$PDL::VERSION\";"** * The command use PDL::parallelCPU* *fails. for me > though. Is there another more recent version of PDL I am not aware of? > > Also, I monitored the system memory while I was running the program and > didn't find that the memory usage got to be too high. Is there a maximum > on the memory that Perl/PDL can consume? How do I go about changing that? > > Thanks, > Perldough > > 2012/10/10 Chris Marshall <[email protected]> > >> Hi Grasswistle- >> >> Maybe there is a possible ithread-safety issue going >> on but my guess would be you are running out of >> memory which will make perl exit. >> >> Also, you don't say what version of PDL you are using, >> but the current release has support for automatic threadloop >> parallelization across processors. Take a look at the >> docs for PDL::ParallelCPU. >> >> --Chris >> >> On Wed, Oct 10, 2012 at 3:33 PM, Grasswistle <[email protected]> >> wrote: >> > Hello, >> > >> > I need to process a bunch of images in very little time. As a result, >> > I >> > have decided to try and parallelise multiple processes. I knew this >> > was >> not >> > going to be trivial, but now I've narrowed down the problem and I'm >> > completely stuck. The problem begins when I try to do convolution: the >> > importation of the image into a piddle. >> > >> > I have written a small test program pasted at the foot of this email. >> > In >> > the code there are two clusters of commented out code. The first >> convolves >> > an image with PDL::ImageND::convolveND while the second convolves an >> image >> > with PDL::Image2D::conv2d. You may try uncommenting one cluster of >> > code >> at >> > a time. >> > >> > When I try to import a small png (300x255), all four threads return >> within >> > the allotted 30 seconds regardless of the import method used. However, >> when >> > a lager image is used (2740x1818, about the size needed for my >> application), >> > the problems occur. Using conv2D, the programs exhibits no problems and >> > behaves as before. Using convolveND, the program exits well before the >> 30 >> > seconds are up and before all 4 threads have returned. This seems to >> > indicate some sort of problem, but without any error messages, it is >> > difficult to tell what is going on... >> > >> > Here are links to the images I used: >> > small image: http://imageshack.us/a/img23/9039/ballr.png >> > large image (yes, it is blank): >> > http://imageshack.us/a/img138/3980/atestblank.png >> > >> > I am using Perl 5.12.2 on a virtualised installation of XP. >> > >> > See code below. >> > >> > Thanks, >> > Perldough >> > >> > <code> >> > use PDL; >> > use threads; >> > >> > my $thread1 = threads->create(\&analysis_A); # start thread running >> > my $thread2 = threads->create(\&analysis_A); >> > my $thread3 = threads->create(\&analysis_A); >> > my $thread4 = threads->create(\&analysis_A); >> > >> > sleep 30; >> > >> > sub analysis_A >> > { >> > require PDL::IO::GD; >> > my $im = PDL::IO::GD->new( {filename => 'A_test-blank.png'} ); >> > my $piddle = $im->to_pdl(); >> > >> > #importation method 1 >> > #require PDL::ImageND; >> > #my $new = PDL::ImageND::convolveND( $piddle, ones(5,5)); # Works >> > in >> 1 >> > thread, fails in multiple >> > >> > #importation method 2 >> > #require PDL::Image2D; >> > #my $new = PDL::Image2D::conv2d( $piddle, ones(5,5)); # Works >> > in >> > multiple threads >> > >> > my $tid = threads->tid(); >> > print "thread $tid: Done \n"; >> > } >> > </code> >> > >> > _______________________________________________ >> > Perldl mailing list >> > [email protected] >> > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl >> > >> > _______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
