I already did the benchmarking exercise and found convolution to be the real time hog. I'm going to continue playing with PDL::ParallelCPU.
Nonetheless, any information helping me to get convolveND to work with conventional threading in the original code is greatly appreciated. Thanks, Perldough 2012/10/11 Chris Marshall <[email protected]> > perldoc PDL::ParallelCPU gives the documentation > on auto-parallel threads it is not a module. PDL > uses 32bit indexing so the maximum size of a piddle > is <2**32. > > Also, consider using profiling and/or benchmarking > to measure where things need to be sped up in > your code. With the example of your code, speeding > up file IO could be more important than speeding > up the convolutions. > > --Chris > > On 10/10/12, Grasswistle <[email protected]> wrote: > > Hello Chris, > > > > I am using PDL v2.4.10, as reported by *perl -MPDL -e "print > > \"$PDL::VERSION\";"** * The command use PDL::parallelCPU* *fails. for > me > > though. Is there another more recent version of PDL I am not aware of? > > > > Also, I monitored the system memory while I was running the program and > > didn't find that the memory usage got to be too high. Is there a maximum > > on the memory that Perl/PDL can consume? How do I go about changing > that? > > > > Thanks, > > Perldough > > > > 2012/10/10 Chris Marshall <[email protected]> > > > >> Hi Grasswistle- > >> > >> Maybe there is a possible ithread-safety issue going > >> on but my guess would be you are running out of > >> memory which will make perl exit. > >> > >> Also, you don't say what version of PDL you are using, > >> but the current release has support for automatic threadloop > >> parallelization across processors. Take a look at the > >> docs for PDL::ParallelCPU. > >> > >> --Chris > >> > >> On Wed, Oct 10, 2012 at 3:33 PM, Grasswistle <[email protected]> > >> wrote: > >> > Hello, > >> > > >> > I need to process a bunch of images in very little time. As a result, > >> > I > >> > have decided to try and parallelise multiple processes. I knew this > >> > was > >> not > >> > going to be trivial, but now I've narrowed down the problem and I'm > >> > completely stuck. The problem begins when I try to do convolution: > the > >> > importation of the image into a piddle. > >> > > >> > I have written a small test program pasted at the foot of this email. > >> > In > >> > the code there are two clusters of commented out code. The first > >> convolves > >> > an image with PDL::ImageND::convolveND while the second convolves an > >> image > >> > with PDL::Image2D::conv2d. You may try uncommenting one cluster of > >> > code > >> at > >> > a time. > >> > > >> > When I try to import a small png (300x255), all four threads return > >> within > >> > the allotted 30 seconds regardless of the import method used. > However, > >> when > >> > a lager image is used (2740x1818, about the size needed for my > >> application), > >> > the problems occur. Using conv2D, the programs exhibits no problems > and > >> > behaves as before. Using convolveND, the program exits well before > the > >> 30 > >> > seconds are up and before all 4 threads have returned. This seems to > >> > indicate some sort of problem, but without any error messages, it is > >> > difficult to tell what is going on... > >> > > >> > Here are links to the images I used: > >> > small image: http://imageshack.us/a/img23/9039/ballr.png > >> > large image (yes, it is blank): > >> > http://imageshack.us/a/img138/3980/atestblank.png > >> > > >> > I am using Perl 5.12.2 on a virtualised installation of XP. > >> > > >> > See code below. > >> > > >> > Thanks, > >> > Perldough > >> > > >> > <code> > >> > use PDL; > >> > use threads; > >> > > >> > my $thread1 = threads->create(\&analysis_A); # start thread running > >> > my $thread2 = threads->create(\&analysis_A); > >> > my $thread3 = threads->create(\&analysis_A); > >> > my $thread4 = threads->create(\&analysis_A); > >> > > >> > sleep 30; > >> > > >> > sub analysis_A > >> > { > >> > require PDL::IO::GD; > >> > my $im = PDL::IO::GD->new( {filename => 'A_test-blank.png'} ); > >> > my $piddle = $im->to_pdl(); > >> > > >> > #importation method 1 > >> > #require PDL::ImageND; > >> > #my $new = PDL::ImageND::convolveND( $piddle, ones(5,5)); # Works > >> > in > >> 1 > >> > thread, fails in multiple > >> > > >> > #importation method 2 > >> > #require PDL::Image2D; > >> > #my $new = PDL::Image2D::conv2d( $piddle, ones(5,5)); # Works > >> > in > >> > multiple threads > >> > > >> > my $tid = threads->tid(); > >> > print "thread $tid: Done \n"; > >> > } > >> > </code> > >> > > >> > _______________________________________________ > >> > Perldl mailing list > >> > [email protected] > >> > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl > >> > > >> > > >
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
