I'm not familiar with ithreads to help but one thought would be to use an optimized 2d spatial convolution specific to your problem.
While PDL computations are done with C code, the generality and flexibility does come at a cost. If you need peak performance, wrapping a specific C convolution routine should offer improved performance. Good luck, Chris On 10/11/12, Grasswistle <[email protected]> wrote: > I already did the benchmarking exercise and found convolution to be the > real time hog. I'm going to continue playing with PDL::ParallelCPU. > > Nonetheless, any information helping me to get convolveND to work with > conventional threading in the original code is greatly appreciated. > > Thanks, > Perldough > 2012/10/11 Chris Marshall <[email protected]> > >> perldoc PDL::ParallelCPU gives the documentation >> on auto-parallel threads it is not a module. PDL >> uses 32bit indexing so the maximum size of a piddle >> is <2**32. >> >> Also, consider using profiling and/or benchmarking >> to measure where things need to be sped up in >> your code. With the example of your code, speeding >> up file IO could be more important than speeding >> up the convolutions. >> >> --Chris >> >> On 10/10/12, Grasswistle <[email protected]> wrote: >> > Hello Chris, >> > >> > I am using PDL v2.4.10, as reported by *perl -MPDL -e "print >> > \"$PDL::VERSION\";"** * The command use PDL::parallelCPU* *fails. for >> me >> > though. Is there another more recent version of PDL I am not aware >> of? >> > >> > Also, I monitored the system memory while I was running the program and >> > didn't find that the memory usage got to be too high. Is there a >> > maximum >> > on the memory that Perl/PDL can consume? How do I go about changing >> that? >> > >> > Thanks, >> > Perldough >> > >> > 2012/10/10 Chris Marshall <[email protected]> >> > >> >> Hi Grasswistle- >> >> >> >> Maybe there is a possible ithread-safety issue going >> >> on but my guess would be you are running out of >> >> memory which will make perl exit. >> >> >> >> Also, you don't say what version of PDL you are using, >> >> but the current release has support for automatic threadloop >> >> parallelization across processors. Take a look at the >> >> docs for PDL::ParallelCPU. >> >> >> >> --Chris >> >> >> >> On Wed, Oct 10, 2012 at 3:33 PM, Grasswistle <[email protected]> >> >> wrote: >> >> > Hello, >> >> > >> >> > I need to process a bunch of images in very little time. As a >> >> > result, >> >> > I >> >> > have decided to try and parallelise multiple processes. I knew this >> >> > was >> >> not >> >> > going to be trivial, but now I've narrowed down the problem and I'm >> >> > completely stuck. The problem begins when I try to do convolution: >> the >> >> > importation of the image into a piddle. >> >> > >> >> > I have written a small test program pasted at the foot of this >> >> > email. >> >> > In >> >> > the code there are two clusters of commented out code. The first >> >> convolves >> >> > an image with PDL::ImageND::convolveND while the second convolves an >> >> image >> >> > with PDL::Image2D::conv2d. You may try uncommenting one cluster of >> >> > code >> >> at >> >> > a time. >> >> > >> >> > When I try to import a small png (300x255), all four threads return >> >> within >> >> > the allotted 30 seconds regardless of the import method used. >> However, >> >> when >> >> > a lager image is used (2740x1818, about the size needed for my >> >> application), >> >> > the problems occur. Using conv2D, the programs exhibits no problems >> and >> >> > behaves as before. Using convolveND, the program exits well before >> the >> >> 30 >> >> > seconds are up and before all 4 threads have returned. This seems >> >> > to >> >> > indicate some sort of problem, but without any error messages, it is >> >> > difficult to tell what is going on... >> >> > >> >> > Here are links to the images I used: >> >> > small image: http://imageshack.us/a/img23/9039/ballr.png >> >> > large image (yes, it is blank): >> >> > http://imageshack.us/a/img138/3980/atestblank.png >> >> > >> >> > I am using Perl 5.12.2 on a virtualised installation of XP. >> >> > >> >> > See code below. >> >> > >> >> > Thanks, >> >> > Perldough >> >> > >> >> > <code> >> >> > use PDL; >> >> > use threads; >> >> > >> >> > my $thread1 = threads->create(\&analysis_A); # start thread >> >> > running >> >> > my $thread2 = threads->create(\&analysis_A); >> >> > my $thread3 = threads->create(\&analysis_A); >> >> > my $thread4 = threads->create(\&analysis_A); >> >> > >> >> > sleep 30; >> >> > >> >> > sub analysis_A >> >> > { >> >> > require PDL::IO::GD; >> >> > my $im = PDL::IO::GD->new( {filename => 'A_test-blank.png'} >> >> > ); >> >> > my $piddle = $im->to_pdl(); >> >> > >> >> > #importation method 1 >> >> > #require PDL::ImageND; >> >> > #my $new = PDL::ImageND::convolveND( $piddle, ones(5,5)); # >> >> > Works >> >> > in >> >> 1 >> >> > thread, fails in multiple >> >> > >> >> > #importation method 2 >> >> > #require PDL::Image2D; >> >> > #my $new = PDL::Image2D::conv2d( $piddle, ones(5,5)); # >> >> > Works >> >> > in >> >> > multiple threads >> >> > >> >> > my $tid = threads->tid(); >> >> > print "thread $tid: Done \n"; >> >> > } >> >> > </code> >> >> > >> >> > _______________________________________________ >> >> > Perldl mailing list >> >> > [email protected] >> >> > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl >> >> > >> >> >> > >> > _______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
