Parallelization is across thread dimensions. See the PDL Book for explanations of how to determine and control thread dimensions: http://pdl.perl.org/content/pdl-book-toc.html
If you want to have parallel operation of a 2D convolution. you need to operate on a number of images that is a multiple of your number of pthreads, e.g. $im4 = random(640,480,4); $kern = ones(3,3); $out4 = conv2d $im4, $kern; should parallelize on 4 threads is the size is large enough. --Chris On 10/12/12, Grasswistle <[email protected]> wrote: > Alright, so I've been playing around with pthreads and I've run into some > troubles there too... > > I am using *get_autopthread_actual()* to check whether the last operation > was threaded or not. When the last operation is a simple addition, * > get_autopthread_actual()** *yields a non-zero value (indicating that > operation was pthreaded). However, when the last operation is one of my > convolution functions, the returned value is 0 (indicating that the > operation was not pthreaded). > > I'm not sure why the convolution operations appear to not get pthreaded. > > I have posted my code at the foot of this email. There are three clusters > of commented code. Feel free to uncomment any one of the three at a time > to see what I mean. > > Thanks, > Afsheen > > <code> > use PDL; > > # Set target of 4 parallel pthreads to create > # pthreads should not be used for piddles with a size smaller than 1Meg. > set_autopthread_targ(4); > set_autopthread_size(1); > > my $big = ones(5000,5000); > > #my $sum = $big + $big; > > #require PDL::ImageND; > #my $new = PDL::ImageND::convolveND( $big, ones(5,5)); > > #require PDL::Image2D; > #my $new = PDL::Image2D::conv2d( $big, ones(5,5)); > > # Get the actual number of pthreads for the last processing operation. > $actualPthreads = get_autopthread_actual(); > print "$actualPthreads\n"; > </code> > > 2012/10/12 Chris Marshall <[email protected]> > >> I'm not familiar with ithreads to help but one >> thought would be to use an optimized 2d >> spatial convolution specific to your problem. >> >> While PDL computations are done with C code, >> the generality and flexibility does come at a >> cost. If you need peak performance, wrapping >> a specific C convolution routine should offer >> improved performance. >> >> Good luck, >> Chris >> >> On 10/11/12, Grasswistle <[email protected]> wrote: >> > I already did the benchmarking exercise and found convolution to be the >> > real time hog. I'm going to continue playing with PDL::ParallelCPU. >> > >> > Nonetheless, any information helping me to get convolveND to work with >> > conventional threading in the original code is greatly appreciated. >> > >> > Thanks, >> > Perldough >> > 2012/10/11 Chris Marshall <[email protected]> >> > >> >> perldoc PDL::ParallelCPU gives the documentation >> >> on auto-parallel threads it is not a module. PDL >> >> uses 32bit indexing so the maximum size of a piddle >> >> is <2**32. >> >> >> >> Also, consider using profiling and/or benchmarking >> >> to measure where things need to be sped up in >> >> your code. With the example of your code, speeding >> >> up file IO could be more important than speeding >> >> up the convolutions. >> >> >> >> --Chris >> >> >> >> On 10/10/12, Grasswistle <[email protected]> wrote: >> >> > Hello Chris, >> >> > >> >> > I am using PDL v2.4.10, as reported by *perl -MPDL -e "print >> >> > \"$PDL::VERSION\";"** * The command use PDL::parallelCPU* *fails. >> for >> >> me >> >> > though. Is there another more recent version of PDL I am not aware >> >> of? >> >> > >> >> > Also, I monitored the system memory while I was running the program >> and >> >> > didn't find that the memory usage got to be too high. Is there a >> >> > maximum >> >> > on the memory that Perl/PDL can consume? How do I go about changing >> >> that? >> >> > >> >> > Thanks, >> >> > Perldough >> >> > >> >> > 2012/10/10 Chris Marshall <[email protected]> >> >> > >> >> >> Hi Grasswistle- >> >> >> >> >> >> Maybe there is a possible ithread-safety issue going >> >> >> on but my guess would be you are running out of >> >> >> memory which will make perl exit. >> >> >> >> >> >> Also, you don't say what version of PDL you are using, >> >> >> but the current release has support for automatic threadloop >> >> >> parallelization across processors. Take a look at the >> >> >> docs for PDL::ParallelCPU. >> >> >> >> >> >> --Chris >> >> >> >> >> >> On Wed, Oct 10, 2012 at 3:33 PM, Grasswistle >> >> >> <[email protected]> >> >> >> wrote: >> >> >> > Hello, >> >> >> > >> >> >> > I need to process a bunch of images in very little time. As a >> >> >> > result, >> >> >> > I >> >> >> > have decided to try and parallelise multiple processes. I knew >> this >> >> >> > was >> >> >> not >> >> >> > going to be trivial, but now I've narrowed down the problem and >> >> >> > I'm >> >> >> > completely stuck. The problem begins when I try to do >> >> >> > convolution: >> >> the >> >> >> > importation of the image into a piddle. >> >> >> > >> >> >> > I have written a small test program pasted at the foot of this >> >> >> > email. >> >> >> > In >> >> >> > the code there are two clusters of commented out code. The first >> >> >> convolves >> >> >> > an image with PDL::ImageND::convolveND while the second convolves >> an >> >> >> image >> >> >> > with PDL::Image2D::conv2d. You may try uncommenting one cluster >> >> >> > of >> >> >> > code >> >> >> at >> >> >> > a time. >> >> >> > >> >> >> > When I try to import a small png (300x255), all four threads >> >> >> > return >> >> >> within >> >> >> > the allotted 30 seconds regardless of the import method used. >> >> However, >> >> >> when >> >> >> > a lager image is used (2740x1818, about the size needed for my >> >> >> application), >> >> >> > the problems occur. Using conv2D, the programs exhibits no >> >> >> > problems >> >> and >> >> >> > behaves as before. Using convolveND, the program exits well >> >> >> > before >> >> the >> >> >> 30 >> >> >> > seconds are up and before all 4 threads have returned. This >> >> >> > seems >> >> >> > to >> >> >> > indicate some sort of problem, but without any error messages, it >> is >> >> >> > difficult to tell what is going on... >> >> >> > >> >> >> > Here are links to the images I used: >> >> >> > small image: http://imageshack.us/a/img23/9039/ballr.png >> >> >> > large image (yes, it is blank): >> >> >> > http://imageshack.us/a/img138/3980/atestblank.png >> >> >> > >> >> >> > I am using Perl 5.12.2 on a virtualised installation of XP. >> >> >> > >> >> >> > See code below. >> >> >> > >> >> >> > Thanks, >> >> >> > Perldough >> >> >> > >> >> >> > <code> >> >> >> > use PDL; >> >> >> > use threads; >> >> >> > >> >> >> > my $thread1 = threads->create(\&analysis_A); # start thread >> >> >> > running >> >> >> > my $thread2 = threads->create(\&analysis_A); >> >> >> > my $thread3 = threads->create(\&analysis_A); >> >> >> > my $thread4 = threads->create(\&analysis_A); >> >> >> > >> >> >> > sleep 30; >> >> >> > >> >> >> > sub analysis_A >> >> >> > { >> >> >> > require PDL::IO::GD; >> >> >> > my $im = PDL::IO::GD->new( {filename => >> >> >> > 'A_test-blank.png'} >> >> >> > ); >> >> >> > my $piddle = $im->to_pdl(); >> >> >> > >> >> >> > #importation method 1 >> >> >> > #require PDL::ImageND; >> >> >> > #my $new = PDL::ImageND::convolveND( $piddle, ones(5,5)); # >> >> >> > Works >> >> >> > in >> >> >> 1 >> >> >> > thread, fails in multiple >> >> >> > >> >> >> > #importation method 2 >> >> >> > #require PDL::Image2D; >> >> >> > #my $new = PDL::Image2D::conv2d( $piddle, ones(5,5)); # >> >> >> > Works >> >> >> > in >> >> >> > multiple threads >> >> >> > >> >> >> > my $tid = threads->tid(); >> >> >> > print "thread $tid: Done \n"; >> >> >> > } >> >> >> > </code> >> >> >> > >> >> >> > _______________________________________________ >> >> >> > Perldl mailing list >> >> >> > [email protected] >> >> >> > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl >> >> >> > >> >> >> >> >> > >> >> >> > >> > _______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
