Alright, so I've been playing around with pthreads and I've run into some troubles there too...
I am using *get_autopthread_actual()* to check whether the last operation was threaded or not. When the last operation is a simple addition, * get_autopthread_actual()** *yields a non-zero value (indicating that operation was pthreaded). However, when the last operation is one of my convolution functions, the returned value is 0 (indicating that the operation was not pthreaded). I'm not sure why the convolution operations appear to not get pthreaded. I have posted my code at the foot of this email. There are three clusters of commented code. Feel free to uncomment any one of the three at a time to see what I mean. Thanks, Afsheen <code> use PDL; # Set target of 4 parallel pthreads to create # pthreads should not be used for piddles with a size smaller than 1Meg. set_autopthread_targ(4); set_autopthread_size(1); my $big = ones(5000,5000); #my $sum = $big + $big; #require PDL::ImageND; #my $new = PDL::ImageND::convolveND( $big, ones(5,5)); #require PDL::Image2D; #my $new = PDL::Image2D::conv2d( $big, ones(5,5)); # Get the actual number of pthreads for the last processing operation. $actualPthreads = get_autopthread_actual(); print "$actualPthreads\n"; </code> 2012/10/12 Chris Marshall <[email protected]> > I'm not familiar with ithreads to help but one > thought would be to use an optimized 2d > spatial convolution specific to your problem. > > While PDL computations are done with C code, > the generality and flexibility does come at a > cost. If you need peak performance, wrapping > a specific C convolution routine should offer > improved performance. > > Good luck, > Chris > > On 10/11/12, Grasswistle <[email protected]> wrote: > > I already did the benchmarking exercise and found convolution to be the > > real time hog. I'm going to continue playing with PDL::ParallelCPU. > > > > Nonetheless, any information helping me to get convolveND to work with > > conventional threading in the original code is greatly appreciated. > > > > Thanks, > > Perldough > > 2012/10/11 Chris Marshall <[email protected]> > > > >> perldoc PDL::ParallelCPU gives the documentation > >> on auto-parallel threads it is not a module. PDL > >> uses 32bit indexing so the maximum size of a piddle > >> is <2**32. > >> > >> Also, consider using profiling and/or benchmarking > >> to measure where things need to be sped up in > >> your code. With the example of your code, speeding > >> up file IO could be more important than speeding > >> up the convolutions. > >> > >> --Chris > >> > >> On 10/10/12, Grasswistle <[email protected]> wrote: > >> > Hello Chris, > >> > > >> > I am using PDL v2.4.10, as reported by *perl -MPDL -e "print > >> > \"$PDL::VERSION\";"** * The command use PDL::parallelCPU* *fails. > for > >> me > >> > though. Is there another more recent version of PDL I am not aware > >> of? > >> > > >> > Also, I monitored the system memory while I was running the program > and > >> > didn't find that the memory usage got to be too high. Is there a > >> > maximum > >> > on the memory that Perl/PDL can consume? How do I go about changing > >> that? > >> > > >> > Thanks, > >> > Perldough > >> > > >> > 2012/10/10 Chris Marshall <[email protected]> > >> > > >> >> Hi Grasswistle- > >> >> > >> >> Maybe there is a possible ithread-safety issue going > >> >> on but my guess would be you are running out of > >> >> memory which will make perl exit. > >> >> > >> >> Also, you don't say what version of PDL you are using, > >> >> but the current release has support for automatic threadloop > >> >> parallelization across processors. Take a look at the > >> >> docs for PDL::ParallelCPU. > >> >> > >> >> --Chris > >> >> > >> >> On Wed, Oct 10, 2012 at 3:33 PM, Grasswistle <[email protected]> > >> >> wrote: > >> >> > Hello, > >> >> > > >> >> > I need to process a bunch of images in very little time. As a > >> >> > result, > >> >> > I > >> >> > have decided to try and parallelise multiple processes. I knew > this > >> >> > was > >> >> not > >> >> > going to be trivial, but now I've narrowed down the problem and I'm > >> >> > completely stuck. The problem begins when I try to do convolution: > >> the > >> >> > importation of the image into a piddle. > >> >> > > >> >> > I have written a small test program pasted at the foot of this > >> >> > email. > >> >> > In > >> >> > the code there are two clusters of commented out code. The first > >> >> convolves > >> >> > an image with PDL::ImageND::convolveND while the second convolves > an > >> >> image > >> >> > with PDL::Image2D::conv2d. You may try uncommenting one cluster of > >> >> > code > >> >> at > >> >> > a time. > >> >> > > >> >> > When I try to import a small png (300x255), all four threads return > >> >> within > >> >> > the allotted 30 seconds regardless of the import method used. > >> However, > >> >> when > >> >> > a lager image is used (2740x1818, about the size needed for my > >> >> application), > >> >> > the problems occur. Using conv2D, the programs exhibits no problems > >> and > >> >> > behaves as before. Using convolveND, the program exits well before > >> the > >> >> 30 > >> >> > seconds are up and before all 4 threads have returned. This seems > >> >> > to > >> >> > indicate some sort of problem, but without any error messages, it > is > >> >> > difficult to tell what is going on... > >> >> > > >> >> > Here are links to the images I used: > >> >> > small image: http://imageshack.us/a/img23/9039/ballr.png > >> >> > large image (yes, it is blank): > >> >> > http://imageshack.us/a/img138/3980/atestblank.png > >> >> > > >> >> > I am using Perl 5.12.2 on a virtualised installation of XP. > >> >> > > >> >> > See code below. > >> >> > > >> >> > Thanks, > >> >> > Perldough > >> >> > > >> >> > <code> > >> >> > use PDL; > >> >> > use threads; > >> >> > > >> >> > my $thread1 = threads->create(\&analysis_A); # start thread > >> >> > running > >> >> > my $thread2 = threads->create(\&analysis_A); > >> >> > my $thread3 = threads->create(\&analysis_A); > >> >> > my $thread4 = threads->create(\&analysis_A); > >> >> > > >> >> > sleep 30; > >> >> > > >> >> > sub analysis_A > >> >> > { > >> >> > require PDL::IO::GD; > >> >> > my $im = PDL::IO::GD->new( {filename => 'A_test-blank.png'} > >> >> > ); > >> >> > my $piddle = $im->to_pdl(); > >> >> > > >> >> > #importation method 1 > >> >> > #require PDL::ImageND; > >> >> > #my $new = PDL::ImageND::convolveND( $piddle, ones(5,5)); # > >> >> > Works > >> >> > in > >> >> 1 > >> >> > thread, fails in multiple > >> >> > > >> >> > #importation method 2 > >> >> > #require PDL::Image2D; > >> >> > #my $new = PDL::Image2D::conv2d( $piddle, ones(5,5)); # > >> >> > Works > >> >> > in > >> >> > multiple threads > >> >> > > >> >> > my $tid = threads->tid(); > >> >> > print "thread $tid: Done \n"; > >> >> > } > >> >> > </code> > >> >> > > >> >> > _______________________________________________ > >> >> > Perldl mailing list > >> >> > [email protected] > >> >> > http://mailman.jach.hawaii.edu/mailman/listinfo/perldl > >> >> > > >> >> > >> > > >> > > >
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
