Grasswistle - You said you needed to process lots of images in "very little time" at the start of this email thread. Does this mean you were facing a deadline, or that you need high throughput? You also said that you needed a convolution, but you didn't say what sort. In particular, all your code works with images, which are two-dimensional. Are you trying to do 2D convolutions, or higher dimensional ones? If higher dimensional ones, I suspect that convolveND allocates some work space, but it is not thread-specific work space; as the first thread exits, it frees the memory so that the other threads suffer a segmentation fault when they try to use this global memory that is no longer allocated. I bet that if we modified the code so that it created thread-specific work space, it would work, but it's not clear to me, having glanced through the code, where those modifications need to be made.
Performing 2D convolutions on many images is the sort of problem that is considered "embarrassingly parallel". That is, parallelizing the code is simply a matter of sending the same job with different data to different processors. When parallelized, the processing time is cut down by the number of processors at hand, so instead of taking T time, it takes T/N time (N being the number of processors). Not all problems are embarrassingly parallel. In particular, scaling algorithms to very large sets of data across many processors is typically *not* embarrassingly parallel and require careful thought to do correctly. At any rate, if you are performing 2D convolutions on images that fit well within memory, it seems to me that you have two options, both of which have already been discussed in this thread. The first is to spawn many Perl threads using Perl's "threads" module and have each of them perform 2D convolutions on your images. In doing this, you would parallelize your code by hand. Whatever you do, you should not try to share a piddle across multiple threads. This is not supported with normal PDL, and unfortunately it does not emit any sensible warnings if you try to do this. (That's something I've meant to work on since this summer, when I wrote PDL::Parallel::threads <https://github.com/run4flat/PDL-Parallel-threads>.) The other solution is to load all the images into a single 3D piddle (or batches at a time, if there is not enough memory to load them all at once), and calling conv2d as Chris describes to get the the pthread parallelization automatically. So, everything I've said in the above paragraphs applies to two-dimensional convolutions. If you actually need higher-dimensional convolutions, I'll have to reconsider. David -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." -- Brian Kernighan
_______________________________________________ Perldl mailing list [email protected] http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
