Re: Parallel processing on shared data structures

2009-03-20 Thread Hendrik van Rooyen
psaff...@lemail.com wrote: I'm filing 160 million data points into a set of bins based on their position. At the moment, this takes just over an hour using interval So why do you not make four sets of bins - one for each core of your quad, and split the points into quarters, and run four

Parallel processing on shared data structures

2009-03-19 Thread psaff...@googlemail.com
I'm filing 160 million data points into a set of bins based on their position. At the moment, this takes just over an hour using interval trees. I would like to parallelise this to take advantage of my quad core machine. I have some experience of Parallel Python, but PP seems to only really work

Re: Parallel processing on shared data structures

2009-03-19 Thread MRAB
psaff...@googlemail.com wrote: I'm filing 160 million data points into a set of bins based on their position. At the moment, this takes just over an hour using interval trees. I would like to parallelise this to take advantage of my quad core machine. I have some experience of Parallel Python,