On 7/21/12 6:17 AM, Gilles Sadowski wrote: > Hi. > > My previous post (with subject "Synchronisation") made me think (again) that > it might be useful to start considering how to take advantage of > multi-threading in Commons Math. > Indeed, it seems that some parts of the library might end up not being used > anymore because their performance simply cannot match competing > implementations that do benefit form parallelization. [The recent example > that comes to mind is the FFT.]
This is an interesting question. I am also -1 on adding dependencies, but it would be a good idea to look at how others have solved the problem of how to support parallel execution by multiple threads without managing threads directly. Lots of [math] algorithms could be parallelized. The question is how to effectively coordinate the work without owning or creating the workers. I would be -0 to any suggestion that involved [math] itself spawning threads, since that 0) creates management headeaches 1) may violate some container contracts and 2) forces execution threads to be in the same process. I think it is worth thinking about how we might support parallel execution by externally managed workers. An obvious thing to look at is how to break our parallelizable algorithms into pieces that could be executed in Hadoop Map/Reduce jobs. Step 0) is the breaking up part. Then step 1) might be either some examples added to the user guide or custom Pig functions (or examples of how to code them). Phil > > > Best regards, > Gilles > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
