Re: question on an embarrassingly parallelism

2019-02-05 Thread Alex Guziel
The scheduler isn't guaranteed to compute them in that order to maximize parallelism. You can imagine in the case where m = n -1, that it just computes the m branches in parallel, then it has to complete the nth branch with parallelism 1. On Tue, Feb 5, 2019 at 7:20 AM soma dhavala wrote: > Imag

question on an embarrassingly parallelism

2019-02-05 Thread soma dhavala
Imagine that you have celery (or another cluster) executor with “m” workers (with equal resources) Say, I’ve the following dad [0]<—[1a]<—[1b]<— [1c] [0]<—[2a]<—[2b]<— [2c] .. [0]<—[n1]<—[n2]<— [n3] In the above, [0] node is the parent of “n” identical branches. Suppose [0]’s computational time