Christian Heimes wrote: > David Cournapeau wrote: > >> No, and it never will. Parallel builds requires to build with >> dependency handling. Even make does not handle it well: it works most >> of the time by accident, but there are numerous problems (try for >> example building lapack with make -j8 on your 8 cores machine - it >> will give a bogus library 90 % of the time, because it starts building >> static library with ar while some object files are still built). >> > > You may call me naive and ignorant. Is it really that hard to archive > some kind of poor man's concurrency? You don't have to parallelize > everything to get a speed up on multi core machines. Usually the compile > process from C/C++ file to an object files takes up most of the time. > > How about > > * assemble a list of all C/C++ source files of all extensions. > * compile all source files in parallel > * do the rest (linking etc.) in serial >
That's more or less how make works - it does not work very well IMHO. And doing the above correctly in distutils may be harder than it seems: both scons and waf had numerous problems with calling subtasks because of race conditions in subprocess for example (both have their own module for that). More fundamentally though, I have no interest in working on distutils. Not working on a DAG is fundamentally and hopelessly broken for a build tool, and this is unfixable in distutils. Everything is wrong, from the concepts to the UI through the implementation, to paraphrase a famous saying. There is nothing to save IMHO. Of course, someone else can work on it. I prefer working on a sane solution myself, cheers, David _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
