Re: Fwd: [RFC]serialize the output of parallel make?
On Tue, Aug 3, 2010 at 2:51 PM, Tim Murphy wrote: > > Since some things happen at the same time there is no single "serial > order". The semaphore mechanism, forces one of the possible orders. > I'm not familiar with source code of make, but I believe the "serial order" of shells is determined by the dependence DAG, it may be unique for a given dependence DAG. Shells can be issued and completed at random order(only need satisfying the dependence relation). But make can print their outputs strictly in their "serial order". -- Chiheng Xu Wuhan,China ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: Fwd: [RFC]serialize the output of parallel make?
Chiheng Xu wrote: On Tue, Aug 3, 2010 at 2:51 PM, Tim Murphy wrote: Since some things happen at the same time there is no single "serial order". The semaphore mechanism, forces one of the possible orders. I'm not familiar with source code of make, but I believe the "serial order" of shells is determined by the dependence DAG, it may be unique for a given dependence DAG. Shells can be issued and completed at random order(only need satisfying the dependence relation). But make can print their outputs strictly in their "serial order". I'm trying very hard to only provide constructive comments in response to this thread, but frankly this is, in a word, stupid. If you want make's output to be in serial order, then don't use parallel make at all. The point to parallel make is that it allows jobs which have no ordering dependency to run in parallel. If you want their output to be fully serialized, then you will force make to wait for them to complete serially. Which automatically also means that make will have to maintain an arbitrarily large internal queue for all of the output, because given the unpredictable completion times of multiple jobs running concurrently, no output can be emitted until the slowest parallel job completes. In particular, if you have recursive makefiles, no parent make process can output anything at all until all of its submakes have completed, because no individual make process has enough knowledge about what the actual serial order is. Given that this discussion seems to have arisen due to the braindead stdio handling in Cygwin, it seems like any de-mangling of parallel make's output should be directed to the Cygwin libraries. In my experience Cygwin is too slow an environment to be useful anyway, which is why I use MSYS for Windows builds. But I have to admit, I only use it inside a single-core VirtualBox these days so I haven't looked at how parallel make behaves there. But the fact is all I/O in Cygwin is funneled through the Cygwin DLL, so there's no reason that it can't be fixed to not mingle/mangle lines from different processes together. But again, that's not gnu-make's problem, that's a Cygwin issue. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: Fwd: [RFC]serialize the output of parallel make?
> Date: Tue, 3 Aug 2010 07:51:22 +0100 > From: Tim Murphy > Cc: e...@opera.com, bug-make@gnu.org > > mytarget: > ->command1 && > ->command2 && > ->command3 > > > Note that I'm using bash syntax here. On windows if you want to use > cmd.exe then good luck - I don't think it's really fit for purpose. cmd.exe supports the same `command1 && command2' semantics as does Bash, so there's no problem here and no need for any ``luck''. ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: Fwd: [RFC]serialize the output of parallel make?
Chiheng Xu wrote: What I want is transparent "parallel make". Make can issue multiple shells simultaneously, but print their outputs in the same order as in a serial make. ElectricAccelerator is a gmake replacement that does exactly this. I wrote about this feature a while back: http://blog.electric-cloud.com/2008/12/01/untangling-parallel-build-logs/ You can read more about Accelerator on the blog, or here: http://www.electric-cloud.com/products/electricaccelerator.php Eric Melski Architect Electric Cloud, Inc. http://blog.electric-cloud.com/ ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: Fwd: [RFC]serialize the output of parallel make?
Chiheng Xu wrote: What I want is transparent "parallel make". Make can issue multiple shells simultaneously, but print their outputs in the same order as in a serial make. ElectricAccelerator is a gmake replacement that does exactly this. I wrote about this feature a while back: http://blog.electric-cloud.com/2008/12/01/untangling-parallel-build-logs/ You can read more about Accelerator on the blog, or here: http://www.electric-cloud.com/products/electricaccelerator.php Eric Melski Architect Electric Cloud, Inc. http://blog.electric-cloud.com/ ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: Fwd: [RFC]serialize the output of parallel make?
On Wed, Aug 4, 2010 at 10:41 AM, Eric Melski wrote: > ElectricAccelerator is a gmake replacement that does exactly this. I wrote > about this feature a while back: > > http://blog.electric-cloud.com/2008/12/01/untangling-parallel-build-logs/ > > You can read more about Accelerator on the blog, or here: > > http://www.electric-cloud.com/products/electricaccelerator.php > > Eric Melski > Architect > Electric Cloud, Inc. > http://blog.electric-cloud.com/ > Excellent ! -- Chiheng Xu Wuhan,China ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
