On Tuesday 25 January 2011, Ralf Wildenhues wrote: > * Stefano Lattarini wrote on Tue, Jan 25, 2011 at 11:02:11AM CET: > > So, what about the attached updated patch? > > I have some nits, otherwise I'll let Eric approve this. > I agree with most of your nits, and I've amended the patch accordingly. See my replies in-line below.
> > Subject: [PATCH] docs: another parallel make issue > > > > * doc/autoconf.texi (Parallel Make): Document that some make > > implementations, when run in parallel mode, connect stdout and > > stderr of child processes to pipes or temporary files, and might > > re-route stderr of spawned processes to stout. Also document > > that FreeBSD make in parallel mode reuses the same shell for > > multiple commands within one recipe (like NetBSD make does). > > > --- a/doc/autoconf.texi > > +++ b/doc/autoconf.texi > > @@ -19475,13 +19475,52 @@ sh > > @cindex Parallel @command{make} > > > > Support for parallel execution in @command{make} implementation varies. > > -Generally, using GNU make is your best bet. When NetBSD > > -@command{make} is invoked with @option{-j@var{N}}, it will reuse the > > -same shell for multiple commands within one recipe. This can have > > -unexpected consequences.@footnote{Note that GNU make has > > -heuristics to avoid spawning a shell at all if the command is deemed > > -safe to be executed directly.} For example, change of directories or > > -variables persist between commands: > > +Generally, using GNU make is your best bet. > > + > > +Some make implementations (among them, FreeBSD @command{make}, NetBSD > > +@command{make}, and Solaris @command{dmake}), when invoked with a > > +@option{-j@var{N}} option, connect the standard output and standard > > +error of all their children processes to pipes or temporary regular > > s/children/child/ (hasn't this been reviewed already?) > Yes :-( Sorry, stupid copy & paste blunder. > > +files. This can lead to slightly but subtly different semantics in > > s/slightly but// > Fixed. > > +the behaviour of the spawned processes. For example, even if the > > +@command{make} standard output is a connected to a tty, a command > > s/is a/is/ > Fixed. > > +like @code{test -t 1} issued in a recipe will return a failure within > > +those make implementations: > > How about: ... is connected to a tty, the recipe commands will not be: > Nice. I've taken it. > > +@example > > +$ @kbd{cat Makefile} > > +all: > > + @@test -t 1 && echo "Is a tty" || echo "Is not a tty" > > +$ @kbd{make -j 2} # FreeBSD make > > +Is not a tty > > +$ @kbd{make -j 2} # NetBSD make > > +--- all --- > > +Is not a tty > > +$ @kbd{dmake -j 2} # Solaris dmake > > +@var{hostname} --> 1 job > > +@var{hostname} --> Job output > > +Is not a tty > > +@end example > > + > > +@noindent > > +while on the other hand: > > + > > +@example > > +$ @kbd{make -j 2} # GNU make, Heirloom make > > +Is a tty > > +@end example > > + > > +Furthermore, parallel run of those @command{make} implementations will > > "a parallel run" or "parallel runs" > I've gone for the second one. > > +route standard error from commands that they spawn into their own > > s/that they spawn // > That was pre-existing; but I've fixed it anyway. > > +standard output, and may remove leading whitespace from output lines. > > + > > +And there's more about NetBSD or FreeBSD @command{make}. When run in > > This sentence is newspaper style. How about just > When NetBSD or FreeBSD @command{make} are run in parallel mode, ... > Fine with me. > > +parallel mode, they will reuse the same shell for multiple commands > > +within one recipe. This can have unexpected > > +consequences.@footnote{Note that GNU make has heuristics to avoid > > +spawning a shell at all if the command is deemed safe to be executed > > +directly.} For example, change of directories or variables persist > > +between commands: > > Footnotes are really ugly, we should think of them as costing 5 bucks or > so each time we use them. > This particular footnote was pre-existing. > How about just omitting the footnote here, > since this isn't about GNU make otherwise at all? > Agreed. Footnote removed. > Also, I don't know what the immediate connection here is. > > FWIW, newish GNU make supports .ONESHELL for reusing shells between > recipe commands. > > > @example > > all: > > @@ -19518,12 +19557,9 @@ Another consequence of this is that, if one > > command in a recipe uses > > @code{exit 0} to indicate a successful exit, the shell will be gone > > and the remaining commands of this recipe will not be executed. > > > > -The above example also shows additional status output NetBSD > > -@command{make} produces in parallel mode for targets being updated. > > - > > -Furthermore, parallel NetBSD @command{make} will route standard error > > -from commands that it spawns into its own standard output, and may > > -remove leading whitespace from output lines. > > +The above example also shows additional status output produced by NetBSD > > +@command{make}@footnote{but @emph{not} by FreeBSD @command{make}} in > > I'd just replace the footnote with the same text in parentheses. > Fine with me. > > +parallel mode for targets being updated. > > > > You can avoid these issues by using the @option{-B} option to enable > > compatibility semantics. However, that will effectively also disable > > Cheers, > Ralf > The updated patch is attached. Regards, Stefano
From 36f9129c2daa538d169e84aa3169f9deb1b11a93 Mon Sep 17 00:00:00 2001 From: Stefano Lattarini <stefano.lattar...@gmail.com> Date: Mon, 24 Jan 2011 22:31:37 +0100 Subject: [PATCH] docs: another parallel make issue * doc/autoconf.texi (Parallel Make): Document that some make implementations, when run in parallel mode, connect stdout and stderr of child processes to pipes or temporary files, and might re-route stderr of spawned processes to stout. Also document that FreeBSD make in parallel mode reuses the same shell for multiple commands within one recipe (like NetBSD make does). --- ChangeLog | 10 +++++++++ doc/autoconf.texi | 55 +++++++++++++++++++++++++++++++++++++++++----------- 2 files changed, 53 insertions(+), 12 deletions(-) diff --git a/ChangeLog b/ChangeLog index 2fc5ab3..5067399 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,13 @@ +2011-01-25 Stefano Lattarini <stefano.lattar...@gmail.com> + + docs: another parallel make issue + * doc/autoconf.texi (Parallel Make): Document that some make + implementations, when run in parallel mode, connect stdout and + stderr of child processes to pipes or temporary files, and might + re-route stderr of spawned processes to stout. Also document + that FreeBSD make in parallel mode reuses the same shell for + multiple commands within one recipe (like NetBSD make does). + 2011-01-23 Ralf Wildenhues <ralf.wildenh...@gmx.de> docs: new section about whitespace trimmmed from make command-lines. diff --git a/doc/autoconf.texi b/doc/autoconf.texi index ae601a0..151b70f 100644 --- a/doc/autoconf.texi +++ b/doc/autoconf.texi @@ -19475,12 +19475,46 @@ sh @cindex Parallel @command{make} Support for parallel execution in @command{make} implementation varies. -Generally, using GNU make is your best bet. When NetBSD -@command{make} is invoked with @option{-j@var{N}}, it will reuse the -same shell for multiple commands within one recipe. This can have -unexpected consequences.@footnote{Note that GNU make has -heuristics to avoid spawning a shell at all if the command is deemed -safe to be executed directly.} For example, change of directories or +Generally, using GNU make is your best bet. + +Some make implementations (among them, FreeBSD @command{make}, NetBSD +@command{make}, and Solaris @command{dmake}), when invoked with a +@option{-j@var{N}} option, connect the standard output and standard +error of all their child processes to pipes or temporary regular +files. This can lead to subtly different semantics in the behaviour +of the spawned processes. For example, even if the @command{make} +standard output is connected to a tty, the recipe command will not be: + +@example +$ @kbd{cat Makefile} +all: + @@test -t 1 && echo "Is a tty" || echo "Is not a tty" +$ @kbd{make -j 2} # FreeBSD make +Is not a tty +$ @kbd{make -j 2} # NetBSD make +--- all --- +Is not a tty +$ @kbd{dmake -j 2} # Solaris dmake +@var{hostname} --> 1 job +@var{hostname} --> Job output +Is not a tty +@end example + +@noindent +On the other hand: + +@example +$ @kbd{make -j 2} # GNU make, Heirloom make +Is a tty +@end example + +Furthermore, parallel runs of those @command{make} implementations will +route standard error from commands that they spawn into their own +standard output, and may remove leading whitespace from output lines. + +When NetBSD or FreeBSD @command{make} are run in parallel mode, they will +reuse the same shell for multiple commands within one recipe. This can +have unexpected consequences. For example, change of directories or variables persist between commands: @example @@ -19518,12 +19552,9 @@ Another consequence of this is that, if one command in a recipe uses @code{exit 0} to indicate a successful exit, the shell will be gone and the remaining commands of this recipe will not be executed. -The above example also shows additional status output NetBSD -@command{make} produces in parallel mode for targets being updated. - -Furthermore, parallel NetBSD @command{make} will route standard error -from commands that it spawns into its own standard output, and may -remove leading whitespace from output lines. +The above example also shows additional status output produced by NetBSD +@command{make} (but @emph{not} by FreeBSD @command{make}) in parallel +mode for targets being updated. You can avoid these issues by using the @option{-B} option to enable compatibility semantics. However, that will effectively also disable -- 1.7.2.3