Hello autoconfers and automakers. In both FreeBSD and NetBSD make, when "parallel mode" is used (e.g., with `make -j2'), the stdout and stderr of child processes get connected to pipes[1][2]. Similarly, in Solaris 10 dmake, the stdout and stderr of child processes get connected to temporary files[3].
IMHO, the developers of a software package should be fully aware of this behaviour when writing their makefiles, since it might cause some small yet potentially subtle semantic differences in the behaviour of the spwaned processes. Here is a concrete example of how such semantic differences can bite: the "colored output" from automake-generated testsuites[4] doesn't work out-of-the box with NetBSD and FreeBSD make run in "parallel mode", even when those make instances are connected to a capable terminal[5][6]. So, what about the attached documentation patches (one for the automake maint branch, the other for the autoconf master branch)? Do you think they could be useful? Regards, Stefano -*-*- "FOOTNOTES" [1] I *think* this is done in order to avoid to intermingle output from different processes, to some degree at least. For example: $ cat Makefile all: a b a: @sleep 0.1; printf a; sleep 0.1; printf a; sleep 0.1; echo b: @sleep 0.15; printf b; sleep 0.1; echo b $ gmake -j2 abab $ heirloom-make -j2 abab $ freebsd-make -j2 bb aa $ netbsd-make -j2 --- a --- --- b --- bb --- a --- aa [2] This can be demonstrated (on Linux) with the following: $ echo 'all: ; ls -l /proc/$$$$/fd' > Makefile $ gmake -j 2 ls -l /proc/$$/fd total 0 lrwx------ 1 stefano stefano 64 Jan 24 22:49 0 -> /dev/pts/4 lrwx------ 1 stefano stefano 64 Jan 24 22:49 1 -> /dev/pts/4 lrwx------ 1 stefano stefano 64 Jan 24 22:49 2 -> /dev/pts/4 lr-x------ 1 stefano stefano 64 Jan 24 22:49 3 -> /proc/14727/fd $ heirloom-make -j 2 ls -l /proc/$$/fd total 0 lrwx------ 1 stefano stefano 64 Jan 24 22:50 0 -> /dev/pts/4 lrwx------ 1 stefano stefano 64 Jan 24 22:50 1 -> /dev/pts/4 lrwx------ 1 stefano stefano 64 Jan 24 22:50 2 -> /dev/pts/4 $ netbsd-make -j 2 --- x --- ls -l /proc/$$/fd total 0 lrwx------ 1 stefano stefano 64 Jan 24 22:50 0 -> /tmp/make4lRgpl (deleted) l-wx------ 1 stefano stefano 64 Jan 24 22:50 1 -> pipe:[34633] l-wx------ 1 stefano stefano 64 Jan 24 22:50 2 -> pipe:[34633] $ freebsd-make -j 2 ls -l /proc/$$/fd total 0 lrwx------ 1 stefano stefano 64 Jan 24 22:53 0 -> /tmp/makeXXXXAYm9oW (deleted) l-wx------ 1 stefano stefano 64 Jan 24 22:53 1 -> pipe:[34818] l-wx------ 1 stefano stefano 64 Jan 24 22:53 2 -> pipe:[34818] [3] This can be demonstrated (on Solaris 10) with the following: $ echo 'all: ; all: ; /usr/proc/bin/pfiles $$$$' > Makefile $ gmake -j 2 Current rlimit: 256 file descriptors 0: S_IFCHR mode:0620 ... O_RDWR|O_NOCTTY|O_LARGEFILE /devices/pseudo/pts@0:6 1: S_IFCHR mode:0620 ... O_RDWR|O_NOCTTY|O_LARGEFILE /devices/pseudo/pts@0:6 2: S_IFCHR mode:0620 ... O_RDWR|O_NOCTTY|O_LARGEFILE /devices/pseudo/pts@0:6 $ dmake -j 2 Current rlimit: 256 file descriptors 0: S_IFCHR mode:0620 ... O_RDWR|O_NOCTTY|O_LARGEFILE /devices/pseudo/pts@0:6 1: S_IFREG mode:0600 ... O_WRONLY|O_DSYNC|O_CREAT|O_TRUNC|O_LARGEFILE /tmp/dmake.stdout.23631.0.7ba4jU 2: S_IFREG mode:0600 ... O_WRONLY|O_DSYNC|O_CREAT|O_TRUNC|O_LARGEFILE /tmp/dmake.stdout.23631.0.7ba4jU [4] For more details, see the description of the 'color-tests' option at: <http://www.gnu.org/software/automake/manual/html_node/Simple-Tests.html> [5] That happens becuse the automake-generated testsuite drivers check whether the controlling make instance is attached to a tty (to decide whether to activate colored output or not) by using `test -t 1' in a proper recipe. But when make is run in "parallel mode", the shell running that test has its stdout connected to a pipe (read my make), so that the test returns "false", and the colored output is not activated. [6] Luckily, there is a simple work around: define the AM_COLOR_TESTS variable to 'always' in the environment of the make process; e.g., the following will produce colored testsuite output even when using NetBSD or FreeBSD make: $ AM_COLOR_TESTS=always make -j2 check
From 0f216c0ad0ab721e25c0d16b9fb66fbcb1a5ae64 Mon Sep 17 00:00:00 2001 From: Stefano Lattarini <stefano.lattar...@gmail.com> Date: Mon, 24 Jan 2011 22:31:37 +0100 Subject: [PATCH] docs: another parallel make issue * doc/autoconf.texi (Parallel Make): Document that some make implementations, when run in parallel mode, connect stdout and stderr of child processes to pipes or temporary files. --- ChangeLog | 7 +++++++ doc/autoconf.texi | 37 ++++++++++++++++++++++++++++++++++++- 2 files changed, 43 insertions(+), 1 deletions(-) diff --git a/ChangeLog b/ChangeLog index 2fc5ab3..e548f8c 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,10 @@ +2011-01-24 Stefano Lattarini <stefano.lattar...@gmail.com> + + docs: another parallel make issue + * doc/autoconf.texi (Parallel Make): Document that some make + implementations, when run in parallel mode, connect stdout and + stderr of child processes to pipes or temporary files. + 2011-01-23 Ralf Wildenhues <ralf.wildenh...@gmx.de> docs: new section about whitespace trimmmed from make command-lines. diff --git a/doc/autoconf.texi b/doc/autoconf.texi index ae601a0..eaeb054 100644 --- a/doc/autoconf.texi +++ b/doc/autoconf.texi @@ -19475,7 +19475,42 @@ sh @cindex Parallel @command{make} Support for parallel execution in @command{make} implementation varies. -Generally, using GNU make is your best bet. When NetBSD +Generally, using GNU make is your best bet. + +Some make implementations (among them, FreeBSD @command{make}, NetBSD +@command{make}, and Solaris @command{dmake}), when invoked with a +@option{-j@var{N}} option, connect the standard output and standard +error of all their children processes to pipes or temporary regular +files. This can lead to slightly but subtly different semantics in +the behaviour of the spawned processes. For example, even if the +@command{make} standard output is a connected to a tty, a command +like @code{test -t 1} issued in a recipe will return a failure within +those make implementations: + +@example +$ @kbd{cat Makefile} +all: + @@test -t 1 && echo "Is a tty" || echo "Is not a tty" +$ @kbd{make -j 2} # FreeBSD make +Is not a tty +$ @kbd{make -j 2} # NetBSD make +--- all --- +Is not a tty +$ @kbd{dmake -j 2} # Solaris dmake +@var{hostname} --> 1 job +@var{hostname} --> Job output +Is not a tty +@end example + +@noindent +while on the other hand: + +@example +$ @kbd{make -j 2} # GNU make, Heirloom make +Is a tty +@end example + +When NetBSD @command{make} is invoked with @option{-j@var{N}}, it will reuse the same shell for multiple commands within one recipe. This can have unexpected consequences.@footnote{Note that GNU make has -- 1.7.2.3
From 3eaf9247e44a354cc0c0606e2c0f2b564e4c1c4e Mon Sep 17 00:00:00 2001 From: Stefano Lattarini <stefano.lattar...@gmail.com> Date: Mon, 24 Jan 2011 22:26:55 +0100 Subject: [PATCH] docs: color-tests issues with parallel make * doc/automake.texi (Simple Tests): Document that automatic recognition of a capable terminal attached to stdout can fail with some make implementation when running in parallel mode, thus causing colored test output not to be automatically activated when it should. --- ChangeLog | 9 +++++++++ doc/automake.texi | 6 ++++++ 2 files changed, 15 insertions(+), 0 deletions(-) diff --git a/ChangeLog b/ChangeLog index c632b44..95790da 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,12 @@ +2011-01-24 Stefano Lattarini <stefano.lattar...@gmail.com> + + docs: color-tests issues with parallel make + * doc/automake.texi (Simple Tests): Document that automatic + recognition of a capable terminal attached to stdout can fail + with some make implementation when running in parallel mode, + thus causing colored test output not to be automatically + activated when it should. + 2011-01-23 Stefano Lattarini <stefano.lattar...@gmail.com> coverage: test semantics of "dummy" per-target flags diff --git a/doc/automake.texi b/doc/automake.texi index 3d6a7b0..2391621 100644 --- a/doc/automake.texi +++ b/doc/automake.texi @@ -8589,6 +8589,12 @@ colored output by setting the @command{make} variable @samp{AM_COLOR_TESTS=no}, or force colored output even without a connecting terminal with @samp{AM_COLOR_TESTS=always}. +Note that incompatible semantics of some @command{make} implementations +when used in parallel mode (@pxref{Parallel make,,, autoconf, The Autoconf +Manual}) can cause the automatic detection of a connection to a capable +terminal to fail spuriously. In that case, you can still resort to the +use of @samp{AM_COLOR_TESTS=always}. + @vindex TESTS @vindex TESTS_ENVIRONMENT The variable @code{TESTS_ENVIRONMENT} can be used to set environment -- 1.7.2.3