[Automake-NG] [FYI] Merge branch 'master' into ng/master
commit f855e98b434d74489e19aded9b598e6969068975 Merge: 20cff0b ebc92e1 Author: Stefano Lattarini Date: Wed Jun 27 12:55:59 2012 +0200 Merge branch 'master' into ng/master * master: missing: --run is supported again (as a no-op) for use by older automake Signed-off-by: Stefano Lattarini
[Automake-NG] [FYI] [ng] missing: suggest Automake-NG, not mainstream Automake
* lib/missing (program_details): Here.
Signed-off-by: Stefano Lattarini
---
lib/missing |5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/lib/missing b/lib/missing
index b398a02..fec9b3d 100755
--- a/lib/missing
+++ b/lib/missing
@@ -109,8 +109,9 @@ program_details ()
{
case $1 in
aclocal|automake)
- echo "The '$1' program is part of the GNU Automake package:"
- echo "<$gnu_software_URL/automake>"
+ echo "The '$1' program is part of the GNU Automake-NG package:"
+ # FIXME: maybe it's time for Automake-NG to have a page of its own?
+ echo ""
echo "It also requires GNU Autoconf, GNU m4 and Perl in order to run:"
echo "<$gnu_software_URL/autoconf>"
echo "<$gnu_software_URL/m4/>"
--
1.7.9.5
[Automake-NG] [FYI] [ng] missing: no need to suggest GNU make explicitly
* lib/missing (give_advice): Here. In mainstream Automake, possible use of GNU make was suggested as a remedy against useless remake of '.info' files triggered by some buggy vendor make implementations. Signed-off-by: Stefano Lattarini --- lib/missing |4 1 file changed, 4 deletions(-) diff --git a/lib/missing b/lib/missing index fec9b3d..cd1f739 100755 --- a/lib/missing +++ b/lib/missing @@ -185,10 +185,6 @@ give_advice () echo "any other file indirectly affecting the aspect of the manual." echo "You might want to install the Texinfo package:" echo "<$gnu_software_URL/texinfo/>" - echo "The spurious makeinfo call might also be the consequence of" - echo "using a buggy 'make' (AIX, DU, IRIX), in which case you might" - echo "want to install GNU make:" - echo "<$gnu_software_URL/make/>" ;; *) echo "You might have modified some files without having the proper" -- 1.7.9.5
Re: [Automake-NG] [PATCH 00/14] Rework cleaning rules APIs and implementation
On 06/21/2012 12:32 PM, Stefano Lattarini wrote: > I will push this series in a couple of days if there is no objection. > > Stefano Lattarini (14): > [01] deptrack: remove an obsolete comment > [02] deptrack: clean depdirs using *clean-am targets > [03] clean: simplify cleaning of compiled objects > [04] clean: revamp recipes and APIs to extend cleaning rules > [05] clean: do not ignore errors while removing files > [06] automake: new global variable '%clean_dirs' > [07] refactoring: move definition of $(am__mv) in depend2.am > [08] cleanup: remove 'depend.am' > [09] clean: don't remove object files in '.' unconditionally > [10] refactoring: no "cleaning" rules in compile.am > [11] cleanup: merge '%compile_clean_files' in '%clean_files' > [12] refactor: prefer '&file_contents' over '&file_contents_internals' > [13] cleanup: remove 'libtool.am' > [14] check: keep cleaning of '.log' and '.trs' files near other rules > [15] cleanup: remove 'clean-hdr.am' > [16] clean: drop minor DOS-related cruft in libtool support > Pushed now, with all the fixlets and improvements we had agreed about in the thread. Thanks, Stefano
[Automake-NG] [PATCH 2/2] warns: fix checks against typos in vars (_LDADD and _SOURCES, for example)
After commit v1.12.1-302-g67d6102 of 2012-06-05, "[ng] warns: typos in
_SOURCES etc. reported at make runtime", the checks against typos in use
of few special variables (like 'bar_SOURCES' or 'libfoo_a_LIBADD') are
done at make runtime rather than at automake runtime.
Such checks have so far been running unconditionally every time make was
invoked, thus creating a noticeable performance degradation in null or
almost-null builds of packages that, like GNU coreutils, have a lot of
programs and/or use several recursive make invocations.
What is worse, such checks have been unable to interact well with the
Makefile automatic remake rules. Say that a user encounters a failure
in those checks; he then fixes his 'Makefile.am' accordingly, and re-run
"make", expecting the situation to be sorted out by the automatic remake
rules (as was the case for mainline Automake). But that doesn't happen,
because those same checks will run again *before* giving the remake rules
a chance to kick in -- thus "make" would fail again, exactly as before,
because the Makefile hasn't been rebuilt, and thus hasn't changed.
So the user would forced to re-run automake and config.status *by hand* to
get back a consistent, correct status in its build system! Such situation
is utterly unacceptable.
The present change takes care at once of both of the performance and the
correctness problems.
On the performance side, after this change, the time for a null build
on GNU coreutils (bootstrapped with Automake-NG) has gone down from ~8
seconds to ~2 seconds. Not bad!
* automake.in (generate_makefile): Pass the '%MAKEFILE%' transform
when processing 'check-typos.am'.
* lib/am/check-typos.am: Take advantage of the GNU make automatic restart
capabilities to re-implement the existing check in a more "lazy" way:
they should be re-run only when Makefile has changes, and only after the
automatic remake rules has rebuilt it (if it needs to be rebuilt).
* lib/am/header-vars.am (.am/nil): New do-nothing target, for now used
only by 'check-typos.am'.
* t/am-dir.sh: Adjust.
* t/maken.sh: Likewise.
* t/output6.sh: Likewise.
* t/spell.sh: Likewise.
* t/spell2.sh: Likewise.
* t/vartypos-whitelist.sh: Likewise.
* t/vartypos.sh: Likewise.
Signed-off-by: Stefano Lattarini
---
automake.in |3 ++-
lib/am/check-typos.am | 55 ++-
lib/am/header-vars.am |6 ++
t/am-dir.sh |3 ++-
t/maken.sh |6 +++---
t/output6.sh|8 +--
t/spell.sh |4 ++--
t/spell2.sh |4 ++--
t/vartypos-whitelist.sh | 20 -
t/vartypos.sh | 13 ---
10 files changed, 98 insertions(+), 24 deletions(-)
diff --git a/automake.in b/automake.in
index 074abf6..a734afc 100644
--- a/automake.in
+++ b/automake.in
@@ -6945,7 +6945,8 @@ sub generate_makefile ($$)
my $output_checks = '';
# See if any _SOURCES (or _LIBADD, or ...) variable were misspelled.
- $output_checks .= preprocess_file ("$libdir/am/check-typos.am");
+ $output_checks .= preprocess_file ("$libdir/am/check-typos.am",
+ MAKEFILE => basename ($makefile));
# Report errors (if any) seen at make runtime.
$output_checks .= '$(if $(am__seen_error),' .
'$(error Some Automake-NG error occurred))' .
diff --git a/lib/am/check-typos.am b/lib/am/check-typos.am
index 71769c9..6e4b804 100644
--- a/lib/am/check-typos.am
+++ b/lib/am/check-typos.am
@@ -18,13 +18,29 @@
##bin_PROGRAMS = bar
##baz_SOURCES = main.c # Should be bar_SOURCES.
-## FIXME: With this, we impose runtime penalty to all make runs. Maybe we
-## FIXME: should make all these checks conditional to whether the user sets
-## FIXME: a AM_SANITY_CHECKS variable or something?
-
## FIXME: We should document the '.am/' namespace as reserved for automake
## FIXME: internals somewhere.
+## FIXME: we should document the user-settable AM_FORCE_SANITY_CHECKS
+## FIXME: variable, and its semantics.
+
+# Running these checks unconditionally can be quite wasteful, especially
+# for projects with lots of programs. So run them only when the values
+# of the checked variables has possibly changed. For the moment, we assume
+# this can only happen if the Makefile has been updated since the later
+# check. And to give user more control, we also allow him to require these
+# checks run unconditionally, by setting AM_FORCE_SANITY_CHECKS to "yes".
+
+# The idiom we employ to implement our "lazy checking" relies on recursive
+# make invocations and creation of an auxiliary makefile fragments, and
+# such an approach do not interact very well with "make -n"; in such a case,
+# it's simpler and safer to go for "greedy checking".
+ifeq ($(am__make_dryrun),true)
+AM_FORCE_SANITY_CHECKS ?= yes
+endif
+
+ifeq ($(AM_FORCE_SANITY_CHECKS),yes)
+
# Variables with these suffixes are candidates for typo checking.
.am/vartypos/suff
[Automake-NG] [PATCH 1/2] [ng] perf: disable GNU make builtin rules, for performance reasons
* lib/am/header-vars (MAKEFLAGS): Append '--no-builtin-rules'. This enhance performance by removing GNU make builtin rules that would be rather irrelevant (e.g., rules to automatically check out files from from RCS or SCCS repositories), or overridden by Automake (e.g., rules to compile C or C++ files). With this simple tweak, the null-build time on GNU coreutils on an aging Debian system goes down from ~ 20 seconds to ~ 9 seconds. Signed-off-by: Stefano Lattarini --- lib/am/header-vars.am |7 +++ 1 file changed, 7 insertions(+) diff --git a/lib/am/header-vars.am b/lib/am/header-vars.am index eccfec5..75e5a3a 100644 --- a/lib/am/header-vars.am +++ b/lib/am/header-vars.am @@ -16,6 +16,13 @@ VPATH = @srcdir@ +## Enhance performance by removing GNU make builtin rules that would be +## rather irrelevant (e.g., rules to automatically check out files from +## from RCS or SCCS repositories), or overridden by Automake (e.g., rules +## to compile C or C++ files). For larger packages (like GNU coreutils), +## this can greatly speed up null or almost-null builds, up to even 50%! +MAKEFLAGS += --no-builtin-rules + ## Declare an error, without immediately terminating the execution (proper ## code will take care later of that). This will allow us to diagnose more ## issues at once, rather than stopping at the first one. -- 1.7.9.5
Re: [Automake-NG] [PATCH 1/2] [ng] perf: disable GNU make builtin rules, for performance reasons
On Wed, 27 Jun 2012, Stefano Lattarini wrote: * lib/am/header-vars (MAKEFLAGS): Append '--no-builtin-rules'. This enhance performance by removing GNU make builtin rules that would be rather irrelevant (e.g., rules to automatically check out files from from RCS or SCCS repositories), or overridden by Automake (e.g., rules to compile C or C++ files). With this simple tweak, the null-build time on GNU coreutils on an aging Debian system goes down from ~ 20 seconds to ~ 9 seconds. This simple approach seems like a huge improvement. The improvement will surely be larger if some/all of the files are accessed over a network. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: [Automake-NG] [PATCH 2/2] warns: fix checks against typos in vars (_LDADD and _SOURCES, for example)
On Wed, Jun 27, 2012 at 16:05 UTC, Stefano Lattarini wrote:
> After commit v1.12.1-302-g67d6102 of 2012-06-05, "[ng] warns: typos in
> _SOURCES etc. reported at make runtime", the checks against typos in use
> of few special variables (like 'bar_SOURCES' or 'libfoo_a_LIBADD') are
> done at make runtime rather than at automake runtime.
>
> Such checks have so far been running unconditionally every time make was
> invoked, thus creating a noticeable performance degradation in null or
> almost-null builds of packages that, like GNU coreutils, have a lot of
> programs and/or use several recursive make invocations.
>
> What is worse, such checks have been unable to interact well with the
> Makefile automatic remake rules. Say that a user encounters a failure
> in those checks; he then fixes his 'Makefile.am' accordingly, and re-run
> "make", expecting the situation to be sorted out by the automatic remake
> rules (as was the case for mainline Automake). But that doesn't happen,
> because those same checks will run again *before* giving the remake rules
> a chance to kick in -- thus "make" would fail again, exactly as before,
> because the Makefile hasn't been rebuilt, and thus hasn't changed.
> So the user would forced to re-run automake and config.status *by hand* to
> get back a consistent, correct status in its build system! Such situation
> is utterly unacceptable.
>
> The present change takes care at once of both of the performance and the
s/at once//
"Takes care of" is an idiom that doesn't read well when interspersed
with other words. Since "at once" isn't really adding any information
here, getting rid of it is the easiest reformulation.
> correctness problems.
>
> On the performance side, after this change, the time for a null build
> on GNU coreutils (bootstrapped with Automake-NG) has gone down from ~8
> seconds to ~2 seconds. Not bad!
>
> * automake.in (generate_makefile): Pass the '%MAKEFILE%' transform
> when processing 'check-typos.am'.
> * lib/am/check-typos.am: Take advantage of the GNU make automatic restart
> capabilities to re-implement the existing check in a more "lazy" way:
> they should be re-run only when Makefile has changes, and only after the
I think s/changes/changed/ is clearer.
> automatic remake rules has rebuilt it (if it needs to be rebuilt).
s/has/have/
At your discretion, s/(if it needs to be rebuilt)/(if needed)/ is more
concise without ambiguity.
> * lib/am/header-vars.am (.am/nil): New do-nothing target, for now used
> only by 'check-typos.am'.
> * t/am-dir.sh: Adjust.
> * t/maken.sh: Likewise.
> * t/output6.sh: Likewise.
> * t/spell.sh: Likewise.
> * t/spell2.sh: Likewise.
> * t/vartypos-whitelist.sh: Likewise.
> * t/vartypos.sh: Likewise.
>
> Signed-off-by: Stefano Lattarini
> ---
> automake.in | 3 ++-
> lib/am/check-typos.am | 55
> ++-
> lib/am/header-vars.am | 6 ++
> t/am-dir.sh | 3 ++-
> t/maken.sh | 6 +++---
> t/output6.sh | 8 +--
> t/spell.sh | 4 ++--
> t/spell2.sh | 4 ++--
> t/vartypos-whitelist.sh | 20 -
> t/vartypos.sh | 13 ---
> 10 files changed, 98 insertions(+), 24 deletions(-)
>
> diff --git a/automake.in b/automake.in
> index 074abf6..a734afc 100644
> --- a/automake.in
> +++ b/automake.in
> @@ -6945,7 +6945,8 @@ sub generate_makefile ($$)
>
> my $output_checks = '';
> # See if any _SOURCES (or _LIBADD, or ...) variable were misspelled.
> - $output_checks .= preprocess_file ("$libdir/am/check-typos.am");
> + $output_checks .= preprocess_file ("$libdir/am/check-typos.am",
> + MAKEFILE => basename ($makefile));
> # Report errors (if any) seen at make runtime.
> $output_checks .= '$(if $(am__seen_error),' .
> '$(error Some Automake-NG error occurred))' .
> diff --git a/lib/am/check-typos.am b/lib/am/check-typos.am
> index 71769c9..6e4b804 100644
> --- a/lib/am/check-typos.am
> +++ b/lib/am/check-typos.am
> @@ -18,13 +18,29 @@
> ## bin_PROGRAMS = bar
> ## baz_SOURCES = main.c # Should be bar_SOURCES.
>
> -## FIXME: With this, we impose runtime penalty to all make runs. Maybe we
> -## FIXME: should make all these checks conditional to whether the user sets
> -## FIXME: a AM_SANITY_CHECKS variable or something?
> -
> ## FIXME: We should document the '.am/' namespace as reserved for automake
> ## FIXME: internals somewhere.
>
> +## FIXME: we should document the user-settable AM_FORCE_SANITY_CHECKS
> +## FIXME: variable, and its semantics.
> +
> +# Running these checks unconditionally can be quite wasteful, especially
> +# for projects with lots of programs. So run them only when the values
> +# of the checked variables has possibly changed. For the moment, we assume
s/has possibly changed/may have changed/
or "might have changed" or "could have changed" if you prefer.
> +#
Re: [Automake-NG] [PATCH 1/2] [ng] perf: disable GNU make builtin rules, for performance reasons
On Wed, Jun 27, 2012 at 17:30 PM, Bob Friesenhahn wrote: > On Wed, 27 Jun 2012, Stefano Lattarini wrote: > >> * lib/am/header-vars (MAKEFLAGS): Append '--no-builtin-rules'. This >> enhance performance by removing GNU make builtin rules that would be >> rather irrelevant (e.g., rules to automatically check out files from >> from RCS or SCCS repositories), or overridden by Automake (e.g., rules >> to compile C or C++ files). With this simple tweak, the null-build >> time on GNU coreutils on an aging Debian system goes down from ~ 20 >> seconds to ~ 9 seconds. > > > This simple approach seems like a huge improvement. The improvement will > surely be larger if some/all of the files are accessed over a network. Yes, good stuff. Anyone who has debugged Makefiles with GNU make -v has seen the insane number of wasted operations due to combinatorial explosion exploring possible matches with builtin rules. Can Automake be similarly improved to disable GNU make's builtin rules without sacraficing compatibility with less-loved make implementations? If it's not practical for Automake to do this for all packages, how might a maintainer do it for his package without requiring GNU make? Cheers, Dave Hart
Re: [Automake-NG] [PATCH 2/2] warns: fix checks against typos in vars (_LDADD and _SOURCES, for example)
Hi Dave, thanks for the review. On 06/27/2012 07:32 PM, Dave Hart wrote: > > [SNIP GOOD SUGGESTIONS] > I agree with all your nits. Consider them addressed. Best regards, Stefano
Re: [Automake-NG] [PATCH 1/2] [ng] perf: disable GNU make builtin rules, for performance reasons
On 06/27/2012 07:40 PM, Dave Hart wrote: > On Wed, Jun 27, 2012 at 17:30 PM, Bob Friesenhahn wrote: >> On Wed, 27 Jun 2012, Stefano Lattarini wrote: >> >>> * lib/am/header-vars (MAKEFLAGS): Append '--no-builtin-rules'. This >>> enhance performance by removing GNU make builtin rules that would be >>> rather irrelevant (e.g., rules to automatically check out files from >>> from RCS or SCCS repositories), or overridden by Automake (e.g., rules >>> to compile C or C++ files). With this simple tweak, the null-build >>> time on GNU coreutils on an aging Debian system goes down from ~ 20 >>> seconds to ~ 9 seconds. >> >> >> This simple approach seems like a huge improvement. The improvement will >> surely be larger if some/all of the files are accessed over a network. > > Yes, good stuff. Anyone who has debugged Makefiles with GNU make -v > has seen the insane number of wasted operations due to combinatorial > explosion exploring possible matches with builtin rules. > > Can Automake be similarly improved to disable GNU make's builtin rules > without sacraficing compatibility with less-loved make implementations? > I think it would be easily doable; it would probably require a configure time tough (similar to the one that is done to initialize '@SET_MAKE@'). Alas, this would have the side effect or rendering a Makefile created from a configuration with GNU make early in PATH unusable with non-GNU makes. Not a big deal, but an annoyance nonetheless. Anybody volunteering a patch? > If it's not practical for Automake to do this for > all packages, how might a maintainer do it for his package without > requiring GNU make? > He could provide a top-level GNUmakefile containing the MAKEFLAGS += --no-builtin-rules line early, and then including the Automake-generated Makefile: include ./Makefile But that will only clobber the built-in rules for builds launched from the top-level directory, not from the subdirectories (another good reason to switch to a non-recursive make setup). Regards, Stefano
Re: [Automake-NG] [PATCH 1/2] [ng] perf: disable GNU make builtin rules, for performance reasons
On Wed, Jun 27, 2012 at 19:31 UTC, Stefano Lattarini wrote: > On 06/27/2012 07:40 PM, Dave Hart wrote: >> Yes, good stuff. Anyone who has debugged Makefiles with GNU make -v >> has seen the insane number of wasted operations due to combinatorial >> explosion exploring possible matches with builtin rules. >> >> Can Automake be similarly improved to disable GNU make's builtin rules >> without sacraficing compatibility with less-loved make implementations? >> > I think it would be easily doable; it would probably require a configure > time tough (similar to the one that is done to initialize '@SET_MAKE@'). > Alas, this would have the side effect or rendering a Makefile created > from a configuration with GNU make early in PATH unusable with non-GNU > makes. Not a big deal, but an annoyance nonetheless. Better than the opposite: I know folks who let configure detect the native non-GNU 'make' on Solaris and BSD systems, then habitually invoke 'gmake' not 'make'. That has not caused trouble, though I don't advise it. I generally prefer to use the host 'make' to expose issues that would not occur with GNU make, but in the more common scenario of just wanting it to work, I'd suggest MAKE=gmake in the environment of those who habitually type 'gmake', to inform configure scripts of the make program that will be used. > Anybody volunteering a patch? I'm not, so far. I'll see how much it itches. >> If it's not practical for Automake to do this for >> all packages, how might a maintainer do it for his package without >> requiring GNU make? >> > He could provide a top-level GNUmakefile containing the > > MAKEFLAGS += --no-builtin-rules > > line early, and then including the Automake-generated Makefile: > > include ./Makefile > > But that will only clobber the built-in rules for builds launched from > the top-level directory, not from the subdirectories (another good > reason to switch to a non-recursive make setup). Thanks. That approach should work for a recursive setup if each directory with Makefile.am also has the MAKEFLAGS-altering GNUmakefile. From my perspecitve, while it would be nicer if Automake handled it for me, this solution appeals. I realize converting to nonrecursive make would yield correctness and performance wins, but so far I have been put off by the loss of ability to build only a subtree. Cheers, Dave Hart
Re: [Automake-NG] [PATCH 1/2] [ng] perf: disable GNU make builtin rules, for performance reasons
On Wed, 27 Jun 2012, Dave Hart wrote: Thanks. That approach should work for a recursive setup if each directory with Makefile.am also has the MAKEFLAGS-altering GNUmakefile. From my perspecitve, while it would be nicer if Automake handled it for me, this solution appeals. I realize converting to nonrecursive make would yield correctness and performance wins, but so far I have been put off by the loss of ability to build only a subtree. It is quite possible to build a subtree of a non-recursive project but you do need to know how to specify the fully-qualified path to the target which should be built. If there are just a few common targets, these can be added as shortcut targets in the Makefile. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
