[bug#60535] [PATCH] depend2: switch echo|sed to automatic vars

[Mike Frysinger]

On 05 Jan 2023 16:47, Karl Berry wrote:
> Please excuse my curmudgeonness, but it's not clear to me that avoiding
> sed is worth these hassles in working around the implementation-specific
> bugs in the automatic variables. Especially if we have to invoke a shell
> and various commands anyway, how about keeping things as they are? -k

i don't quite buy the argument of "we've got a lot of per-object overhead,
so what's a little bit more".  we should be trying to minimize overhead of
generated code as much as possible.  forking a subshell to fork sed just to
munge a string that we can have make itself generate is pure overhead.

but you're appealing to avoiding hassle.  to that end, i'll point out the
current depdir logic is already in an unhealthy state: we're generating
similar logic 3 times with automake-processed conditionals depending on 3
runtime settings:
?!GENERIC?  %VERBOSE%%COMPILE% -MT %OBJ% -MD -MP -MF %DEPBASE%.Tpo 
%-c% -o %OBJ% %SOURCEFLAG%`test -f '%SOURCE%' || echo '$(srcdir)/'`%SOURCE%
?!GENERIC?  %SILENT%$(am__mv) %DEPBASE%.Tpo %DEPBASE%.Po
?GENERIC??!SUBDIROBJ?   %VERBOSE%%COMPILE% -MT %OBJ% -MD -MP -MF %DEPBASE%.Tpo 
%-c% -o %OBJ% %SOURCEFLAG%%SOURCE%
?GENERIC??!SUBDIROBJ?   %SILENT%$(am__mv) %DEPBASE%.Tpo %DEPBASE%.Po
?GENERIC??SUBDIROBJ?%VERBOSE%depbase=`echo %OBJ% | sed 
's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\
?GENERIC??SUBDIROBJ?%COMPILE% -MT %OBJ% -MD -MP -MF %DEPBASE%.Tpo %-c% -o 
%OBJ% %SOURCEFLAG%%SOURCE% &&\
?GENERIC??SUBDIROBJ?$(am__mv) %DEPBASE%.Tpo %DEPBASE%.Po

this directly translates to overhead in the generated Makefile.in:
.c.o:
@am__fastdepCC_TRUE@$(AM_V_CC)depbase=`echo $@ | sed 
's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\
@am__fastdepCC_TRUE@$(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo 
-c -o $@ $< &&\
@am__fastdepCC_TRUE@$(am__mv) $$depbase.Tpo $$depbase.Po
@AMDEP_TRUE@@am__fastdepCC_FALSE@   $(AM_V_CC)source='$<' object='$@' 
libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepCC_FALSE@   DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) 
@AMDEPBACKSLASH@
@am__fastdepCC_FALSE@   $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $<

switching to make variables will allow us to collapse these.  i'm fairly
confident i can at least collapse the last 5 lines ?GENERIC? into 2 lines.
-mike


signature.asc
Description: PGP signature

[bug#60747] [PATCH] dirstamp: switch to a pattern rule

[Mike Frysinger]

We can leverage $(@D) to generate a single pattern rule for all dirstamp
rules.  This saves many lines in the output -- normally we create 2 rules
(or 6 lines) per subdir, and projects that use subdirs tend to use them
quite a bit.

In the most extreme & unlikely case (1 subdir, no depdir support), the
line count is the same.  In every other case, it's always a win.

Looking at a few real world projects, the line deltas:
* GNU libgloss: +3 -66
* GNU newlib:   +3 -714
* GNU sim:  +3 -138

There shouldn't be any concerns about portability with $(@D) because:
(1) We only generate this rule iff we know the dirstamp is in a subdir
(so we'd never have a case where $(@D) would expand to the cwd)
(2) We already rely on $(@D) in our depdir code, and have since 2014
(the Automake 1.16 release).
---
 bin/automake.in | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/bin/automake.in b/bin/automake.in
index 1c13a3187f46..139d5ad93a9a 100644
--- a/bin/automake.in
+++ b/bin/automake.in
@@ -7893,6 +7893,14 @@ sub require_build_directory
   $directory_map{$directory} = $dirstamp;
   $directory_map{$cdir} = $dirstamp;
 
+  # Generate the pattern rule only once.
+  if (! vardef ('am__dirstamp', TRUE))
+{
+  $output_rules .= ("%/\$(am__dirstamp):\n"
+   . "\t\@\$(MKDIR_P) \$(\@D)\n"
+   . "\t\@: >>\$\@\n");
+}
+
   # Set a variable for the dirstamp basename.
   define_pretty_variable ('am__dirstamp', TRUE, INTERNAL,
  '$(am__leading_dot)dirstamp');
@@ -7900,10 +7908,6 @@ sub require_build_directory
   # Directory must be removed by 'make distclean'.
   $clean_files{$dirstamp} = DIST_CLEAN;
 
-  $output_rules .= ("$dirstamp:\n"
-   . "\t\@\$(MKDIR_P) $directory\n"
-   . "\t\@: >>$dirstamp\n");
-
   return $dirstamp;
 }
 
-- 
2.39.0

[bug#60746] [PATCH] dirstamp: use append too instead of truncate

[Mike Frysinger]

We changed the depfiles logic to use >> (append) instead of > (truncate)
due to it being slightly faster & nicer to the disk.  Do the same with
the dirstamp files as we only need the files to exist -- we don't care
about their content, and we never put anything in them ourselves.  If
someone else were to, we clean them up normally with `make clean`.

Simple test case on my Linux 6.1 w/ext4 on SSD:

@: > foo.txt
for (i = 0; i < 100; ++i) close(open("foo.txt", O_WRONLY|O_CREAT|O_TRUNC, 
0666));
-> 769 msec

@: >>foo.txt
for (i = 0; i < 100; ++i) close(open("foo.txt", O_WRONLY|O_CREAT|O_APPEND, 
0666));
-> 2 sec
---
 bin/automake.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bin/automake.in b/bin/automake.in
index 3069132796f1..1c13a3187f46 100644
--- a/bin/automake.in
+++ b/bin/automake.in
@@ -7902,7 +7902,7 @@ sub require_build_directory
 
   $output_rules .= ("$dirstamp:\n"
. "\t\@\$(MKDIR_P) $directory\n"
-   . "\t\@: > $dirstamp\n");
+   . "\t\@: >>$dirstamp\n");
 
   return $dirstamp;
 }
-- 
2.39.0