Hi Karl,
> I understand that equal timestamps are considered up to date
Yes, except for old HP-UX 'make', which no one uses any more, all 'make'
programs consider equal timestamps as "up-to-date".
> Ok, but
> then why is configure generating config.status/etc. such a special case
> that it requires the sleep, and nothing else? I mean, I know the sleep
> is needed; I've experienced the problems without that sleep myself. But
> I don't understand why it's the only place (in normal compilations;
> forget the Automake test suite specifically) that needs it.
The 'configure' or 'config.status' are not _inherently_ special. They are
special
- because of the AM_SANITY_CHECK,
- because there are Makefile rules that depend on configure and config.status:
Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status
$(top_builddir)/config.status: $(top_srcdir)/configure
$(CONFIG_STATUS_DEPENDENCIES)
Let's look at the history.
1) In the beginning, i.e. with just Autoconf and no Automake, or with
Automake before 1996, no sleep was ever done.
But systems often had a skewed clock. Building a 30-seconds old tarball
on a machine with a 5-minutes skew would spew pretty obscure errors.
Therefore:
2) The AM_SANITY_CHECK macro was added.
https://git.savannah.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=c7f27ed5f5e40f768876c63d6380795c4436e935
It uses 'ls -t' of the configure file and a freshly created file
and expects the freshly created file to be newer.
The need for sleeps starts here. Namely, if the configure file is 0.1 sec
old but the file system time resolution is 1 second, with up to 90% chance
(depending on the current time mod 1 second, and depending on the order
in which 'ls' scans the two inodes) the sanity check will bail out.
Therefore:
3) A 'sleep 1' was added.
https://git.savannah.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=c1bf7cf1850526431765c461ef198375ffe7f0cb
Now the sanity check no longer bails out unnecessarily. But the configure
script takes 1 extra second, always, and for all packages.
Therefore:
4) The 'ls -t' test was changed to be done twice, so that in the normal
case (no skewed clock, and a tarball older than 1 second) no 'sleep 1'
is needed.
https://git.savannah.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=f6b3f7fb620580356865ebedfbaf76af3e534369
But this caused "time stamp issues with makefile rules rerunning
autotools programs." To fix them:
5) A parallel 'sleep 1' was introduced, from the beginning of the
configure execution until where config.status is being generated.
https://lists.gnu.org/archive/html/automake-commit/2010-10/msg00005.html
https://lists.gnu.org/archive/html/bug-automake/2010-10/msg00000.html
https://git.savannah.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=333c18a898e9042938be0e5709ec46ff0ead0797
This was fine for 11 years. Then:
6)
> This whole thing started
> with Mike Vapier's change in Feb 2022 (commit 720a11531):
> https://lists.gnu.org/archive/html/automake-commit/2022-02/msg00009.html
https://git.savannah.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=720a1153134b833de9298927a432b4ea266216fb
>
> As I read it now, his goal was to speed up other projects, not Automake,
> by reducing the "sleep 1" to "sleep <mtime_resolution>" in
> AM_SANITY_CHECK, via AC_CONFIG_COMMANDS_PRE, i.e., before creating
> config.status.
His goal was not to speed up other packages in general, but specifically
"projects with many smallish configure scripts with many cached vars",
and specifically the case of re-running the configuration. In this
case, the parallel 'sleep 1' from the beginning of the configure
execution until the creation of config.status is what dominates the
consumed wall time.
The problem is that in doing so, he added the unconditional AC_REQUIRE
of _AM_FILESYSTEM_TIMESTAMP_RESOLUTION, which sleeps for at least
2 seconds (either 20x 'sleep 0.1' or 2x 'sleep 1'). Thus, while
optimizing the case of re-running the configuration, this patch penalizes
the first run of 'configure'. For all packages.
You and Zack have rewritten _AM_FILESYSTEM_TIMESTAMP_RESOLUTION, but
it still sleeps for 2 seconds on NetBSD 10. For all packages.
> But that is only one instance of generating files. I must be missing
> something obvious. There are zillions of generated files in the
> world. For instance, why aren't there problems when a small C file is
> created and compiled? That could easily take less than 1 second, if that
> is the mtime resolution.
I think, *if* the problem that was noticed in 2010 and that motivated the
change 5) still exists:
What is particular is that 'configure' and 'config.status' are part of a
circular dependency cycle, where Makefiles are being regenerated (and thus
'make' is reinitialized). Care must be taken to avoid these regenerations,
as they are costly. When a C or C++ file is created and compiled, the
Makefile stays the same.
Bruno
