Re: Bug Resilience Program of German Sovereign Tech Fund
I received an answer from the German Sovereign Tech Fund regarding our application for their bug resilience program. Automake is on the waiting list for next year. Thanks Christoph!
Re: First draft of application to Sovereign Tech Fund
Hi Detlef - I think Christoph has already submitted the application to the fund, in somewhat different form than the one you commented on. There are some subsequent msgs between he and I on the list. (In short, I basically agree[d] with all your comments. We probably didn't get everything right for the submission, but hopefully close enough.) We need a list of "todo projects", similar to what is available for projects, who apply for Google Summer of Code. There are, in fact, no such projects. Maintenance is not the same as summer of code. What I put near the top of the https://gnu.org/s/automake web page, years ago, was a link to a message I wrote that said, basically, "please look at the open bug list if you're willing to help". https://lists.gnu.org/archive/html/automake/2021-03/msg00018.html Zack wrote: I'd suggest that the most useful way for a new contributor to help at this time is to analyze the backlog of bug reports. That's a good point. I guess I should combine that idea with my post and recast it as a web page in the automake directory. Or maybe this is something a volunteer would like to do :)? BTW, I'd like to take this opportunity to thank Bogdan, who spontaneously came forward a couple years ago and has, indeed, analyzed a ton of reports, written good patches for a bunch, and generally been extremely helpful in keeping Automake going. Happy hacking to all, Karl
Re: First draft of application to Sovereign Tech Fund
I added your paragraph to the section "Describe why your project needs those services?" Sounds good. Can I submit the application? Yes. Time to give it a try. Thanks for the initiative. --best, karl.
Re: First draft of application to Sovereign Tech Fund
1) It looks to me like GCC uses autoconf but not automake. Maybe change the answer to: > Where is your open source technology project being > used (describe all user bases)? GNU Automake is used by a very large number of GNU and non-GNU packages. Here are just a couple of examples: GNU coreutils, GNU findutils, GNU gettext, libpng, TeX Live. 2) ust a typo in: Plenty of projects haved switched to other build systems for various s/haved/have/ 3) > Which BRP services are you interested in? Maybe here, or somewhere, I wonder about inserting: Since the Free Software Foundation holds the sole copyright on GNU Automake, it will be necessary for any significant contributor, whether individual or corporate, to sign a disclaimer or assignment of copyright before contributions can be accepted. Please confirm that this is acceptable. I still worry that this is going to be a stopper. Or at least a big delayer. The last thing I want is for us to go through this whole process and then neighborhoodie "can't" sign the assignment. Or for them to start posting changes but I can't install them because the assignment hasn't happened. Wdyt? --thanks, karl.
Re: improve display of filesystem timestamp resolution
A physical entity should always be displayed with its unit. Agreed in principle, but sorry, I'm not ready to install this one. Adding a unit to the value and then removing it everywhere the value is used seems error-prone to me. How about just displaying it with the unit (somehow)? Or having two values, one for the display with the unit and one without for use. Also, I don't much want to use a new shell construct just for a cosmetic purpose like this, even if it is supposedly sufficiently portable. Which I simply doubt on principle, given our past experiences. Finally, and trivially, I think "s" would be more in keeping with what I've seen used for time units in other places (e.g., the sleep option) than " sec". --thanks, karl.
Re: improve display of whether sleep supports fractional seconds
checking whether sleep supports fractional seconds... true is inconsistent with the rest of the GNU Build System. Namely, in configure output, results are "yes"/"no", not "true"/"false". The attached patch makes automake consistent with the rest. Thanks Bruno. I applied, plus "quoting" some of the $am_cv values that I happened to see around there. (No doubt more could be done in that regard, but anyway.) -k - automake: display whether sleep supports fractional seconds as yes/no. https://lists.gnu.org/archive/html/automake/2024-07/msg3.html * m4/sanity.m4 (_AM_SLEEP_FRACTIONAL_SECONDS): Set am_cv_sleep_fractional_seconds to yes/no, not true/false, for consistency with other results. Also "quote" this and other $am_cv values. (_AM_FILESYSTEM_TIMESTAMP_RESOLUTION): Test for yes/no. * t/ax/test-defs.in: Set am_cv_sleep_fractional_seconds to 'no', not 'false'. diff --git a/m4/sanity.m4 b/m4/sanity.m4 index 9e556b33a..c7f32daf8 100644 --- a/m4/sanity.m4 +++ b/m4/sanity.m4 @@ -11,8 +11,8 @@ AC_DEFUN([_AM_SLEEP_FRACTIONAL_SECONDS], [dnl AC_CACHE_CHECK([whether sleep supports fractional seconds], am_cv_sleep_fractional_seconds, [dnl -AS_IF([sleep 0.001 2>/dev/null], [am_cv_sleep_fractional_seconds=true], - [am_cv_sleep_fractional_seconds=false]) +AS_IF([sleep 0.001 2>/dev/null], [am_cv_sleep_fractional_seconds=yes], + [am_cv_sleep_fractional_seconds=no]) ])]) # _AM_FILESYSTEM_TIMESTAMP_RESOLUTION @@ -43,7 +43,7 @@ am_cv_filesystem_timestamp_resolution=2 # packages. # am_try_resolutions= -if $am_cv_sleep_fractional_seconds; then +if test "$am_cv_sleep_fractional_seconds" = yes; then # Even a millisecond often causes a bunch of false positives, # so just try a hundredth of a second. The time saved between .001 and # .01 is not terribly consequential. @@ -198,12 +198,12 @@ for am_try in 1 2; do break fi # Just in case. - sleep $am_cv_filesystem_timestamp_resolution + sleep "$am_cv_filesystem_timestamp_resolution" am_has_slept=yes done AC_MSG_RESULT([$am_build_env_is_sane]) -if test $am_build_env_is_sane = no; then +if test "$am_build_env_is_sane" = no; then AC_MSG_ERROR([newly created file is older than distributed files! Check your system clock]) fi @@ -212,7 +212,7 @@ fi # generated files are strictly newer. am_sleep_pid= AS_IF([test -e conftest.file || grep 'slept: no' conftest.file >/dev/null 2>&1],, [dnl - ( sleep $am_cv_filesystem_timestamp_resolution ) & + ( sleep "$am_cv_filesystem_timestamp_resolution" ) & am_sleep_pid=$! ]) AC_CONFIG_COMMANDS_PRE( diff --git a/t/ax/test-defs.in b/t/ax/test-defs.in index aac5d60e3..542305788 100644 --- a/t/ax/test-defs.in +++ b/t/ax/test-defs.in @@ -211,7 +211,7 @@ case $MTIME_RESOLUTION in MTIME_RESOLUTION=1 # we must not sleep for fractional seconds when autom4te does not # support subsecond-mtimes, even when sleep supports it. See bug#67670. - am_cv_sleep_fractional_seconds=false + am_cv_sleep_fractional_seconds=no export am_cv_sleep_fractional_seconds fi ;; @@ -232,7 +232,7 @@ esac # scripts in other packages, too. # # At any rate, with this set to "sleep 1", there were still random -# parallelization failures; setting am_cv_sleep_fractional_seconds=false +# parallelization failures; setting am_cv_sleep_fractional_seconds=no # as above was still needed. # sleep="sleep $MTIME_RESOLUTION"
Re: [platform-testers] automake-1.16.92 released
Hi Frederic,
NetworkManager:
Use of uninitialized value $var in string eq at
/usr/share/automake-1.16/Automake/Variable.pm line 754, line 1169.
>From the Makefile.am you sent me separately (attached here for the
record), it seems that is coming from the use of $() in:
introspection/%.h: introspection/%.c
$()
Accordingly, I changed scan_variable_expansions (in Variable.pm) to not
worry if the variable name is empty (patch and test below). Could you
retry NetworkManager or the other packages and make sure the Perl
warning is gone, please?
It wasn't entirely clear to me if the line
_vapi_name = $(subst /,_,$(subst -,_,$(subst .,_,$(1
might also be causing trouble, but I think Automake already recognizes
that ok (and gives a warning, by default, but an Automake warning, not a
Perl warning.)
As an aside, I'm curious as to why the $() is used. It seems
a mysterious way to do nothing. Do you know? --thanks, karl.
-
automake: avoid Perl-level warning on empty variable $().
https://lists.gnu.org/archive/html/automake/2024-06/msg00085.html
* lib/Automake/Variable.pm (scan_variable_expansions): recognize
and do nothing if the variable name is empty: $().
* t/varempty.sh: new test.
* t/list-of-tests.mk (handwritten_TESTS): add it.
diff --git a/lib/Automake/Variable.pm b/lib/Automake/Variable.pm
index db1f6378d..f97aab59f 100644
--- a/lib/Automake/Variable.pm
+++ b/lib/Automake/Variable.pm
@@ -751,7 +751,11 @@ sub scan_variable_expansions ($)
while ($text =~ m{\$(?:\{([^\}]*)\}|\(([^\)]*)\)|(\$))}g)
{
my $var = $1 || $2 || $3;
- next if $var eq '$';
+ next if (! defined $var) || ($var eq '$');
+ # we check for $var being defined because NetworkManager and other
+ # packages use the strange construct $().
+ # https://lists.gnu.org/archive/html/automake/2024-06/msg00085.html
+
# The occurrence may look like $(string1[:subst1=[subst2]]) but
# we want only 'string1'.
$var =~ s/:[^:=]*=[^=]*$//;
diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk
index 1e0f364ba..e80ace470 100644
--- a/t/list-of-tests.mk
+++ b/t/list-of-tests.mk
@@ -1282,6 +1282,7 @@ t/vala-per-target-flags.sh \
t/vala-recursive-setup.sh \
t/vala-vapi.sh \
t/vala-vpath.sh \
+t/varempty.sh \
t/vars.sh \
t/vars3.sh \
t/var-recurs.sh \
diff --git a/t/varempty.sh b/t/varempty.sh
new file mode 100644
index 0..9eb45c421
--- /dev/null
+++ b/t/varempty.sh
@@ -0,0 +1,36 @@
+#! /bin/sh
+# Copyright (C) 2024 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+# An empty variable name $() should not cause a Perl warning, namely:
+# Use of uninitialized value $var in string eq at
+# .../lib/Automake/Variable.pm line 754, line 3.
+# (in scan_variable_expansions)
+#
+# This showed up with the NetworkManager and other packages in Fedora:
+# https://lists.gnu.org/archive/html/automake/2024-06/msg00085.html
+# (The actual purpose of the "$()" is unclear.)
+
+. test-init.sh
+
+cat > Makefile.am << 'END'
+x:
+ $()
+END
+
+$ACLOCAL
+$AUTOMAKE
+
+:
Running command: git commit \-q \-F \.\/vc\-dwim\-log\-wBh6_U \-\-author\=Karl\
Berry\ \ \-\- ChangeLog
+ set +x
compile finished at Sun Jun 30 13:26:51 2024
Makefile.am.gz
Description: Binary data
Makefile.vapigen.gz
Description: Binary data
Re: First draft of application to Sovereign Tech Fund
Hi Christoph and all - back on the Sovereign Tech Fund application. Here's my first cut at tweaking the wording. Thanks for doing all this. -k Draft of applications (questions are with > at the left of the line): > I acknowledge: > > All code and documentation to be supported must be licensed such > that it may be freely reusable, changeable and redistributable As mentioned, we will need Neighborhoodie to sign a copyright assignment/disclaimer to the FSF. > Project title Autoconf & Automake Direct Contribution 2024 I think autoconf and automake should be submitted separately? Even though there is clearly a lot of overlap, and technically automake is entirely dependent on autoconf, the two projects are implemented quite differently, have separate bug lists, separate maintainers, etc. It is possible to work on Automake without ever delving deeply into Autoconf (and of course the reverse is even more true :). I'm going to adjust the rest to emphasize Automake. > Link to project website https://www.gnu.org/software/automake/ > Link to project repository https://git.savannah.gnu.org/cgit/automake.git > Where is your open source technology project being used (describe > all user bases)? GNU Autoconf and GNU Automake are the core of the GNU Autotools. They help in making source code packages portable to virtually all Unix-like systems. They are used by GNU GCC, LibreOffice, and OpenJDK, among thousands of other packages. The interface that they implement is the standard for GNU. > Why do you consider your open source technology project to be > relevant and critical? Many projects rely on Automake (and Autoconf) as crucial part of their build system. Moving to alternatives is challenging. Without a working build system, users might not be able to build the software using the build system. > How does your open source technology benefit the public interest? A build system is like water or electricity, it is supposed to work and nobody thinks about it. Once service is disrupted, we realize how much depends on it. Many users never know of Automake. The programmers and packagers work behind the scenes, relying on the build systems effectiveness. > Please describe the history and state of development of your open > source technology Automake automatically generates input files for Autoconf and adds dependency tracking. Version 1.0 was released in 1996. Automake is (primarily) implemented in Perl, and also uses m4 and shell scripts. Automake is completely dependent on Autoconf, which is an extensible package of M4 macros that produce shell scripts to automatically configure software source code packages. Autoconf and to a somewhat lesser extent Automake became very popular in the 2000s, as extremely portable and adaptable build systems. It is often possible to configure a package for compilation on a new system without user intervention. Plenty of projects haved switched to other build systems for various reasons, but numerous projects still rely on the Autotools. > Which BRP services are you interested in? Direct contributions, particularly fixes for open bugs. https://debbugs.gnu.org/cgi/pkgreport.cgi?package=automake >Describe why your project needs those services? Automake (and Autoconf) have relatively few contributors, and thus releases are infrequent and some bugs, especially system-dependent ones, have remained unfixed for years. The many dependencies (GNU m4, GNU libtool, plus (f)lex, bison/yacc, compilers, etc.) make it relatively difficult for newcomers to make substantive contributions without significant effort. On the other hand, C, C++, Python, and other languages are evolving, and the new versions of these language standards are often backward-incompatible with existing code and practice. Thus the Autotools must adjust.
Re: automake-1.16.92 released
Hi Dave, in case it affects a decision to release 1.17 Indeed. Thank you very much for the report (and the followup). The first question that comes to mind: are you using the same version of libtool in the various cases? --thanks again, karl.
Re: RFC: Add public macros AC_LOG_CMD and AC_LOG_FILE.
Subject: RFC: Add public macros AC_LOG_CMD and AC_LOG_FILE. FWIW, it sounds good to me. To my mind, logging is one of the most important features of autoconf, so I'm all for macros to support it further. --thanks, karl.
Re: [platform-testers] automake-1.16.92 released
I ran a mass rebuild on Fedora for packages that depend on automake with
this version.
Thank you!!
Out of 1330 packages built, I found the following failures.
Could be a lot worse!
NetworkManager
x2gokdrive
xorg-x11-server
Use of uninitialized value $var in string eq at
/usr/share/automake-1.16/Automake/Variable.pm line 754, line 47.
Any chance you can send the Makefile.am for any of those packages?
Line 754 of Variable.pm is "scan_variable_expansions":
751 while ($text =~ m{\$(?:\{([^\}]*)\}|\(([^\)]*)\)|(\$))}g)
752{
753 my $var = $1 || $2 || $3;
754 next if $var eq '$';
So I guess something in the variable definition is not being recognized
properly. (And I don't think this has changed, and I also doubt the
Makefile.am's are in error, so it's a bit puzzling.) It would be easy
enough to force it to the empty string and avoid the uninitialized
warning, but it would be nice to understand what's going on.
ddclient:
automake: warnings are treated as errors
Makefile.am:20: warning: escaping \# comment markers is not portable
Indeed, I think ddclient's Makefile.am just needs to avoid \#.
I don't see anything to do on the automake side for that one.
Thanks again,
Karl
Re: branch master updated: lib scripts: add "(GNU Automake)" to --version output, etc.
Why two blank lines, not just one as for the other scripts? Slip of the editor. Will fix. Thanks for the sharp eyes. -k
Re: 1.16.90 regression: configure now takes 7 seconds to start
have to say --enable-maintainer-mode Personally I agree with you, in theory, but I think it is too big of a change to make. As I understand it. GNU package authors have their source repos and have never had to say --enable-maintainer-mode before to get their Makefiles etc. rebuilt. I don't think that would be a welcome change. Maintainers who want it can choose that (AM_MAINTAINER_MODE), but I don't think we should change the default. -k
Re: 1.16.90 regression: configure now takes 7 seconds to start
Let's look at the history. Thanks very much for all that research and explanations, Bruno. Likewise, Jacob. https://lists.gnu.org/archive/html/bug-automake/2010-10/msg0.html Aha, the explanation for some of the $sleep commands scattered throughout the Automake tests! I had no idea. I'll add a comment at some point. (I myself added more, in vain attempts to work around the timing problems, but I don't think they should be necessary, in theory. Not that I plan to remove them.) This patch is conservative; IMO safe for a 1.17 release. I like the idea of a conservative patch, for sure. I see that what you're doing here is simply omitting the 1 second trial for the sake of saving that time, in exchange for accepting a 2s delay in the Automake tests on 1s filesystems. That seems fine to me. This patch is risky. I think it's safe to commit only if followed by a 1-month testing period, I installed your first patch. where we can see if people report failures like they were reported in September 2010. I don't see any reason why the problem would have gone away. The problem with this patch is that it reverts the change 5) in the Perhaps the AC_CONFIG_COMMANDS_POST as proposed by Nick and Jacob would ameliorate that. It doesn't incur any delay on reasonable systems. I'll reply separately about that. --thanks, karl.
Re: 1.16.90 regression: configure now takes 7 seconds to start
Nick>
AC_CONFIG_COMMANDS_POST([cat >conftest.mk <<'EOF'
configure: config.status
false
EOF
while ${MAKE-make} -f conftest.mk >/dev/null 2>&1
do
touch config.status
done])
One missing element is that there is no limit, which would be a bit of a
problem if the clock skew is severe (e.g., if configure's mtime is years
or even minutes in the future), so something extra is probably desirable
to bound the amount of time this runs to something practical.
It's easy enough to use a shell variable and limit it to 100(0) runs or
whatever.
But I have to wonder about inserting the filesystem sleep :(. Otherwise
it seems like all the runs could easily happen "at once" (e.g., in
memory/caches) and nothing would change.
Since on reasonable systems, as you say, the loop would never be
entered, it shouldn't create a delay.
Anyway, later for that. --thanks, karl.
Re: Bug Resilience Program of German Sovereign Tech Fund
Christoph - I had one more thought about the tech fund: Automake is copyright FSF. So Neighorhoodie is going to need to sign a disclaimer or (preferably) assignment for any patches to be usable. Is there precedent in the fund for doing that? --thanks, karl.
Re: use of make in AM_SANITY_CHECK (was: improved timestamp resolution test)
make(1) in AM_SANITY_CHECK seems to be a logic error, since the user may want to build with a different $MAKE, You're right. Crap. It never ends. In practice it probably doesn't matter, though. Although in theory one can imagine that "make" succeeds while $MAKE fails, resulting in a false positive, in practice that seems next to zero probability to me. Much more likely is that "make" fails and $MAKE succeeds, and the only downside of that is an extra second of sleeping. have a way to revise AM_SANITY_CHECK that can avoid any sleep in the most common cases. Bruno's last patch already does that, doesn't it? I'll apply it shortly. Are you willing to consider patches on this? I guess. The more I look at this, the less I understand it. Especially because I generally cannot reproduce the problems/situations that people are trying to fix with this endless series of patches. --thanks, karl.
Re: 1.16.90 regression: configure now takes 7 seconds to start
Find here attached a revised proposed patch. Ok on the reorg, but sorry, I remain confused. This whole thing started with Mike Vapier's change in Feb 2022 (commit 720a11531): https://lists.gnu.org/archive/html/automake-commit/2022-02/msg9.html As I read it now, his goal was to speed up other projects, not Automake, by reducing the "sleep 1" to "sleep " in AM_SANITY_CHECK, via AC_CONFIG_COMMANDS_PRE, i.e., before creating config.status. But that is only one instance of generating files. I must be missing something obvious. There are zillions of generated files in the world. For instance, why aren't there problems when a small C file is created and compiled? That could easily take less than 1 second, if that is the mtime resolution. I understand that equal timestamps are considered up to date, and presumably the .c and .o (say) would be equal in such a case. Ok, but then why is configure generating config.status/etc. such a special case that it requires the sleep, and nothing else? I mean, I know the sleep is needed; I've experienced the problems without that sleep myself. But I don't understand why it's the only place (in normal compilations; forget the Automake test suite specifically) that needs it. Can someone please educate me as to what is really going on underneath all this endless agonizing tweaking of the mtime tests? Sorry for my stupidity. --thanks, karl.
Re: Bug Resilience Program of German Sovereign Tech Fund
Hi Christoph, All Automake release announcements since 2.13 warn about future Automake 2.0 incompatibilities. I moved that noisy warning from the top of NEWS into a separate NEWS-2.0 file a couple of releases ago. At the beginning there is a meeting between you (maybe more people, I can take part, too) and the developers. Citing the FAQ: I see. Well, that's fine, but I won't have much to say beyond "chase open bugs". Many are system-specific. Others have had long discussions with no clear resolution possible (but I still left them open). Although it would be nice if we could say "these are the most important bugs to work on", I can't. Triaging the bug list is itself a time-consuming job that I can't take on. Does anyone else reading here have ideas, or would like to participate in such a meeting? Any other thoughts in general on this? Please share text improvements with me, once we decide to apply. Ack. Thanks, Karl
Re: Bug Resilience Program of German Sovereign Tech Fund
Hi Christoph, I advice everybody so seek alternatives. I'm sorry to hear it. My own personal experience (with my sysadmin hat on) is that all alternatives have been inferior. Which is, ultimately, why I choose to spend my time here. I would love to see any minor improvement, as the documentation left me puzzled multiple times. Me too. But no single document can easily answer everyone's questions every time. If there are specific lacuna or unclear things in the manual, then I'm happy to work on that. Nobody wants to rewrite Automake in Rust or replace M4 by Python ;-) Ok, good. That sort of thing is exactly what I was imagining :). My interpretation for Autocond and Automake would be to adjust issues with newer version of the used technologies I don't think there is any "technology", i.e., program, where we can assume a newer version. Whenever we try to use some "new" feature of, say, Perl (like, a mere decade or two old), we get complaints. And let's not talk about shells. or adjusting tests to newer C and C++ standards and the stricter interpretation of modern compilers. Ok, but we have to continue to support older compilers. This is one of the biggest problems nowadays. Trying to support tool X on system Y, where every release of the tool and every release of the OS have a different set of bugs and/or requirements. It's maddening. It could also help out on the way to Automake 2.0. As long as I'm the primary person driving Automake, there will not be an Automake 2.0. I have no plans to ever work on that, and see no need for it, either. Just keeping up with the churn induced by language "standard" committees and compiler and OS backward incompatibilities, which are done at the drop of a hat nowadays, is clearly beyond me. It is very frustrating to have to work so hard to stay in the same place. Do you feel like not trying it in the first place? Let me ask this: if we end up being accepted, what are we committing ourselves to do? Anything? (wrt to the neighborhoodies or the government.) Submit a report? Assuming we aren't forced to do things like accept patches we don't want, or explain endlessly how m4 works, overall, I guess my feeling is that there's probably no harm in trying. Despite our skepticism, maybe we will be pleasantly surprised and get some useful results. That would be nice. I have a bunch of wording issues with your proposed text, but that is minor. Thanks, Karl
Re: improved timestamp resolution test (was: 1.16.90 regression: configure now takes 7 seconds to start)
Jacob, [*sigh*] You said it. About this whole thing. I rather wish this bright idea had never come to pass. It has delayed the release by months. Oh well. Still, could we use make(1) for *all* of the testing and not use `ls -t` I guess it is technically possible, but I somehow feel doubtful about relying entirely on make. Using ls already has plenty of portability issues; I shudder to think how many strange problems we'll run into when we start exercising timing edge cases in make. Also, it's taken months to get the current code to even be vaguely usable; I don't much like the idea of starting over. Also, with Bruno's patch (once I understand it, anyway), I guess the whole problem goes away. Which would be the best outcome. Thanks, Karl
Re: 1.16.90 regression: configure now takes 7 seconds to start
Sorry, but I don't really understand your patch. With it, it would seem that _AM_FILESYSTEM_TIMESTAMP_RESOLUTION is only called if the build environment is not "sane", i.e., a created file is not newer than configure. Is that right? (That embedded shell construct is hard to grok.) is in _AM_FILESYSTEM_TIMESTAMP_RESOLUTION, whose sole purpose is to determine a variable am_cv_filesystem_timestamp_resolution. The sole use of this variable is in AM_SANITY_CHECK, The timestamp resolution also used in Automake's t/ax/test-defs.in (->.sh), and Automake's configure.ac, aka MTIME_RESOLUTION. With your patch, how does it ever get defined? Did you intend that Automake's configure.ac would then explicitly call _AM_FILESYSTEM_TIMESTAMP_RESOLUTION (in which case it should probably be renamed without a leading underscore), or am I misunderstanding the whole thing? (Quite likely.) --thanks, karl.
Re: 1.16.90 regression: configure now takes 7 seconds to start
Does BSD ls(1) support "--time=ctime --time-style=full-iso"? BSD ls does not support any --longopts. Looking at the man page, I don't see "millisecond" or "subsecond" etc. mentioned, though I could easily be missing it. E.g., https://man.freebsd.org/cgi/man.cgi?ls Even if there is such an option, I am skeptical of how portable it would be, or trying to discern whether it is really working or not. All the evidence so far is that it is very difficult to determine whether subsecond mtimes are sufficiently supported or not. Speaking in general, I don't think trying to get into system-specific behaviors, of whatever kind, is going to help. --thanks, karl.
Re: 1.16.90 regression: configure now takes 7 seconds to start
It should save 6 seconds. Because it goes like this: - Test whether 0.01 sec works. - Test whether 0.1 sec works. - If not, set the variable to 2, because that's the worst case and it *must* work. At which places will then a 'sleep 2' be done where (if not VFAT) a 'sleep 1' would be sufficient? Sorry, I don't understand what you're saying. When I look at the code, I see it checking .01, .1, and then 1. If all those fail, then it falls back to 2 (but never tests it). am_cv_filesystem_timestamp_resolution=2 am_try_resolutions=1 .. am_try_resolutions="0.01 0.1 $am_try_resolutions" .. for am_try_res in $am_try_resolutions; do ... sleep $am_try_res twice for ls and four times for make ... if ; then am_cv_filesystem_timestamp_resolution=$am_try_res break fi done At which places will then a 'sleep 2' be done sleep 2 is currently never done, so far as I can see. Anyway, I just committed the change to avoid the make-related sleeps if the mtime_resolution is 1s. It's what I can see ... -k - automake: omit make subsecond tests if at one-second resolution. Partially ameliorates report from Bruno Haible, thread at https://lists.gnu.org/archive/html/automake/2024-06/msg00015.html * m4/sanity.m4 (_AM_FILESYSTEM_TIMESTAMP_RESOLUTION): if resolution is only one second, assume make works, to avoid several sleeps. diff --git a/m4/sanity.m4 b/m4/sanity.m4 index 09489ffaa..68fbf36f7 100644 --- a/m4/sanity.m4 +++ b/m4/sanity.m4 @@ -90,40 +90,46 @@ for am_try_res in $am_try_resolutions; do test "$[]3" = conftest.ts2 && test "$[]4" = conftest.ts1); then # -# Ok, ls -t worked. We have one more thing to check: make. -# It can happen that everything else supports the subsecond mtimes, -# but make doesn't, notably on macOS, which ships make 3.81 from -# 2006 (the last one released under GPLv2). https://bugs.gnu.org/68808 -# -# So, first let's create a Makefile: -rm -f conftest.mk -echo 'conftest.ts1: conftest.ts2' >conftest.mk -echo ' touch conftest.ts2' >>conftest.mk -# -# Now, running -# touch conftest.ts1; touch conftest.ts2; make -# should touch ts1 because ts2 is newer. This could happen by luck, -# but most often, it will fail if make's support is insufficient. So -# test for several consecutive successes. -# -# (We reuse conftest.ts[12] because we still want to modify existing -# files, not create new ones, per above.) -n=0 +# Ok, ls -t worked. If we're at a resolution of 1 second, we're done, +# because we don't need to test make. make_ok=true -until test $n -eq 4; do - echo one > conftest.ts1 - sleep $am_try_res - echo two > conftest.ts2 # ts2 should now be newer than ts1 - if make -f conftest.mk | grep 'up to date' >/dev/null; then -make_ok=false -break # out of $n loop - fi - n=`expr $n + 1` -done +if test $am_try_res != 1; then + # But if we've succeeded so far with a subsecond resolution, we + # have one more thing to check: make. It can happen that + # everything else supports the subsecond mtimes, but make doesn't; + # notably on macOS, which ships make 3.81 from 2006 (the last one + # released under GPLv2). https://bugs.gnu.org/68808 + # + # So, first let's create a Makefile (real tab character): + rm -f conftest.mk + echo 'conftest.ts1: conftest.ts2' >conftest.mk + echo ' touch conftest.ts2' >>conftest.mk + # + # Now, running + # touch conftest.ts1; touch conftest.ts2; make + # should touch ts1 because ts2 is newer. This could happen by luck, + # but most often, it will fail if make's support is insufficient. So + # test for several consecutive successes. + # + # (We reuse conftest.ts[12] because we still want to modify existing + # files, not create new ones, per above.) + n=0 + until test $n -eq 3; do +echo one > conftest.ts1 +sleep $am_try_res +echo two > conftest.ts2 # ts2 should now be newer than ts1 +if make -f conftest.mk | grep 'up to date' >/dev/null; then + make_ok=false + break # out of $n loop +fi +n=`expr $n + 1` + done +fi +# if $make_ok; then # Everything we know to check worked out, so call this resolution good. am_cv_filesystem_timestamp_resolution=$am_try_res - break # out of resolution loop + break # out of $am_try_res loop fi # Otherwise, we'll go on to check the next resolution. fi compile finished at Wed Jun 12 18:24:14 2024
Re: 1.16.90 regression: configure now takes 7 seconds to start
> Any other ideas? A horrible one: :) split the test so it can be performed in parallel with others -- upper half starts the sleep in the background, lower half waits for it to complete. No thanks. I can't even begin to imagine the portability and edge case problems that would result :). -k
Re: 1.16.90 regression: configure now takes 7 seconds to start
Maybe this is a silly question, but, is there a reason why this test needs to be performed in every single package that uses Automake? I was under the impression that the purpose of this test was merely to speed up running Automake's own test suite. There are other packages that have many tests that presumably also substantially benefit? If it only benefitted automake, it would not have been worth it, given the significant complications and hassle that have ensued. But since all that work has already been done, I'd hate to throw it away now. I'll put a note in NEWS about how to effectively override the test, i.e., am_cv_filesystem_timestamp_resolution=1 export am_cv_filesystem_timestamp_resolution I believe (untested). -k
Re: Bug Resilience Program of German Sovereign Tech Fund
Hi Christoph, https://www.sovereigntechfund.de/programs/bug-resilience Thanks for bringing this up. > Our partner 'Neighbourhoodie Software' provides a variety > of types of contributions to participating projects to > address known issues, If that means providing patches for open bugs, then great. That is what is needed. https://debbugs.gnu.org/cgi/pkgreport.cgi?package=automake However, going along with what Paul said, I am skeptical that there is any real chance that someone from "Neighborhoodie Software" knows or will learn m4 + sh + perl + etc. enough to actually do anything useful. Looking at https://neighbourhood.ie/areas-of-expertise (assuming that's the company in question), I see nothing about low-level Unix stuff. Just current high-level web things. On the other hand, as you say, spending tax money on free software in whatever way is better than many other uses, regardless of whether anything useful comes out of it. > improve documentation, I am equally skeptical that that would happen. > and reduce technical debt. I don't know what that means. I instinctively shy away from such vague buzzwords. As for "modernizing" autoconf/make, mentioned in other msgs, that's the last thing that should be done. We go to a lot of trouble to make the tools work on old systems that no one else supports. For example, I can just picture them saying "oh yes, you should use $(...) instead of `...`" and other such "modern" shell constructs. Or "use Perl module xyz to simplify", where xyz only became available a few years ago. Etc. My biggest concern is that I do not want to spend the little time I have "explaining" to people, who are supposedly there to help, how these packages work and what the basic approach is. (And I'm sure that all the rest of us doing the development feel the same way.) I fear that is exactly what will happen. --best, karl.
Re: 1.16.90 regression: configure now takes 7 seconds to start
bh> Seen e.g. on NetBSD 10.0. Which doesn't support subsecond mtimes? jb> Maybe the best answer is to test for subsecond timestamp granularity first, and then only do the slow test to distinguish between 1-second and 2-second granularity if the subsecond granularity test gives a negative result? Unfortunately, that is already the case. The function (_AM_FILESYSTEM_TIMESTAMP_RESOLUTION in m4/sanity.m4) does the tests starting with .01, then .1, then 1. Searching for sleep [012] in Bruno's log confirms this. If I understand correctly, Bruno's goal is to omit the "1" test if we can detect that we're not on a fat filesystem. But I admit I don't like trying to inspect filesystem types. That way lies madness, it seems to me, and this whole function is already maddening enough. E.g., mount and/or df could hang if NFS is involved. It seems to me that using stat doesn't help because it's not available by default on the bsds etc., as Bruno pointed out. The simple change is to omit the make test if we are at resolution 1. That will save 4 seconds. Omitting it is justified because the make test is only there for the sake of makes that are broken wrt subsecond timestamps. I will do that. That will leave 2+ sec of sleeping, but if we are to reliably support fat, I don't see a good alternative. At least it's not as bad as 6+. Any other ideas? karl
Re: 1.16.90 regression: configure now takes 7 seconds to start
Date: Sat, 08 Jun 2024 00:27:39 +0200 From: Bruno Haible To: Subject: 1.16.90 regression: configure now takes 7 seconds to start [I'm writing to automake@gnu.org because bug-autom...@gnu.org appears to be equivalent to /dev/null: [...] It seems this has been fixed, whatever was going on then. I just sent a test message to bug-automake around 15:12 PDT today, and got an acknowledgement about 15 minutes later that bug#71497 had been created. (Then I closed the bug.) -k
Re: automake 1.17 release plan?
Hi Christoph, end of 2023 you shared the news that automake release 1.17 could happen Indeed. Unfortunately since then I have had to attend to other priority projects, and there's no one else driving automake. I hope to be able to spend some time on automake again soon to bring the release to fruition. I have no ETA though, as ever. Sorry. There has been no public communication for the last couple of weeks. Well, I (or someone) is still replying to questions and bugs on automake@gnu.org and bug-autom...@gnu.org. Not about the release, though, granted. What are your plans regarding 1.17? Release as soon as possible. Are there blockers? Yes. The last pretest in December turned up at least one: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68808 It requires hacking a function in sanity.m4 to probe make for its "subsecond mtime" behavior. There may be others, I'd have to look at the recent pending bugs. Is there anything we as the community can help? Well, a patch for the bug above would certainly be helpful. Patches in general for any other bugs, especially the recent ones, would be all to the good. Other than actual patches, reviewing the other recent pending bugs to see if any others are important enough to require fixing before the release, or simple enough to just fix and be done with it, would also be helpful. (I know there are a few other patches simply waiting for me to review and install.) Beyond that, three years ago I wrote a message requesting more volunteers. The information is (sadly enough) still current: https://lists.gnu.org/archive/html/automake/2021-03/msg00018.html Thanks, Karl P.S. BTW, did something happen somewhere that prompted you to send this? I ask because another person wrote me off-list today with the same question. Seems a bit much for coincidence.
Re: How to force libtool to use CXX mode?
Automake does have a critical bug in that for a target which only optionally has C++ sources, that target is always linked using C++. So it should only use C++ if the "option" is selected? Can you provide a test tree? Without this issue, the trick of including an empty optional C++ source file in the build would work. I don't yet understand how all this is playing together ... -k
Re: How to force libtool to use CXX mode?
convince Automake to force libtool to link using the C++ compiler When there are no C++ sources? Why? Just trying to understand ... I'm sorry Bob, but I just don't know. Maybe the just-released libtool-2.5.0 alpha offers some new help? If there is some bug in or feature for Automake that would help, I'm open to suggestions (and patches). It kind of sounds like more on the libtool side? --sorry, karl.
Re: GNU Coding Standards, automake, and the recent xz-utils backdoor
I'm also wondering whether the GNU system should recommend using zstd instead of or in addition to xz for compression purposes. I'm not sure GNU explicitly recommends anything. Although the tarball examples in standards.texi and maintain.texi all use gz, I don't think even gz is explicitly recommended. (Which seems ok to me.) Personally, I would support lz4 over zstd simply because more GNU packages already use lz4.(*) Both lz4 and zstd are quite a bit less resource-hungry than xz, especially for compression. I don't know if there is a technical reason to prefer zstd. In general, I think it can continue to be left up to individual maintainers, vs. making any decrees. Automake supports them all (among others). --best, karl. (*) Looking at a listing of ftp.gnu.org, I see only gmp using zst, and perhaps a dozen or so packages using lz. Basically always in addition to another format.
Re: GNU Coding Standards, automake, and the recent xz-utils backdoor
`distcheck` target's prominence to recommend it in the "Standard Targets for All Users" section of the GCS? Replying as an Automake developer, I have nothing against it in principle, but it's clearly up to the GNU coding standards maintainers. As far as I know, that's still rms (for anything substantive) FWIW, I expect that few users would actually run make distcheck, regardless of anything in the GCS. And of those that do, I suspect there would be many failures because make distcheck is a complex target that is not, so far as I understand it, intended to be as perfectly portable or prerequisite-free as other targets. No doubt it could be improved in that regard. Not a stopper, just my thoughts. Thanks for the suggestion, Karl
Re: C library promoted to C++ linkage due to optional C++ source module
Hi Bob, It is my opinion that if a library or program is linked with C++ libraries (and especially if it is a static build!) that it should be linked using the C++ linker. Likewise, if a library or program does not depend on, or contain any C++ code, it should be linked with the C linker. I surely agree, in principle. But that doesn't mean I have the faintest idea what to change in Automake to make that be the case. You probably know better than me :). Can you provide (as simple as possible) test cases? I have a continuing problem that when libtool tests bug-libt...@gnu.org for that one, I think ... --best, karl.
purpose of line ".. contents:: :depth: 2" in test-suite.log?
Does anyone know what this line in automake-generated test-suite.log files is used by or intended for? . contents:: :depth: 2 It appears to be machine-parsable, but I can't find any reference to it in the code or the doc. There doesn't seem to be anything that outputs :depth: 3 (or anything else), either. The above is just a static string in lib/am/check.am: echo ".. contents:: :depth: 2"; \ Just wondering. I don't plan to change it. --thanks, karl.
automake-1.16j pretest available: please test
The GNU Automake 1.16j development snapshot is now available. Download here: https://alpha.gnu.org/gnu/automake/automake-1.16j.tar.xz https://alpha.gnu.org/gnu/automake/automake-1.16j.tar.gz We intend for automake 1.17 to be released soon, essentially with only bug fixes for whatever is found in this pretest. So please do test if at all possible. The main change from 1.16i was another attempt at making the "subsecond mtime" support reliable and backward compatible. That is, to work with autoconf-2.72 (which supports subsecond mtimes) and earlier versions (which don't). See below for the detailed list of changes since the previous version, as summarized by the NEWS file. In general, it's our primary goal to preserve compatibility. If this release of Automake induces failure in a setup that was previously working, please let us know. Please report bugs and problems to , and send general comments and feedback to , and patches to . Thanks to everyone who has reported problems, contributed patches, and helped test Automake! With special thanks to Bogdan, Mike Frysinger, Nick Bowler, and Zack Weinberg for their numerous contributions. This release could not have happened without them. New in (what will be) 1.17: * New features added - AM_PATH_PYTHON will, after checking "python", prefer any Python 3 version (latest versions checked first) over any Python 2 version. If a specific version of Python 2 is still needed, the $PYTHON variable should be set beforehand. - AM_PATH_PYTHON will also search for Python versions 3.15 through 3.10. It previously searched for 3.9 through 3.0. (bug#53530) - RANLIB may be overridden on a per-target basis. - AM_TEXI2FLAGS may be defined to pass extra flags to TEXI2DVI & TEXI2PDF. - New option "posix" to emit the special target .POSIX for make. - Systems with non-POSIX "rm -f" behavior are now supported, and the prior intent to drop support for them has been reversed. The ACCEPT_INFERIOR_RM_PROGRAM setting no longer exists. - Variables using escaped \# will trigger portability warnings, but be retained when appended. GNU Make & BSD Makes are known to support it. (bug#7610) - GNU Make's default pattern rules are disabled, for speed and debugging. (.SUFFIXES was already cleared.) (bug#64743) - For Texinfo documents, if a .texi.in file exists, but no .texi, the .texi.in will be read. Texinfo source files need not be present at all, and if present, need not contain @setfilename. Then the file name as given in the Makefile.am will be used. If @setfilename is present, it should be the basename of the Texinfo file, extended with .info. (bug#54063) - The missing script also supports autoreconf, autogen, and perl. * Bugs fixed - Generated file timestamp checks handle filesystems with sub-second timestamp granularity dynamically, greatly speeding up make check, etc. However, this requires an autom4te from Autoconf 2.72 or later (or random test failures and other timing problems may ensue), as well as a Perl, sleep utility, and filesystem that supports sub-second resolution; otherwise, we fall back to one-second granularity as before. When everything is supported, a line `Features: subsecond-mtime' is now printed by automake --version and autom4te --version. (bug#64756, bug#67670) - The default value of $ARFLAGS is now "cr" instead of "cru", to better support deterministic builds. (bug#20082) - Dependency files are now empty, instead of '# dummy', for speed. - Compiling Python modules with Python 3.5+ uses multiple optimization levels. (bug#38043) - When compiling Emacs Lisp files, emacs is run with --no-site-file to disable user config files that might hang or access the terminal; and -Q is not used, since its support and behavior varies. (bug#58102) - Emacs Lisp compilations respects silent make output. - Automake no longer incorrectly warns that the POSIX make variables $(*D) and the like are non-POSIX. Unfortunately, the make implementations which do not correctly implement all the POSIX variables are not detected, but this seems to have little impact in practice. (bug#9587) - Pass libtool tags OBJC and OBJCXX for the respective languages. (bug#67539) - distcleancheck ignores "silly rename" files (.nfs* .smb* .__afs*) that can show up on network file systems. - tests: avoid some declaration conflicts for lex et al. on SunOS. (bug#34151 and others) - Pass any options given to AM_PROG_LEX on to AC_PROG_LEX. (bug#65600, bug#65730) * Obsolescence: - py-compile no longer supports Python 0.x or 1.x versions. Python 2.0, released in 2000, is currently the minimum required version.
Re: Typo in NEWS section "New in 1.17" and "New in 1.15"
While reading the announcement for 1.16i, I found a typo in the NEWS file "New in 1.17" section. I have also accidentally found a typo in the "New in 1.15" section, Thanks x 2, Hans. Applied. which might need a line rewrap after the fix. Nah, it's ok. I have not systematically looked for typos in any of the "New in ..." sections older than 1.17. I ran a spell checker and nothing obvious showed up. Not that that's conclusive, but it will have to do :). --thanks, karl.
Re: What does (did?) it mean when aclocal exits with code 63?
Hi Zack,
*automake* can exit with code 63 under
some circumstances, but it really looks like aclocal never will.
Agreed. I searched for "63" in automake distributions back to 1.11.3, as
well as the current sources, and no trace of any 63's in aclocal, only
in automake. Thus I suspect that the autoconf test meant to invoke
automake, not aclocal?
Does anyone remember what situation this might have been trying to detect?
"Version mismatch". Of exactly what kind, I don't know :).
The change was made in automake in 2003, according to old/ChangeLog.03.
BTW, I don't see a way that Channels.pm (which aclocal.in uses as of
this change, apparently) exits with code 63, either.
Here are the ChangeLog and NEWS entries. --hope this helps, karl.
2003-09-06 Alexandre Duret-Lutz
* lib/Automake/XFile.pm: Update to use ChannelDefs.
(close): Call Automake::FileUtils::handle_exec_errors on errors.
* automake.in (scan_autoconf_files): Exit with $?=63 on version
mismatch.
(scan_autoconf_traces): Likewise. Close the autoconf pipe to
capture abnormal exits.
* aclocal.in ($exit_status): Remove, and use Channels.pm's $exit_code
instead.
(trace_used_macros): Close the autom4te pipe to capture abnormal exits.
* lib/missing: When a tool has run and exited with $?=63, emulate
it. Adjust the diagnostic and pretend the tool is tool old in this
case. Use an emacs-updated "scriptversion" variable.
* configure.in: Output tests/aclocal-${APIVERSION} and
tests/automake-${APIVERSION}.
* tests/aclocal.in, tests/automake.in,
tests/missing.test, tests/missing2.test: New files.
* tests/defs.in (AUTOMAKE, ACLOCAL, PATH): Define
to use tests/aclocal-$APIVERSION and tests/automake-$APIVERSION.
* tests/dup3.test: Remove. alocal9.test, acloca10.test, and others are
already testing for this.
* tests/depacl2.test, tests/dup2.test, tests/order.test: Fix
configure.in so that aclocal works.
* tests/defun.test: Quote the AC_DEFUN body.
* tests/Makefile.am (TESTS): Add missing.test and missing2.test,
and remove dup3.test.
(check_SCRIPTS): Add aclocal and automake.
New in 1.8:
..
- Automake will exit with $? = 63 on version mismatch. (So does
Autoconf 2.58) missing knows this, and in this case it will
emulate the tools as if they were absent. Because older versions
of Automake and Autoconf did not use this exit code, this change
will only be useful in projects generated with future versions of
these tools.
automake-1.16i pretest available: please test
We are pleased to announce the GNU Automake 1.16i development snapshot. We intend for automake 1.17 to be released soon, essentially with only bug fixes for whatever is found in this pretest. So please do test if at all possible. It's our primary goal to preserve compatibility. If this release of Automake induces failure in a setup that was previously working, please let us know. See below for the detailed list of changes since the previous version, as summarized by the NEWS file. Download here: https://alpha.gnu.org/gnu/automake/automake-1.16i.tar.xz https://alpha.gnu.org/gnu/automake/automake-1.16i.tar.gz Please report bugs and problems to , and send general comments and feedback to , and patches to . Thanks to everyone who has reported problems, contributed patches, and helped test Automake! With special thanks to Bogdan, Mike Frysinger, Nick Bowler, and Zack Weinberg for their numerous contributions. This release could not have happened without them. New volunteers to help maintain Automake are needed. Please help if you can. For info, see this post (nothing much has changed since): https://lists.gnu.org/archive/html/automake/2021-03/msg00018.html and the open bug list: https://debbugs.gnu.org/cgi/pkgreport.cgi?package=automake New in (what will be) 1.17: * New features added - AM_PATH_PYTHON will, after checking "python", prefer any Python 3 version (latest versions checked first) over any Python 2 version. If a specific version of Python 2 is still needed, the $PYTHON variable should be set beforehand. - AM_PATH_PYTHON will also search for Python versions 3.15 through 3.10. It previously searched for 3.9 through 3.0. (bug#53530) - RANLIB may be overridden on a per-target basis. - AM_TEXI2FLAGS may be defined to pass extra flags to TEXI2DVI & TEXI2PDF. - New option "posix" to emit the special target .POSIX for make. - Systems with non-POSIX "rm -f" behavior are now supported, and the prior intent to drop support for them has been reversed. The ACCEPT_INFERIOR_RM_PROGRAM setting no longer exists. - Variables using escaped \# will trigger portability warnings, but be retained when appended. GNU Make & BSD Makes are known to support it. (bug#7610) - GNU Make's default pattern rules are disabled, for speed and debugging. (.SUFFIXES was already cleared.) (bug#64743) - For Texinfo documents, if a .texi.in file exists, but no .texi, the .texi.in will be read. Texinfo source files need not be present at all, and if present, need not contain @setfilename. Then the file name as given in the Makefile.am will be used. If @setfilename is present, it should be the basename of the Texinfo file, extended with .info. (bug#54063) - The missing script also supports autoreconf, autogen, and perl. * Bugs fixed - Generated file timestamp checks handle filesystems with sub-second timestamp granularity dynamically, greatly speeding up make check, etc. However, this requires an autom4te from Autoconf 2.72 (as yet unreleased) or later (or random test failures and other timing problems may ensue), as well as a Perl, sleep utility, and filesystem that supports sub-second resolution; otherwise, we fall back to one-second granularity as before. When everything necessary is supported, a new line `Features: subsecond-mtime' is printed by automake --version (and autom4mte --version). (bug#64756 and bug#67670) - The default value of $ARFLAGS is now "cr" instead of "cru", to better support deterministic builds. (bug#20082) - Dependency files are now empty, instead of '# dummy', for speed. - Compiling Python modules with Python 3.5+ uses multiple optimization levels. (bug#38043) - When compiling Emacs Lisp files, emacs is run with --no-site-file to disable user config files that might hang or access the terminal; and -Q is not used, since its support and behavior varies. (bug#58102) - Emacs Lisp compilations respects silent make output. - Automake no longer incorrectly warns that the POSIX make variables $(*D) and the like are non-POSIX. Unfortunately, the make implementations which do not correctly implement all the POSIX variables are not detected, but this seems to have little impact in practice. (bug#9587) - Pass libtool tags OBJC and OBJCXX for the respective languages. (bug#67539) - distcleancheck ignores "silly rename" files (.nfs* .smb* .__afs*) that can show up on network file systems. - tests: avoid some declaration conflicts for lex et al. on SunOS. (bug#34151 and others) - Pass any options given to AM_PROG_LEX on to AC_PROG_LEX. (bug#65600 and bug#65730) * Miscellaneous changes * Obsolescence: - py-compile no longer supports Python 0.x or 1.x versions. Python 2.0, released in 2000, is currently the minimum required versi
Re: [RFC PATCH]: autom4te: report subsecond timestamp support in --version
Features: subsecond-timestamps Sounds good to me, FWIW. Thanks to all. -k
Re: rhel8 test failure confirmation?
I thought we were talking about automake's testsuite probing the behavior of *autom4te* Yes, exactly. Sorry for the confusion. $ ./tests/autom4te --version autom4te (GNU Autoconf) 2.72d.6-49ab3-dirty (subsecond timestamps supported) Looks good to me. Thanks. a third party CPAN module Good point. The implementation is just Your patch sent to the autoconf list looked fine to me, FWIW. Perhaps worth mentioning somewhere that Automake is the consumer of this information that caused the change. You could point to https://bugs.gnu.org/64756 if you want something more convenient than the long multi-month thread on the automake discussion list (which I pointed to from that bug). Jacob, can you confirm that's an accurate test, +1 for Jacob's confirmation being welcome :).
Re: rhel8 test failure confirmation?
upthread somewhere Karl (iirc) threw out a bikeshed idea like --has=. Pretty sure it wasn't me :).
Re: rhel8 test failure confirmation?
> There would not need to be much parsing, just "automake --version | grep > HiRes" in that case, or "case `automake --version` in *HiRes*) ...;; > easc" to avoid running grep if you want. I specifically want to hear what Karl thinks. I lean towards Jacob's view that automake --version | grep HiRes will suffice. Not having a new option seems simpler/better in terms of later understanding, too. --thanks, karl. P.S. As for case vs. grep, personally I find a simple if...grep easier to comprehend/test/debug than a case statement. (Especially the macro-ized AS_CASE, which just makes me have to look up its syntax every time I see it.) Also fewer lines of source. Granted calling the external grep is less efficient, but that seems insignificant to me. I understand Paul and others may disagree ...
Re: rhel8 test failure confirmation?
The best way to check if high-resolution timestamps are available to autom4te is to have perl load Autom4te::FileUtils and check if that also loaded Time::HiRes. The problem with that turned out to be that Time::HiRes got loaded from other system modules, resulting in the test thinking that autom4te used it when that wasn't actually the case. That's what happened in practice with your patch. Hence the current test that merely greps the autom4te source for HiRes, which is not sufficient, granted. But we have to do something. Distributions unbundling Time::HiRes ... at some point I don't mind saying that people have to set am_cv_sleep_fractional_seconds=false by hand. karl
Re: armflang: error: unknown argument: '-soname'
While libtool has a new maintainer (Alex Ameen), essentially nothing happens, which is quite unfortunate... 1) libtool 2.4.7 was released in March 2022 (I don't know if Alex did it), which isn't so terribly long ago. Sure, maintainers should at least confirm bug reports and patches, even if releases don't happen quickly (pace my own efforts with automake), but ... 2) When a package appears to be unmaintained, the first thing to do is simply write the maintainer to see what the story is. If there is no response, the next step is to write maintain...@gnu.org. It's their responsibility to make continued efforts to contact the maintainer, up to and including sending paper mail. And look for a new maintainer if need be. 3) If that also fails, the final fallback is to write gnu-advis...@gnu.org and/or r...@gnu.org. Best, Karl
Re: armflang: error: unknown argument: '-soname'
Hi Anton - thanks for the report. https://github.com/HDFGroup/hdf5/issues/366 with the solution that "-Wl," must be prepended somehow to "-soname". Why do you think this is not a libtool issue? Isn't it libtool's job to figure out the arguments that need to be passed? The fact that the -Wl is provided for gfortran and not for armflang makes me think even more that it is libtool. At first glance, the string "soname" does not appear in the Automake sources. But I'm not sure what changes I need to do to my configure.ac and/or Makefile.am to fix this correctly. I wish I had a workaround for you, but I just don't know the answer. Someone else here probably knows better, else I suggest posting to bug-libt...@gnu.org, ideally with a reproducible case. Happy linking, Karl
Re: [PATCH v2] automake: rewrite scan_variable_expansions regex
The regex pattern in function scan_variable_expansions() fails to report a portability warning when a dollar-escaped dollar sign precedes the variable: foo_SOURCES = a.c $$$(patsubst a.c,a,b) Thanks much, Jan. It all worked for me. Pushed. --karl
Re: [PATCH] automake: rewrite scan_variable_expansions regex
The regex pattern in function scan_variable_expansions() fails to report a portability warning when a dollar-escaped dollar sign precedes the variable: foo_SOURCES = a.c $$$(patsubst a.c,a,b) Thanks Jan. Any chance you could add a check to one of the tests, I guess dollarvar.sh or dollarvar2.sh? (I haven't looked at them closely.) -k
Re: Issue with AM_PROG_LEX
This will work iff AM_PROG_LEX uses AC_REQUIRE to invoke
AC_PROG_LEX.
Evidently so. From automake's m4/lex.m4:
AC_DEFUN([AM_PROG_LEX],
[AC_PREREQ([2.50])dnl
AC_REQUIRE([AM_MISSING_HAS_RUN])dnl
AC_REQUIRE([AC_PROG_LEX])dnl
if test "$LEX" = :; then
LEX=${am_missing_run}flex
fi])
Re: Getting long SOURCES lines with subdirs shorter
Hi Jan, Current automake likely won't have anything in store already, Not that I know of. a_SOURCES = $(addprefix aprog/,main.c foo.c bar.c baz.c) I've often wanted this myself. I'd certainly welcome a patch for it. Please work from automake trunk. None of the various branches are kept to date. (Sad but that's the reality.) Happy hacking, Karl
Re: _AM_FILESYSTEM_TIMESTAMP_RESOLUTION incorrect result (was Re: make check(s) pre-release problems)
Hi Zack, Date: Fri, 07 Oct 2022 11:35:41 -0400 From: Zack Weinberg [...] the filesystem timestamp resolution was incorrectly detected: Your analysis sounds plausible to me, but it's not obvious to me how best to fix it. ls --full-time or stat may not be available. Maybe just do the test again if the first ls -t "succeeds", i.e., if the first 0.1sec straddles a second boundary, the second presumably won't? Or is there a better way? I wonder if you could easily make a patch for it (in m4/sanity.m4), if you already have a setup on a second-granularity fs? (I don't.) --thanks, karl. [...] The filesystem I'm working in only records timestamps at second granularity. (I don't know why ls is printing .1 instead of .0 but it's always .1.) $ touch A && sleep 0.1 && touch B && ls --full-time -t A B -rw-r--r-- 1 zweinber users 0 2022-10-07 11:20:05.1 -0400 A -rw-r--r-- 1 zweinber users 0 2022-10-07 11:20:05.1 -0400 B I *think* this is a bug in _AM_FILESYSTEM_TIMESTAMP_RESOLUTION where, if it starts looping at *just the wrong time*, the first 0.1-second sleep will straddle a second boundary and we'll break out of the loop immediately and think we have 0.1-second timestamp resolution. zw
Re: faster tests [was: rhel8 test failure confirmation?]
Hi Bogdan, Then, I analysed the files and added the trick from t/backcompat2.sh (if possible) and/or removed the extra calls to $ACLOCAL (if possible). Thanks much for looking into this. Short version: after a few hours of testing and modifications, I *may* have saved up to 1 minute and 12 seconds of testing... Well, at least you get kudos for doing all the research :). You may look at the attached patch as a result of the investigation and then ... you're free to completely ignore it :). It works for me, but I wonder if it won't cause more confusion than it's worth... I agree. Not worth the complications. t/backcompat-acout.sh: 35 -> 24s That seems to me like the only one that might be worth applying the patch for. Quite a bit more savings than anything else in the list. >> # A trick to make the test run muuuch faster, by avoiding repeated >> # runs of aclocal (one order of magnitude improvement in speed!). >> echo 'AC_INIT(x,0) AM_INIT_AUTOMAKE' > configure.ac Alternatively, I wonder how much this is really saving. Maybe the trick should not be used anywhere. - having 1277 .sh files in 't/' means that even if each runs in 30 seconds, you have 10 hours of testing just from the number of tests, Indeed. The only practical way to run make check is in parallel. I discovered that early on :). It still takes painfully long for me (10-15min at best, on a new and fast machine). - it may be better to determine if there are duplicate tests Sounds awfully hard to do. My impression is that the (vast?) majority of tests are the direct result of bug reports. I would not be inclined to tweak, remove, merge, or change them. Even if two tests are nearly identical, that "nearly" can make all the difference. - as you see above, t/pr401b.sh takes 1m42s to run. I wonder if e.g. running the 'distcheck' target in tests would be the main factor Sounds very likely to me. Distcheck is inherently a lengthy process. I can't imagine how it can be sped up. Although I agree that 1:42 seems rather long for a trivial package like those in the tests. Same case for t/pr401c.sh and t/pr401.sh, although shorter times. At a glance, I see required='cc libtoolize' in 401b, whereas 401c and 401 only have cc. Testing libtool really is different, and really does take time. So I'm not sure there's any low-hanging fruit here. Thanks again for doing all this work, Karl
Re: rhel8 test failure confirmation?
Are you sure about that? Yes. Well, as sure as I can be. I don't see any $(...) constructs in Automake's *.m4 now, and this is certainly deliberate. We discussed shell portability many times over the decades of Automake development. I can't quickly find where it's documented, if anywhere, but as far as I know it is the status quo and I certainly see no reason to break new ground for this small patch. I got a fair bit of pushback on removing $(...) from config.guess Yes, I know. I was one of the people who reported problems because of that. on the grounds that Autoconf locates a POSIX shell and uses it for the bulk of configure My understanding is that Zack implemented that precisely to defend against the config.guess snafu. So for all but the latest Autoconf versions, it will not happen. Let's try: Thanks much! --all the best, karl.
Re: rhel8 test failure confirmation?
Hi Jacob,
The guess was the two most probable locations: /usr/share/autoconf and
/usr/local/share/autoconf.
Wouldn't have worked on my own system :).
Challenge accepted.
Thanks!
if $PERL -I${autom4te_perllibdir:-$(sed -n \
'/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}' \
<$(command -v autom4te))} -MAutom4te::FileUtils \
-e 'exit defined $INC{q[Time/HiRes.pm]} ? 0 : 1'; then
# autom4te uses Time::HiRes
unfortunately we are highly restricted in what we can use in basic
automake/conf shell code (as opposed to in the tests). Neither the
"command" command nor $(...) syntax can be used.
For the former, I think there's an autoconf/make macro to look up a
program name along PATH? For the latter, good old `...` suffices. (Not
sure there can be newlines in `...` though, even backslashed.)
Would you be up for tweaking the check to use such
least-common-denominator shell stuff?
Ordinarily Perl could not be used either, but since Automake is written
in Perl, I don't see a problem with doing so here. (If the system
doesn't have Perl, Automake won't get far.) Not sure if $PERL is already
defined by the time at which this would be run, but it should be
possible to arrange with an ac prerequisite if needed.
Thanks,
Karl
Re: rhel8 test failure confirmation?
jb> The test also guesses the location of autoconf's Perl libraries;
I'm skeptical that any "guessing" of library locations would be reliable
enough.
jb> a more thorough test would locate the autom4te script and grep it
for the perllibdir that was substituted when autoconf was
configured.
I guess that would work. A simpler option would be to run
autom4te --version
and if it's "ge" 2.72, the patch is applied.
Unless there is another pretest with the patch, granted, in which case
there would be need to be some finagling. But dealing with pretest
versions will become irrelevant as soon as the release is made.
Although ordinarily we would not want to test version numbers instead of
features, in this particular case I don't see a problem with it. If, for
instance, RH patches an autoconf with a lesser version and so the check
gives the "wrong" answer, the only result is that the sleep will be
1 sec, as long as it's been in every released automake ever. Hence, not
our problem. RH can patch the test too if they want to.
bd> [version number comparison] Best, not easy :).
Comparing version numbers in complete generality is hard. In the
particular case of autom4te, it doesn't seem problematic to me.
Compare '2.71' to '2.72-beta1-snapshot-12345678' or
vice versa with the suffixes :).
I think ge gives the right answer for our particular question ("is
autom4te patched") until autom4te reaches version 10. I don't think we
need to protect against that, but if you feel the need to do so, not too
hard.
it could be possible to make a loop similar to the one
I was not imagining running the same sort of loop to somehow try to
infer if autom4te is patched. That doesn't sound right. Determining if
autom4te is patched should be as definitive as possible, not just a
heuristic. And I don't think it's any harder to accomplish that.
I agree, so it would be best to have a workaround or a plan "B" for
those unpatched, for some reasonable time, at least. What would the
plan be is the question.
The workaround I think is (by far) the best is: set the sleep to 1 if
autom4te is not patched. That's it. Thus:
if () then
;
else
sleep = 1 second; # as it always was before
fi
Only a matter of where to put it...
I was imagining at the beginning of _AM_FILESYSTEM_TIMESTAMP_RESOLUTION
in m4/sanity.m4. I.e., when computing the sleep to use.
an issue with SOME OTHER TOOL.
Automake and autoconf are not two independent tools. Automake completely
relies on autoconf.
It's not for me to hand down any final pronouncements, but personally I
feel strongly that the tests should not paper over this problem by
changing the way tests work in general. With rm -rf of the cache, or
autoconf -f, etc. That is not what users do, so that's not what the
tests should do, either. Such global changes could have all kinds of
other unknown/undesirable effects on the tests.
In contrast to setting the sleep value "as appropriate", which is what
is/should be already done, so changing the conditions under which it is
set is unlikely to cause any unforeseen additional problems.
My $0.02 FWIW. Unfortunately I still have other projects I must attend
to, so can't dig into the actual code any time soon. But you guys are
doing that perfectly well anyway :). --thanks, karl.
Re: faster tests [was: rhel8 test failure confirmation?]
Is there a way to time individual tests I don't know. Maybe one of the more experienced automakers here can advise. Makes me wonder about having the test infrastructure start each test by running date to get the start time into the log file. Does anyone see a problem with doing that? The mtime of the file should be good enough for the end time, I think? (Filesystem inaccuracies shouldn't matter for that purpose. :) -k
faster tests [was: rhel8 test failure confirmation?]
# A trick to make the test run muuuch faster, by avoiding repeated # runs of aclocal (one order of magnitude improvement in speed!). echo 'AC_INIT(x,0) AM_INIT_AUTOMAKE' > configure.ac Hadn't noticed this before. Maybe you could see what tests are currently taking the longest to run, and see if the above helps speed them up? It seems a somewhat weird thing to do, but if it saves enough time, I guess it would be worth it. --thanks, karl.
Re: rhel8 test failure confirmation?
I modified my autom4te using the attached patch, Thank you very very much for finding this, Bogdan. I'm glad that Paul has already installed it in Autoconf. I can't easily confirm it for myself right now, but I'm hopeful. (Maybe Frederic or others can try with the development autoconf.) What can we do about this? As for automake: can we (you :) somehow modify the computation of the sleep value to determine if autom4te can handle the HiRes testing or not (i.e., has the patch installed)? And then use the longer sleep in automake testing if needed. In fact, I wonder about autom4te providing an explicit way to test whether it's been patched, because I suspect that a functionality test (run autom4te a bunch of times on a bunch of newly-created files) would be highly susceptible to wrong results. Checking the autom4te --version number might be the easiest and best thing to do. I think it's pretty crucial that the automake (or autoconf) tests not spuriously fail due to filesystem timing, even for people that have the "old" (unpatched) autom4te. People will certainly keep using current and older releases of autoconf for years to come. It seems to me that using autoconf -f or similar is papering over the problem, so that the tests are no longer testing the normal behavior. Which does not sound desirable. Wdyt? --thanks again, karl.
Re: rhel8 test failure confirmation?
Note that 'config.h' is older (4 seconds) than './configure', which shouldn't be the case as it should get updated with new values. Indeed. That is the same sort of thing as I was observing with nodef. But what (at any level) could be causing that to happen? Files just aren't getting updated as they should be. I haven't yet tried older releases of automake to see if their tests succeed on the systems that are failing now. That's next on my list. Thanks for looking into it, Karl
Re: rhel8 test failure confirmation?
Hi Frédéric, While building the trunk directly led to check failures, Confirmation is good. rebuilding the RPM in a mock environment didn't. Puzzling. I'll likely spend more time next week to perform more testing. It may simply be an environment problem: I guess it's possible, though I've tried make check in quite a few different environments by now (latest tools, distro tools, minimal env, maximal env, different kinds of filesystems, etc.), and the make check failures have persisted in all cases. Also, I'd expect an environment issue to cause a test (say, nodef.sh) to consistently succeed or consistently fail. But what I observe is a "probabilistic" failure. If I run nodef on its own 10 times, it might fail 7 times and succeed 3 times, say. Or 6/4 or 8/2 or 5/5 or whatever. That smells like a caching bug somewhere to me. But ... another friend tried make check on a freebsd system and *also* got failures (more or less the same set of failing tests, including nodef.sh). So evidently it's unlikely to be a low-level kernel/disk subsystem timing or caching bug. I'll likely spend more time next week to perform more testing. For sure, any digging will be greatly appreciated! --thanks, karl.
Re: rhel8 test failure confirmation?
FAIL: t/remake-aclocal-version-mismatch.sh Thanks for trying it, Nick. I'm glad it's not just me. And I sure wonder wtf is going on :(. -k
rhel8 test failure confirmation?
Does anyone have access to an RHEL 8-based machine? Alma Linux, Rocky Linux, original RHEL, or even (sort of) CentOS 8? It would be nice if someone could run a make check there (from automake dev). git clone -q git://git.savannah.gnu.org/automake.git cd automake ./bootstrap ./configure && make >&cm.out make -j8 VERBOSE=1 check keep_testdirs=yes >&ch8.out (choose whatever -j value you like) Even in a pristine environment, on a different physical machines, I find that a few tests fail, usually taken from the list below. Not always the same tests fail, but I don't think it's directly related to parallelism. In the one I've looked into most, t/nodef.sh, it seems to be about disk writes not taking effect. That test runs the usual autotool sequence twice, both times writing to the same file named `output' and then grepping it, e.g.: grep 'DEFS.*-DVERSION=\\"UnIqUe' output && exit 1 If I put a sleep 1 between the two autotool invocations, it passes. If I don't, it fails, because the second grep reads the `output' file written by the first. Or so it seems. Which is crazy. I could chalk it up to weird (extremely broken) hard disk behavior on one machine, but not two different machines with completely dissimilar hardware (including disks). It seems it must be in the os, in my case Alma Linux: $ grep ^VERSION= /etc/os-release VERSION="8.7 (Stone Smilodon)" (But I'd be very surprised if it is Alma-specific.) All the tests pass fine on CentOS 7. Sigh. --thanks, karl. Approximate lists of tests that "usually" fail: FAIL: t/backcompat2.sh FAIL: t/backcompat3.sh FAIL: t/get-sysconf.sh FAIL: t/lex-depend.sh FAIL: t/nodef.sh FAIL: t/remake-aclocal-version-mismatch.sh FAIL: t/subdir-add2-pr46.sh FAIL: t/testsuite-summary-reference-log.sh
Re: On generating compilation databases
Hi Arsen - sorry for the delayed reply. Replying as one of the automake developers, though I know nothing ... src/tangle-main.o \ `test -f 'src/main.cpp' || echo '$(srcdir)/'`src/main.cpp ... which relies on a shell construct, making it hard to 'deduce' from the point of a pure make processor (note that I didn't actually go too The shell construct is something about the package you're compiling, isn't it? With the minimal amhello, the command I see with make V=1 is (modulo line breaks): gcc -DHAVE_CONFIG_H -I. -I.. -g -O2 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c without any ||. However, regardless, for the reasons you describe in your message (stamp files and the like), merely getting the compilation commands is not enough. A full package build is not just a sequence of compiles, in general. So, as for the general question you raise: I don't know. I have no background on pretty much anything you mention in your message (LSP, compile_commands.json, ...). Thus, I have nothing against the idea, but you or someone else who wants to make it happen will need to implement it. If Automake needs to do something differently, and patches (that don't affect its normal operation) can be provided, then that's fine. Happy hacking, Karl
Re: Automatic dependency tracking when using non-depcomp tools
Can I actually rely on Automake making a verbatim
copy of the include line from Makefile.am to Makefile.im?
As far as I can make out, your logic is exactly correct. Looking at
the bin/automake script, I see:
my $PATH_PATTERN = '(\w|[+/.-])+';
# This will pass through anything not of the prescribed form.
my $INCLUDE_PATTERN = ('^include\s+'
. '((\$\(top_srcdir\)/' . $PATH_PATTERN . ')'
. '|(\$\(srcdir\)/' . $PATH_PATTERN . ')'
. '|([^/\$]' . $PATH_PATTERN . '))\s*(#.*)?' . "\$");
The first two alternatives are the ones mentioned in
https://www.gnu.org/software/automake/manual/automake.html#Include
The third matches any "plain" filename ("foobar", but not "foo/bar"), I
think? Then, in the code that handles this (approx line 6700ff.),
$(srcdir) is prepended. This is apparently a feature not documented in
the manual -- a plain "include foo" turns into "include
$(srcdir)/foo". I think. Does that seem right to you? Clearly a
desirable feature, but missed in the manual, it seems.
Anyway, for your purposes, as soon you include a / or other
non-$PATH_PATTERN character in the include argument, Automake leaves it
alone and copies it. I can't conceive of changing that now, so yes, I
think it is reasonably safe for you to rely on it.
and the Makefile.am rules therefore now look as follows, for the benefit
Thank you very much for posting all this. --happy hacking, karl.
Re: TAR variable no effect when tar-pax is used
Hi Jan,
But when you set `tar_pax` in AM_INIT_AUTOMAKE, this comes out:
I surmise that's left over from when the pax support was added.
I don't see any reason not to use $(TAR) there now.
It presumably shouldn't be too hard to fix. If you can devise a patch
(and ideally test case), that would be great. Else, it's on my list
.. someday :( ...
Also, ${AMTAR} seems totally unused. Why is it even there?
As far as I can see, it is a relic of when AMTAR was used.
It seems it was added and then later removed in 1999:
1999-06-01 Alexandre Oliva
...
* m4/init.m4: Remove AMTAR-related stuff.
...
As you say, I see no use of it anywhere in the current source tree,
apart from its being defined in m4/tar.m4. As far as I can see, can be
removed too. --thanks, karl.
Re: Automatic dependency tracking when using non-depcomp tools
Hi Hans - first, congratulations on getting this to work at all. I don't recall anyone else undertaking this. I'm afraid I don't have any particular insights, but just so you know someone read your mail: Question: Is it OK for me to hook into ./$(DEPDIR)/ at all? I could use just some file in $(builddir), after all. I don't see a specific problem with using $(DEPDIR), yet if you can use your own directory in builddir, that seems safer. Just in principle. Question: Is *.cdep a good name? I want to avoid name collisions. I don't know of anything else that uses .cdep (not that I would, necessarily), and have nothing against it myself. OTOH, using xml seems more unusual than targeting C, so maybe .xmldep or even .xmlcdep, unless the long extension is irritating in some way. Question: Is it ok for me to use $(am__mv)? It seems no am__ variables are documented. So, although in practice I expect it would be fine, yet since it is merely "mv -f", in your place I would be tempted to define my own variable for it. I have one other comment, about the dummy file that is created: provide an explicit recipe to create a dummy dep file to In a relatively recent change (not in 1.16.5), Automake's dep setup was changed to create an empty file instead of one with the line "# dummy". In large enough projects, this turned out to make a noticeable speed difference (having to create a regular file and not just an inode). I'll append the ChangeLog entry. Happy hacking, Karl 2022-05-23 Jan Engelhardt deps: create empty file instead of dummy file. This change is per an automake thread, see both before and after: https://lists.gnu.org/archive/html/automake/2022-05/msg6.html * lib/am/depend.am ($(am__depfiles_remade)): create empty files for dependencies instead of files with a line '# dummy'. Turns out this is noticeably faster. * THANKS: update Jan's email address. * NEWS: mention this.
Re: 3-rd party aux files.
automake -af --libdir=$HOME/aux it breaks automake (it can't find other files, e.g. am/header-vars.am). So, my question: Is there any way to provide 3-rd party aux files? It seems not :(. If you, or anyone, can draft a patch, that would be great. As can be seen by how long it took me to send this non-reply (sorry), my automake time is woefully scarce at the moment :(. Meanwhile, I have resent your message to bug-automake so it will become more trackable. --thanks for writing, karl.
Re: make check(s) pre-release problems
I actually wonder if your sudden "parallelism" failure could be somehow linked to an update of bash, similar to mine ? Good idea, but my bash hasn't changed ... I don't doubt there would be plenty more failures with the new SHLVL change (any such change seems like a terrible idea, but oh well). Well, more fun in the future. -k
Re: make check(s) pre-release problems
No errors on RHEL7+autoconf2.71 Puzzling. Can you easily try RHEL8 or one of its derivatives? It surprises me that that is the culprit, but it seems possible. I'm using autoconf-2.71, make-4.3, etc., compiled from source, but am using the OS-provided coreutils. I think I'll try compiling that from source. Thanks to everyone for the suggestions and help. -k
Re: make check(s) pre-release problems
What version of GNU make are you using? I've been using make 4.3 since its release in 2020. No changes, no prereleases. I'm afraid the problem, whatever it is, is not that simple :(. What troubles me most is that there's no obvious way to debug any test failure involving parallelism, since they go away with serial execution. Any ideas about how to determine what is going wrong in the parallel make? Any way to make parallel failures more reproducible? Right now, all I know is "some random test(s) fail(s)". Not helpful. The test logs show the set -x execution of the test, so the actual command that fails can be seen. I have keep_testdirs=yes, of course, but then running the command by hand in the shell after the failure often does not reproduce the problem. Something else is going on, but my imagination about what that might be has failed so far :(. Argh. --thanks, karl.
make check(s) pre-release problems
With Zack's latest Python fixes, I was hoping to move towards an Automake release, but I find myself stymied by apparently random and unreproducible test failures. I haven't exhausted every conceivable avenue yet, but I thought I would write in hopes that others (Zack, past Automake developers, anyone else ...) could give it a try, and/or have some insights. For me, running a parallel make check (with or without parallelizing the "internal" makes), or make distcheck, fails some tests, e.g., nodef, nodef2, testsuite-summary-reference-log. The exact tests that fail changes from run to run. Running the tests on their own succeeds. Ok, so it's something in the parallelism. But why? And how to debug? Nothing has changed in the tests. Nothing has changed in the automake infrastructure. Everything worked for me a few weeks ago. Furthermore, Jim ran make check with much more parallelism than my machine can muster, and everything succeeded for him. That was with: make check TESTSUITEFLAGS=-j20 So what the heck? Perhaps easier to debug: there are two targets to be run before making a release, check-no-trailing-backslash-in-recipes and check-cc-no-c-o, to try to ensure no reversion wrt these features. A special shell and compiler are configured, respectively (shell scripts that check the behavior). These always worked before. But now, Jim gets hundreds of failures with the first (didn't have time to try the second). I get a couple, with both, instead of hundreds. Again the failing tests vary. In this case, they fail for me even without parallelism. So what the heck x 2? Any ideas, directions, fixes, greatly appreciated. --thanks, karl.
Re: python debugging on trunk
The appended patch should address both issues. Thanks Zack!! I installed the changes (separately). Note that I have only tested it with Python 2.7 and 3.6. It _may_ not be correct for Python in the 3.0--3.3 (inclusive) range I didn't test any of those versions either. Certainly your change is a vast improvement in any case. The test suite now passes on my Alma Linux 8 system (I had one unrelated failure which I could now fix). For the record, this has Python 2.7.18 and 3.9.7. Thanks again! -k
python debugging on trunk
Is anyone up for debugging some Python-related test failures on RHEL-based systems? Mike Vapier from gentoo made many improvements to the Python support. (Mike, if you're still out there, would love to hear back.) Unfortunately, the end result is that 13 tests (listed below) now fail for me on Alma Linux 8 (and, presumably, Rocky 8; and I believe CentOS 7) with the trunk automake. My appetite for (and experience in) debugging Python problems is pretty low. But we can't make another release until this is fixed. So, I wonder if there's anyone else who's interested in figuring this out. I can probably figure out access to a system where it fails, if that's a problem. I have a suspicion the problem is that on RHEL systems, "python" invokes "python2" (because that's what the Python maintainers recommend, as I understand it), but the tests are assuming it invokes "python3". Knowing Python, I feel pretty sure that there will be many complications in dealing with this. Thanks, Karl FAIL: t/instmany-python.sh FAIL: t/nobase-python.sh FAIL: t/py-compile-basic.sh FAIL: t/py-compile-basedir.sh FAIL: t/py-compile-destdir.sh FAIL: t/py-compile-option-terminate.sh FAIL: t/python3.sh FAIL: t/python12.sh FAIL: t/python-prefix.sh FAIL: t/python-pr10995.sh FAIL: t/python-vars.sh FAIL: t/subobj-pr13928-more-langs.sh For example, test-suite.log shows this for the nobase-python.sh failure: .. Byte-compiling python modules... one.py Traceback (most recent call last): File "", line 13, in AttributeError: 'module' object has no attribute 'implementation' make: *** [Makefile:304: install-myPYTHON] Error 1
Re: man_MANS install locations
Hi Jan, As for GNU/Linux, what was the rationale to only permit [0-9ln]? No idea. Maybe just didn't think about "m", or maybe it didn't exist at that time? Jim, Paul, anyone? Should automake be relaxed? I see no harm in allowing more (any) letters, if that's what you mean. When running automake on Solaris, placing svcadm.1m into man1 rather than man1m seems outright wrong. But is Automake's purpose to reproduce platform-specific behavior, or to have consistent behavior across platforms? I think the latter. I guess a new option to install *.1m in man1m/, etc., would be ok, if you want it. If you or anyone can provide a patch, that would be great. Unfortunately I doubt it's anything I will ever implement myself. Should the rpmlint check be adjusted to cater to the GNU FHS? I guess that's a question for the rpmlint people, whoever they are. I don't see that Automake's default behavior is going to change. Also, GNU (as an organization) never had anything to do with the FHS, so far as I know. I don't think the GNU coding standards/maintainer information have anything to say about this topic ... --thanks, karl.
Re: How to speed up 'automake'
I was going to bisect but if it doesn't fail for me in the first place... :( Thanks. Indeed, reconfiguring etc. got rid of those errors. Now a bunch (12) of the Python tests are failing for me, presumably because of previous Python changes not playing nicely with my older Python version (2.7.5 on CentOS 7). Sigh. Not looking forward to figuring that out. Anyway, that's surely independent of empty dependency files, so I pushed Jan's change. Thanks! --karl
Re: How to speed up 'automake'
Unrelated to Jan's depend.am change, but it turns out that a previous change broke make distcheck (768 failures). I don't feel right about committing anything else until that is fixed. Error below. Jim (or anyone), can you easily advise? I haven't delved into this part of things before. (Not sure how many of the failures are due to these errors, but I suspect most.) --thanks,karl. FAIL: contrib/t/multilib .. configure.ac:4: warning: _AM_PROG_RM_F is m4_require'd but not m4_defun'd aclocal.m4:139: AM_INIT_AUTOMAKE is expanded from... configure.ac:4: the top level configure.ac:4: warning: _AM_PROG_XARGS_N is m4_require'd but not m4_defun'd aclocal.m4:139: AM_INIT_AUTOMAKE is expanded from... configure.ac:4: the top level configure:2411: error: possibly undefined macro: _AM_PROG_RM_F If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. configure:2412: error: possibly undefined macro: _AM_PROG_XARGS_N
Re: How to speed up 'automake'
Fancy doing it? Jim agreed :) Yeah, sorry. Other priority things have intervened :(. I have a mild hope of getting to it by tomorrow. If someone else wants to make the commit, certainly fine by me :). -k
Re: How to speed up 'automake'
I see no reason why mv would be so crucial. Hmm, I guess you're right. Thanks for the analysis. The purpose of the stamp files is to avoid partial files being written (and screwing up future makes), but if the file is zero bytes, it seems that problem cannot happen. Jim, do you agree? I'll install the change if yes. --thanks, karl.
Re: How to speed up 'automake'
- @echo '# dummy' >$@-t && $(am__mv) $@-t $@ + @: >>$@ 1) does it actually speed anything up? 2) without the mv I fear we are no longer noticing write failure. -k
Re: How to speed up 'automake'
automake --verbose --warnings=all --add-missing --copy Since you're talking about cron, I'm guessing the terminal output is being redirected to a file? Because if it's going to an actual tty, that can consume a surprising amount of time. Is there a way to speed 'automake' up? Only real idea I have is to run automake under the Perl profiler and see what comes up. As in, install Devel::NYTProf from CPAN and then run perl -d:NYTProf /path/to/automake/executable Best, Karl
Re: Wrong order of preprocessor and compiler flags
It seems the basic inconsistency is whether CPPFLAGS is considered a "user variable" or not. In earlier eras, it wasn't, but from your msg, I gather it is now. The GNU standards node about it, that you mentioned, https://www.gnu.org/prep/standards/standards.html#Command-Variables does not clearly state it one way or another. But its example shows CFLAGS after CPPFLAGS. Thus I think a prior step is to write bug-standa...@gnu.org and suggest clarifying the status of CPPFLAGS, its relationship to CFLAGS, etc. We could consider changing automake to follow autoconf even if the GCS is not changed, but it would be better if the GCS were clear. 2. Use AM_CFLAGS CFLAGS AM_CPPFLAGS CPPFLAGS. This is more aligned with current flags grouping, but CFLAGS will not override definitions in AM_CPPFLAGS (less aligned with GNU Standards). It seems wrong (and disastrously backwards-incompatible) to me for CFLAGS not to override AM_everything. Your option 1: 1. Use AM_CFLAGS AM_CPPFLAGS CFLAGS CPPFLAGS. I think this is the best option. As required by GNU Standards, CFLAGS still override all upstream-defined flags. seems like the best option to me too. --thanks, karl.
Re: multiple online manual versions
> https://www.gnu.org/software/automake/manual/index-full.html It's better, but how about a non-blank line? (does obey table cells? I'm never sure.) Obviously we're getting down to trivialities, feel free to ignore :). so i guess once gnulib merges my update, If nothing happens soon, let me know and I can commit that update. (I'd do it now but some kind of network problem, apparently at the FSF end ...) --thanks, karl.
Re: multiple online manual versions
Hi Mike, https://www.gnu.org/software/automake/manual/index-full.html It looks nice, but the plethora of versions becomes rather an undifferentiated mass. Maybe make each major release its own , as in: Automake 1.16 releases: 1.16* versions Automake 1.15 releases: 1.15* versions Just to break it up a little. Also, some simple intro text seems desirable (edit as you see fit): Below are links to the manual for all released versions of GNU Automake. The current manual is also available separately. And maybe an outro too: Above are links to the manual for all released versions of GNU Automake. Wdyt? i actually regenerated them from scratch rather than extract them Sounds good. The copyright at the bottom says 2020. I don't know if that means gendocs.sh needs updating, or the template, or something on the web site, but something somewhere should be changed to 2022. Can you check? --thanks, karl.
Re: multiple online manual versions
i was planning on the full index being maintained here: https://www.gnu.org/software/automake/manual/index-full.html Sounds good. >>See the [full version index] for other versions of the manual. Good. Maybe something like: >>See the [full version index] for the manual for older releases of Automake. (Since it's not just "versions of the manual", but "versions of the software". :) Not sure. Will be easier to think about seeing the pages actually differ and setup symlinks in case the manual didn't change between point releases. I doubt there are significant changes to the manual for most point releases, but still, I think it would be confusing to ask for the manual for 1.16.2 and get the manual for 1.16.1 instead. I think it'd be best to just do the full thing for any versions published. --thanks for all, karl.
Re: multiple online manual versions
per your request, the default is unchanged. I understand (and thanks), but my question was about the "table" (whether it's actually a or not): * (Feb 2018) GNU Automake 1.16 (HTML PDF) * (Dec 2014) GNU Automake 1.15 (HTML PDF) * (Jun 2013) GNU Automake 1.14 (HTML PDF) On what page were you going to put this table of all available manual versions? BTW I still think it's more useful to have the manual for the latest 1.xx.y release of each 1.xx than the original. Why propagate known-old information in preference to newer? tbh the only reason i'd prefer a table is for the column alignment. Sure. Fine by me. However it works out best. Thanks. -k
Re: multiple online manual versions
* i'm assuming that we don't want to modify lib/gendocs_template since it's synced with upstream gnulib For sure. so the default manual/ landing page & manual will be unchanged from today other than having a link to the full versioned index What url/filename are you thinking for the "full versioned index"? GNU Automake 1.16 ... vs. ... Feb 2018 GNU Automake 1.16 HTML PDF I doubt the old manuals will get a tremendous amount of use, so one line per version seems sufficient to me, vs. a table. But I have no objection either way. OTOH, the date and the direct link to the html seem like the most interesting information (more so than the PDF). Could be something like: GNU Automake 1.16 - Feb 2018 - HTML But I don't feel strongly about it. Whatever seems sensible. It might take seeing the page with a few versions actually done before it's clear what's best. --thanks, karl.
Re: community long term x.y release branches
i would like to help coordinate these downstream distros so they don't have to keep all repeating the same work. basically: Sounds sensible to me. * commits are on a volunteer/request basis -- there is no expectation that people working on master/whatever think about these older branches I like that in particular :). * maybe we cut a new release once a year from those branches if any changes were actually made to them. i would like this, but even if we do all the other things above and omit this, i'd still be happy. Sounds reasonable, not that I'd be the one making such releases. I'm guessing Jim would want to push making such "back releases" to you ... so since the last release is still 1.16.z, the newest such branch would be refs/heads/release/1.15 and it would start life at v1.15.1. Fine by me. Jim, can you chime in? As you know, I have little clue about git development conventions, so can't make any sensible comments at that level. BTW, the next release will surely be 1.17. Some of the stuff I already did in the 1.16.x releases really should have provoked 1.17, but I just didn't adhere strictly to the rules (because I didn't know about them). --thanks, karl.
Re: multiple online manual versions
* the current page, but with an entry/link like "For older manuals, please see this index." Agreed this is preferable. Not a fan of the gcc index page. changes to the manual the rename or reorder chapters, we're breaking those historical links. Reordering isn't a problem; that doesn't break links. Renaming nodes is what breaks links. And the answer is, 1) don't do it, and 2) if you feel you simply must, leave an @anchor behind, and then existing links will not be broken. Anyway, the manual hardly changes nowadays, so it's not an issue in practice. To me, the redirection to a versioned url a la autoconf often leads to the wrong behavior in the other direction: typically people want to link or refer to the current version of node Foobar (whatever version it might be), but it's too much trouble to do when you end up getting redirected. So it doesn't happen, and then X years now people still think autoconf-2.68, for example, is current because that's what the link say. (Rather like bugs.gnu.org/ being preferred, but since that redirects, hardly anyone uses it and it's just manual labor to do so. But I digress.) any other manual/ access would redirect to the latest version. that way we don't break links already out in the wild. Agreed we certainly must not break existing links. But I'd rather have a copy as the unversioned/canonical page than forced redirection in all cases. Both canonical and versioned urls are useful. https://www.gnu.org/software/automake/manual/1.15/ https://www.gnu.org/software/automake/manual/1.16/ (or if you prefer, 1.15.x and such) If not publishing all versions, it would seem better to me to publish the latest version of the manual for any given 1.x, not the first version. I.e., 1.13.4, 1.14.1, 1.15.1, 1.16.5. I don't see that there's anything especially magical about the "non-dot" releases like 1.15, although I have no objection to publishing those too. --thanks, karl.
Re: adding a command line option for ACLOCAL_PATH-type search paths
Hi Mike, the ACLOCAL_PATH functionality is useful (adding search dirs after -I), but a bit unwieldy as an env var. any reason we can't add a command line option for this ? Seems like a fine idea to me. call it --aclocal-path ? or --extra-system-acdir ? Reading the doc, my hunch is to call it --aclocal-path and make the semantics of its argument be exactly the same as $ACLOCAL_PATH. Wdyt? But if both are set, should the --aclocal-path argument replace $ACLOCAL_PATH, or augment it (earlier of course)? I'm not sure. By the way, would it be feasible to have --help show the actual values for --automake-acdir and --system-acdir? It is far from obvious. --thanks, karl.
Re: multiple online manual versions
Having multiple versions of the manual online sounds all to the good to me. As long as it's being done at all, I wouldn't hesitate to put up the manuals for every release, not just the major releases. For 1.16.x, I'm afraid I rather broke the previous rules for major releases anyway. https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-/ What I don't like about this approach is that it redirects from the generic url to the versioned url. Also, the /savannah-checkouts/ seems pretty ugly. I think it would be cleaner to create and commit /automake/manual-/ for whatever s desired. Could probably get them out of CVS. Then automake/manual/ could remain unchanged, as a copy of the current . FWIW ... --best, karl.
Re: rm -f # no more args failure?
bf> I thought that systems deriving from OpenSolaris (e.g. Illumos,
...
However, testing in an empty directory on a system without the
upated ksh93 this looks ok to me:
Bob, what you wrote before (approx. a year ago) is here:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=42529
Ending with:
Behavior of ksh93 (which has a bug) appears to depend on the PATH
setting so it will behave differently if /usr/xpg4/bin or
/usr/xpg6/bin appear in the path.
Maybe that is why you didn't see the behavior just now?
In any case, if a broken "rm -f" still exists in a free system as late
as last year, it seems premature to me to force this change now.
I grant Moritz's point that the ubiquitious "test -z ... || rm ..."
adds noise when trying to understand Automake recipes, but IMHO that is
not enough reason to induce this incompatibility.
mf> collected here as blockers:
https://debbugs.gnu.org/10828
Thanks. No one active (i.e., me) was aware this was being done, so I
didn't add Bob's report as a blocker when it came in.
I grepped the bug archive for the error message ("Your.*rm.*program")
and didn't see anything other than Bob's, but that could have missed
some.
there's only 3 reports filed from the last 10 years.
Which I see are:
May 2016 - MKS Platform component 9.X - bugs.gnu.org/23563
Dec 2015 - Unixware 7.1.4 - bugs.gnu.org/22074
Jan 2015 - Qnx 6.3.2 - bugs.gnu.org/19692
Looks like Unixware 7.1.4 may still be current, but since there are no
later reports, I guess they're not really using Automake, or set the
necessary envvar, or something. --karl
rm -f # no more args failure?
Does anyone here use or know of an active system where plain rm -f with no arguments fails? I mean, exits with bad status? We are considering changing Automake to assume this works, although we'd provide for a workaround just in case, something like make RM_F="rm -f nosuchfile" But if there are still active systems where it's a problem, we will probably make no change. Thanks, Karl
Re: automake 1.16.4 and new PYTHON_PREFIX
If anyone who weighed in on the Python prefix stuff (or didn't, for that matter) has a chance to try the new pretest at https://meyering.net/automake/automake-1.16g.tar.xz that'd be great. It'd be nice not to have to do another patch-up release. Thanks, Karl
Re: generated lex/yacc sources?
Hi Nick, Jan, all, nick> I think all that should be needed is to list the .l (or .y) file in _SOURCES normally Thanks much. I was thinking I should avoid that since the .[ly] are not ultimate sources, but if it works, fine with me. jan> BUILT_SOURCES = foo.y foo.y: foo.cweb somecommands That would be sensible, but I failed to mention the problem with BUILT_SOURCES (sorry): the manual says it only works with the general targets (all, check, install, install-exec). I need something that works with individual targets. I'll give the _SOURCES stuff a whirl. Thanks again. -k
Re: [platform-testers] automake-1.16g snapshot
Redoing the tests with 1.16g I now have 9 failed tests, the testsuite.log is attached. Thanks much for giving it a whirl right away. But are those failures anything new? FAIL: t/fn99subdir FAIL: t/lex-clean-cxx FAIL: t/lex-depend-cxx FAIL: t/test-extensions-empty FAIL: t/subpkg FAIL: t/subpkg2 FAIL: t/subpkg3 FAIL: t/subpkg-yacc FAIL: t/vala-mix2 They don't look like anything that was touched in these small changes since 1.16.4; the only real purpose for the release is to get back to the historical Python directory behavior. At least the lex/yacc stuff hasn't worked with Solaris tools for quite a while. There are a number of open bugs relating to Solaris at https://debbugs.gnu.org/cgi/pkgreport.cgi?package=automake#_0_4_3 but no one has come forward to look into them. (They will never make it to the top of my list to work on, sorry to say.) Thanks again, Karl
generated lex/yacc sources?
Suppose I want to generate a lex or yacc input file from another file, e.g., a CWEB literate program. Is there a way to tell Automake about this so that the ultimately-generated parser/lexer [.ch] files are saved in srcdir, as happens when [.ly] are direct sources, listed in *_SOURCES? I should know the answer to this, but sadly, I don't. I couldn't find any hints in the manual or sources or online, although that probably only indicates insufficient searching. Thanks for any clues. --karl
Re: automake 1.16.4 and new PYTHON_PREFIX
Regarding PYTHON_PREFIX setting in automake, I've pushed a change that (I hope) reverts to the previous behavior of using the usual GNU prefix variables by default. It's attached. The new configure option --with-python-sys-prefix yields the the 1.16.4 behavior of using the sys.* Python values. The --with-python_[exec_]prefix options are still present and unchanged, setting the prefixes explicitly. It would be really fantastic if there could be some testing of this by other people before we push out another problematic release. Jim, could you roll a test release please? --thanks, karl. P.S. Oops, I see the brief description in the change is only supposed to be one line. Well, too bad, not going to adjust now. P.P.S. Although the diff shows nearly every line being changed, in fact most of the changes are about indentation. Unfortunately. But separating the formatting changes from the real changes proved too problematic and time-consuming, and I wanted to end up with a correctly-formatted source (as best I could manage). Sorry. ampy.diff Description: Binary data
Re: [PATCH v2] automake: add install dep on install-libLTLIBRARIES to all targets
Hi Jan and all. depending on install-libLTLIBRARIES not just for bin_PROGRAMS, but all PROGRAMS and LTLIBRARIES. I installed your patch (for the record: the thread starts at https://lists.gnu.org/archive/html/automake/2021-08/msg00016.html). Thanks much. (Now, on another and unrelated front ... time to see if I can restore the previous Python dir behavior, retain the new setting with an option, and not break anything new. I sure wish there was someone out there who wanted to devote some real time to Automake maintenance.) Happy hacking, Karl
Re: [PATCH v2] automake: add install dep on install-libLTLIBRARIES to all targets
Subject: [PATCH v2] automake: add install dep on install-libLTLIBRARIES to all targets Thanks. Have you run make check? (In practice, make -j12 check or similar.) Always good to make sure nothing old breaks ... -k
