RE: Future plans for Autotools
On Thu, 13 May 2021, FOURNIER Yvan wrote: Hi Karl, Regarding the possible addition of a libtool option to ignore .la files, I would need to take a deeper look into how libtool works (I have only scratched the surface and experimented with it as a "black box" so far, but If I do get around to it, I will definitely send a patch or post it here). It would be useful to know why .la files are "incorrect". The most common concern is that libtool links using full dependencies (evaluated using information from the .la files) rather than relying on a target's implicit dependency support. This means that libraries may be "over-linked" and thus record undesirable (e.g. to the distribution maintainers) dependency information. If recorded dependency information in a .la file becomes incorrect because an implicit dependency was re-built, or an alternate implementation library was substituted, then that creates a problem. Libtool is first and foremost a portability tool. It also helps support static linkage. Solving the library depencency issue properly (while not relying on non-portable implicit dependency support or 3rd party software) might require more controls/knobs for libtool as well as an update to the .la file format so it is clear which dependencies are direct, and which are simply dependencies of the direct dependencies. If the ability to static link is not important, or if only using/supporting GNU Linux with GCC then libtool is not needed. GCC itself knows how to create shared libraries on popular targets. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
RE: Future plans for Autotools
Hi Karl,
Regarding the possible addition of a libtool option to ignore .la files, I
would need to take a deeper look into how libtool works (I have only scratched
the surface and experimented with it as a "black box" so far, but If I do get
around to it, I will definitely send a patch or post it here).
The strong link between autoconf and automake is fine for me. I remarked on
some slight discrepancies (multiple vs single generated ".h" files assumed),
but this is really not a showstopper.
One thing I might try to look at is how to handle dependencies in Fortran the
way they are handled in C. If this depends on support from the compiler, I am
afraid that as always, there may be no "standard" compiler command syntax, but
if at least there is a "common" one, that would help. So I'll share anything
interesting I find on this when I find some time, though it might wait several
weeks. If C++ precompiled headers use the same logic, it will add additional
motivation.
Best regards,
Yvan
De : k...@freefriends.org
Envoyé : jeudi 13 mai 2021 03:20
À : FOURNIER Yvan
Cc : automake@gnu.org
Objet : Re: Future plans for Autotools
Hi Yvan - sorry for the delayed reply.
While configure/automale/libtool seem to be designed to work together,
Yes, they were. It seems your major issues are with libtool. I can
(uselessly) sympathize, but unfortunately that's all I can do. Libtool
is currently unmaintained (according to GNU records), so until someone
volunteers to work on it, there will presumably be no further changes to
it. (Maybe you or someone reading this wants to take up the challenge?)
- but when it encounters .la files with incorrect .la files, libtool
adds the incorrect dependencies or options, with no workaround
If you (or anyone) can provide a patch for that specific critical issue
in libtool, it might be possible to somehow make a bug fix release.
- or make them more modular, and better able to work
independently. This is probably more of a documentation / example
code issue than an actual development issue.
I expect you know this, but Automake completely depends on
Autoconf. That's not going to change, in the current reality. OTOH,
using libtool is completely optional. Many auto{make,conf} projects
don't use libtool.
For Automake, and I expect for Autoconf, specific documentation
suggestions are welcome, but I at least can't write anything from
"make them more modular", however much I agree in principle.
There are quite a few "best practices" autotools documents around, some
written by past autotools developers, others by autotools users. Sounds
like you have enough experience that you could write your own useful
addition :).
Best regards,
Karl
Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à
l'intention exclusive des destinataires et les informations qui y figurent sont
strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination, toute diffusion ou toute publication totale ou partielle, est
interdite sauf autorisation expresse.
Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le
copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si
vous avez reçu ce Message par erreur, merci de le supprimer de votre système,
ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support
que ce soit. Nous vous remercions également d'en avertir immédiatement
l'expéditeur par retour du message.
Il est impossible de garantir que les communications par messagerie
électronique arrivent en temps utile, sont sécurisées ou dénuées de toute
erreur ou virus.
This message and any attachments (the 'Message') are intended solely for the
addressees. The information contained in this Message is confidential. Any use
of information contained in this Message not in accord with its purpose, any
dissemination or disclosure, either whole or partial, is prohibited except
formal approval.
If you are not the addressee, you may not copy, forward, disclose or use any
part of it. If you have received this message in error, please delete it and
all copies from your system and notify the sender immediately by return message.
E-mail communication cannot be guaranteed to be timely secure, error or
virus-free.
Re: Future plans for Autotools
Hi Yvan - sorry for the delayed reply.
While configure/automale/libtool seem to be designed to work together,
Yes, they were. It seems your major issues are with libtool. I can
(uselessly) sympathize, but unfortunately that's all I can do. Libtool
is currently unmaintained (according to GNU records), so until someone
volunteers to work on it, there will presumably be no further changes to
it. (Maybe you or someone reading this wants to take up the challenge?)
- but when it encounters .la files with incorrect .la files, libtool
adds the incorrect dependencies or options, with no workaround
If you (or anyone) can provide a patch for that specific critical issue
in libtool, it might be possible to somehow make a bug fix release.
- or make them more modular, and better able to work
independently. This is probably more of a documentation / example
code issue than an actual development issue.
I expect you know this, but Automake completely depends on
Autoconf. That's not going to change, in the current reality. OTOH,
using libtool is completely optional. Many auto{make,conf} projects
don't use libtool.
For Automake, and I expect for Autoconf, specific documentation
suggestions are welcome, but I at least can't write anything from
"make them more modular", however much I agree in principle.
There are quite a few "best practices" autotools documents around, some
written by past autotools developers, others by autotools users. Sounds
like you have enough experience that you could write your own useful
addition :).
Best regards,
Karl
Re: Future plans for Autotools
On Thu, 6 May 2021, Andy Tai wrote: a general question: would a rewrite in Python or some other language, to keep the same functionality as the current implementation, a viable goal, or that would not be a productive thing to do? There are several major aspects of Automake. One is the use of Perl scripts to transform a user-provided high-level description into a portable Makefile. Another is the design and syntax of the Makefile itself. There are also m4 Autoconf macros to integrate Automake into Autoconf scripts. Supporting a feature such as two targets from the same source file apparently is aided by depending on GNU Make features so the result would no longer be a portable Makefile. It has yet to be decided to require use of GNU Make. Perl is not necessarily more difficult to read and understand than Python but some popular Perl syntax may be very difficult for new users to understand whereas Python mostly avoids such syntax. Regardless, the current major problem is now how difficult the Automake software is to maintain. The major problem is that there is no one fully dedicated to maintain it. All work appears to be done on an intermittent part-time basis. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: Future plans for Autotools
a general question: would a rewrite in Python or some other language, to keep the same functionality as the current implementation, a viable goal, or that would not be a productive thing to do? On Thu, May 6, 2021 at 1:44 PM Bob Friesenhahn wrote: > > On Thu, 6 May 2021, Karl Berry wrote: > > > > (*) https://lists.gnu.org/archive/html/automake/2021-03/msg00018.html > > So far the response has been nil. > > I don't recall seeing that email. I did see an email thread regarding > Autoconf which immediately became a lot of "need to support this soon" > and "wouldn't it be nice" discussion, totally missing the point of the > text they were reponding to (i.e. "project is on the gurney and may > expire without help"). > > Regardless, "baby bird" syndrome seems to be striking a great many > established free software projects which are mature and heavily used. > > Projects operated by billion dollar companies with teams of developers > paid to sit in a cubicles and write free sofware seem to be doing > fine. > > Unfortunately, this is not the model for most GNU projects. > > Bob > -- > Bob Friesenhahn > bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ > Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt > -- Andy Tai, a...@atai.org, Skype: licheng.tai, Line: andy_tai, WeChat: andytai1010 Year 2021 民國110年 自動的精神力是信仰與覺悟 自動的行為力是勞動與技能
Re: Future plans for Autotools
On Thu, May 6, 2021 at 4:44 PM Bob Friesenhahn wrote: > > On Thu, 6 May 2021, Karl Berry wrote: > > > > (*) https://lists.gnu.org/archive/html/automake/2021-03/msg00018.html > > So far the response has been nil. > > I don't recall seeing that email. I did see an email thread regarding > Autoconf which immediately became a lot of "need to support this soon" > and "wouldn't it be nice" discussion, totally missing the point of the > text they were reponding to (i.e. "project is on the gurney and may > expire without help"). > > Regardless, "baby bird" syndrome seems to be striking a great many > established free software projects which are mature and heavily used. I don't think it's fair to expect that a user of autotools is capable of becoming expert enough in Perl to become a developer of autotools. > Projects operated by billion dollar companies with teams of developers > paid to sit in a cubicles and write free sofware seem to be doing > fine. > > Unfortunately, this is not the model for most GNU projects. I think this is a larger problem with companies tending to put support into projects with licenses that they find more appealing. For instance, IMO, the GPLv3 provoked strong support for billion dollar companies to latch on to llvm. There is also a mindset shift with current developers. The autotools projects generally were guided by the idea of spreading GNU software everywhere, and thus they take great care to, first and foremost, make the job trivial for the person compiling the software at the expense of more work on the side of the developer; and secondly, to support a great many systems. Whereas tools like CMake take a different approach. They aim to both support only a few systems (comparatively), and they aim to make the job of the developer easier while the job of the person compiling software becomes quite difficult and cumbersome. Said more simply, software from earlier days is better for "them", while software from today is better for "us", for flexible definitions of "them" and "us". I don't particularly like the current trend.
Re: Future plans for Autotools
On Thu, 6 May 2021, Karl Berry wrote: (*) https://lists.gnu.org/archive/html/automake/2021-03/msg00018.html So far the response has been nil. I don't recall seeing that email. I did see an email thread regarding Autoconf which immediately became a lot of "need to support this soon" and "wouldn't it be nice" discussion, totally missing the point of the text they were reponding to (i.e. "project is on the gurney and may expire without help"). Regardless, "baby bird" syndrome seems to be striking a great many established free software projects which are mature and heavily used. Projects operated by billion dollar companies with teams of developers paid to sit in a cubicles and write free sofware seem to be doing fine. Unfortunately, this is not the model for most GNU projects. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: Future plans for Autotools
I think automake really needs to support this soon. Sounds reasonable to me, but to be clear, Automake will only support things that volunteers care enough about to actually dig into the code and write patches for. New developers/maintainers are needed. As I previously explained(*) / pleaded, Jim and I simply cannot commit the necessary time for substantive development -- we can't even keep up with incoming bug reports and interesting suggestions. --best, karl. (*) https://lists.gnu.org/archive/html/automake/2021-03/msg00018.html So far the response has been nil.
Re: Future plans for Autotools
On Mon, May 3, 2021 at 5:34 AM Thomas Jahns wrote: > > - Our code is a mix of Fortran and C, with a bit of C++. Automake still > > deos not support Fortran 90-type module dependencies, so we have to manage > > manual dependencies in one of our Makefile.am's. More modern systems handle > > Fortran (not quite the latest fad) much better. > > At our site, there are a number of scripts in use that generate the Fortran > module/file dependencies on demand. For those not knowing Fortran: source > files > are compiled to both, .mod files for use in runs of the compiler and .o files > for the later link step. The .mod files can follow a number of conventions in > respect to suffix and upper/lower case basename and suffix. This is going to become very important to keep automake relevant, as c++ is adopting modules using the same two-file-output mechanism as Fortran. GNU Make supports having a rule generate multiple files and tracking them accordingly. I think automake really needs to support this soon.
Re: Future plans for Autotools
On Sun, 2021-05-02 at 17:49 +, FOURNIER Yvan wrote: > - When using Gettext, "make update-po" modifies code in the source > tree based on a command in the build tree. When using separate source > and build trees, this is somewhat surprising, so having an equivalent > command (not based on the generated Makefiles) for the source tree > would be more consistent as regards source/build separation. Yes, I agree this is a weird one for the reason stated. On a related note, and correct me if I'm wrong, but automake doesn't automatically generate that target. You still have to implement it yourself? -- Kip Warner -- Senior Software Engineer OpenPGP signed/encrypted mail preferred https://www.thevertigo.com signature.asc Description: This is a digitally signed message part
Re: Future plans for Autotools
Hello, some comments with a similar HPC background: On 2021-05-02 19:49, FOURNIER Yvan wrote: [...] - it is very easy to find the configure options from a previous run at the top of the config.log and config.status files, and then copy/past them so as to generate an new build in a clean (empty) build directory. This is IMHO a strong point compared to CMake, whose caching aspects can lead to complex issues, and where regenerating a configuration command is not so easy. This is indeed valuable, but could be improved by adding the appropriate quotes. [...] - autoconf can generate multiple configuration files (for example public/private header files), but automake assumes only one is used. I don't remeber the details, but had to work around this 15 years ago to re-implement the generation of a "light" external config.h in m4 and shell instead of using built-n features due to this. We also found that we could not install config.h and use a Perl script to insert config.h macros into the installed headers. - Our code is a mix of Fortran and C, with a bit of C++. Automake still deos not support Fortran 90-type module dependencies, so we have to manage manual dependencies in one of our Makefile.am's. More modern systems handle Fortran (not quite the latest fad) much better. At our site, there are a number of scripts in use that generate the Fortran module/file dependencies on demand. For those not knowing Fortran: source files are compiled to both, .mod files for use in runs of the compiler and .o files for the later link step. The .mod files can follow a number of conventions in respect to suffix and upper/lower case basename and suffix. [...] - .la files broken by Linux packaging: in the case of an MPI library built with various dependencies, some of which are loaded dynamically as plugins, and not installed by default with the minimal packages, the .la file lists some non-installed (and non-needed) libraries as dependencies. I found that in general it's advisable to look through .la files and edit them if needed. - incorrect options: recently, trying to use the Cray compiler toolchain on an Arm64 machine, some low-level dependency related to back-end gcc libraries added a "-pthread" option in the link line, while the top-level compiler used a different syntax and did not accept that. We encounter the same problem with the NAG Fortran compiler (equivalent options are -Wc,-pthread -Wl,-pthread). The simplest solution was a wrapper around to the compiler to patch up the options, another colleague patched ltmain.sh to adjust appropriately. Although I have quite a bit of experience working around libtool's quirlks, I was simply not able to build my code using that toolchain on that machine. Total failure just before the finish line... -I have had a similar issue generating shared libraries with with CUDA, though in that case I could work around this by duplicating and patching the generated libtool file, and adding a Makefile.am rule for .cu files. But I have never seen another project optionally using CUDA which bothered doing this rather than using another build system (ie. if there is a better solution, I did not find it). Since CUDA depends on the installed kernel module, it's also giving us headaches regarding stability of installed libraries like MPI. There is support for optional .cu files in our highest level software, I'll have to look up what the responsible colleague did that might have eluded you. We already depend on a build environment with the PGI/NVIDIA compiler for OpenACC when GPU support is needed and that adds everything for cuda below the layer visible to libtool, we might not be seeing your specific issue. [...] Libtool has many nice aspects, such as handling of versioning, rpaths, and more, but its insistence that it is always right, even in situations where it produces unusable commands can really be a showstopper. I can second that, we now use 7 patch sets for ltmain.sh to address issues regarding compilers not matching the expectations of libtool. I have not looked at Meson much so far. In the short term, the fact that it requires ninja, which is not yet ubiquitous on HPC systems, makes things a bit less comfortable in our case, but the fact that is uses Python is an advantage (Python is so prevalent in scientific software that just about all users and developers in our community need to learn at least some rudiments anyways, which is not true of m4, Makefiles, or cmake). So it could become a serious contender (whereas the other Python-based contenders such as scons and Waf seem to have less momentum). I found scons to be sufficiently complex to be impenetrable to a number of scientists, who otherwise knew and used Python. Regards, Thomas -- Thomas Jahns HPC-Gruppe Abteilung Anwendungssoftware Deutsches Klimarechenzentrum GmbH Bundesstraße 45a • D-20146 Hamburg • Germany Phone: +49 40 460094-151
Re: Future plans for Autotools
Hello, Sorry for reacting a bit late to the January discussion about Autotool's future. As a longtime user of the GNU Autotools for a large computational dynamics code (code_saturne.org), here are a few hopefully constructive remarks about our experience so far. Sorry if my post is a bit long. Most-liked aspects - - excellent cross-compilation support - `configure --help` is very practical (CMake's equivalent is much messier, and one of the reasons we did not switch) - "almost" no dependencies on the target system except for a shell and Make. This is especially useful on compute clusters which may have a minimal system. - pretty good documentation - it is very easy to find the configure options from a previous run at the top of the config.log and config.status files, and then copy/past them so as to generate an new build in a clean (empty) build directory. This is IMHO a strong point compared to CMake, whose caching aspects can lead to complex issues, and where regenerating a configuration command is not so easy. Some Inconsistencies - While configure/automale/libtool seem to be designed to work together, there are a few inconsistencies : - autoconf can generate multiple configuration files (for example public/private header files), but automake assumes only one is used. I don't remeber the details, but had to work around this 15 years ago to re-implement the generation of a "light" external config.h in m4 and shell instead of using built-n features due to this. - Though one of the initial advantages of the GNU autotools was that a shell and Make were sufficient on the target system, GNOME devs started breking that with pkg-config, and now it seems resorting to that has become ubiquitous in many autotools-based projects, so the dependency list has increased in practice. Tools like autopoint also add complexity, as they are available on some systems not on others. - Our code is a mix of Fortran and C, with a bit of C++. Automake still deos not support Fortran 90-type module dependencies, so we have to manage manual dependencies in one of our Makefile.am's. More modern systems handle Fortran (not quite the latest fad) much better. - When using Gettext, "make update-po" modifies code in the source tree based on a command in the build tree. When using separate source and build trees, this is somewhat surprising, so having an equivalent command (not based on the generated Makefiles) for the source tree would be more consistent as regards source/build separation. Major issue --- Worse, autoconf does not take advantage of libtool's ".am" files to find dependencies, while libtool does not allow bypassing these files. This means that: - we need to manually search for dependencies anyways - but when it encounters .la files with incorrect .la files, libtool adds the incorrect dependencies or options, with no workaround So while .la files were probably a great idea in the early days of the GNU project, they are of limited use now (especially since tools not based on autotools do not provide them), and are a source of issues. I have encountered several examples of this : - .la files broken by Linux packaging: in the case of an MPI library built with various dependencies, some of which are loaded dynamically as plugins, and not installed by default with the minimal packages, the .la file lists some non-installed (and non-needed) libraries as dependencies. - incorrect options: recently, trying to use the Cray compiler toolchain on an Arm64 machine, some low-level dependency related to back-end gcc libraries added a "-pthread" option in the link line, while the top-level compiler used a different syntax and did not accept that. Although I have quite a bit of experience working around libtool's quirlks, I was simply not able to build my code using that toolchain on that machine. Total failure just before the finish line... -I have had a similar issue generating shared libraries with with CUDA, though in that case I could work around this by duplicating and patching the generated libtool file, and adding a Makefile.am rule for .cu files. But I have never seen another project optionally using CUDA which bothered doing this rather than using another build system (ie. if there is a better solution, I did not find it). Libtool has many nice aspects, such as handling of versioning, rpaths, and more, but its insistence that it is always right, even in situations where it produces unusable commands can really be a showstopper. I am seriously contemplating removing libtool (while keeping autoconf/automake) due to the blocking "no option to ignore .la files even when incorrect" issue, as I have no other workaround. This will probably require quite a bit of extra work, but is better than hitting a wall. Comparison with other tools ---
Re: Future plans for Autotools
I'd just like to suggest that in the event of future significant development on a new automake, or a revamped build system in whatever way, that the new system not be called "autoconf" or "automake". It seems inevitable to me that any such new system would have incompatibilities with the old, and this would cause trouble both for the new developers (retaining perfect compatibility is hard and ugly, thus typically does not happen), and existing users (their long-time build scripts would be broken).(*) It would surely be nice for a new system to be "as compatible as possible" with the old, but pretending it is a drop-in replacement seems like it would just lead to bad feelings all around. --best, karl. (*) This already happened with autoconf 2.70, unfortunately. Even the minor incompatibilities it introduced are a significant barrier for some of us (at least me) to adoption. The kinds of rewrites being talked about here would surely have far more compatibility issues.
Re: Future plans for Autotools
On Tue, 2021-01-26 at 11:01 -0800, Andy Tai wrote: > GNU Make integrates with guile. Maybe such extension can be > done using guile for flexibility? The problem is that hardly any standard distributions build GNU make with Guile support enabled. If this was used basically it would end up requiring anyone who wanted to use automake to build their own version of GNU make first. And, we'd have a catch-22 where Guile (if it used automake) would need a GNU make built with Guile before it could be built...
Re: Future plans for Autotools
GNU Make integrates with guile. Maybe such extension can be done using guile for flexibility? (ref: https://www.gnu.org/software/make/manual/html_node/Guile-Integration.html#Guile-Integration) On Thu, Jan 21, 2021 at 12:11 PM Paul Eggert wrote: > > One possible way forward is to have an Autoconf 2 that builds atop GNU > Make, both as a partial replacement for Automake (which is what Emacs > does already), and as a way to speed up and simplify configuration. If > 'configure' were mostly a front end to a GNU Make invocation, it could > run configuration probes in parallel which would certainly be a win for > me. And perhaps configuration probes could be written in GNU Make rather > than m4, which would also be a win because it'd be one less language to > learn. (Of course we could continue to support existing m4-based > probes, run sequentially, as well as letting Automake do its thing for > people who prefer Automake.) >
Re: Future plans for Autotools
As always, thanks for all your effort Zack! I wanted to share some of my thoughts on Autoconf and friends. Maybe I wrote too much. For me the most important requirement of the GNU build system is that it must be as straightforward as possible for novice users to build free software packages from source code, with or without local modification. This is what empowers users with the benefits of free software. If users are unable to build or modify the software that they use, they are unable to take advantage of those benefits. For me, every other consideration is secondary. The interface consistency prescribed by the GNU coding standards goes a long way: you learn the steps for one package and can apply that knowledge to almost any other package. The trend towards requiring everyone to build from VCS snapshots and requiring zillions of specific versions of various build tools is concerning. Unfortunately I think many developers don't really care about the user experience when it comes to building their software releases from source. This brings me to another important strength of the GNU Build System: if I prepare a package today I want to be confident that people will still be able to build it 5, 10, 20 or more years from now. Now obviously we can't predict the future but we can look to past experience: just today, I unpacked GNU Bison 1.25 (ca. 1996) on a modern GNU/Linux system, running on a processor architecture and distribution that didn't even exist back then, and it builds *out of the box*. Typical issues encountered with old GNU packages are usually very minor if you have any problems at all. For a more complex example, I tried building glib-1.2.10 (ca. 2001). I had to update config.sub/config.guess to the latest, set CC='gcc -std=gnu89' (because the code does not work with C99 inline) and edit one line of code to disable use of an obsolete GNU C extension (both compilation problems are due to not following the Autoconf philosophy and using version checks instead of feature checks, oops!) My general experience with CMake is that you probably can't build any old packages because whatever version of CMake you have available simply doesn't understand the package's build scripts, and the version which could understand them just doesn't work on your system because you have a newer processor or something. I don't have enough experience with Meson to say. Mainstream free software packages have only very recently started using it. On the GNU side, glib-2.60 (ca. 2019) converted to meson and I am able to build it. If possible, I will have to try again in 2039. I bet the autoconf-based glib-1.2.10 tarball from 2001 will still mostly work, and so will the 1996 version of GNU Bison. Cheers, Nick
Re: Future plans for Autotools
> "Gavin" == Gavin Smith writes: Gavin> I remember somebody was Gavin> complaining about this page: Gavin> https://www.gnu.org/software/automake/manual/html_node/Program-and-Library-Variables.html Gavin> and asking what "maude" meant - it turned out it was the name of the Gavin> dog or something of the person who first wrote the program, who didn't Gavin> want it to change. Yeah, that was my dog's name. Feel free to change it if you think that's important. I don't recall objecting, but I wouldn't put it past me. Tom
Re: Future plans for Autotools
On 1/21/21 8:01 AM, Zack Weinberg wrote: I know that at least one person has tried to write a set of GNU Make library files intended to replace it altogether, but I've never seen anyone *finish* that project. I'd very much like to see someone give that another go. GNU Emacs never used Automake but its developers eventually decided to require GNU Make. This has not proved to be a problem in practice, as GNU Make is ubiquitous, and Automake features not directly supported by GNU Make don't seem to be needed by Emacs builds. One possible way forward is to have an Autoconf 2 that builds atop GNU Make, both as a partial replacement for Automake (which is what Emacs does already), and as a way to speed up and simplify configuration. If 'configure' were mostly a front end to a GNU Make invocation, it could run configuration probes in parallel which would certainly be a win for me. And perhaps configuration probes could be written in GNU Make rather than m4, which would also be a win because it'd be one less language to learn. (Of course we could continue to support existing m4-based probes, run sequentially, as well as letting Automake do its thing for people who prefer Automake.) Whatever way forward is chosen will surely need coordination with Gnulib, which has essentially taken over most of the low-level system-specific porting tasks that Autoconf used to have. When Emacs adopted Gnulib but did not want to use Automake, we had to hack on Gnulib to support that; bigger hacks to Gnulib will surely be needed to support any of the changes proposed here.
Re: Future plans for Autotools
My two cents: the competing build systems, cmake, meson, have in their "features" (and major motivation for their original development) of supporting Xcode and Microsoft Visual Studio. Supporting for these seem to become necessary for GNU Autotools to compete, even if these two may be outside of interests for people who have traditionally worked on autotools? And supporting such may be distraction for developer resources?
Re: Future plans for Autotools
Zack, On Thu, Jan 21, 2021 at 9:12 AM Zack Weinberg wrote: > On Wed, Jan 20, 2021 at 5:15 PM Zack Weinberg wrote: > > Now we've all had a while to recover from the long-awaited Autoconf > > 2.70 release, I'd like to start a conversation about where the > > Autotools in general might be going in the future. > > > Now we've all had a while to recover from the long-awaited Autoconf > > 2.70 release, I'd like to start a conversation about where the > > Autotools in general might be going in the future. > > Thanks for all the great thoughtful responses that came in overnight! > I see many people expressing similar sentiments, so rather than > respond to each of you individually I'm going to summarize and reply > to each strand in this one message. > > I like the way you think. But, as your response so clearly indicates, there's actually very little we can change in a radical way: 1. We shouldn't change the underlying language(s) and tools because one of the key strengths of the Autotools is (their products') ability to run everywhere without off-box dependencies. In my humble opinion, nothing the GNU project has ever done comes close to the value they created by designing the Autotools to work as seamlessly (for the end-user) as they do. Download a tarball, unpack, ./configure && make. How much simpler could it be? No additional tools to install - everything required is already there - for the end user. Of course, developers may need to install a few additional tools to build these tarballs from the base source repository, but developers are not end-users and those are the "customers" GNU was trying to appease. If you build from source repositories, you're effectively a developer, not an end-user - not always the case, but 98% of the time this is true. Additionally, making such changes would require long man-hours of unpaid effort - something the world is not willing to give. Gone are the days of university grad students who know enough of what they're talking about to effectively take on this task for the sake of their educational goals (this is the way these tools started, after all). In fact, as much as they'd hate to admit it, the GNU project has become much more corporate than educational today. But let's not get bent out of shape by it - the truth of the matter is that no one does anything for free (you yourself solicited funding for the 2.70 update - which did NOT ultimately come from philanthropic individuals. 2. The Autotools are actually more transparent than any other build tools out there. All these other tools' (cmake, maven, etc) - that purport to be so much simpler because they insulate the user from the underlying details of the build process - these tool's primary failure is that this very insulation keeps users from being able to make the changes they need to accomplish their unique project-specific build goals. Anyone who has nothing but good things to say about this aspect of cmake, maven, gradle, or whatever, has simply not worked on a project that requires them to move far enough away from the defaults. I've used them all and I've spent hours in frustration trying to determine how to work around the shortcomings of some "do-all" (except what I want) tool function. This is simply not an issue with the Autotools. As someone mentioned earlier in this thread, you can drop shell script into a configure.ac file, and make script into a Makefile.am file. That is the very definition of transparency. No other tool in existence allows this level of flexibility. The most interesting feedback I read in the responses was that the Autotools had an appalling lack of documentation which - ironically - is actually not true. At the risk of sounding self-serving, I'll say this: in the research I did for my book (Autotools, No Starch Press, 2019), I primarily (95%) used the GNU Autotools documentation as reference sources. The information is all there and actually very concise - too concise sometimes. The problem with it is two fold. First, the documentation for GNU projects is most often written by the author of the code - that's the open source way, after all. This brings with it all sorts of issues that most corporate documentation (inadvertently) bypasses by standard corporate siloing protocol - that is, they relegate doc writing to doc writers, who are often not experts in the code, so they have to spend the time researching the product from a user's perspective. This is exactly the perfect storm of events that (usually) generates pretty good documentation. Second, the documentation is written in a manner that is not conducive to fast learning. There are two types of instructive material. The first is reference material that, by its very nature, requires the user to read and absorb significant amounts of information until they hit a critical mass of understanding - an exponential learning curve that starts out long and slow - at which point they avalanche into the steeper portion of the curve.
Re: Future plans for Autotools
On Thu, Jan 21, 2021 at 11:01:34AM -0500, Zack Weinberg wrote:
> Having said that, switching to *anything else* would be a gigantic
> task -- multiple full-time person-years of effort just for the core --
> and would mean either porting or losing all of the third-party macro
> libraries. I don't think it makes sense to explore this possibility
> any further unless both a lot of funding and a lot of developers
> materialize.
It could be possible for autoconf to become one option of several for
use with automake. Then it would not be necessary to replicate the
entire functionality of autoconf in a new system. People could have
(part of) the configuration written in Perl or other languages.
For this to work, automake would need to change quite a lot, for
example not always using configure.ac as an input file. There would
be the question of information sharing between automake's own checks and
checks done in user configuration code. It would never be trivial
to add a new configuration language, but they could all have similar
characteristics, for example communicating their results as a list
of names and values (i.e. the "name=@name@" of autoconf), some of which
would be required for Automake rules (e.g. whatever configuration method
was used, it would have to yield a value for CC if C files were being
compiled).
Perhaps there could be an Automake variable CONFIGURATION_METHOD
that could select which system was to used ("autoconf" being the
current system). Other options could be to generate the configure
script completely automatically, or to access other programs that
allow some user code in the script.
This all depends on somebody creating a replacement for autoconf, though,
and integrating it with automake.
> A lesser possibility we could think about, would be dusting off GNU M4
> 2.0 and thinking about what we could change about the M4 language to
> make it friendlier for Autoconf's purposes. Again I would want
> funding and developers lined up, but we'd not need quite so much of
> either.
If you ask if the current design of the autotools is something that
should be preserved forever, when you look at what its use is (creating
build systems for software), you would answer that it shouldn't be. M4
is part of this.
> [Bob Friesenhahn]
> > There is a fork of Automake which was re-done to be based on GNU
> > Make. This assumes that makefiles written in GNU make can noticeably
> > improve things. I like your idea of supporting other underlying
> > build tools besides 'make'. Make's dependence on matching rules and
> > file time stamps is problematic and it does not scale.
>
> I haven't ever followed automake development closely. I know that
> at least one person has tried to write a set of GNU Make library files
> intended to replace it altogether, but I've never seen anyone *finish*
> that project. I'd very much like to see someone give that another go.
>
> The alternative to the Make family of tools, that I'm most familiar
> with, is Ninja, and it also depends on matching rules and file time
> stamps. What do you have in mind as a replacement for these things?
> Is there already an alternative tool that you like?
One issue here is that Makefile.am can contain Makefile rules, which
transfer cleanly to use by "make", but possibly not for other programs
like ninja. (I don't know anything about ninja, though.)
> I have a lot more sympathy with people who are annoyed with the GNU
> *project* and/or the FSF. I have any number of my own grievances with
> them. However, I would prefer not to get into that in this
> conversation, because it's liable to drag it off-topic and/or turn
> into a flame war. Let's declare the known obstacles imposed on
> Autotools by the FSF -- concretely, the requirement for copyright
> assignment paperwork and the requirement not to use proprietary CI and
> forge services -- immobile and out-of-scope for this discussion.
There could be a discussion to be had about project governance and how
to stimulate development in GNU packages: advertising for contributors,
promoting use of the package, encouraging exploratory development. I
don't have any concrete suggestions, though, and many changes are also
possible for the worse.
Re: Future plans for Autotools
On Wed, Jan 20, 2021 at 5:15 PM Zack Weinberg wrote: > Now we've all had a while to recover from the long-awaited Autoconf > 2.70 release, I'd like to start a conversation about where the > Autotools in general might be going in the future. > Now we've all had a while to recover from the long-awaited Autoconf > 2.70 release, I'd like to start a conversation about where the > Autotools in general might be going in the future. Thanks for all the great thoughtful responses that came in overnight! I see many people expressing similar sentiments, so rather than respond to each of you individually I'm going to summarize and reply to each strand in this one message. > continuous integration for Autotools This is one of my highest priorities for what volunteer time I have in the next few months. The biggest difficulty with CI, as I see it, is that Autotools needs to be tested on a very wide variety of operating systems, and ideally *versions* of operating systems, and unusual compilers, and so on, in order to confirm that it's still fulfilling its contract. Testing just on the "big three" operating systems (Linux, Windows, MacOS) typically supported by commercial CI providers ... would be better than nothing, but would not have helped all that much with the 2.70 release, if I'm honest. GNU's Hydra instance (https://www.gnu.org/software/devel.en.html#Hydra -- I didn't previously know about this, thank you Dan for pointing it out) is a strong contender just because it has FreeBSD and Solaris as well. > implementation languages - can't we do better than shell+perl+m4? I deeply sympathize with frustration with M4 and portable shell as implementation languages. Perl isn't so bad IMO, but being restricted to perl 5.6.0 with no CPAN modules is also a headache. Having said that, switching to *anything else* would be a gigantic task -- multiple full-time person-years of effort just for the core -- and would mean either porting or losing all of the third-party macro libraries. I don't think it makes sense to explore this possibility any further unless both a lot of funding and a lot of developers materialize. A lesser possibility we could think about, would be dusting off GNU M4 2.0 and thinking about what we could change about the M4 language to make it friendlier for Autoconf's purposes. Again I would want funding and developers lined up, but we'd not need quite so much of either. And we probably could bump up the minimum version requirement for Perl quite some ways, since it's only needed at autoreconf time. I'm not sure how much Autoconf would gain from this, though, since it's only incrementally changed since 5.6 and Autoconf doesn't do *that* much with it. Automake uses it much more heavily, though, perhaps people from that project would like to comment? [Nate Bargmann] > One strength of the Autotools that I think stands above the rest is > the fact that a user of a distributed package does not need to > install any of the Autotools packages. I definitely agree with this, and I'm putting it right next to the "implementation languages" note because this is a compelling reason to stick to shell as the language for generated configure scripts. The only alternative that would come anywhere near as close in terms of "just unpack and run" portability is Perl. Which is a tempting possibility -- it would wipe out dozens of internal headaches due to shell's limitations and portability quirks -- but again, multiple full-time person-years of effort, losing the third-party macro libraries, and we *would* be making life harder for people starting from scratch on OSes that don't bundle Perl. > Autotools are too difficult to work on Yes indeed. On the assumption that we are stuck with shell, M4, and Perl, however, what can we do about it? I have a few ideas, maybe you have more: - A linter for configure.ac and third-party macro libraries. This wouldn't be any fun to write, because it would have to parse both M4 and shell, *accurately*, but it would be valuable to every user of autoconf. - The parser for the linter could also be used as the basis for a reimplementation of autoupdate that *doesn't* rely on running configure.ac through M4 with most of the macros disabled. That implementation choice is responsible for most of the bugs in autoupdate, and for most of its limitations (e.g. not being able to fix quotation errors). - Audit existing configure.ac's looking for pain points: where are people having trouble doing the checks they need, where are people not realizing that they don't need to work so hard, where are people not realizing that they don't need to check for that at all... - Audit the core for workarounds that are no longer necessary now we can rely on shell functions (I'm looking at you, AC_LANG_*). - Better documentation and better debugging tools. Speaking of which, here's an interesting bit from Russell: [Russell Shaw] > The biggest problem i had that wasted t
Re: Future plans for Autotools
On 21/1/21 9:15 am, Zack Weinberg wrote: Now we've all had a while to recover from the long-awaited Autoconf 2.70 release, I'd like to start a conversation about where the Autotools in general might be going in the future. Clearly any future development depends on finding people who will do the work, but before we worry about that I think we might want to figure out what work we _want_ to do. ... Agree with all that. The biggest problem i had that wasted the most time is that none of the documentation gave a good overview of the sequence of steps the main perl script did to process the files and produce the output, and intermediate files involved. Using a language like C++ where a step-by-step debugger like gdb can be used would have been good. Having a nuts and bolts low level detail of how it all works makes one much more productive. I'm talking of real software engineering ppl that understand compilers and not programmers that are really artist dabbling users. It is the real engineers that are more motivated to understand and be contributors. As a result, i only gradually got proficient over a few years, and having to comprehend a large manual on ancient shell variants and study M4 didn't help. M4 is easy if there's a decent tutorial readily available. Currently one needs to download other manuals and tutorials to get a good idea of it, and it's a big step for starting programmers. I understand mostly how things work, but got bogged down in understanding the perl script. If i mastered it all, i'd think more about what can be changed or improved. I did one early iteration of a tool that did the functions of 'make' using an input description of the files one wanted in a project. It also had its own shell implementation. Users would need a copy of that shell installed. An idea was to have the users script automatically build and install the shell in a location the user or system dictated. Another option was an automatic download from a trusted place. I could get more done if i had another go at it, but i'm too busy on a different tool. If GNU paperwork gets in the way, just bypass it and release new stuff as BSD. Then it's truly free.
Re: Future plans for Autotools
On Wed, 2021-01-20 at 17:15 -0500, Zack Weinberg wrote: > As a starting point, I wrote up a "strengths, weaknesses, > opportunities, and threats" analysis for Autotools -- this is a > standard project management technique, if you're not familiar with > it, there's a nice writeup in the draft of the book my friend and > colleague Sumana Harihareswara is writing [ > https://changeset.nyc/resources/getting-unstuck-sampler-offer.html ]. Thank you for sharing, Zack (and Sumana). This looks very thoughtful and worth all of us reading at some point. I for one have no intention of migrating away from the Autotools for any of my projects. My personal experience is that every time someone tries to develop a replacement, they tend to reinvent it and in a poorer way. -- Kip Warner -- Senior Software Engineer OpenPGP signed/encrypted mail preferred https://www.thevertigo.com signature.asc Description: This is a digitally signed message part
Re: Future plans for Autotools
* On 2021 20 Jan 17:33 -0600, Bob Friesenhahn wrote: > Autotools is in great danger of becoming irrelevant, at least for new > software development. A lot of people feel hostile toward it. This is quite true. As a co-maintainer of a library project that uses Autoconf, Automake, and Libtool, I've been asked point blank after the most recent major release why we have not switched to cmake. My answer was that I don't have the time nor the desire after spending a considerable amount of time getting my head around Autotools enough to improve and maintain the project's build system with a modicum of sanity. The project builds on any POSIX platform that Autotools can support and MS Windows. Portability of the build system is vital to this project and that is the second priority of the build system. I've built projects from cmake and the Qt equivalent and while their build definition files often seem simple, it appears to me there is a lot that goes on behind the scenes. One strength of the Autotools is that Autoconf and Automake are almost infinitely malleable. Each supports dropping shell syntax in configure.ac or make syntax in a Makefile.am respectively. There just simply need to be ways for the project maintain to extend the build system in ways specific to that project. I've never cared to investigate cmake to know whether it is as capable thus I am unsure if cmake trades control for simplicity. > It seems to me that Autoconf is too difficult to work on. There is too much > to become expert at in order for new volunteers to make an impact. The same > is true for libtool. > > In my opinion, a new "language" designed specifically to meet the needs of > Autoconf should be developed and Autoconf should be re-implemented using it. > There should be no more need to run external utilities like 'sed', or 'awk' > or other things which can be implemented internally in a way which does not > suffer from the white space and delimiter issues that shell script does. I'm not here to defend m4, on the contrary, my life would continue on just fine if I never had to look at m4 ever again. The very few times I have tried have not been pleasant. On the other hand, is there any enthusiasm for reimplementing sed, awk, or other such utilities since Perl did it over 30 years ago and other scripting languages have done so since? The good thing about Automake is that though it is written in Perl, the implementation language doesn't seem to bleed through into the syntax of Makefile.am. With Autoconf, so long as the maintainer can stick with the provided macros or third party ones, life is tolerable. On the rare occasion that I started looking at enhancing a macro, I began questioning some life choices! On the other hand, I am not so quick to abandon shell syntax. It is the lowest common denominator on POSIX like systems. That said, perhaps a reasonable approach would be to target a somewhat older POSIX specification and abandon support for shells that cannot reach even that low bar. As an aside, I have worked to ensure that any shell script that I write is free of Bashisms even though they can make writing scripts easier. > It seems that the core precept that Automake should produce portable > makefiles should be re-evaluated if most systems provide GNU make, or can > easily be updated have it. At the moment the project(s) that seem to be working toward replacing all things GNU do allow the later installation of GNU software. Will such projects eventually become hostile to even the premise of installing GPL software, let alone GNU software, ala iOS? If so, my concerns about support for other make implementations likely become moot as the project I'm involved with is GPL/LGPL licensed. > There is a fork of Automake which was re-done to > be based on GNU Make. This assumes that makefiles written in GNU make can > noticeably improve things. What negative impact would relying solely on GNU make have on third party projects intending to build on platforms where the installation of GNU make may become discouraged? While I've not read of any public plans of such, the actions of FreeBSD make me curious if they might head in that direction eventually. > The support for 'distcheck' is excellent. > > Regardless, thanks for your ideas and the red alert. Agreed on both of these counts, Bob. - Nate -- "The optimist proclaims that we live in the best of all possible worlds. The pessimist fears this is true." Web: https://www.n0nb.us Projects: https://github.com/N0NB GPG fingerprint: 82D6 4F6B 0E67 CD41 F689 BBA6 FB2C 5130 D55A 8819 signature.asc Description: PGP signature
Re: Future plans for Autotools
It seems better not to start another language. with already lack of resources, that will further dilate available resources, and hard to compete with other tools already us9ng Python's mature ecosystem On Wed, Jan 20, 2021 at 3:32 PM Bob Friesenhahn < bfrie...@simple.dallas.tx.us> wrote: > > In my opinion, a new "language" designed specifically to meet the > needs of Autoconf should be developed and Autoconf should be > re-implemented using it. There should be no more need to run external > utilities like 'sed', or 'awk' or other things which can be > implemented internally in a way which does not suffer from the white > space and delimiter issues that shell script does. > -- > Bob Friesenhahn >
Re: Future plans for Autotools
On Wed, 20 Jan 2021, Zack Weinberg wrote: Now we've all had a while to recover from the long-awaited Autoconf 2.70 release, I'd like to start a conversation about where the Autotools in general might be going in the future. Clearly any future development depends on finding people who will do the work, but before we worry about that I think we might want to figure out what work we _want_ to do. Zack, your text was excellent and I agree with all of it. Autotools is in great danger of becoming irrelevant, at least for new software development. A lot of people feel hostile toward it. It seems to me that Autoconf is too difficult to work on. There is too much to become expert at in order for new volunteers to make an impact. The same is true for libtool. In my opinion, a new "language" designed specifically to meet the needs of Autoconf should be developed and Autoconf should be re-implemented using it. There should be no more need to run external utilities like 'sed', or 'awk' or other things which can be implemented internally in a way which does not suffer from the white space and delimiter issues that shell script does. It seems that the core precept that Automake should produce portable makefiles should be re-evaluated if most systems provide GNU make, or can easily be updated have it. There is a fork of Automake which was re-done to be based on GNU Make. This assumes that makefiles written in GNU make can noticeably improve things. I like your idea of supporting other underlying build tools besides 'make'. Make's dependence on matching rules and file time stamps is problematic and it does not scale. It is unfortunate that GNU produced a much more powerful 'make' tool (a paradigm invented in 1976), but not a new build tool with fresh ideas to improve build quality and reduce build times on modern systems. The macro definitions provided by Autoconf have been proven by the test of time and allow the underlying implementation to be changed. Only surrounding shell script might need to be changed if the underlying implementation changes. The support for 'distcheck' is excellent. Regardless, thanks for your ideas and the red alert. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: Future plans for Autotools
Thanks for writing all of this. I'm writing from the perspective of a long-term user of the autotools. A discussion like the one you have started will likely attract many opinions. Some will be contradictory. However, somebody in the end will have to decide. The challenge seems to be to evolve the system with regard to backward compatibility. You mention the use of many languages, like m4. This brings me to my first suggestion: make automake work without autoconf. This would potentially allow configuration scripts (if there is to be such a thing) to be written in a language that doesn't use m4. The purpose of autoconf, and of configure scripts, could be clarified. The manual says, "Autoconf is a tool for producing shell scripts that automatically configure software source code packages to adapt to many kinds of Posix-like systems." Posix-like systems, and thus portability, is implied to be a major issue. Many would not care about portability, especially when starting a small personal project. (This was my reaction when I first started using the autotools.) I see the purpose of a configure script to be two-fold: * Allowing a user to make choices about what features are built, and how the package is built (e.g. installation paths) * Adapting to operating system variants The first will always be relevant, even if you don't care about the second. I think configuration scripts on GNU-like systems would always have a heavy element of shell script in them, even if m4 or autoconf macros were abandoned. There would be no need to move so far away from this. If automake could generate its own configure script without autoconf, then the interface of the GNU Build System could continue unchanged for people building software. It would also simplify matters for people learning to use the autotools, that they could just create a single Makefile.am to describe their project (like a CMakefile for cmake or .pro file for qmake), rather than having to create Makefile.am and configure.ac. It could be made easier to get started with automake. For example, "AUTOMAKE_OPTIONS=foreign" should be the default; otherwise automake refuses to finish if required files aren't found in the project. There's a political problem here to persuade people of this. The documentation could also be improved. I remember somebody was complaining about this page: https://www.gnu.org/software/automake/manual/html_node/Program-and-Library-Variables.html and asking what "maude" meant - it turned out it was the name of the dog or something of the person who first wrote the program, who didn't want it to change. So think getting consensus is impossible for changes that are needed to make the program simpler, easier and better. There's a vacuum of responsibility where nobody wants to step on others' feet. Forking the project would go nowhere as nobody would use the fork. Your idea about adding support for new programming languages is good and if someone did this along with other changes, it could stimulate adoption of the other changes. It would be nice if all the code required to support a new language could occur as a self-contained "module" within automake. On Wed, Jan 20, 2021 at 10:15 PM Zack Weinberg wrote: > > Now we've all had a while to recover from the long-awaited Autoconf > 2.70 release, I'd like to start a conversation about where the > Autotools in general might be going in the future. Clearly any future > development depends on finding people who will do the work, but before > we worry about that I think we might want to figure out what work we > _want_ to do. > > As a starting point, I wrote up a "strengths, weaknesses, > opportunities, and threats" analysis for Autotools -- this is a > standard project management technique, if you're not familiar with it, > there's a nice writeup in the draft of the book my friend and > colleague Sumana Harihareswara is writing [ > https://changeset.nyc/resources/getting-unstuck-sampler-offer.html ]. > I'm going to paste the full text of this analysis below, please reply > inline. You can also read it on my blog at > https://www.owlfolio.org/development/autoconf-swot/ . > > zw > > > I’ve been a contributor to GNU projects for many years, notably both > GCC and GNU libc, and recently I led the effort to make the first > release of Autoconf since 2012 (release announcement for Autoconf > 2.70). For background and context, see the LWN article my colleague > Sumana Harihareswara of Changeset Consulting wrote. > > Autoconf not having made a release in eight years is a symptom of a > deeper problem. Many GNU projects, including all of the other > components of the Autotools (Automake, Libtool, Gnulib, etc.) and the > software they depend upon (GNU M4, GNU Make, etc.) have seen a steady > decline in both contributor enthusiasm and user base over the past > decade. I include myself in the group of declining enthusiasts; I > would not have done the work leading up to the Autoconf 2.70 release > if I ha
Future plans for Autotools
Now we've all had a while to recover from the long-awaited Autoconf 2.70 release, I'd like to start a conversation about where the Autotools in general might be going in the future. Clearly any future development depends on finding people who will do the work, but before we worry about that I think we might want to figure out what work we _want_ to do. As a starting point, I wrote up a "strengths, weaknesses, opportunities, and threats" analysis for Autotools -- this is a standard project management technique, if you're not familiar with it, there's a nice writeup in the draft of the book my friend and colleague Sumana Harihareswara is writing [ https://changeset.nyc/resources/getting-unstuck-sampler-offer.html ]. I'm going to paste the full text of this analysis below, please reply inline. You can also read it on my blog at https://www.owlfolio.org/development/autoconf-swot/ . zw I’ve been a contributor to GNU projects for many years, notably both GCC and GNU libc, and recently I led the effort to make the first release of Autoconf since 2012 (release announcement for Autoconf 2.70). For background and context, see the LWN article my colleague Sumana Harihareswara of Changeset Consulting wrote. Autoconf not having made a release in eight years is a symptom of a deeper problem. Many GNU projects, including all of the other components of the Autotools (Automake, Libtool, Gnulib, etc.) and the software they depend upon (GNU M4, GNU Make, etc.) have seen a steady decline in both contributor enthusiasm and user base over the past decade. I include myself in the group of declining enthusiasts; I would not have done the work leading up to the Autoconf 2.70 release if I had not been paid to do it. (I would like to say thank you to the project funders: Bloomberg, Keith Bostic, and the GNU Toolchain Fund of the FSF.) The Autotools are in particularly bad shape due to the decline in contributor enthusiasm. Preparation for the Autoconf 2.70 release took almost twice as long as anticipated; I made five beta releases between July and December 2020, and merged 157 patches, most of them bugfixes. On more than one occasion I was asked why I was going to the trouble—isn’t Autoconf (and the rest of the tools by implication) thoroughly obsolete? Why doesn’t everyone switch to something newer, like CMake or Meson? (See the comments on Sumana’s LWN article for examples.) I personally don’t think that the Autotools are obsolete, or even all that much more difficult to work with than some of the alternatives, but it is a fair question. Should development of the Autotools continue? If they are to continue, we need to find people who have the time and the inclination (and perhaps also the funding) to maintain them steadily, rather than in six-month release sprints every eight years. We also need a proper roadmap for where further development should take these projects. As a starting point for the conversation about whether the projects should continue, and what the roadmap should be, I was inspired by Sumana’s book in progress on open source project management (sample chapters are available from her website) to write up a strengths, weaknesses, opportunities, and threats analysis of Autotools. This inventory can help us figure out how to build on new opportunities, using the Autotools’ substantial strengths, and where to invest to guard against threats and shore up current weaknesses. Followup discussion should go to the Autoconf mailing list. Strengths In summary: as the category leader for decades, the Autotools benefit from their architectural approach, interoperability, edge case coverage, standards adherence, user trust, and existing install base. Autoconf’s feature-based approach to compiled-code portability scales better than lists of system quirks. The Autotools carry 30+ years’ worth of embedded knowledge about portability traps for C programs and shell-based build scripting on Unix (and to a lesser extent Windows and others), including variants of Unix that no other comparable configuration tool supports. Autoconf and Automake support cross-compilation better than competing build systems. Autoconf and Automake support software written in multiple languages better than some competing build systems (but see below). Autoconf is very extensible, and there are lots of third-party macros available. Tarball releases produced by Autotools have fewer build dependencies than tarball releases produced by competing tools. Tarball releases produced by Autotools have a predictable, standardized (literally; it’s a key aspect of the GNU Coding Standards) interface for setting build-time options, building them, testing them, and installing them. Automake tries very hard to generate Makefiles that will work with any Make implementation, not just GNU make, and not even just (GNU or BSD) make. The Autotools have excellent reference-level documentation (better than CMake and Meson’s). As they are GNU projects, users can hav
