Re: Constantly changing libtool
On Apr 15, 2021, at 11:23 AM, Paul Smith wrote: > > On Thu, 2021-04-15 at 08:33 -0500, Laurence Marks wrote: >> there should be a way to autoupdate autoXYZ files for each system autoreconf? https://linux.die.net/man/1/autoreconf >> without user intervention. Post-checkout hooks? https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks > The problem comes (IMO) from checking in generated files. Ding-ding-ding! Correct answer. You (the OP) should be shipping a bootstrap/autogen.sh type script with your software to bring the version controlled checkout’s state up to the level you’d use for a distribution tarball, running the tools that autoreconf would do, in the proper order. Here’s the relevant section of one of mine: https://tangentsoft.com/mysqlpp/file?name=bootstrap&ln=124-129 As that script shows, there’s a lot more you may want to put in such a script. What all you write here depends on build system details of the software. For instance, my current version control system doesn’t like to version-control symlinks all that much, so in packages that need symlinks to avoid committing redundant files, I have the bootstrap script regenerate them.
Re: Constantly changing libtool
Hi Laurence, from what I can see, you are trying to solve an issue outside the scope autotools, which are AFAIK used in two contexts successfully: 1. users are provided a combination of source program and autotools generated files, and, unless they want to change the build system itself, have no need to worry about the autotools underpinnings but just run configure and make. 2. users receive source without autotools artifacts and are expected to run autotools at least once successfully. Scenario 1. is obviously not helpful if people are expected to also change the build system to e.g. include new files. In scenario 2. more work than absolutely needed might be expected of users. It seems your approach is somewhere in between: you supply autotools-generated files but expect users to also run autotools. I think you can improve outcomes by setting minimum version requirements in configure.ac so the consistency of the dev environment improves but that would probably not take care of all issues with e.g. new gcc versions. My suggestion would be to supply more than just the sources: consider handing out virtual machine images or docker containers. Doing so would enable you to keep much more under control than only the sources. Users would receive a complete environment that's almost guaranteed to produce identical results to what you prepared for lectures/hands-on sessions. Thomas Jahns On 4/15/21 4:41 PM, Laurence Marks wrote: As you say, it has files generated by autotools in a cvs (anonymous), so when checking out there is a clash. --- The software is at http://www.numis.northwestern.edu/Software/, mainly edm & semper-7.0beta. They are both old, not so great but I still use them every year or so for teaching, and there are a few others who use them. They are updated via cvs (yes, they are that old). I am updating them currently (for a class) so they may be changing. At least for edm most of the problems are with fftw-2.1.5 ( http://www.fftw.org/download.html) which is also somewhat old. (Updating the code to use more recent fftw would be a pain.) On Thu, Apr 15, 2021 at 9:27 AM Peter Johansson wrote: Hi Laurence, On 15/4/21 6:35 am, Bob Friesenhahn wrote: Most problems seem to stem from packages providing the generated files from Autoconf, Automake, and libtool so that end users don't need to have the tools available. It will be easier for people here to help if you provide a bit more information about your set-up. Is the problem when building from a tarball or when building from git/subversion? If the latter, do have you checked in files generated by autotools into git/subversion, so when checking out/cloning the source to a system with other autotools versions, there becomes a clash between versions? Cheers, Peter -- Thomas Jahns HPC-Gruppe Abteilung Anwendungssoftware Deutsches Klimarechenzentrum GmbH Bundesstraße 45a • D-20146 Hamburg • Germany Phone: +49 40 460094-151 Fax:+49 40 460094-270 Email: Thomas Jahns URL:www.dkrz.de Geschäftsführer: Prof. Dr. Thomas Ludwig Sitz der Gesellschaft: Hamburg Amtsgericht Hamburg HRB 39784 smime.p7s Description: S/MIME Cryptographic Signature
Re: Constantly changing libtool
On Thu, 2021-04-15 at 08:33 -0500, Laurence Marks wrote: > In the same way as autoXZY sets up Makefiles in an OS independent > fashion, there should be a way to autoupdate autoXYZ files for each > system without user intervention. (I don't mean automake itself or > similar, I do mean only the local files.) In an ideal world this > would be a macro in configure.ac The problem comes (IMO) from checking in generated files. If you only check in actual source files, not generated files, then you don't run into these issues with different people having different versions of build tools. For GNU make I never check in any files generated by running autotools: I check in configure.ac and Makefle.am and that's all. I don't even check in .po files (latest versions pulled automatically by the build config) or gnulib-installed files (installed during config as well). Of course, this means that everyone who wants to build the software needs to have their own local copy of ALL the tools needed to do so, including autoconf, automake, and libtool (actually we don't use libtool for GNU make but...). There's never a free lunch so you just have to decide which cost is more dear.
Re: Constantly changing libtool
As you say, it has files generated by autotools in a cvs (anonymous), so when checking out there is a clash. --- The software is at http://www.numis.northwestern.edu/Software/, mainly edm & semper-7.0beta. They are both old, not so great but I still use them every year or so for teaching, and there are a few others who use them. They are updated via cvs (yes, they are that old). I am updating them currently (for a class) so they may be changing. At least for edm most of the problems are with fftw-2.1.5 ( http://www.fftw.org/download.html) which is also somewhat old. (Updating the code to use more recent fftw would be a pain.) On Thu, Apr 15, 2021 at 9:27 AM Peter Johansson wrote: > Hi Laurence, > > On 15/4/21 6:35 am, Bob Friesenhahn wrote: > > Most problems seem to stem from packages providing the generated files > > from Autoconf, Automake, and libtool so that end users don't need to > > have the tools available. > > It will be easier for people here to help if you provide a bit more > information about your set-up. Is the problem when building from a > tarball or when building from git/subversion? If the latter, do have you > checked in files generated by autotools into git/subversion, so when > checking out/cloning the source to a system with other autotools > versions, there becomes a clash between versions? > > Cheers, > > Peter > > > > -- Professor Laurence Marks Department of Materials Science and Engineering Northwestern University www.numis.northwestern.edu "Research is to see what everybody else has seen, and to think what nobody else has thought" Albert Szent-Györgyi
Re: Constantly changing libtool
Hi Laurence, On 15/4/21 6:35 am, Bob Friesenhahn wrote: Most problems seem to stem from packages providing the generated files from Autoconf, Automake, and libtool so that end users don't need to have the tools available. It will be easier for people here to help if you provide a bit more information about your set-up. Is the problem when building from a tarball or when building from git/subversion? If the latter, do have you checked in files generated by autotools into git/subversion, so when checking out/cloning the source to a system with other autotools versions, there becomes a clash between versions? Cheers, Peter
Re: Constantly changing libtool
On Thu, 15 Apr 2021, Laurence Marks wrote: In the same way as autoXZY sets up Makefiles in an OS independent fashion, there should be a way to autoupdate autoXYZ files for each system without user intervention. (I don't mean automake itself or similar, I do mean only the local files.) In an ideal world this would be a macro in configure.ac When dealing with updated software there might not be any viable approach other than to install the various Autotools packages using a common installation prefix. Sometimes executing autoreconf --install --force using an existing Autotools installation is a viable solution. Given that all of the software you are using is already released and sometimes very archaic, you are going to need to deal with the situation in front of you. There is not going to be some new release which is somehow going to solve this issue to your satisfaction. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: Constantly changing libtool
Unfortunately things are not consistent, and in general have not been for some time. It all depends upon which version has been installed which is sometimes (often) beyond the control/competence of users. Even someone who is moderately competent can have problems. For instance, on my laptop I have both cygwin & Ubuntu; my cluster has Centos. Some of my students have VM and/or Mac-Unix, and who knows what other users in Russia/Japan/etc have -- I can't control that. I typically install autoXYZ on my systems in /use/local, but that is not a reliable or general fix. When moving between machines/OS to test (as gcc & now gfortran are moving targets) I almost always have to reconfig. Libtool is part of the issue, but there are also couplings to aclocal macros and sometimes other components as well. In the same way as autoXZY sets up Makefiles in an OS independent fashion, there should be a way to autoupdate autoXYZ files for each system without user intervention. (I don't mean automake itself or similar, I do mean only the local files.) In an ideal world this would be a macro in configure.ac _ Professor Laurence Marks "Research is to see what everybody else has seen, and to think what nobody else has thought", Albert Szent-Györgyi www.numis.northwestern.edu On Thu, Apr 15, 2021, 08:12 Bob Friesenhahn wrote: > On Wed, 14 Apr 2021, Laurence Marks wrote: > > > It is not timestamp issues, it is version issues -- e.g. libtool 2.4.2 > > versus 2.4.6 (for instance, those are just invented numbers). In many > cases > > aclocal, libtool, ltmain.sh and maybe a few others do not work if they > were > > created with a different autoXYZ version than is on the computer where > they > > are being installed. It seems to be "special" to libtool. > > There has not been a libtool release since 2014 and it is 2021 > already. > > The issues you are complaining about should not be existing for quite > a long time already, and are easily corrected by installing consistent > Autotools versions under the same installation prefix. If you don't > like the archaic versions your system provides, then simply uninstall > (or not use) those packages and install using current Autotools > release versions. > > It is intended that libtool components are installed into the software > which uses them. This assures consistent versions. If the libtool > components were not installed and distributed with the package, then > there could be problems as you describe. > > Bob > -- > Bob Friesenhahn > bfrie...@simple.dallas.tx.us, > https://urldefense.com/v3/__http://www.simplesystems.org/users/bfriesen/__;!!Dq0X2DkFhyF93HkjWTBQKhk!CjQjRsGtyWBePsFky9wEHrKJFYZgwc9ACi_I8ll7bvAa-RaEo1asjDsmsAO-D4rcH7MlDA$ > GraphicsMagick Maintainer, > https://urldefense.com/v3/__http://www.GraphicsMagick.org/__;!!Dq0X2DkFhyF93HkjWTBQKhk!CjQjRsGtyWBePsFky9wEHrKJFYZgwc9ACi_I8ll7bvAa-RaEo1asjDsmsAO-D4oDSZ6wRw$ > Public Key, > https://urldefense.com/v3/__http://www.simplesystems.org/users/bfriesen/public-key.txt__;!!Dq0X2DkFhyF93HkjWTBQKhk!CjQjRsGtyWBePsFky9wEHrKJFYZgwc9ACi_I8ll7bvAa-RaEo1asjDsmsAO-D4od8dCEwg$ >
Re: Constantly changing libtool
On Wed, 14 Apr 2021, Laurence Marks wrote: It is not timestamp issues, it is version issues -- e.g. libtool 2.4.2 versus 2.4.6 (for instance, those are just invented numbers). In many cases aclocal, libtool, ltmain.sh and maybe a few others do not work if they were created with a different autoXYZ version than is on the computer where they are being installed. It seems to be "special" to libtool. There has not been a libtool release since 2014 and it is 2021 already. The issues you are complaining about should not be existing for quite a long time already, and are easily corrected by installing consistent Autotools versions under the same installation prefix. If you don't like the archaic versions your system provides, then simply uninstall (or not use) those packages and install using current Autotools release versions. It is intended that libtool components are installed into the software which uses them. This assures consistent versions. If the libtool components were not installed and distributed with the package, then there could be problems as you describe. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: Constantly changing libtool
It is not timestamp issues, it is version issues -- e.g. libtool 2.4.2 versus 2.4.6 (for instance, those are just invented numbers). In many cases aclocal, libtool, ltmain.sh and maybe a few others do not work if they were created with a different autoXYZ version than is on the computer where they are being installed. It seems to be "special" to libtool. On Wed, Apr 14, 2021 at 3:36 PM Bob Friesenhahn < bfrie...@simple.dallas.tx.us> wrote: > On Wed, 14 Apr 2021, Laurence Marks wrote: > > > I have some software which has been "fine" for about 15 years, with one > > exception: every 6-12 months or so when I (or students/others) compile it > > we run into version problems. The typical ones are version mismatch > between > > libtool and/or aclocal (called from libtool). While I can futze around > with > > libtoolize/aclocal/autoconf/automake etc until it works, this is not > viable > > for users who barely understand linux. > > > > Is there some way in configure.ac (Makefile.am) to setup so the relevant > > version fixes can be done automatically? There are so many posts about > > version mismatches that this should be there. Maybe it is and I just > missed > > it. > > Most problems seem to stem from packages providing the generated files > from Autoconf, Automake, and libtool so that end users don't need to > have the tools available. This would be combined with file time-stamp > issues. Many source control systems do not preserve file time-stamps. > > If the operating system distribution (or some other source) provides > versions which differ from what the package provided, then there will > be a problem. > > One thing that can help quite a lot is to include AM_MAINTAINER_MODE > in configure.ac so that the user needs to add the > --enable-maintainer-mode option in order to enable self-maintenance > rules. If the user does not provide the option, then the existing > files will be used, which is usually just fine unless a file has > actually been modified. > > It is sometimes said that 'autoreconf --force' will solve all > problems, but in my experience this is not always true. > > Bob > -- > Bob Friesenhahn > bfrie...@simple.dallas.tx.us, > https://urldefense.com/v3/__http://www.simplesystems.org/users/bfriesen/__;!!Dq0X2DkFhyF93HkjWTBQKhk!G3ek0gkcIrC1V4ZVdsgymVIc8_myhWPWhHAgikVKVkhnWQpQZyjj1MyMzheM1ajhg0kYkg$ > GraphicsMagick Maintainer, > https://urldefense.com/v3/__http://www.GraphicsMagick.org/__;!!Dq0X2DkFhyF93HkjWTBQKhk!G3ek0gkcIrC1V4ZVdsgymVIc8_myhWPWhHAgikVKVkhnWQpQZyjj1MyMzheM1ahChk89tg$ > Public Key, > https://urldefense.com/v3/__http://www.simplesystems.org/users/bfriesen/public-key.txt__;!!Dq0X2DkFhyF93HkjWTBQKhk!G3ek0gkcIrC1V4ZVdsgymVIc8_myhWPWhHAgikVKVkhnWQpQZyjj1MyMzheM1ahdtcF-sA$ > -- Professor Laurence Marks Department of Materials Science and Engineering Northwestern University www.numis.northwestern.edu "Research is to see what everybody else has seen, and to think what nobody else has thought" Albert Szent-Györgyi
Re: Constantly changing libtool
On Wed, 14 Apr 2021, Laurence Marks wrote: I have some software which has been "fine" for about 15 years, with one exception: every 6-12 months or so when I (or students/others) compile it we run into version problems. The typical ones are version mismatch between libtool and/or aclocal (called from libtool). While I can futze around with libtoolize/aclocal/autoconf/automake etc until it works, this is not viable for users who barely understand linux. Is there some way in configure.ac (Makefile.am) to setup so the relevant version fixes can be done automatically? There are so many posts about version mismatches that this should be there. Maybe it is and I just missed it. Most problems seem to stem from packages providing the generated files from Autoconf, Automake, and libtool so that end users don't need to have the tools available. This would be combined with file time-stamp issues. Many source control systems do not preserve file time-stamps. If the operating system distribution (or some other source) provides versions which differ from what the package provided, then there will be a problem. One thing that can help quite a lot is to include AM_MAINTAINER_MODE in configure.ac so that the user needs to add the --enable-maintainer-mode option in order to enable self-maintenance rules. If the user does not provide the option, then the existing files will be used, which is usually just fine unless a file has actually been modified. It is sometimes said that 'autoreconf --force' will solve all problems, but in my experience this is not always true. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: Constantly changing libtool
Hi Laurence - sorry, but I can't imagine how to automatically correct version mismatches. The possible problems are both unpredictable and endless, seems to me. Maybe others here have some more useful idea. --best, karl.
Constantly changing libtool
I have some software which has been "fine" for about 15 years, with one exception: every 6-12 months or so when I (or students/others) compile it we run into version problems. The typical ones are version mismatch between libtool and/or aclocal (called from libtool). While I can futze around with libtoolize/aclocal/autoconf/automake etc until it works, this is not viable for users who barely understand linux. Is there some way in configure.ac (Makefile.am) to setup so the relevant version fixes can be done automatically? There are so many posts about version mismatches that this should be there. Maybe it is and I just missed it. -- Professor Laurence Marks Department of Materials Science and Engineering Northwestern University www.numis.northwestern.edu "Research is to see what everybody else has seen, and to think what nobody else has thought" Albert Szent-Györgyi
