Preserve ordering of GNU ld library grouping flags

[Thomas Jahns]

Hello,

I noticed that libtool unfortunately puts -Wl,--start-group -Wl,--end-group and 
their short-hands -Wl,-( and -Wl,-) into an early position in the argument list 
of the link command instead of the libraries they are meant to go with, i.e. as 
originally specified.


Having libraries with cyclic dependencies is nothing I advocate for, but I think 
projects using libtool should be able to support it in their dependencies if 
necessary for external reasons.


So I had a look at what ltmain.sh does and figured out what special handling was 
needed. The corresponding patch is quite simple and straigh-forward. So I hope 
it can be included with an upcoming release.


The attached patch is vs. the master branch of
git://git.savannah.gnu.org/libtool.git at commit 
08c5524fb24b923ad91c2bdca836f1eafa56fd6c

which I hope is the correct place to use.

Regards, Thomas
From 01fe8e27da2cbebfe2db79606e7bb846554a6863 Mon Sep 17 00:00:00 2001
From: Thomas Jahns 
Date: Mon, 31 Oct 2016 11:08:43 +0100
Subject: [PATCH] Fix handling of flags for GNU ld library grouping.

* -Wl,--start-group -Wl,--end-group -Wl,-\( -Wl,-) must preserve their
  respective position within the list of libraries to work as expected.
---
 build-aux/ltmain.in | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 4cd3818..41e95d9 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -5324,6 +5324,11 @@ func_mode_link ()
 	arg=$func_stripname_result
 	;;
 
+  -Wl,--start-group|-Wl,--end-group|-Wl,-\(|-Wl,-\))
+	func_append deplibs " $arg"
+	continue
+	;;
+
   -Wl,*)
 	func_stripname '-Wl,' '' "$arg"
 	args=$func_stripname_result
@@ -5938,6 +5943,15 @@ func_mode_link ()
 	  alldeplibs=:
 	  continue
 	  ;;
+-Wl,--start-group|-Wl,--end-group|-Wl,-\(|-Wl,-\))
+	  if test "$linkmode,$pass" = "prog,link"; then
+	compile_deplibs="$deplib $compile_deplibs"
+	finalize_deplibs="$deplib $finalize_deplibs"
+	  else
+	deplibs="$deplib $deplibs"
+	  fi
+	  continue
+	  ;;
 	esac # case $deplib
 
 	$found || test -f "$lib" \
-- 
2.7.0.79.gdc08a19



smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Plus characters ("++") in library name

[Thomas Jahns]

On 01/02/2017 10:14 AM, c...@roberthairgrove.com wrote:

I tried removing "argppbaseimpl.hpp" from the library's __la_SOURCES list. The
tar ball was created, but that file was missing from it. I cannot add it to
include_HEADERS because it should not be installed in /usr/local/include (it's a
private header).


There's the noinst prefix. But that is totally an automake issue that's well 
documented.


Regards, Thomas



smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: How can I keep -lstdc++ out of $postdeps?

[Thomas Jahns]

On 02/23/2017 05:01 PM, Jack Bates wrote:

On 22/02/17 12:16 PM, Mike Frysinger wrote:

On 22 Feb 2017 10:32, Jack Bates wrote:

How can I avoid dynamically linking with libstdc++?
or, how can I keep -lstdc++ out of $postdeps?


let's back up.  what is it you're actually trying to do ?
you want to create a statically linked C++ program ?
you want to link a C program ?
something else ?


Well, I'm asking for a friend [1]. It sounds like he's got one version
of libstdc++ in his build infrastructure and another (older) version on
his production machines. He wants to avoid using the older version (I
think because of an issue with the STL) which is why he's trying to keep
-lstdc++ out of the linker arguments.

Regardless of whether -static-libstdc++ is the solution he ultimately
runs with, I think he's confused about _LT_SYS_HIDDEN_LIBDEPS -- as am
I. The underlying compiler/linker seems to do a more refined job of
adding/not adding these dependencies? Is there a recommended way to
disable _LT_SYS_HIDDEN_LIBDEPS?


I think the path of least resistance is still to install the newer 
libstc++ (CentOS can use newer developer tools [1] and has corresponding 
library packages to my knowledge).


Thomas

[1] 




___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Writing Autoconf tests

[Thomas Jahns]

Hello,

On 03/08/2017 08:51 PM, Yumiko wrote:

How should I write my macro so that the test program is
compiled and linked through the libtool program?


I've used a macro, originally from TeX-Live, that I extended for Fortran and 
which simply changes the compiler etc. to use libtool.


I found that for one compiler I needed to rewrite FCFLAGS at the same time 
because one of its options collides with those of libtool. But this shouldn't be 
an issue for C/C++.


If you simply add _KPSE_USE_LIBTOOL after all the necessary AC_PROG_* macros, 
you should be fine.


Some tests might need to be adapted if you e.g. try to deduce something from 
inspecting programs (which are wrapper scripts when using libtool).


Regards, Thomas


kpse_libtool.m4
Description: application/m4


smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Thomas Jahns]

On 05/30/2017 12:02 PM, Vincent Lefevre wrote:

Note that binutils 2.28 is breaking libtool for test programs
("make check") when -no-install is used. I've reported the
following bug:

  https://sourceware.org/bugzilla/show_bug.cgi?id=21476

The problem is that with this version, RUNPATH is used instead
of RPATH in the executable generated by the linker, so that
LD_LIBRARY_PATH now overrides the run path set up by libtool.

I wonder whether libtool can do something as a workaround.


The error is your setting of LD_LIBRARY_PATH, if you want the old behaviour back 
because you cannot get rid of LD_LIBRARY_PATH, consider adding


-Wl,--disable-new-dtags

to LDFLAGS. Setting LD_LIBRARY_PATH as some "permanent" solution is always an 
error waiting to happen and indicative of something that went wrong beforehand.


I repeat: don't set LD_LIBRARY_PATH, that's the real problem, libtool not 
working for you is just a symptom of that.


Regards, Thomas



smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Thomas Jahns]

On 05/30/2017 06:30 PM, Vincent Lefevre wrote:

On 2017-05-30 17:39:14 +0200, Thomas Jahns wrote:

I repeat: don't set LD_LIBRARY_PATH, that's the real problem, libtool not
working for you is just a symptom of that.


So, how can I make things work *automatically* under Linux without
setting LD_LIBRARY_PATH?


Your binaries and shared objects need to contain the correct RPATH or RUNPATH 
entries, there is several tools to help with that:


chrpath and patchelf[1] can be used to change the RPATH or RUNPATH of an already 
installed program or shared object


If you cannot ensure absolute paths, the relative option might still work, where 
an element of the RPATH starts with $ORIGIN.


Regards, Thomas

[1] patchelf is more general and can handle RPATHs getting longer which chrpath 
cannot IIRC




smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Thomas Jahns]

On 05/31/2017 01:15 PM, Vincent Lefevre wrote:

On 2017-05-31 11:58:05 +0200, Thomas Jahns wrote:

On 05/30/2017 06:30 PM, Vincent Lefevre wrote:

On 2017-05-30 17:39:14 +0200, Thomas Jahns wrote:

I repeat: don't set LD_LIBRARY_PATH, that's the real problem, libtool not
working for you is just a symptom of that.


So, how can I make things work *automatically* under Linux without
setting LD_LIBRARY_PATH?


Your binaries and shared objects need to contain the correct RPATH or
RUNPATH entries, there is several tools to help with that:


GCC doesn't generate such entries, and the user shouldn't have
to use alternate tools.


GCC doesn't generate binaries or shared libraries, ld does. Passing

-Wl,-rpath,/path/to/dependency/lib

will do just what's needed (for most people on this list libtool does so for 
them).

Regards, Thomas



smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: binutils 2.28 breaking libtool for test programs when -no-install is used

[Thomas Jahns]

On 06/01/2017 11:09 AM, Vincent Lefevre wrote:

On 2017-06-01 09:56:29 +0200, Thomas Jahns wrote:

GCC doesn't generate binaries or shared libraries, ld does. Passing

-Wl,-rpath,/path/to/dependency/lib


But this is not automatic. When typing

  $CC program.c -o program -lmpfr -lgmp

(where $CC can be gcc, tcc or icc, for instance), things should just
work, i.e. the correct options should be passed to ld *by default*.


That would be nice, on AIX directories passed with -L are also added to the 
run-time library search path. But since some people don't want that either for 
places that only contain static archives, -L and -Wl,-rpath both need to be 
accounted for by the person/automaton invoking the link editor.



Perhaps defining the LD_RUN_PATH environment variable instead of
LD_LIBRARY_PATH could be a workaround, but gold does not support
it, so that this cannot be a general solution.


Linking is very platform- and tool-dependant. LD_LIBRARY_PATH is also not 
general (read what it does on BSD or Solaris to find out more, it does not do 
anything on e.g. AIX, macOS, Windows...).



Moreover, what path should be given when the library is on NFS, where
the mount point (e.g. $HOME directory) depends on the machine?


That's just an accident waiting to happen and shows more about bad system 
management than tooling. But as I wrote, library paths that are invariant 
relative to the binary can still work.



Note that it is important that several machines can be used at the
same time, e.g. for parallel computing on a network of machines.


I don't know of any site with such a setup, seems like to little thought went 
into planning somewhere.



will do just what's needed (for most people on this list libtool
does so for them).


libtool is used to compile libraries, but in general, the user does
not use it to build an executable.


On the contrary, in autotools projects, libtool is automatically used for 
linking both, libraries and executables.


Thomas



smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Developing against $HOME/lib libraries and LD_LIBRARY_PATH

[Thomas Jahns]

Posting again, since at the first attempt I only sent directly to Paul.

On 11/22/17 15:45, Paul "LeoNerd" Evans wrote:

On Thu, 16 Nov 2017 14:49:01 -0500
Nick Bowler  wrote:


If that is not happening then maybe the libraries and/or programs are
being linked incorrectly, or perhaps there's a bug in libtool.

Example:

build+install libfoo to non-system location:
   % libtool --tag=CC --mode=compile gcc -c foo.c
   % libtool --tag=CC --mode=link gcc -rpath /tmp/instlib -o libfoo.la
foo.lo % libtool --mode=install cp libfoo.la /tmp/instlib

build libbar, linked against libfoo:
   % libtool --tag=CC --mode=compile gcc -c bar.c
   % libtool --tag=CC --mode=link gcc -rpath /usr/local/lib -o
libbar.la \ bar.lo /tmp/instlib/libfoo.la

build application linked against uninstalled libbar:
   % libtool --tag=CC --mode=compile gcc -c app.c
   % libtool --tag=CC --mode=link gcc -o app app.lo libbar.la

Inspection of results:
   % ldd .libs/app
   [...]
   libbar.so.0 => not found
   libfoo.so.0 => /tmp/instlib/libfoo.so.0 (0x7fec8f7a9000)

   [we see that that the executable has the nonstandard location for
libfoo, and libbar is not installed so it is not found by ldd, and
finally...]

   % libtool --mode=execute ./app
   [works correctly]


Ah interesting. I see a number of `-rpath` arguments in theabove, which
I don't have in my case. I wonder if that explains it. Should the .pc
file from the first library be emitting that as part of the --libs
output then?


The -rpath argument is for libtool (to tell it where the library will be 
installed). To tell libtool (or ld) where to search for libraries at run-time 
(if no .la files are installed) one needs to add -R or -Wl,-rpath options.


Regards, Thomas




smime.p7s
Description: S/MIME Cryptographic Signature
___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Accounting for -rpath when libtool isn't helping

[Thomas Jahns]

Hello Paul,

On 2/6/19 2:21 PM, Paul "LeoNerd" Evans wrote:

Nowthen - my actual question concerns what happens when I am not using
libtool to build my program. Such a case is common when building
C-level integration libraries for dynamic languages (Perl in my case).
I can't get libtool to directly build the code in question, because
Perl has its own build tooling. I can ask pkg-config what it thinks of
the --cflags and --libs to link against the library, but without
getting libtool involved in the magic step above, I don't get the
-rpath argument added.


Have you tried setting something like

CC='libtool cc'
CFLAGS='--mode=compile -some -other -flags -youd -use'
LDFLAGS='--mode=link -some -further -link -flags'

?

Regards, Thomas
--
Thomas Jahns
HD(CP)^2
Abteilung Anwendungssoftware

Deutsches Klimarechenzentrum GmbH
Bundesstraße 45a • D-20146 Hamburg • Germany

Phone:  +49 40 460094-151
Fax:+49 40 460094-270
Email:  Thomas Jahns 
URL:www.dkrz.de

Geschäftsführer: Prof. Dr. Thomas Ludwig
Sitz der Gesellschaft: Hamburg
Amtsgericht Hamburg HRB 39784

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: GCC LTO options not correctly handled

[Thomas Jahns]

On 4/12/19 6:04 PM, Bob Friesenhahn wrote:

On Fri, 12 Apr 2019, Laurent Stacul wrote:


As I don't have the ownership on the project, I need to patch (and
provide a patch upstream). But this can be tedious on big projects, so
it would great if libtool don't filter the options like -fno-lto,
-fno-whopr.


The path of least resistance is to use the GCC -Wl, or -Wc, syntax to 
pass options to the linker or compiler, respectively.  This already 
works well with Autoconf, Automake, and libtool.


Adding support to libtool to support particular GCC options is the path 
of greatest resistance.


Compiler and linker options tend to expand continually, and it is not 
reasonable to support them all.


You are right about that, but libtool considers anything with a -Wl 
prefix to be directed at itself unfortunately. Also the -Xlinker and 
-Xcompiler prefix cannot normally be used during configure tests. I 
maintain a package that uses libtool, and what I have come to rely on is to:


* switch all compilation and link tests to use libtool as early as
  possible (i.e. directly after AC_PROG_CC and LT_INIT)
* mask known flags that need to survive (like icc -shared-intel flag)
  with -Xcompiler and/or -Xlinker

For the reason you state above (the list of problematic flags only 
becoming longer), this is not entirely safe, but much better than 
educating users about the known problems.


Thomas

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Avoid -Wl,--whole-archive

[Thomas Jahns]

On 5/16/19 5:00 PM, Bob Friesenhahn wrote:
Convenience libraries are evil.  Convenience libraries slow down your 
build process dramatically and they cause 'make' not to be aware of the 
actual underlying dependencies, so that it does much more work than is 
required each type you invoke 'make'.  If you can enumerate the actual 
dependencies in the Makefile and avoid needless intermediate product 
generation, then you will achive a maximally parallel build which builds 
much faster on modern systems.


Except when they don't actually slow you down: ar and ranlib provide for 
a lookup table that in my experience is beneficial to build times, 
especially when you have a network filesystem, where access to 
individual file system objects can be more expensive.


Regards, Thomas

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: How to make libtool to use the external linker to link only one shared library in a large project?

[Thomas Jahns]

Hello,

On 5/22/19 10:46 AM, Yuri wrote:
The fftw3 project needs to link one of many libraries (libfftw3_mpi.so) 
with the externally supplied linked (mpicc executable). The rest of the 
project needs to be linked in a default way.


Could you please advise how to do this?


while it's usually easier to use the mpicc wrapper, for almost any MPI 
it's equally possible to discover the libraries linked by mpicc. For 
some MPI's its --shome:link and for others -link_info, you could then 
add those instead of swapping CC.


Regards, Thomas
--
Thomas Jahns
HD(CP)^2
Abteilung Anwendungssoftware

Deutsches Klimarechenzentrum GmbH
Bundesstraße 45a • D-20146 Hamburg • Germany

Phone:  +49 40 460094-151
Fax:+49 40 460094-270
Email:  Thomas Jahns 
URL:www.dkrz.de

Geschäftsführer: Prof. Dr. Thomas Ludwig
Sitz der Gesellschaft: Hamburg
Amtsgericht Hamburg HRB 39784

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: How to make libtool to use the external linker to link only one shared library in a large project?

[Thomas Jahns]

On 5/23/19 3:04 PM, Thomas Jahns wrote:
some MPI's its --shome:link and for others -link_info, you could then 


I meant --showme:link, of course.

Thomas

___
https://lists.gnu.org/mailman/listinfo/libtool