Re: Ian's solution [was: What hack in ld.so?]
From: Olaf Weber <[EMAIL PROTECTED]> Date: 31 Jan 1999 11:39:23 +0100 > Hi, all! > There's been so much traffic on this thread, that I suspect most > people have missed the fact that Ian Lance Taylor has analyzed and > *solved* the problems with interaction between libtool and > libc5-compat shared libraries. By, as far as I can tell, breaking the ABI. The ABI was broken a long time ago. That patch breaks it further. Ian
Re: Ian's solution [was: What hack in ld.so?]
Date: Sun, 31 Jan 1999 13:08:51 -0600 From: David Engel <[EMAIL PROTECTED]> I am the Debian and upstream maintainer of the libc5 ld.so. Ian's patch will not be going in. I think most people understand this, but I should make clear that it's not my patch. I assume it's from Eric Troan. I found it in the RedHat distribution. FWIW, I cringed the first time I saw what RedHat had done. They did not foresee the evils of -rpath and the problems it would cause in the libc5/libc6 transition. I can sympathize. I cringed the first time I saw how the dynamic linker had been hacked to no longer do a straight path search. There is some very ugly code in the binutils linker to deal with that. I guess it's something of a standoff. Somebody made what I consider to be an unfortunate decision a while back, with an incomplete hack to the dynamic linker. Now that decision is repercussing out to other software packages. I accepted the repurcussions into the binutils, overriding my personal judgement. Alexandre doesn't want to accept them into libtool, and I personally don't blame him. Alexandre has said that he's willing to accept a patch to not generate a -rpath argument for any directory listed in /etc/ld.so.conf. It's possible to construct cases for which this will fail--because of the dynamic linker hack, /etc/ld.so.conf is not synonymous with the list of directories the dynamic linker will search--but there will probably be fewer failure cases than the current situation. I encourage the people who can't abide the current situation to write such a patch. Let's not forget that this is only a temporary problem. Programs built using the current libtool on a current Linux system will work on all foreseeable future Linux systems, because nobody will ever have to make this type of unfortunate decision again. Ian
Re: What hack in ld.so?
Date: Sat, 30 Jan 1999 16:52:48 -0700 (MST) From: Jason Gunthorpe <[EMAIL PROTECTED]> > That's not what I'd like libtool to do. I agree there is a problem to > be fixed, I just think that libtool is not the only piece of software > that may have to be changed to fix it, because it is not the only > piece of software that uses -rpath. libtool is however the only piece of software that we cannot easially change. I don't understand the reasoning here. The libtool files that a package will use are distributed with that package. If you are willing to modify a package's Makefile to remove -rpath, then it ought to be just as easy or difficult to modify the package's ltconfig file. Ian
Re: What hack in ld.so?
Date: Sat, 30 Jan 1999 16:06:04 -0700 (MST)
From: Jason Gunthorpe <[EMAIL PROTECTED]>
On 30 Jan 1999, Alexandre Oliva wrote:
> Obviously, this would have never been needed if old libraries had not
> been replaced with (in)compatible versions, but the maintainers of
> Debian have already taken this step, despite the fact that this would
Look, it was not us that decided to do this. It was the upstream
maintainers, other dists and a huge combination of factors. It is not in
our power to choose a different direction to solve these problems, we must
have libc6 xlib called libX11.so.6 and we must have libc5 called
libX11.so.6 - that is what all the other dists did, that is the default
and expected compilation behavoir of xlib and it is what all the new glibc
binary-only programs are using (ie netscape)
If you want to say that is a dumb way then fine, but you have not proposed
an alternative to solving the versioning problem and you have not proposed
an alternative way to handle the requirement of identical sonames and
libtool continues to perpetuate this 'bad' behavoir and makes it worse by
providing no way to get around it with the standard linux ld.so
What you are saying, I think, is that you have two programs with
A) DT_NEEDED libc.so.5, DT_NEEDED libX11.so.6
B) DT_NEEDED libc.so.6, DT_NEEDED libX11.so.6
Neither has DT_RPATH. The system has four libraries:
libc.so.5
libc.so.6
libX11.so.6 with DT_NEEDED libc.so.5
libX11.so.6 with DT_NEEDED libc.so.6
You want programs A and B to both work, without modification.
This was done by modifying the search algorithm used by the dynamic
linker so that it chooses the version of libX11.so.6 which matches the
version of libc.so.N used by the program. This was done by recording
in /etc/ld.so.cache the version of libc which the shared library uses.
The dynamic linker was modified to look in /etc/ld.so.cache for
libraries which were not found in DT_RPATH, but to only select
libraries listed in /etc/ld.so.cache which matched the version of the
dynamic linker being used. Programs which are linked against
different versions of libc must then use different dynamic linkers.
There are in fact three different dynamic linkers which understand
this ld.so.cache algorithm: the old a.out dynamic linker, the libc5
dynamic linker, and the glibc dynamic linker.
I just spent some time looking at the ld.so sources. Interestingly,
it seems to me that everything will work correctly in the sources I
was looking at. That is because the libc5 dynamic linker on my system
(RedHat 5.2) was modified to search the library cache before using the
application's DT_RPATH.
I think that is a hack that Debian is missing: it is the final hack to
the libc5 dynamic linker to change the search path to account for the
moved shared libraries even when rpath is used. I have appended the
RedHat patch below. This is to ld.so-1.9.5. Of course, this will
technically break the handling of DT_RPATH. However, we've already
determined that DT_RPATH binaries will not work correctly anyhow,
because the shared libraries were moved. So using this patch should
not make us any worse off.
Indeed libtool causes such a severe problem that if you take a libtool
program, compile it on a libc5 Slackware and try to run it on -any- glibc
system IT WILL NOT WORK.
This is not quite right. As I described above, glibc and libc5
binaries use a different dynamic linker. So the behaviour of your
libc5 binary depends upon the behaviour of your libc5 dynamic linker.
That linker does not come from glibc. Although I can not test this, I
now believe that if you take a libtool program, compile it on a libc5
Slackware and try to run it on a RedHat 5.2 system, it will work.
Ian
--- ld.so-1.9.5/d-link/readelflib1.c.ewtMon Nov 17 10:04:15 1997
+++ ld.so-1.9.5/d-link/readelflib1.cMon Nov 17 10:23:15 1997
@@ -179,38 +179,10 @@
goto goof;
}
- /*
- * The ABI specifies that RPATH is searched before LD_*_PATH or
- * the default path of /usr/lib.
- * Check in rpath directories
- */
- for (tpnt = _dl_loaded_modules; tpnt; tpnt = tpnt->next) {
-if (tpnt->libtype == elf_executable) {
- pnt1 = (char *)tpnt->dynamic_info[DT_RPATH];
- if(pnt1) {
- pnt1 += (unsigned int) tpnt->loadaddr + tpnt->dynamic_info[DT_STRTAB];
- while(*pnt1){
- pnt2 = mylibname;
- while(*pnt1 && *pnt1 != ':') {
- if (pnt2 - mylibname < 1024)
- *pnt2++ = *pnt1++;
- else
- pnt1++;
- }
- if (pnt2 - mylibname >= 1024)
- break;
- if(pnt2[-1] != '/') *pnt2++ = '/';
- pnt = libname;
- while(*pnt) *pnt2++ = *pnt++;
- *pnt2++ = 0;
- tpnt1 = _dl_load_elf_shared_library(mylibname, 0);
- if(tpnt1) return tpnt1;
- if(*pnt1 == ':') pnt1++;
- }
- }
-}
- }
-
+ /* EWT - change thing
Re: -rpath with libtool and Debian Linux
Date: Sat, 30 Jan 1999 23:42:32 +0100 From: Marcus Brinkmann <[EMAIL PROTECTED]> > In general, it's convenient to store the path in the executable any > time a shared library is installed in a directory which the dynamic > linker does not search by default. Yes, I should have narrowed my question to system libraries. Unfortunately, system libraries are most likely to cause the problems, for example if people hard code /usr/X11R6/lib/ for xaw library and you want to use xaw3d... It's hard to distinguish system libraries from non-system libraries, except by the distinction quoted above: a non-system library is a library installed in a directory which the dynamic linker does not search by default. Unfortunately, the GNU/Linux dynamic linker reportedly uses a rather complex algorithm, in which it makes decisions based on the libc version number which libraries are linked against, which would seem to make it hard to determine just which directories the dynamic linker searches by default. In the normal case I think one can assume that the dynamic linker will search any directory listed in /etc/ld.so.conf, and it would be OK to omit a -rpath argument for any shared library installed in one of the directories listed in that file. Note that although you can set up a case in which the xaw library is found in /usr/X11R6/lib, it's not normal. Normally the program will be linked against libxaw.so.N, and will have a specified search path, the rpath, to find that file. > Incidentally, I don't know what you mean by saying both soname and > library name. There is only one name recorded for a shared library: > the soname. Ignorance I think. I thought soname is only the number, and a lib is stored in libfoo.x.y.z, where foo is the library name and x.y.z the soname. If I got it wrong, I apologize. When I, and I think most others, use the word soname, I refer to the DT_SONAME tag in a shared library which appears in a DT_NEEDED tag in the executable. The soname is set in a shared library using the -h/--soname option, and is copied into the executable by the program linker. In this case the soname is the full name of the file: libfoo.x.y.z. Ian
Re: -rpath with libtool and Debian Linux
Date: Sat, 30 Jan 1999 23:10:26 +0100 From: Marcus Brinkmann <[EMAIL PROTECTED]> * Is there a better way to do a library transition? I think it is very obvious, that the only correct behaviour is to change the library/soname of all involeved libraries when doing a transition. So we had to move either from libfoo.2 to libfoog.2, libfoo.libc6.2, libc6/libfoo.2 or whatever, until a new library with a new, incompatible, soname is released. Changing the name does not look correct, so we had to change soname or path. When you make an incompatible change to a shared library, change the soname. That's just a matter of choosing a name for the new library and tweaking a symlink. There is no reason to do anything else. What do you mean when you say ``changing the name does not look correct?'' Don't confuse the name of the library found by the runtime linker (libc.so) with the soname used by the dynamic linker (libc.so.6). The runtime linker will normally find a symlink to the soname. When you build a shared library, use the -h/--soname option to set the soname of the shared library. Shouldn't there be a way to override rpath? Currently,LD_LIBRARY_PATH does override rpath, right? No, LD_LIBRARY_PATH does not override rpath. The rpath is searched first, and then the LD_LIBRARY_PATH is searched. I think you may have agreed with that later in your message. Ian
Re: -rpath with libtool and Debian Linux
Date: Sat, 30 Jan 1999 23:30:43 +0100 From: Marcus Brinkmann <[EMAIL PROTECTED]> Why should the application choose to hard code the PATH in the binary? AFAICS, there is no apparent reason for it. What has the path to do with the library? I think the only thing that should be hard coded is the exact soname and library name. Maybe I am missing something? Suppose you have your own set of shared libraries, in your own directory. Suppose you want to let other people use your programs linked against your own shared libraries. You can tell everyone who uses your programs to set LD_LIBRARY_PATH, or you can simply use -rpath so that your programs can always find your shared libraries. In general, it's convenient to store the path in the executable any time a shared library is installed in a directory which the dynamic linker does not search by default. Incidentally, I don't know what you mean by saying both soname and library name. There is only one name recorded for a shared library: the soname. Ian
