RE: -rpath with libtool and Debian Linux
-Original Message- From: Ian Lance Taylor [mailto:[EMAIL PROTECTED] Sent: Saturday, January 30, 1999 11:40 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; debian-devel@lists.debian.org; [EMAIL PROTECTED] Subject: Re: -rpath with libtool and Debian Linux Date: Sat, 30 Jan 1999 23:30:43 +0100 From: Marcus Brinkmann [EMAIL PROTECTED] Why should the application choose to hard code the PATH in the binary? AFAICS, there is no apparent reason for it. What has the path to do with the library? I think the only thing that should be hard coded is the exact soname and library name. Maybe I am missing something? Suppose you have your own set of shared libraries, in your own directory. Suppose you want to let other people use your programs linked against your own shared libraries. You can tell everyone who uses your programs to set LD_LIBRARY_PATH, or you can simply use -rpath so that your programs can always find your shared libraries. I fully agree; I've already said that in this case -rpath is perfectly justified (and whenever possible the use of paths that are interpreted relative to the executable directory is a plus) In general, it's convenient to store the path in the executable any time a shared library is installed in a directory which the dynamic linker does not search by default. Especially if it is related to the executable. If it's a requisite (that is if you say: My program will only work if you installed libraries libsomething on your system) you should expect this library to be *installed*, that is the system setup to be changed so that the linker will find it, and then -rpath is a nuisance. Incidentally, I don't know what you mean by saying both soname and library name. There is only one name recorded for a shared library: the soname. Regards, Bernard Bernard Dautrevaux Microprocess Ingéniérie 97 bis, rue de Colombes 92400 COURBEVOIE FRANCE Tel:+33 (0) 1 47 68 80 80 Fax:+33 (0) 1 47 88 97 85 e-mail: [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: -rpath with libtool and Debian Linux
On Feb 1, 1999, Bernard Dautrevaux [EMAIL PROTECTED] wrote: In general, it's convenient to store the path in the executable any time a shared library is installed in a directory which the dynamic linker does not search by default. Especially if it is related to the executable. If it's a requisite (that is if you say: My program will only work if you installed libraries libsomething on your system) you should expect this library to be *installed*, that is the system setup to be changed so that the linker will find it, and then -rpath is a nuisance. I can't see it as a nuisance. If the library is installed at the hard-coded path, it will be used. If it is installed elsewhere, and the dynamic linker is properly configured, it will also be found. If it is not installed, or the dynamic linker is misconfigured, it will not be found (of course :-). But if an unrelated library with the same soname is installed in the hard-coded path or in some directory the dynamic linker is configured to use, the user (or the packager) is asking for trouble, and that's what he'll get. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
use -rpath /usr/lib for their programs. Just to make it clear, since I don't think this has come up yet, /usr/lib isn't the only problem -- /usr/X11R6/lib is as well (or was, at some point; I haven't looked at the upstream XFree86 Imake configuration recently, but it did use --rpath at one point in the libc5 days.) Thus the substitution approach needs to be a slightly more general mapping.
Re: -rpath with libtool and Debian Linux
On Jan 29, 1999, Jason Gunthorpe [EMAIL PROTECTED] wrote: Didn't we decide that all of the available alternatives that you have suggested are not a feasable solution (does this mail help make it clear why)? On 29 Jan 1999, Alexandre Oliva wrote: You may have missed the ugly one I was referring to, that I suggested in the very beginning of this discussion, that would work even for packages that were distributed with older versions of libtool: configure the packages to use a gcc or ld wrapper that removes switches such as -rpath /usr/lib from the command line then call the appropriate program. So you agree that we should be able to choose to disable rpath but you feel that we should do extra work to advoid it for libtool because it does not fit your beliefs of how shared libraries work? What about other dists that do the same thing? We are not the only linux dist to use this scheme! I agree that libtool may, some day, allow users to do that in a portable fashion. But I still don't see how modifying the current version of libtool would help you fix the problem of backward compatibility for already compiled programs or for packages distributed with older versions of libtool. Therefore, the fix you really need is not in libtool, is in the dynamic loader. This will have the extra benefit of fixing other packages that don't use libtool, but happen to specify -rpath on their own. Actually virtually every other package we would just hand edit the makefiles, libtool kinda makes that impossible. Yep. Now you have to edit a single libtool script, instead of all the Makefiles of the package. Ain't tha progress? - rpath is bad because it disables LD_LIBRARYPATH It prevents you from using LD_LIBRARY_PATH to superceed the default location of libraries, which is partially what it does Not according to the Posix specification. rpath prevents library searching and thus kills this functionality. It doesn't prevent library searching, it just takes precedence over it. If the library is not found in the rpath specified at link-time, the library is searched in other directories, such as the ones specified in LD_LIBRARY_PATH. - rpath is bad because it disables the linkers automatic versioning mechanism Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the library search path in certain circumstances? The hack is incomplete, you just have to fix it. See the other messages - it is not a hack and it is not horribly incomplete. It doesn't work for applications that have chosen to hard-code /usr/lib or /usr/lib/X11R6 in their code, for whatever reason, therefore I can only see two possible conclusions: 1) your choice to move libraries around was a bad idea, because it causes certain applications to break 2) the code in the dynamic loader that chooses the `right' version of a library is incomplete, in the sense that it doesn't choose the `right' version when shared library paths are hard-coded in the application -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Jan 29, 1999, Hamish Moffatt [EMAIL PROTECTED] wrote: On Fri, Jan 29, 1999 at 07:11:54AM -0200, Alexandre Oliva wrote: On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: Therefore, we chose to solve that particular problem (the libc5-6 transition) by moving libraries around, knowing that our linker was up to the job. It is now clear that it is not. :-( It IS, as long as you don't use rpath. And libtool works perfectly well, as long as you don't replace libraries with incompatible versions. What makes you think the maintainers of libtool should introduce potential problems and break backward compatibility just to fix a (IMHO) bad choice you have made, even knowing that modifying libtool at this point would not contribute in *any* way to fix the problem you currently have? The user is obviously free to use them for locally compiled stuff, and AFAIK it will behave as advertised. Yes, when Debian moves those libraries in the future, those programs will break. The user shouldn't really use rpath. Maybe you should ask distributors of Debian to print this advice in any CD-ROM containing Debian distributions they sell. But there are plenty of other ways for a user to hose their system; we really can't stop them doing it. That's not the point. By replacing libraries with (in)compatible versions, you're actively working to hose users' systems. Modifying libtool to remove -rpath fixes the problem at our end. Nope, because the current problem has to do with pre-installed programs. Modifying the libtool script will, by no means, fix this problem. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Jan 29, 1999, Hamish Moffatt [EMAIL PROTECTED] wrote: On Fri, Jan 29, 1999 at 07:27:28AM -0200, Alexandre Oliva wrote: Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the library search path in certain circumstances? The hack is incomplete, you just have to fix it. Have you checked our ld.so source? The only mentioned of libc5-compat is documentation. Nope, I'm just believing what the people that have complained have told me about it. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
Hi, On Fri, Jan 29, 1999 at 03:41:46PM -0600, Gordon Matzigkeit wrote: I don't understand this comment. Which trouble with --rpath do you mean? AO The exact problem the Debian developers have been complaining AO about. The more I think about the problem, the more I see that AO the problem they're facing is an incomplete hack of ld.so, in AO that it modifies the library search path under certain AO circumstances, but not in all circumstances that would need it. Exactly. Mmh. I think I see the point now, and I have to agree. So, the problem we are facing is twofold: * How can Debian hack around the rpath problem, so user can use third party software which is libc5+rpath. * Is there a better way to do a library transition? I think it is very obvious, that the only correct behaviour is to change the library/soname of all involeved libraries when doing a transition. So we had to move either from libfoo.2 to libfoog.2, libfoo.libc6.2, libc6/libfoo.2 or whatever, until a new library with a new, incompatible, soname is released. Changing the name does not look correct, so we had to change soname or path. I am sorry if this sounds quite confusing, but I hope you get the idea (Gordon, this is exactly what we discussed on the Hurd list, right?). Fortunately, libc6 will use symbol versioning, so we will not experience problems at this scale again (hopefully). Furthermore, it would be great if upstream author, compiler and installer could all influence the linking conditions (rpath or not etc), which seems to be a good goal. This could fix our problem, too, but only as a hack. The theoretical solution is outlined above. There is still the issue of xaw wrappers, can you please comment on this? Shouldn't there be a way to override rpath? Currently,LD_LIBRARY_PATH does override rpath, right? But then, if I want to link a program which was compiled for xaw with xaw3d, to get a better look+feel, I would need a way to override the rpath inside the binary. Maybe another environment variable is needed for this, or should the priority changed? Because, I really think the sysadmin/user should always have the last word on this issue. Currently, rpath overrides everything, which is set by the distributor of the binary. This means, the package can provide a default, which can be overridden at compilation time. Finally, the installer can override both. I should add here that ideally, a user/sysadmin should always be able to override all three. AO That's exactly what I'm looking for. But I wouldn't like to AO install a quick hack now that would later reveal to be a step in AO the wrong direction. That's why I'm being so conservative about AO all this issue. For the record, Alexandre's conservativeness is well-justified. I still don't see if libtools default, rpath, is correct, but I see now what Debian did wrong. I also see that if Debian would have done it correctly, the use of rpath would be a non-issue. The best solution I can come up with is to *always* change a library's soname when its dependencies change. Ah, here you say it what I cludged together above with my limited understanding of the issues :) Thanks, Marcus -- Rhubarb is no Egyptian god.Debian GNU/Linuxfinger brinkmd@ Marcus Brinkmann http://www.debian.orgmaster.debian.org [EMAIL PROTECTED]for public PGP Key http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Re: -rpath with libtool and Debian Linux
On Jan 29, 1999, Steve Dunham [EMAIL PROTECTED] wrote: Maybe we should just agree that libtool is broken, that it won't be fixed upstream, and just fix the Debian version? This would mean that we would have to rerun autoconf co when we build packages Actually, you'd just have to modify the libtool script, after it is generated, so as to set hardcode_libdir_flag_spec to the empty string. The attached script should do it; just run it after configure in any Debian package and you're done. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil /home/msc/oliva/norpath.sh Description: Bourne shell script
Re: -rpath with libtool and Debian Linux
On Sat, Jan 30, 1999 at 07:46:21PM -0200, Alexandre Oliva wrote: On Jan 29, 1999, Jason Gunthorpe [EMAIL PROTECTED] wrote: rpath prevents library searching and thus kills this functionality. It doesn't prevent library searching, it just takes precedence over it. If the library is not found in the rpath specified at link-time, the library is searched in other directories, such as the ones specified in LD_LIBRARY_PATH. I think a way to override even rpath would be great. It doesn't work for applications that have chosen to hard-code /usr/lib or /usr/lib/X11R6 in their code, for whatever reason, therefore I can only see two possible conclusions: 1) your choice to move libraries around was a bad idea, because it causes certain applications to break 2) the code in the dynamic loader that chooses the `right' version of a library is incomplete, in the sense that it doesn't choose the `right' version when shared library paths are hard-coded in the application Why should the application choose to hard code the PATH in the binary? AFAICS, there is no apparent reason for it. What has the path to do with the library? I think the only thing that should be hard coded is the exact soname and library name. Maybe I am missing something? Thanks, Marcus -- Rhubarb is no Egyptian god.Debian GNU/Linuxfinger brinkmd@ Marcus Brinkmann http://www.debian.orgmaster.debian.org [EMAIL PROTECTED]for public PGP Key http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Re: -rpath with libtool and Debian Linux
Date: Sat, 30 Jan 1999 23:30:43 +0100 From: Marcus Brinkmann [EMAIL PROTECTED] Why should the application choose to hard code the PATH in the binary? AFAICS, there is no apparent reason for it. What has the path to do with the library? I think the only thing that should be hard coded is the exact soname and library name. Maybe I am missing something? Suppose you have your own set of shared libraries, in your own directory. Suppose you want to let other people use your programs linked against your own shared libraries. You can tell everyone who uses your programs to set LD_LIBRARY_PATH, or you can simply use -rpath so that your programs can always find your shared libraries. In general, it's convenient to store the path in the executable any time a shared library is installed in a directory which the dynamic linker does not search by default. Incidentally, I don't know what you mean by saying both soname and library name. There is only one name recorded for a shared library: the soname. Ian
Re: -rpath with libtool and Debian Linux
On Sat, Jan 30, 1999 at 05:40:24PM -0500, Ian Lance Taylor wrote: Suppose you have your own set of shared libraries, in your own directory. Suppose you want to let other people use your programs linked against your own shared libraries. You can tell everyone who uses your programs to set LD_LIBRARY_PATH, or you can simply use -rpath so that your programs can always find your shared libraries. Okay. In general, it's convenient to store the path in the executable any time a shared library is installed in a directory which the dynamic linker does not search by default. Yes, I should have narrowed my question to system libraries. Unfortunately, system libraries are most likely to cause the problems, for example if people hard code /usr/X11R6/lib/ for xaw library and you want to use xaw3d... Incidentally, I don't know what you mean by saying both soname and library name. There is only one name recorded for a shared library: the soname. Ignorance I think. I thought soname is only the number, and a lib is stored in libfoo.x.y.z, where foo is the library name and x.y.z the soname. If I got it wrong, I apologize. Marcus -- Rhubarb is no Egyptian god.Debian GNU/Linuxfinger brinkmd@ Marcus Brinkmann http://www.debian.orgmaster.debian.org [EMAIL PROTECTED]for public PGP Key http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Re: -rpath with libtool and Debian Linux
Date: Sat, 30 Jan 1999 23:10:26 +0100 From: Marcus Brinkmann [EMAIL PROTECTED] * Is there a better way to do a library transition? I think it is very obvious, that the only correct behaviour is to change the library/soname of all involeved libraries when doing a transition. So we had to move either from libfoo.2 to libfoog.2, libfoo.libc6.2, libc6/libfoo.2 or whatever, until a new library with a new, incompatible, soname is released. Changing the name does not look correct, so we had to change soname or path. When you make an incompatible change to a shared library, change the soname. That's just a matter of choosing a name for the new library and tweaking a symlink. There is no reason to do anything else. What do you mean when you say ``changing the name does not look correct?'' Don't confuse the name of the library found by the runtime linker (libc.so) with the soname used by the dynamic linker (libc.so.6). The runtime linker will normally find a symlink to the soname. When you build a shared library, use the -h/--soname option to set the soname of the shared library. Shouldn't there be a way to override rpath? Currently,LD_LIBRARY_PATH does override rpath, right? No, LD_LIBRARY_PATH does not override rpath. The rpath is searched first, and then the LD_LIBRARY_PATH is searched. I think you may have agreed with that later in your message. Ian
Re: -rpath with libtool and Debian Linux
On Sat, Jan 30, 1999 at 05:49:39PM -0500, Ian Lance Taylor wrote: * Is there a better way to do a library transition? I think it is very obvious, that the only correct behaviour is to change the library/soname of all involeved libraries when doing a transition. So we had to move either from libfoo.2 to libfoog.2, libfoo.libc6.2, libc6/libfoo.2 or whatever, until a new library with a new, incompatible, soname is released. Changing the name does not look correct, so we had to change soname or path. When you make an incompatible change to a shared library, change the soname. That's just a matter of choosing a name for the new library and tweaking a symlink. There is no reason to do anything else. Yes, this is what I meant. Debian should have changed the sonames of the libc6 libraries when the library exists for libc5, too. What do you mean when you say ``changing the name does not look correct?'' Well, you _could_ rename the library, and recompile applications using the new name... for a transition period... OTOH I realized that this would be very ugly and require changes to Makefiles etc... so it does not look correct == is a stupid and brain dead idea. Shouldn't there be a way to override rpath? Currently,LD_LIBRARY_PATH does override rpath, right? No, LD_LIBRARY_PATH does not override rpath. The rpath is searched first, and then the LD_LIBRARY_PATH is searched. I think you may have agreed with that later in your message. Sorry about the typo. I meant to say does not override rpath. Thanks, Marcus -- Rhubarb is no Egyptian god.Debian GNU/Linuxfinger brinkmd@ Marcus Brinkmann http://www.debian.orgmaster.debian.org [EMAIL PROTECTED]for public PGP Key http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Re: -rpath with libtool and Debian Linux
On Sat, Jan 30, 1999 at 05:49:39PM -0500, Ian Lance Taylor wrote: Shouldn't there be a way to override rpath? Currently,LD_LIBRARY_PATH does override rpath, right? No, LD_LIBRARY_PATH does not override rpath. The rpath is searched first, and then the LD_LIBRARY_PATH is searched. I think you may have agreed with that later in your message. This is another irksome thing about libtool and -rpath. Test programs, even if they are marked noinst, use -rpath, and when they are run using the installed version instead of the newly built one. Which is annoying, since the whole point of test programs is to make sure the library is working *before* you install it. So maybe we should have an explicit -no-rpath switch to libtool to fix this. -Yosh
Re: -rpath with libtool and Debian Linux
Date: Sat, 30 Jan 1999 23:42:32 +0100 From: Marcus Brinkmann [EMAIL PROTECTED] In general, it's convenient to store the path in the executable any time a shared library is installed in a directory which the dynamic linker does not search by default. Yes, I should have narrowed my question to system libraries. Unfortunately, system libraries are most likely to cause the problems, for example if people hard code /usr/X11R6/lib/ for xaw library and you want to use xaw3d... It's hard to distinguish system libraries from non-system libraries, except by the distinction quoted above: a non-system library is a library installed in a directory which the dynamic linker does not search by default. Unfortunately, the GNU/Linux dynamic linker reportedly uses a rather complex algorithm, in which it makes decisions based on the libc version number which libraries are linked against, which would seem to make it hard to determine just which directories the dynamic linker searches by default. In the normal case I think one can assume that the dynamic linker will search any directory listed in /etc/ld.so.conf, and it would be OK to omit a -rpath argument for any shared library installed in one of the directories listed in that file. Note that although you can set up a case in which the xaw library is found in /usr/X11R6/lib, it's not normal. Normally the program will be linked against libxaw.so.N, and will have a specified search path, the rpath, to find that file. Incidentally, I don't know what you mean by saying both soname and library name. There is only one name recorded for a shared library: the soname. Ignorance I think. I thought soname is only the number, and a lib is stored in libfoo.x.y.z, where foo is the library name and x.y.z the soname. If I got it wrong, I apologize. When I, and I think most others, use the word soname, I refer to the DT_SONAME tag in a shared library which appears in a DT_NEEDED tag in the executable. The soname is set in a shared library using the -h/--soname option, and is copied into the executable by the program linker. In this case the soname is the full name of the file: libfoo.x.y.z. Ian
Re: -rpath with libtool and Debian Linux
On Jan 30, 1999, Ian Lance Taylor [EMAIL PROTECTED] wrote: In the normal case I think one can assume that the dynamic linker will search any directory listed in /etc/ld.so.conf, and it would be OK to omit a -rpath argument for any shared library installed in one of the directories listed in that file. This seems to be a reasonable assumption, as long as no directory is ever removed from /etc/ld.so.conf. But then, there's also the problem of finding the wrong version of a library, if it is found before the correct one in the search path. Anyway, as Thomas Tanner argued, hard-coding rpath won't always solve this problem either (although it would solve it in some cases), so I'd welcome a patch that would cause libtool to: 1) use information from /etc/ld.so.conf, instead of having a hard-coded list of directories, to set sys_lib_search_path_spec 2) refrain from hard-coding directories listed in sys_lib_search_path_spec (but not in $shlibpath_var) in executables -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
Marcus Brinkmann [EMAIL PROTECTED] writes: On Fri, Jan 29, 1999 at 03:41:46PM -0600, Gordon Matzigkeit wrote: I don't understand this comment. Which trouble with --rpath do you mean? AO The exact problem the Debian developers have been complaining AO about. The more I think about the problem, the more I see that AO the problem they're facing is an incomplete hack of ld.so, in AO that it modifies the library search path under certain AO circumstances, but not in all circumstances that would need it. Exactly. Mmh. I think I see the point now, and I have to agree. So, the problem we are facing is twofold: * How can Debian hack around the rpath problem, so user can use third party software which is libc5+rpath. * Is there a better way to do a library transition? I think it is very obvious, that the only correct behaviour is to change the library/soname of all involeved libraries when doing a transition. So we had to move either from libfoo.2 to libfoog.2, libfoo.libc6.2, libc6/libfoo.2 or whatever, until a new library with a new, incompatible, soname is released. Changing the name does not look correct, so we had to change soname or path. Caveat: I'm a novice at these issues. IMHO (with 20/20 hindsight), it would have been much nicer if the libc5-libc6 transition had used a different mechanism -- something akin to $_RLD_ROOT of IRIX. The idea of _RLD_ROOT is that if that env. variable is set, it is prepended to every runpath in the binary + every default path. E.g. if _RLD_ROOT=/mnt1:/mnt2:, and you have a binary with -rpath set to /usr/foo/lib and, the default library search path is /lib:/usr/lib. The set of paths searched by the linker are, in order: /mnt1/usr/foo/lib /mnt2/usr/foo/lib /usr/foo/lib $LD_LIBRARY_PATH /mnt1/lib /mnt2/lib /lib /mnt1/usr/lib /mnt2/usr/lib /usr/lib (I may have got the specifics wrong, but this should be the general idea.) The better way for libc5/6 hack would have been to modify ld-linux.so.1 (the libc5 ELF loader) to act _as if_ the _RLD_ROOT env. var. was set to `/usr/compat-glibc1:'. This way, the libc5 libraries could have been moved into to the /usr/compat-glibc1 tree maintaining the same tree structure, and would automatically have worked with any `-rpath's. E.g., xlib6 (the libc5 compatible X library) could have put its libraries in /usr/compat-glibc1/usr/X11R6/lib (if it originally put it in `/usr/X11R6/lib') and libc5 could have put its library either in /lib or in /usr/compat-glibc1/lib. Moving from `bo' to `hamm' for libc5 libraries could just have been a matter of dpkg-repack or some similar tool. Of course, I'm not conversant with all the issues, and could be completely wrong. - Hari -- Raja R Harinath -- [EMAIL PROTECTED] When all else fails, read the instructions. -- Cahn's Axiom Our policy is, when in doubt, do the right thing. -- Roy L Ash
Re: -rpath with libtool and Debian Linux
On Jan 30, 1999, Manish Singh [EMAIL PROTECTED] wrote: No, LD_LIBRARY_PATH does not override rpath. The rpath is searched first, and then the LD_LIBRARY_PATH is searched. I think you may have agreed with that later in your message. This is another irksome thing about libtool and -rpath. Test programs, even if they are marked noinst, use -rpath, and when they are run using the installed version instead of the newly built one. Which is annoying, since the whole point of test programs is to make sure the library is working *before* you install it. Yep, this is a known bug in libtool, and a partial fix for it, by Edouard Parmelan, is already installed in the CVS tree. Hopefully, someone will soon be able to complete his work, based on the short description of what is missing he recently provided. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Marcus Brinkmann [EMAIL PROTECTED] wrote: On Wed, Jan 27, 1999 at 08:22:09PM -0200, Alexandre Oliva wrote: 3) I don't want to regret having introduced a flag that caused as much or more trouble than -rpath; and I don't understand this comment. Which trouble with --rpath do you mean? The exact problem the Debian developers have been complaining about. The more I think about the problem, the more I see that the problem they're facing is an incomplete hack of ld.so, in that it modifies the library search path under certain circumstances, but not in all circumstances that would need it. I.e., if ld.so would replace /usr/lib with /usr/lib/libc5:/usr/lib whenever it found that the application was linked with libc5, even if /usr/lib had been hard-coded in the program's rpath, everything would be working beautifully. The fact that libtool always hard-codes /usr/lib in a program that is linked with a libtool library that is (going to be) installed in /usr/lib is a side issue, because users may do it on their own, and, IMHO, they're not to be blamed because of doing that. Until now, I only heard from you that rpath is the ideal solution. Maybe you haven't read (or even received) one of my messages in the beginning of this discussion, in which I stated that even rpath is wrong in certain circumstances. For example, if a program is linked with libfoo, that lives in /foo, and libbar, that lives in /bar, but there's a compatible (WRT to version numbers, not necessarily WRT version of libc) version of libbar in /foo, the linker will pick the one in /foo, not the one in libbar. In fact, Thomas Tanner's suggestion of dropping -rpath when it would hard-code a standard library just makes this problem more apparent: if a (l)user installs a library in /usr/local/lib, but there's a library with the same name in /usr/lib, the version in /usr/lib will be used at run-time. This may have very bad consequences, such as the never-ending problem of (l)users installing gcc 2.[78].* in /usr and egcs 1.0.* in /usr/local, each one with its particular (and incompatible) version of libstdc++ (and libg++, for some). The only way to avoid this potential problem is to hard-code the full library path in the soname of the library itself, but then, this is not portable, and it is not desirable because then you can't move a library at all (read it again: I'm not talking about replacing a library with an apparently compatible but actually incompatible version of it :-) No rpath makes it harder for a user to determine exactly which system libraries he links with. (With rpath, though, it only works when the system administrator never changes the library file at this place, too). It is not a problem to move a library, as long as it can be found in other hard-coded rpath, in the default search path or in LD_LIBRARY_PATH. A problem only arises if an apparently compatible but actually incompatible library is found first, which is exactly the problem that the Debian developers are facing. 4) I have already suggested a (dirty and ugly, I admit) hack that is sufficient for your needs of not using -rpath at all in Debian packages. We can find our own hacks (and do so since a long time). Now we are interested in a compatible, portable and general solution. As the libtool maintainer, you should be the ideal person to talk about such a solution. I think we understood by now that a simple disable flag does not satisfy your high ambitions wrt to portability. Let's move on to better solutions. And, in fact, such a flag, that I said I was willing to accept, wouldn't actually help you much, because applications are distributed with their own versions of libtool, and you'd have to modify them all, or use your own hacked version of libtool to build all applications (via make LIBTOOL=/hacked/for/debian/libtool), so there'd not be much point in introducing this change only in libtool 1.3 and newer... :-( I hope you understand my position now. I should also note that I myself have already wanted flags such as -hardcode-libdir for hardcoding the full pathname of a shared library into itself (it's mentioned in libtool/TODO) and -no-implicit-rpath (which is what you happen to be asking for), but I have some trouble deciding who should be responsible for deciding which flags to use for which libraries and programs. Maybe you should not be the one to decide at all? I'm certainly not. Offer flexible solutions, where the last person can override the (possible) default given by others. That's the hard part. Overriding may have to occur in a per-library and/or per-program basis, so I haven't been able to come up with a satisfactory solution. This means, the package can provide a default, which can be overridden at compilation time. Finally, the installer can override both. That's exactly what I'm looking for. But I wouldn't like to install a quick hack now that would later reveal to be a step in the
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: 1) it would be hard to make it behave correctly in a portable way (and libtool would be useless if it were not for being portable); Special case-it for linux, if you will. Libtool has plenty of special cases as it is. Not in the interface it presents to its users. Internally, it's obviously full of special cases to support all the crazyness of OS developers and their wonderful dynamic linkers. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: Therefore, we chose to solve that particular problem (the libc5-6 transition) by moving libraries around, knowing that our linker was up to the job. It is now clear that it is not. :-( rpath is broken. You said as much yourself. rpath is broken because it *overrides* all other sorts of library searching. But isn't overriding library searching exactly what the ld.so hack is doing? Why is one of those blessed for doing that, while the other is crucified for guilt of all the sins of the world? :-) However, our dynamic linker *does* understand the problem. And it *does* have a solution to it. And -rpath turns it off. So we cannot afford to use -rpath. As I have already pointed out, the solution is not complete, otherwise you'd not be observing any problem. What do you mean the solution is not complete? It does not replace /usr/lib with /usr/lib/libc5:/usr/lib in the rpath. That's what is causing you trouble, not the fact that /usr/lib is hard-coded in the program. It works. It works well. If it worked well you wouldn't be complaining about a problem that is caused by its incomplete behavior, but that could be avoided by modifying other pieces of software that are doing their jobs correctly. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Ulrich Drepper [EMAIL PROTECTED] wrote: Jules Bean [EMAIL PROTECTED] writes: rpath is broken. You said as much yourself. rpath is broken because it *overrides* all other sorts of library searching. I think people here do not know about $ORIGIN. This allows to define relative rpaths. E.g., a package with a program foo and a library libbar.so where foo is installed in $PATH/bin and libbar.so is defined in $PATH/lib should use -rpath \$ORIGIN/../lib The $ORIGIN is defined relative to the location of the object containing the reference. This is a great feature, but I think it is hardly usable by libtool, since, in order to use it, libtool would have to know at program link time where the program is going to be installed (it currently doesn't need this information), and it would have to check whether the libtool libraries that the program is linked with are going to be installed in paths that are easily accessible via \$ORIGIN relative pathnames, and that no soft-linking (say, /bin - /usr/bin but not /lib - /usr/lib) is going to break it, and probably many other potential problems that I can't foresee. But I agree, it's a nice feature, and we may be able to adopt it in the future, on platforms that support it. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Jason Gunthorpe [EMAIL PROTECTED] wrote: Actually you want to know why I remember this? I used libtool a while back and I installed a copy of my program in /usr/bin and /usr/lib and wanted to us a new local copy of my libtool program. Of course libtool had used -rpath to make sure that my local binary used /usr/lib (as it was intended to be installed there someday) and then used LD_LIBRARY_PATH in the wrapper script to try to override this. Needless to say it did not work and it took me a bit of figuring to determine why my changes had no effect. Even in an all libtool environment rpath causes pain. This is a known bug in the current libtool, and we're working on a fix. I have already told you one (ugly) way to do it, but I still don't think it is a good idea in the general case. Didn't we decide that all of the available alternatives that you have suggested are not a feasable solution (does this mail help make it clear why)? You may have missed the ugly one I was referring to, that I suggested in the very beginning of this discussion, that would work even for packages that were distributed with older versions of libtool: configure the packages to use a gcc or ld wrapper that removes switches such as -rpath /usr/lib from the command line then call the appropriate program. This will have the extra benefit of fixing other packages that don't use libtool, but happen to specify -rpath on their own. - rpath is good because it allows a binary to have a shared library in a non-standard place without requiring the user to use LD_LIBRARYPATH or the sysadmin to add that place to the search path - rpath is bad because it disables LD_LIBRARYPATH It does not disable it, it just precedes LD_LIBRARY_PATH. AFAIK, the purpose of LD_LIBRARY_PATH is to help a program find a library that was moved, and it does fulfil this purpose as long as you don't install another (in)compatible library in place of the moved library. - rpath is bad because it disables the linkers automatic versioning mechanism Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the library search path in certain circumstances? The hack is incomplete, you just have to fix it. - rpath is bad because it prevents you from moving shared libraries around freely. It does not. It just prevents you from arbitrarily replacing a library with an (in)compatible version of it. Since you shouldn't do that, libtool is not the piece of software to be blamed for using -rpath. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Jan 28, 1999, Bernard Dautrevaux [EMAIL PROTECTED] wrote: You say the contract is I want to find THERE the lib that does THIS.x and THAT.x; I think (and that's at least true for Linux) the contract the compiler and linker has signed was twofold; it says: 1) I will give you the library that makes THIS.x and THAT.x 2) The prefered library I want you to use to obtain THIS and THAT is there and makes THIS.x and THAT.x Now you trick it with -rpath in finding both the .x and THERE and all the problem comes from there... An analogy: When I wand to get hot water in restrooms, I just look at the tap, and turn the red one; I do not INSIST on opening the left one, risking to get cold water... Good analogy. What's happening here is that Debian is placing the red lable on the cold water tap. I.e., they're replacing a library with an incompatible version of it, and getting in trouble because some programs are now getting cold water where they expected hot water. Under Linux at least the incompatibilities we are talking here ARE managed by the dynamic linker, IFF we do not trick it saying him (using -rpath) Do not be smart, just load the library from there. YOU are breaking the Linux contract... ld.so is trying to outsmart everybody, but it is not smart enough to do it. When you moved libc5-compatible libraries from /usr/lib to /usr/lib/libc5, you established a rule that, if any program was linked with libc5, it should look for libraries in /usr/lib/libc5 first, right? Then why doesn't ld.so follow this rule, by replacing /usr/lib with /usr/lib/libc5 when it finds this directory in the rpath too? -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
Alexandre Oliva wrote: ld.so is trying to outsmart everybody, but it is not smart enough to do it. When you moved libc5-compatible libraries from /usr/lib to /usr/lib/libc5, you established a rule that, if any program was linked with libc5, it should look for libraries in /usr/lib/libc5 first, right? Then why doesn't ld.so follow this rule, by replacing /usr/lib with /usr/lib/libc5 when it finds this directory in the rpath too? No, that's not how it works. To the best of my understanding, it works by adding a libc5 or libc6 field to its cache. When it looks for a cached library, and it finds two entries, it picks the one with the correct libc. It always searches all of its directories. It allows -rpath and LD_LIBRARY_PATH to override this behaviour. I think that that is correct -- these _are_ overrides. They're to be used when the default behaviour gets things wrong. I think the dynamic linker could be further changed to always ignore a library that would introduce a mixed libc5/libc6 linkage. That would give the correct behaviour even with these overrides. However, that only solves the _previous_ problem, not any future ones. A general solution would require that soname be split into a library name and a major version, so that the dynamic linker can detect incompatible versions of the same library. That would be a major change. Richard Braakman
Re: -rpath with libtool and Debian Linux
On Jan 29, 1999, Richard Braakman [EMAIL PROTECTED] wrote: Alexandre Oliva wrote: ld.so is trying to outsmart everybody, but it is not smart enough to do it. When you moved libc5-compatible libraries from /usr/lib to /usr/lib/libc5, you established a rule that, if any program was linked with libc5, it should look for libraries in /usr/lib/libc5 first, right? Then why doesn't ld.so follow this rule, by replacing /usr/lib with /usr/lib/libc5 when it finds this directory in the rpath too? No, that's not how it works. To the best of my understanding, it works by adding a libc5 or libc6 field to its cache. When it looks for a cached library, and it finds two entries, it picks the one with the correct libc. It always searches all of its directories. It allows -rpath and LD_LIBRARY_PATH to override this behaviour. I think that that is correct -- these _are_ overrides. They're to be used when the default behaviour gets things wrong. Since -rpath is specified at program link time, and libc5 is supposed to be used only by old programs, it would be correct for ld.so to use /usr/lib/libc5 instead of /usr/lib if it finds that the program was linked with libc5. This would make the transition as transparent as possible, even for users that, inadvertently or not, have decided to use -rpath /usr/lib for their programs. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Fri, Jan 29, 1999 at 07:11:54AM -0200, Alexandre Oliva wrote: On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: Therefore, we chose to solve that particular problem (the libc5-6 transition) by moving libraries around, knowing that our linker was up to the job. It is now clear that it is not. :-( It IS, as long as you don't use rpath. We don't use rpath for vendor-supplied parts of the system. The user is obviously free to use them for locally compiled stuff, and AFAIK it will behave as advertised. Yes, when Debian moves those libraries in the future, those programs will break. The user shouldn't really use rpath. But there are plenty of other ways for a user to hose their system; we really can't stop them doing it. If it worked well you wouldn't be complaining about a problem that is caused by its incomplete behavior, but that could be avoided by modifying other pieces of software that are doing their jobs correctly. Modifying libtool to remove -rpath fixes the problem at our end. The documentation for our package checker (lintian) includes two ways to do this easily. Hamish -- Hamish Moffatt VK3TYD [EMAIL PROTECTED], [EMAIL PROTECTED] Latest Debian packages at ftp://ftp.rising.com.au/pub/hamish. PGP#EFA6B9D5 CCs of replies from mailing lists are welcome. http://hamish.home.ml.org
Re: -rpath with libtool and Debian Linux
On Fri, Jan 29, 1999 at 07:27:28AM -0200, Alexandre Oliva wrote: Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the library search path in certain circumstances? The hack is incomplete, you just have to fix it. Have you checked our ld.so source? The only mentioned of libc5-compat is documentation. Hamish -- Hamish Moffatt VK3TYD [EMAIL PROTECTED], [EMAIL PROTECTED] Latest Debian packages at ftp://ftp.rising.com.au/pub/hamish. PGP#EFA6B9D5 CCs of replies from mailing lists are welcome. http://hamish.home.ml.org
Re: -rpath with libtool and Debian Linux
Previously Alexandre Oliva wrote: Good analogy. What's happening here is that Debian is placing the red lable on the cold water tap. I.e., they're replacing a library with an incompatible version of it, and getting in trouble because some programs are now getting cold water where they expected hot water. No, what Debian (and RH and probably other distributions) do is exchanging the two taps, there is no messing with labels involved since the dynamic linker needs those to determine which library to use. When you moved libc5-compatible libraries from /usr/lib to /usr/lib/libc5, you established a rule that, if any program was linked with libc5, it should look for libraries in /usr/lib/libc5 first, No, what we did was making libc6 libraries the default since the linker uses stuff in /usr/lib. The dynamic linker doesn't really care about the order, it just uses the first library that it sees for will work with the appplication (ie libc5/libc6 check and soname check). But you override the dynamic linkers' safe choice by using rpath.. Wichert. -- == This combination of bytes forms a message written to you by Wichert Akkerman. E-Mail: [EMAIL PROTECTED] WWW: http://www.wi.leidenuniv.nl/~wichert/ pgpSr9ZJxKJmZ.pgp Description: PGP signature
Re: -rpath with libtool and Debian Linux
On 29 Jan 1999, Alexandre Oliva wrote: Didn't we decide that all of the available alternatives that you have suggested are not a feasable solution (does this mail help make it clear why)? You may have missed the ugly one I was referring to, that I suggested in the very beginning of this discussion, that would work even for packages that were distributed with older versions of libtool: configure the packages to use a gcc or ld wrapper that removes switches such as -rpath /usr/lib from the command line then call the appropriate program. So you agree that we should be able to choose to disable rpath but you feel that we should do extra work to advoid it for libtool because it does not fit your beliefs of how shared libraries work? What about other dists that do the same thing? We are not the only linux dist to use this scheme! This will have the extra benefit of fixing other packages that don't use libtool, but happen to specify -rpath on their own. Actually virtually every other package we would just hand edit the makefiles, libtool kinda makes that impossible. - rpath is good because it allows a biary to have a shared library in a non-standard place without requiring the user to use LD_LIBRARYPATH or the sysadmin to add that place to the search path - rpath is bad because it disables LD_LIBRARYPATH It does not disable it, it just precedes LD_LIBRARY_PATH. AFAIK, the purpose of LD_LIBRARY_PATH is to help a program find a library that was moved, and it does fulfil this purpose as long as you don't install another (in)compatible library in place of the moved library. It prevents you from using LD_LIBRARY_PATH to superceed the default location of libraries, which is partially what it does - rpath prevents library searching and thus kills this functionality. - rpath is bad because it disables the linkers automatic versioning mechanism Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the library search path in certain circumstances? The hack is incomplete, you just have to fix it. See the other messages - it is not a hack and it is not horribly incomplete. Jason
Re: -rpath with libtool and Debian Linux
Hi! [Creaaak... Gordon pops out of the grave reserved for former libtool maintainers to make some comments.] Alexandre Oliva writes: I don't understand this comment. Which trouble with --rpath do you mean? AO The exact problem the Debian developers have been complaining AO about. The more I think about the problem, the more I see that AO the problem they're facing is an incomplete hack of ld.so, in AO that it modifies the library search path under certain AO circumstances, but not in all circumstances that would need it. Exactly. This means, the package can provide a default, which can be overridden at compilation time. Finally, the installer can override both. AO That's exactly what I'm looking for. But I wouldn't like to AO install a quick hack now that would later reveal to be a step in AO the wrong direction. That's why I'm being so conservative about AO all this issue. For the record, Alexandre's conservativeness is well-justified. Debian maintainers should feel free to patch individual Debian packages, but fixing this problem upstream is a lot harder than it appears at first glance. The best solution I can come up with is to *always* change a library's soname when its dependencies change. I believe it was Joel Klecker who mentioned something about `libapi' patches for egcs that were supposed to implement this automatically. Joel, can you comment on this (or somebody else who knows the details)? -- Gordon Matzigkeit [EMAIL PROTECTED] //\ I'm a FIG (http://www.fig.org/) Lovers of freedom, unite! \// I use GNU (http://www.gnu.org/) [Unfortunately, www.fig.org is broken. Please stay tuned for details.]
Re: -rpath with libtool and Debian Linux
Hamish Moffatt [EMAIL PROTECTED] writes: On Fri, Jan 29, 1999 at 07:27:28AM -0200, Alexandre Oliva wrote: Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the library search path in certain circumstances? The hack is incomplete, you just have to fix it. Have you checked our ld.so source? The only mentioned of libc5-compat is documentation. It's a magic hack. Somehow (according to Alexandre) it magically adds /usr/lib/libc5 to the path, even though libc5 occurs nowhere in the binary. (go figure.) Maybe we should just agree that libtool is broken, that it won't be fixed upstream, and just fix the Debian version? This would mean that we would have to rerun autoconf co when we build packages - but it beats arguing with this guy till the end of time. Steve [EMAIL PROTECTED]
Re: -rpath with libtool and Debian Linux
At 15:41 -0600 1999-01-29, Gordon Matzigkeit wrote: The best solution I can come up with is to *always* change a library's soname when its dependencies change. I believe it was Joel Klecker who mentioned something about `libapi' patches for egcs that were supposed to implement this automatically. Joel, can you comment on this (or somebody else who knows the details)? That patch merely applies to the soname of the libstdc++ library that is part of egcs, it imparts no other functionality. -- Joel Klecker (aka Espy) URL:http://web.espy.org/ URL:mailto:[EMAIL PROTECTED] URL:mailto:[EMAIL PROTECTED] Debian GNU/Linux PowerPC -- URL:http://www.debian.org/ports/powerpc/
Re: -rpath with libtool and Debian Linux
On 27 Jan 1999, Alexandre Oliva wrote: You know, I seem to remember that there is another rather unpleasent side-effect of rpath - it basically completely disables library searching and thus disables LD_LIBRARY_PATH, once you have used rpath it is not easy for a user replace that library (for whatever reason) and I find that highly annoying. Well, he could always move the library elsewhere and put whatever he wants in its place... Some people might be consider it a bug in the dynamic linker, that could look at LD_LIBRARY_PATH before -rpath. I think -rpath really means it, so I consider the behavior correct (albeit a bit inconvenient for us, maintainers of libtool) Actually you want to know why I remember this? I used libtool a while back and I installed a copy of my program in /usr/bin and /usr/lib and wanted to us a new local copy of my libtool program. Of course libtool had used -rpath to make sure that my local binary used /usr/lib (as it was intended to be installed there someday) and then used LD_LIBRARY_PATH in the wrapper script to try to override this. Needless to say it did not work and it took me a bit of figuring to determine why my changes had no effect. Even in an all libtool environment rpath causes pain. Actually you might have something there with the search order. What if the linker were to examine it's library list before examining rpath, rpath would be like a compiled in default search path instead of a compiled in search path override. The ld-linux people probably would not go for that though. Incidently from what I've read I think that description of rpath more closely matches what you want to use it for than does it's current behavior. The linux dynamic linker will resolve things in some magical way based on the library dependencies and the program dependencies to locate the proper library in many cases - rpath does cripples, not enhances, this function. Since you do support -rpath in your system, you should probably extend your dynamic linker to work in this case too, or risk taking the blame for silently breaking applications, if the poor user ever understands what happened to his program. Well then you change the meaning of rpath, if you accept rpath means 'use this library not matter what the consequences' then it is arguable the user should expect that from rpath. This is a good point. But since all of you face this same problem, and all of you use the same major versions of libc, you can probably agree on a solution that would benefit you all. Or arrange, via Lol! There has been little luck in that area, the only way to do it would really be to convince all the upstream maintianers to do that and then force those changes on the distributions. In any event as we agree below this is not an adaquate solution. inter-package dependencies, that required libraries are installed in the proper places. No, we are talking about 3rd party binary applications such as netscape - arguably they are a compelling reason why we use the scheme we do. It seems to me that the main problem has to do with library versioning. Even though the existing library versioning mechanisms are great to describe the evolution of the API offerred by a library, they by no means describe the dependencies of a library, so we end up with libraries that have the same version numbers but that are not compatible at all. Bingo. We have a dynamic linker that can complete the second half of the puzzle if all available major versions of a library are in the search path, it will pick the one with the matching dependencies, we as a distribution vendor take steps to make sure that any compatibility libraries remain within the linkers search path. What we all are desperately looking for is a mechanism to allow the dynamic loader to pick the right version of a shared library out of a set of multiple builds of the same version of a library, the main difference between them being their dependencies. I belive we already have this capablility - the only problem is that rpath disables it. If rpath did not break the normal library searching mechanisms then nobody would care that libtool uses it. [See above] If there is more than one dependency, there is no way to ensure that the major version number is increased whenever it has to be. Or I just can't find out how :-) No, you will never be able to work this out because you have to get the upstream author to do it, assume for instance that we changed our xlib6 soname to be libX11.so.6-C6-ICE6 then anything that is linked on our system will not work on a redhat one unless they did the same and then people who compile out of the tar ball would be in trouble and so on. Permuting the soname is not a good enough solution because it cannot be applied universally and consistently. We must be able to turn it off for libraries used on our system! I have already told you one (ugly) way to do it, but I still don't think
Re: -rpath with libtool and Debian Linux
Alexandre == Alexandre Oliva [EMAIL PROTECTED] writes: Ben With Debian and Red Hat, it's totally the opposite. Moving Ben libraries around is what leads to upgrades being possible. Alexandre Then why do you find so much trouble with it? Because of -rpath. :) That's the only reason we've run into trouble with it. -- Brought to you by the letters D and P and the number 9. What's green and sits in the corner? ... A naughty frog! Debian GNU/Linux maintainer of Gimp and GTK+ -- http://www.debian.org/ I'm on FurryMUCK as Che, and EFNet/Open Projects IRC as Che_Fox.
RE: -rpath with libtool and Debian Linux
-Message d'origine- De: Ulrich Drepper [SMTP:[EMAIL PROTECTED] Date: jeudi 28 janvier 1999 00:54 À:Jules Bean Cc: Alexandre Oliva; Debian Developers; [EMAIL PROTECTED] Objet:Re: -rpath with libtool and Debian Linux Jules Bean [EMAIL PROTECTED] writes: rpath is broken. You said as much yourself. rpath is broken because it *overrides* all other sorts of library searching. I think people here do not know about $ORIGIN. This allows to define relative rpaths. E.g., a package with a program foo and a library libbar.so where foo is installed in $PATH/bin and libbar.so is defined in $PATH/lib should use -rpath \$ORIGIN/../lib The $ORIGIN is defined relative to the location of the object containing the reference. This is available in Solaris 7 (maybe 2.6?) and Linux w/ glibc 2.1. This is the perfect way of doing if the same package install a common shared library and a set of programs using it; then relative paths are OK. By this does not solve the problem of finding independently installed libraries or system ones... There -rpath will force to use the absolute path of these libraries on the development system and if installed as a binary package, these may be in slightly differing places (I'm sure there is system libs that are in /usr/lib in some Linux distribs but in /lib or /usr/local/lib in others...) AFAIK this is the subject of this whole thread about -rpath: how could we create binary distributions that WORK... (other than statically linking all executables of course...) Regards, Bernard -- Bernard Dautrevaux Microprocess Ingéniérie 97 bis, rue de Colombes 92400 COURBEVOIE FRANCE Tel:+33 (0) 1 47 68 80 80 Fax:+33 (0) 1 47 88 97 85 e-mail: [EMAIL PROTECTED] [EMAIL PROTECTED] --
RE: -rpath with libtool and Debian Linux
-Message d'origine- De: Alexandre Oliva [SMTP:[EMAIL PROTECTED] Date: mercredi 27 janvier 1999 20:53 À:J.H.M. Dassen Cc: debian-devel@lists.debian.org; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] Objet:Re: -rpath with libtool and Debian Linux On Jan 27, 1999, J.H.M. Dassen [EMAIL PROTECTED] wrote: On Wed, Jan 27, 1999 at 17:07:30 -0200, Alexandre Oliva wrote: You might have included my suggestion to prevent having to move libraries in the first place: creating a libc6-specific directory right now, instead of installing libraries in /usr/lib and having to move them into another directory when libc7 should be released. I'm sorry, but this is IMHO completely backwards. On Linux systems, there is nothing wrong with moving libraries around as the need arises. Except that you risk replacing a library with an incompatible one. That's what has caused you so much trouble. If, instead of moving the X11 libraries to another dir and creating new, incompatible versions under the same pathname, you had created new versions in other directories, no unexpected crashes would have occurred. Having libtool default to -rpath is what's causing problems. This is IMHO completely backwards :-) When a program is linked with a shared library, a contract is established between them stating that the library (or any newer but compatible version thereof, on systems that support library versioning) will supply the symbols that the program resolved from it, and the program will be able to find the library at that point. If you move the library and replace it with an incompatible one, you're breaking the contract and the versioning mechanism, so you can't blame the program for crashing, nor the tool that created the program. You mark a point here: everything is about a contract passed by you, using the compiler, to the system on which you install. The problem is with the exact contract passed, and interpretation of this contract. You say the contract is I want to find THERE the lib that does THIS.x and THAT.x; I think (and that's at least true for Linux) the contract the compiler and linker has signed was twofold; it says: 1) I will give you the library that makes THIS.x and THAT.x 2) The prefered library I want you to use to obtain THIS and THAT is there and makes THIS.x and THAT.x Now you trick it with -rpath in finding both the .x and THERE and all the problem comes from there... An analogy: When I wand to get hot water in restrooms, I just look at the tap, and turn the red one; I do not INSIST on opening the left one, risking to get cold water... I've seen one too many instances of foo crashes on Debian that turned out to be foo is a libc5 binary with an RPATH of /usr/X11R6/lib which on any reasonably up to date Debian system contains libc6 X libraries. See? You replaced one library with an incompatible one without modifying its version number to mark it as incompatible. Isn't this breaking the contract? How could you expect it to work? Moving the library was just fulfilling the 2nd part of the contract: You will find here the prefered library to use for THIS and THAT. The X example also shows that the problem isn't limited to /usr/lib either. What's next? /usr/local/lib/libc6 ? Maybe some library versioning mechanism that allows incompatible changes to be marked as such. Maybe an environment variable or some file in /etc containing a number that will be added to the major version number of any library libtool creates, so that they will be marked as incompatible. Under Linux at least the incompatibilities we are talking here ARE managed by the dynamic linker, IFF we do not trick it saying him (using -rpath) Do not be smart, just load the library from there. YOU are breaking the Linux contract... It's possible the way you are using -rpath correspond to the contract of some OSes, but surely not LINUX :-(; and if there is different contracts to be adhered to depending on the OS, it seems that exaactly why we use autoconf/automake/libtool: allow us NOT to bother about OS diferences but rather be able to ask libtool: Do what teh OS expect you to do to fulfill its contracts So my opinion, at least on Linux, would be not to use -rpath, except if we need to access shared libraries that WE install together with the executables; and then install it in some package-dependant directory where we have no risk to get ANY other lib, pass ONLY this dir to the exec using -rpath, and then all should be OK. Now for the one that want to compile and install a set of interdependant packages in it's home directory on some OS where he is not able to pass some kind of an LD_LIBRARY_PATH to the exec, then it may use some option to each compiles asking that -rpath be passed for all links, and warn him he should not try to distribute or move the resultant installed packages, and may need to reinstall them if changing
RE: -rpath with libtool and Debian Linux
-Message d'origine- De: Alexandre Oliva [SMTP:[EMAIL PROTECTED] Date: mercredi 27 janvier 1999 22:23 À:Jules Bean Cc: J.H.M. Dassen; debian-devel@lists.debian.org; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] Objet:Re: -rpath with libtool and Debian Linux On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: On 27 Jan 1999, Alexandre Oliva wrote: snipped We even support separate versions of software, to some extent, although I'd agree completely that our support in this area is not what it might be. And that's the very reason why I've never been able to adopt any of the available packaging systems: the only way to keep multiple working versions is to use a separate directory for each version of each package. Good point. I'm not either using any standard packaging system like rpm or deb, just due to this. I use a (quite primitive :-)) homegrown packaging system taht allows my users to install each realease in a different location, anywhere they want on their disk. Period. (I think difficult to be more opened...) My problem is that if in my package I create a shared library that will be used by the exec in this package, and I use libtool for this, then the use of -rpath for this cause the user to have to install exactly where I build it (that is in some highly non-standard place due to our complex development environment), or else the -rpath will point in some unexistent place and (as Linux will ignore LD_LIBRARY_PATH) I can't override it... Not to talk about having to cope with you, Linux packagers, moving system libraries from release to release :-) But I'm confident your tools are coherent and that if you move libraries, your linker will find them in the proper place. So I think -rpath may be useful INSIDE packages, IFF we use relative pathnames (that \$ORIGIN I saw in a message yesterday I think), but should only be used for standard libraries if the OS distributor advised us to, and NOT USED IF THE CONTRACT I PROPOSE US say NOT TO USE IT! The purpose of libtool is to help us build portable code, whose built rules are adapted to the build platform, isn't-it :-), so let's do it: if Linux distribution maintainers advise us not to use -rpath on their distribution, we must trust them (and blame them if the solution they recommend do not work 8-(, but then THEY have to correct their implementation! :-)) How does the current packaging system allows me to test one version of a package while other users of the same host are running a stable version of that tool? Or are the GNU/Linux distributions all moving towards the Micro$oft model of single-user workstations? Of course not ;-) Jeez!, I was sure I had added a smiley after that last sentence. Sorry about that... If you want to test one version, you simply run it with ./configure --prefix /home/me/nicepackage But isn't this exactly what the packaging systems are trying to avoid, i.e., that people have to compile systems on their own? And then, how could I make sure that my test build works exactly as the pre-compiled upgrade, so that I could use the packaging system for the update? This is something that I feel that should be taken care of by the existing GNU/Linux distributions. In fact, I've got a bunch of ideas that I'll probably never find time to implement :-( about how to maintain multiple versions of packages installed and allowing each user to select which version he wants to use, either by explicit version number or by tags such as `stable', `test', `old', etc, tags that are determined by the system manager when he installs the package. I think this is slightly off topic, or we may have to start a whole new mailing list to discuss packaging systems (I don't know but I'm quite sure there exist mailing lists or newsgroups on RPM, DEB, etc. :-)) Here we are talking about how libtool could help isolate us from the peculiarities of different OSes, and I think this is enough work :-) Regards, Bernard -- Bernard Dautrevaux Microprocess Ingéniérie 97 bis, rue de Colombes 92400 COURBEVOIE FRANCE Tel:+33 (0) 1 47 68 80 80 Fax:+33 (0) 1 47 88 97 85 e-mail: [EMAIL PROTECTED] [EMAIL PROTECTED] --
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Ben Gertzfield [EMAIL PROTECTED] wrote: This is bad for Debian, because in all binary packaging systems, shared libraries can and will be moved around from time to time, as policy and major upgrades (like libc5 - libc6) mandate. You might have included my suggestion to prevent having to move libraries in the first place: creating a libc6-specific directory right now, instead of installing libraries in /usr/lib and having to move them into another directory when libc7 should be released. However, Alexandre Oliva [EMAIL PROTECTED] brings up an important point: -rpath is necessary if one is installing libraries and binaries linked to those libraries in one's home directory, or if your Unix has no support for library search paths via environment variables like Linux's LD_LIBRARY_PATH. More than that (and it was my fault to have failed to mention that before): libtool will hard-code the installation directory of the library into the `libdir' variable of the .la script it installs. Therefore, if one moves the library then tries to link with the .la file, he loses. There's also the dlopening issue: libltdl (to be released with libtool 1.3) will dlopen a library in the directory pointed to by `libdir' too. In general, I feel that moving libraries around is a very bad idea, because it will lead to failure most of the times, and that's why I don't feel libtool should help people doing that. Basically, I have been asking Alexandre if it's possible to add a --no-rpath option to libtool when calling it to tell it to not use -rpath when linking binaries, but he refused, saying he'd have to port that to 'hundreds of platforms'. Actually, not issuing -rpath or equivalent is quite easy to do, but choosing *when* not to do it is the part that is hard to port. Thomas Tanner has suggested that we could omit the -rpath switch for libraries that are supposed to be installed in directories listed in the default search path. However, the default search PATH might change, and, even if it did not, it would be possible to link an application with a library in, say, /usr/local/lib, and then find out that at run-time it loads a library with the same name in /usr/lib, because /usr/lib appears first in the standard search path. The other issue is *how* to specify that -rpath should be ommitted. Should it be a configure option, that would permanently disable any -rpath switches? Or should it be an additional argument to the libtool script at program-linking time? Or should it be specified at library linking time, with one of three options: hardcode path in the library, hardcode path in any program linked with it, or do not hardcode path at all? Or something else? What to do on a platform that doesn't support the requested hardcoding strategy? The issue is very complex because we can't think just of GNU/Linux with all its bells and whistles, because libtool is supposed to present an homogeneous, portable interface to creating libraries. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Wed, Jan 27, 1999 at 17:07:30 -0200, Alexandre Oliva wrote: You might have included my suggestion to prevent having to move libraries in the first place: creating a libc6-specific directory right now, instead of installing libraries in /usr/lib and having to move them into another directory when libc7 should be released. I'm sorry, but this is IMHO completely backwards. On Linux systems, there is nothing wrong with moving libraries around as the need arises. Having libtool default to -rpath is what's causing problems. I've seen one too many instances of foo crashes on Debian that turned out to be foo is a libc5 binary with an RPATH of /usr/X11R6/lib which on any reasonably up to date Debian system contains libc6 X libraries. The X example also shows that the problem isn't limited to /usr/lib either. What's next? /usr/local/lib/libc6 ? However, Alexandre Oliva [EMAIL PROTECTED] brings up an important point: -rpath is necessary if one is installing libraries and binaries linked to those libraries in one's home directory, That is a special case. The default should be sane for regular cases. or if your Unix has no support for library search paths via environment variables like Linux's LD_LIBRARY_PATH. While I appreciate concerns about supporting less fortunate operating environments, I don't think their existence should hold us back from doing the right thing on Linux. In general, I feel that moving libraries around is a very bad idea, because it will lead to failure most of the times, and that's why I don't feel libtool should help people doing that. I see no reason why moving libraries around is a bad idea. I see defaulting to -rpath as a bad idea, which breaks moving libraries around. Ray -- POPULATION EXPLOSION Unique in human experience, an event which happened yesterday but which everyone swears won't happen until tomorrow. - The Hipcrime Vocab by Chad C. Mulligan
Re: -rpath with libtool and Debian Linux
Alexandre == Alexandre Oliva [EMAIL PROTECTED] writes: Alexandre You might have included my suggestion to prevent having Alexandre to move libraries in the first place: creating a Alexandre libc6-specific directory right now, instead of Alexandre installing libraries in /usr/lib and having to move Alexandre them into another directory when libc7 should be Alexandre released. This is rather frustrating, because then we will need to make a /lib/libc6, /usr/lib/libc6, /usr/X11R6/libc6, /usr/local/lib/libc6.. it never ends. :) Alexandre More than that (and it was my fault to have failed to Alexandre mention that before): libtool will hard-code the Alexandre installation directory of the library into the `libdir' Alexandre variable of the .la script it installs. Therefore, if Alexandre one moves the library then tries to link with the .la Alexandre file, he loses. There's also the dlopening issue: Alexandre libltdl (to be released with libtool 1.3) will dlopen a Alexandre library in the directory pointed to by `libdir' too. I've never understood what the .la scripts are for. Why are they installed into /usr/lib/, where libraries live? This is kind of off-the-subject, but they have always confused me, and I delete them out of any libtool-using library package I maintain. Alexandre In general, I feel that moving libraries around is a Alexandre very bad idea, because it will lead to failure most of Alexandre the times, and that's why I don't feel libtool should Alexandre help people doing that. With Debian and Red Hat, it's totally the opposite. Moving libraries around is what leads to upgrades being possible. Alexandre The issue is very complex because we can't think just Alexandre of GNU/Linux with all its bells and whistles, because Alexandre libtool is supposed to present an homogeneous, portable Alexandre interface to creating libraries. Totally agreed. You are worrying just a bit too much about this, though -- we don't need to worry about a switch that has to decide WHEN to disable -rpath, just a switch that understands, Okay, the builder knows what he's talking about, no -rpath is fine with me. Ben -- Brought to you by the letters V and D and the number 3. Porcoga daisuki! Debian GNU/Linux maintainer of Gimp and GTK+ -- http://www.debian.org/ I'm on FurryMUCK as Che, and EFNet/Open Projects IRC as Che_Fox.
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, J.H.M. Dassen [EMAIL PROTECTED] wrote: On Wed, Jan 27, 1999 at 17:07:30 -0200, Alexandre Oliva wrote: You might have included my suggestion to prevent having to move libraries in the first place: creating a libc6-specific directory right now, instead of installing libraries in /usr/lib and having to move them into another directory when libc7 should be released. I'm sorry, but this is IMHO completely backwards. On Linux systems, there is nothing wrong with moving libraries around as the need arises. Except that you risk replacing a library with an incompatible one. That's what has caused you so much trouble. If, instead of moving the X11 libraries to another dir and creating new, incompatible versions under the same pathname, you had created new versions in other directories, no unexpected crashes would have occurred. Having libtool default to -rpath is what's causing problems. This is IMHO completely backwards :-) When a program is linked with a shared library, a contract is established between them stating that the library (or any newer but compatible version thereof, on systems that support library versioning) will supply the symbols that the program resolved from it, and the program will be able to find the library at that point. If you move the library and replace it with an incompatible one, you're breaking the contract and the versioning mechanism, so you can't blame the program for crashing, nor the tool that created the program. I've seen one too many instances of foo crashes on Debian that turned out to be foo is a libc5 binary with an RPATH of /usr/X11R6/lib which on any reasonably up to date Debian system contains libc6 X libraries. See? You replaced one library with an incompatible one without modifying its version number to mark it as incompatible. Isn't this breaking the contract? How could you expect it to work? The X example also shows that the problem isn't limited to /usr/lib either. What's next? /usr/local/lib/libc6 ? Maybe some library versioning mechanism that allows incompatible changes to be marked as such. Maybe an environment variable or some file in /etc containing a number that will be added to the major version number of any library libtool creates, so that they will be marked as incompatible. However, Alexandre Oliva [EMAIL PROTECTED] brings up an important point: -rpath is necessary if one is installing libraries and binaries linked to those libraries in one's home directory, That is a special case. The default should be sane for regular cases. You see the regular case as the one you use the most. I see it as the one I use the most. I don't install any packages in /usr or /usr/local because I find it *horrible* to have a huge /usr/local/bin without any clear separation between packages. It's a pity that the GNU/Linux distributions don't care about clearly separating packages from one another... or if your Unix has no support for library search paths via environment variables like Linux's LD_LIBRARY_PATH. While I appreciate concerns about supporting less fortunate operating environments, I don't think their existence should hold us back from doing the right thing on Linux. For which definition of the right thing? :-) In general, I feel that moving libraries around is a very bad idea, because it will lead to failure most of the times, and that's why I don't feel libtool should help people doing that. I see no reason why moving libraries around is a bad idea. I see defaulting to -rpath as a bad idea, which breaks moving libraries around. Because you break a contract every time you remove a library from the place in which it used to live. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Ben Gertzfield [EMAIL PROTECTED] wrote: I've never understood what the .la scripts are for. They contain inter-library dependency information, the location and the name of the actual library, and any additional run-time paths needed for the library dependencies. libtool (1.2d) is able to link with an installed .la file, meaning that it will select the appropriate switches to link the library, its dependencies and its run-time paths into the program or libtool library being linked. .la files are also used by libltdl, a portable dlopening library that is available in the current libtool CVS tree, and that will be released with libtool 1.3. libltdl will use the .la file to find out the pathname of the file to be dlopened, as well as any dependencies that must be dlopened before the library. It is possible that libtool 1.3 will be able to find and use .la files even when -L and -l switches are used to refer to it (currently it requires the pathname of the .la file). With Debian and Red Hat, it's totally the opposite. Moving libraries around is what leads to upgrades being possible. Then why do you find so much trouble with it? Totally agreed. You are worrying just a bit too much about this, though -- we don't need to worry about a switch that has to decide WHEN to disable -rpath, just a switch that understands, Okay, the builder knows what he's talking about, no -rpath is fine with me. In this case, you already have the suggestion of the ld wrapper script, right? :-) The point is that, if we support this flag, it must be supported in a portable way, otherwise GNU/Linux developers may feel inclined to enable it by default in the packages they maintain, and this will result in their packages *having* to be installed with --prefix=/usr/local, and even then, they will only work on GNU/Linux. I want to avoid this situation at all costs. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On 27 Jan 1999, Alexandre Oliva wrote: Having libtool default to -rpath is what's causing problems. This is IMHO completely backwards :-) When a program is linked with a shared library, a contract is established between them stating that the library (or any newer but compatible version thereof, on systems that support library versioning) will supply the symbols that the program resolved from it, and the program will be able to find the library at that point. If you move the library and replace it with an incompatible one, you're breaking the contract and the versioning mechanism, so you can't blame the program for crashing, nor the tool that created the program. The contract does not state that the library will be found in a particular location on the filesystem hierarchy. The contract simply states that the library will be found. Which library is used can be determined by the linker. Where is the need for rpath here? The combination of library versions uniquely identifies, to a suitably well configured linker, which library to find. I've seen one too many instances of foo crashes on Debian that turned out to be foo is a libc5 binary with an RPATH of /usr/X11R6/lib which on any reasonably up to date Debian system contains libc6 X libraries. See? You replaced one library with an incompatible one without modifying its version number to mark it as incompatible. Isn't this breaking the contract? How could you expect it to work? It did work. On all binaries without -rpath. It worked. What do you mean, 'How could we expect it to work?' However, Alexandre Oliva [EMAIL PROTECTED] brings up an important point: -rpath is necessary if one is installing libraries and binaries linked to those libraries in one's home directory, That is a special case. The default should be sane for regular cases. You see the regular case as the one you use the most. I see it as the one I use the most. I don't install any packages in /usr or /usr/local because I find it *horrible* to have a huge /usr/local/bin without any clear separation between packages. It's a pity that the GNU/Linux distributions don't care about clearly separating packages from one another... I'm sorry, but I'm flabbergasted. To think that the person in charge of libtool actively disagrees with a) The whole point of packaging systems b) The FHS Our distribution cares greatly about separating packages from each other. And it does it very well. There is no need to acheive this by enforcing such package structure on the file-system - which package a file belongs to is a concept orthogonal to where the file lives in a sensible filesystem. Worried, Jules /+---+-\ | Jelibean aka | [EMAIL PROTECTED] | 6 Evelyn Rd| | Jules aka | [EMAIL PROTECTED] | Richmond, Surrey | | Julian Bean | [EMAIL PROTECTED]| TW9 2TF *UK* | ++---+-+ | War doesn't demonstrate who's right... just who's left. | | When privacy is outlawed... only the outlaws have privacy. | \--/
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: On 27 Jan 1999, Alexandre Oliva wrote: Having libtool default to -rpath is what's causing problems. This is IMHO completely backwards :-) When a program is linked with a shared library, a contract is established [...] If you move the library and replace it with an incompatible one, you're breaking the contract and the versioning mechanism, so you can't blame the program for crashing, nor the tool that created the program. The contract does not state that the library will be found in a particular location on the filesystem hierarchy. Correct. The contract simply states that the library will be found. Which library is used can be determined by the linker. Except that, if you replace the library with an incompatible one, you *are* breaking the contract. Furthermore, there's no reason to assume that the user will *not* have used -rpath himself, so moving libraries does have a potential for breaking user programs, therefore it should best be avoided. Where is the need for rpath here? There's no need for it, but it doesn't cause any harm unless you break the rules, i.e., replace a library with an incompatible one that holds an apparently compatible version number. *This* is the root of all your problems, not that fact that a rpath is specified. If you had not replaced libraries with incompatible versions, the dynamic linker would not have found it in the hard-coded rpath and would look for it in the default search path, and find it in the appropriate alternate location. See? You replaced one library with an incompatible one without modifying its version number to mark it as incompatible. Isn't this breaking the contract? How could you expect it to work? It did work. On all binaries without -rpath. It worked. What do you mean, 'How could we expect it to work?' Except for this `minor' restriction. You may be silently breaking *user* programs because they have chosed to specify -rpath where it was not necessary. If you think people *must not* use -rpath on your system, you'd better completely disable it, not blame the user for making use of a (IMO nice) feature of your system. I think a solution that is not general is not a good solution. Since the solution of moving shared libraries around has the potential of causing problems, if I were you, I'd work harder to try to find a complete solution (which I happen to have already suggested) instead of trying to blame libtool or other users for doing things that are perfectly correct, but not in the way that would let you replace incompatible libraries. Our distribution cares greatly about separating packages from each other. Not from the point of view of the end user. When they want to find a tool, they may `ls /usr/bin /usr/local/bin' and will be presented *thousands* of files. I'd rather have a classification system in which I could have links (or similar) to all programs in a common directory, but in which I could search for programs by subject. I'd like to have development tools such as compilers in one directory, text writing tools such as word processors and text editors in another, system administration utilities in another, and so on. Additionally, I'd like to be able to keep multiple versions of a package installed, and the only way I could do that is by installing each version in a separate directory, and selecting which version to use via soft-links in the directory that happens to be in my PATH. How does the current packaging system allows me to test one version of a package while other users of the same host are running a stable version of that tool? Or are the GNU/Linux distributions all moving towards the Micro$oft model of single-user workstations? -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On 27 Jan 1999, Alexandre Oliva wrote: The contract simply states that the library will be found. Which library is used can be determined by the linker. Except that, if you replace the library with an incompatible one, you *are* breaking the contract. We don't replace libraries with incompatible ones. We bring in new libraries, which are incompatible. We keep the old ones, which are compatible. Our linker knows how to tell which is which. Who cares where they are stored? Actually, we do care where they are stored. My point is, that it doesn't matter if the new library is in the location the old library used to occupy. Our linker knows which is which. Furthermore, there's no reason to assume that the user will *not* have used -rpath himself, so moving libraries does have a potential for breaking user programs, therefore it should best be avoided. This is a valid point. It is my feeling that -rpath should not be used for libraries in the 'standard' paths, which allows them to move (which is a useful feature). It is reasonable to use it for libraries in unusual places (another useful feature). I think a solution that is not general is not a good solution. Since the solution of moving shared libraries around has the potential of causing problems, if I were you, I'd work harder to try to find a complete solution (which I happen to have already suggested) instead of trying to blame libtool or other users for doing things that are perfectly correct, but not in the way that would let you replace incompatible libraries. I agree with your suggested solution, which amounted to more complex understanding and use of inter-library dependencies by the tools. However, I don't have the expertise to implement this. And I disagree with you (strongly) about the correct use of the existing system. Our distribution cares greatly about separating packages from each other. Not from the point of view of the end user. When they want to find a tool, they may `ls /usr/bin /usr/local/bin' and will be presented *thousands* of files. I'd rather have a classification system in which I could have links (or similar) to all programs in a common directory, but in which I could search for programs by subject. I'd like to have development tools such as compilers in one directory, text writing tools such as word processors and text editors in another, system administration utilities in another, and so on. Additionally, I'd like to be able to keep multiple versions of a package installed, and the only way I could do that is by installing each version in a separate directory, and selecting which version to use via soft-links in the directory that happens to be in my PATH. We do have a classification system. And we don't use the filesystem for it. We have lists of packages, with descriptions. We even support separate versions of software, to some extent, although I'd agree completely that our support in this area is not what it might be. How does the current packaging system allows me to test one version of a package while other users of the same host are running a stable version of that tool? Or are the GNU/Linux distributions all moving towards the Micro$oft model of single-user workstations? Of course not ;-) If you want to test one version, you simply run it with ./configure --prefix /home/me/nicepackage or whatever. But you knew that. Jules /+---+-\ | Jelibean aka | [EMAIL PROTECTED] | 6 Evelyn Rd| | Jules aka | [EMAIL PROTECTED] | Richmond, Surrey | | Julian Bean | [EMAIL PROTECTED]| TW9 2TF *UK* | ++---+-+ | War doesn't demonstrate who's right... just who's left. | | When privacy is outlawed... only the outlaws have privacy. | \--/
Re: -rpath with libtool and Debian Linux
On 27 Jan 1999, Alexandre Oliva wrote: Having libtool default to -rpath is what's causing problems. This is IMHO completely backwards :-) You know, I seem to remember that there is another rather unpleasent side-effect of rpath - it basically completely disables library searching and thus disables LD_LIBRARY_PATH, once you have used rpath it is not easy for a user replace that library (for whatever reason) and I find that highly annoying. I've seen one too many instances of foo crashes on Debian that turned out to be foo is a libc5 binary with an RPATH of /usr/X11R6/lib which on any reasonably up to date Debian system contains libc6 X libraries. See? You replaced one library with an incompatible one without modifying its version number to mark it as incompatible. Isn't this breaking the contract? How could you expect it to work? Not exactly, the x libraries are fully compatible and were built with the same source - the trouble is that one is linked with libc5 and the other with libc6. In normal cases the dymanic linker would figure this out one way or another with rpath this functionality is disabled as it overrides the library versioning scheme. The linux dynamic linker will resolve things in some magical way based on the library dependencies and the program dependencies to locate the proper library in many cases - rpath does cripples, not enhances, this function. On another note, you have suggested that we use /usr/lib/libc6 and other things to isolate libraries that have been recompiled with updated dependencies - the trouble with this is that all the distributions and all the distributers would have to agree on a scheme for this - otherwise a 'debian' binary would not function on a RH system because they used a different scheme and their rpaths would be incompatibly different. Furthermore this idea of a /usr/lib/libc6 becomes entirely unmanageable when we start having soname changes for things like libgtk - we will have the -same- problem with all the millions of libs that link with libgtk as we did with libc6! The linux dynamic linker has the ability to resolve these issues on it's own by carefully relocating the old libraries, rpath simply does not. We must be able to turn it off for libraries used on our system! Jason
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: On 27 Jan 1999, Alexandre Oliva wrote: Except that, if you replace the library with an incompatible one, you *are* breaking the contract. We don't replace libraries with incompatible ones. Oh yes, you are. We bring in new libraries, which are incompatible. We keep the old ones, which are compatible. Our linker knows how to tell which is which. Except that it can't do that if the user happened to use a switch that was available, and its use didn't trigger any blinking light saying that this would cause any future upgrade he might do to silently break his program. If you do want to be able to freely move libraries around, -rpath must be forbidden. If -rpath is available for users, you can't move libraries around and expect things to work. Who cares where they are stored? The user who used -rpath? Actually, we do care where they are stored. Ah! And that's where the potential conflict comes from. My point is, that it doesn't matter if the new library is in the location the old library used to occupy. It may matter for the user who used -rpath. Our linker knows which is which. Not always. Or maybe you can fix your linker to do the right thing even if the user inadvertently (incorrectly, in your terms) specified -rpath. We even support separate versions of software, to some extent, although I'd agree completely that our support in this area is not what it might be. And that's the very reason why I've never been able to adopt any of the available packaging systems: the only way to keep multiple working versions is to use a separate directory for each version of each package. How does the current packaging system allows me to test one version of a package while other users of the same host are running a stable version of that tool? Or are the GNU/Linux distributions all moving towards the Micro$oft model of single-user workstations? Of course not ;-) Jeez!, I was sure I had added a smiley after that last sentence. Sorry about that... If you want to test one version, you simply run it with ./configure --prefix /home/me/nicepackage But isn't this exactly what the packaging systems are trying to avoid, i.e., that people have to compile systems on their own? And then, how could I make sure that my test build works exactly as the pre-compiled upgrade, so that I could use the packaging system for the update? This is something that I feel that should be taken care of by the existing GNU/Linux distributions. In fact, I've got a bunch of ideas that I'll probably never find time to implement :-( about how to maintain multiple versions of packages installed and allowing each user to select which version he wants to use, either by explicit version number or by tags such as `stable', `test', `old', etc, tags that are determined by the system manager when he installs the package. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On 27 Jan 1999, Alexandre Oliva wrote: If you do want to be able to freely move libraries around, -rpath must be forbidden. If -rpath is available for users, you can't move libraries around and expect things to work. There are lots of things which users can do which might appear to work, and then break later. The user should understand what -rpath is for, and why he should use it, and when he should use it. As far as I can, it is *not* useful for the packages which are part of a 'managed' system like Debian. It *is* useful for users custom-compiling things in custom dirs. Can you tell -rpath to store the rpath for libmycustomthing.so and not for libc.so? (Not a rhetorical question - I genuinely don't know the answer). If not, then I'd suggest that it's rpath that's broken. We even support separate versions of software, to some extent, although I'd agree completely that our support in this area is not what it might be. And that's the very reason why I've never been able to adopt any of the available packaging systems: the only way to keep multiple working versions is to use a separate directory for each version of each package. In general, it is not useful to have multiple versions of the same package. In specific cases, it can be. I don't think that libtool is the right vehicle for you to evangelise your dislike of packaging systems and the FHS. If you want to test one version, you simply run it with ./configure --prefix /home/me/nicepackage But isn't this exactly what the packaging systems are trying to avoid, i.e., that people have to compile systems on their own? And then, how could I make sure that my test build works exactly as the pre-compiled upgrade, so that I could use the packaging system for the update? You would download the debian package. You would very slightly edit the rules file (which is like the makefile) to change the prefix passed to configure. You would then run dpkg-buildpackage. Or simply debian/rules binary. If the program wasn't debian packaged at all, you'd read some documentation on how to create a debian package ;-) This is something that I feel that should be taken care of by the existing GNU/Linux distributions. In fact, I've got a bunch of ideas that I'll probably never find time to implement :-( about how to maintain multiple versions of packages installed and allowing each user to select which version he wants to use, either by explicit version number or by tags such as `stable', `test', `old', etc, tags that are determined by the system manager when he installs the package. These are all worthy ideas. The idea of a 'fluid file system' or a 'tagged file system' which allows to mark abitrary subsets of your files with tags (like 'belongs_to_package_x' or 'part_of_debian_version_2.0') is a very worthy idea. The idea of extending debian's package system to cope elegantly with concurrent versions is (probably) a good idea. However, none of these things are currently available for Linux. What is available, is a distribution with well-thought (not arbitrary) structure and standards. A distribution used by many people, although not as many as RedHat (which has similar standards in most respects anyway). We have considered library dependencies. We have a working system. It is reasonably elegant. It is convenient for the users. It is not the only answer. Nonetheless, you are refusing to support it. Jules /+---+-\ | Jelibean aka | [EMAIL PROTECTED] | 6 Evelyn Rd| | Jules aka | [EMAIL PROTECTED] | Richmond, Surrey | | Julian Bean | [EMAIL PROTECTED]| TW9 2TF *UK* | ++---+-+ | War doesn't demonstrate who's right... just who's left. | | When privacy is outlawed... only the outlaws have privacy. | \--/
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Jason Gunthorpe [EMAIL PROTECTED] wrote: On 27 Jan 1999, Alexandre Oliva wrote: Having libtool default to -rpath is what's causing problems. This is IMHO completely backwards :-) You know, I seem to remember that there is another rather unpleasent side-effect of rpath - it basically completely disables library searching and thus disables LD_LIBRARY_PATH, once you have used rpath it is not easy for a user replace that library (for whatever reason) and I find that highly annoying. Well, he could always move the library elsewhere and put whatever he wants in its place... Some people might be consider it a bug in the dynamic linker, that could look at LD_LIBRARY_PATH before -rpath. I think -rpath really means it, so I consider the behavior correct (albeit a bit inconvenient for us, maintainers of libtool) See? You replaced one library with an incompatible one without modifying its version number to mark it as incompatible. Isn't this breaking the contract? How could you expect it to work? Not exactly, the x libraries are fully compatible except that they don't work with programs or libraries that had been compiled with the old version. When I talk of compatibility, I'm talking about full binary compatibility, not just apparent compatibility. In normal cases the dymanic linker would figure this out one way or another with rpath this functionality is disabled as it overrides the library versioning scheme. The linux dynamic linker will resolve things in some magical way based on the library dependencies and the program dependencies to locate the proper library in many cases - rpath does cripples, not enhances, this function. Since you do support -rpath in your system, you should probably extend your dynamic linker to work in this case too, or risk taking the blame for silently breaking applications, if the poor user ever understands what happened to his program. On another note, you have suggested that we use /usr/lib/libc6 and other things to isolate libraries that have been recompiled with updated dependencies - the trouble with this is that all the distributions and all the distributers would have to agree on a scheme for this - otherwise a 'debian' binary would not function on a RH system because they used a different scheme and their rpaths would be incompatibly different. This is a good point. But since all of you face this same problem, and all of you use the same major versions of libc, you can probably agree on a solution that would benefit you all. Or arrange, via inter-package dependencies, that required libraries are installed in the proper places. Furthermore this idea of a /usr/lib/libc6 becomes entirely unmanageable when we start having soname changes for things like libgtk - we will have the -same- problem with all the millions of libs that link with libgtk as we did with libc6! It seems to me that the main problem has to do with library versioning. Even though the existing library versioning mechanisms are great to describe the evolution of the API offerred by a library, they by no means describe the dependencies of a library, so we end up with libraries that have the same version numbers but that are not compatible at all. What we all are desperately looking for is a mechanism to allow the dynamic loader to pick the right version of a shared library out of a set of multiple builds of the same version of a library, the main difference between them being their dependencies. Using the version number of a dependency to modify the version number of the dependent library, in order to ensure that incompatibilities are automatically handled, seems perfect, except that it will only work when there is only one dependency. :-( If there is more than one dependency, there is no way to ensure that the major version number is increased whenever it has to be. Or I just can't find out how :-) Anyway, I strongly believe that being able to tell libtool not to use -rpath and to freely move libraries around will only give you more trouble to worry about in the future. We must be able to turn it off for libraries used on our system! I have already told you one (ugly) way to do it, but I still don't think it is a good idea in the general case. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On 27 Jan 1999, Alexandre Oliva wrote: In normal cases the dymanic linker would figure this out one way or another with rpath this functionality is disabled as it overrides the library versioning scheme. The linux dynamic linker will resolve things in some magical way based on the library dependencies and the program dependencies to locate the proper library in many cases - rpath does cripples, not enhances, this function. Since you do support -rpath in your system, you should probably extend your dynamic linker to work in this case too, or risk taking the blame for silently breaking applications, if the poor user ever understands what happened to his program. You may be right. I haven't examined that situation in enough detail to explain why the designers of the dynamic linker implemented it the way they did. Maybe they got it wrong. Note that 'we' (Debian) write neither the kernel, nor the dynamic linker. You have previously objected to a proposed solution on the grounds that 'you want libtool to work on all systems'. It seems to me that if you want to make libtool work on Linux, you should find a way of disabling rpath, since rpath is broken on Linux. It seems to me that the main problem has to do with library versioning. Even though the existing library versioning mechanisms are great to describe the evolution of the API offerred by a library, they by no means describe the dependencies of a library, so we end up with libraries that have the same version numbers but that are not compatible at all. You're right. That is the underlying problem. However, our dynamic linker *does* understand the problem. And it *does* have a solution to it. And -rpath turns it off. So we cannot afford to use -rpath. Jules /+---+-\ | Jelibean aka | [EMAIL PROTECTED] | 6 Evelyn Rd| | Jules aka | [EMAIL PROTECTED] | Richmond, Surrey | | Julian Bean | [EMAIL PROTECTED]| TW9 2TF *UK* | ++---+-+ | War doesn't demonstrate who's right... just who's left. | | When privacy is outlawed... only the outlaws have privacy. | \--/
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: On 27 Jan 1999, Alexandre Oliva wrote: Can you tell -rpath to store the rpath for libmycustomthing.so and not for libc.so? No, but, on some systems (for example, GNU/Linux), it is possible to hard-code the full pathname of libmycustomthing.so into the shared library itself, so that -rpath isn't needed, and the library cannot ever be moved. And that's the very reason why I've never been able to adopt any of the available packaging systems: the only way to keep multiple working versions is to use a separate directory for each version of each package. In general, it is not useful to have multiple versions of the same package. You're probably talking about the single-user workstation model. I'm talking about a networked multi-user model, in which some users need (for reasons only they understand :-) particular versions of, say, GNU Emacs and gcc installed. I don't think that libtool is the right vehicle for you to evangelise your dislike of packaging systems and the FHS. But debian-devel is probably a good place to talk about these ideas. What is available, is a distribution with well-thought (not arbitrary) structure and standards. A distribution used by many people, although not as many as RedHat (which has similar standards in most respects anyway). I should note that I happen to appreciate both of them. Nonetheless, you are refusing to support it. I'm not refusing to support it. I'm just inclined to avoid having an easy-to-use flag to disable explicit hard-coding of library paths because: 1) it would be hard to make it behave correctly in a portable way (and libtool would be useless if it were not for being portable); 2) it should not be necessary if you play by libtool rules, i.e., you pre-declare where libraries are going to be installed and keep them there forever (or until they're no longer needed); 3) I don't want to regret having introduced a flag that caused as much or more trouble than -rpath; and 4) I have already suggested a (dirty and ugly, I admit) hack that is sufficient for your needs of not using -rpath at all in Debian packages. I hope you understand my position now. I should also note that I myself have already wanted flags such as -hardcode-libdir for hardcoding the full pathname of a shared library into itself (it's mentioned in libtool/TODO) and -no-implicit-rpath (which is what you happen to be asking for), but I have some trouble deciding who should be responsible for deciding which flags to use for which libraries and programs. I feel this decision should be left for the installer of the package, but such installer-customizations aren't easy to introduce in the existing installation meta-tools. I'd really appreciate if someone had a bright idea about how to do that; I'd just like to avoid providing such dangerous flags for inadvertent use. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: Since you do support -rpath in your system, you should probably extend your dynamic linker to work in this case too, or risk taking the blame for silently breaking applications, if the poor user ever understands what happened to his program. Note that 'we' (Debian) write neither the kernel, nor the dynamic linker. Sorry, I had assumed you had hacked the dynamic linker in order to support the /libc5 compatibility libraries. You have previously objected to a proposed solution on the grounds that 'you want libtool to work on all systems'. It seems to me that if you want to make libtool work on Linux, you should find a way of disabling rpath, since rpath is broken on Linux. rpath is not broken, it just won't let one replace a library (meaning moving a library to another directory and installing a new library in its place) with one that appears to be compatible, according to the version numbers, but is not. That's the root of the problem, and that's why you want so much libtool not to use -rpath. I feel we're not going to make progress in this discussion unless someone comes up with a bright idea about how to allow the user to select when/how to hardcode rpaths or not. In the beginning of this discussion, Thomas Tanner suggested that -rpath could be dropped if it would specify a standard library search directory. Although I see problems in this behavior, that I have exposed in other messages, I think it is a reasonable trade-off, and I'm willing to accept a patch for libtool that would add to AC_PROG_LIBTOOL some code that would check whether enable_libtool_hardcoding_of_default_search_dirs=no (but --disable-libtool-... should remain undocumented by now) and, in this case, pass some flag to ltconfig that causes it to create a libtool script that drops rpaths (but not xrpaths) that specify directories listed in sys_lib_search_path_spec. However, I'd like to ask you to avoid distributing packages with pre-``compiled'' libtool scripts with such hardcoding disabled, particularly the libtool Debian package. This arrangement should take care of your immediate needs, while we try to find a better way to do it, or even decide to do it by default. However, our dynamic linker *does* understand the problem. And it *does* have a solution to it. And -rpath turns it off. So we cannot afford to use -rpath. As I have already pointed out, the solution is not complete, otherwise you'd not be observing any problem. -- Alexandre Oliva http://www.dcc.unicamp.br/~oliva [EMAIL PROTECTED] [EMAIL PROTECTED],gnu.org,egcs.cygnus.com,samba.org} Universidade Estadual de Campinas, SP, Brasil
Re: -rpath with libtool and Debian Linux
Jason Gunthorpe wrote: On 27 Jan 1999, Alexandre Oliva wrote: Having libtool default to -rpath is what's causing problems. This is IMHO completely backwards :-) You know, I seem to remember that there is another rather unpleasent side-effect of rpath - it basically completely disables library searching and thus disables LD_LIBRARY_PATH, once you have used rpath it is not easy for a user replace that library (for whatever reason) and I find that highly annoying. Yes, for example if a X program is linked with rpath, and you want to use something like xaw39 (or xaw95 or nextaw) instead of the usual xaw widgets with it, you can't. This has it's uses if you know the program won't work with those replacement libraries, but most programs do so it's a horrid default. -- see shy jo
Re: -rpath with libtool and Debian Linux
On Wed, Jan 27, 1999 at 08:22:09PM -0200, Alexandre Oliva wrote: I don't think that libtool is the right vehicle for you to evangelise your dislike of packaging systems and the FHS. But debian-devel is probably a good place to talk about these ideas. Please start another thread under a different subject for this. It is an important but mostly unrelated topic. Nonetheless, you are refusing to support it. I'm not refusing to support it. I'm just inclined to avoid having an easy-to-use flag to disable explicit hard-coding of library paths because: 1) it would be hard to make it behave correctly in a portable way (and libtool would be useless if it were not for being portable); So why not move the discussion in a direction where we can talk about portability, instead trying to convince each other that the own ideas are the best. We are faced here with two different ideas about library dependencies (handling them at compile/installation time on your side, or handling them at run time on our side), so we better talk about a way how to address both. IMHO, both have their positive and their negative. Now let's try to get constructive. 2) it should not be necessary if you play by libtool rules, i.e., you pre-declare where libraries are going to be installed and keep them there forever (or until they're no longer needed); See above. Libtools rules are not necessarily the only true rules. Let's try to get support for diversity in library handling. 3) I don't want to regret having introduced a flag that caused as much or more trouble than -rpath; and I don't understand this comment. Which trouble with --rpath do you mean? Until now, I only heard from you that rpath is the ideal solution. Maybe we can get a better understanding of each other when we outline the negative of the own solution. I'll try a beginning: No rpath makes it harder for the linker to find the library. In fact, on some systems, it may make it impossible, so rpath is essential on some systems (though not GNU/Linux or GNU/Hurd). On other systems, the linker needs some smart way to resolve dependencies and search for libraries. No rpath makes it harder for a user to determine exactly which system libraries he links with. (With rpath, though, it only works when the system administrator never changes the library file at this place, too). This is in a hurry. Could you do me tha favour and list some negative points of using rpath, too? 4) I have already suggested a (dirty and ugly, I admit) hack that is sufficient for your needs of not using -rpath at all in Debian packages. We can find our own hacks (and do so since a long time). Now we are interested in a compatible, portable and general solution. As the libtool maintainer, you should be the ideal person to talk about such a solution. I think we understood by now that a simple disable flag does not satisfy your high ambitions wrt to portability. Let's move on to better solutions. I hope you understand my position now. I should also note that I myself have already wanted flags such as -hardcode-libdir for hardcoding the full pathname of a shared library into itself (it's mentioned in libtool/TODO) and -no-implicit-rpath (which is what you happen to be asking for), but I have some trouble deciding who should be responsible for deciding which flags to use for which libraries and programs. Maybe you should not be the one to decide at all? Offer flexible solutions, where the last person can override the (possible) default given by others. This means, the package can provide a default, which can be overridden at compilation time. Finally, the installer can override both. I feel this decision should be left for the installer of the package, but such installer-customizations aren't easy to introduce in the existing installation meta-tools. Implement what works and provide a framework for things which should be made to work. This means, implement those flags for compilation, and leave installation possible for later. Marcus -- Rhubarb is no Egyptian god.Debian GNU/Linuxfinger brinkmd@ Marcus Brinkmann http://www.debian.orgmaster.debian.org [EMAIL PROTECTED]for public PGP Key http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Re: -rpath with libtool and Debian Linux
Alexandre Oliva wrote: More than that (and it was my fault to have failed to mention that before): libtool will hard-code the installation directory of the library into the `libdir' variable of the .la script it installs. Therefore, if one moves the library then tries to link with the .la file, he loses. There's also the dlopening issue: libltdl (to be released with libtool 1.3) will dlopen a library in the directory pointed to by `libdir' too. Well, that's actually not true. libltdl will try to dlopen the library in the directory where the .la file was found, too. In general, I feel that moving libraries around is a very bad idea, because it will lead to failure most of the times, and that's why I don't feel libtool should help people doing that. Moving libraries should be avoided, but libtool shouldn't make it impossible. Actually, not issuing -rpath or equivalent is quite easy to do, but choosing *when* not to do it is the part that is hard to port. Thomas Tanner has suggested that we could omit the -rpath switch for libraries that are supposed to be installed in directories listed in the default search path. However, the default search PATH might change, and, even if it did not, it would be possible to link an application with a library in, say, /usr/local/lib, and then find out that at run-time it loads a library with the same name in /usr/lib, because /usr/lib appears first in the standard search path. I still support my proposal. We have the same problem with the -L option and we can't do anything against it. For example, if you link with -L/usr/local/lib -lfoo -L/usr/lib -lbar the linker might link the library libbar in /usr/local/lib instead of the one in /usr/lib. AFAIK there's no clean and portable way to avoid this. So _the user_ has to ensure that this won't happen. If it's impossible for -L, then it's not necessary for -rpath. -- Thomas Tanner email: tanner@(gmx.de|ggi-project.org|gnu.org|mucl.de) web: http://home.pages.de/~tanner visit: www.ggi-project.org www.gnustep.org
Re: -rpath with libtool and Debian Linux
On 27 Jan 1999, Alexandre Oliva wrote: On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: On 27 Jan 1999, Alexandre Oliva wrote: [watch indenting carefully : I wrote this next bit, of course] In general, it is not useful to have multiple versions of the same package. You're probably talking about the single-user workstation model. I'm talking about a networked multi-user model, in which some users need (for reasons only they understand :-) particular versions of, say, GNU Emacs and gcc installed. In general, such a situation only arises because of a bug in the software. That is why I say that, in general, it is not useful. Nonetheless, you are refusing to support it. I'm not refusing to support it. I'm just inclined to avoid having an easy-to-use flag to disable explicit hard-coding of library paths because: 1) it would be hard to make it behave correctly in a portable way (and libtool would be useless if it were not for being portable); Special case-it for linux, if you will. Libtool has plenty of special cases as it is. 2) it should not be necessary if you play by libtool rules, i.e., you pre-declare where libraries are going to be installed and keep them there forever (or until they're no longer needed); We don't want to play by libtool rules. We don't see that as a sensible restriction. [more information to follow in a separate follow-up] Jules /+---+-\ | Jelibean aka | [EMAIL PROTECTED] | 6 Evelyn Rd| | Jules aka | [EMAIL PROTECTED] | Richmond, Surrey | | Julian Bean | [EMAIL PROTECTED]| TW9 2TF *UK* | ++---+-+ | War doesn't demonstrate who's right... just who's left. | | When privacy is outlawed... only the outlaws have privacy. | \--/
Re: -rpath with libtool and Debian Linux
On 27 Jan 1999, Alexandre Oliva wrote: On Jan 27, 1999, Jules Bean [EMAIL PROTECTED] wrote: Since you do support -rpath in your system, you should probably extend your dynamic linker to work in this case too, or risk taking the blame for silently breaking applications, if the poor user ever understands what happened to his program. Note that 'we' (Debian) write neither the kernel, nor the dynamic linker. Sorry, I had assumed you had hacked the dynamic linker in order to support the /libc5 compatibility libraries. Absolutely not. This is one of my main points, in fact. Our dynamic linker is already very good at solving versioning problems. It doesn't need us to guarantee the locations of libraries to work - it can get it all correct anyway. Therefore, we chose to solve that particular problem (the libc5-6 transition) by moving libraries around, knowing that our linker was up to the job. You are now saying that libtool, by giving no way of avoiding rpath, will not support this scheme. You have previously objected to a proposed solution on the grounds that 'you want libtool to work on all systems'. It seems to me that if you want to make libtool work on Linux, you should find a way of disabling rpath, since rpath is broken on Linux. rpath is not broken, it just won't let one replace a library (meaning moving a library to another directory and installing a new library in its place) with one that appears to be compatible, according to the version numbers, but is not. That's the root of the problem, and that's why you want so much libtool not to use -rpath. rpath is broken. You said as much yourself. rpath is broken because it *overrides* all other sorts of library searching. For all I know, there may be a design justification for this. My understanding of rpath was that it was an 'option of last resort' when you couldn't make it work any other way (and hence, the overriding of all other search options is justified). However, using rpath as default is bad for our setup. Since you think it is right to use rpath as default, I therefore claim it is broken. I feel we're not going to make progress in this discussion unless someone comes up with a bright idea about how to allow the user to select when/how to hardcode rpaths or not. In the beginning of this discussion, Thomas Tanner suggested that -rpath could be dropped if it would specify a standard library search directory. Easy. Allow libtool to be run with the option libtool --no-rpath. This would then be run in all debian-maintained versions of libtoolized packages, and they'd work. Furthermore, when talking to software developers, I'd recommend they use this option by default, disabling it with a ./configure --use-libtool-rpath if the user really needs the functionality. However, our dynamic linker *does* understand the problem. And it *does* have a solution to it. And -rpath turns it off. So we cannot afford to use -rpath. As I have already pointed out, the solution is not complete, otherwise you'd not be observing any problem. What do you mean the solution is not complete? It works. It works well. People developing only for linux, and not using libtool, wouldn't even *think* of using rpath since it is a) unnecessary and b) a bad thing. However, people don't develop only for linux (this is a very good thing). And because various OSes have wildly different rules for dynamic libraries, libtool was born to help developers keep their makefiles sane. And libtool is now deciding not to support the linux model. Jules /+---+-\ | Jelibean aka | [EMAIL PROTECTED] | 6 Evelyn Rd| | Jules aka | [EMAIL PROTECTED] | Richmond, Surrey | | Julian Bean | [EMAIL PROTECTED]| TW9 2TF *UK* | ++---+-+ | War doesn't demonstrate who's right... just who's left. | | When privacy is outlawed... only the outlaws have privacy. | \--/
Re: -rpath with libtool and Debian Linux
Jules Bean [EMAIL PROTECTED] writes: rpath is broken. You said as much yourself. rpath is broken because it *overrides* all other sorts of library searching. I think people here do not know about $ORIGIN. This allows to define relative rpaths. E.g., a package with a program foo and a library libbar.so where foo is installed in $PATH/bin and libbar.so is defined in $PATH/lib should use -rpath \$ORIGIN/../lib The $ORIGIN is defined relative to the location of the object containing the reference. This is available in Solaris 7 (maybe 2.6?) and Linux w/ glibc 2.1. -- ---. drepper at gnu.org ,-. 1325 Chesapeake Terrace Ulrich Drepper \,---' \ Sunnyvale, CA 94089 USA Cygnus Solutions `--' drepper at cygnus.com `
