Re: libtool - building both 32 and 64-bit members for archives in one "pass"
Hi Michael, On 01/19/2016 09:44 PM, Michael Felt wrote: > L.S., > > If I understand the documentation correctly libtool places non-shareable > members (.o files) in the src directory > and "shareable aka PIC objects in src/.libs". However, libtool makes only one > size of objects - either 32-bit or 64-bit. > > As "aixtools" I am trying to package (common) opensource projects - and > mainly I have done so as as 32-bit only and was not bothered by the lack > of 64-bit build issues. Recently there are packages that may only work well > in 64-bit mode (e.g., R language) and I have started to package > in 64-bit as well. > > As part of my research I see that Linux distros seem to use > .../lib/libsomething* and .../lib64/libsomething* for 32 and 64-bit > (respectively) library > packaging. The AIX convention - as you may, or may not know, is not to have > multiple .so files in two directories but to have one .../lib/libsomething.a > with multiple members as static and/or shareable objects. The same object > name may be used multiple times, although the legacy convention is to > use shr.o as a shared object of multiple .o members and shr64.o for the same > .o files - but compiled in 64-bit mode). > > I am wondering - and willing to work on - with assistence - an addition to > the .lo target such that both 32 and 64 bit (shared) objects are made > for inclusion in a single .a archive - to be compareable with a library as it > would be released by IBM AIX developers/labs. IMO, there is no point in having multilib support within libtool for AIX only. If ever, multilib support inside libtool should be implemented in some generic way - to allow for porting to Linux and others eventually. Instead, for the time being even on AIX, it is up to the package manager to separately compile the package in 32-bit and 64-bit mode, and then merge the members of both the 32-bit Archive Library and the 64-bit Archive Library into one single Archive Library. Agreed this is a little tricky, but possible. However, this works as long as you have libtool to create Archive Libraries at all - which is the default when LDFLAGS does not contain "-brtl". My 2 cents, /haubi/ PS: Probably worth a note: Since libtool-2.4.4 there is the "--with-aix-soname" configure option for optional filename-based shared library versioning on AIX, which is designed to work with above multilib merging strategy as well. ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: [RFC] Moving ltmain.sh and libtool.m4 into Automake
On 10/17/2012 11:41 AM, Gary V. Vaughan wrote: > 1. libltdl as a standalone runtime loader wrapper > 2. libtool.m4/ltmain.sh to generate the libtool script While I don't care about such a split in general, ... > I think (2) belongs better into Automake alongside the other tool > wrappers it already carries, where it can decide whether to run the > libtool m4 macros and roll an appropriate compiler wrapper tailored for > the project using it (no need for all the C++/Java/Fortran goo in a C- > only project for example). > > Another consideration is that rolling Libtool into Automake would make > using Libtool as a standalone script rather more difficult. Having > said that, my impression is that Libtool is rarely used that way in > any case, and further simplification may be possible by deliberately > dropping explicit support for that use case. > > If I make this split and contribute the macros and ltmain.sh to Automake, > is this something anyone else would like? If so, do you like it enough > to wire it into Automake with an appropriate hunk of Perl? ... please do not consider moving libtool.m4/ltmain.sh into Automake! There are build-tools around, often non-public, designed for either one specific platform, and/or for static libs only, which does handle all the build dependency hell (generators for domain specific languages for example) of the specific projects they're used for. Libtool does help a lot either to port these projects to other platforms, and/or to introduce shared libraries, by simply using CC="libtool gcc" (more or less), without the need to switch to Automake, which does do its own dependency thing. Thank you! /haubi/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: lt_dlopen fails on AIX
On 08/29/2012 11:18 AM, Brice Goglin wrote: > Hello, > > I cannot get ltdl to load plugins on AIX 6.1. I am compiling plugins > with -module -avoid-version. libtool generates a ".so" but then embeds > it inside a ".a" and it looks like it really wants me to use this ".a" > since the ".so" doesn't even get installed. > > When lt_dlopen tries to open the plugins (tried dlopen with an extension > and dlopenext without extension), it fails saying "file not found". The > raw dlopen doesn't want to load the .a file because of a "bad magic number". > > dlopen succeeds if I use "foo.a(foo.so)" as a file name instead of > "foo.a", and if I add RTLD_MEMBER to the flags. But I don't see any > reference to RTLD_MEMBER in libltdl, so I assume I am not building the > plugins correctly. Yep, AIX support in libtool is far from ideal. As indicated by Perry, during the AIX port of Gentoo Prefix, besides making ltdl more flexible in where/how to load shared objects from, I've grown up a working solution for filename-based shared library versioning - known as the "soname" feature in ELF/Linux world, as well as ltdl As it applies to libgcc_s too, I've outlined this already in http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52623#c8 To keep static linking possible, this should be done with runtime-linking enabled only. On the other hand, projects using libtool usually aren't designed just for AIX, but mainly for platforms that cannot disable runtime-linking at all ... However, the libtool (and ltdl) patches for that still aren't ready for upstream, but I plan to resume working on that... (can't promise anything unfortunately). Anyway, I've rebased them to current master these days, made them working again, and pushed to github: https://github.com/haubi/libtool/commits/aix-soname Use 'configure --enable-aix-soname' to see how it works, although bootstrapping is a little tricky at the moment, it doesn't work with current gnulib... Other open issues: * The "--enable-aix-soname=compat" mode isn't implemented yet. * If that compat mode actually is necessary at all. * If the name "aix-soname" and the values "no,compat,yes" are the best choice. * If it is possible to get rid of doing different things upon -brtl in LDFLAGS. * Outstanding tests with xlc and xlC (we do have gcc only here). * ... /haubi/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: lt_dlopen fails on AIX
On 08/30/12 15:29, Perry Smith wrote: > ... that you can have 32 bit objects (shared or not shared) and 64 bit > objects > in the same archive ... > You can do similar tricks with executables and loadable kernel extensions. Offtopic: How does this work with executables? > Of topic slightly but… there are a ton of magic features that are available > for > selecting and loading libraries which the open source community doesn't grok > because "its not like Linux". I'm not 100% sure but from my perspective it > appears that whatever the .la file does is mostly built into ld and the > loader > in AIX. The .la files in AIX are not needed. > I don't know what "ltdl" is "ltdl", provided by libtool, is a wrapper library around dlopen, ought to provide /one/ API to dynamically load modules on /any/ platform (that libtool supports). It does read the .la file, to know how to load the module created by libtool, and uses a few fallbacks when there is no .la file for the requested module. > On that topic, the "prefix" set of tools (...) has been ported to AIX. http://prefix.gentoo.org/ Indeed: Gentoo Prefix on AIX forces runtime-linking on, and because of that, it can use a working variant of filename-based shared library versioning. Unfortunately, Gentoo Prefix may not bootstrap on AIX right now because of myself slacking due to workwork load... /haubi/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: RFC: on AIX, which "soname"-equivalent to prefer with runtime linking?
Hello again, some updates regarding the '# autoload' property in general, as well as in combination with both 32 and 64 bit shared objects in one archive: On 01/21/11 15:19, Michael Haubenwallner wrote: > *) Create the Import File "shr.imp", containing >- this header: >#! libNAME.so.1(shr.o) ># autoload >#! libNAME.so.1(shr.o) >- the list of symbols exported. > *) '# autoload' keeps a reference to the shared object in the >target binary even when not referenced by some static object. The problem here is that there is no way to tell the '# autoload' to fire in either 32 _or_ 64 bit mode only. So when using '# autoload', both 32 and 64 bit shared objects must have the _same_ member name, e.g. "(shr.o)". When not using '# autoload' (or even independent of that), an additional line indicating the bitness of the exported symbols would make sense - either '# 32' or '# 64', to load the correct archive member when needed. I tend to omit the '# autoload' (and the subsequent '#!') line, as this is how it works with traditional shared libraries anyway: >Unfortunately, another shared object having undefined symbols provided >by this shared object does not trigger the latter to be referenced. Additionally, the ld manpage for AIX [1] says for undefined symbols, in section 'Import and Export File Format' [2]: "The main program must export symbols imported by other modules, or loading will fail." [1] http://publib.boulder.ibm.com/infocenter/aix/v6r1/topic/com.ibm.aix.cmds/doc/aixcmds3/ld.htm [2] http://publib.boulder.ibm.com/infocenter/aix/v6r1/topic/com.ibm.aix.cmds/doc/aixcmds3/ld.htm#a3119106d /haubi/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: RFC: on AIX,which "soname"-equivalent to prefer with runtime linking?
On 01/23/11 07:31, Daniel Sands wrote:
The reasons to choose this way:
*) It is possible to dlopen() with or without a version number:
- dlopen("libNAME.so(shr.o)", RTLD_MEMBER), besides the preferred
- dlopen("libNAME.so.1(shr.o)", RTLD_MEMBER), and even
- dlopen("libNAME.so.1.2.3(shr.o)", RTLD_MEMBER) does work.
>>>
>>> Does dlopen("libNAME.so", RTLD_GLOBAL|RTLD_NOW) (for example) without
>>> RTLD_MEMBER work?
Oh, overlooked this question first.
>> However, it seems that for all of the requested functionality, it may be
>> necessary to have an
>> AIX-specific libtool script and maybe even an AIX-specific LTDL.
libtool script is generated and is platform specific anyway, and
LTDL is designed to have platform specific implementation when necessary.
So the libtool part is not a problem here.
> So the libs are named with .so, but they are apparently in .a format,
> according to the above dlopens.
While this actually is the shiny new thing here, it may be the biggest
problem: Not being obvious that the .so file actually is an archive.
However, I do think this is acceptable, given that the traditional
way is different anyway (in having an archive file as shared library).
If not to say: "Shared libraries on AIX are archive files, period."
> In that case, RTLD_MEMBER is required and must include the member object name.
Exactly.
I haven't found a way to be compatible to the current way. So, as said before,
for current versions (=sonames) of packages it has to be up to the package
manager
to choose whether to use the old (no soname) or new (with soname) way, while
for new package versions (=sonames) it could be up to the package maintainer
to force the new way (with soname), not allowing the old one any more.
/haubi/
___
http://lists.gnu.org/mailman/listinfo/libtool
Re: RFC: on AIX, which "soname"-equivalent to prefer with runtime linking?
On 01/22/11 17:41, Peter O'Gorman wrote:
> On 01/21/2011 08:19 AM, Michael Haubenwallner wrote:
>> Hello!
>
> Hi.
>
> Library versioning for AIX would be a great feature.
>
>>
>> After playing around with different ways[1], this is my proposed one,
>> using an AIX feature called "Import Files":
>>
>> *) Create the shared object "shr.o" (using '-G' linker flag).
>> *) Set the LOADONLY flag for "shr.o" (using 'strip -e').
>> *) Create the Import File "shr.imp", containing
>> - this header:
>> #! libNAME.so.1(shr.o)
>> # autoload
>> #! libNAME.so.1(shr.o)
>> - the list of symbols exported.
>> *) Create the archive library "libNAME.so.1.2.3" from both
>> "shr.imp" and "shr.o".
>> *) Create the symlinks as usual:
>> libNAME.so.1 -> libNAME.so.1.2.3
>> libNAME.so -> libNAME.so.1.2.3
>>
>> Using this way, for existing packages it might be necessary to give the
>> package manager the choice whether to "--enable-aix-soname", or to keep
>> the old way the package used to create "libNAME.so" before.
>
> If I upgrade a library from say version 1.2.3 to 1.2.5, with both having the
> same libtool version information,
> but with 1.2.3 linked with the current -brtl libtool system, and 1.2.5 using
> your proposed method,
> will clients of the library still work?
(assuming 1.2.5 is binary compatible to 1.x, the soname (on Linux) is
libNAME.so.1,
while version 2.0.0 is incompatible and does have soname libNAME.so.2)
Currently, subsequent executables require "libNAME.so" to be a standalone
shared object. When using the new way, "libNAME.so" is (a symlink to) an
archive file.
The simple answer is: No.
However, when upgrading to version 2.0.0, existing binaries will be broken
anyway, even with the old way where "libNAME.so" is the shared object.
Unless you --enable-aix-soname even for version 1.x, you are unable to upgrade
to 2.0.0 without breaking existing binaries, even when keeping the old shared
object around somewhere.
So in both cases, either enabling aix-soname or upgrading to 2.0,
you have to rebuild the reverse dependencies anyway.
This is the reason why the package /manager/ has to have the choice whether
to --enable-aix-soname or not, as only she may have control over the packages
currently linked against that "soname" of the library.
OTOH, when the package /maintainer/ does a soname bump, she might want to
force the aix-soname without leaving the choice to --disable-aix-soname,
as existing binaries using this libraries will break anyway.
>
>>
>> The reasons to choose this way:
>>
>> *) It is possible to dlopen() with or without a version number:
>> - dlopen("libNAME.so(shr.o)", RTLD_MEMBER), besides the preferred
>> - dlopen("libNAME.so.1(shr.o)", RTLD_MEMBER), and even
>> - dlopen("libNAME.so.1.2.3(shr.o)", RTLD_MEMBER) does work.
>
> Does dlopen("libNAME.so", RTLD_GLOBAL|RTLD_NOW) (for example) without
> RTLD_MEMBER work?
>
> Do you currently have patches for libtool, or do you want pre-approval before
> working on them?
> If you have any, even if only half finished, I would like to see them.
For the last few years, I've played around with different ways here to learn
which way could be the best one.
While the patches I currently have do provide shared library versioning, they
do not do this proposed way yet. Unfortunately, they do not create the soname
libNAME.so.1 as symlink to an archive file, but as standalone shared object,
while libNAME.so is the symlink to an archive file containing the import file.
Additionally, they do force -no-undefined ATM.
And yes, I'd like to have some commitment now upon /how/ I break all my
packages on AIX once again.
Anyway, you've asked for half finished patches:
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/eclass/ELT-patches/aixrtl/
/haubi/
___
http://lists.gnu.org/mailman/listinfo/libtool
Re: RFC: on AIX, which "soname"-equivalent to prefer with runtime linking?
On 01/21/11 17:48, Robert Boehne wrote: > Michael, > The "-brtl" isn't a "run-time linking" flag, but more exactly, it's a > system-5 shared library compatibility mode flag. > It is intended to provide a more familiar shared library style, not > additional features. As far as I can tell, runtime linking is not done on AIX without the '-brtl' flag to both the shared objects and the executable. Eventually it is the '-brtllib' flag that actually /enables/ runtime linking for an executable (which is added by the '-brtl' flag and gets disabled with '-bnortllib' when creating shared objects, either explicitly or via '-G' linker flag), but '-brtl' is still necessary to /allow/ a shared object to be subject to runtime linking. But yes, '-brtl' also adds some extra System-V compatibility at linktime, however something like 'DT_SONAME' is still missing. /haubi/ > With AIX style shared libraries, you can put multiple versions of a shared > library, and static versions, > in the same archive file, 32 bit & 64 bit etc. AIX shared libraries are very > flexible, however, > because no other operating system (to my knowledge) supports the same > features, those features are not portable. > > HTH, > > Rob > > On Fri, Jan 21, 2011 at 8:19 AM, Michael Haubenwallner > mailto:michael.haubenwall...@salomon.at>> > wrote: > > Hello! > > On AIX, with runtime linking (-brtl linker flag) enabled, the current way > how libtool creates shared libraries prevents any form of "soname" > support, > as there is no way to have the runtime loader to load a different version > of > some shared object (either standalone or as archive member) than the > linker > would record into the next binary when linking against this library. > ___ http://lists.gnu.org/mailman/listinfo/libtool
RFC: on AIX, which "soname"-equivalent to prefer with runtime linking?
Hello!
On AIX, with runtime linking (-brtl linker flag) enabled, the current way
how libtool creates shared libraries prevents any form of "soname" support,
as there is no way to have the runtime loader to load a different version of
some shared object (either standalone or as archive member) than the linker
would record into the next binary when linking against this library.
However, there are ways to get an equivalent of "soname" support even with
runtime linking enabled.
What I'm after now is to get a consensus over which way to prefer as
"best practice" to create shared libraries with "soname" support with
runtime linking enabled, even for projects not using libtool but doing
their own platform specifics instead.
So please comment on how my proposed way below is wrong, how it should
be improved/completed/whatever, as well as whether you can or would
accept/support/push it for libtool/your project/whatever, etc.
After playing around with different ways[1], this is my proposed one,
using an AIX feature called "Import Files":
*) Create the shared object "shr.o" (using '-G' linker flag).
*) Set the LOADONLY flag for "shr.o" (using 'strip -e').
*) Create the Import File "shr.imp", containing
- this header:
#! libNAME.so.1(shr.o)
# autoload
#! libNAME.so.1(shr.o)
- the list of symbols exported.
*) Create the archive library "libNAME.so.1.2.3" from both
"shr.imp" and "shr.o".
*) Create the symlinks as usual:
libNAME.so.1 -> libNAME.so.1.2.3
libNAME.so -> libNAME.so.1.2.3
Using this way, for existing packages it might be necessary to give the
package manager the choice whether to "--enable-aix-soname", or to keep
the old way the package used to create "libNAME.so" before.
The reasons to choose this way:
*) Need to distinguish different sonames on filename level rather than within
an unversioned archive file. This is possible with Import Files only, being
either standalone or an archive member.
Having multiple unversioned archive files around in directories potentially
searched at runtime is calling for problems like this one I really had[2]:
- AIX provides /usr/lib/libiconv.a containing "shr.o"
- GNU-iconv provides /myprefix/lib/libiconv.a containing "libiconv.so.2",
and eventually "shr.o" extracted from /usr/lib/libiconv.a to keep existing
binaries running even when LIBPATH is set to "/myprefix/lib".
- mybinary linked against GNU-iconv has an encoded runpath "/myprefix/lib:"
"/usr/lib:/lib" to find the correct libiconv.a even with LIBPATH unset.
- When calling mybinary from within IBM-Java, "/usr/lib:/lib" is _prepended_
to LIBPATH, so /usr/lib/libiconv.a is found first - which obviously does
not contain "libiconv.so.2" and causes mybinary to fail.
*) It is possible to dlopen() with or without a version number:
- dlopen("libNAME.so(shr.o)", RTLD_MEMBER), besides the preferred
- dlopen("libNAME.so.1(shr.o)", RTLD_MEMBER), and even
- dlopen("libNAME.so.1.2.3(shr.o)", RTLD_MEMBER) does work.
I've choosen "shr.o" as the shared object member name to have at least
some recognition value when one already knows the traditional AIX way
of shared libraries.
While the name "shr.o" of the shared object is irrelevant, it should be
identical among different libraries, so developers using dlopen() could
take "(shr.o)" as another "AIX-constant" like RTLD_MEMBER.
However, it is necessary to create the "shr.o" and "shr.imp" within some
library-specific subdirectory, to avoid race-conditions when building
multiple libraries in parallel, or when there is some sourcefile "shr.c".
*) Immediately setting the LOADONLY flag avoids resolving symbols by
"libNAME.so(shr.o)" when these symbols are not listed for whatever
reason in the Import File to be provided by "libNAME.so.1(shr.o)".
I did have the situation where a subsequent binary got references
to both "libNAME.so.1(shr.o)" and "libNAME.so(shr.o)".
*) '# autoload' keeps a reference to the shared object in the
target binary even when not referenced by some static object.
Linking against standalone shared objects does this nevertheless.
Unfortunately, another shared object having undefined symbols provided
by this shared object does not trigger the latter to be referenced.
So omitting the '# autoload' to get some '--as-needed' equivalent may
not work as expected in all circumstances.
*) The AIX way of multilib (32bit/64bit) does work, by having both
the 32bit "shr.o" and the 64bit "shr_64.o" in the same archive
(other platforms call this something like "fat binary"), even if
libtool does not support that by design (yet?).
Import Files can declare symbols for either 32bit, 64bit or both
if necessary. And it is possible to have multiple Import Files in
one archive. This would be "shr.imp" and "shr_64.imp" respectively.
While it is even possible to have a 32bit "shr.o" and a 64bit "shr.o"
in
