-dlopen self on AIX
Does -dlopen self work on AIX? While building epiphany-1.6.5 on this
platform, seems that not all the correct symbols are being exported.
On AIX 5.3, dlopen_self=no. If -dlopen self is specified, then the
following code is executed (from the 1.5 branch):
if test "$dlself" = yes; then
$show "generating symbol list for \`$output'"
test -z "$run" && $echo ': @PROGRAM@ ' > "$nlist"
# Add our own program objects to the symbol list.
progfiles=`$echo "X$objs$old_deplibs" | $SP2NL | $Xsed -e "$lo2o" |
$NL2SP`
for arg in $progfiles; do
$show "extracting global C symbols from \`$arg'"
$run eval "$NM $arg | $global_symbol_pipe >> '$nlist'"
done
...
First, if noinst_LTLIBRARIES *.la files are specified on the
command-line, their symbol list will not be searched because the only
files searched for symbols is "X$objs$old_deplibs". This should really
be "X$objs$libobjs$old_deplibs$old_convenience".
Running the link line and the progfiles change I get:
$ ../libtool --mode=link xlC_r -O2 -qro -qroconst -qmaxmem=-1
-qarch=com -L/opt/TWWfsw/gettext014/lib -Wl,-brtl
-o epiphany -R/opt/TWWfsw/firefox15/lib -dlopen
self epiphany-ephy-main.o libephymain.la
../src/bookmarks/libephybookmarks.la ../embed/libephyembedfactory.la
../embed/mozilla/libephymozillaembed.la ../embed/libephyembed.la
../lib/widgets/libephywidgets.la ../lib/libephymisc.la
../lib/egg/libegg.la -L/opt/TWWfsw/firefox15/lib -lgtkembedmoz
-lxpcom -lxpcom_core -lplds4 -lplc4 -lnspr4 -lpthreads -ldl ...
rm -f .libs/epiphany.nm .libs/epiphany.nmS .libs/epiphany.nmT
creating .libs/epiphanyS.c
generating symbol list for `epiphany'
extracting global C symbols from `epiphany-ephy-main.o'
extracting global C symbols from `./.libs/libephymain.a'
extracting global C symbols from `../src/bookmarks/.libs/libephybookmarks.a'
extracting global C symbols from `../embed/.libs/libephyembedfactory.a'
extracting global C symbols from
`../embed/mozilla/.libs/libephymozillaembed.a'
extracting global C symbols from `../embed/.libs/libephyembed.a'
extracting global C symbols from `../lib/widgets/.libs/libephywidgets.a'
extracting global C symbols from `../lib/.libs/libephymisc.a'
extracting global C symbols from `../lib/egg/.libs/libegg.a'
(cd .libs && xlc_r -O2 -qro -qroconst -qmaxmem=-1 -qarch=com -c "epiphanyS.c")
rm -f .libs/epiphanyS.c .libs/epiphany.nm .libs/epiphany.nmS
.libs/epiphany.nmT
xlC_r -O2 -qro -qroconst -qmaxmem=-1 -qarch=com -brtl
-o epiphany .libs/epiphanyS.o epiphany-ephy-main.o
-L/opt/TWWfsw/gettext014/lib ./.libs/libephymain.a
../src/bookmarks/.libs/libephybookmarks.a
../embed/.libs/libephyembedfactory.a
../embed/mozilla/.libs/libephymozillaembed.a
../embed/.libs/libephyembed.a ../lib/widgets/.libs/libephywidgets.a
../lib/.libs/libephymisc.a ../lib/egg/.libs/libegg.a
-L/opt/TWWfsw/firefox15/lib -lgtkembedmoz -lxpcom -lxpcom_core -lplds4
-lplc4 -lnspr4 -lpthreads -ldl ...
One of the symbols not being exported is
"language_editor_up_button_clicked_cb". So, let's check out the above:
$ grep language_editor_up_button_clicked_cb .libs/epiphanyS.c
{"language_editor_up_button_clicked_cb", (lt_ptr)
&language_editor_up_button_clicked_cb},
$ nm -BCpg .libs/epiphanyS.o | grep language_editor_up_button_clicked_cb
- U language_editor_up_button_clicked_cb
$ nm -BCpg epiphany | grep language_editor_up_button_clicked_cb
269185316 T .language_editor_up_button_clicked_cb
But what I want is the following:
$ nm -BCpg epiphany | grep language_editor_up_button_clicked_cb
269185316 T .language_editor_up_button_clicked_cb
536936672 D language_editor_up_button_clicked_cb
--
albert chin ([EMAIL PROTECTED])
___
http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
On Sun, Oct 30, 2005 at 12:10:23PM -0800, Howard Chu wrote: > Albert Chin wrote: > > [snip ...] > > > >Dynamic module support in openldap-2.3.x isn't working on AIX for the > >above reason. Using -bexpall or -bE when linking the main binary does > >work though. While the main binary, slapd, doesn't dlopen itself, > >modules it dlopen()'s cannot resolve symbols in slapd because they > >weren't exported. > > I don't recall what particular hacks we've done to libtool for this on > AIX, but Symas builds everything as dynamic modules on all of our > platforms, and mostly It Just Works. (Including IBM AIX and z/OS, as > well as Solaris, HPUX, Linux, Windows, etc.) In general the only thing > I've hacked was to prevent build directories from being embedded into > the generated files, and to eliminate all relinking. I've submitted most > of our changes back to this list in the past; a lot may have fallen on > the floor since those were patches against libtool 1.4. I'll follow up on the OpenLDAP mailing list. I can reproduce my error with a virgin build/install of 2.3.11. -- albert chin ([EMAIL PROTECTED]) ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
On Sun, Oct 30, 2005 at 12:16:01PM -0600, Albert Chin wrote: > On Sun, Oct 30, 2005 at 11:56:30AM -0600, Albert Chin wrote: > > On Sun, Oct 30, 2005 at 11:14:14AM -0600, Albert Chin wrote: > > > [snip ...] > > > > > > This means the method libtool uses to support -dlopen self on AIX > > > won't work (by building a list of symbols into some xxS.o file and > > > linking it with the executable). The real solution is to -bE the > > > symbol list. What's the cleanest way to do this? > > > > > > Dynamic module support in openldap-2.3.x isn't working on AIX for the > > > above reason. Using -bexpall or -bE when linking the main binary does > > > work though. While the main binary, slapd, doesn't dlopen itself, > > > modules it dlopen()'s cannot resolve symbols in slapd because they > > > weren't exported. > > > > Seems the openldap folks should really use -export-dynamic, which > > isn't defined for AIX. We could make it -bexpall but as documented in > > the previous mail, it doesn't export everything. So, should we create > > a new variable, export_dynamic_cmds? If so, and temporary files are > > created by it, how do we clean up? > > Related to this, if always_export_symbols=yes, shouldn't we run > $export_symbols_cmds against programs as well? export_symbols_cmds=yes > only on AIX but it's equally important for programs and libraries. I finished a patch for this but it's not robust. There's nothing we can do. If a program links against an archive library in the build process, the linker will pull in whatever symbols it needs from the archive library. If we export a full set of symbols from both the program and the library, we might get unresolved symbols from the linker because we've included symbols in the export list that the linker wouldn't pull in from the archive library because the program never references them. Oh well. -- albert chin ([EMAIL PROTECTED]) ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
Albert Chin wrote: On Sun, Oct 30, 2005 at 11:38:26AM -0800, Howard Chu wrote: Albert Chin wrote: On Sun, Oct 30, 2005 at 11:14:14AM -0600, Albert Chin wrote: [snip ...] This means the method libtool uses to support -dlopen self on AIX won't work (by building a list of symbols into some xxS.o file and linking it with the executable). The real solution is to -bE the symbol list. What's the cleanest way to do this? Dynamic module support in openldap-2.3.x isn't working on AIX for the above reason. Using -bexpall or -bE when linking the main binary does work though. While the main binary, slapd, doesn't dlopen itself, modules it dlopen()'s cannot resolve symbols in slapd because they weren't exported. Seems the openldap folks should really use -export-dynamic, which isn't defined for AIX. We could make it -bexpall but as documented in the previous mail, it doesn't export everything. So, should we create a new variable, export_dynamic_cmds? If so, and temporary files are created by it, how do we clean up? Wait. The fact that -bexpall "doesn't export everything" is A Good Thing. Symbol tables on AIX include a lot of junk that other modules simply don't need to see. The symbols they omit are generally not useful for user code anyway. It might be a "A Good Thing" in terms of being "pure", but it's different than the behavior on Solaris, HP-UX, Tru64 UNIX, IRIX, and Linux. It's just not worth it being that different today. AIX certainly has no advantage because of it. Yes, it's a given that AIX is different from every other Unix in the world. But you have no idea what you're asking for here. If -bexpall actually exported every symbol, it would be utterly unusable. As someone who has been (suffering through) building dynamic modules on AIX since 3.2.0, I know what I'm talking about. The fact that the linker hides extraneous symbols today merely brings it inline with normal expectations / behavior on other platforms. You aren't missing anything with the symbols that aren't exported, none of them are user-level. E.g., in AIX, an exported function appears two or three times in the symbol table. Once for the actual body of the function, and a couple more times in the Data or BSS segments, as function pointers that are relocated in the calling modules address space. These extra symbols are only of importance to the dynamic loader, and it always knows how to find them. On older versions of AIX, that actually exported everything with -bexpall, you got duplicate symbols and clashes. The only usable way to link anything was using an explicit exports file. The current behavor of -bexpall is a significant improvement. It ain't broke, leave it alone. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc OpenLDAP Core Teamhttp://www.openldap.org/project/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
On Sun, Oct 30, 2005 at 11:38:26AM -0800, Howard Chu wrote: > Albert Chin wrote: > >On Sun, Oct 30, 2005 at 11:14:14AM -0600, Albert Chin wrote: > >>[snip ...] > >> > >>This means the method libtool uses to support -dlopen self on AIX > >>won't work (by building a list of symbols into some xxS.o file and > >>linking it with the executable). The real solution is to -bE the > >>symbol list. What's the cleanest way to do this? > >> > >>Dynamic module support in openldap-2.3.x isn't working on AIX for the > >>above reason. Using -bexpall or -bE when linking the main binary does > >>work though. While the main binary, slapd, doesn't dlopen itself, > >>modules it dlopen()'s cannot resolve symbols in slapd because they > >>weren't exported. > > > >Seems the openldap folks should really use -export-dynamic, which > >isn't defined for AIX. We could make it -bexpall but as documented in > >the previous mail, it doesn't export everything. So, should we create > >a new variable, export_dynamic_cmds? If so, and temporary files are > >created by it, how do we clean up? > > Wait. The fact that -bexpall "doesn't export everything" is A Good > Thing. Symbol tables on AIX include a lot of junk that other modules > simply don't need to see. The symbols they omit are generally not useful > for user code anyway. It might be a "A Good Thing" in terms of being "pure", but it's different than the behavior on Solaris, HP-UX, Tru64 UNIX, IRIX, and Linux. It's just not worth it being that different today. AIX certainly has no advantage because of it. -- albert chin ([EMAIL PROTECTED]) ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
Albert Chin wrote: On AIX, ld doesn't explicitly export all symbols (not even with -bexpall). From ld(1): autoexp Automatically exports some symbols from the output module without having to list them in an export file. (This option does not export all symbols from the output module. Use the -bexpall option to export all symbols.) This is the default. Use this option when linking a main program. The linker assumes that you are linking a main program when you do not specify a module type (with the M or modtype option) beginning with S and you do not use the noentry option. When you use the autoexp option, if any shared object listed on the command-line imports a symbol from the special file . (dot), and the module being linked contains a local definition of the symbol, the symbol is exported automatically. Other symbols are also exported automatically when you link with the rtl option. If a symbol defined in the module being linked has one or more additional definitions exported from a shared object listed on the command-line, and if any of the definitions is a BSS symbol, the symbol is exported automatically. If the definition in the module being linked is a BSS symbol, the symbol is exported with the nosymbolic attribute. Otherwise, the symbol is exported with the symbolic attribute. If the symbol is listed in an export file with another export attribute, the explicit attribute is used. If the autoexp option would automatically export a symbol, but the symbol is listed in an export file with the list attribute, the symbol is not exported. expall Exports all global symbols, except imported symbols, unreferenced symbols defined in archive members, and symbols beginning with an underscore (_). You can export additional symbols by listing them in an export file. This option does not affect symbols exported by the autoexp option. When you use this option, you might be able to avoid using an export file. On the other hand, using an export file provides explicit control over which symbols are exported, and allows you to use other global symbols within your shared object without worrying about conflicting with names exported from other shared objects. The default is noexpall. This means the method libtool uses to support -dlopen self on AIX won't work (by building a list of symbols into some xxS.o file and linking it with the executable). The real solution is to -bE the symbol list. What's the cleanest way to do this? Dynamic module support in openldap-2.3.x isn't working on AIX for the above reason. Using -bexpall or -bE when linking the main binary does work though. While the main binary, slapd, doesn't dlopen itself, modules it dlopen()'s cannot resolve symbols in slapd because they weren't exported. I don't recall what particular hacks we've done to libtool for this on AIX, but Symas builds everything as dynamic modules on all of our platforms, and mostly It Just Works. (Including IBM AIX and z/OS, as well as Solaris, HPUX, Linux, Windows, etc.) In general the only thing I've hacked was to prevent build directories from being embedded into the generated files, and to eliminate all relinking. I've submitted most of our changes back to this list in the past; a lot may have fallen on the floor since those were patches against libtool 1.4. Maybe you should just download our CDS3 packages... -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc OpenLDAP Core Teamhttp://www.openldap.org/project/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
Bob Friesenhahn wrote: On Sun, 30 Oct 2005, Albert Chin wrote: Seems the openldap folks should really use -export-dynamic, which isn't defined for AIX. We could make it -bexpall but as documented in the previous mail, it doesn't export everything. So, should we create a new variable, export_dynamic_cmds? If so, and temporary files are created by it, how do we clean up? A better approach for openldap is to have the daemon and its modules depend on a common shared library. As far as I am aware, this works ok under AIX and other systems. Designing modules so that they depend on symbols from the executable which loads them is just asking for trouble. We've talked about taking this approach before. Essentially all of the slapd code would become a shared library, and the main program would just be a stub that loads it. (This became an issue back in release 2.1, when the slap tools were still separate executables. Since shared modules record the name of the file that exports their symbols, a shared module linked against slapd would not work when loaded by slapadd. And this was before the AIX linker supported importing files against "." and "..". The solution at that time was to merge all of the slap tools into the main slapd executable, which we did in release 2.2. The same issue bit us on Windows, and the same fix worked there too.) Unfortunately the shared library-slapd approach is unmanageable because the code uses on a number of static libraries that are not usually compiled as PIC code. It would be a pretty drastic change to get that all working. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc OpenLDAP Core Teamhttp://www.openldap.org/project/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
Albert Chin wrote: On Sun, Oct 30, 2005 at 11:14:14AM -0600, Albert Chin wrote: [snip ...] This means the method libtool uses to support -dlopen self on AIX won't work (by building a list of symbols into some xxS.o file and linking it with the executable). The real solution is to -bE the symbol list. What's the cleanest way to do this? Dynamic module support in openldap-2.3.x isn't working on AIX for the above reason. Using -bexpall or -bE when linking the main binary does work though. While the main binary, slapd, doesn't dlopen itself, modules it dlopen()'s cannot resolve symbols in slapd because they weren't exported. Seems the openldap folks should really use -export-dynamic, which isn't defined for AIX. We could make it -bexpall but as documented in the previous mail, it doesn't export everything. So, should we create a new variable, export_dynamic_cmds? If so, and temporary files are created by it, how do we clean up? Wait. The fact that -bexpall "doesn't export everything" is A Good Thing. Symbol tables on AIX include a lot of junk that other modules simply don't need to see. The symbols they omit are generally not useful for user code anyway. (In contrast to older releases of AIX, where they really did export everything, and you'd get clashes for stupid things like __init and such that never should have been exported in the first place.) -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc OpenLDAP Core Teamhttp://www.openldap.org/project/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
On Sun, 30 Oct 2005, Albert Chin wrote: Seems the openldap folks should really use -export-dynamic, which isn't defined for AIX. We could make it -bexpall but as documented in the previous mail, it doesn't export everything. So, should we create a new variable, export_dynamic_cmds? If so, and temporary files are created by it, how do we clean up? A better approach for openldap is to have the daemon and its modules depend on a common shared library. As far as I am aware, this works ok under AIX and other systems. Designing modules so that they depend on symbols from the executable which loads them is just asking for trouble. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
On Sun, Oct 30, 2005 at 11:56:30AM -0600, Albert Chin wrote: > On Sun, Oct 30, 2005 at 11:14:14AM -0600, Albert Chin wrote: > > [snip ...] > > > > This means the method libtool uses to support -dlopen self on AIX > > won't work (by building a list of symbols into some xxS.o file and > > linking it with the executable). The real solution is to -bE the > > symbol list. What's the cleanest way to do this? > > > > Dynamic module support in openldap-2.3.x isn't working on AIX for the > > above reason. Using -bexpall or -bE when linking the main binary does > > work though. While the main binary, slapd, doesn't dlopen itself, > > modules it dlopen()'s cannot resolve symbols in slapd because they > > weren't exported. > > Seems the openldap folks should really use -export-dynamic, which > isn't defined for AIX. We could make it -bexpall but as documented in > the previous mail, it doesn't export everything. So, should we create > a new variable, export_dynamic_cmds? If so, and temporary files are > created by it, how do we clean up? Related to this, if always_export_symbols=yes, shouldn't we run $export_symbols_cmds against programs as well? export_symbols_cmds=yes only on AIX but it's equally important for programs and libraries. -- albert chin ([EMAIL PROTECTED]) ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: -dlopen self on AIX
On Sun, Oct 30, 2005 at 11:14:14AM -0600, Albert Chin wrote: > [snip ...] > > This means the method libtool uses to support -dlopen self on AIX > won't work (by building a list of symbols into some xxS.o file and > linking it with the executable). The real solution is to -bE the > symbol list. What's the cleanest way to do this? > > Dynamic module support in openldap-2.3.x isn't working on AIX for the > above reason. Using -bexpall or -bE when linking the main binary does > work though. While the main binary, slapd, doesn't dlopen itself, > modules it dlopen()'s cannot resolve symbols in slapd because they > weren't exported. Seems the openldap folks should really use -export-dynamic, which isn't defined for AIX. We could make it -bexpall but as documented in the previous mail, it doesn't export everything. So, should we create a new variable, export_dynamic_cmds? If so, and temporary files are created by it, how do we clean up? -- albert chin ([EMAIL PROTECTED]) ___ http://lists.gnu.org/mailman/listinfo/libtool
-dlopen self on AIX
On AIX, ld doesn't explicitly export all symbols (not even with -bexpall). From ld(1): autoexp Automatically exports some symbols from the output module without having to list them in an export file. (This option does not export all symbols from the output module. Use the -bexpall option to export all symbols.) This is the default. Use this option when linking a main program. The linker assumes that you are linking a main program when you do not specify a module type (with the M or modtype option) beginning with S and you do not use the noentry option. When you use the autoexp option, if any shared object listed on the command-line imports a symbol from the special file . (dot), and the module being linked contains a local definition of the symbol, the symbol is exported automatically. Other symbols are also exported automatically when you link with the rtl option. If a symbol defined in the module being linked has one or more additional definitions exported from a shared object listed on the command-line, and if any of the definitions is a BSS symbol, the symbol is exported automatically. If the definition in the module being linked is a BSS symbol, the symbol is exported with the nosymbolic attribute. Otherwise, the symbol is exported with the symbolic attribute. If the symbol is listed in an export file with another export attribute, the explicit attribute is used. If the autoexp option would automatically export a symbol, but the symbol is listed in an export file with the list attribute, the symbol is not exported. expall Exports all global symbols, except imported symbols, unreferenced symbols defined in archive members, and symbols beginning with an underscore (_). You can export additional symbols by listing them in an export file. This option does not affect symbols exported by the autoexp option. When you use this option, you might be able to avoid using an export file. On the other hand, using an export file provides explicit control over which symbols are exported, and allows you to use other global symbols within your shared object without worrying about conflicting with names exported from other shared objects. The default is noexpall. This means the method libtool uses to support -dlopen self on AIX won't work (by building a list of symbols into some xxS.o file and linking it with the executable). The real solution is to -bE the symbol list. What's the cleanest way to do this? Dynamic module support in openldap-2.3.x isn't working on AIX for the above reason. Using -bexpall or -bE when linking the main binary does work though. While the main binary, slapd, doesn't dlopen itself, modules it dlopen()'s cannot resolve symbols in slapd because they weren't exported. -- albert chin ([EMAIL PROTECTED]) ___ http://lists.gnu.org/mailman/listinfo/libtool
