libgcc: On AIX, increase chances to find landing pads for exceptions
Hi David, still experiencing exception-not-caught problems with gcc-4.2.4 on AIX leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in 2004 already, ought to be fixed by some different commit since 3.4.0. As long as build systems (even libtool right now) on AIX do export these _GLOBAL__* symbols from shared libraries, overlapping frame-base address ranges may become registered, even if newer gcc (seen with 4.8) does name the FDE symbols more complex to reduce these chances. But still, just think of linking some static library into multiple shared libraries and/or the main executable. Or sometimes there is just need for some hackery to override a shared object's implementation detail and rely on runtime linking to do the override at runtime. Agreed both is "wrong" to some degree, but the larger an application is, the higher is the chance for this to happen. Thoughts? Thanks! /haubi/ 2016-02-08 Michael Haubenwallner On AIX, increase chances to find landing pads for exceptions. * unwind-dw2-fde.c (_Unwind_Find_FDE): Stop assuming registered object's address ranges to not overlap. --- libgcc/unwind-dw2-fde.c +++ libgcc/unwind-dw2-fde.c @@ -1013,7 +1013,25 @@ _Unwind_Find_FDE (void *pc, struct dwarf_eh_bases *bases) f = search_object (ob, pc); if (f) goto fini; + /* break; + In an ideal world, even on AIX, we could break here because + objects would not overlap. But the larger an application is, + the more likely an "overlap" may happen (on AIX) because of: + - Shared libraries do export the FDE symbols ("_GLOBAL__F*"), + which is a bug in their build system, but out of gcc's control. + - Other shared libraries, or the main executable, do contain + identical or similar object files - which is suboptimal, + but may be intentional. However, exporting their FDE symbols, + which may have identical names as seen in the former shared + libraries, again is a bug in their build system, but still + out of gcc's control. + - Finally, run time linking is enabled, redirecting adresses of + identically named exported symbols from their original shared + library's address range into another shared library's or the + main executable's address range. + This results in address ranges being registered by different + objects to potentially overlap. */ } /* Classify and search the objects we've not yet processed. */
Re: libgcc: On AIX, increase chances to find landing pads for exceptions
On 02/10/2016 02:27 PM, David Edelsohn wrote: > On Wed, Feb 10, 2016 at 1:52 AM, Michael Haubenwallner > wrote: >> >> On 02/08/2016 02:59 PM, David Edelsohn wrote: >>> Runtime linking is disabled by default on AIX, and I disabled it for >>> libstdc++. >> >> For large applications mainly developed on/for Linux I do prefer/need >> runtime linking even on AIX. Still I do believe there is no AIX-based >> reason to leave runtime linking disabled, but build-/linktime issues >> instead that cause things to fail with runtime linking enabled. > > What do you mean by the term "runtime linking"? Runtime linking means > runtime overloading of symbols -- preloading -- not dynamic linking > and loading. dlopen does not require runtime linking. There also are > issues with searching for shared objects with .a or .so file > extension, but that can be addressed separately. We're actually talking about the same "runtime linking" understanding. > Runtime linking causes every global, exported function call to be > invoked through indirect glue code. And each function must be > inserted into the TOC. The indirect call overhead is very expensive, > and potential TOC overflow can cause even more performance > degradation. Well, I'm in the lucky situation to not need to care for performance that much. Don't get me wrong - I don't want to change any default settings, but keep symbol-overloading at runtime just working if requested for the final executable (by linking with -brtl). What I indeed do not know: Is the performance overhead still active even when the final executable is _not_ linked with -brtl, but the shared libraries only are linked with -G ? > Your statement of no AIX-based reason to leave runtime linking > disabled is fundamentally flawed. Agreed performance wise. But still I fail to see a functional reason. >>> There are two remaining issues: >>> >>> 1) FDEs with overlapping ranges causing problems with exceptions. I'm >>> not sure of the best way to work around this. Your patch is one >>> possible solution. >> >> This patch is not meant as a final solution, but to improve current >> situation with broken build systems exporting even _GLOBAL__ symbols. >> I'm about to prepare another libtool patch to fix that one. >> >>> 2) AIX linker garbage collection conflicting with scanning for >>> symbols. collect2 scanning needs to better emulate SVR4 linker >>> semantics for object files and archives. >> >> Probably collect2 should filter the symbol list originating in either >> an explicit -bexport:file or the -bexpall/-bexpfull flags and pass the >> resulting symbol list as explicit -bexport:file only to the AIX linker? > > -bexpall and -bexpfull cause numerous problem by re-exporting symbols. > > All of the suggestions will produce programs that function, but have > severe performance impacts and unintended consequences that you seem > to be ignoring. Erm... I don't think so - what I mean is: Probably collect2 should _not_ passthrough the -bexpall/-bexpfull/-bexport:file flags to the linker, but itself create an export file and pass this one _instead_. However, this feels like a workaround for broken build systems - which do provide the breaking -bexp* flags to gcc (collect2) already. Thanks! /haubi/ > > - David > >> >> /haubi/ >> >>> >>> Thanks, David >>> >>> >>> On Mon, Feb 8, 2016 at 7:14 AM, Michael Haubenwallner >>> wrote: Hi David, still experiencing exception-not-caught problems with gcc-4.2.4 on AIX leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in 2004 already, ought to be fixed by some different commit since 3.4.0. As long as build systems (even libtool right now) on AIX do export these _GLOBAL__* symbols from shared libraries, overlapping frame-base address ranges may become registered, even if newer gcc (seen with 4.8) does name the FDE symbols more complex to reduce these chances. But still, just think of linking some static library into multiple shared libraries and/or the main executable. Or sometimes there is just need for some hackery to override a shared object's implementation detail and rely on runtime linking to do the override at runtime. Agreed both is "wrong" to some degree, but the larger an application is, the higher is the chance for this to happen. Thoughts? Thanks! /haubi/
Re: libgcc: On AIX, increase chances to find landing pads for exceptions
Hi David, On 02/10/2016 10:52 AM, Michael Haubenwallner wrote: >> There are two remaining issues: >> >> 1) FDEs with overlapping ranges causing problems with exceptions. I'm >> not sure of the best way to work around this. Your patch is one >> possible solution. > > This patch is not meant as a final solution, but to improve current > situation with broken build systems exporting even _GLOBAL__ symbols. > I'm about to prepare another libtool patch to fix that one. so this is the libtool patch I'm about to submit. What do you think? Reasonable? Thanks! /haubi/ >From 18daacef170d0b0ad67869173cd521dc991d2264 Mon Sep 17 00:00:00 2001 From: Michael Haubenwallner Date: Mon, 29 Feb 2016 16:07:32 +0100 Subject: [PATCH] AIX: stop exporting _GLOBAL__ syms * m4/libtool.m4 (_LT_LINKER_SHLIBS): On AIX, GNU g++ generates _GLOBAL__* symbols as landing pads for C++ exceptions. These symbols must not be exported from shared libraries, or exception handling may break for applications with runtime linking enabled. --- m4/libtool.m4 | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/m4/libtool.m4 b/m4/libtool.m4 index ee292af..656d2cd 100644 --- a/m4/libtool.m4 +++ b/m4/libtool.m4 @@ -4939,9 +4939,9 @@ m4_if([$1], [CXX], [ # it in the Import File for the 'aix-soname' feature, so we have # to replace the "-B" option with "-P" for AIX nm. if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then - _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' + _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".") && ([substr](\$ 3,1,9) != "_GLOBAL__")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' else - _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' + _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".") && ([substr](\$ 1,1,9) != "_GLOBAL__")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' fi ;; pw32*) @@ -5394,9 +5394,9 @@ _LT_EOF # it in the Import File for the 'aix-soname' feature, so we have # to replace the "-B" option with "-P" for AIX nm. if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then - _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' + _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".") && ([substr](\$ 3,1,9) != "_GLOBAL__")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' else - _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' + _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "L") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".") && ([substr](\$ 1,1,9) != "_GLOBAL__")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' fi aix_use_runtimelinking=no -- 2.4.6
Re: libgcc: On AIX, increase chances to find landing pads for exceptions
On Tue, Mar 1, 2016 at 7:09 AM, Michael Haubenwallner wrote: > Hi David, > > On 02/10/2016 10:52 AM, Michael Haubenwallner wrote: > >>> There are two remaining issues: >>> >>> 1) FDEs with overlapping ranges causing problems with exceptions. I'm >>> not sure of the best way to work around this. Your patch is one >>> possible solution. >> >> This patch is not meant as a final solution, but to improve current >> situation with broken build systems exporting even _GLOBAL__ symbols. >> I'm about to prepare another libtool patch to fix that one. > > so this is the libtool patch I'm about to submit. > > What do you think? Reasonable? I don't think that Ian really cares about this. I guess that the patch is reasonable, but the libtool command is becoming extremely complicated. - David
Re: libgcc: On AIX, increase chances to find landing pads for exceptions
Runtime linking is disabled by default on AIX, and I disabled it for libstdc++. There are two remaining issues: 1) FDEs with overlapping ranges causing problems with exceptions. I'm not sure of the best way to work around this. Your patch is one possible solution. 2) AIX linker garbage collection conflicting with scanning for symbols. collect2 scanning needs to better emulate SVR4 linker semantics for object files and archives. Thanks, David On Mon, Feb 8, 2016 at 7:14 AM, Michael Haubenwallner wrote: > Hi David, > > still experiencing exception-not-caught problems with gcc-4.2.4 on AIX > leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in > 2004 already, ought to be fixed by some different commit since 3.4.0. > > As long as build systems (even libtool right now) on AIX do export these > _GLOBAL__* symbols from shared libraries, overlapping frame-base address > ranges may become registered, even if newer gcc (seen with 4.8) does name > the FDE symbols more complex to reduce these chances. > > But still, just think of linking some static library into multiple shared > libraries and/or the main executable. Or sometimes there is just need for > some hackery to override a shared object's implementation detail and rely > on runtime linking to do the override at runtime. > > Agreed both is "wrong" to some degree, but the larger an application is, > the higher is the chance for this to happen. > > Thoughts? > > Thanks! > /haubi/
Re: libgcc: On AIX, increase chances to find landing pads for exceptions
On 02/08/2016 02:59 PM, David Edelsohn wrote: > Runtime linking is disabled by default on AIX, and I disabled it for > libstdc++. For large applications mainly developed on/for Linux I do prefer/need runtime linking even on AIX. Still I do believe there is no AIX-based reason to leave runtime linking disabled, but build-/linktime issues instead that cause things to fail with runtime linking enabled. > There are two remaining issues: > > 1) FDEs with overlapping ranges causing problems with exceptions. I'm > not sure of the best way to work around this. Your patch is one > possible solution. This patch is not meant as a final solution, but to improve current situation with broken build systems exporting even _GLOBAL__ symbols. I'm about to prepare another libtool patch to fix that one. > 2) AIX linker garbage collection conflicting with scanning for > symbols. collect2 scanning needs to better emulate SVR4 linker > semantics for object files and archives. Probably collect2 should filter the symbol list originating in either an explicit -bexport:file or the -bexpall/-bexpfull flags and pass the resulting symbol list as explicit -bexport:file only to the AIX linker? /haubi/ > > Thanks, David > > > On Mon, Feb 8, 2016 at 7:14 AM, Michael Haubenwallner > wrote: >> Hi David, >> >> still experiencing exception-not-caught problems with gcc-4.2.4 on AIX >> leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in >> 2004 already, ought to be fixed by some different commit since 3.4.0. >> >> As long as build systems (even libtool right now) on AIX do export these >> _GLOBAL__* symbols from shared libraries, overlapping frame-base address >> ranges may become registered, even if newer gcc (seen with 4.8) does name >> the FDE symbols more complex to reduce these chances. >> >> But still, just think of linking some static library into multiple shared >> libraries and/or the main executable. Or sometimes there is just need for >> some hackery to override a shared object's implementation detail and rely >> on runtime linking to do the override at runtime. >> >> Agreed both is "wrong" to some degree, but the larger an application is, >> the higher is the chance for this to happen. >> >> Thoughts? >> >> Thanks! >> /haubi/
Re: libgcc: On AIX, increase chances to find landing pads for exceptions
On Wed, Feb 10, 2016 at 1:52 AM, Michael Haubenwallner wrote: > > On 02/08/2016 02:59 PM, David Edelsohn wrote: >> Runtime linking is disabled by default on AIX, and I disabled it for >> libstdc++. > > For large applications mainly developed on/for Linux I do prefer/need > runtime linking even on AIX. Still I do believe there is no AIX-based > reason to leave runtime linking disabled, but build-/linktime issues > instead that cause things to fail with runtime linking enabled. What do you mean by the term "runtime linking"? Runtime linking means runtime overloading of symbols -- preloading -- not dynamic linking and loading. dlopen does not require runtime linking. There also are issues with searching for shared objects with .a or .so file extension, but that can be addressed separately. Runtime linking causes every global, exported function call to be invoked through indirect glue code. And each function must be inserted into the TOC. The indirect call overhead is very expensive, and potential TOC overflow can cause even more performance degradation. Your statement of no AIX-based reason to leave runtime linking disabled is fundamentally flawed. > >> There are two remaining issues: >> >> 1) FDEs with overlapping ranges causing problems with exceptions. I'm >> not sure of the best way to work around this. Your patch is one >> possible solution. > > This patch is not meant as a final solution, but to improve current > situation with broken build systems exporting even _GLOBAL__ symbols. > I'm about to prepare another libtool patch to fix that one. > >> 2) AIX linker garbage collection conflicting with scanning for >> symbols. collect2 scanning needs to better emulate SVR4 linker >> semantics for object files and archives. > > Probably collect2 should filter the symbol list originating in either > an explicit -bexport:file or the -bexpall/-bexpfull flags and pass the > resulting symbol list as explicit -bexport:file only to the AIX linker? -bexpall and -bexpfull cause numerous problem by re-exporting symbols. All of the suggestions will produce programs that function, but have severe performance impacts and unintended consequences that you seem to be ignoring. - David > > /haubi/ > >> >> Thanks, David >> >> >> On Mon, Feb 8, 2016 at 7:14 AM, Michael Haubenwallner >> wrote: >>> Hi David, >>> >>> still experiencing exception-not-caught problems with gcc-4.2.4 on AIX >>> leads me to some patch proposed in http://gcc.gnu.org/PR13878 back in >>> 2004 already, ought to be fixed by some different commit since 3.4.0. >>> >>> As long as build systems (even libtool right now) on AIX do export these >>> _GLOBAL__* symbols from shared libraries, overlapping frame-base address >>> ranges may become registered, even if newer gcc (seen with 4.8) does name >>> the FDE symbols more complex to reduce these chances. >>> >>> But still, just think of linking some static library into multiple shared >>> libraries and/or the main executable. Or sometimes there is just need for >>> some hackery to override a shared object's implementation detail and rely >>> on runtime linking to do the override at runtime. >>> >>> Agreed both is "wrong" to some degree, but the larger an application is, >>> the higher is the chance for this to happen. >>> >>> Thoughts? >>> >>> Thanks! >>> /haubi/