[I've found the problem at the low level for my context of using WITH_LIBCPLUSPLUS= with the likes of devel/powerpc64-gcc but I do not have a solution for WITH_LIBCPLUSPLUS= so far. I give details of what I found.]
On 2018-Oct-14, at 12:40 AM, Mark Millard <marklmi at yahoo.com> wrote: > On 2018-Oct-12, at 1:59 PM, Mark Millard <marklmi at yahoo.com> wrote: > >> I built a powerpc64 head -r339076 via devel/powerpc64-gcc >> and the like that built clang as cc as well (and >> WITHOUT_LIB32). This included use of base/binutils to >> the the powerpc64 set up. The system and kernel are >> non-debug builds (but with symbols). [system-clang is not >> used for buildworld buildkernel because of known >> issues (last I tried).] >> >> booting, buildworld, buildkernel, poudriere building >> what totaled to be somewhat under 400 ports all seem >> to work. But . . . >> >> It been a long time since I've done something analogous >> and a significant item in the result is different than in >> the past once I started testing the throwing of C++ >> exceptions in code produced by system-clang or by >> devel/powerpc64-gcc : >> >> Such code ends up stuck using around 100% of a CPU. >> An example is the program: >> >> # more exception_test.cpp >> #include <exception> >> >> int main(void) >> { >> try { throw std::exception(); } >> catch (std::exception& e) {} >> return 0; >> } >> >> For system-clang it ended up with: >> >> # ldd a.out >> a.out: >> libc++.so.1 => /usr/lib/libc++.so.1 (0x81006d000) >> libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x810184000) >> libm.so.5 => /lib/libm.so.5 (0x8101ab000) >> libc.so.7 => /lib/libc.so.7 (0x8101eb000) >> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x810554000) >> >> That program goes into an possibly unbounded execution. >> (Historically when this program had problems it would >> stop and produce a core file.) >> >> When compiled by devel/powerpc64-gcc the a.out that results >> does the same thing. ( /usr/local/bin/powerpc64-unknown-freebsd12.0-c++ >> as the compiler path ) So this is not really clang specific >> in any way. This ended up with: >> >> # ldd a.out >> a.out: >> libc++.so.1 => /usr/lib/libc++.so.1 (0x81006d000) >> libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x810184000) >> libm.so.5 => /lib/libm.so.5 (0x8101ab000) >> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x8101eb000) >> libc.so.7 => /lib/libc.so.7 (0x810211000) >> >> (That should not have involved clang or llvm at all.) >> >> But compiled by lang/gcc8's g++8 the a.out that results works >> fine. This ends up with: >> >> # ldd a.out >> a.out: >> libstdc++.so.6 => /usr/local/lib/gcc8/libstdc++.so.6 (0x81006e000) >> libm.so.5 => /lib/libm.so.5 (0x8102c7000) >> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x810307000) >> libc.so.7 => /lib/libc.so.7 (0x81032d000) >> >> It is not clear if using base/gcc as system cc >> would do any better than using system-clang does >> or than devel/powerpc64-gcc does: it is sort of >> a variant of devel/powerpc64-gcc . >> >> It will probably be some time before I figure out >> much about what is going on. >> >> Two things common to the problem cases are: >> >> libc++.so.1 => /usr/lib/libc++.so.1 (0x81006d000) >> libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x810184000) >> >> lang/gcc8 avoids those being involved. >> >> >> Notes: >> >> . . . >> >> WITHOUT_LIB32= is because, for every post-gcc 4.2.1 >> that I've tried, the lib32 produced misuses R30 in >> crtbeginS code (vs. the ABI for FreeBSD) and 32-bit >> code just produces core files from the bad so-called >> address dereference that results. >> >> I'd rather have throwing C++ exceptions working and >> lack of lib32 than have lib32 but not have throwing >> C++ exceptions working. But at the moment how to have >> such is not obvious when fairly modern compilers >> and toolchains are involved. > > Here is what I've found so far. > > The code is looping in the following routine. > (I've inserted 2 NOTE: lines for what the > sustained looping is like.) > . . . (the routine was _Unwind_RaiseException) . . . So far I've found that the following in _Unwind_RaiseException stays invariant once initialized --despite the uw_frame_state_for and uw_update_context calls in _Unwind_RaiseException 's loop that normally update fs: (gdb) print fs $15 = {regs = {reg = {{loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED} <repeats 31 times>, {loc = {reg = 18446744073709551608, offset = -8, exp = 0xfffffffffffffff8 <Error reading address 0xfffffffffffffff8: Bad address>}, how = REG_SAVED_OFFSET}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED} <repeats 33 times>, { loc = {reg = 16, offset = 16, exp = 0x10 <Error reading address 0x10: Bad address>}, how = REG_SAVED_OFFSET}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED} <repeats 80 times>}, prev = 0x0}, cfa_offset = 0, cfa_reg = 1, cfa_exp = 0x0, cfa_how = CFA_REG_OFFSET, pc = 0x8101999f8, personality = 0, data_align = -8, code_align = 4, retaddr_column = 65, fde_encoding = 27 '\033', lsda_encoding = 255 '?', saw_z = 1 '\001', signal_frame = 0 '\0', eh_ptr = 0x0} It turns out that pc value 0x8101999f8 is a little after the libcxxrt.so call to _Unwind_RaiseException that is in throw_exception. But _Unwind_RaiseException returning would be a failure and would end up in a non-returning, error-reporting code path. In other words: this is not an appropriate context for following the return path to unwind out of _Unwind_RaiseException and its internal caller (throw_exception). It got to that address from lr containing the address of the instruction after the one that does bl to the _Unwind_RaiseException routine. Overall it needs to unwind past this in the normal case but is stuck handling the error/no-return case as "the" case. Supporting details follow. What lead up to 0x8101999f8 for initialization was the lr value related to calling _Unwind_RaiseException (see the address in lr below, also copied to r5): . . . (gdb) c Continuing. Breakpoint 9, <signal handler called> 1: x/i $pc 0x8101f35d8 <_Unwind_RaiseException+216>: bl 0x8101f2bc0 <uw_init_context_1> Current language: auto; currently minimal where the register values being supplied are (see r5 and lr): (gdb) info reg r0 0x8101999f0 34629851632 r1 0x3fffffffffffc320 4611686018427372320 r2 0x810217900 34630367488 r3 0x3fffffffffffd280 4611686018427376256 r4 0x3fffffffffffd930 4611686018427377968 r5 0x8101999f0 34629851632 r6 0xa0 160 r7 0x0 0 r8 0x1 1 r9 0x8101aac10 34629921808 r10 0x1 1 r11 0x28 40 r12 0x28000282 671089282 r13 0x81005d020 34628554784 r14 0x0 0 r15 0x0 0 r16 0x0 0 r17 0x0 0 r18 0x0 0 r19 0x0 0 r20 0x0 0 r21 0x0 0 r22 0x0 0 r23 0x0 0 r24 0x0 0 r25 0x0 0 r26 0x0 0 r27 0x3fffffffffffd280 4611686018427376256 r28 0x810041060 34628440160 r29 0x3fffffffffffc390 4611686018427372432 r30 0x3fffffffffffcd10 4611686018427374864 r31 0x810041008 34628440072 pc 0x8101f35d8 34630219224 ps 0x0 0 cr 0x0 0 lr 0x8101999f0 34629851632 ctr 0x8101f3500 34630219008 xer 0x0 0 fpscr 0xfff80000 -524288 vscr 0x0 0 vrsave 0x0 0 The pc listed in print fs (0x8101999f8) is in the following from libcxxrt, as is the value in r5 and lr (0x8101999f0): (some blank lines inserted to help identify the area and some related material) (gdb) disass throw_exception Dump of assembler code for function throw_exception: 0x0000000810199960 <throw_exception+0>: mflr r0 0x0000000810199964 <throw_exception+4>: std r31,-8(r1) 0x0000000810199968 <throw_exception+8>: mr r31,r3 0x000000081019996c <throw_exception+12>: std r0,16(r1) 0x0000000810199970 <throw_exception+16>: stdu r1,-128(r1) 0x0000000810199974 <throw_exception+20>: bl 0x810197ab0 <thread_info> 0x0000000810199978 <throw_exception+24>: ld r10,8(r3) 0x000000081019997c <throw_exception+28>: mr r9,r3 0x0000000810199980 <throw_exception+32>: cmpdi cr7,r10,0 0x0000000810199984 <throw_exception+36>: std r10,24(r31) 0x0000000810199988 <throw_exception+40>: beq- cr7,0x810199a10 <throw_exception+176> 0x000000081019998c <throw_exception+44>: ld r10,0(r9) 0x0000000810199990 <throw_exception+48>: cmpdi cr7,r10,0 0x0000000810199994 <throw_exception+52>: std r10,32(r31) 0x0000000810199998 <throw_exception+56>: beq- cr7,0x8101999d0 <throw_exception+112> 0x000000081019999c <throw_exception+60>: lwz r10,48(r9) 0x00000008101999a0 <throw_exception+64>: addi r3,r31,88 0x00000008101999a4 <throw_exception+68>: addi r10,r10,1 0x00000008101999a8 <throw_exception+72>: stw r10,48(r9) 0x00000008101999ac <throw_exception+76>: bl 0x81018e500 <00000018.plt_call._Unwind_RaiseException@@GCC_3.0> 0x00000008101999b0 <throw_exception+80>: ld r2,40(r1) 0x00000008101999b4 <throw_exception+84>: addi r1,r1,128 0x00000008101999b8 <throw_exception+88>: mr r4,r31 0x00000008101999bc <throw_exception+92>: ld r0,16(r1) 0x00000008101999c0 <throw_exception+96>: ld r31,-8(r1) 0x00000008101999c4 <throw_exception+100>: mtlr r0 0x00000008101999c8 <throw_exception+104>: b 0x8101996b0 <report_failure> 0x00000008101999cc <throw_exception+108>: nop 0x00000008101999d0 <throw_exception+112>: nop 0x00000008101999d4 <throw_exception+116>: addi r3,r31,88 0x00000008101999d8 <throw_exception+120>: ld r10,-30008(r2) 0x00000008101999dc <throw_exception+124>: std r10,32(r31) 0x00000008101999e0 <throw_exception+128>: lwz r10,48(r9) 0x00000008101999e4 <throw_exception+132>: addi r10,r10,1 0x00000008101999e8 <throw_exception+136>: stw r10,48(r9) 0x00000008101999ec <throw_exception+140>: bl 0x81018e500 <00000018.plt_call._Unwind_RaiseException@@GCC_3.0> 0x00000008101999f0 <throw_exception+144>: ld r2,40(r1) 0x00000008101999f4 <throw_exception+148>: addi r1,r1,128 0x00000008101999f8 <throw_exception+152>: mr r4,r31 0x00000008101999fc <throw_exception+156>: ld r0,16(r1) 0x0000000810199a00 <throw_exception+160>: ld r31,-8(r1) 0x0000000810199a04 <throw_exception+164>: mtlr r0 0x0000000810199a08 <throw_exception+168>: b 0x8101996b0 <report_failure> 0x0000000810199a0c <throw_exception+172>: nop 0x0000000810199a10 <throw_exception+176>: nop 0x0000000810199a14 <throw_exception+180>: ld r10,-30000(r2) 0x0000000810199a18 <throw_exception+184>: std r10,24(r31) 0x0000000810199a1c <throw_exception+188>: b 0x81019998c <throw_exception+44> 0x0000000810199a20 <throw_exception+192>: .long 0x0 0x0000000810199a24 <throw_exception+196>: .long 0x90001 0x0000000810199a28 <throw_exception+200>: lwz r0,0(r1) End of assembler dump. For: 0x00000008101999f8 (above) objdump shows: 00000000000159f8 (below): 0000000000015960 <.__cxa_end_catch+0x460> mflr r0 0000000000015964 <.__cxa_end_catch+0x464> std r31,-8(r1) 0000000000015968 <.__cxa_end_catch+0x468> mr r31,r3 000000000001596c <.__cxa_end_catch+0x46c> std r0,16(r1) 0000000000015970 <.__cxa_end_catch+0x470> stdu r1,-128(r1) 0000000000015974 <.__cxa_end_catch+0x474> bl 0000000000013ab0 <._ZdaPv+0x590> 0000000000015978 <.__cxa_end_catch+0x478> ld r10,8(r3) 000000000001597c <.__cxa_end_catch+0x47c> mr r9,r3 0000000000015980 <.__cxa_end_catch+0x480> cmpdi cr7,r10,0 0000000000015984 <.__cxa_end_catch+0x484> std r10,24(r31) 0000000000015988 <.__cxa_end_catch+0x488> beq cr7,0000000000015a10 <.__cxa_end_catch+0x510> 000000000001598c <.__cxa_end_catch+0x48c> ld r10,0(r9) 0000000000015990 <.__cxa_end_catch+0x490> cmpdi cr7,r10,0 0000000000015994 <.__cxa_end_catch+0x494> std r10,32(r31) 0000000000015998 <.__cxa_end_catch+0x498> beq cr7,00000000000159d0 <.__cxa_end_catch+0x4d0> 000000000001599c <.__cxa_end_catch+0x49c> lwz r10,48(r9) 00000000000159a0 <.__cxa_end_catch+0x4a0> addi r3,r31,88 00000000000159a4 <.__cxa_end_catch+0x4a4> addi r10,r10,1 00000000000159a8 <.__cxa_end_catch+0x4a8> stw r10,48(r9) 00000000000159ac <.__cxa_end_catch+0x4ac> bl 000000000000a500 <CXXABI_1.3@@CXXABI_1.3+0xa500> 00000000000159b0 <.__cxa_end_catch+0x4b0> ld r2,40(r1) 00000000000159b4 <.__cxa_end_catch+0x4b4> addi r1,r1,128 00000000000159b8 <.__cxa_end_catch+0x4b8> mr r4,r31 00000000000159bc <.__cxa_end_catch+0x4bc> ld r0,16(r1) 00000000000159c0 <.__cxa_end_catch+0x4c0> ld r31,-8(r1) 00000000000159c4 <.__cxa_end_catch+0x4c4> mtlr r0 00000000000159c8 <.__cxa_end_catch+0x4c8> b 00000000000156b0 <.__cxa_end_catch+0x1b0> 00000000000159cc <.__cxa_end_catch+0x4cc> nop 00000000000159d0 <.__cxa_end_catch+0x4d0> nop 00000000000159d4 <.__cxa_end_catch+0x4d4> addi r3,r31,88 00000000000159d8 <.__cxa_end_catch+0x4d8> ld r10,-30008(r2) 00000000000159dc <.__cxa_end_catch+0x4dc> std r10,32(r31) 00000000000159e0 <.__cxa_end_catch+0x4e0> lwz r10,48(r9) 00000000000159e4 <.__cxa_end_catch+0x4e4> addi r10,r10,1 00000000000159e8 <.__cxa_end_catch+0x4e8> stw r10,48(r9) 00000000000159ec <.__cxa_end_catch+0x4ec> bl 000000000000a500 <CXXABI_1.3@@CXXABI_1.3+0xa500> 00000000000159f0 <.__cxa_end_catch+0x4f0> ld r2,40(r1) 00000000000159f4 <.__cxa_end_catch+0x4f4> addi r1,r1,128 00000000000159f8 <.__cxa_end_catch+0x4f8> mr r4,r31 00000000000159fc <.__cxa_end_catch+0x4fc> ld r0,16(r1) 0000000000015a00 <.__cxa_end_catch+0x500> ld r31,-8(r1) 0000000000015a04 <.__cxa_end_catch+0x504> mtlr r0 0000000000015a08 <.__cxa_end_catch+0x508> b 00000000000156b0 <.__cxa_end_catch+0x1b0> 0000000000015a0c <.__cxa_end_catch+0x50c> nop 0000000000015a10 <.__cxa_end_catch+0x510> nop 0000000000015a14 <.__cxa_end_catch+0x514> ld r10,-30000(r2) 0000000000015a18 <.__cxa_end_catch+0x518> std r10,24(r31) 0000000000015a1c <.__cxa_end_catch+0x51c> b 000000000001598c <.__cxa_end_catch+0x48c> 0000000000015a20 <.__cxa_end_catch+0x520> .long 0x0 0000000000015a24 <.__cxa_end_catch+0x524> .long 0x90001 0000000000015a28 <.__cxa_end_catch+0x528> lwz r0,0(r1) And dwarfdump shows starting at 0x00015960 as: < 117><0x00015960:0x00015a2c><><cie offset 0x0000101c::cie index 0><fde offset 0x00001018 length: 0x00000030> <eh aug data len 0x0> 0x00015960: <off cfa=00(r1) > 0x00015968: <off cfa=00(r1) > <off r31=-8(cfa) > <off r65=r0 > 0x00015974: <off cfa=128(r1) > <off r31=-8(cfa) > <off r65=16(cfa) > 0x000159b8: <off cfa=00(r1) > <off r31=-8(cfa) > <off r65=16(cfa) > 0x000159c8: <off cfa=00(r1) > 0x000159d0: <off cfa=128(r1) > <off r31=-8(cfa) > <off r65=16(cfa) > 0x000159f8: <off cfa=00(r1) > <off r31=-8(cfa) > <off r65=16(cfa) > 0x00015a08: <off cfa=00(r1) > 0x00015a10: <off cfa=128(r1) > <off r31=-8(cfa) > <off r65=16(cfa) > fde section offset 4120 0x00001018 cie offset for fde: 4124 0x0000101c 0 DW_CFA_advance_loc 8 (2 * 4) 1 DW_CFA_register r65 = r0 4 DW_CFA_offset r31 -8 (1 * -8) 6 DW_CFA_advance_loc 12 (3 * 4) 7 DW_CFA_def_cfa_offset 128 10 DW_CFA_offset_extended_sf r65 16 (-2 * -8) 13 DW_CFA_advance_loc 68 (17 * 4) 14 DW_CFA_remember_state 15 DW_CFA_def_cfa_offset 0 17 DW_CFA_advance_loc 16 (4 * 4) 18 DW_CFA_restore_extended r65 20 DW_CFA_restore r31 21 DW_CFA_advance_loc 8 (2 * 4) 22 DW_CFA_restore_state 23 DW_CFA_advance_loc 40 (10 * 4) 24 DW_CFA_remember_state 25 DW_CFA_def_cfa_offset 0 27 DW_CFA_advance_loc 16 (4 * 4) 28 DW_CFA_restore_extended r65 30 DW_CFA_restore r31 31 DW_CFA_advance_loc 8 (2 * 4) 32 DW_CFA_restore_state 33 DW_CFA_nop 34 DW_CFA_nop /usr/src/contrib/libstdc++/libsupc++/eh_throw.cc has something that /usr/src/contrib/libcxxrt/exception.cc does not have in its error handling code path: a use of __cxa_begin_catch in __cxxabiv1::__cxa_throw : #ifdef _GLIBCXX_SJLJ_EXCEPTIONS _Unwind_SjLj_RaiseException (&header->unwindHeader); #else _Unwind_RaiseException (&header->unwindHeader); #endif // Some sort of unwinding error. Note that terminate is a handler. __cxa_begin_catch (&header->unwindHeader); std::terminate (); It looks to me like __cxa_begin_catch might do either of terminate or _Unwind_Resume and that the (conditional) _Unwind_Resume case is possibly needed here for the normal execution. There are also no other calls (bl's) before the terminate: more directly indicated to not return. I do not know if one or both of those points helped the code unwind correctly or not. But it seems suggestive. Other notes: I've demonstrated the problem on my prior head -r333594 environment that had been build via a 6.3 vintage of devel/powerpc64-gcc (but that was then updated to ports -r469844 and so had 6.4 of devel/powerpc64-gcc installed). Also: base/binutils was of a 2.30 vintage and base/gcc was of a 6.3 vintage. (system-clang was not cc, base/gcc provided system-cc.) Compiling to produce the a.out via: /usr/bin/powerpc64-unknown-freebsd12.0-g++ (and so via a 6.3 vintage g++) made no difference. It still had the problem. I have taken to having buildworld buildkernel use -gdwarf-2 so that /usr/libexec/gdb has access to the information in a format it is (mostly) designed for. (/usr/local/bin/gdb is broken by the thrown-C++-exception problem that I'm investigating: gdb internally throws exceptions in normal operation.) === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) _______________________________________________ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"