Folks, more on this topic:
This ---------8<-------------- (gdb) x/5i 0x45fd3efc 0x45fd3efc: add r12, pc, #1048576 ; 0x100000 0x45fd3f00: add r12, r12, #4096 ; 0x1000 0x45fd3f04: ldr pc, [r12, #1652]! ; 0x674 0x45fd3f08: add r12, pc, #1048576 ; 0x100000 0x45fd3f0c: add r12, r12, #4096 ; 0x1000 --------------------------- is not garbage. It's kind of a link table (plt) inside of glib2. It makes the first 2 calls in g_dpgettext2() go to strlen() and the third go to memcpy(). memcpy() saves the r0 and lr regisers at the very beginning and pops them into r0 and pc at a later time in order to return to the caller. However somewhere in the function, the stack data seems to become corrupted and a junk value is loaded into pc, leading to the crash. Either its value was overwritten or sp is pointing to the wrong place. (gdb) x/50i 0x45e80f98 0x45e80f98 <memcpy>: push {r0, lr} =====> SAVE REGISTERS 0x45e80f9c <memcpy+4>: subs r2, r2, #4 0x45e80fa0 <memcpy+8>: blt 0x45e81028 <memcpy+144> 0x45e80fa4 <memcpy+12>: ands r12, r0, #3 0x45e80fa8 <memcpy+16>: bne 0x45e81050 <memcpy+184> 0x45e80fac <memcpy+20>: ands r12, r1, #3 0x45e80fb0 <memcpy+24>: bne 0x45e81080 <memcpy+232> 0x45e80fb4 <memcpy+28>: subs r2, r2, #8 0x45e80fb8 <memcpy+32>: blt 0x45e81008 <memcpy+112> 0x45e80fbc <memcpy+36>: subs r2, r2, #20 0x45e80fc0 <memcpy+40>: blt 0x45e80ff4 <memcpy+92> 0x45e80fc4 <memcpy+44>: push {r4} ; (str r4, [sp, #-4]!) 0x45e80fc8 <memcpy+48>: ldm r1!, {r3, r4, r12, lr} 0x45e80fcc <memcpy+52>: stmia r0!, {r3, r4, r12, lr} 0x45e80fd0 <memcpy+56>: ldm r1!, {r3, r4, r12, lr} 0x45e80fd4 <memcpy+60>: stmia r0!, {r3, r4, r12, lr} 0x45e80fd8 <memcpy+64>: subs r2, r2, #32 0x45e80fdc <memcpy+68>: bge 0x45e80fc8 <memcpy+48> 0x45e80fe0 <memcpy+72>: cmn r2, #16 0x45e80fe4 <memcpy+76>: ldmge r1!, {r3, r4, r12, lr} 0x45e80fe8 <memcpy+80>: stmiage r0!, {r3, r4, r12, lr} 0x45e80fec <memcpy+84>: subge r2, r2, #16 0x45e80ff0 <memcpy+88>: pop {r4} ; (ldr r4, [sp], #4) 0x45e80ff4 <memcpy+92>: adds r2, r2, #20 0x45e80ff8 <memcpy+96>: ldmge r1!, {r3, r12, lr} 0x45e80ffc <memcpy+100>: stmiage r0!, {r3, r12, lr} 0x45e81000 <memcpy+104>: subsge r2, r2, #12 0x45e81004 <memcpy+108>: bge 0x45e80ff8 <memcpy+96> 0x45e81008 <memcpy+112>: adds r2, r2, #8 0x45e8100c <memcpy+116>: blt 0x45e81028 <memcpy+144> 0x45e81010 <memcpy+120>: subs r2, r2, #4 0x45e81014 <memcpy+124>: ldrlt r3, [r1], #4 0x45e81018 <memcpy+128>: strlt r3, [r0], #4 0x45e8101c <memcpy+132>: ldmge r1!, {r3, r12} 0x45e81020 <memcpy+136>: stmiage r0!, {r3, r12} 0x45e81024 <memcpy+140>: subge r2, r2, #4 0x45e81028 <memcpy+144>: adds r2, r2, #4 0x45e8102c <memcpy+148>: popeq {r0, pc} 0x45e81030 <memcpy+152>: cmp r2, #2 0x45e81034 <memcpy+156>: ldrb r3, [r1], #1 0x45e81038 <memcpy+160>: strb r3, [r0], #1 0x45e8103c <memcpy+164>: ldrbge r3, [r1], #1 0x45e81040 <memcpy+168>: strbge r3, [r0], #1 0x45e81044 <memcpy+172>: ldrbgt r3, [r1], #1 0x45e81048 <memcpy+176>: strbgt r3, [r0], #1 => 0x45e8104c <memcpy+180>: pop {r0, pc} ======> CRASH: PC gets corrupted value 0x45e81050 <memcpy+184>: rsb r12, r12, #4 Further investigation is to come ;) 2015-05-31 19:05 GMT+00:00 Jared McNeill <jmcne...@invisible.ca>: > I saw the same issue a while back, don't remember exactly which version of > glib2 it was. I LD_PRELOAD'd a version of g_dpgettext2 that always returned > NULL and it made the crashes go away. g_dpgettext2 uses alloca internally, > so I increased stack size and the crashes went away again. > > https://git.gnome.org/browse/glib/tree/glib/ggettext.c#n271 > > > On Sun, 31 May 2015, Stephan wrote: > >> >> That does not help unfortunately - just tested again with ulimit -s >> unlimited. I dont think we are hitting the stack boundary >> because sp is still inside the stack region outlined by pmap. Have you >> seen the very same issue as I do? >> >> Am 31.05.2015 19:48 schrieb "Jared McNeill" <jmcne...@invisible.ca>: >> Default stack size limits on evbarm are too low for webkit-gtk. Bump >> it up with ulimit -s and the g_dpgettext2 >> problem should go away. >> >> >> >> On Sun, 31 May 2015, Stephan wrote: >> >> Hi folks, >> >> I am currently testing some applications on the RPI 2. Some >> work >> pretty well, others not yet. As for webkit-gtk based browsers, >> I am >> experiencing crashes from time to time. >> >> One problem that occurs often seems to be related to >> g_dpgettext2 () >> from glib2. The top of the stack looks like this: >> >> (gdb) bt >> #0 0x636f7452 in ?? () >> #1 0x45ff3fa8 in g_dpgettext2 () from >> /usr/pkg/lib/libglib-2.0.so.0 >> #2 0x42ad6030 in gtk_stock_lookup () from >> /usr/pkg/lib/libgtk-x11-2.0.so.0 >> #3 0x42987b98 in gtk_action_set_stock_id () from >> /usr/pkg/lib/libgtk-x11-2.0.so.0 >> #4 0x45f55cfc in g_object_set_valist () from >> /usr/pkg/lib/libgobject-2.0.so.0 >> #5 0x45f5642c in g_object_set () from >> /usr/pkg/lib/libgobject-2.0.so.0 >> #6 0x4298a27c in gtk_action_group_add_actions_full () from >> /usr/pkg/lib/libgtk-x11-2.0.so.0 >> #7 0x4298a388 in gtk_action_group_add_actions () from >> /usr/pkg/lib/libgtk-x11-2.0.so.0 >> #8 0x0004238c in ?? () >> #9 0x45f5322c in g_object_new_internal () from >> /usr/pkg/lib/libgobject-2.0.so.0 >> #10 0x45f5587c in g_object_new_valist () from >> /usr/pkg/lib/libgobject-2.0.so.0 >> #11 0x45f55a24 in g_object_new () from >> /usr/pkg/lib/libgobject-2.0.so.0 >> #12 0x00043ff0 in ephy_window_new_with_chrome () >> #13 0x0003ac94 in ephy_shell_new_tab_full () >> #14 0x0003f81c in ?? () >> #15 0x40a090e8 in webkit_marshal_OBJECT__OBJECT () from >> /usr/pkg/lib/libwebkitgtk-1.0.so.0 >> #16 0x45f4e070 in g_closure_invoke () from >> /usr/pkg/lib/libgobject-2.0.so.0 >> #17 0x45f6154c in signal_emit_unlocked_R () from >> /usr/pkg/lib/libgobject-2.0.so.0 >> #18 0x45f69278 in g_signal_emit_valist () from >> /usr/pkg/lib/libgobject-2.0.so.0 >> #19 0x45f69cac in g_signal_emit_by_name () from >> /usr/pkg/lib/libgobject-2.0.so.0 >> #20 0x409d9074 in >> >> WebKit::FrameLoaderClient::dispatchCreatePage(WebCore::NavigationAction >> const&) () from /usr/pkg/lib/libwebkitgtk-1.0.so.0 >> ... >> >> The call to the topmost frame happens from 0x45ff3fa4 in the >> said >> funtion. The disassembly from within gdb loos like this: >> >> (gdb) disass g_dpgettext2 >> Dump of assembler code for function g_dpgettext2: >> 0x45ff3f44 <+0>: mov r12, sp >> 0x45ff3f48 <+4>: push {r4, r5, r6, r7, r8, r9, r10, >> r11, r12, lr, pc} >> 0x45ff3f4c <+8>: sub r11, r12, #4 >> 0x45ff3f50 <+12>: sub sp, sp, #12 >> 0x45ff3f54 <+16>: mov r9, r0 >> 0x45ff3f58 <+20>: mov r0, r1 >> 0x45ff3f5c <+24>: mov r6, r2 >> 0x45ff3f60 <+28>: str r1, [r11, #-48] ; 0x30 >> 0x45ff3f64 <+32>: bl 0x45fd3f44 >> 0x45ff3f68 <+36>: mov r5, r0 >> 0x45ff3f6c <+40>: mov r0, r6 >> 0x45ff3f70 <+44>: bl 0x45fd3f44 >> 0x45ff3f74 <+48>: add r10, r5, #1 >> 0x45ff3f78 <+52>: add r8, r0, #1 >> 0x45ff3f7c <+56>: add r3, r8, r10 >> 0x45ff3f80 <+60>: add r3, r3, #14 >> 0x45ff3f84 <+64>: bic r3, r3, #7 >> 0x45ff3f88 <+68>: sub sp, sp, r3 >> 0x45ff3f8c <+72>: mov r3, sp >> 0x45ff3f90 <+76>: lsr r7, r3, #3 >> 0x45ff3f94 <+80>: lsl r4, r7, #3 >> 0x45ff3f98 <+84>: ldr r1, [r11, #-48] ; 0x30 >> 0x45ff3f9c <+88>: mov r2, r5 >> 0x45ff3fa0 <+92>: mov r0, r4 >> 0x45ff3fa4 <+96>: bl 0x45fd3efc >> 0x45ff3fa8 <+100>: mov r3, #4 >> 0x45ff3fac <+104>: mov r2, r8 >> 0x45ff3fb0 <+108>: strb r3, [r5, r7, lsl #3] >> ... >> >> Now the jump target actually seems to be some random memory: >> >> (gdb) info symbol 0x45fd3efc >> No symbol matches 0x45fd3efc. >> >> At least it is mapped to the text section of libglib2. It >> decodes like >> follows which just looks like random garbage: >> >> (gdb) x/5i 0x45fd3efc >> 0x45fd3efc: add r12, pc, #1048576 ; 0x100000 >> 0x45fd3f00: add r12, r12, #4096 ; 0x1000 >> 0x45fd3f04: ldr pc, [r12, #1652]! ; 0x674 >> 0x45fd3f08: add r12, pc, #1048576 ; 0x100000 >> 0x45fd3f0c: add r12, r12, #4096 ; 0x1000 >> >> The disassembly of g_dpgettext2() from objdump looks weird as >> well: >> >> 00033f44 <g_dpgettext2>: >> 33f44: e1a0c00d mov ip, sp >> 33f48: e92ddff0 push {r4, r5, r6, r7, r8, >> r9, sl, >> fp, ip, lr, pc} >> 33f4c: e24cb004 sub fp, ip, #4 >> 33f50: e24dd00c sub sp, sp, #12 >> 33f54: e1a09000 mov r9, r0 >> 33f58: e1a00001 mov r0, r1 >> 33f5c: e1a06002 mov r6, r2 >> 33f60: e50b1030 str r1, [fp, #-48] ; 0x30 >> 33f64: ebff7ff6 bl 13f44 >> <g_mem_chunk_new-0x7c> >> 33f68: e1a05000 mov r5, r0 >> 33f6c: e1a00006 mov r0, r6 >> 33f70: ebff7ff3 bl 13f44 >> <g_mem_chunk_new-0x7c> >> 33f74: e285a001 add sl, r5, #1 >> 33f78: e2808001 add r8, r0, #1 >> 33f7c: e088300a add r3, r8, sl >> 33f80: e283300e add r3, r3, #14 >> 33f84: e3c33007 bic r3, r3, #7 >> 33f88: e04dd003 sub sp, sp, r3 >> 33f8c: e1a0300d mov r3, sp >> 33f90: e1a071a3 lsr r7, r3, #3 >> 33f94: e1a04187 lsl r4, r7, #3 >> 33f98: e51b1030 ldr r1, [fp, #-48] ; 0x30 >> 33f9c: e1a02005 mov r2, r5 >> 33fa0: e1a00004 mov r0, r4 >> 33fa4: ebff7fd4 bl 13efc >> <g_mem_chunk_new-0xc4> >> 33fa8: e3a03004 mov r3, #4 >> 33fac: e1a02008 mov r2, r8 >> 33fb0: e7c53187 strb r3, [r5, r7, lsl #3] >> 33fb4: e1a01006 mov r1, r6 >> 33fb8: e084000a add r0, r4, sl >> 33fbc: ebff7fce bl 13efc >> <g_mem_chunk_new-0xc4> >> 33fc0: e1a00009 mov r0, r9 >> 33fc4: e1a01004 mov r1, r4 >> 33fc8: ebffff8a bl 33df8 <g_dgettext> >> 33fcc: e1500004 cmp r0, r4 >> 33fd0: e1a08000 mov r8, r0 >> >> The jumps refer to negative function offsets (for example >> <g_mem_chunk_new-0x7c>). Glibc (2.42.2) is from pkgsrc from >> ftp.netbsd.org. I compiled an own one which has exactly the >> same >> issue. >> >> Ideas? >> >> >> >