[
https://issues.apache.org/jira/browse/DISPATCH-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334743#comment-17334743
]
michael goulish commented on DISPATCH-2088:
-------------------------------------------
Here you go!
(gdb) thread apply all bt
{color:#172b4d}Thread 33{color} (Thread 0x7fa320ff9640 (LWP 53393)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2f0000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
{color:#172b4d}Thread 32{color} (Thread 0x7fa2e8ff9640 (LWP 53408)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2a8000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
--Type <RET> for more, q to quit, c to continue without paging--
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
{color:#172b4d}Thread 31{color} (Thread 0x7fa2e37fe640 (LWP 53409)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2bc000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 30 (Thread 0x7fa30effd640 (LWP 53396)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa300000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
--Type <RET> for more, q to quit, c to continue without paging--
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 29 (Thread 0x7fa30f7fe640 (LWP 53395)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2fc000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 28 (Thread 0x7fa30cff9640 (LWP 53400)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2e4000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-di--Type <RET> for more, q to quit, c to continue
without paging--c
spatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
*{color:#de350b}Thread 27{color}* (Thread 0x7fa2eb7fe640 (LWP 53403)):
#0 0x00007fa343d8350c in send () from /lib64/libpthread.so.0
#1 0x00007fa343dbe718 in snd (s=512, b=<optimized out>, fd=25) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll_raw_connection.c:333
#2 pni_raw_write (send=<optimized out>, set_error=<optimized out>,
sock=<optimized out>, conn=<optimized out>) at
/home/mick/latest/qpid-proton/c/src/proactor/raw_connection.c:566
#3 pni_raw_write (send=<optimized out>, set_error=<optimized out>, sock=25,
conn=0x7fa2dc129cf0) at
/home/mick/latest/qpid-proton/c/src/proactor/raw_connection.c:554
#4 pni_raw_connection_process (sched_ready=<optimized out>, t=0x7fa2dc129c30)
at /home/mick/latest/qpid-proton/c/src/proactor/epoll_raw_connection.c:388
#5 process (tsk=0x7fa2dc129c30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2230
#6 next_event_batch (p=<optimized out>, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2419
#7 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#8 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#9 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
{color:#de350b}*Thread 26*{color} (Thread 0x7fa2eaffd640 (LWP 53404)):
#0 0x00007fa343dbafa5 in memory_barrier (ee=<optimized out>) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:123
#1 rearm_polling (epollfd=<optimized out>, ee=<optimized out>) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:186
#2 pni_raw_connection_done (rc=0x7fa2dc12ac70) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll_raw_connection.c:413
#3 pn_proactor_done (batch=0x7fa2dc12b228, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2692
#4 pn_proactor_done (p=0xd46d30, batch=batch@entry=0x7fa2dc12b228) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2672
#5 0x00007fa343e9d081 in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1138
#6 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#7 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 25 (Thread 0x7fa3217fa640 (LWP 53392)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa304000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 24 (Thread 0x7fa2ebfff640 (LWP 53401)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2c0000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
*{color:#de350b}Thread 23{color}* (Thread 0x7fa2e3fff640 (LWP 53402)):
#0 0x00007fa343d8350c in send () from /lib64/libpthread.so.0
#1 0x00007fa343dbe718 in snd (s=512, b=<optimized out>, fd=27) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll_raw_connection.c:333
#2 pni_raw_write (send=<optimized out>, set_error=<optimized out>,
sock=<optimized out>, conn=<optimized out>) at
/home/mick/latest/qpid-proton/c/src/proactor/raw_connection.c:566
#3 pni_raw_write (send=<optimized out>, set_error=<optimized out>, sock=27,
conn=0x7fa2f413a820) at
/home/mick/latest/qpid-proton/c/src/proactor/raw_connection.c:554
#4 pni_raw_connection_process (sched_ready=<optimized out>, t=0x7fa2f413a760)
at /home/mick/latest/qpid-proton/c/src/proactor/epoll_raw_connection.c:388
#5 process (tsk=0x7fa2f413a760) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2230
#6 next_event_batch (p=<optimized out>, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2419
#7 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#8 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#9 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 22 (Thread 0x7fa2ea7fc640 (LWP 53405)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2cc000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 21 (Thread 0x7fa30d7fa640 (LWP 53399)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2c8000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 20 (Thread 0x7fa321ffb640 (LWP 53391)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2ec000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 19 (Thread 0x7fa2e9ffb640 (LWP 53406)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2d0000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 18 (Thread 0x7fa323fff640 (LWP 53387)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2d4000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 17 (Thread 0x7fa30dffb640 (LWP 53398)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2c4000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 16 (Thread 0x7fa30ffff640 (LWP 53394)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2dc000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
{color:#de350b}*Thread 15*{color} (Thread 0x7fa330c7a640 (LWP 53386)):
#0 0x00007fa343d82ea0 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007fa343d7b763 in pthread_mutex_lock () from /lib64/libpthread.so.0
#2 0x00007fa343dbd469 in lock (m=0xd47050) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll-internal.h:330
#3 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2407
#4 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#5 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#6 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 14 (Thread 0x7fa322ffd640 (LWP 53389)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2f4000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 13 (Thread 0x7fa33347f640 (LWP 53381)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa318000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 12 (Thread 0x7fa3237fe640 (LWP 53388)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa310000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 11 (Thread 0x7fa33247d640 (LWP 53383)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa31c000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
{color:#de350b}*Thread 10*{color} (Thread 0x7fa33147b640 (LWP 53385)):
#0 0x00007fa343d7ce99 in __pthread_mutex_unlock_usercnt () from
/lib64/libpthread.so.0
#1 0x00007fa343dbde0a in unlock (m=0xd47050) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll-internal.h:331
#2 poller_do_epoll (can_block=true, ts=0x7fa314000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2490
#3 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2438
#4 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#5 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#6 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 9 (Thread 0x7fa332c7e640 (LWP 53382)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa328000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 8 (Thread 0x7fa331c7c640 (LWP 53384)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2ac000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x7fa30e7fc640 (LWP 53397)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2b0000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x7fa333c80640 (LWP 53380)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa324000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7fa3227fc640 (LWP 53390)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2f8000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x7fa2e2ffd640 (LWP 53410)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa29c000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7fa2e97fa640 (LWP 53407)):
#0 0x00007fa343d7f6c2 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fa343dbdcbb in suspend (ts=0x7fa2a4000b60, p=0xd46d30) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:393
#2 next_event_batch (p=0xd46d30, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2455
#3 0x00007fa343e9cf9f in thread_run (arg=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#4 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
{color:#de350b}*Thread 2*{color} (Thread 0x7fa3431672c0 (LWP 53378)):
#0 0x00007fa343d8350c in send () from /lib64/libpthread.so.0
#1 0x00007fa343dbe718 in snd (s=512, b=<optimized out>, fd=31) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll_raw_connection.c:333
#2 pni_raw_write (send=<optimized out>, set_error=<optimized out>,
sock=<optimized out>, conn=<optimized out>) at
/home/mick/latest/qpid-proton/c/src/proactor/raw_connection.c:566
#3 pni_raw_write (send=<optimized out>, set_error=<optimized out>, sock=31,
conn=0x7fa2f413bba0) at
/home/mick/latest/qpid-proton/c/src/proactor/raw_connection.c:554
#4 pni_raw_connection_process (sched_ready=<optimized out>, t=0x7fa2f413bae0)
at /home/mick/latest/qpid-proton/c/src/proactor/epoll_raw_connection.c:388
#5 process (tsk=0x7fa2f413bae0) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2230
#6 next_event_batch (p=<optimized out>, can_block=true) at
/home/mick/latest/qpid-proton/c/src/proactor/epoll.c:2419
#7 0x00007fa343e9cf9f in thread_run (arg=arg@entry=0xb52c00) at
/home/mick/latest/qpid-dispatch/src/server.c:1105
#8 0x00007fa343e9d730 in qd_server_run (qd=<optimized out>) at
/home/mick/latest/qpid-dispatch/src/server.c:1487
#9 0x00000000004026a8 in main_process (config_path=0x7ffc80b5bd99
"./Br_1.conf", python_pkgdir=<optimized out>, test_hooks=<optimized out>, fd=2)
at /home/mick/latest/qpid-dispatch/router/src/main.c:115
#10 0x000000000040240b in main (argc=3, argv=0x7ffc80b5a808) at
/home/mick/latest/qpid-dispatch/router/src/main.c:369
*{color:#de350b}Thread 1{color}* (Thread 0x7fa334481640 (LWP 53379)):
#0 sys_atomic_sub (value=1, ref=0x14) at
/home/mick/latest/qpid-dispatch/include/qpid/dispatch/atomic.h:48
#1 sys_atomic_dec (ref=0x14) at
/home/mick/latest/qpid-dispatch/include/qpid/dispatch/atomic.h:212
#2 qd_buffer_dec_fanout (buf=0x0) at
/home/mick/latest/qpid-dispatch/include/qpid/dispatch/buffer.h:177
#3 qd_message_stream_data_release (stream_data=0x7fa2a817acc8) at
/home/mick/latest/qpid-dispatch/src/message.c:2627
#4 0x00007fa343e48895 in flush_outgoing_buffs (conn=conn@entry=0x7fa2f411f888)
at /home/mick/latest/qpid-dispatch/src/adaptors/tcp_adaptor.c:431
#5 0x00007fa343e4c05e in free_qdr_tcp_connection (tc=0x7fa2f411f888) at
/home/mick/latest/qpid-dispatch/src/adaptors/tcp_adaptor.c:455
#6 0x00007fa343e8791d in router_core_thread (arg=0xdb6cb0) at
/home/mick/latest/qpid-dispatch/src/router_core/router_core_thread.c:239
#7 0x00007fa343d793f9 in start_thread () from /lib64/libpthread.so.0
#8 0x00007fa3439f8b53 in clone () from /lib64/libc.so.6
(gdb)
> SEGV in qd_buffer_dec_fanout
> ----------------------------
>
> Key: DISPATCH-2088
> URL: https://issues.apache.org/jira/browse/DISPATCH-2088
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Protocol Adaptors
> Reporter: michael goulish
> Assignee: Charles E. Rolke
> Priority: Blocker
> Fix For: 1.16.0
>
>
> *code from 2021-04-26-afternoon*
> {
> dispatch: (main) 22689e4f95ae1945e61eec814d3ab3e2d4259f04
> proton: (main) 08b301a97c834e002d41ee852bba1288fe83b936
> }
>
> *Test*
> * Doing 1-router TCP throughput testing across high-bandwidth link.
> * Router has 32 worker threads.
> * iperf client is using "-P 10" flag, i.e. doing 10 parallel streams.
> * Router is sustaining 10+ Gbit/sec during test.
> * SEGV happens at end of test.
>
> Here's the backtrace:
>
> {color:#de350b}#0 sys_atomic_sub (value=1, ref=0x14){color}
> {color:#de350b} at
> /home/mick/latest/qpid-dispatch/include/qpid/dispatch/atomic.h:48{color}
> {color:#de350b}#1 sys_atomic_dec (ref=0x14){color}
> {color:#de350b} at
> /home/mick/latest/qpid-dispatch/include/qpid/dispatch/atomic.h:212{color}
> {color:#de350b}#2 qd_buffer_dec_fanout (buf=0x0){color}
> {color:#de350b} at
> /home/mick/latest/qpid-dispatch/include/qpid/dispatch/buffer.h:177{color}
> {color:#de350b}#3 qd_message_stream_data_release
> (stream_data=0x7f01b80038c8){color}
> {color:#de350b} at /home/mick/latest/qpid-dispatch/src/message.c:2627{color}
> {color:#de350b}#4 0x00007f0237035895 in flush_outgoing_buffs
> (conn=conn@entry=0x7f0218012a88){color}
> {color:#de350b} at
> /home/mick/latest/qpid-dispatch/src/adaptors/tcp_adaptor.c:431{color}
> {color:#de350b}#5 0x00007f023703905e in free_qdr_tcp_connection
> (tc=0x7f0218012a88){color}
> {color:#de350b} at
> /home/mick/latest/qpid-dispatch/src/adaptors/tcp_adaptor.c:455{color}
> {color:#de350b}#6 0x00007f023707491d in router_core_thread
> (arg=0x1e6ccb0){color}
> {color:#de350b} at
> /home/mick/latest/qpid-dispatch/src/router_core/router_core_thread.c:239{color}
> {color:#de350b}#7 0x00007f0236f663f9 in start_thread () from
> /lib64/libpthread.so.0{color}
> {color:#de350b}#8 0x00007f0236be5b53 in clone () from /lib64/libc.so.6{color}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
