[jira] [Assigned] (DISPATCH-902) Intermittent crash with link to broker when broker closed
[
https://issues.apache.org/jira/browse/DISPATCH-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Conway reassigned DISPATCH-902:
Assignee: Ganesh Murthy (was: Alan Conway)
> Intermittent crash with link to broker when broker closed
> -
>
> Key: DISPATCH-902
> URL: https://issues.apache.org/jira/browse/DISPATCH-902
> Project: Qpid Dispatch
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Kim van der Riet
>Assignee: Ganesh Murthy
>Priority: Blocker
> Fix For: 1.1.0
>
> Attachments: qdrouterd.node1.conf, qdrouterd.node2.conf,
> qpidd.d2n.conf, testme.tgz
>
>
> When using dispatch in a 2-node configuration with a broker between them:
> {noformat}
> 9002 10001 100019003
> sender > dispatch1 -> qpid-cpp -> dispatch2 -> receiver
> {noformat}
> and initializing in the following order:
> # start dispatch1
> # start dispatch2
> # start qpid-cpp
> # wait for "Link Route Activated" messages on both dispatch nodes
> # stop qpid-cpp
> then the dispatch nodes will core after a random amount of time and after
> sending a random number of
> {noformat}
> (info) Connection to localhost:10001 failed: proton:io Connection refused -
> on read from localhost:10001
> {noformat}
> messages.
> The stack trace is as follows for all occurrences:
> {noformat}
> Thread 3 "qdrouterd" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fffea269700 (LWP 10954)]
> pn_transport_tail_closed (transport=0x0) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/core/transport.c:3044
> 3044 bool pn_transport_tail_closed(pn_transport_t *transport) { return
> transport->tail_closed; }
> (gdb) thread apply all bt
> Thread 5 (Thread 0x7fffe9267700 (LWP 10956)):
> #0 0x767eb6d3 in epoll_wait () at
> ../sysdeps/unix/syscall-template.S:84
> #1 0x777327e2 in proactor_do_epoll (p=0x89b550,
> can_block=can_block@entry=true) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:1978
> #2 0x777337ca in pn_proactor_wait (p=) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2025
> #3 0x77bbc219 in thread_run (arg=0x89ec20) at
> /home/kpvdr/RedHat/qpid-dispatch/src/server.c:932
> #4 0x775185ca in start_thread (arg=0x7fffe9267700) at
> pthread_create.c:333
> #5 0x767eb0cd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> Thread 4 (Thread 0x7fffe9a68700 (LWP 10955)):
> #0 0x767eb6d3 in epoll_wait () at
> ../sysdeps/unix/syscall-template.S:84
> #1 0x777327e2 in proactor_do_epoll (p=0x89b550,
> can_block=can_block@entry=true) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:1978
> #2 0x777337ca in pn_proactor_wait (p=) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2025
> #3 0x77bbc219 in thread_run (arg=0x89ec20) at
> /home/kpvdr/RedHat/qpid-dispatch/src/server.c:932
> #4 0x775185ca in start_thread (arg=0x7fffe9a68700) at
> pthread_create.c:333
> #5 0x767eb0cd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> Thread 3 (Thread 0x7fffea269700 (LWP 10954)):
> #0 pn_transport_tail_closed (transport=0x0) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/core/transport.c:3044
> #1 0x7794f4f9 in pn_connection_driver_read_closed
> (d=d@entry=0x7fffdc054288) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/core/connection_driver.c:109
> #2 0x77731ef1 in pconnection_rclosed (pc=0x7fffdc053ce0) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:898
> #3 pconnection_process (pc=0x7fffdc053ce0, events=,
> timeout=timeout@entry=false, topup=topup@entry=false) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:1084
> #4 0x77732945 in proactor_do_epoll (p=0x89b550,
> can_block=can_block@entry=true) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2007
> #5 0x777337ca in pn_proactor_wait (p=) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2025
> #6 0x77bbc219 in thread_run (arg=0x89ec20) at
> /home/kpvdr/RedHat/qpid-dispatch/src/server.c:932
> #7 0x775185ca in start_thread (arg=0x7fffea269700) at
> pthread_create.c:333
> #8 0x767eb0cd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> Thread 2 (Thread 0x7fffeaa6a700 (LWP 10953)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at
> ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
> #1 0x77ba2949 in sys_cond_wait (cond=,
> held_mutex=) at
> /home/kpvdr/RedHat/qpid-dispatch/src/posix/threading.c:91
> #2 0x77bb0cf5 in router_core_thread (arg=0x8f8c90) at
>
[jira] [Assigned] (DISPATCH-902) Intermittent crash with link to broker when broker closed
[
https://issues.apache.org/jira/browse/DISPATCH-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Conway reassigned DISPATCH-902:
Assignee: Alan Conway (was: Ted Ross)
> Intermittent crash with link to broker when broker closed
> -
>
> Key: DISPATCH-902
> URL: https://issues.apache.org/jira/browse/DISPATCH-902
> Project: Qpid Dispatch
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Kim van der Riet
>Assignee: Alan Conway
>Priority: Blocker
> Fix For: 1.1.0
>
> Attachments: qdrouterd.node1.conf, qdrouterd.node2.conf,
> qpidd.d2n.conf
>
>
> When using dispatch in a 2-node configuration with a broker between them:
> {noformat}
> 9002 10001 100019003
> sender > dispatch1 -> qpid-cpp -> dispatch2 -> receiver
> {noformat}
> and initializing in the following order:
> # start dispatch1
> # start dispatch2
> # start qpid-cpp
> # wait for "Link Route Activated" messages on both dispatch nodes
> # stop qpid-cpp
> then the dispatch nodes will core after a random amount of time and after
> sending a random number of
> {noformat}
> (info) Connection to localhost:10001 failed: proton:io Connection refused -
> on read from localhost:10001
> {noformat}
> messages.
> The stack trace is as follows for all occurrences:
> {noformat}
> Thread 3 "qdrouterd" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fffea269700 (LWP 10954)]
> pn_transport_tail_closed (transport=0x0) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/core/transport.c:3044
> 3044 bool pn_transport_tail_closed(pn_transport_t *transport) { return
> transport->tail_closed; }
> (gdb) thread apply all bt
> Thread 5 (Thread 0x7fffe9267700 (LWP 10956)):
> #0 0x767eb6d3 in epoll_wait () at
> ../sysdeps/unix/syscall-template.S:84
> #1 0x777327e2 in proactor_do_epoll (p=0x89b550,
> can_block=can_block@entry=true) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:1978
> #2 0x777337ca in pn_proactor_wait (p=) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2025
> #3 0x77bbc219 in thread_run (arg=0x89ec20) at
> /home/kpvdr/RedHat/qpid-dispatch/src/server.c:932
> #4 0x775185ca in start_thread (arg=0x7fffe9267700) at
> pthread_create.c:333
> #5 0x767eb0cd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> Thread 4 (Thread 0x7fffe9a68700 (LWP 10955)):
> #0 0x767eb6d3 in epoll_wait () at
> ../sysdeps/unix/syscall-template.S:84
> #1 0x777327e2 in proactor_do_epoll (p=0x89b550,
> can_block=can_block@entry=true) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:1978
> #2 0x777337ca in pn_proactor_wait (p=) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2025
> #3 0x77bbc219 in thread_run (arg=0x89ec20) at
> /home/kpvdr/RedHat/qpid-dispatch/src/server.c:932
> #4 0x775185ca in start_thread (arg=0x7fffe9a68700) at
> pthread_create.c:333
> #5 0x767eb0cd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> Thread 3 (Thread 0x7fffea269700 (LWP 10954)):
> #0 pn_transport_tail_closed (transport=0x0) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/core/transport.c:3044
> #1 0x7794f4f9 in pn_connection_driver_read_closed
> (d=d@entry=0x7fffdc054288) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/core/connection_driver.c:109
> #2 0x77731ef1 in pconnection_rclosed (pc=0x7fffdc053ce0) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:898
> #3 pconnection_process (pc=0x7fffdc053ce0, events=,
> timeout=timeout@entry=false, topup=topup@entry=false) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:1084
> #4 0x77732945 in proactor_do_epoll (p=0x89b550,
> can_block=can_block@entry=true) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2007
> #5 0x777337ca in pn_proactor_wait (p=) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2025
> #6 0x77bbc219 in thread_run (arg=0x89ec20) at
> /home/kpvdr/RedHat/qpid-dispatch/src/server.c:932
> #7 0x775185ca in start_thread (arg=0x7fffea269700) at
> pthread_create.c:333
> #8 0x767eb0cd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> Thread 2 (Thread 0x7fffeaa6a700 (LWP 10953)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at
> ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
> #1 0x77ba2949 in sys_cond_wait (cond=,
> held_mutex=) at
> /home/kpvdr/RedHat/qpid-dispatch/src/posix/threading.c:91
> #2 0x77bb0cf5 in router_core_thread (arg=0x8f8c90) at
> /home/kpvdr/RedHat/qpid-dispatch/src/router_core/router_core_thread.c:66
> #3
[jira] [Assigned] (DISPATCH-902) Intermittent crash with link to broker when broker closed
[
https://issues.apache.org/jira/browse/DISPATCH-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Ross reassigned DISPATCH-902:
-
Assignee: Ted Ross
> Intermittent crash with link to broker when broker closed
> -
>
> Key: DISPATCH-902
> URL: https://issues.apache.org/jira/browse/DISPATCH-902
> Project: Qpid Dispatch
> Issue Type: Bug
>Reporter: Kim van der Riet
>Assignee: Ted Ross
> Attachments: qdrouterd.node1.conf, qdrouterd.node2.conf,
> qpidd.d2n.conf
>
>
> When using dispatch in a 2-node configuration with a broker between them:
> {noformat}
> 9002 10001 100019003
> sender > dispatch1 -> qpid-cpp -> dispatch2 -> receiver
> {noformat}
> and initializing in the following order:
> # start dispatch1
> # start dispatch2
> # start qpid-cpp
> # wait for "Link Route Activated" messages on both dispatch nodes
> # stop qpid-cpp
> then the dispatch nodes will core after a random amount of time and after
> sending a random number of
> {noformat}
> (info) Connection to localhost:10001 failed: proton:io Connection refused -
> on read from localhost:10001
> {noformat}
> messages.
> The stack trace is as follows for all occurrences:
> {noformat}
> Thread 3 "qdrouterd" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fffea269700 (LWP 10954)]
> pn_transport_tail_closed (transport=0x0) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/core/transport.c:3044
> 3044 bool pn_transport_tail_closed(pn_transport_t *transport) { return
> transport->tail_closed; }
> (gdb) thread apply all bt
> Thread 5 (Thread 0x7fffe9267700 (LWP 10956)):
> #0 0x767eb6d3 in epoll_wait () at
> ../sysdeps/unix/syscall-template.S:84
> #1 0x777327e2 in proactor_do_epoll (p=0x89b550,
> can_block=can_block@entry=true) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:1978
> #2 0x777337ca in pn_proactor_wait (p=) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2025
> #3 0x77bbc219 in thread_run (arg=0x89ec20) at
> /home/kpvdr/RedHat/qpid-dispatch/src/server.c:932
> #4 0x775185ca in start_thread (arg=0x7fffe9267700) at
> pthread_create.c:333
> #5 0x767eb0cd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> Thread 4 (Thread 0x7fffe9a68700 (LWP 10955)):
> #0 0x767eb6d3 in epoll_wait () at
> ../sysdeps/unix/syscall-template.S:84
> #1 0x777327e2 in proactor_do_epoll (p=0x89b550,
> can_block=can_block@entry=true) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:1978
> #2 0x777337ca in pn_proactor_wait (p=) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2025
> #3 0x77bbc219 in thread_run (arg=0x89ec20) at
> /home/kpvdr/RedHat/qpid-dispatch/src/server.c:932
> #4 0x775185ca in start_thread (arg=0x7fffe9a68700) at
> pthread_create.c:333
> #5 0x767eb0cd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> Thread 3 (Thread 0x7fffea269700 (LWP 10954)):
> #0 pn_transport_tail_closed (transport=0x0) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/core/transport.c:3044
> #1 0x7794f4f9 in pn_connection_driver_read_closed
> (d=d@entry=0x7fffdc054288) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/core/connection_driver.c:109
> #2 0x77731ef1 in pconnection_rclosed (pc=0x7fffdc053ce0) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:898
> #3 pconnection_process (pc=0x7fffdc053ce0, events=,
> timeout=timeout@entry=false, topup=topup@entry=false) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:1084
> #4 0x77732945 in proactor_do_epoll (p=0x89b550,
> can_block=can_block@entry=true) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2007
> #5 0x777337ca in pn_proactor_wait (p=) at
> /home/kpvdr/RedHat/qpid-proton/proton-c/src/proactor/epoll.c:2025
> #6 0x77bbc219 in thread_run (arg=0x89ec20) at
> /home/kpvdr/RedHat/qpid-dispatch/src/server.c:932
> #7 0x775185ca in start_thread (arg=0x7fffea269700) at
> pthread_create.c:333
> #8 0x767eb0cd in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> Thread 2 (Thread 0x7fffeaa6a700 (LWP 10953)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at
> ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
> #1 0x77ba2949 in sys_cond_wait (cond=,
> held_mutex=) at
> /home/kpvdr/RedHat/qpid-dispatch/src/posix/threading.c:91
> #2 0x77bb0cf5 in router_core_thread (arg=0x8f8c90) at
> /home/kpvdr/RedHat/qpid-dispatch/src/router_core/router_core_thread.c:66
> #3 0x775185ca in start_thread (arg=0x7fffeaa6a700) at
> pthread_create.c:333
> #4 0x767eb0cd in clone ()
