[ https://issues.apache.org/jira/browse/DISPATCH-963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Conway reassigned DISPATCH-963: ------------------------------------ Assignee: Alan Conway > Router crash during shutdown in system_tests_distribution > --------------------------------------------------------- > > Key: DISPATCH-963 > URL: https://issues.apache.org/jira/browse/DISPATCH-963 > Project: Qpid Dispatch > Issue Type: Improvement > Components: Tests > Affects Versions: 1.0.1 > Reporter: Ganesh Murthy > Assignee: Alan Conway > Priority: Major > > The router crashes during shutdown in system_tests_distribution.py > Here is the backtrace > > {noformat} > (gdb) bt > #0 0x00007f361ca5ae40 in pn_ep_decref (endpoint=0x7f35f01c2dd0) at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/core/engine.c:447 > #1 0x00007f361ca5b58b in pn_ep_decref (endpoint=<optimized out>) at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/core/engine.c:445 > #2 0x00007f361ca5f588 in pni_transport_unbind_handles > (handles=0x7f35f00764a0, reset_state=reset_state@entry=true) at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/core/transport.c:748 > #3 0x00007f361ca5f666 in pni_transport_unbind_channels (channels=0x9d1ce0) > at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/core/transport.c:761 > #4 0x00007f361ca5f777 in pn_transport_unbind (transport=0xa863d0) at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/core/transport.c:795 > #5 0x00007f361ca5a63e in pn_connection_driver_release_connection > (d=d@entry=0xa86248) at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/core/connection_driver.c:81 > #6 0x00007f361ca5a679 in pn_connection_driver_destroy (d=d@entry=0xa86248) > at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/core/connection_driver.c:92 > #7 0x00007f361c83a69c in pconnection_final_free (pc=0xa85ca0) at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/proactor/epoll.c:827 > #8 0x00007f361c83b3ac in pconnection_cleanup (pc=<optimized out>) at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/proactor/epoll.c:843 > #9 0x00007f361c83db37 in pconnection_forced_shutdown (pc=0xa85ca0) at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/proactor/epoll.c:878 > #10 pn_proactor_free (p=0x916fd0) at > /home/gmurthy/opensource/qpid-proton-0.22.0/proton-c/src/proactor/epoll.c:1815 > #11 0x00007f361cce7bb5 in qd_server_free (qd_server=0x919190) at > /home/gmurthy/opensource/qpid-dispatch/src/server.c:1176 > #12 0x00007f361cca878e in qd_dispatch_free (qd=0x6164b0) at > /home/gmurthy/opensource/qpid-dispatch/src/dispatch.c:318 > #13 0x0000000000401864 in main_process (config_path=0x7ffcb36d88e2 "B.conf", > python_pkgdir=0x7ffcb36d88ec "/home/gmurthy/opensource/qpid-dispatch/python", > fd=2) at /home/gmurthy/opensource/qpid-dispatch/router/src/main.c:116 > #14 0x00000000004022b0 in main (argc=5, argv=0x7ffcb36d8158) at > /home/gmurthy/opensource/qpid-dispatch/router/src/main.c:360 > (gdb){noformat} > > Running the test under valgrind, it seems that the pn_proactor_free is trying > to free already freed link endpoint. Here are two outputs from valgrind > > {noformat} > Process 3493 error: exit code 42, expected 0 > qdrouterd -c B.conf -I /home/gmurthy/opensource/qpid-dispatch/python > /home/gmurthy/opensource/qpid-dispatch/build/system_test.dir/system_tests_distribution/DistributionTests/setUpClass/B-2.cmd > >>>> > ==3493== Invalid write of size 8 > ==3493== at 0x50E82B0: pn_link_unbound (engine.c:1202) > ==3493== by 0x50EB5D0: pni_transport_unbind_handles (transport.c:746) > ==3493== by 0x50EB665: pni_transport_unbind_channels (transport.c:761) > ==3493== by 0x50EB776: pn_transport_unbind (transport.c:795) > ==3493== by 0x50E663D: pn_connection_driver_release_connection > (connection_driver.c:81) > ==3493== by 0x50E6678: pn_connection_driver_destroy > (connection_driver.c:92) > ==3493== by 0x530A69B: pconnection_final_free (epoll.c:827) > ==3493== by 0x530DB36: pconnection_forced_shutdown (epoll.c:878) > ==3493== by 0x530DB36: pn_proactor_free (epoll.c:1815) > ==3493== by 0x4EA5BB4: qd_server_free (server.c:1176) > ==3493== by 0x4E6678D: qd_dispatch_free (dispatch.c:318) > ==3493== by 0x401863: main_process (main.c:116) > ==3493== by 0x4022AF: main (main.c:360) > ==3493== Address 0x15c06188 is 376 bytes inside a block of size 488 free'd > ==3493== at 0x4C2DD18: free (vg_replace_malloc.c:530) > ==3493== by 0x50DD938: pn_class_decref (object.c:101) > ==3493== by 0x50EA03F: pn_event_finalize (event.c:226) > ==3493== by 0x50EA03F: pn_event_finalize_cast (event.c:271) > ==3493== by 0x50DD928: pn_class_decref (object.c:95) > ==3493== by 0x50EA361: pn_collector_next (event.c:197) > ==3493== by 0x50E6408: batch_next (connection_driver.c:51) > ==3493== by 0x530C544: pconnection_batch_next (epoll.c:884) > ==3493== by 0x4EA50F2: thread_run (server.c:957) > ==3493== by 0x551950A: start_thread (in /usr/lib64/libpthread-2.26.so) > ==3493== by 0x629916E: clone (in /usr/lib64/libc-2.26.so) > ==3493== Block was alloc'd at > ==3493== at 0x4C2EA1E: calloc (vg_replace_malloc.c:711) > ==3493== by 0x50DD811: pn_object_new (object.c:202) > ==3493== by 0x50DD88B: pn_class_new (object.c:61) > ==3493== by 0x50E8163: pn_link_new (engine.c:1153) > ==3493== by 0x50ED3C4: pn_do_attach (transport.c:1366) > ==3493== by 0x50E6227: pni_dispatch_action (dispatcher.c:74) > ==3493== by 0x50E6227: pni_dispatch_frame (dispatcher.c:116) > ==3493== by 0x50E6227: pn_dispatcher_input (dispatcher.c:135) > ==3493== by 0x50EC96B: pn_input_read_amqp (transport.c:2561) > ==3493== by 0x50ECA17: transport_consume (transport.c:1817) > ==3493== by 0x50F0135: pn_transport_process (transport.c:2946) > ==3493== by 0x530C43F: pconnection_process (epoll.c:1128) > ==3493== by 0x530C9EA: proactor_do_epoll (epoll.c:2010) > ==3493== by 0x4EA50C4: thread_run (server.c:955) > {noformat} > > {noformat} > ==3488== Invalid read of size 8 > ==3488== at 0x50E6E11: pni_ep_get_connection (engine.c:50) > ==3488== by 0x50E6E11: pn_ep_decref.part.10 (engine.c:446) > ==3488== by 0x50EB587: pni_transport_unbind_handles (transport.c:748) > ==3488== by 0x50EB665: pni_transport_unbind_channels (transport.c:761) > ==3488== by 0x50EB776: pn_transport_unbind (transport.c:795) > ==3488== by 0x50E663D: pn_connection_driver_release_connection > (connection_driver.c:81) > ==3488== by 0x50E6678: pn_connection_driver_destroy > (connection_driver.c:92) > ==3488== by 0x530A69B: pconnection_final_free (epoll.c:827) > ==3488== by 0x530DF4A: pconnection_done (epoll.c:964) > ==3488== by 0x530DF4A: pn_proactor_done (epoll.c:2038) > ==3488== by 0x4EA5114: thread_run (server.c:960) > ==3488== by 0x551950A: start_thread (in /usr/lib64/libpthread-2.26.so) > ==3488== by 0x629916E: clone (in /usr/lib64/libc-2.26.so) > ==3488== Address 0x127e15d0 is 400 bytes inside a block of size 488 free'd > ==3488== at 0x4C2DD18: free (vg_replace_malloc.c:530) > ==3488== by 0x50DD938: pn_class_decref (object.c:101) > ==3488== by 0x50EA03F: pn_event_finalize (event.c:226) > ==3488== by 0x50EA03F: pn_event_finalize_cast (event.c:271) > ==3488== by 0x50DD928: pn_class_decref (object.c:95) > ==3488== by 0x50EA361: pn_collector_next (event.c:197) > ==3488== by 0x50E6408: batch_next (connection_driver.c:51) > ==3488== by 0x530C544: pconnection_batch_next (epoll.c:884) > ==3488== by 0x4EA50F2: thread_run (server.c:957) > ==3488== by 0x551950A: start_thread (in /usr/lib64/libpthread-2.26.so) > ==3488== by 0x629916E: clone (in /usr/lib64/libc-2.26.so) > ==3488== Block was alloc'd at > ==3488== at 0x4C2EA1E: calloc (vg_replace_malloc.c:711) > ==3488== by 0x50DD811: pn_object_new (object.c:202) > ==3488== by 0x50DD88B: pn_class_new (object.c:61) > ==3488== by 0x50E8163: pn_link_new (engine.c:1153) > ==3488== by 0x50ED3C4: pn_do_attach (transport.c:1366) > ==3488== by 0x50E6227: pni_dispatch_action (dispatcher.c:74) > ==3488== by 0x50E6227: pni_dispatch_frame (dispatcher.c:116) > ==3488== by 0x50E6227: pn_dispatcher_input (dispatcher.c:135) > ==3488== by 0x50EC96B: pn_input_read_amqp (transport.c:2561) > ==3488== by 0x50ECA17: transport_consume (transport.c:1817) > ==3488== by 0x50F0135: pn_transport_process (transport.c:2946) > ==3488== by 0x530C43F: pconnection_process (epoll.c:1128) > ==3488== by 0x530C9EA: proactor_do_epoll (epoll.c:2010) > ==3488== by 0x4EA50C4: thread_run (server.c:955) > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org