[ https://issues.apache.org/jira/browse/DISPATCH-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ganesh Murthy resolved DISPATCH-1954. ------------------------------------- Fix Version/s: 1.16.0 Resolution: Fixed Adding back the bitmask suppression fixed this issue - https://github.com/apache/qpid-dispatch/commit/5c0de9c31b13f7f0795e158f00f8176c25480d19 > system_tests_management fails due to qd_bitmask_t leak > ------------------------------------------------------ > > Key: DISPATCH-1954 > URL: https://issues.apache.org/jira/browse/DISPATCH-1954 > Project: Qpid Dispatch > Issue Type: Test > Components: Tools > Reporter: Ganesh Murthy > Priority: Major > Fix For: 1.16.0 > > > {noformat} > Router router1 debug dump file: > 19: >>>> > 19: alloc.c: Items of type 'qd_iterator_t' remain allocated at shutdown: 9 > (SUPPRESSED) > 19: alloc.c: Items of type 'qd_timer_t' remain allocated at shutdown: 1 > (SUPPRESSED) > 19: alloc.c: Items of type 'qd_bitmask_t' remain allocated at shutdown: 1 > 19: Leak: 2021-02-08 15:33:33.819717 +0000 type: qd_bitmask_t address: > 0x611000086090 > 19: /lib/x86_64-linux-gnu/libasan.so.5(+0x6cd30) [0x7f14c8ed1d30] > 19: > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/src/libqpid-dispatch.so(qd_alloc+0x37b) > [0x7f14c86813db] > 19: > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/src/libqpid-dispatch.so(qd_bitmask+0x12) > [0x7f14c8691112] > 19: > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/src/libqpid-dispatch.so(qd_tracemask_create+0xaf) > [0x7f14c88b00cf] > 19: > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/src/libqpid-dispatch.so(+0x730542) > [0x7f14c888a542] > 19: > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/src/libqpid-dispatch.so(qd_container_handle_event+0x16b7) > [0x7f14c86f0507] > 19: > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/src/libqpid-dispatch.so(+0x7403ca) > [0x7f14c889a3ca] > 19: > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/src/libqpid-dispatch.so(+0x748911) > [0x7f14c88a2911] > 19: /lib/x86_64-linux-gnu/libpthread.so.0(+0x9609) [0x7f14c8140609] > 19: /lib/x86_64-linux-gnu/libc.so.6(clone+0x43) [0x7f14c76fa293] > 19: > 19: alloc.c: Items of type 'qdr_action_t' remain allocated at shutdown: 2 > (SUPPRESSED) > 19: alloc.c: Items of type 'qdr_field_t' remain allocated at shutdown: 1 > (SUPPRESSED) > 19: alloc.c: Items of type 'qd_buffer_t' remain allocated at shutdown: 7 > (SUPPRESSED) > 19: alloc.c: Items of type 'qd_connector_t' remain allocated at shutdown: 1 > (SUPPRESSED) > 19: alloc.c: Items of type 'qdr_link_work_t' remain allocated at shutdown: 1 > (SUPPRESSED) > 19: alloc.c: Items of type 'qd_message_t' remain allocated at shutdown: 2 > (SUPPRESSED) > 19: alloc.c: Items of type 'qd_message_content_t' remain allocated at > shutdown: 2 (SUPPRESSED) > 19: alloc.c: Items of type 'qd_parsed_field_t' remain allocated at shutdown: > 3 (SUPPRESSED) > 19: alloc.c: Items of type 'qdr_delivery_t' remain allocated at shutdown: 1 > (SUPPRESSED) > 19: > 19: <<<< > 17: > 17: ---------------------------------------------------------------------- > 17: Ran 15 tests in 17.318s > 17: > 17: OK > 19: > 19: Router router1 output file: > 19: >>>> > 19: ERROR: Aborted due to unexpected alloc pool leak of type 'qd_bitmask_t' > 19: > 19: <<<< > 19: > 19: Router router1 command file: > 19: >>>> > 19: qdrouterd -c router1.conf -I > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/python > 19: pid=2431 > 19: > 19: <<<< > 19: > 19: Router router1 log file tail: > 19: >>>> > 19: 2021-02-08 15:33:33.816853 +0000 ROUTER_CORE (trace) Core action > 'link_flow' (../src/router_core/router_core_thread.c:238) > 19: 2021-02-08 15:33:33.816887 +0000 ROUTER_CORE (trace) Core action > 'link_detach' (../src/router_core/router_core_thread.c:238) > 19: 2021-02-08 15:33:33.816924 +0000 ROUTER_CORE (info) [C2][L23] Link lost: > del=16 presett=0 psdrop=0 acc=16 rej=0 rel=0 mod=0 delay1=0 delay10=0 > blocked=no (../src/router_core/connections.c:1049) > 19: 2021-02-08 15:33:33.816955 +0000 ROUTER_CORE (trace) Core action > 'link_detach' (../src/router_core/router_core_thread.c:238) > 19: 2021-02-08 15:33:33.816987 +0000 ROUTER_CORE (info) [C2][L24] Link lost: > del=16 presett=16 psdrop=0 acc=0 rej=0 rel=0 mod=0 delay1=0 delay10=0 > blocked=no (../src/router_core/connections.c:1049) > 19: 2021-02-08 15:33:33.817023 +0000 ROUTER_CORE (trace) Core action > 'connection_closed' (../src/router_core/router_core_thread.c:238) > 19: 2021-02-08 15:33:33.817049 +0000 ROUTER_CORE (info) [C2] Connection > Closed (../src/router_core/connections.c:1531) > 19: 2021-02-08 15:33:33.817133 +0000 PROTOCOL (trace) [C1]:FRAME: 0 <- > @flow(19) [next-incoming-id=20, incoming-window=2147483647, > next-outgoing-id=18, outgoing-window=2147483647, handle=1, delivery-count=20, > link-credit=250, drain=false] (../src/server.c:113) > 19: 2021-02-08 15:33:33.817209 +0000 ROUTER_CORE (trace) Core action > 'link_flow' (../src/router_core/router_core_thread.c:238) > 19: 2021-02-08 15:33:33.819245 +0000 PROTOCOL (trace) [C1]:FRAME: 0 <- > @transfer(20) [handle=0, delivery-id=18, > delivery-tag=b"#\x00\x00\x00\x00\x00\x00\x00", message-format=0, > settled=true] (246) > "\x00Sp\xd0\x00\x00\x00\x05\x00\x00\x00\x01B\x00Sr\xd1\x00\x00\x00o\x00\x00\x00\x0a\xa3\x0ex-opt-qd.trace\xd0\x00\x00\x00\x0f\x00\x00\x00\x01\xa1\x090/router0\xa3\x10x-opt-qd.ingress\xa1\x090/router0\xa3\x09x-opt-qd.\xa1\x01X\xa3\x09x-opt-qd.\xa1\x01X\xa3\x09x-opt-qd.\xa1\x01X\x00Ss\xd0\x00\x00\x00)\x00\x00\x00\x06@@\xa1\x1eamqp:/_topo/0/router1/qdrouter@@@\x00St\xd1\x00\x00\x00\x11\x00\x00\x00\x02\xa1\x06opcode\xa1\x03LSR\x00Sw\xd1\x00\x00\x00 > > \x00\x00\x00\x06\xa1\x02id\xa1\x07router0\xa1\x02pvT\x01\xa1\x04area\xa1\x010" > (../src/server.c:113) > 19: 2021-02-08 15:33:33.819765 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery created qdr_link_deliver_to (../src/router_core/transfer.c:97) > 19: 2021-02-08 15:33:33.819792 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery incref: rc:1 qdr_link_deliver_to - newly created delivery, add > to action list (../src/router_core/delivery.c:112) > 19: 2021-02-08 15:33:33.819810 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery incref: rc:2 qdr_link_deliver_to - protect returned value > (../src/router_core/delivery.c:112) > 19: 2021-02-08 15:33:33.819837 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery incref: rc:3 referenced by a pn_delivery > (../src/router_core/delivery.c:112) > 19: 2021-02-08 15:33:33.819862 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery decref: rc:2 release protection of return from deliver > (../src/router_core/delivery.c:146) > 19: 2021-02-08 15:33:33.819892 +0000 ROUTER_CORE (trace) Core action > 'link_deliver' (../src/router_core/router_core_thread.c:238) > 19: 2021-02-08 15:33:33.819956 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery incref: rc:3 qdr_delivery_update_disposition - add to action > list (../src/router_core/delivery.c:112) > 19: 2021-02-08 15:33:33.819995 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery decref: rc:3 removed reference from pn_delivery > (../src/router_core/delivery.c:146) > 19: 2021-02-08 15:33:33.820024 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery incref: rc:4 qdr_forward_on_message_CT - adding to general work > item (../src/router_core/delivery.c:112) > 19: 2021-02-08 15:33:33.820093 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery decref_CT: rc:2 qdr_link_forward_CT - removed from action (2) > (../src/router_core/delivery.c:639) > 19: 2021-02-08 15:33:33.820137 +0000 ROUTER_CORE (trace) Core action > 'update_delivery' (../src/router_core/router_core_thread.c:238) > 19: 2021-02-08 15:33:33.820164 +0000 ROUTER_CORE (trace) [C1][L1][D132] > Remote updated mcast delivery disp=0x0 settled=True > (../src/router_core/delivery.c:791) > 19: 2021-02-08 15:33:33.820184 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery decref_CT: rc:1 qdr_update_delivery_CT - remove from action > (../src/router_core/delivery.c:639) > 19: 2021-02-08 15:33:33.820247 +0000 HTTP (trace) lws_context_destroy: ctx > 0x620000013080 (../src/http-libwebsockets.c:61) > 19: 2021-02-08 15:33:33.820277 +0000 HTTP (trace) lws_destroy_event_pipe > (../src/http-libwebsockets.c:61) > 19: 2021-02-08 15:33:33.820306 +0000 HTTP (trace) lws_context_destroy2: ctx > 0x620000013080 (../src/http-libwebsockets.c:61) > 19: 2021-02-08 15:33:33.820321 +0000 HTTP (trace) 0x7f14c6c9b720: post vh > listl (../src/http-libwebsockets.c:61) > 19: 2021-02-08 15:33:33.820334 +0000 HTTP (trace) 0x7f14c6c9b720: post pdl > (../src/http-libwebsockets.c:61) > 19: 2021-02-08 15:33:33.820450 +0000 HTTP (trace) 0x7f14c6c9b720: baggage > (../src/http-libwebsockets.c:61) > 19: 2021-02-08 15:33:33.820466 +0000 HTTP (trace) 0x7f14c6c9b720: post dc2 > (../src/http-libwebsockets.c:61) > 19: 2021-02-08 15:33:33.820481 +0000 HTTP (trace) lws_context_destroy3 > (../src/http-libwebsockets.c:61) > 19: 2021-02-08 15:33:33.820604 +0000 HTTP (trace) lws_context_destroy3: ctx > 0x620000013080 freed (../src/http-libwebsockets.c:61) > 19: 2021-02-08 15:33:33.820622 +0000 SERVER (notice) Shut Down > (../src/server.c:1493) > 19: 2021-02-08 15:33:33.820691 +0000 CONTAINER (trace) Default node removed > (../src/container.c:848) > 19: 2021-02-08 15:33:33.820728 +0000 ROUTER_CORE (info) Router Core thread > exited (../src/router_core/router_core_thread.c:259) > 19: 2021-02-08 15:33:33.820845 +0000 HTTP_ADAPTOR (trace) Shutting down HTTP2 > Protocol adaptor (../src/adaptors/http2/http2_adaptor.c:2444) > 19: 2021-02-08 15:33:33.820875 +0000 TCP_ADAPTOR (info) Shutting down TCP > protocol adaptor (../src/adaptors/tcp_adaptor.c:1249) > 19: 2021-02-08 15:33:33.821177 +0000 ROUTER_LS (trace) RCVD: LSR(id=router0 > pv=1 area=0) > (/home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/python/qpid_dispatch_internal/router/engine.py:165) > 19: 2021-02-08 15:33:33.821428 +0000 ROUTER_LS (trace) SENT: LSU(id=router1 > pv=1 area=0 inst=1612798405 ls_seq=3 ls=LS(id=router1 area=0 ls_seq=3 > peers={'router0': 1})) > (/home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/python/qpid_dispatch_internal/router/link.py:73) > 19: 2021-02-08 15:33:33.821456 +0000 ROUTER_CORE (debug) [C1][L1][D132] > Delivery decref: rc:0 qdr_forward_on_message - remove from general work > (../src/router_core/delivery.c:146) > 19: 2021-02-08 15:33:33.821693 +0000 ROUTER_CORE (info) Finalizing core > module: streaming_link_scrubber (../src/router_core/router_core_thread.c:138) > 19: 2021-02-08 15:33:33.821721 +0000 ROUTER_CORE (info) Finalizing core > module: mobile_sync (../src/router_core/router_core_thread.c:138) > 19: 2021-02-08 15:33:33.821738 +0000 ROUTER_CORE (info) Finalizing core > module: stuck_delivery_detection (../src/router_core/router_core_thread.c:138) > 19: 2021-02-08 15:33:33.821752 +0000 ROUTER_CORE (info) Finalizing core > module: address_lookup_client (../src/router_core/router_core_thread.c:138) > 19: 2021-02-08 15:33:33.821767 +0000 ROUTER_CORE (info) Finalizing core > module: address_lookup_server (../src/router_core/router_core_thread.c:138) > 19: 2021-02-08 15:33:33.821781 +0000 ROUTER_CORE (info) Finalizing core > module: edge_addr_tracking (../src/router_core/router_core_thread.c:138) > 19: 2021-02-08 15:33:33.821856 +0000 SERVER (info) [C1] Closing connection on > shutdown (../src/server.c:1377) > 19: 2021-02-08 15:33:33.821883 +0000 SERVER (info) [C4] Closing connection on > shutdown (../src/server.c:1377) > 19: 2021-02-08 15:33:33.821900 +0000 SERVER (info) [C6] Closing connection on > shutdown (../src/server.c:1377) > 19: 2021-02-08 15:33:33.821917 +0000 SERVER (info) [C8] Closing connection on > shutdown (../src/server.c:1377) > 19: > 19: <<<< > 9/37 Test #17: system_tests_autolinks ......................... Passed > 17.44 sec > test 21 > Start 21: system_tests_handle_failover21: Test command: /usr/bin/bwrap > "--bind" "/" "/" "--unshare-net" "--dev" "/dev" "--die-with-parent" "--" > "/opt/hostedtoolcache/Python/3.7.9/x64/bin/python" > "/home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/tests/run.py" > "-m" "unittest" "-v" "system_tests_handle_failover" > 21: Test timeout computed to be: 600 > 19: ERROR > 19: > 19: ====================================================================== > 19: ERROR: tearDownClass (system_tests_management.ManagementTest) > 19: ---------------------------------------------------------------------- > 19: Traceback (most recent call last): > 19: File > "/home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py", > line 839, in tearDownClass > 19: cls.tester.teardown() > 19: File > "/home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py", > line 782, in teardown > 19: raise RuntimeError("Errors during teardown: \n\n%s" % > "\n\n".join([str(e) for e in errors])) > 19: RuntimeError: Errors during teardown: > 19: > 19: Process 2431 error: exit code -6, expected 0 > 19: qdrouterd -c router1.conf -I > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/python > 19: > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/tests/system_test.dir/system_tests_management/ManagementTest/setUpClass/router1-3.cmd > 19: >>>> > 19: ERROR: Aborted due to unexpected alloc pool leak of type 'qd_bitmask_t' > 19: <<<< > 19: > 19: ---------------------------------------------------------------------- > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org