Ganesh Murthy created DISPATCH-2297: ---------------------------------------
Summary: OneRouter Q2 test failing in system_tests_http2 on rhel 9 Key: DISPATCH-2297 URL: https://issues.apache.org/jira/browse/DISPATCH-2297 Project: Qpid Dispatch Issue Type: Test Components: Tests Affects Versions: 1.18.0 Reporter: Ganesh Murthy {noformat} 71: Test command: /usr/bin/python3.9 "/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/build/tests/run.py" "-m" "unittest" "-v" "system_tests_http2" 71: Test timeout computed to be: 600 71: test_q2_block_unblock (system_tests_http2.Http2Q2OneRouterTest) ... ERROR 71: 71: Router QDR output file: 71: >>>> 71: 71: <<<< 71: 71: Router QDR command file: 71: >>>> 71: qdrouterd -c http2-test-router.conf -I /home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/python 71: pid=7610 71: 71: <<<< 71: 71: Router QDR log file tail: 71: >>>> 71: 2021-12-08 20:08:23.643322 +0000 ROUTER_CORE (debug) [C3][L2][D1] Delivery decref_CT: rc:1 qdr_delivery_unlink_peers_CT - unlinked from peer (delivery) (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/delivery.c:640) 71: 2021-12-08 20:08:23.643338 +0000 ROUTER_CORE (debug) [C1][L4][D2] Delivery decref_CT: rc:2 qdr_delivery_unlink_peers_CT - unlinked from delivery (peer) (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/delivery.c:640) 71: 2021-12-08 20:08:23.643361 +0000 ROUTER_CORE (debug) [C3][L2][D1] Delivery outcome : is unknown (0x0) (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/delivery.c:392) 71: 2021-12-08 20:08:23.643373 +0000 ROUTER_CORE (debug) [C3][L2][D1] Delivery decref_CT: rc:0 qdr_link_cleanup_deliveries_CT - remove from unsettled list (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/delivery.c:640) 71: 2021-12-08 20:08:23.643472 +0000 ROUTER_CORE (trace) Core action 'link_detach' (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/router_core_thread.c:235) 71: 2021-12-08 20:08:23.643493 +0000 ROUTER_CORE (trace) Core action 'connection_closed' (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/router_core_thread.c:235) 71: 2021-12-08 20:08:23.643514 +0000 ROUTER_CORE (info) [C3][L2] Link closed due to connection loss: del=1 presett=0 psdrop=0 acc=0 rej=0 rel=0 mod=0 delay1=0 delay10=1 blocked=no (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/connections.c:1109) 71: 2021-12-08 20:08:23.643546 +0000 ROUTER_CORE (info) [C3][L3] Link closed due to connection loss: del=0 presett=0 psdrop=0 acc=0 rej=0 rel=0 mod=0 delay1=0 delay10=0 blocked=no (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/connections.c:1109) 71: 2021-12-08 20:08:23.643559 +0000 ROUTER_CORE (info) [C3] Connection Closed (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/connections.c:1604) 71: 2021-12-08 20:08:23.643613 +0000 ROUTER_CORE (trace) Core action 'delete_http2_connection' (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/router_core_thread.c:235) 71: 2021-12-08 20:08:23.643631 +0000 HTTP_ADAPTOR (trace) [C3] Freeing http2 connection in free_qdr_http2_connection (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:459) 71: 2021-12-08 20:08:23.643654 +0000 HTTP_ADAPTOR (trace) [C1] Activation triggered, calling pn_raw_connection_wake() (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:1704) 71: 2021-12-08 20:08:23.643765 +0000 HTTP_ADAPTOR (trace) [C1] PN_RAW_CONNECTION_WAKE Wake-up (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:2539) 71: 2021-12-08 20:08:23.643791 +0000 ROUTER_CORE (debug) [C1][L4][D2] Delivery incref: rc:3 qdr_link_process_deliveries - holding the undelivered delivery locally (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/delivery.c:115) 71: 2021-12-08 20:08:23.643825 +0000 HTTP_ADAPTOR (debug) [C1][S1] qdr_http_deliver - call handle_outgoing_http (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:2076) 71: 2021-12-08 20:08:23.643838 +0000 PROTOCOL (trace) [C1] Starting to handle_outgoing_http (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:1744) 71: 2021-12-08 20:08:23.643853 +0000 PROTOCOL (trace) [C1][S1] Headers already submitted, Proceeding with the body (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:1864) 71: 2021-12-08 20:08:23.643865 +0000 PROTOCOL (trace) [C1][S1] nghttp2_session_send - write_buffers done for resumed stream (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:1872) 71: 2021-12-08 20:08:23.643878 +0000 PROTOCOL (trace) [C1][S1] Message has no footer (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:1979) 71: 2021-12-08 20:08:23.643896 +0000 PROTOCOL (trace) [C1] Finished handle_outgoing_http (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:2008) 71: 2021-12-08 20:08:23.643908 +0000 ROUTER_CORE (debug) [C1][L4][D2] Delivery decref: rc:2 qdr_link_process_deliveries - release local reference - not send_complete (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/delivery.c:149) 71: 2021-12-08 20:08:23.649364 +0000 SERVER (notice) Shut Down (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/server.c:1534) 71: 2021-12-08 20:08:23.649535 +0000 HTTP (trace) lws_context_destroy: ctx 0x20f9f70 (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.649656 +0000 HTTP (trace) _lws_state_transition: system: changed 1 'CONTEXT_CREATED' -> 13 'POLICY_INVALID' (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.649695 +0000 HTTP (trace) lws_destroy_event_pipe (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.649729 +0000 HTTP (trace) lws_pt_destroy: pt destroyed (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.649748 +0000 HTTP (trace) lws_context_destroy2: ctx 0x20f9f70 (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.649778 +0000 HTTP (trace) 0x7f016ca449c0: post vh listl (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.649795 +0000 HTTP (trace) 0x7f016ca449c0: post pdl (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.649815 +0000 HTTP (trace) 0x7f016ca449c0: baggage (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.649833 +0000 HTTP (trace) 0x7f016ca449c0: post dc2 (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.649860 +0000 HTTP (trace) lws_context_destroy3: ctx 0x20f9f70 freed (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/http-libwebsockets.c:62) 71: 2021-12-08 20:08:23.650037 +0000 CONTAINER (trace) Default node removed (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/container.c:849) 71: 2021-12-08 20:08:23.650078 +0000 ROUTER_CORE (info) Router Core thread exited (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/router_core_thread.c:256) 71: 2021-12-08 20:08:23.650170 +0000 HTTP_ADAPTOR (trace) Shutting down HTTP2 Protocol adaptor (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:2702) 71: 2021-12-08 20:08:23.650214 +0000 HTTP_ADAPTOR (info) [C1] Freeing stream_data (stream_dispatcher, qdr_http2_adaptor_final) (225de50) (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:2710) 71: 2021-12-08 20:08:23.650233 +0000 HTTP_ADAPTOR (info) [C1] Freeing http2 connection (calling free_qdr_http2_connection) (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:2714) 71: 2021-12-08 20:08:23.650248 +0000 HTTP_ADAPTOR (trace) [C1][S1] Freeing stream in free_qdr_http2_connection (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:80) 71: 2021-12-08 20:08:23.650273 +0000 HTTP_ADAPTOR (debug) Adding http request info o_127.0.0.1_examples (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http_common.c:697) 71: 2021-12-08 20:08:23.650315 +0000 HTTP_ADAPTOR (trace) [C1][S1] Freeing stream_data in free_http2_stream_data (7f01540af450) (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:374) 71: 2021-12-08 20:08:23.650500 +0000 ROUTER_CORE (debug) [C1][L4][D2] Delivery decref: rc:1 HTTP2 adaptor out_dlv - free_http2_stream_data (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/delivery.c:149) 71: 2021-12-08 20:08:23.650767 +0000 HTTP_ADAPTOR (trace) [C1] Freeing http2 connection in free_qdr_http2_connection (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http2/http2_adaptor.c:459) 71: 2021-12-08 20:08:23.650826 +0000 TCP_ADAPTOR (info) Shutting down TCP protocol adaptor (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/tcp_adaptor.c:1760) 71: 2021-12-08 20:08:23.651215 +0000 ROUTER_CORE (debug) [C1][L4][D2] Delivery outcome : is unknown (0x0) (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/delivery.c:392) 71: 2021-12-08 20:08:23.651246 +0000 ROUTER_CORE (debug) [C1][L4][D2] Delivery decref_CT: rc:0 qdr_link_cleanup_deliveries_CT - remove from undelivered list (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/delivery.c:640) 71: 2021-12-08 20:08:23.651298 +0000 ROUTER_CORE (info) Finalizing core module: streaming_link_scrubber (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/router_core_thread.c:138) 71: 2021-12-08 20:08:23.651316 +0000 ROUTER_CORE (info) Finalizing core module: stuck_delivery_detection (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/router_core_thread.c:138) 71: 2021-12-08 20:08:23.651333 +0000 ROUTER_CORE (info) Finalizing core module: heartbeat_server (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/router_core_thread.c:138) 71: 2021-12-08 20:08:23.651348 +0000 ROUTER_CORE (info) Finalizing core module: address_lookup_client (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/router_core/router_core_thread.c:138) 71: 2021-12-08 20:08:23.651381 +0000 HTTP_ADAPTOR (debug) Added http request info o_127.0.0.1_examples (2) (/home/jenkins/workspace/dispatch/dispatch-rhel9-main/source/dispatch/src/adaptors/http_common.c:626) 71: 71: <<<< 71: ERROR {noformat} -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org