[ https://issues.apache.org/jira/browse/DISPATCH-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484347#comment-17484347 ]
Jiri Daněk commented on DISPATCH-2323: -------------------------------------- I actually never before realized what the elephantine problem at the centre of the whole thing is: Python cannot shut down its HTTPServer if there is a client connected to it. Because that client is the qdrouterd, which is supposed to stay up over the duration of the entire testclass, this is... unpleasant. Currently the tests solve this by POSTING a special /SHUTDOWN request through the qdrouterd to the HTTPServer. When processing that, there is opportune moment for the server to shut down itself. > [http1] system_tests_http1_adaptor failure due to Address already in use > issue (second round) > --------------------------------------------------------------------------------------------- > > Key: DISPATCH-2323 > URL: https://issues.apache.org/jira/browse/DISPATCH-2323 > Project: Qpid Dispatch > Issue Type: Bug > Components: Protocol Adaptors > Affects Versions: 1.15.0 > Reporter: Jiri Daněk > Assignee: Jiri Daněk > Priority: Major > Fix For: 1.17.0 > > > https://github.com/apache/qpid-dispatch/runs/4993651356?check_suite_focus=true#step:27:1904 > {noformat} > 36/37 Test #70: system_tests_http1_adaptor ........................***Failed > 351.39 sec > ============================= test session starts > ============================== > platform linux -- Python 3.6.8, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- > /usr/bin/python3 > cachedir: .pytest_cache > rootdir: /__w/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/tests, > configfile: tox.ini > collecting ... collected 28 items > ::Http1AdaptorManagementTest::test_01_create_delete FAILED > ::Http1AdaptorManagementTest::test_01_delete_active_connector FAILED > ::Http1AdaptorOneRouterTest::test_000_stats PASSED > ::Http1AdaptorOneRouterTest::test_001_get PASSED > ::Http1AdaptorOneRouterTest::test_002_head PASSED > ::Http1AdaptorOneRouterTest::test_003_post PASSED > ::Http1AdaptorOneRouterTest::test_004_put PASSED > ::Http1AdaptorOneRouterTest::test_005_get_10 PASSED > ::Http1AdaptorOneRouterTest::test_006_head_10 PASSED > ::Http1AdaptorOneRouterTest::test_007_post_10 PASSED > ::Http1AdaptorOneRouterTest::test_008_put_10 PASSED > ::Http1AdaptorEdge2EdgeTest::test_01_concurrent_requests ERROR > ::Http1AdaptorEdge2EdgeTest::test_02_credit_replenish ERROR > ::Http1AdaptorEdge2EdgeTest::test_03_server_reconnect ERROR > ::Http1AdaptorEdge2EdgeTest::test_04_server_pining_for_the_fjords ERROR > ::Http1AdaptorEdge2EdgeTest::test_05_large_streaming_msg ERROR > ::Http1AdaptorEdge2EdgeTest::test_1001_client_request_close ERROR > ::Http1AdaptorEdge2EdgeTest::test_1002_client_response_close ERROR > ::Http1AdaptorEdge2EdgeTest::test_2000_curl_get ERROR > ::Http1AdaptorEdge2EdgeTest::test_2001_curl_put ERROR > ::Http1AdaptorEdge2EdgeTest::test_2002_curl_post ERROR > ::Http1AdaptorBadEndpointsTest::test_01_unsolicited_response PASSED > ::Http1AdaptorBadEndpointsTest::test_02_bad_request_message PASSED > ::Http1AdaptorBadEndpointsTest::test_03_bad_response_message PASSED > ::Http1AdaptorBadEndpointsTest::test_04_client_request_close PASSED > ::Http1AdaptorBadEndpointsTest::test_05_client_response_close PASSED > ::Http1AdaptorQ2Standalone::test_01_backpressure_client PASSED > ::Http1AdaptorQ2Standalone::test_02_backpressure_server PASSED > ==================================== ERRORS > ==================================== > ___ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_01_concurrent_requests > ____ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > name = 'EA2', mode = 'edge' > extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', > 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': > 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, > 'protocolVersion': 'HTTP1'})] > @classmethod > def router(cls, name, mode, extra): > config = [ > ('router', {'mode': mode, > 'id': name, > 'allowUnsettledMulticast': 'yes'}), > ('listener', {'role': 'normal', > 'port': cls.tester.get_port()}), > ('address', {'prefix': 'closest', 'distribution': 'closest'}), > ('address', {'prefix': 'multicast', 'distribution': 'multicast'}), > ] > > if extra: > config.extend(extra) > config = Qdrouterd.Config(config) > > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > _____ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_02_credit_replenish > _____ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > name = 'EA2', mode = 'edge' > extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', > 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': > 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, > 'protocolVersion': 'HTTP1'})] > @classmethod > def router(cls, name, mode, extra): > config = [ > ('router', {'mode': mode, > 'id': name, > 'allowUnsettledMulticast': 'yes'}), > ('listener', {'role': 'normal', > 'port': cls.tester.get_port()}), > ('address', {'prefix': 'closest', 'distribution': 'closest'}), > ('address', {'prefix': 'multicast', 'distribution': 'multicast'}), > ] > > if extra: > config.extend(extra) > config = Qdrouterd.Config(config) > > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > _____ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_03_server_reconnect > _____ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > name = 'EA2', mode = 'edge' > extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', > 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': > 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, > 'protocolVersion': 'HTTP1'})] > @classmethod > def router(cls, name, mode, extra): > config = [ > ('router', {'mode': mode, > 'id': name, > 'allowUnsettledMulticast': 'yes'}), > ('listener', {'role': 'normal', > 'port': cls.tester.get_port()}), > ('address', {'prefix': 'closest', 'distribution': 'closest'}), > ('address', {'prefix': 'multicast', 'distribution': 'multicast'}), > ] > > if extra: > config.extend(extra) > config = Qdrouterd.Config(config) > > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > _ ERROR at setup of > Http1AdaptorEdge2EdgeTest.test_04_server_pining_for_the_fjords _ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > name = 'EA2', mode = 'edge' > extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', > 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': > 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, > 'protocolVersion': 'HTTP1'})] > @classmethod > def router(cls, name, mode, extra): > config = [ > ('router', {'mode': mode, > 'id': name, > 'allowUnsettledMulticast': 'yes'}), > ('listener', {'role': 'normal', > 'port': cls.tester.get_port()}), > ('address', {'prefix': 'closest', 'distribution': 'closest'}), > ('address', {'prefix': 'multicast', 'distribution': 'multicast'}), > ] > > if extra: > config.extend(extra) > config = Qdrouterd.Config(config) > > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > ___ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_05_large_streaming_msg > ____ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > name = 'EA2', mode = 'edge' > extra = [('connector', {'host': '127.0.0.1', 'idleTimeoutSeconds': '120', > 'name': 'uplink', 'port': 25519, ...}), ('httpConnec...protocolVersion': > 'HTTP1'}), ('httpConnector', {'address': 'testServer10', 'port': 25522, > 'protocolVersion': 'HTTP1'})] > @classmethod > def router(cls, name, mode, extra): > config = [ > ('router', {'mode': mode, > 'id': name, > 'allowUnsettledMulticast': 'yes'}), > ('listener', {'role': 'normal', > 'port': cls.tester.get_port()}), > ('address', {'prefix': 'closest', 'distribution': 'closest'}), > ('address', {'prefix': 'multicast', 'distribution': 'multicast'}), > ] > > if extra: > config.extend(extra) > config = Qdrouterd.Config(config) > > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > __ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_1001_client_request_close > __ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > @classmethod > def setUpClass(cls): > """Start a router""" > super(Http1AdaptorEdge2EdgeTest, cls).setUpClass() > > # configuration: > # one edge, one interior > # > # +-------+ +---------+ +-------+ > # | EA1 |<==>| INT.A |<==>| EA2 | > # +-------+ +---------+ +-------+ > # ^ ^ > # | | > # V V > # <clients> <servers> > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('INT.A', 'interior', [('listener', {'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('listener', {'role': 'edge', > 'port': cls.INTA_edge2_port}), > ]) > cls.INT_A = cls.routers[0] > cls.INT_A.listener = cls.INT_A.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA1', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('httpListener', {'port': cls.http_listener11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpListener', {'port': cls.http_listener10_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer10'}) > ]) > cls.EA1 = cls.routers[1] > cls.EA1.listener = cls.EA1.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA2', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge2_port}), > ('httpConnector', {'port': cls.http_server11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpConnector', {'port': cls.http_server10_port, > 'protocolVersion': 'HTTP1', > > 'address': 'testServer10'}) > ]) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > in router > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > _ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_1002_client_response_close > __ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > @classmethod > def setUpClass(cls): > """Start a router""" > super(Http1AdaptorEdge2EdgeTest, cls).setUpClass() > > # configuration: > # one edge, one interior > # > # +-------+ +---------+ +-------+ > # | EA1 |<==>| INT.A |<==>| EA2 | > # +-------+ +---------+ +-------+ > # ^ ^ > # | | > # V V > # <clients> <servers> > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('INT.A', 'interior', [('listener', {'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('listener', {'role': 'edge', > 'port': cls.INTA_edge2_port}), > ]) > cls.INT_A = cls.routers[0] > cls.INT_A.listener = cls.INT_A.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA1', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('httpListener', {'port': cls.http_listener11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpListener', {'port': cls.http_listener10_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer10'}) > ]) > cls.EA1 = cls.routers[1] > cls.EA1.listener = cls.EA1.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA2', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge2_port}), > ('httpConnector', {'port': cls.http_server11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpConnector', {'port': cls.http_server10_port, > 'protocolVersion': 'HTTP1', > > 'address': 'testServer10'}) > ]) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > in router > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > ________ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_2000_curl_get > ________ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > @classmethod > def setUpClass(cls): > """Start a router""" > super(Http1AdaptorEdge2EdgeTest, cls).setUpClass() > > # configuration: > # one edge, one interior > # > # +-------+ +---------+ +-------+ > # | EA1 |<==>| INT.A |<==>| EA2 | > # +-------+ +---------+ +-------+ > # ^ ^ > # | | > # V V > # <clients> <servers> > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('INT.A', 'interior', [('listener', {'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('listener', {'role': 'edge', > 'port': cls.INTA_edge2_port}), > ]) > cls.INT_A = cls.routers[0] > cls.INT_A.listener = cls.INT_A.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA1', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('httpListener', {'port': cls.http_listener11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpListener', {'port': cls.http_listener10_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer10'}) > ]) > cls.EA1 = cls.routers[1] > cls.EA1.listener = cls.EA1.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA2', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge2_port}), > ('httpConnector', {'port': cls.http_server11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpConnector', {'port': cls.http_server10_port, > 'protocolVersion': 'HTTP1', > > 'address': 'testServer10'}) > ]) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > in router > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > ________ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_2001_curl_put > ________ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > @classmethod > def setUpClass(cls): > """Start a router""" > super(Http1AdaptorEdge2EdgeTest, cls).setUpClass() > > # configuration: > # one edge, one interior > # > # +-------+ +---------+ +-------+ > # | EA1 |<==>| INT.A |<==>| EA2 | > # +-------+ +---------+ +-------+ > # ^ ^ > # | | > # V V > # <clients> <servers> > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('INT.A', 'interior', [('listener', {'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('listener', {'role': 'edge', > 'port': cls.INTA_edge2_port}), > ]) > cls.INT_A = cls.routers[0] > cls.INT_A.listener = cls.INT_A.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA1', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('httpListener', {'port': cls.http_listener11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpListener', {'port': cls.http_listener10_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer10'}) > ]) > cls.EA1 = cls.routers[1] > cls.EA1.listener = cls.EA1.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA2', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge2_port}), > ('httpConnector', {'port': cls.http_server11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpConnector', {'port': cls.http_server10_port, > 'protocolVersion': 'HTTP1', > > 'address': 'testServer10'}) > ]) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > in router > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > _______ ERROR at setup of Http1AdaptorEdge2EdgeTest.test_2002_curl_post > ________ > cls = <class 'system_tests_http1_adaptor.Http1AdaptorEdge2EdgeTest'> > @classmethod > def setUpClass(cls): > """Start a router""" > super(Http1AdaptorEdge2EdgeTest, cls).setUpClass() > > # configuration: > # one edge, one interior > # > # +-------+ +---------+ +-------+ > # | EA1 |<==>| INT.A |<==>| EA2 | > # +-------+ +---------+ +-------+ > # ^ ^ > # | | > # V V > # <clients> <servers> > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('INT.A', 'interior', [('listener', {'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('listener', {'role': 'edge', > 'port': cls.INTA_edge2_port}), > ]) > cls.INT_A = cls.routers[0] > cls.INT_A.listener = cls.INT_A.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA1', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge1_port}), > ('httpListener', {'port': cls.http_listener11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpListener', {'port': cls.http_listener10_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer10'}) > ]) > cls.EA1 = cls.routers[1] > cls.EA1.listener = cls.EA1.addresses[0] > > super(Http1AdaptorEdge2EdgeTest, cls).\ > router('EA2', 'edge', > [('connector', {'name': 'uplink', 'role': 'edge', > 'port': cls.INTA_edge2_port}), > ('httpConnector', {'port': cls.http_server11_port, > 'protocolVersion': 'HTTP1', > 'address': 'testServer11'}), > ('httpConnector', {'port': cls.http_server10_port, > 'protocolVersion': 'HTTP1', > > 'address': 'testServer10'}) > ]) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:391: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/http1_tests.py:1198: > in router > cls.routers.append(cls.tester.qdrouterd(name, config, wait=True)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:795: > in qdrouterd > return self.cleanup(Qdrouterd(*args, **kwargs)) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:481: > in __init__ > self.wait_ready() > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:705: > in wait_ready > self.wait_connectors(**retry_kwargs) > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > self = <system_test.Qdrouterd object at 0x7fed8feb5390>, retry_kwargs = {} > @py_assert1 = <function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8> > @py_assert4 = None > @py_format6 = 'Port not connected 25519\n>assert None\n{None = > retry(<function Qdrouterd.wait_connectors.<locals>.<lambda> at > 0x7fed8fe2e9d8>, **{})\n}' > def wait_connectors(self, **retry_kwargs): > """ > Wait for all connectors to be connected > @param retry_kwargs: keyword args for L{retry} > """ > for c in self.config.sections('connector'): > > assert retry(lambda: self.is_connected(port=c['port'], > > host=self.get_host(c.get('protocolFamily'))), > **retry_kwargs), "Port not connected %s" % c['port'] > E AssertionError: Port not connected 25519 > E assert None > E + where None = retry(<function > Qdrouterd.wait_connectors.<locals>.<lambda> at 0x7fed8fe2e9d8>, **{}) > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_test.py:697: > AssertionError > =================================== FAILURES > =================================== > _______________ Http1AdaptorManagementTest.test_01_create_delete > _______________ > self = <system_tests_http1_adaptor.Http1AdaptorManagementTest > testMethod=test_01_create_delete> > def test_01_create_delete(self): > """ Create and delete HTTP1 connectors and listeners. The > connectors/listeners are created on the edge router. Verify that the > adaptor properly notifies the interior of the subscribers/producers. > """ > e_mgmt = self.e_router.management > self.assertEqual(0, > len(e_mgmt.query(type=self.LISTENER_TYPE).results)) > self.assertEqual(0, > len(e_mgmt.query(type=self.CONNECTOR_TYPE).results)) > > e_mgmt.create(type=self.CONNECTOR_TYPE, > name="ServerConnector", > attributes={'address': 'closest/http1Service', > 'port': self.http_server_port, > 'protocolVersion': 'HTTP1'}) > > e_mgmt.create(type=self.LISTENER_TYPE, > name="ClientListener", > attributes={'address': 'closest/http1Service', > 'port': self.http_listener_port, > 'protocolVersion': 'HTTP1'}) > > # verify the entities have been created and http traffic works > > self.assertEqual(1, > len(e_mgmt.query(type=self.LISTENER_TYPE).results)) > self.assertEqual(1, > len(e_mgmt.query(type=self.CONNECTOR_TYPE).results)) > > count, error = http1_ping(sport=self.http_server_port, > cport=self.http_listener_port) > self.assertIsNone(error) > self.assertEqual(1, count) > > # now check the interior router for the closest/http1Service address > self.i_router.wait_address("closest/http1Service", subscribers=1) > > # > # delete the connector and listener; wait for the associated > connection > # to be removed > # > e_mgmt.delete(type=self.CONNECTOR_TYPE, name="ServerConnector") > self.assertEqual(0, > len(e_mgmt.query(type=self.CONNECTOR_TYPE).results)) > e_mgmt.delete(type=self.LISTENER_TYPE, name="ClientListener") > self.assertEqual(0, > len(e_mgmt.query(type=self.LISTENER_TYPE).results)) > > # will hit test timeout on failure: > while True: > hconns = 0 > obj = e_mgmt.query(type=self.CONNECTION_TYPE, > attribute_names=["protocol"]) > for item in obj.get_dicts(): > if "http/1.x" in item["protocol"]: > hconns += 1 > if hconns == 0: > break > sleep(0.25) > > # When a connector is configured the router will periodically attempt > # to connect to the server address. To prove that the connector has > # been completely removed listen for connection attempts on the server > # port. > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) > s.bind(("", self.http_server_port)) > s.setblocking(True) > s.settimeout(3) # reconnect attempts every 2.5 seconds > s.listen(1) > with self.assertRaises(socket.timeout): > > conn, addr = s.accept() > E AssertionError: timeout not raised > /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/tests/system_tests_http1_adaptor.py:158: > AssertionError > __________ Http1AdaptorManagementTest.test_01_delete_active_connector > __________ > self = <system_tests_http1_adaptor.Http1AdaptorManagementTest > testMethod=test_01_delete_active_connector> > def test_01_delete_active_connector(self): > """Delete an HTTP1 connector that is currently connected to a server. > Verify the connection is dropped. > """ > e_mgmt = self.e_router.management > self.assertEqual(0, > len(e_mgmt.query(type=self.CONNECTOR_TYPE).results)) > > e_mgmt.create(type=self.CONNECTOR_TYPE, > name="ServerConnector", > attributes={'address': 'closest/http1Service', > 'port': self.http_server_port, > 'protocolVersion': 'HTTP1'}) > > # verify the connector has been created and attach a dummy server > self.assertEqual(1, > len(e_mgmt.query(type=self.CONNECTOR_TYPE).results)) > > server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) > > server.bind(("", self.http_server_port)) > E OSError: [Errno 98] Address already in use > {noformat} > One scenario against which the tests are not protecting is that something > without SO_REUSEADDR was listening on the socket before. In that case, > attempt to connect will fail (nothing is there listening now) but a new bind > will fail too (we are lingering after use). The port range dispatch is using > is chosen not to fall into > {code} > $ cat /proc/sys/net/ipv4/ip_local_port_range > 32768 60999 > {code} > so we should be safe in that respect (local randomly assigned ports don't > clash). Still, it seems sensible to me to try checking a bind() before we > hand over the port. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org