[ https://issues.apache.org/jira/browse/KAFKA-17084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863361#comment-17863361 ]
Mickael Maison commented on KAFKA-17084: ---------------------------------------- I can get both of these tests to pass in our CI [^TEST-kafka.xml] > Network Degrade Test fails in System Tests > ------------------------------------------ > > Key: KAFKA-17084 > URL: https://issues.apache.org/jira/browse/KAFKA-17084 > Project: Kafka > Issue Type: Bug > Components: system tests > Affects Versions: 3.8.0 > Reporter: Josep Prat > Priority: Critical > Attachments: TEST-kafka.xml > > > Tests for NetworkDegradeTest fail consistently on the 3.8 branch. > > Tests failing are: > > {noformat} > Module: kafkatest.tests.core.network_degrade_test > Class: NetworkDegradeTest > Method: test_latency > Arguments: > { > "device_name": "eth0", > "latency_ms": 50, > "rate_limit_kbit": 1000, > "task_name": "latency-100-rate-1000" > } > {noformat} > > and > > {noformat} > Module: kafkatest.tests.core.network_degrade_test > Class: NetworkDegradeTest > Method: test_latency > Arguments: > { > "device_name": "eth0", > "latency_ms": 50, > "rate_limit_kbit": 0, > "task_name": "latency-100" > } > {noformat} > > Failure for the first one is: > {noformat} > RemoteCommandError({'ssh_config': {'host': 'worker30', 'hostname': > '10.140.34.105', 'user': 'ubuntu', 'port': 22, 'password': None, > 'identityfile': '/home/semaphore/kafka-overlay/semaphore-muckrake.pem'}, > 'hostname': 'worker30', 'ssh_hostname': '10.140.34.105', 'user': 'ubuntu', > 'externally_routable_ip': '10.140.34.105', '_logger': <Logger > kafkatest.tests.core.network_degrade_test.NetworkDegradeTest.test_latency.task_name=latency-100-rate-1000.device_name=eth0.latency_ms=50.rate_limit_kbit=1000-1790 > (DEBUG)>, 'os': 'linux', '_ssh_client': <paramiko.client.SSHClient object at > 0x7f17a237dc10>, '_sftp_client': <paramiko.sftp_client.SFTPClient object at > 0x7f17a2393910>, '_custom_ssh_exception_checks': None}, 'ping -i 1 -c 20 > worker21', 1, b'') > Traceback (most recent call last): > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", > line 184, in _do_run > data = self.run_test() > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", > line 262, in run_test > return self.test_context.function(self.test) > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/mark/_mark.py", > line 433, in wrapper > return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs) > File > "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/network_degrade_test.py", > line 66, in test_latency > for line in zk0.account.ssh_capture("ping -i 1 -c 20 %s" % > zk1.account.hostname): > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py", > line 680, in next > return next(self.iter_obj) > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py", > line 347, in output_generator > raise RemoteCommandError(self, cmd, exit_status, stderr.read()) > ducktape.cluster.remoteaccount.RemoteCommandError: ubuntu@worker30: Command > 'ping -i 1 -c 20 worker21' returned non-zero exit status 1.{noformat} > And for the second one is: > {noformat} > RemoteCommandError({'ssh_config': {'host': 'worker28', 'hostname': > '10.140.41.79', 'user': 'ubuntu', 'port': 22, 'password': None, > 'identityfile': '/home/semaphore/kafka-overlay/semaphore-muckrake.pem'}, > 'hostname': 'worker28', 'ssh_hostname': '10.140.41.79', 'user': 'ubuntu', > 'externally_routable_ip': '10.140.41.79', '_logger': <Logger > kafkatest.tests.core.network_degrade_test.NetworkDegradeTest.test_latency.task_name=latency-100.device_name=eth0.latency_ms=50.rate_limit_kbit=0-1791 > (DEBUG)>, 'os': 'linux', '_ssh_client': <paramiko.client.SSHClient object at > 0x7f17a1c7b7c0>, '_sftp_client': <paramiko.sftp_client.SFTPClient object at > 0x7f17a1c7b2b0>, '_custom_ssh_exception_checks': None}, 'ping -i 1 -c 20 > worker27', 1, b'') > Traceback (most recent call last): > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", > line 184, in _do_run > data = self.run_test() > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", > line 262, in run_test > return self.test_context.function(self.test) > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/mark/_mark.py", > line 433, in wrapper > return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs) > File > "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/core/network_degrade_test.py", > line 66, in test_latency > for line in zk0.account.ssh_capture("ping -i 1 -c 20 %s" % > zk1.account.hostname): > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py", > line 680, in next > return next(self.iter_obj) > File > "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py", > line 347, in output_generator > raise RemoteCommandError(self, cmd, exit_status, stderr.read()) > ducktape.cluster.remoteaccount.RemoteCommandError: ubuntu@worker28: Command > 'ping -i 1 -c 20 worker27' returned non-zero exit status 1.{noformat} > -- This message was sent by Atlassian Jira (v8.20.10#820010)