Josep Prat created KAFKA-17085: ---------------------------------- Summary: Streams Cooperative Rebalance Upgrade Test fails in System Tests Key: KAFKA-17085 URL: https://issues.apache.org/jira/browse/KAFKA-17085 Project: Kafka Issue Type: Bug Components: system tests Affects Versions: 3.8.0 Reporter: Josep Prat
StreamsCooperativeRebalanceUpgradeTest fails on system tests when upgrading from: 2.1.1, 2.2.2 and 2.3.1. Tests that fail: {noformat} Module: kafkatest.tests.streams.streams_cooperative_rebalance_upgrade_test Class: StreamsCooperativeRebalanceUpgradeTest Method: test_upgrade_to_cooperative_rebalance Arguments: { "upgrade_from_version": "2.1.1" } {noformat} and {noformat} Module: kafkatest.tests.streams.streams_cooperative_rebalance_upgrade_test Class: StreamsCooperativeRebalanceUpgradeTest Method: test_upgrade_to_cooperative_rebalance Arguments: { "upgrade_from_version": "2.2.2" } {noformat} and {noformat} Module: kafkatest.tests.streams.streams_cooperative_rebalance_upgrade_test Class: StreamsCooperativeRebalanceUpgradeTest Method: test_upgrade_to_cooperative_rebalance Arguments: { "upgrade_from_version": "2.3.1" } {noformat} Failure for 2.1.1 is: {noformat} TimeoutError("Never saw 'first_bounce_phase-Processed [0-9]* records so far' message ubuntu@worker28") Traceback (most recent call last): File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run data = self.run_test() File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", line 262, in run_test return self.test_context.function(self.test) File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/mark/_mark.py", line 433, in wrapper return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs) File "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/streams/streams_cooperative_rebalance_upgrade_test.py", line 101, in test_upgrade_to_cooperative_rebalance self.maybe_upgrade_rolling_bounce_and_verify(processors, File "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/streams/streams_cooperative_rebalance_upgrade_test.py", line 182, in maybe_upgrade_rolling_bounce_and_verify stdout_monitor.wait_until(verify_processing_msg, File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py", line 736, in wait_until return wait_until(lambda: self.acct.ssh("tail -c +%d %s | grep '%s'" % (self.offset + 1, self.log, pattern), File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/utils/util.py", line 58, in wait_until raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception ducktape.errors.TimeoutError: Never saw 'first_bounce_phase-Processed [0-9]* records so far' message ubuntu@worker28{noformat} Failure for 2.2.2 is: {noformat} TimeoutError("Never saw 'first_bounce_phase-Processed [0-9]* records so far' message ubuntu@worker5") Traceback (most recent call last): File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run data = self.run_test() File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", line 262, in run_test return self.test_context.function(self.test) File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/mark/_mark.py", line 433, in wrapper return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs) File "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/streams/streams_cooperative_rebalance_upgrade_test.py", line 101, in test_upgrade_to_cooperative_rebalance self.maybe_upgrade_rolling_bounce_and_verify(processors, File "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/streams/streams_cooperative_rebalance_upgrade_test.py", line 182, in maybe_upgrade_rolling_bounce_and_verify stdout_monitor.wait_until(verify_processing_msg, File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py", line 736, in wait_until return wait_until(lambda: self.acct.ssh("tail -c +%d %s | grep '%s'" % (self.offset + 1, self.log, pattern), File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/utils/util.py", line 58, in wait_until raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception ducktape.errors.TimeoutError: Never saw 'first_bounce_phase-Processed [0-9]* records so far' message ubuntu@worker5{noformat} Failure for 2.3.1 is: {noformat} TimeoutError("Never saw 'first_bounce_phase-Processed [0-9]* records so far' message ubuntu@worker21") Traceback (most recent call last): File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run data = self.run_test() File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/tests/runner_client.py", line 262, in run_test return self.test_context.function(self.test) File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/mark/_mark.py", line 433, in wrapper return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs) File "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/streams/streams_cooperative_rebalance_upgrade_test.py", line 101, in test_upgrade_to_cooperative_rebalance self.maybe_upgrade_rolling_bounce_and_verify(processors, File "/home/semaphore/kafka-overlay/kafka/tests/kafkatest/tests/streams/streams_cooperative_rebalance_upgrade_test.py", line 182, in maybe_upgrade_rolling_bounce_and_verify stdout_monitor.wait_until(verify_processing_msg, File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/cluster/remoteaccount.py", line 736, in wait_until return wait_until(lambda: self.acct.ssh("tail -c +%d %s | grep '%s'" % (self.offset + 1, self.log, pattern), File "/home/semaphore/kafka-overlay/kafka/venv/lib/python3.8/site-packages/ducktape/utils/util.py", line 58, in wait_until raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception ducktape.errors.TimeoutError: Never saw 'first_bounce_phase-Processed [0-9]* records so far' message ubuntu@worker21{noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)