[
https://issues.apache.org/jira/browse/KAFKA-19925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Godinho updated KAFKA-19925:
------------------------------------
Description:
Multiple tests in transactions_upgrade_test.py
([https://github.com/apache/kafka/blob/trunk/tests/kafkatest/tests/core/transactions_upgrade_test.py])
timeout
!image-2025-11-26-01-22-37-797.png|width=516,height=157!
E.g. Log:
{code:java}
INFO:2025-11-26 01:02:07,863]: Triggering test 3 of 9...
[INFO:2025-11-26 01:02:07,889]: RunnerClient: Loading test {'directory':
'/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name':
'transactions_upgrade_test.py', 'cls_name': 'TransactionsUpgradeTest',
'method_name': 'test_transactions_upgrade', 'injected_args':
{'from_kafka_version': '3.5.2', 'metadata_quorum': 'ISOLATED_KRAFT',
'group_protocol': None}}
[INFO:2025-11-26 01:02:07,903]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
on run 1/1
[INFO:2025-11-26 01:02:07,906]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Setting up...
[INFO:2025-11-26 01:02:07,908]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Running...
[INFO:2025-11-26 01:08:18,352]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Tearing down...
[INFO:2025-11-26 01:11:42,521]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
FAIL: TimeoutError('copier-0 - Failed to copy all messages in 180s.')
Traceback (most recent call last):
File
"/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line
351, in _do_run
data = self.run_test()
File
"/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line
411, in run_test
return self.test_context.function(self.test)
File "/usr/local/lib/python3.10/dist-packages/ducktape/mark/_mark.py", line
438, in wrapper
return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
File
"/opt/kafka-dev/tests/kafkatest/tests/core/transactions_upgrade_test.py", line
235, in test_transactions_upgrade
concurrently_consumed_messages =
self.copy_messages_transactionally_during_upgrade(
File
"/opt/kafka-dev/tests/kafkatest/tests/core/transactions_upgrade_test.py", line
184, in copy_messages_transactionally_during_upgrade
wait_until(lambda: copier.is_done,
File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line
58, in wait_until
raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from
last_exception
ducktape.errors.TimeoutError: copier-0 - Failed to copy all messages in
180s.[WARNING - 2025-11-26 01:11:42,521 - runner_client - log - lineno:459]:
RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Test requested 8 nodes, used only 7
[WARNING:2025-11-26 01:11:42,522]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Test requested 8 nodes, used only 7
[INFO:2025-11-26 01:11:42,523]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Data: None {code}
was:
Multiple tests in
[transactions_upgrade_test.py|[https://github.com/apache/kafka/blob/trunk/tests/kafkatest/tests/core/transactions_upgrade_test.py]]
sometimes times out
!image-2025-11-26-01-22-37-797.png|width=516,height=157!
E.g. Log:
{code:java}
INFO:2025-11-26 01:02:07,863]: Triggering test 3 of 9...
[INFO:2025-11-26 01:02:07,889]: RunnerClient: Loading test {'directory':
'/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name':
'transactions_upgrade_test.py', 'cls_name': 'TransactionsUpgradeTest',
'method_name': 'test_transactions_upgrade', 'injected_args':
{'from_kafka_version': '3.5.2', 'metadata_quorum': 'ISOLATED_KRAFT',
'group_protocol': None}}
[INFO:2025-11-26 01:02:07,903]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
on run 1/1
[INFO:2025-11-26 01:02:07,906]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Setting up...
[INFO:2025-11-26 01:02:07,908]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Running...
[INFO:2025-11-26 01:08:18,352]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Tearing down...
[INFO:2025-11-26 01:11:42,521]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
FAIL: TimeoutError('copier-0 - Failed to copy all messages in 180s.')
Traceback (most recent call last):
File
"/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line
351, in _do_run
data = self.run_test()
File
"/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line
411, in run_test
return self.test_context.function(self.test)
File "/usr/local/lib/python3.10/dist-packages/ducktape/mark/_mark.py", line
438, in wrapper
return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
File
"/opt/kafka-dev/tests/kafkatest/tests/core/transactions_upgrade_test.py", line
235, in test_transactions_upgrade
concurrently_consumed_messages =
self.copy_messages_transactionally_during_upgrade(
File
"/opt/kafka-dev/tests/kafkatest/tests/core/transactions_upgrade_test.py", line
184, in copy_messages_transactionally_during_upgrade
wait_until(lambda: copier.is_done,
File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line
58, in wait_until
raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from
last_exception
ducktape.errors.TimeoutError: copier-0 - Failed to copy all messages in
180s.[WARNING - 2025-11-26 01:11:42,521 - runner_client - log - lineno:459]:
RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Test requested 8 nodes, used only 7
[WARNING:2025-11-26 01:11:42,522]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Test requested 8 nodes, used only 7
[INFO:2025-11-26 01:11:42,523]: RunnerClient:
kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
Data: None {code}
> Flaky E2E transactions_upgrade_test
> -----------------------------------
>
> Key: KAFKA-19925
> URL: https://issues.apache.org/jira/browse/KAFKA-19925
> Project: Kafka
> Issue Type: Test
> Components: system tests
> Reporter: Francis Godinho
> Assignee: Francis Godinho
> Priority: Major
> Labels: flaky-test
> Attachments: image-2025-11-26-01-22-37-797.png
>
>
> Multiple tests in transactions_upgrade_test.py
> ([https://github.com/apache/kafka/blob/trunk/tests/kafkatest/tests/core/transactions_upgrade_test.py])
> timeout
> !image-2025-11-26-01-22-37-797.png|width=516,height=157!
>
> E.g. Log:
> {code:java}
> INFO:2025-11-26 01:02:07,863]: Triggering test 3 of 9...
> [INFO:2025-11-26 01:02:07,889]: RunnerClient: Loading test {'directory':
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name':
> 'transactions_upgrade_test.py', 'cls_name': 'TransactionsUpgradeTest',
> 'method_name': 'test_transactions_upgrade', 'injected_args':
> {'from_kafka_version': '3.5.2', 'metadata_quorum': 'ISOLATED_KRAFT',
> 'group_protocol': None}}
> [INFO:2025-11-26 01:02:07,903]: RunnerClient:
> kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
> on run 1/1
> [INFO:2025-11-26 01:02:07,906]: RunnerClient:
> kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
> Setting up...
> [INFO:2025-11-26 01:02:07,908]: RunnerClient:
> kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
> Running...
> [INFO:2025-11-26 01:08:18,352]: RunnerClient:
> kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
> Tearing down...
> [INFO:2025-11-26 01:11:42,521]: RunnerClient:
> kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
> FAIL: TimeoutError('copier-0 - Failed to copy all messages in 180s.')
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py",
> line 351, in _do_run
> data = self.run_test()
> File
> "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py",
> line 411, in run_test
> return self.test_context.function(self.test)
> File "/usr/local/lib/python3.10/dist-packages/ducktape/mark/_mark.py", line
> 438, in wrapper
> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
> File
> "/opt/kafka-dev/tests/kafkatest/tests/core/transactions_upgrade_test.py",
> line 235, in test_transactions_upgrade
> concurrently_consumed_messages =
> self.copy_messages_transactionally_during_upgrade(
> File
> "/opt/kafka-dev/tests/kafkatest/tests/core/transactions_upgrade_test.py",
> line 184, in copy_messages_transactionally_during_upgrade
> wait_until(lambda: copier.is_done,
> File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line
> 58, in wait_until
> raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from
> last_exception
> ducktape.errors.TimeoutError: copier-0 - Failed to copy all messages in
> 180s.[WARNING - 2025-11-26 01:11:42,521 - runner_client - log - lineno:459]:
> RunnerClient:
> kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
> Test requested 8 nodes, used only 7
> [WARNING:2025-11-26 01:11:42,522]: RunnerClient:
> kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
> Test requested 8 nodes, used only 7
> [INFO:2025-11-26 01:11:42,523]: RunnerClient:
> kafkatest.tests.core.transactions_upgrade_test.TransactionsUpgradeTest.test_transactions_upgrade.from_kafka_version=3.5.2.metadata_quorum=ISOLATED_KRAFT.group_protocol=None:
> Data: None {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
