[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-24 Thread Josep Prat (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17859746#comment-17859746
 ] 

Josep Prat commented on KAFKA-16990:


Then I'll change the priority to "major" as it's not a blocker anymore for 3.8. 
Unless you want to change the "fix version"

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.8.0
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Major
> Fix For: 3.8.0
>
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remot

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-24 Thread Justine Olshan (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17859743#comment-17859743
 ] 

Justine Olshan commented on KAFKA-16990:


Sorry I will actually reopen until trunk is merged. 

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.8.0
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Blocker
> Fix For: 3.8.0
>
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", 
> line 35, in wrapper
>     return method(self,

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-24 Thread Justine Olshan (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17859733#comment-17859733
 ] 

Justine Olshan commented on KAFKA-16990:


[https://github.com/apache/kafka/commit/35b34a85a7bed1f2e9bd23ae961cac85602435e6]
I will need to do a similar fix for trunk, but was trying to unblock 3.8 
quickly.

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.8.0
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Blocker
> Fix For: 3.8.0
>
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-24 Thread Josep Prat (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17859730#comment-17859730
 ] 

Josep Prat commented on KAFKA-16990:


Which was the PR fixing this?

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.8.0
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Blocker
> Fix For: 3.8.0
>
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", 
> line 35, in wrapper
>     return method(self, *args, **kwargs)
>   File 
> "

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-24 Thread Justine Olshan (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17859727#comment-17859727
 ] 

Justine Olshan commented on KAFKA-16990:


This was a blocker, but I merged the fix. (Believe it or not there were two 
bugs in this test :( )
I will mark this as complete.

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.8.0
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Blocker
> Fix For: 3.8.0
>
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktap

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-24 Thread Josep Prat (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17859589#comment-17859589
 ] 

Josep Prat commented on KAFKA-16990:


[~jolshan] Shall we then remove this as a blocker for 3.8 and leave 
https://issues.apache.org/jira/browse/KAFKA-17011 as the blocker one?

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.8.0
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Blocker
> Fix For: 3.8.0
>
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/lib/python3.9/dist-packages/duckta

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-20 Thread Justine Olshan (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856622#comment-17856622
 ] 

Justine Olshan commented on KAFKA-16990:


new Jira here: https://issues.apache.org/jira/browse/KAFKA-17011

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.8.0
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Blocker
> Fix For: 3.8.0
>
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", 
> line 35, in wrapper
>     return

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-20 Thread Justine Olshan (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856616#comment-17856616
 ] 

Justine Olshan commented on KAFKA-16990:


While fixing this bug I encountered a new one :) 


With this change: 
[https://github.com/apache/kafka/commit/bfe81d622979809325c31d549943c40f6f0f7337],
 we made it possible for the minimum version to be 0. However, when we upgrade, 
we can send to an old node and get this response:
{{java.lang.IllegalArgumentException: Expected minValue >= 1, maxValue >= 1 and 
maxValue >= minValue, but received minValue: 0, maxValue: 0 at 
org.apache.kafka.common.feature.BaseVersionRange.(BaseVersionRange.java:65)
 at 
org.apache.kafka.common.feature.SupportedVersionRange.(SupportedVersionRange.java:32)
 at org.apache.kafka.clients.NodeApiVersions.(NodeApiVersions.java:112) 
at 
org.apache.kafka.clients.NetworkClient.handleApiVersionsResponse(NetworkClient.java:920)
 at 
org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:888)
 at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:569) at 
kafka.common.InterBrokerSendThread.pollOnce(InterBrokerSendThread.scala:74) at 
kafka.server.BrokerToControllerRequestThread.doWork(BrokerToControllerChannelManager.scala:368)
 at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)}}
I think maybe we should either: # Not allow min version to be 0 (and just 
assume if a value is not set it is zero) I believe this is the approach 
group.version took
 # Have some gating on the api versions request so we don’t send 0 if the 
version is too low (can’t support the change)

I can file a new Jira for this if we think it is easier to track, but I can't 
fully test the other tests until it is.
 

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.8.0
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Blocker
> Fix For: 3.8.0
>
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-forma

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-20 Thread Josep Prat (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856563#comment-17856563
 ] 

Josep Prat commented on KAFKA-16990:


Thanks [~jolshan] 

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 3.8.0
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Blocker
> Fix For: 3.8.0
>
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", 
> line 35, in wrapper
>     return method(self, *args, **kwargs)
>   File 
> "/usr/local/

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-20 Thread Justine Olshan (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856535#comment-17856535
 ] 

Justine Olshan commented on KAFKA-16990:


[~jlprat] I believe this is a 3.8 blocker since the tests will fail. [~dajac] 
and I will take a loot and fix as soon as we can.

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Reporter: Gaurav Narula
>Assignee: Justine Olshan
>Priority: Major
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", 
> line 35, in wrapper
>     retur

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-18 Thread Gaurav Narula (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856078#comment-17856078
 ] 

Gaurav Narula commented on KAFKA-16990:
---

Thanks [~jolshan]!


{quote}I'm wondering why this is set this way considering new group coordinator 
is not available for the original "old" version.{quote}

IIUC, the flag is gated on {{use_new_coordinator}} and that was introduced with 
KAFKA-15578/[PR-14582|https://github.com/apache/kafka/commit/150b0e8290cda57df668ba89f6b422719866de5a#diff-0f57108563ac01c46ead76bc036e43a5ac988cbc4b17e314739fbd2fe7a1bf9e].
 Perhaps the intention is to start the cluster with the new group coordinator 
_after_ the upgrade?

I'm assigning the JIRA to you. Hope that's okay.

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Reporter: Gaurav Narula
>Assignee: Gaurav Narula
>Priority: Major
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671,

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-18 Thread Justine Olshan (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856020#comment-17856020
 ] 

Justine Olshan commented on KAFKA-16990:


Likely we are trying to set the new group coordinator with an old storage tool 
that doesn't recognize it. I'm wondering why this is set this way considering 
new group coordinator is not available for the original "old" version.

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Reporter: Gaurav Narula
>Assignee: Gaurav Narula
>Priority: Major
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/loca

[jira] [Commented] (KAFKA-16990) Unrecognised flag passed to kafka-storage.sh in system test

2024-06-18 Thread Justine Olshan (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856016#comment-17856016
 ] 

Justine Olshan commented on KAFKA-16990:


I can take a look.
cc: [~dajac] 

> Unrecognised flag passed to kafka-storage.sh in system test
> ---
>
> Key: KAFKA-16990
> URL: https://issues.apache.org/jira/browse/KAFKA-16990
> Project: Kafka
>  Issue Type: Test
>Reporter: Gaurav Narula
>Assignee: Gaurav Narula
>Priority: Major
>
> Running 
> {{TC_PATHS="tests/kafkatest/tests/core/kraft_upgrade_test.py::TestKRaftUpgrade"
>  bash tests/docker/run_tests.sh}} on trunk (c4a3d2475f) fails with the 
> following:
> {code:java}
> [INFO:2024-06-18 09:16:03,139]: Triggering test 2 of 32...
> [INFO:2024-06-18 09:16:03,147]: RunnerClient: Loading test {'directory': 
> '/opt/kafka-dev/tests/kafkatest/tests/core', 'file_name': 
> 'kraft_upgrade_test.py', 'cls_name': 'TestKRaftUpgrade', 'method_name': 
> 'test_isolated_mode_upgrade', 'injected_args': {'from_kafka_version': 
> '3.1.2', 'use_new_coordinator': True, 'metadata_quorum': 'ISOLATED_KRAFT'}}
> [INFO:2024-06-18 09:16:03,151]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  on run 1/1
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Setting up...
> [INFO:2024-06-18 09:16:03,153]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Running...
> [INFO:2024-06-18 09:16:05,999]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  Tearing down...
> [INFO:2024-06-18 09:16:12,366]: RunnerClient: 
> kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT:
>  FAIL: RemoteCommandError({'ssh_config': {'host': 'ducker10', 'hostname': 
> 'ducker10', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': 
> '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker10', 
> 'ssh_hostname': 'ducker10', 'user': 'ducker', 'externally_routable_ip': 
> 'ducker10', '_logger':  kafkatest.tests.core.kraft_upgrade_test.TestKRaftUpgrade.test_isolated_mode_upgrade.from_kafka_version=3.1.2.use_new_coordinator=True.metadata_quorum=ISOLATED_KRAFT-2
>  (DEBUG)>, 'os': 'linux', '_ssh_client':  0x85bccc70>, '_sftp_client':  0x85bccdf0>, '_custom_ssh_exception_checks': None}, 
> '/opt/kafka-3.1.2/bin/kafka-storage.sh format --ignore-formatted --config 
> /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w -f 
> group.version=1', 1, b"usage: kafka-storage format [-h] --config CONFIG 
> --cluster-id CLUSTER_ID\n                     
> [--ignore-formatted]\nkafka-storage: error: unrecognized arguments: '-f'\n")
> Traceback (most recent call last):
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 186, in _do_run
>     data = self.run_test()
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", 
> line 246, in run_test
>     return self.test_context.function(self.test)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 
> 433, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 132, in test_isolated_mode_upgrade
>     self.run_upgrade(from_kafka_version, group_protocol)
>   File "/opt/kafka-dev/tests/kafkatest/tests/core/kraft_upgrade_test.py", 
> line 96, in run_upgrade
>     self.kafka.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 669, in 
> start
>     self.isolated_controller_quorum.start()
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 671, in 
> start
>     Service.start(self, **kwargs)
>   File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", 
> line 265, in start
>     self.start_node(node, **kwargs)
>   File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 902, in 
> start_node
>     node.account.ssh(cmd)
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", 
> line 35, in wrapper
>     return method(self, *args, **kwargs)
>   File 
> "/usr/local/lib/python3.9/dist-packages/ducktape/clus