Yash Mayya created KAFKA-14569:
----------------------------------
Summary: Migrate EmbeddedKafkaCluster used by Connect integration
tests from EmbeddedZookeeper to KRaft
Key: KAFKA-14569
URL: https://issues.apache.org/jira/browse/KAFKA-14569
Project: Kafka
Issue Type: Task
Components: KafkaConnect
Reporter: Yash Mayya
Assignee: Yash Mayya
ZooKeeper mode is going to be deprecated in Apache Kafka 4.0. Connect currently
uses an
[EmbeddedKafkaCluster|https://github.com/apache/kafka/blob/b8ab09820cd96290176afd24cf7b03e7cda7f783/connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java#L95]
for integration tests that depends on an
[EmbeddedZookeeper.|https://github.com/apache/kafka/blob/b8ab09820cd96290176afd24cf7b03e7cda7f783/connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java#L147]
This should be migrated to remove the ZooKeeper dependency, instead working in
the KRaft mode (probably with co-located brokers and controllers). We could
potentially leverage the existing test kit for KRaft clusters
([https://github.com/apache/kafka/tree/b8ab09820cd96290176afd24cf7b03e7cda7f783/core/src/test/java/kafka/testkit)]
which handles a bunch of stuff including the listeners configuration setup,
formatting the metadata log directory, allowing usage of non-static random
ports for `controller.quorum.voters`, initialization of the shared server(s),
broker(s), and controller(s) etc.
One more thing to note is that some Connect integration tests currently use the
`kafka.security.authorizer.AclAuthorizer` which requires ZooKeeper. These tests
should be migrated to use the new authorizer from
[KIP-801|https://cwiki.apache.org/confluence/display/KAFKA/KIP-801%3A+Implement+an+Authorizer+that+stores+metadata+in+__cluster_metadata]
if we want to completely eliminate the dependency on ZooKeeper.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
