C0urante commented on code in PR #11781:
URL: https://github.com/apache/kafka/pull/11781#discussion_r896234648
##########
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java:
##########
@@ -1327,30 +1334,39 @@ public WorkerTask doBuild(Task task,
connectorClientConfigOverridePolicy, kafkaClusterId);
KafkaProducer<byte[], byte[]> producer = new
KafkaProducer<>(producerProps);
- TopicAdmin topicAdmin;
+ // Prepare to create a topic admin if the task requires one, but
do not actually create an instance
+ // until/unless one is needed
+ final AtomicReference<TopicAdmin> topicAdmin = new
AtomicReference<>();
+ final Supplier<TopicAdmin> topicAdminCreator = () ->
topicAdmin.updateAndGet(existingAdmin -> {
+ if (existingAdmin != null) {
+ return existingAdmin;
+ }
+ Map<String, Object> adminOverrides =
adminConfigs(id.connector(), "connector-adminclient-" + id, config,
+ sourceConfig, connectorClass,
connectorClientConfigOverridePolicy, kafkaClusterId, ConnectorType.SOURCE);
+ Admin adminClient = Admin.create(adminOverrides);
+ return new
TopicAdmin(adminOverrides.get(BOOTSTRAP_SERVERS_CONFIG), adminClient);
+ });
+
Map<String, TopicCreationGroup> topicCreationGroups;
if (config.topicCreationEnable() &&
sourceConfig.usesTopicCreation()) {
topicCreationGroups =
TopicCreationGroup.configuredGroups(sourceConfig);
// Create a topic admin that the task can use for topic
creation
- Map<String, Object> adminOverrides =
adminConfigs(id.connector(), "connector-adminclient-" + id, config,
- sourceConfig, connectorClass,
connectorClientConfigOverridePolicy, kafkaClusterId, ConnectorType.SOURCE);
- topicAdmin = new TopicAdmin(adminOverrides);
+ topicAdminCreator.get();
Review Comment:
This call won't result in a leak because the `AbstractWorkerSourceTask`
class will always close its `TopicAdmin` during shutdown; see
[AbstractWorkerSourceTask::close](https://github.com/C0urante/kafka/blob/fe675683b4671714d02c8e674bd36347d0c345e9/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L306-L308).
And, given the idempotent nature of `Admin::close`, it's fine to close it from
both the `AbstractWorkerSourceTask` and `ConnectorOffsetBackingStore` classes.
The underlying point about ownership being unclear is fair, though, and the
control flow that creates the topic admin in this method is especially
convoluted. I've added a couple comments that hopefully clear things up. I wish
the logic itself could be better but as it is, things are easy to test and
that's taken advantage of by adding a ton of coverage in the [WorkerTest
suite](https://github.com/C0urante/kafka/blob/fe675683b4671714d02c8e674bd36347d0c345e9/connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTest.java#L1592-L1653).
Happy to refactor if there's an easily-testable approach that's clearer,
though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
