C0urante commented on code in PR #11781:
URL: https://github.com/apache/kafka/pull/11781#discussion_r896806289
##########
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java:
##########
@@ -1327,30 +1334,39 @@ public WorkerTask doBuild(Task task,
connectorClientConfigOverridePolicy, kafkaClusterId);
KafkaProducer<byte[], byte[]> producer = new
KafkaProducer<>(producerProps);
- TopicAdmin topicAdmin;
+ // Prepare to create a topic admin if the task requires one, but
do not actually create an instance
+ // until/unless one is needed
+ final AtomicReference<TopicAdmin> topicAdmin = new
AtomicReference<>();
+ final Supplier<TopicAdmin> topicAdminCreator = () ->
topicAdmin.updateAndGet(existingAdmin -> {
+ if (existingAdmin != null) {
+ return existingAdmin;
+ }
+ Map<String, Object> adminOverrides =
adminConfigs(id.connector(), "connector-adminclient-" + id, config,
+ sourceConfig, connectorClass,
connectorClientConfigOverridePolicy, kafkaClusterId, ConnectorType.SOURCE);
+ Admin adminClient = Admin.create(adminOverrides);
+ return new
TopicAdmin(adminOverrides.get(BOOTSTRAP_SERVERS_CONFIG), adminClient);
+ });
+
Map<String, TopicCreationGroup> topicCreationGroups;
if (config.topicCreationEnable() &&
sourceConfig.usesTopicCreation()) {
topicCreationGroups =
TopicCreationGroup.configuredGroups(sourceConfig);
// Create a topic admin that the task can use for topic
creation
- Map<String, Object> adminOverrides =
adminConfigs(id.connector(), "connector-adminclient-" + id, config,
- sourceConfig, connectorClass,
connectorClientConfigOverridePolicy, kafkaClusterId, ConnectorType.SOURCE);
- topicAdmin = new TopicAdmin(adminOverrides);
+ topicAdminCreator.get();
Review Comment:
👍 I added the logic for checking those conditions to `doBuild`. It's still a
little clunky since we're checking
`regularSourceTaskUsesConnectorSpecificOffsetsStore` in two places now
(`doBuild` and `offsetStoreForRegularSourceTask`), but for testing purposes
it's easier to work with since we can continue to unit test
`offsetStoreForRegularSourceTask` independently without having to go through
`doBuild`. And it's certainly more readable than the caching `Supplier` logic
that it replaced.
Good point about ensuring that we stop `WorkerSourceTask` instances, we
definitely do a poor job of that when we fail to build a task. In fact, this
has already been documented as a bug with [an accompanying fix
PR](https://github.com/apache/kafka/pull/11608). If that's worth pursuing at
some point let me know and I can rebase onto the latest trunk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
