gharris1727 commented on code in PR #16053: URL: https://github.com/apache/kafka/pull/16053#discussion_r1626593367
########## connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java: ########## @@ -1123,6 +1125,24 @@ private void processTasksCommitRecord(String connectorName, SchemaAndValue value connectorTaskConfigGenerations.compute(connectorName, (ignored, generation) -> generation != null ? generation + 1 : 0); } inconsistent.remove(connectorName); + + Map<String, String> rawConnectorConfig = connectorConfigs.get(connectorName); + Map<String, String> appliedConnectorConfig; + if (configTransformer != null) { + try { + appliedConnectorConfig = configTransformer.transform(rawConnectorConfig); Review Comment: Let's say that there's 1000 connector configs in the config topic. Today I think the configTransformer only transforms that config when it's in-use (like a connector being started, fenced, updated, etc). That should only be a handful of times after the startup completes. With the code as-written, I think this would make 1000 configTransformer requests, including ones on out-of-date configurations. If the ConfigProvider is making a network request, this has the potential to be slow/expensive, and generate a spike of load that wasn't there before. Do you think there's a simple way to avoid this burst of ConfigProvider calls during startup? The built-in AK providers can handle the burst safely, but I wonder if there are third-party providers that could have bad side-effects. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org