soondenana commented on a change in pull request #9347:
URL: https://github.com/apache/kafka/pull/9347#discussion_r496983721
##########
File path:
connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java
##########
@@ -133,11 +133,14 @@ public void start() {
List<TopicPartition> partitions = new ArrayList<>();
// We expect that the topics will have been created either manually by
the user or automatically by the herder
- List<PartitionInfo> partitionInfos = null;
- long started = time.milliseconds();
- while (partitionInfos == null && time.milliseconds() - started <
CREATE_TOPIC_TIMEOUT_MS) {
+ List<PartitionInfo> partitionInfos = consumer.partitionsFor(topic);
+ long started = time.nanoseconds();
+ long maxSleepMs = 1_000;
+ long sleepMs = 10;
+ while (partitionInfos == null && time.nanoseconds() - started <
CREATE_TOPIC_TIMEOUT_NS) {
+ time.sleep(sleepMs);
+ sleepMs = Math.min(2 * sleepMs, maxSleepMs);
partitionInfos = consumer.partitionsFor(topic);
Review comment:
I think the semantics needed here is different. The timeout in
`partitionsFor` is the max amount of time the api can block waiting for
response before it fails with `TimeoutException`. However, the api can return
within timeout with empty results as newly created topics data has not been
propagated yet. We then have to retry again until `partitionsFor` returns the
partition data (upto a max time).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
