[GitHub] [kafka] dajac commented on a change in pull request #9060: KAFKA-9274: Remove `retries` from InternalTopicManager
dajac commented on a change in pull request #9060:
URL: https://github.com/apache/kafka/pull/9060#discussion_r460053049
##
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java
##
@@ -96,13 +104,15 @@ public InternalTopicManager(final Admin adminClient, final
StreamsConfig streams
// have existed with the expected number of partitions, or some create
topic returns fatal errors.
log.debug("Starting to validate internal topics {} in partition
assignor.", topics);
-int remainingRetries = retries;
+long currentWallClockMs = time.milliseconds();
+final long deadlineMs = currentWallClockMs + retryTimeoutMs;
Review comment:
Thinking a bit more about this, with the default, you may end up not
honouring the deadline. `createTopics` can take up to 1m so if you invoke one
when less than 1m is reaming before the deadline, you may not honour the
deadline. It may not be that important though.
If we want to strictly enforce it, we could calculate the maximum timeout
for each call, something like deadline - now, and set it with
`CreateTopicsOptions.timeoutMs`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] dajac commented on a change in pull request #9060: KAFKA-9274: Remove `retries` from InternalTopicManager
dajac commented on a change in pull request #9060:
URL: https://github.com/apache/kafka/pull/9060#discussion_r459582710
##
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java
##
@@ -96,13 +104,15 @@ public InternalTopicManager(final Admin adminClient, final
StreamsConfig streams
// have existed with the expected number of partitions, or some create
topic returns fatal errors.
log.debug("Starting to validate internal topics {} in partition
assignor.", topics);
-int remainingRetries = retries;
+long currentWallClockMs = time.milliseconds();
+final long deadlineMs = currentWallClockMs + retryTimeoutMs;
Review comment:
That's right. I misread the default value of `max.poll.interval.ms`, too
many zeros for my eyes ;). The default works fine then. Do we want to protect
ourselves if the user changes the default? Or shall we just call out that
`api.default.timeout.ms` should be lower than `max.poll.interval.ms` somewhere?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] dajac commented on a change in pull request #9060: KAFKA-9274: Remove `retries` from InternalTopicManager
dajac commented on a change in pull request #9060:
URL: https://github.com/apache/kafka/pull/9060#discussion_r459258170
##
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java
##
@@ -96,13 +104,15 @@ public InternalTopicManager(final Admin adminClient, final
StreamsConfig streams
// have existed with the expected number of partitions, or some create
topic returns fatal errors.
log.debug("Starting to validate internal topics {} in partition
assignor.", topics);
-int remainingRetries = retries;
+long currentWallClockMs = time.milliseconds();
+final long deadlineMs = currentWallClockMs + retryTimeoutMs;
Review comment:
If we want to guarantee that the `deadlineMs` is respected, I think that
we must set the timeout of the AdminClient's call accordingly:
`CreateTopicsOptions.timeoutMs`. With the default, I think that the call could
be longer than half of `MAX_POLL_INTERVAL_MS_CONFIG`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] dajac commented on a change in pull request #9060: KAFKA-9274: Remove `retries` from InternalTopicManager
dajac commented on a change in pull request #9060:
URL: https://github.com/apache/kafka/pull/9060#discussion_r459258170
##
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java
##
@@ -96,13 +104,15 @@ public InternalTopicManager(final Admin adminClient, final
StreamsConfig streams
// have existed with the expected number of partitions, or some create
topic returns fatal errors.
log.debug("Starting to validate internal topics {} in partition
assignor.", topics);
-int remainingRetries = retries;
+long currentWallClockMs = time.milliseconds();
+final long deadlineMs = currentWallClockMs + retryTimeoutMs;
Review comment:
If we want to guarantee that the `deadlineMs` is respected, I think that
we must set the timeout of the AdminClient's call accordingly:
'CreateTopicsOptions.timeoutMs`. With the default, I think that the call could
be longer than half of `MAX_POLL_INTERVAL_MS_CONFIG`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
