[
https://issues.apache.org/jira/browse/KAFKA-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733344#comment-16733344
]
ASF GitHub Bot commented on KAFKA-6928:
---------------------------------------
guozhangwang commented on pull request #6085: KAFKA-6928: Refactor
StreamsPartitionAssignor retry logic
URL: https://github.com/apache/kafka/pull/6085
1. The retry loop of the InternalTopicManager would just be: a) describe
topics, and exclude those which already exist with the right num.partitions, b)
for the remaining topics, try to create them. Remove any inner loops.
2. In CreateTopicResponse and MetadataResponse (for describe topic), handle
the special error code of TopicExist and UnknownTopicOrPartition in order to
retry in the next loop.
3. Do not handle TimeoutException since it should already been handled
inside AdminClient.
Add corresponding unit tests for a) topic marked for deletion but not
complete yet, in which case metadata response would not contain this topic, but
create topic would return error TopicExists; b) request keep getting timed out.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> StreamsPartitionAssignor is double retrying within InternalTopicManager
> -----------------------------------------------------------------------
>
> Key: KAFKA-6928
> URL: https://issues.apache.org/jira/browse/KAFKA-6928
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Guozhang Wang
> Priority: Major
>
> Today inside StreamsPartitionAssingor, we are using the admin client to 1)
> check num.partitions for internal topics, 2) try to create internal topics
> that are not exist yet. We are using the StreamsConfig.RETRIES_CONFIG inside
> the InternalTopicManager to capture thrown exceptions from the resulted
> KafkaFuture; however inside the AdminClient, we are again using the same
> config with its own retrying logic.
> We should get rid of the outer retrying logic later; as of now (2018.05)
> there are still some issues with the admin client such that its own retrying
> logic is not safe. I'll leave it to [~hachikuji] to decide when admin
> client's own retrying is safe to rely on.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
