[ https://issues.apache.org/jira/browse/KAFKA-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias J. Sax resolved KAFKA-6047. ------------------------------------ Resolution: Duplicate The new {{KafkaAdminClient}} allows to configure number of retries. After we have replace out internal {{StreamsKafkaClient}} with the new client, this issue is resolved. > Allow retries configuration for InternalTopicManager > ---------------------------------------------------- > > Key: KAFKA-6047 > URL: https://issues.apache.org/jira/browse/KAFKA-6047 > Project: Kafka > Issue Type: Improvement > Components: streams > Affects Versions: 0.11.0.0 > Reporter: Dmitry Vsekhvalnov > Assignee: Matthias J. Sax > Priority: Minor > > There is hardcoded number of retries when kafka-streams attempts to create > internal topics. > *InternalTopicManager.MAX_TOPIC_READY_TRY* > Which is not resilient in some scenarios. Consider setup where replication > factor for internal streams topics == number of brokers. (RF=3 and x3 kafka > brokers). When any of brokers dies kafka-streams can shutdown before broker > is resurrected with approximate log: > {code} > [WARN ] [org.apache.kafka.streams.processor.internals.InternalTopicManager] > [Could not create internal topics: Found only 2 brokers, but replication > factor is 3. Decrease replication factor for internal topics via > StreamsConfig parameter "replication.factor" or add more brokers to your > cluster. Retry #2] > [WARN ] [org.apache.kafka.streams.processor.internals.InternalTopicManager] > [Could not create internal topics: Found only 2 brokers, but replication > factor is 3. Decrease replication factor for internal topics via > StreamsConfig parameter "replication.factor" or add more brokers to your > cluster. Retry #3] > [WARN ] [org.apache.kafka.streams.processor.internals.InternalTopicManager] > [Could not create internal topics: Found only 2 brokers, but replication > factor is 3. Decrease replication factor for internal topics via > StreamsConfig parameter "replication.factor" or add more brokers to your > cluster. Retry #4] > [INFO ] [org.apache.kafka.streams.processor.internals.StreamThread] > [stream-thread [Shutting down] > {code} > Would be nice if kafka-streams provides configuration for > InternalTopicManager retries and ideally for retry backoff strategy. To have > possibility for resilience tuning. Possibly can re-use corresponding producer > configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029)