[
https://issues.apache.org/jira/browse/KAFKA-10340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282498#comment-17282498
]
Goltseva Taisiia commented on KAFKA-10340:
------------------------------------------
Hi, guys!
According to [KAFKA-5295](https://issues.apache.org/jira/browse/KAFKA-5295) I
supposed that now topics are always created by AdminClient if
"topic.creation.enable = true".
But now I see that topics are created by AdminClient, only when
"topic.creation.enable = true" and "topic.creation" parameters are used.
But I wonder why?
Why do not create topics always by AdminClient if "topic.creation.enable =
true"?
It will partially solve the current problem (described in this ticket) and only
the case with "topic.creation.enable = false" is the problem.
And also current approach can be a problem for us - we wanted to set
"auto.create.topics.enable = false" on Kafka cluster by default. But we cannot
do that, because our connectors can be created with
"topic.creation" parameters or without them, we cannot guarantee that. And in
case connectors have not this property, we get errors.
Any comments and your opinion is appreciated!
> Source connectors should report error when trying to produce records to
> non-existent topics instead of hanging forever
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-10340
> URL: https://issues.apache.org/jira/browse/KAFKA-10340
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Reporter: Arjun Satish
> Assignee: Chris Egerton
> Priority: Major
>
> Currently, a source connector will blindly attempt to write a record to a
> Kafka topic. When the topic does not exist, its creation is controlled by the
> {{auto.create.topics.enable}} config on the brokers. When auto.create is
> disabled, the producer.send() call on the Connect worker will hang
> indefinitely (due to the "infinite retries" configuration for said producer).
> In setups where this config is usually disabled, the source connector simply
> appears to hang and not produce any output.
> It is desirable to either log an info or an error message (or inform the user
> somehow) that the connector is simply stuck waiting for the destination topic
> to be created. When the worker has permissions to inspect the broker
> settings, it can use the {{listTopics}} and {{describeConfigs}} API in
> AdminClient to check if the topic exists, the broker can
> {{auto.create.topics.enable}} topics, and if these cases do not exist, either
> throw an error.
> With the recently merged
> [KIP-158|https://cwiki.apache.org/confluence/display/KAFKA/KIP-158%3A+Kafka+Connect+should+allow+source+connectors+to+set+topic-specific+settings+for+new+topics],
> this becomes even more specific a corner case: when topic creation settings
> are enabled, the worker should handle the corner case where topic creation is
> disabled, {{auto.create.topics.enable}} is set to false and topic does not
> exist.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
