Jun Yao created KAFKA-4385:
------------------------------
Summary: producer is sending too many unnecessary meta data
request if the meta data for a topic is not available
Key: KAFKA-4385
URL: https://issues.apache.org/jira/browse/KAFKA-4385
Project: Kafka
Issue Type: Bug
Reporter: Jun Yao
All current kafka-client producer implementation (<= 0.10.1.0),
When sending a msg to a topic, it will first check if meta data for this topic
is available or not,
when not available, it will set "metadata.requestUpdate()" and wait for meta
data from brokers,
The thing is inside "org.apache.kafka.clients.Metadata.awaitUpdate()", it's
already doing a "while (this.version <= lastVersion)" loop waiting for new
version response,
So the loop inside
"org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata() is not
needed,
When "auto.create.topics.enable" is false, sending msgs to a non-exist topic
will trigger too many meta requests, everytime a metadata response is returned,
because it does not contain the metadata for the topic, it's going to try again
until TimeoutException is thrown;
This is a waste and sometimes causes too much overhead when unexpected msgs are
arrived.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
