Hi All, (I have asked this on SO as well, but happy to paste the response there if I get a answer here or vice-versa).
I would like to know the impact of setting both of these options on the Producer API. Can someone please let me know why metadata has not expired in the Scenario-1 even when metadata.max.age.ms has elapsed. Scenario-1 ------------ set metadata.max.age.ms to 10 seconds. Leave metadata.max.idle.ms as default of 5 minutes. start Kafka producer. publish some messages to a topic using this producer. shutdown the Kafka cluster. allow more than 10 seconds to elapse.----> I am hoping that this would cause the producer to automatically trigger a request to update Metadata. Now,try to produce a message to the topic using KafkaProducer#send from the producer. This operation returns immediately, without blocking. I was expecting this to block as more than 10 seconds have elapsed and was expecting metadata to expire based on the metadata.max.age.ms being set to 10 seconds. Scenario-2 ------------ However, if do the following, then the KafkaProducer#send operation blocks : set metadata.max.age.ms and metadata.max.idle.ms both to 10 seconds. start Kafka producer publish some messages to a topic using this producer. shutdown the Kafka cluster. allow more than 10 seconds to elapse. try to produce a message to the topic using KafkaProducer#send from the producer. This operation blocks trying to fetch metadata. I am not sure why it blocks now, but not in the first scenario.
