Hi I wonder how the KafkaProducer Metadata version works. I have some KafkaProducers in Production that started failing and will not recover after a topic was recreated (deleted and created again). Those are shared producers, so I stopped publication of that topic before the procedure, but I believe when we re-enabled publication, our system is picking the shared producers with the old metadata, and therefore is failing with the following error:
Flush failed org.apache.kafka.common.errors.TimeoutException: Topic mytopic not present in metadata after 60000 ms. https://github.com/a0x8o/kafka/blob/master/clients/src/main/java/org/apache/kafka/clients/producer/internals/ProducerMetadata.java#L122 By looking at the source code, it looks like the Producer will try to update the metadata, waiting for some expected version to be retrieved, and will timeout. How does that work? Is this version stored in Kafka somehow, so that when we recreated the topic the versions will not match? Isn't the KafkaProducer able to refresh its metadata when a topic is recreated? Thanks Murilo