Hi Apache kafka team, As per the documentation, the kafka producer directly communicates with the leader of the topic partition to produce a message. In order to fetch the leader information, it makes a meta data fetch call before its first produce, further caches it and refreshes depending upon metadata.max.age.ms and metadata.max.idle.ms.
During the meta data fetch call, it makes use of partitionsFor() call which has an upper bound of max.block.ms. What if the broker it tried to connect to is not reachable, how can we enable the retry of the same call preferably with another broker (as we have multiple brokers in the bootstrap servers list)? It looks like the retry configured for the producer comes into picture when there is a transient error while receiving the ack back from the broker when the message is sent by the client (aka the sender thread) and is not applicable for steps that happen before facilitating it aka fetching the metadata. Can you help us understand if there is any property which will enable the said retry or will we need to take care of this situation by ourselves? I am wondering if properties like connections.max.idle.ms, request.timeout.ms, rec could play a role in facilitating the retries? And properties like reconnect.backoff.ms and reconnect.backoff.max.ms can help reconnect with other brokers when one has already failed? *Yash Tailor*
