Re: Kafka consumer is taking time to recognize new partition

2020-05-20 Thread Liam Clarke-Hutchinson
Hi, You want metadata.max.age.ms which, as you noticed, defaults to 5 minutes :) https://kafka.apache.org/documentation/#metadata.max.age.ms Cheers, Liam Clarke-Hutchinson On Thu, May 21, 2020 at 1:06 PM Kafka Shil wrote: > I was running a test where kafka consumer was reading data from mult

Kafka consumer is taking time to recognize new partition

2020-05-20 Thread Kafka Shil
I was running a test where kafka consumer was reading data from multiple partitions of a topic. While the process was running I added more partitions. It took around 5 minutes for consumer thread to read data from the new partition. I have found this configuration " topic.metadata.refresh.interval.