heartbeat.interval.ms should be lower than session.timeout.ms. Check here: http://kafka.apache.org/0101/documentation.html#newconsumerconfigs
On Thu, May 24, 2018 at 2:39 PM, Shantanu Deshmukh <shantanu...@gmail.com> wrote: > Someone please help me. I am suffering due to this issue since a long time > and not finding any solution. > > On Wed, May 23, 2018 at 3:48 PM Shantanu Deshmukh <shantanu...@gmail.com> > wrote: > > > We have a 3 broker Kafka 0.10.0.1 cluster. There we have 3 topics with 10 > > partitions each. We have an application which spawns threads as > consumers. > > We spawn 5 consumers for each topic. I am observing that consider group > > randomly keeps rebalancing. Then many times we see logs saying "Revoking > > partitions for". This happens almost every 10 minutes. Consumption during > > this time completely stops. > > > > I have applied this configuration > > max.poll.records 20 > > heartbeat.interval.ms 10000 > > Session.timeout.ms 6000 > > > > Still this did not help. Strange thing is I observed consumer writing > logs > > saying "auto commit failed because poll() loop spent too much time > > processing records" even when there was no data in partition to process. > We > > have polling interval of 500 ms, specified as argument in poll(). > Initially > > I had set same consumer group for all three topics' consumers. Then I > > specified different CGs for different topics' consumers. Even this is not > > helping. > > > > I am trying to search over the web, checked my code, tried many > > combinations of configuration but still no luck. Please help me. > > > > *Thanks & Regards,* > > > > *Shantanu Deshmukh* > > >
