I would like to add a little more to this context, the problem is not hard
to reproduce.
If you are using
- auto commit
- heartbeat time = commit time
- more than one consumer
It seems that is always failing to send the heart beat. Changing the values
for the heartbeat and commit to be
Hi Jason
Thanks for looking into this.
Created this: https://issues.apache.org/jira/browse/KAFKA-3470
First time do it. Not sure if I have followed any necessary convention
there.
To your question:
No, this should not be a *significant* problem for any client as there
are workarounds (commit les
Hi Zaiming,
Yeah, you're right. Changing coordinator won't cause a rebalance (it hasn't
been that way since we added group metadata persistence). I went back and
checked the code and we actually do not reset the heartbeat timer when a
commit is received. I'm not sure whether there's a good reason
Hi Jason
If I understand correctly, when coordinator is changed the consumer
should get 'NotCoordinatorForGroup' exception not 'IllegalGenerationId'.
Topic metadata change? like number of partitions changed ?
I was testing it in a pretty stable cluster, and it was reproduced several
times,
I had n
Hi Zaiming,
It rules out the most likely cause of rebalance, but not the only one.
Rebalances can also be caused by a topic metadata change or a coordinator
change. Can you post some logs from the consumer around the time that the
unexpected rebalance occurred?
-Jason
On Fri, Mar 25, 2016 at 12:
Hi Jason
thanks for the reply!
Forgot to mention that in we tried to test the simplest scenario in which
there was only one member in the group. I think that should rule out group
rebalancing right?
On Thursday, March 24, 2016, Jason Gustafson wrote:
> HI Zaiming,
>
> I think the problem is n
HI Zaiming,
I think the problem is not that commit requests aren't considered as
effective as heartbeats (they are), but that you can't rejoin the group
using only commits/heartbeats. Every time the group rebalances, all members
must rejoin the group by sending a JoinGroup request. Once a rebalanc
Hi there!
We have noticed that when committing requests are sent intensively, we
receive IllegalGenerationId.
Here is the settings we had problem with: session-timeout: 30 sec,
heartbeat-rate: 3 sec.
Problem resolved by increasing the session timeout to 180 sec.
So I suppose, due to whatever reas
