Guozhang Wang created KAFKA-6534:
------------------------------------
Summary: Consumer.poll may not trigger rebalance in time when
there is a task migration
Key: KAFKA-6534
URL: https://issues.apache.org/jira/browse/KAFKA-6534
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Guozhang Wang
When Streams detect a task migration event in one of its thread, today it will
always let its trigger to call {{consumer.poll}} hoping it could trigger the
rebalance and hence clean up the records buffered from the partitions that on
longer owned. However, because the rebalance is based on HB responses which has
a window of race, the rebalance is not always guaranteed to be triggered when
task migration happens. As a result it could cause the records buffered in
consumer to not be cleaned up and later be processed by Streams, realizing it
no longer belongs to the thread, causing:
{code}
java.lang.IllegalStateException: Record's partition does not belong to this
partition-group.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
