HI Jason, Thanks for your feedback.
I believe your suggestion on how to take advantage of this assignor is valid. We can leverage onPartitionsRevoked() and onPartitionsAssigned() callbacks and do a comparison of assigned partitions before and after the re-balance and do the cleanup only if there is a change (e.g., if some previously assigned partition is not in the assignment). On your second question, a number of tests that I ran shows that the old assignments are preserved in the current implementation; except for when the consumer group leader is killed; in which case, a fresh assignment is performed. This is something that needs to be fixed. I tried to use your pointers to find out where the best place is to preserve the old assignment in such circumstances but have not been able to pinpoint it. If you have any suggestion on this please share. Thanks. Regards, Vahid Hashemian From: Jason Gustafson <ja...@confluent.io> To: dev@kafka.apache.org Date: 04/14/2016 11:37 AM Subject: Re: [DISCUSS] KIP-54 Sticky Partition Assignment Strategy Hi Vahid, Thanks for the proposal. I think one of the advantages of having sticky assignment would be reduce the need to cleanup local partition state between rebalances. Do you have any thoughts on how the user would take advantage of this assignor in the consumer to do this? Maybe one approach is to delay cleanup until you detect a change from the previous assignment in the onPartitionsAssigned() callback? Also, can you provide some detail on how the sticky assignor works at the group protocol level? For example, do you pass old assignments through the "UserData" field in the consumer's JoinGroup? Thanks, Jason On Thu, Apr 14, 2016 at 11:05 AM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi all, > > I have started a new KIP under > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-54+-+Sticky+Partition+Assignment+Strategy > The corresponding JIRA is at > https://issues.apache.org/jira/browse/KAFKA-2273 > The corresponding PR is at https://github.com/apache/kafka/pull/1020 > > Your feedback is much appreciated. > > Regards, > Vahid Hashemian > >
