[jira] [Commented] (KAFKA-16944) Range assignor doesn't co-partition with stickiness
[ https://issues.apache.org/jira/browse/KAFKA-16944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17856061#comment-17856061 ] Ritika Reddy commented on KAFKA-16944: -- Hey! Thanks for showing interest in this task! We've already started working on this internally so I think we're good for now! > Range assignor doesn't co-partition with stickiness > --- > > Key: KAFKA-16944 > URL: https://issues.apache.org/jira/browse/KAFKA-16944 > Project: Kafka > Issue Type: Sub-task >Reporter: Ritika Reddy >Assignee: Ritika Reddy >Priority: Major > > When stickiness is considered during range assignments, it is possible that > in certain cases where co-partitioning is guaranteed we fail. > An example would be: > Consider two topics T1, T2 with 3 partitions each and three members A, B, C. > Let's say the existing assignment (for whatever reason) is: > {quote}A -> T1P0 || B -> T1P1, T2P0, T2P1, T2P2 || C -> T1P2 > {quote} > Now we trigger a rebalance with the following subscriptions where all members > are subscribed to both topics everything else is the same > {quote}A -> T1, T2 || B -> T1, T2 || C -> T1, T2 > {quote} > Since all the topics have an equal number of partitions and all the members > are subscribed to the same set of topics we would expect co-partitioning > right so would we want the final assignment returned to be > {quote}A -> T1P0, T2P0 || B -> T1P1, T2P1 || C -> T1P2, T2P2 > {quote} > SO currently the client side assignor returns the following but it's because > they don't assign sticky partitions > {{{}C=[topic1-2, topic2-2], B=[topic1-1, topic2-1], A=[topic1-0, > topic2-0]{}}}Our > > Server side assignor returns: > (The partitions in bold are the sticky partitions) > {{{}A=MemberAssignment(targetPartitions={topic2=[1], > }}\{{{}{*}topic1=[0]{*}{}}}{{{}}), > B=MemberAssignment(targetPartitions={{}}}{{{}*topic2=[0]*{}}}{{{}, > {{{{{}*topic1=[1]*{}}}{{{}}), > C=MemberAssignment(targetPartitions={topic2=[2], {{{{{}*topic1=[2]*{}}} > *As seen above co-partitioning is expected but not returned.* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-16944) Range assignor doesn't co-partition with stickiness
[ https://issues.apache.org/jira/browse/KAFKA-16944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17854593#comment-17854593 ] dujian0068 commented on KAFKA-16944: Hello, does this problem need to be fixed quickly? If not, can it be assigned to me? It is not easy for me to find a problem, and it may take me some time to deal with it > Range assignor doesn't co-partition with stickiness > --- > > Key: KAFKA-16944 > URL: https://issues.apache.org/jira/browse/KAFKA-16944 > Project: Kafka > Issue Type: Sub-task >Reporter: Ritika Reddy >Assignee: Ritika Reddy >Priority: Major > > When stickiness is considered during range assignments, it is possible that > in certain cases where co-partitioning is guaranteed we fail. > An example would be: > Consider two topics T1, T2 with 3 partitions each and three members A, B, C. > Let's say the existing assignment (for whatever reason) is: > {quote}A -> T1P0 || B -> T1P1, T2P0, T2P1, T2P2 || C -> T1P2 > {quote} > Now we trigger a rebalance with the following subscriptions where all members > are subscribed to both topics everything else is the same > {quote}A -> T1, T2 || B -> T1, T2 || C -> T1, T2 > {quote} > Since all the topics have an equal number of partitions and all the members > are subscribed to the same set of topics we would expect co-partitioning > right so would we want the final assignment returned to be > {quote}A -> T1P0, T2P0 || B -> T1P1, T2P1 || C -> T1P2, T2P2 > {quote} > SO currently the client side assignor returns the following but it's because > they don't assign sticky partitions > {{{}C=[topic1-2, topic2-2], B=[topic1-1, topic2-1], A=[topic1-0, > topic2-0]{}}}Our > > Server side assignor returns: > (The partitions in bold are the sticky partitions) > {{{}A=MemberAssignment(targetPartitions={topic2=[1], > }}\{{{}{*}topic1=[0]{*}{}}}{{{}}), > B=MemberAssignment(targetPartitions={{}}}{{{}*topic2=[0]*{}}}{{{}, > {{{{{}*topic1=[1]*{}}}{{{}}), > C=MemberAssignment(targetPartitions={topic2=[2], {{{{{}*topic1=[2]*{}}} > *As seen above co-partitioning is expected but not returned.* -- This message was sent by Atlassian Jira (v8.20.10#820010)