[jira] [Comment Edited] (KAFKA-13226) Partition expansion may cause uneven distribution
[
https://issues.apache.org/jira/browse/KAFKA-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435380#comment-17435380
]
shizhenzhen edited comment on KAFKA-13226 at 11/1/21, 2:40 AM:
---
I hava push a PR, please review it
https://github.com/apache/kafka/pull/11453
was (Author: shizhenzhen):
I hava push a PR, please review it
https://github.com/apache/kafka/pull/11445
> Partition expansion may cause uneven distribution
> -
>
> Key: KAFKA-13226
> URL: https://issues.apache.org/jira/browse/KAFKA-13226
> Project: Kafka
> Issue Type: Bug
> Components: controller
>Affects Versions: 2.5.0, 2.8.0, 2.7.1, 2.6.2, 2.8.1, 3.0.0
> Environment: mac
> kafka-2.5.0
>Reporter: shizhenzhen
>Priority: Major
>
>
> {color:#ff}*Partition expansion may cause uneven distribution*{color}
>
> 1. Create a Topic , 3-partition 1-replical
> !https://img-blog.csdnimg.cn/561112064b114acfb03882aa09100e0e.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_55,color_FF,t_70,g_se,x_16!
>
> 2. partition expansion to 5 - partiton
> !https://img-blog.csdnimg.cn/f7c3c33b6662457080d9bb5bb190c0c2.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_49,color_FF,t_70,g_se,x_16!
>
> 3. Does this meet expectations ？
>
> !https://img-blog.csdnimg.cn/20cc1007c4214c4ebfcb1b2c2eeb98e4.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_18,color_FF,t_70,g_se,x_16!
>
> {color:#ff}*so this is a bug ?*{color}
>
> The problem may arise here ;
> When we create a new topic . get the broker list is a Object Map ;
> *This is disordered*
> you can read the code , first it have sortBy brokerId, but finally it convert
> to a *disorde Map;*
>
>
> !https://img-blog.csdnimg.cn/131b9bf0c19e4753a73512af4c9c5854.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_66,color_FF,t_70,g_se,x_16!
>
>
>
> The important thing is that it has been sorted when expanding the partition
> and parition-reassignment ;
> {color:#ff}*So why not sort when creating topics?*{color}
>
> If the topic is sorted when create a new topic , this problem will not occur
> ;
>
> so it maybe is a tiny bug ?
>
>
> if you can read Chinese ,
> You can look at this article. I describe it in detail
>
> We look forward to receiving your reply
>
> 如果你能看懂中文,可以看看这篇文章 我描述的很详细!
> [This may be a Kafka
> bug？|[https://shirenchuang.blog.csdn.net/article/details/119912418]]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (KAFKA-13226) Partition expansion may cause uneven distribution
[
https://issues.apache.org/jira/browse/KAFKA-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435384#comment-17435384
]
shizhenzhen edited comment on KAFKA-13226 at 10/28/21, 1:03 PM:
When I tried to repair it, I found that even if I repaired the problem that the
toMap in kafkazkclient#getallbrokerandepochsincluster is not sorted, it still
can not solve this problem, because the brokerList will be cover when
updateMedata request is made,It's still not sorted, , so updatetata request
also needs to be processed, However, even if the whole cluster is not sorted,
there is no problem, except for createtopic, so when I solve this bug, I just
sort it in createtopic, so the change is the smallest,
was (Author: shizhenzhen):
When I tried to repair it, I found that even if I repaired the problem that the
toMap in kafkazkclient#getallbrokerandepochsincluster is not sorted, it still
can not solve this problem, because the brokerList will be cover when
updateMedata request is made,It's still not ordered, , so updatetata request
also needs to be processed, However, even if the whole cluster is not sorted,
there is no problem, except for createtopic, so when I solve this bug, I just
sort it in createtopic, so the change is the smallest,
> Partition expansion may cause uneven distribution
> -
>
> Key: KAFKA-13226
> URL: https://issues.apache.org/jira/browse/KAFKA-13226
> Project: Kafka
> Issue Type: Bug
> Components: controller
>Affects Versions: 2.5.0, 2.8.0, 2.7.1, 2.6.2, 2.8.1, 3.0.0
> Environment: mac
> kafka-2.5.0
>Reporter: shizhenzhen
>Priority: Major
>
>
> {color:#ff}*Partition expansion may cause uneven distribution*{color}
>
> 1. Create a Topic , 3-partition 1-replical
> !https://img-blog.csdnimg.cn/561112064b114acfb03882aa09100e0e.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_55,color_FF,t_70,g_se,x_16!
>
> 2. partition expansion to 5 - partiton
> !https://img-blog.csdnimg.cn/f7c3c33b6662457080d9bb5bb190c0c2.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_49,color_FF,t_70,g_se,x_16!
>
> 3. Does this meet expectations ？
>
> !https://img-blog.csdnimg.cn/20cc1007c4214c4ebfcb1b2c2eeb98e4.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_18,color_FF,t_70,g_se,x_16!
>
> {color:#ff}*so this is a bug ?*{color}
>
> The problem may arise here ;
> When we create a new topic . get the broker list is a Object Map ;
> *This is disordered*
> you can read the code , first it have sortBy brokerId, but finally it convert
> to a *disorde Map;*
>
>
> !https://img-blog.csdnimg.cn/131b9bf0c19e4753a73512af4c9c5854.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_66,color_FF,t_70,g_se,x_16!
>
>
>
> The important thing is that it has been sorted when expanding the partition
> and parition-reassignment ;
> {color:#ff}*So why not sort when creating topics?*{color}
>
> If the topic is sorted when create a new topic , this problem will not occur
> ;
>
> so it maybe is a tiny bug ?
>
>
> if you can read Chinese ,
> You can look at this article. I describe it in detail
>
> We look forward to receiving your reply
>
> 如果你能看懂中文,可以看看这篇文章 我描述的很详细!
> [This may be a Kafka
> bug？|[https://shirenchuang.blog.csdn.net/article/details/119912418]]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (KAFKA-13226) Partition expansion may cause uneven distribution
[
https://issues.apache.org/jira/browse/KAFKA-13226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435384#comment-17435384
]
shizhenzhen edited comment on KAFKA-13226 at 10/28/21, 1:02 PM:
When I tried to repair it, I found that even if I repaired the problem that the
toMap in kafkazkclient#getallbrokerandepochsincluster is not sorted, it still
can not solve this problem, because the brokerList will be cover when
updateMedata request is made,It's still not ordered, , so updatetata request
also needs to be processed, However, even if the whole cluster is not sorted,
there is no problem, except for createtopic, so when I solve this bug, I just
sort it in createtopic, so the change is the smallest,
was (Author: shizhenzhen):
When I tried to repair it, I found that even if I repaired the problem that the
tomap in kafkazkclient#getallbrokerandepochsincluster is not sorted, it still
can not solve this problem, because the update is also made when the updatetata
request is made, which will result in overwriting the previously sorted ones,
so updatetata also needs to be processed, However, even if the whole cluster is
not sorted, there is no problem, except for createtopic, so when I solve this
bug, I just sort it in createtopic, so the change is the smallest,
> Partition expansion may cause uneven distribution
> -
>
> Key: KAFKA-13226
> URL: https://issues.apache.org/jira/browse/KAFKA-13226
> Project: Kafka
> Issue Type: Bug
> Components: controller
>Affects Versions: 2.5.0, 2.8.0, 2.7.1, 2.6.2, 2.8.1, 3.0.0
> Environment: mac
> kafka-2.5.0
>Reporter: shizhenzhen
>Priority: Major
>
>
> {color:#ff}*Partition expansion may cause uneven distribution*{color}
>
> 1. Create a Topic , 3-partition 1-replical
> !https://img-blog.csdnimg.cn/561112064b114acfb03882aa09100e0e.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_55,color_FF,t_70,g_se,x_16!
>
> 2. partition expansion to 5 - partiton
> !https://img-blog.csdnimg.cn/f7c3c33b6662457080d9bb5bb190c0c2.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_49,color_FF,t_70,g_se,x_16!
>
> 3. Does this meet expectations ？
>
> !https://img-blog.csdnimg.cn/20cc1007c4214c4ebfcb1b2c2eeb98e4.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_18,color_FF,t_70,g_se,x_16!
>
> {color:#ff}*so this is a bug ?*{color}
>
> The problem may arise here ;
> When we create a new topic . get the broker list is a Object Map ;
> *This is disordered*
> you can read the code , first it have sortBy brokerId, but finally it convert
> to a *disorde Map;*
>
>
> !https://img-blog.csdnimg.cn/131b9bf0c19e4753a73512af4c9c5854.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_Q1NETiBA55-z6Ie76Ie755qE5p2C6LSn6ZO6,size_66,color_FF,t_70,g_se,x_16!
>
>
>
> The important thing is that it has been sorted when expanding the partition
> and parition-reassignment ;
> {color:#ff}*So why not sort when creating topics?*{color}
>
> If the topic is sorted when create a new topic , this problem will not occur
> ;
>
> so it maybe is a tiny bug ?
>
>
> if you can read Chinese ,
> You can look at this article. I describe it in detail
>
> We look forward to receiving your reply
>
> 如果你能看懂中文,可以看看这篇文章 我描述的很详细!
> [This may be a Kafka
> bug？|[https://shirenchuang.blog.csdn.net/article/details/119912418]]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
