[ https://issues.apache.org/jira/browse/KAFKA-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dong Lin updated KAFKA-5879: ---------------------------- Description: Currently controller can be very inefficient in handling IsrChangeNotification event because it may need to access znode O(n^2) times to handle O (n) IsrChangeNotification znodes. For example, say there are 100 IsrChangeNotification nodes added to the zookeeper. This will generate 100 IsrChangeNotification events with children [1], [1, 2], [1, 2, 3], ... [1, 2, .. 100]. Let's say the controller now needs to handle the IsrChangeNotification event with children [1, 2, ... 100]. Controller will read zookeeper 100 times, delete these 100 znodes, which further generates 100 IsrChangeNotification events with children count from 0 to 99. The main cause of the problem is that, controller will attempt to access zookeeper n times, where n is the count of the children at the time the IsrChangeNotification event is generated, even though there is no IsrChangeNotification znodes in the zookeeper. was: Currently controller can be very inefficient in handling IsrChangeNotification event because it may need to access znode O(n^2) times to handle O(n) IsrChangeNotification znodes. For example, say there are 100 IsrChangeNotification nodes added to the zookeeper. This will generate 100 IsrChangeNotification events with children [1], [1, 2], [1, 2, 3], ... [1, 2, .. 100]. Let's say the controller now needs to handle the IsrChangeNotification event with children [1, 2, ... 100]. Controller will read zookeeper 100 times, delete these 100 znodes, which further generates 100 IsrChangeNotification events with children count from 0 to 99. The main cause of the problem is that, controller will attempt to access zookeeper n times, where n is the count of the children at the time the IsrChangeNotification event is generated, even though there is no IsrChangeNotification znodes in the zookeeper. > Controller should read the latest IsrChangeNotification znodes when handling > IsrChangeNotification event > -------------------------------------------------------------------------------------------------------- > > Key: KAFKA-5879 > URL: https://issues.apache.org/jira/browse/KAFKA-5879 > Project: Kafka > Issue Type: Bug > Reporter: Dong Lin > Assignee: Dong Lin > Priority: Critical > > Currently controller can be very inefficient in handling > IsrChangeNotification event because it may need to access znode O(n^2) times > to handle O (n) IsrChangeNotification znodes. > For example, say there are 100 IsrChangeNotification nodes added to the > zookeeper. This will generate 100 IsrChangeNotification events with children > [1], [1, 2], [1, 2, 3], ... [1, 2, .. 100]. Let's say the controller now > needs to handle the IsrChangeNotification event with children [1, 2, ... > 100]. Controller will read zookeeper 100 times, delete these 100 znodes, > which further generates 100 IsrChangeNotification events with children count > from 0 to 99. > The main cause of the problem is that, controller will attempt to access > zookeeper n times, where n is the count of the children at the time the > IsrChangeNotification event is generated, even though there is no > IsrChangeNotification znodes in the zookeeper. -- This message was sent by Atlassian JIRA (v6.4.14#64029)