[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state
[ https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030397#comment-16030397 ] ASF GitHub Bot commented on KAFKA-3761: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2935 > Controller has RunningAsBroker instead of RunningAsController state > --- > > Key: KAFKA-3761 > URL: https://issues.apache.org/jira/browse/KAFKA-3761 > Project: Kafka > Issue Type: Bug >Reporter: Ismael Juma >Assignee: Roger Hoover > Fix For: 0.10.1.0 > > > In `KafkaServer.start`, we start `KafkaController`: > {code} > /* start kafka controller */ > kafkaController = new KafkaController(config, zkUtils, brokerState, > kafkaMetricsTime, metrics, threadNamePrefix) > kafkaController.startup() > {code} > Which sets the state to `RunningAsController` in > `KafkaController.onControllerFailover`: > `brokerState.newState(RunningAsController)` > And this later gets set to `RunningAsBroker`. > This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we > should start the controller after we register the broker in ZK, but this > seems tricky as we need to controller in `KafkaApis`. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state
[ https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988745#comment-15988745 ] ASF GitHub Bot commented on KAFKA-3761: --- GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/2935 MINOR: onControllerResignation should be invoked if triggerControllerMove is called This fixes a transient test failure due to a NPE in ControllerFailoverTest.testMetadataUpdate: ```text Caused by: java.lang.NullPointerException at kafka.controller.ControllerBrokerRequestBatch.addUpdateMetadataRequestForBrokers(ControllerChannelManager.scala:338) at kafka.controller.KafkaController.sendUpdateMetadataRequest(KafkaController.scala:975) at kafka.controller.ControllerFailoverTest.testMetadataUpdate(ControllerFailoverTest.scala:141) ``` The underlying issue is that setting `activeControllerId.set(-1)` in `triggerControllerMove` causes `Reelect` not to invoke `onControllerResignation`. Among other things, this causes an IllegalStateException to be thrown when `KafkaScheduler.startup` is invoked for the second time without the corresponding `shutdown`. I also updated the test so that we can trigger this issue deterministically instead of transiently. Finally, I included a few clean-ups: 1. No longer update the broker state in `onControllerFailover`. This is no longer needed since we removed the `RunningAsController` state (KAFKA-3761). 2. Trivial clean-ups in KafkaController 3. Removed unused parameter in `ZkUtils.getPartitionLeaderAndIsrForTopics` You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka on-controller-resignation-if-trigger-controller-move Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/2935.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2935 commit c01d29b3a95e7ddffb91550397a5505d9711d5c8 Author: Ismael JumaDate: 2017-04-28T12:30:03Z MINOR: onControllerResignation should be invoked if triggerControllerMove is called commit f28de697f7d109893537652e8b8216c4d06677a7 Author: Ismael Juma Date: 2017-04-28T12:30:47Z Remove remnant broker state update in `onControllerFailover` commit 898b88b59cffbfdb7df864d0b070ed7a4960601e Author: Ismael Juma Date: 2017-04-28T12:31:15Z A few trivial clean-ups in KafkaController commit 241b9890ab47b4670e61f3f9d3b51c6aa92a8a94 Author: Ismael Juma Date: 2017-04-28T12:31:38Z Remove unused parameter in `getPartitionLeaderAndIsrForTopics` > Controller has RunningAsBroker instead of RunningAsController state > --- > > Key: KAFKA-3761 > URL: https://issues.apache.org/jira/browse/KAFKA-3761 > Project: Kafka > Issue Type: Bug >Reporter: Ismael Juma >Assignee: Roger Hoover > Fix For: 0.10.1.0 > > > In `KafkaServer.start`, we start `KafkaController`: > {code} > /* start kafka controller */ > kafkaController = new KafkaController(config, zkUtils, brokerState, > kafkaMetricsTime, metrics, threadNamePrefix) > kafkaController.startup() > {code} > Which sets the state to `RunningAsController` in > `KafkaController.onControllerFailover`: > `brokerState.newState(RunningAsController)` > And this later gets set to `RunningAsBroker`. > This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we > should start the controller after we register the broker in ZK, but this > seems tricky as we need to controller in `KafkaApis`. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state
[ https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353508#comment-15353508 ] ASF GitHub Bot commented on KAFKA-3761: --- Github user theduderog closed the pull request at: https://github.com/apache/kafka/pull/1437 > Controller has RunningAsBroker instead of RunningAsController state > --- > > Key: KAFKA-3761 > URL: https://issues.apache.org/jira/browse/KAFKA-3761 > Project: Kafka > Issue Type: Bug >Reporter: Ismael Juma >Assignee: Roger Hoover > > In `KafkaServer.start`, we start `KafkaController`: > {code} > /* start kafka controller */ > kafkaController = new KafkaController(config, zkUtils, brokerState, > kafkaMetricsTime, metrics, threadNamePrefix) > kafkaController.startup() > {code} > Which sets the state to `RunningAsController` in > `KafkaController.onControllerFailover`: > `brokerState.newState(RunningAsController)` > And this later gets set to `RunningAsBroker`. > This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we > should start the controller after we register the broker in ZK, but this > seems tricky as we need to controller in `KafkaApis`. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state
[ https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353509#comment-15353509 ] ASF GitHub Bot commented on KAFKA-3761: --- GitHub user theduderog reopened a pull request: https://github.com/apache/kafka/pull/1437 KAFKA-3761: Remove BrokerState "RunningAsController" The reasons to remove it are: 1. It's currently broken. The purpose of the [JIRA](https://issues.apache.org/jira/browse/KAFKA-3761) was to report that the RunningAsController state gets overwritten back to "RunningAsBroker". 2. It's not a useful state. a. If clients want to use this metric to know whether a broker is ready to receive requests or not, they do not care whether or not the broker is the controller b. there is already a separate boolean property, KafkaController.isActive which contains this information. You can merge this pull request into a Git repository by running: $ git pull https://github.com/theduderog/kafka KAFKA-3761-broker-state Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/1437.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1437 commit c0201c5ab8125ea885ff52b234d235d288cd86fb Author: Roger HooverDate: 2016-06-23T21:52:32Z Removed RunningAsController state commit cefead1bed0f0699908bc2819535be4a8f046fd4 Author: Roger Hoover Date: 2016-06-23T22:15:30Z Better formatting for ASCII art commit 974ba7d4a27dd5ac6b43bc888ece5170737614c5 Author: Roger Hoover Date: 2016-06-24T18:26:40Z Added upgrade note commit 887fbc0aca264178bb4428b1fdb98dcbe8f2ffdd Author: Roger Hoover Date: 2016-06-24T20:18:17Z Updated upgrade note based on @ijuma's feedback commit 5db6479970472fdc8366c17b1cad66971e542a48 Author: Roger Hoover Date: 2016-06-24T21:38:43Z Moved 0.10.1.0 section to the top > Controller has RunningAsBroker instead of RunningAsController state > --- > > Key: KAFKA-3761 > URL: https://issues.apache.org/jira/browse/KAFKA-3761 > Project: Kafka > Issue Type: Bug >Reporter: Ismael Juma >Assignee: Roger Hoover > > In `KafkaServer.start`, we start `KafkaController`: > {code} > /* start kafka controller */ > kafkaController = new KafkaController(config, zkUtils, brokerState, > kafkaMetricsTime, metrics, threadNamePrefix) > kafkaController.startup() > {code} > Which sets the state to `RunningAsController` in > `KafkaController.onControllerFailover`: > `brokerState.newState(RunningAsController)` > And this later gets set to `RunningAsBroker`. > This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we > should start the controller after we register the broker in ZK, but this > seems tricky as we need to controller in `KafkaApis`. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state
[ https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303571#comment-15303571 ] ASF GitHub Bot commented on KAFKA-3761: --- GitHub user theduderog opened a pull request: https://github.com/apache/kafka/pull/1437 KAFKA-3761: Controller has RunningAsBroker instead of RunningAsController state @junrao @ijuma This works for me in local testing. Do you see any issues? You can merge this pull request into a Git repository by running: $ git pull https://github.com/theduderog/kafka KAFKA-3761-broker-state Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/1437.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1437 commit 6278fdec8c75a6ccc610a602c8cf27c2c1706bfb Author: Roger HooverDate: 2016-05-27T05:54:50Z Setting RunningAsController after RunningAsBroker > Controller has RunningAsBroker instead of RunningAsController state > --- > > Key: KAFKA-3761 > URL: https://issues.apache.org/jira/browse/KAFKA-3761 > Project: Kafka > Issue Type: Bug >Reporter: Ismael Juma >Assignee: Roger Hoover > > In `KafkaServer.start`, we start `KafkaController`: > {code} > /* start kafka controller */ > kafkaController = new KafkaController(config, zkUtils, brokerState, > kafkaMetricsTime, metrics, threadNamePrefix) > kafkaController.startup() > {code} > Which sets the state to `RunningAsController` in > `KafkaController.onControllerFailover`: > `brokerState.newState(RunningAsController)` > And this later gets set to `RunningAsBroker`. > This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we > should start the controller after we register the broker in ZK, but this > seems tricky as we need to controller in `KafkaApis`. -- This message was sent by Atlassian JIRA (v6.3.4#6332)