[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030397#comment-16030397
 ] 

ASF GitHub Bot commented on KAFKA-3761:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/2935


> Controller has RunningAsBroker instead of RunningAsController state
> ---
>
> Key: KAFKA-3761
> URL: https://issues.apache.org/jira/browse/KAFKA-3761
> Project: Kafka
>  Issue Type: Bug
>Reporter: Ismael Juma
>Assignee: Roger Hoover
> Fix For: 0.10.1.0
>
>
> In `KafkaServer.start`, we start `KafkaController`:
> {code}
> /* start kafka controller */
> kafkaController = new KafkaController(config, zkUtils, brokerState, 
> kafkaMetricsTime, metrics, threadNamePrefix)
> kafkaController.startup()
> {code}
> Which sets the state to `RunningAsController` in 
> `KafkaController.onControllerFailover`:
> `brokerState.newState(RunningAsController)`
> And this later gets set to `RunningAsBroker`.
> This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we 
> should start the controller after we register the broker in ZK, but this 
> seems tricky as we need to controller in `KafkaApis`.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988745#comment-15988745
 ] 

ASF GitHub Bot commented on KAFKA-3761:
---

GitHub user ijuma opened a pull request:

https://github.com/apache/kafka/pull/2935

MINOR: onControllerResignation should be invoked if triggerControllerMove 
is called

This fixes a transient test failure due to a NPE in 
ControllerFailoverTest.testMetadataUpdate:

```text
Caused by: java.lang.NullPointerException
at 
kafka.controller.ControllerBrokerRequestBatch.addUpdateMetadataRequestForBrokers(ControllerChannelManager.scala:338)
at 
kafka.controller.KafkaController.sendUpdateMetadataRequest(KafkaController.scala:975)
at 
kafka.controller.ControllerFailoverTest.testMetadataUpdate(ControllerFailoverTest.scala:141)
```

The underlying issue is that setting `activeControllerId.set(-1)` in 
`triggerControllerMove`
causes `Reelect` not to invoke `onControllerResignation`. Among other 
things, this
causes an IllegalStateException to be thrown when `KafkaScheduler.startup` 
is invoked
for the second time without the corresponding `shutdown`.

I also updated the test so that we can trigger this issue deterministically 
instead of
transiently.

Finally, I included a few clean-ups:
1. No longer update the broker state in `onControllerFailover`. This is no 
longer needed
since we removed the `RunningAsController` state (KAFKA-3761).
2. Trivial clean-ups in KafkaController
3. Removed unused parameter in `ZkUtils.getPartitionLeaderAndIsrForTopics`

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ijuma/kafka 
on-controller-resignation-if-trigger-controller-move

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/2935.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2935


commit c01d29b3a95e7ddffb91550397a5505d9711d5c8
Author: Ismael Juma 
Date:   2017-04-28T12:30:03Z

MINOR: onControllerResignation should be invoked if triggerControllerMove 
is called

commit f28de697f7d109893537652e8b8216c4d06677a7
Author: Ismael Juma 
Date:   2017-04-28T12:30:47Z

Remove remnant broker state update in `onControllerFailover`

commit 898b88b59cffbfdb7df864d0b070ed7a4960601e
Author: Ismael Juma 
Date:   2017-04-28T12:31:15Z

A few trivial clean-ups in KafkaController

commit 241b9890ab47b4670e61f3f9d3b51c6aa92a8a94
Author: Ismael Juma 
Date:   2017-04-28T12:31:38Z

Remove unused parameter in `getPartitionLeaderAndIsrForTopics`




> Controller has RunningAsBroker instead of RunningAsController state
> ---
>
> Key: KAFKA-3761
> URL: https://issues.apache.org/jira/browse/KAFKA-3761
> Project: Kafka
>  Issue Type: Bug
>Reporter: Ismael Juma
>Assignee: Roger Hoover
> Fix For: 0.10.1.0
>
>
> In `KafkaServer.start`, we start `KafkaController`:
> {code}
> /* start kafka controller */
> kafkaController = new KafkaController(config, zkUtils, brokerState, 
> kafkaMetricsTime, metrics, threadNamePrefix)
> kafkaController.startup()
> {code}
> Which sets the state to `RunningAsController` in 
> `KafkaController.onControllerFailover`:
> `brokerState.newState(RunningAsController)`
> And this later gets set to `RunningAsBroker`.
> This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we 
> should start the controller after we register the broker in ZK, but this 
> seems tricky as we need to controller in `KafkaApis`.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353508#comment-15353508
 ] 

ASF GitHub Bot commented on KAFKA-3761:
---

Github user theduderog closed the pull request at:

https://github.com/apache/kafka/pull/1437


> Controller has RunningAsBroker instead of RunningAsController state
> ---
>
> Key: KAFKA-3761
> URL: https://issues.apache.org/jira/browse/KAFKA-3761
> Project: Kafka
>  Issue Type: Bug
>Reporter: Ismael Juma
>Assignee: Roger Hoover
>
> In `KafkaServer.start`, we start `KafkaController`:
> {code}
> /* start kafka controller */
> kafkaController = new KafkaController(config, zkUtils, brokerState, 
> kafkaMetricsTime, metrics, threadNamePrefix)
> kafkaController.startup()
> {code}
> Which sets the state to `RunningAsController` in 
> `KafkaController.onControllerFailover`:
> `brokerState.newState(RunningAsController)`
> And this later gets set to `RunningAsBroker`.
> This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we 
> should start the controller after we register the broker in ZK, but this 
> seems tricky as we need to controller in `KafkaApis`.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353509#comment-15353509
 ] 

ASF GitHub Bot commented on KAFKA-3761:
---

GitHub user theduderog reopened a pull request:

https://github.com/apache/kafka/pull/1437

KAFKA-3761: Remove BrokerState "RunningAsController"

The reasons to remove it are:
1. It's currently broken.  The purpose of the 
[JIRA](https://issues.apache.org/jira/browse/KAFKA-3761) was to report that the 
RunningAsController state gets overwritten back to "RunningAsBroker".
2. It's not a useful state.
  a. If clients want to use this metric to know whether a broker is ready 
to receive requests or not, they do not care whether or not the broker is the 
controller
  b. there is already a separate boolean property, KafkaController.isActive 
which contains this information.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/theduderog/kafka KAFKA-3761-broker-state

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1437.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1437


commit c0201c5ab8125ea885ff52b234d235d288cd86fb
Author: Roger Hoover 
Date:   2016-06-23T21:52:32Z

Removed RunningAsController state

commit cefead1bed0f0699908bc2819535be4a8f046fd4
Author: Roger Hoover 
Date:   2016-06-23T22:15:30Z

Better formatting for ASCII art

commit 974ba7d4a27dd5ac6b43bc888ece5170737614c5
Author: Roger Hoover 
Date:   2016-06-24T18:26:40Z

Added upgrade note

commit 887fbc0aca264178bb4428b1fdb98dcbe8f2ffdd
Author: Roger Hoover 
Date:   2016-06-24T20:18:17Z

Updated upgrade note based on @ijuma's feedback

commit 5db6479970472fdc8366c17b1cad66971e542a48
Author: Roger Hoover 
Date:   2016-06-24T21:38:43Z

Moved 0.10.1.0 section to the top




> Controller has RunningAsBroker instead of RunningAsController state
> ---
>
> Key: KAFKA-3761
> URL: https://issues.apache.org/jira/browse/KAFKA-3761
> Project: Kafka
>  Issue Type: Bug
>Reporter: Ismael Juma
>Assignee: Roger Hoover
>
> In `KafkaServer.start`, we start `KafkaController`:
> {code}
> /* start kafka controller */
> kafkaController = new KafkaController(config, zkUtils, brokerState, 
> kafkaMetricsTime, metrics, threadNamePrefix)
> kafkaController.startup()
> {code}
> Which sets the state to `RunningAsController` in 
> `KafkaController.onControllerFailover`:
> `brokerState.newState(RunningAsController)`
> And this later gets set to `RunningAsBroker`.
> This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we 
> should start the controller after we register the broker in ZK, but this 
> seems tricky as we need to controller in `KafkaApis`.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303571#comment-15303571
 ] 

ASF GitHub Bot commented on KAFKA-3761:
---

GitHub user theduderog opened a pull request:

https://github.com/apache/kafka/pull/1437

KAFKA-3761: Controller has RunningAsBroker instead of RunningAsController 
state

@junrao @ijuma 

This works for me in local testing.  Do you see any issues?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/theduderog/kafka KAFKA-3761-broker-state

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1437.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1437


commit 6278fdec8c75a6ccc610a602c8cf27c2c1706bfb
Author: Roger Hoover 
Date:   2016-05-27T05:54:50Z

Setting RunningAsController after RunningAsBroker




> Controller has RunningAsBroker instead of RunningAsController state
> ---
>
> Key: KAFKA-3761
> URL: https://issues.apache.org/jira/browse/KAFKA-3761
> Project: Kafka
>  Issue Type: Bug
>Reporter: Ismael Juma
>Assignee: Roger Hoover
>
> In `KafkaServer.start`, we start `KafkaController`:
> {code}
> /* start kafka controller */
> kafkaController = new KafkaController(config, zkUtils, brokerState, 
> kafkaMetricsTime, metrics, threadNamePrefix)
> kafkaController.startup()
> {code}
> Which sets the state to `RunningAsController` in 
> `KafkaController.onControllerFailover`:
> `brokerState.newState(RunningAsController)`
> And this later gets set to `RunningAsBroker`.
> This doesn't match the diagram in `BrokerStates`. [~junrao] suggested that we 
> should start the controller after we register the broker in ZK, but this 
> seems tricky as we need to controller in `KafkaApis`.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)