[DISCUSS] KIP-39 Pinning controller to a broker

2015-10-20 Thread Abhishek Nigam
Hi, Can we please discuss this KIP. The background for this is that it allows us to pin controller to a broker. This is useful in a couple of scenarios: a) If we want to do a rolling bounce we can reduce the number of controller moves down to 1. b) Again pick a designated broker and reduce the

Re: [DISCUSS] KIP-39 Pinning controller to a broker

2015-10-20 Thread Neha Narkhede
Agree with Jay on staying away from pinning roles to brokers. This is actually harder to operate and monitor. Regarding the problems you mentioned- 1. Reducing the controller moves during rolling bounce is useful but really something that should be handled by the tooling. The root cause is that

Re: [DISCUSS] KIP-39 Pinning controller to a broker

2015-10-20 Thread Jay Kreps
This seems like a step backwards--we really don't want people to manually manage the location of the controller and try to manually balance partitions off that broker. I think it might make sense to consider directly fixing the things you actual want to fix: 1. Two many controller moves--we could

Re: [DISCUSS] KIP-39 Pinning controller to a broker

2015-10-20 Thread Abhishek Nigam
Hi Jay/Neha, I just subscribed to the mailing list so I read your response but did not receive your email so adding the context into this email thread. " Agree with Jay on staying away from pinning roles to brokers. This is actually harder to operate and monitor. Regarding the problems you

Re: [DISCUSS] KIP-39 Pinning controller to a broker

2015-10-20 Thread Neha Narkhede
> > I will update the KIP on how we can optimize the placement of controller > (pinning it to a preferred broker id (potentially config enabled) ) if that > sounds reasonable. The point I (and I think Jay too) was making is that pinning a controller to a broker through config is what we should

Re: [DISCUSS] KIP-39 Pinning controller to a broker

2015-10-20 Thread Aditya Auradkar
Hi Abhishek - Perhaps it would help if you explained the motivation behind your proposal. I know there was a bunch of discussion on KAFKA-1778, can you summarize? Currently, I'd agree with Neha and Jay that there isn't really a strong reason to pin the controller to a given broker or restricted