Hey Joe, Can you elaborate what you mean by a stop the world change? In this protocol, we can target notifications to a subset of brokers in the cluster (controller if we need to). Is the AdminChangeNotification a ZK notification or a request type exposed by each broker?
Thanks, Aditya ________________________________________ From: Joe Stein [joe.st...@stealth.ly] Sent: Friday, May 01, 2015 5:25 AM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-21 Configuration Management Hi Aditya, thanks for the write up and focusing on this piece. Agreed we need something that we can do broker changes dynamically without rolling restarts. I think though if every broker is getting changes it with notifications it is going to limit which configs can be dynamic. We could never deliver a "stop the world" configuration change because then that would happen on the entire cluster to every broker on the same time. Can maybe just the controller get the notification? And we provide a layer for brokers to work with the controller to-do the config change operations at is discretion (so it can stop things if needs). controller gets notification, sends AdminChangeNotification to broker [X .. N] then brokers can do their things, even send a response for heartbeating while it takes the few milliseconds it needs or crashes. We need to go through both scenarios. I am worried we put this change in like this and it works for quotas and maybe a few other things but nothing else gets dynamic and we don't get far enough for almost no more rolling restarts. ~ Joe Stein - - - - - - - - - - - - - - - - - http://www.stealth.ly - - - - - - - - - - - - - - - - - On Thu, Apr 30, 2015 at 8:14 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > > 1. I have deep concerns about managing configuration in ZooKeeper. > > First, Producers and Consumers shouldn't depend on ZK at all, this > seems > > to add back a dependency we are trying to get away from. > > The KIP probably needs to be clarified here - I don't think Aditya was > referring to client (producer/consumer) configs. These are global > client-id-specific configs that need to be managed centrally. > (Specifically, quota overrides on a per-client basis). > >
