Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
On Thu, Sep 26, 2019 at 2:16 PM Jun Rao wrote: > [...] > 100. It would be useful to think through how to support SASL scram in the > new world. If the only broker listener is SASL scram, we need to bootstrap > the scram credential in each broker before we can start the broker. > Currently, this can be done by adding the scram credential in ZK first and > then start the brokers. It's not clear how this works when the metadata > service is part of the brokers. > [...] > Without wishing to derail the discussion here, I wrote KIP-506 a while ago which raises some other issues with setting SCRAM credentials via the Admin interface, though I admit I overlooked the bootstrapping problem. This makes me wonder whether trying to set SCRAM credentials via the Admin interface is the wrong approach. Perhaps a tool which could be run within the controller quorum (similar to the existing script) is the only way to provide equivalent security. Cheers, Tom
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
Hi, Colin, Thanks for the KIP. Sorry for the late reply. A couple of comments below. 100. It would be useful to think through how to support SASL scram in the new world. If the only broker listener is SASL scram, we need to bootstrap the scram credential in each broker before we can start the broker. Currently, this can be done by adding the scram credential in ZK first and then start the brokers. It's not clear how this works when the metadata service is part of the brokers. 101. It would be useful to think through the broker failure detection logic. From the diagram, it seems that in the new approach, the controller can fence off a broker but a broker can also fence itself off if it can't contact the controller. This is a bit different from the current approach that ZK server solely determines which broker's session expires. There are tradeoffs between the two approaches. One downslide of the new approach is that it's not clear if the decision from the controller is always consistent with that from the broker and whose decision is the source of truth. 102. For the upgrade process, could we make it clear whether downgrading is supported or not? Jun On Fri, Sep 13, 2019 at 6:12 PM Jason Gustafson wrote: > Late here, but I am a big +1. Great to see this finally happening. > > -Jason > > On Fri, Sep 13, 2019 at 11:43 AM Colin McCabe wrote: > > > Hi all, > > > > With 3 binding +1 votes from Gwen Shapira, Ismael Juma, and Bill Bejeck > > and 5 non-binding +1 votes from Tom Bentley, Ron Dagostino, David Jacot, > > Dhruvil Shah, Mickael Maison, the vote passes. > > > > Thanks, everyone! > > > > best, > > Colin > > > > > > On Fri, Sep 13, 2019, at 01:14, Mickael Maison wrote: > > > +1 non-binding > > > Thanks Colin > > > > > > On Fri, Sep 13, 2019 at 5:21 AM Dhruvil Shah > > wrote: > > > > > > > > This is exciting! +1 (non-binding) > > > > > > > > - Dhruvil > > > > > > > > On Thu, Sep 12, 2019 at 12:29 PM Bill Bejeck > > wrote: > > > > > > > > > Thanks for the KIP! > > > > > > > > > > +1 (binding) > > > > > > > > > > -Bill > > > > > > > > > > On Thu, Sep 12, 2019 at 3:26 PM Ismael Juma > > wrote: > > > > > > > > > > > Thanks for the KIP, +1 (binding). > > > > > > > > > > > > Ismael > > > > > > > > > > > > On Mon, Sep 9, 2019, 8:28 AM Colin McCabe > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I'd like to start the vote for KIP-500: Replace ZooKeeper with > a > > > > > > > Self-Managed Metadata Quorum. > > > > > > > > > > > > > > The DISCUSS thread from the mailing list is here: > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > > > > > > > > > > > > > The KIP is here: > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > > > > > > > > > > > > > regards, > > > > > > > Colin > > > > > > > > > > > > > > > > > > > > > > > >
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
Late here, but I am a big +1. Great to see this finally happening. -Jason On Fri, Sep 13, 2019 at 11:43 AM Colin McCabe wrote: > Hi all, > > With 3 binding +1 votes from Gwen Shapira, Ismael Juma, and Bill Bejeck > and 5 non-binding +1 votes from Tom Bentley, Ron Dagostino, David Jacot, > Dhruvil Shah, Mickael Maison, the vote passes. > > Thanks, everyone! > > best, > Colin > > > On Fri, Sep 13, 2019, at 01:14, Mickael Maison wrote: > > +1 non-binding > > Thanks Colin > > > > On Fri, Sep 13, 2019 at 5:21 AM Dhruvil Shah > wrote: > > > > > > This is exciting! +1 (non-binding) > > > > > > - Dhruvil > > > > > > On Thu, Sep 12, 2019 at 12:29 PM Bill Bejeck > wrote: > > > > > > > Thanks for the KIP! > > > > > > > > +1 (binding) > > > > > > > > -Bill > > > > > > > > On Thu, Sep 12, 2019 at 3:26 PM Ismael Juma > wrote: > > > > > > > > > Thanks for the KIP, +1 (binding). > > > > > > > > > > Ismael > > > > > > > > > > On Mon, Sep 9, 2019, 8:28 AM Colin McCabe > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a > > > > > > Self-Managed Metadata Quorum. > > > > > > > > > > > > The DISCUSS thread from the mailing list is here: > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > > > > > > > > > > > The KIP is here: > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > > > > > > > > > > > regards, > > > > > > Colin > > > > > > > > > > > > > > > > > >
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
Hi all, With 3 binding +1 votes from Gwen Shapira, Ismael Juma, and Bill Bejeck and 5 non-binding +1 votes from Tom Bentley, Ron Dagostino, David Jacot, Dhruvil Shah, Mickael Maison, the vote passes. Thanks, everyone! best, Colin On Fri, Sep 13, 2019, at 01:14, Mickael Maison wrote: > +1 non-binding > Thanks Colin > > On Fri, Sep 13, 2019 at 5:21 AM Dhruvil Shah wrote: > > > > This is exciting! +1 (non-binding) > > > > - Dhruvil > > > > On Thu, Sep 12, 2019 at 12:29 PM Bill Bejeck wrote: > > > > > Thanks for the KIP! > > > > > > +1 (binding) > > > > > > -Bill > > > > > > On Thu, Sep 12, 2019 at 3:26 PM Ismael Juma wrote: > > > > > > > Thanks for the KIP, +1 (binding). > > > > > > > > Ismael > > > > > > > > On Mon, Sep 9, 2019, 8:28 AM Colin McCabe wrote: > > > > > > > > > Hi all, > > > > > > > > > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a > > > > > Self-Managed Metadata Quorum. > > > > > > > > > > The DISCUSS thread from the mailing list is here: > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > > > > > > > > > The KIP is here: > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > > > > > > > > > regards, > > > > > Colin > > > > > > > > > > > > >
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
+1 non-binding Thanks Colin On Fri, Sep 13, 2019 at 5:21 AM Dhruvil Shah wrote: > > This is exciting! +1 (non-binding) > > - Dhruvil > > On Thu, Sep 12, 2019 at 12:29 PM Bill Bejeck wrote: > > > Thanks for the KIP! > > > > +1 (binding) > > > > -Bill > > > > On Thu, Sep 12, 2019 at 3:26 PM Ismael Juma wrote: > > > > > Thanks for the KIP, +1 (binding). > > > > > > Ismael > > > > > > On Mon, Sep 9, 2019, 8:28 AM Colin McCabe wrote: > > > > > > > Hi all, > > > > > > > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a > > > > Self-Managed Metadata Quorum. > > > > > > > > The DISCUSS thread from the mailing list is here: > > > > > > > > > > > > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > > > > > > > The KIP is here: > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > > > > > > > regards, > > > > Colin > > > > > > > > >
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
This is exciting! +1 (non-binding) - Dhruvil On Thu, Sep 12, 2019 at 12:29 PM Bill Bejeck wrote: > Thanks for the KIP! > > +1 (binding) > > -Bill > > On Thu, Sep 12, 2019 at 3:26 PM Ismael Juma wrote: > > > Thanks for the KIP, +1 (binding). > > > > Ismael > > > > On Mon, Sep 9, 2019, 8:28 AM Colin McCabe wrote: > > > > > Hi all, > > > > > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a > > > Self-Managed Metadata Quorum. > > > > > > The DISCUSS thread from the mailing list is here: > > > > > > > > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > > > > > The KIP is here: > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > > > > > regards, > > > Colin > > > > > >
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
Thanks for the KIP! +1 (binding) -Bill On Thu, Sep 12, 2019 at 3:26 PM Ismael Juma wrote: > Thanks for the KIP, +1 (binding). > > Ismael > > On Mon, Sep 9, 2019, 8:28 AM Colin McCabe wrote: > > > Hi all, > > > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a > > Self-Managed Metadata Quorum. > > > > The DISCUSS thread from the mailing list is here: > > > > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > > > The KIP is here: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > > > regards, > > Colin > > >
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
Thanks for the KIP, +1 (binding). Ismael On Mon, Sep 9, 2019, 8:28 AM Colin McCabe wrote: > Hi all, > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a > Self-Managed Metadata Quorum. > > The DISCUSS thread from the mailing list is here: > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > The KIP is here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > regards, > Colin >
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
+1 (non-binding) Le jeu. 12 sept. 2019 à 20:35, Gwen Shapira a écrit : > +1 (binding) > > On Mon, Sep 9, 2019 at 8:28 AM Colin McCabe wrote: > > > > Hi all, > > > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a > Self-Managed Metadata Quorum. > > > > The DISCUSS thread from the mailing list is here: > > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > > > The KIP is here: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > > > regards, > > Colin >
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
+1 (binding) On Mon, Sep 9, 2019 at 8:28 AM Colin McCabe wrote: > > Hi all, > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a Self-Managed > Metadata Quorum. > > The DISCUSS thread from the mailing list is here: > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > The KIP is here: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > regards, > Colin
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
+1 (non-binding) Ron On Thu, Sep 12, 2019 at 5:04 AM Tom Bentley wrote: > +1 (non-binding) > > On Mon, Sep 9, 2019 at 4:28 PM Colin McCabe wrote: > > > Hi all, > > > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a > > Self-Managed Metadata Quorum. > > > > The DISCUSS thread from the mailing list is here: > > > > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > > > The KIP is here: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > > > regards, > > Colin > > >
Re: [VOTE] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
+1 (non-binding) On Mon, Sep 9, 2019 at 4:28 PM Colin McCabe wrote: > Hi all, > > I'd like to start the vote for KIP-500: Replace ZooKeeper with a > Self-Managed Metadata Quorum. > > The DISCUSS thread from the mailing list is here: > > https://lists.apache.org/thread.html/cce5313ebe72bde34bf0da3af5a1723db3ee871667b1fd8edf2ee7ab@%3Cdev.kafka.apache.org%3E > > The KIP is here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > regards, > Colin >
