Hi Colin, Thanks for the KIP. A couple of questions:
1) The KIP says: *However, when the controller is active, its Authorizer state may be slightly ahead of the the broker's Authorizer state. This will happen in the time between when a new ACL is created (or deleted) and the time that this change is persisted durably by a majority of controller quorum peers.* Once the ACL takes effect on the controller, do we guarantee that it will be applied everywhere or can there be scenarios where the controller has applied the change, but others don't due to a subsequent failure in persisting the ACL changes? 2) Have we considered using a different config with limited privileges for bootstrapping instead of the all-powerful *super.users*? Regards, Rajini On Wed, Jan 26, 2022 at 1:50 AM Colin McCabe <cmcc...@apache.org> wrote: > How about this: > > We create a configuration key called early.start.listeners which contains > a list of listener names. If this is not specified, its value defaults to > just the controller listeners. Optionally, other listeners can be added too. > > If super.users contains any user names, early start listeners will start > immediately. In the beginning they only authorize users that are in > super.users. All other listeners receive a new error code, > AUTHORIZER_NOT_READY_ERROR. If super.users does not contain any user names, > then early start listeners will not be treated differently than other > listeners. > > This will allow the controller listeners to get started immediately if the > broker user is in super.users, which will speed up startup. It also will be > useful for breaking chicken/egg cycles like needing to pull the SCRAM > metadata to authorize pulling the SCRAM metadata. > > There are still a few use cases where super.users won't be required, but > it may be useful in many cases to have this early start functionality. > > Leaving aside the preceding discussion, do you agree with starting up all > endpoints (including non-early start ones) once we load a metadata > snapshot? How feasible would it be for us to get a callback from the Raft > layer the first time we caught up to the last stable offset? (we only want > the callback the first time, not any other time). (I think the metadata > shell also would want something like this, at least as an option). > > best, > Colin > > > On Tue, Jan 25, 2022, at 13:34, Jason Gustafson wrote: > > Hi Colin, > > > > Thanks for the writeup. I had one question about bootstrapping. For the > > brokers, I understand that listener startup is delayed until the > Authorizer > > is ready. However, I was not very clear how this would work for the > > controller listeners. We may need them to startup before the metadata log > > is ready so that a quorum can be established (as noted in the KIP). This > > works fine if we assume that the controller principals are among > > `super.users`. For requests forwarded from brokers, on the other hand, we > > need to ensure the ACLs have been loaded properly before we begin > > authorizing. The problem is that we currently use the same listener for > > quorum requests and for forwarded requests. So my question is how does > the > > Authorizer communicate to the controller when it is safe to begin > > authorizing different request types? > > > > There are a couple ways I can see this working. First, we could allow the > > user to configure the listener used for forwarded requests separately. > That > > would work with the existing `Authorizer.start` API. Alternatively, > perhaps > > we could modify `Authorizer.start` to work with something more granular > > than `EndPoint`. This would allow the controller to begin accepting > > requests from the other quorum members before it is ready to authorize > > forwarded requests from clients. Then we would need some way to let > > brokers know when the controller is ready to accept these forwarded > > requests (e.g. through an error code in the `Envelope` response). > > > > What do you think? > > > > Thanks, > > Jason > > > > > > > > > > > > > > On Wed, Jan 12, 2022 at 12:57 PM David Arthur > > <david.art...@confluent.io.invalid> wrote: > > > >> +1 binding, thanks Colin! > >> > >> On Mon, Dec 13, 2021 at 7:47 PM Colin McCabe <cmcc...@apache.org> > wrote: > >> > >> > Hi all, > >> > > >> > I'd like to start the vote on KIP-801: Implement an Authorizer that > >> stores > >> > metadata in __cluster_metadata > >> > > >> > The KIP is here: https://cwiki.apache.org/confluence/x/h5KqCw > >> > > >> > The original DISCUSS thread is here: > >> > > >> > https://lists.apache.org/thread/3d5o7h17ztjztjhblx4fln0wbbs1rmdq > >> > > >> > Please take a look and vote if you can. > >> > > >> > best, > >> > Colin > >> > > >> > >> > >> -- > >> -David > >> >
