Thanks Colin for the summary! And Guozhang, regarding the future use cases, consider a scenario where there are temporary connectivity issue between controller to a fellow broker A, the controller could then leverage another healthy broker B to do a forwarding request to A in order to maintain a communication.
Even for KIP-590 scope, the forwarding mechanism shall not be transit as we do need to support older version of admin clients for a couple of years IIUC. Boyang On Wed, Apr 22, 2020 at 1:29 PM Colin McCabe <cmcc...@apache.org> wrote: > Hi all, > > I guess the way I see this working is that the request gets sent from the > client, to the initial broker, and then forwarded to the final broker. > > The initial broker should do authentication (who are you?) and come up > with a principal name. Then it creates an envelope request, which will > contain that principal name, and sends it along with the unmodified > original request to the final broker. (I agree with Tom and Rajini that we > can't forward the information needed to do authentication on the final > broker, but I also think we don't need to, since we can do it on the > initial broker.) > > The final broker knows it can trust the principal name in the envelope > (since EnvelopeRequest requires CLUSTERACTION on CLUSTER). So it can use > that principal name for authorization (given who you are, what can you > do?) The forwarded principal name will also be used for logging. > > One question is why we need an EnvelopeRequest. Well, if we don't have an > EnvelopeRequest, we need somewhere else to put the forwarded principal > name. I don't think overriding an existing field (like clientId) is a good > option for this. It's messy, and loses information. It also raises the > question of how the final broker knows that the clientId in the received > message is not "really" a clientId, but is a principal name. Without an > envelope, there's no way to clearly mark a request as forwarded, so there's > no reason for the final broker to treat this differently than a regular > clientId (or whatever). > > We talked about using optional fields to contain the forwarded principal > name, but this is also messy and potentially dangerous. Older brokers will > simply ignore the optional fields, which could result in them executing > operations as the wrong principal. Of course, this would require a > misconfiguration in order to happen, but it still seems better to set up > the system so that this misconfiguration is detected, rather than silently > ignored. > > It's true that the need for forwarding is "temporary" in some sense, since > we only need it for older clients. However, we will want to support these > older clients for many years to come. > > I agree that the usefulness of EnvelopeRequest is limited by it being a > superuser-only request at the moment. Perhaps there are some changes to > how custom principals work that would allow us to get around this in the > future. We should think about that so that we have this functionality in > the future if it's needed. > > best, > Colin > > > On Wed, Apr 22, 2020, at 11:21, Guozhang Wang wrote: > > Hello Gwen, > > > > The purpose here is for maintaining compatibility old clients, who do not > > have functionality to do re-routing admin requests themselves. New > clients > > can of course do this themselves by detecting who's the controller. > > > > > > Hello Colin / Boyang, > > > > Regarding the usage of the envelope, I'm curious what are the potential > > future use cases that would require request forwarding and hence envelope > > would be useful? Originally I thought that the forwarding mechanism is > only > > temporary as we need it for the bridge release, and moving forward we > will > > get rid of this to simplify the code base. If we do have valid use cases > in > > the future which makes us believe that request forwarding would actually > be > > a permanent feature retained on the broker side, I'm on board with adding > > the envelope request protocol. > > > > > > > > Guozhang > > > > > > > > > > On Wed, Apr 22, 2020 at 8:22 AM Gwen Shapira <g...@confluent.io> wrote: > > > > > Hey Boyang, > > > > > > Sorry if this was already discussed, but I didn't see this as rejected > > > alternative: > > > > > > Until now, we always did client side routing - the client itself found > the > > > controller via metadata and directed requests accordingly. Brokers that > > > were not the controller, rejected those requests. > > > > > > Why did we decide to move to broker side routing? Was the client-side > > > option discussed and rejected somewhere and I missed it? > > > > > > Gwen > > > > > > On Fri, Apr 3, 2020, 4:45 PM Boyang Chen <reluctanthero...@gmail.com> > > > wrote: > > > > > > > Hey all, > > > > > > > > I would like to start off the discussion for KIP-590, a follow-up > > > > initiative after KIP-500: > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-590%3A+Redirect+Zookeeper+Mutation+Protocols+to+The+Controller > > > > > > > > This KIP proposes to migrate existing Zookeeper mutation paths, > including > > > > configuration, security and quota changes, to controller-only by > always > > > > routing these alterations to the controller. > > > > > > > > Let me know your thoughts! > > > > > > > > Best, > > > > Boyang > > > > > > > > > > > > > -- > > -- Guozhang > > >
