Hi Guys, This discussion was dead for a while. Are there still contentious points? If not, why are there no votes?
On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <j...@confluent.io> wrote: > Ashish, > > Yes, I will send out a KIP invite for next week to discuss KIP-48 and other > remaining KIPs. > > Thanks, > > Jun > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <asi...@cloudera.com> wrote: > >> Thanks Harsha! >> >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did not >> actually make a call on when we should have next KIP call. As there are a >> few outstanding KIPs that could not be discussed this week, can we have a >> KIP hangout call next week? >> >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani <ka...@harsha.io> >> wrote: >> >>> Ashish, >>> Yes we are working on it. Lets discuss in the next KIP meeting. >>> I'll join. >>> -Harsha >>> >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <asi...@cloudera.com> >>> wrote: >>> >>> > Hello Harsha, >>> > >>> > Are you still working on this? Wondering if we can discuss this in next >>> KIP >>> > meeting, if you can join. >>> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <ka...@harsha.io> >>> > wrote: >>> > >>> > > Hi Grant, >>> > > We are working on it. Will add the details to KIP about the >>> > > request protocol. >>> > > >>> > > Thanks, >>> > > Harsha >>> > > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <ghe...@cloudera.com> >>> wrote: >>> > > >>> > > > Hi Parth, >>> > > > >>> > > > Are you still working on this? If you need any help please don't >>> > hesitate >>> > > > to ask. >>> > > > >>> > > > Thanks, >>> > > > Grant >>> > > > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <j...@confluent.io> wrote: >>> > > > >>> > > > > Parth, >>> > > > > >>> > > > > Thanks for the reply. >>> > > > > >>> > > > > It makes sense to only allow the renewal by users that >>> authenticated >>> > > > using >>> > > > > *non* delegation token mechanism. Then, should we make the >>> renewal a >>> > > > list? >>> > > > > For example, in the case of rest proxy, it will be useful for >>> every >>> > > > > instance of rest proxy to be able to renew the tokens. >>> > > > > >>> > > > > It would be clearer if we can document the request protocol like >>> > > > > >>> > > > > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >>> > > 4+-+Command+line+and+centralized+administrative+operations#KIP-4- >>> > > Commandlineandcentralizedadministrativeoperations- >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0) >>> > > > > . >>> > > > > >>> > > > > It would also be useful to document the client APIs. >>> > > > > >>> > > > > Thanks, >>> > > > > >>> > > > > Jun >>> > > > > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt < >>> > > > > brahmbhatt.pa...@gmail.com> wrote: >>> > > > > >>> > > > > > Hi, >>> > > > > > >>> > > > > > I am suggesting that we will only allow the renewal by users >>> that >>> > > > > > authenticated using *non* delegation token mechanism. For >>> example, >>> > If >>> > > > > user >>> > > > > > Alice authenticated using kerberos and requested delegation >>> tokens, >>> > > > only >>> > > > > > user Alice authenticated via non delegation token mechanism can >>> > > renew. >>> > > > > > Clients that have access to delegation tokens can not issue >>> > renewal >>> > > > > > request for renewing their own token and this is primarily >>> > important >>> > > to >>> > > > > > reduce the time window for which a compromised token will be >>> valid. >>> > > > > > >>> > > > > > To clarify, Yes any authenticated user can request delegation >>> > tokens >>> > > > but >>> > > > > > even here I would recommend to avoid creating a chain where a >>> > client >>> > > > > > authenticated via delegation token request for more delegation >>> > > tokens. >>> > > > > > Basically anyone can request delegation token, as long as they >>> > > > > authenticate >>> > > > > > via a non delegation token mechanism. >>> > > > > > >>> > > > > > Aren't classes listed here >>> > > > > > < >>> > > > > > >>> > > > > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >>> > > 48+Delegation+token+support+for+Kafka#KIP-48Delegationtokens >>> upportforKaf >>> > > ka-PublicInterfaces >>> > > > > > > >>> > > > > > sufficient? >>> > > > > > >>> > > > > > Thanks >>> > > > > > Parth >>> > > > > > >>> > > > > > >>> > > > > > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <j...@confluent.io> >>> wrote: >>> > > > > > >>> > > > > > > Parth, >>> > > > > > > >>> > > > > > > Thanks for the reply. A couple of comments inline below. >>> > > > > > > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt < >>> > > > > > > brahmbhatt.pa...@gmail.com> wrote: >>> > > > > > > >>> > > > > > > > 1. Who / how are tokens renewed? By original requester >>> only? or >>> > > > using >>> > > > > > > > Kerberos >>> > > > > > > > auth only? >>> > > > > > > > My recommendation is to do this only using Kerberos auth and >>> > only >>> > > > > threw >>> > > > > > > the >>> > > > > > > > renewer specified during the acquisition request. >>> > > > > > > > >>> > > > > > > > >>> > > > > > > Hmm, not sure that I follow this. Are you saying that any >>> client >>> > > > > > > authenticated with the delegation token can renew, i.e. there >>> is >>> > no >>> > > > > > renewer >>> > > > > > > needed? >>> > > > > > > >>> > > > > > > Also, just to be clear, any authenticated client (either >>> through >>> > > SASL >>> > > > > or >>> > > > > > > SSL) can request a delegation token for the authenticated >>> user, >>> > > > right? >>> > > > > > > >>> > > > > > > >>> > > > > > > > 2. Are tokens stored on each broker or in ZK? >>> > > > > > > > My recommendation is still to store in ZK or not store them >>> at >>> > > all. >>> > > > > The >>> > > > > > > > whole controller based distribution is too much overhead >>> with >>> > not >>> > > > > much >>> > > > > > to >>> > > > > > > > achieve. >>> > > > > > > > >>> > > > > > > > 3. How are tokens invalidated / expired? >>> > > > > > > > Either by expiration time out or through an explicit >>> request to >>> > > > > > > invalidate. >>> > > > > > > > >>> > > > > > > > 4. Which encryption algorithm is used? >>> > > > > > > > SCRAM >>> > > > > > > > >>> > > > > > > > 5. What is the impersonation proposal (it wasn't in the KIP >>> but >>> > > was >>> > > > > > > > discussed >>> > > > > > > > in this thread)? >>> > > > > > > > There is no imperonation proposal. I tried and explained how >>> > its >>> > > a >>> > > > > > > > different problem and why its not really necessary to >>> discuss >>> > > that >>> > > > as >>> > > > > > > part >>> > > > > > > > of this KIP. This KIP will not support any impersonation, >>> it >>> > > will >>> > > > > just >>> > > > > > > be >>> > > > > > > > another way to authenticate. >>> > > > > > > > >>> > > > > > > > 6. Do we need new ACLs, if so - for what actions? >>> > > > > > > > We do not need new ACLs. >>> > > > > > > > >>> > > > > > > > >>> > > > > > > Could we document the format of the new request/response and >>> > their >>> > > > > > > associated Resource and Operation for ACL? >>> > > > > > > >>> > > > > > > >>> > > > > > > > 7. How would the delegation token be configured in the >>> client? >>> > > > > > > > Should be through config. I wasn't planning on supporting >>> JAAS >>> > > for >>> > > > > > > tokens. >>> > > > > > > > I don't believe hadoop does this either. >>> > > > > > > > >>> > > > > > > > Thanks >>> > > > > > > > Parth >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <j...@confluent.io> >>> > > wrote: >>> > > > > > > > >>> > > > > > > > > Harsha, >>> > > > > > > > > >>> > > > > > > > > Another question. >>> > > > > > > > > >>> > > > > > > > > 9. How would the delegation token be configured in the >>> > client? >>> > > > The >>> > > > > > > > standard >>> > > > > > > > > way is to do this through JAAS. However, we will need to >>> > think >>> > > > > > through >>> > > > > > > if >>> > > > > > > > > this is convenient in a shared environment. For example, >>> > when a >>> > > > new >>> > > > > > > task >>> > > > > > > > is >>> > > > > > > > > added to a Storm worker node, do we need to dynamically >>> add a >>> > > new >>> > > > > > > section >>> > > > > > > > > in the JAAS file? It may be more convenient if we can >>> pass in >>> > > the >>> > > > > > token >>> > > > > > > > > through the config directly w/o going through JAAS. >>> > > > > > > > > >>> > > > > > > > > Are you or Parth still actively working on this KIP? >>> > > > > > > > > >>> > > > > > > > > Thanks, >>> > > > > > > > > >>> > > > > > > > > Jun >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao < >>> j...@confluent.io> >>> > > > wrote: >>> > > > > > > > > >>> > > > > > > > > > Just to add on that list. >>> > > > > > > > > > >>> > > > > > > > > > 2. It would be good to document the format of the data >>> > stored >>> > > > in >>> > > > > > ZK. >>> > > > > > > > > > 7. Earlier, there was a discussion on whether the tokens >>> > > should >>> > > > > be >>> > > > > > > > > > propagated through ZK like config/acl/quota, or through >>> the >>> > > > > > > controller. >>> > > > > > > > > > Currently, the controller is only designed for >>> propagating >>> > > > topic >>> > > > > > > > > metadata, >>> > > > > > > > > > but not other data. >>> > > > > > > > > > 8. Should we use SCRAM to send the token instead of >>> > > DIGEST-MD5 >>> > > > > > since >>> > > > > > > > it's >>> > > > > > > > > > deprecated? >>> > > > > > > > > > >>> > > > > > > > > > Also, the images in the wiki seem broken. >>> > > > > > > > > > >>> > > > > > > > > > Thanks, >>> > > > > > > > > > >>> > > > > > > > > > Jun >>> > > > > > > > > > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira < >>> > > > > g...@confluent.io> >>> > > > > > > > > wrote: >>> > > > > > > > > > >>> > > > > > > > > >> From what I can see, remaining questions are: >>> > > > > > > > > >> >>> > > > > > > > > >> 1. Who / how are tokens renewed? By original requester >>> > only? >>> > > > or >>> > > > > > > using >>> > > > > > > > > >> Kerberos auth only? >>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK? >>> > > > > > > > > >> 3. How are tokens invalidated / expired? >>> > > > > > > > > >> 4. Which encryption algorithm is used? >>> > > > > > > > > >> 5. What is the impersonation proposal (it wasn't in the >>> > KIP >>> > > > but >>> > > > > > was >>> > > > > > > > > >> discussed in this thread)? >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what actions? >>> > > > > > > > > >> >>> > > > > > > > > >> Gwen >>> > > > > > > > > >> >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha < >>> ka...@harsha.io> >>> > > > wrote: >>> > > > > > > > > >> > Jun & Ismael, >>> > > > > > > > > >> > Unfortunately I couldn't >>> attend >>> > > the >>> > > > > KIP >>> > > > > > > > > meeting >>> > > > > > > > > >> > when delegation tokens >>> > discussed. >>> > > > > > > > Appreciate >>> > > > > > > > > if >>> > > > > > > > > >> > you can update the thread if >>> > you >>> > > > have >>> > > > > > any >>> > > > > > > > > >> > further questions. >>> > > > > > > > > >> > Thanks, >>> > > > > > > > > >> > Harsha >>> > > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote: >>> > > > > > > > > >> >> It seems that the links to images in the KIP are >>> > broken. >>> > > > > > > > > >> >> >>> > > > > > > > > >> >> Liquan >>> > > > > > > > > >> >> >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt < >>> > > > > > > > > >> >> brahmbhatt.pa...@gmail.com> wrote: >>> > > > > > > > > >> >> >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean? >>> > > > > > > > > >> >> > In the current proposal we only allow a user to >>> get >>> > > > > > delegation >>> > > > > > > > > token >>> > > > > > > > > >> for >>> > > > > > > > > >> >> > the identity that it authenticated as using >>> another >>> > > > > > mechanism, >>> > > > > > > > i.e. >>> > > > > > > > > >> A user >>> > > > > > > > > >> >> > that authenticate using a keytab for principal >>> > > > > > > us...@example.com >>> > > > > > > > > >> will get >>> > > > > > > > > >> >> > delegation tokens for that user only. In future I >>> > think >>> > > > we >>> > > > > > will >>> > > > > > > > > have >>> > > > > > > > > >> to >>> > > > > > > > > >> >> > extend support such that we allow some set of >>> users ( >>> > > > > > > > > >> >> > kafka-rest-u...@example.com, >>> > storm-nim...@example.com) >>> > > > to >>> > > > > > > > acquire >>> > > > > > > > > >> >> > delegation tokens on behalf of other users whose >>> > > identity >>> > > > > > they >>> > > > > > > > have >>> > > > > > > > > >> >> > verified independently. Kafka brokers will have >>> ACLs >>> > > to >>> > > > > > > control >>> > > > > > > > > >> which >>> > > > > > > > > >> >> > users are allowed to impersonate other users and >>> get >>> > > > tokens >>> > > > > > on >>> > > > > > > > > >> behalf of >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole different >>> > > problem >>> > > > in >>> > > > > > my >>> > > > > > > > > >> opinion and >>> > > > > > > > > >> >> > I think we can tackle it in separate KIP. >>> > > > > > > > > >> >> > >>> > > > > > > > > >> >> > 111. What's the typical rate of getting and >>> renewing >>> > > > > > delegation >>> > > > > > > > > >> tokens? >>> > > > > > > > > >> >> > Typically this should be very very low, 1 request >>> per >>> > > > > minute >>> > > > > > > is a >>> > > > > > > > > >> >> > relatively high estimate. However it depends on >>> the >>> > > token >>> > > > > > > > > >> expiration. I am >>> > > > > > > > > >> >> > less worried about the extra load it puts on >>> > controller >>> > > > vs >>> > > > > > the >>> > > > > > > > > added >>> > > > > > > > > >> >> > complexity and the value it offers. >>> > > > > > > > > >> >> > >>> > > > > > > > > >> >> > Thanks >>> > > > > > > > > >> >> > Parth >>> > > > > > > > > >> >> > >>> > > > > > > > > >> >> > >>> > > > > > > > > >> >> > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma < >>> > > > > > > ism...@juma.me.uk> >>> > > > > > > > > >> wrote: >>> > > > > > > > > >> >> > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a >>> separate >>> > > KIP >>> > > > > as >>> > > > > > it >>> > > > > > > > > will >>> > > > > > > > > >> >> > > introduce user visible changes. We could also >>> > update >>> > > > > KIP-48 >>> > > > > > > to >>> > > > > > > > > >> have this >>> > > > > > > > > >> >> > > information, but it seems cleaner to do it >>> > > separately. >>> > > > We >>> > > > > > can >>> > > > > > > > > >> discuss >>> > > > > > > > > >> >> > that >>> > > > > > > > > >> >> > > in the KIP call today. >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > Ismael >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram >>> < >>> > > > > > > > > >> >> > > rajinisiva...@googlemail.com> wrote: >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > > Ismael, >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > I have created a JIRA ( >>> > > > > > > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751) >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would >>> that >>> > > need >>> > > > > > > another >>> > > > > > > > > >> KIP? If >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this just >>> be >>> > a >>> > > > JIRA >>> > > > > > > that >>> > > > > > > > > gets >>> > > > > > > > > >> >> > > reviewed >>> > > > > > > > > >> >> > > > when the PR is ready? >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > Thank you, >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > Rajini >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma < >>> > > > > > > > > ism...@juma.me.uk> >>> > > > > > > > > >> >> > wrote: >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good >>> > candidate. >>> > > > > > > > > >> >> > > > > >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned this as a >>> SASL >>> > > > > > mechanism >>> > > > > > > > > that >>> > > > > > > > > >> might >>> > > > > > > > > >> >> > be >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been meaning to >>> > check >>> > > > it >>> > > > > in >>> > > > > > > > more >>> > > > > > > > > >> detail. >>> > > > > > > > > >> >> > > Good >>> > > > > > > > > >> >> > > > > to know that you are willing to contribute >>> an >>> > > > > > > > implementation. >>> > > > > > > > > >> Maybe >>> > > > > > > > > >> >> > we >>> > > > > > > > > >> >> > > > > should file a separate JIRA for this? >>> > > > > > > > > >> >> > > > > >>> > > > > > > > > >> >> > > > > Ismael >>> > > > > > > > > >> >> > > > > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini >>> > Sivaram < >>> > > > > > > > > >> >> > > > > rajinisiva...@googlemail.com> wrote: >>> > > > > > > > > >> >> > > > > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response >>> > Authentication >>> > > > > > > > Mechanism) >>> > > > > > > > > >> is a >>> > > > > > > > > >> >> > > better >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't >>> come >>> > > > with a >>> > > > > > > > > built-in >>> > > > > > > > > >> SCRAM >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be >>> happy >>> > > to >>> > > > > add >>> > > > > > > > > support >>> > > > > > > > > >> in >>> > > > > > > > > >> >> > Kafka >>> > > > > > > > > >> >> > > > > since >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism to support >>> > > anyway. >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677 >>> > describes >>> > > > the >>> > > > > > > > protocol >>> > > > > > > > > >> for >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256. >>> > > > > > > > > >> >> > > > > > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao < >>> > > > > > > > j...@confluent.io >>> > > > > > > > > > >>> > > > > > > > > >> wrote: >>> > > > > > > > > >> >> > > > > > >>> > > > > > > > > >> >> > > > > > > Parth, >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A couple of >>> > more >>> > > > > > > questions. >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs >>> mean? >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of getting >>> and >>> > > > > > renewing >>> > > > > > > > > >> delegation >>> > > > > > > > > >> >> > > > tokens? >>> > > > > > > > > >> >> > > > > > > That may have an impact on whether they >>> > > should >>> > > > be >>> > > > > > > > > directed >>> > > > > > > > > >> to the >>> > > > > > > > > >> >> > > > > > > controller. >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > > > > Jun >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth >>> > > > > brahmbhatt < >>> > > > > > > > > >> >> > > > > > > brahmbhatt.pa...@gmail.com> wrote: >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > > > > > Hi Jun, >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing. >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster action to add >>> > acls >>> > > > on >>> > > > > > who >>> > > > > > > > can >>> > > > > > > > > >> request >>> > > > > > > > > >> >> > > > > > delegation >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use case for >>> that >>> > > yet >>> > > > > but >>> > > > > > > > down >>> > > > > > > > > >> the line >>> > > > > > > > > >> >> > > > when >>> > > > > > > > > >> >> > > > > we >>> > > > > > > > > >> >> > > > > > > > start supporting getDelegationTokenAs >>> it >>> > > will >>> > > > > be >>> > > > > > > > > >> necessary. >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be only >>> > > > > > > used/distributed >>> > > > > > > > > >> over >>> > > > > > > > > >> >> > secure >>> > > > > > > > > >> >> > > > > > > channels. >>> > > > > > > > > >> >> > > > > > > > * Depending on what design we end up >>> > > choosing >>> > > > > > > > > >> Invalidation will >>> > > > > > > > > >> >> > > be >>> > > > > > > > > >> >> > > > > > > > responsibility of every broker or >>> > > controller. >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I documented >>> somewhere >>> > > > that >>> > > > > > > > > >> invalidation >>> > > > > > > > > >> >> > will >>> > > > > > > > > >> >> > > > > > directly >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but that is not >>> the >>> > > > > intent. >>> > > > > > > > > >> Invalidation >>> > > > > > > > > >> >> > > will >>> > > > > > > > > >> >> > > > > > either >>> > > > > > > > > >> >> > > > > > > > be request based or due to >>> expiration. No >>> > > > > direct >>> > > > > > > > > >> zookeeper >>> > > > > > > > > >> >> > > > > interaction >>> > > > > > > > > >> >> > > > > > > from >>> > > > > > > > > >> >> > > > > > > > any client. >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the >>> DelegationToken >>> > > > > without >>> > > > > > > the >>> > > > > > > > > >> hmac in >>> > > > > > > > > >> >> > the >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the >>> confusion. >>> > > The >>> > > > > sole >>> > > > > > > > > >> purpose of >>> > > > > > > > > >> >> > > > > zookeeper >>> > > > > > > > > >> >> > > > > > in >>> > > > > > > > > >> >> > > > > > > > this design is as distribution channel >>> > for >>> > > > > tokens >>> > > > > > > > > >> between all >>> > > > > > > > > >> >> > > > brokers >>> > > > > > > > > >> >> > > > > > > and a >>> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens that >>> were >>> > > > > > generated >>> > > > > > > by >>> > > > > > > > > >> making a >>> > > > > > > > > >> >> > > > > request >>> > > > > > > > > >> >> > > > > > > to a >>> > > > > > > > > >> >> > > > > > > > broker will be accepted (more on this >>> in >>> > > > second >>> > > > > > > > > >> paragraph). The >>> > > > > > > > > >> >> > > > token >>> > > > > > > > > >> >> > > > > > > > consists of few elements (owner, >>> renewer, >>> > > > uuid >>> > > > > , >>> > > > > > > > > >> expiration, >>> > > > > > > > > >> >> > > hmac) >>> > > > > > > > > >> >> > > > , >>> > > > > > > > > >> >> > > > > > one >>> > > > > > > > > >> >> > > > > > > of >>> > > > > > > > > >> >> > > > > > > > which is the finally generated hmac >>> but >>> > > hmac >>> > > > it >>> > > > > > > self >>> > > > > > > > is >>> > > > > > > > > >> >> > derivable >>> > > > > > > > > >> >> > > > if >>> > > > > > > > > >> >> > > > > > you >>> > > > > > > > > >> >> > > > > > > > have all the other elements of the >>> token >>> > + >>> > > > > secret >>> > > > > > > key >>> > > > > > > > > to >>> > > > > > > > > >> >> > generate >>> > > > > > > > > >> >> > > > > hmac. >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not provide SSL >>> > > support >>> > > > we >>> > > > > > do >>> > > > > > > > not >>> > > > > > > > > >> want the >>> > > > > > > > > >> >> > > > > entire >>> > > > > > > > > >> >> > > > > > > > token to be wire transferred to >>> zookeeper >>> > > as >>> > > > > that >>> > > > > > > > will >>> > > > > > > > > >> be an >>> > > > > > > > > >> >> > > > insecure >>> > > > > > > > > >> >> > > > > > > wire >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only store all >>> the >>> > > other >>> > > > > > > > elements >>> > > > > > > > > >> of a >>> > > > > > > > > >> >> > > > > delegation >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these >>> elements >>> > and >>> > > > > > because >>> > > > > > > > > they >>> > > > > > > > > >> also >>> > > > > > > > > >> >> > > have >>> > > > > > > > > >> >> > > > > > access >>> > > > > > > > > >> >> > > > > > > > to secret key they will be able to >>> > generate >>> > > > > hmac >>> > > > > > on >>> > > > > > > > > >> their end. >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > One of the alternative proposed is to >>> > avoid >>> > > > > > > zookeeper >>> > > > > > > > > >> >> > > altogether. A >>> > > > > > > > > >> >> > > > > > > Client >>> > > > > > > > > >> >> > > > > > > > will call broker with required >>> > information >>> > > > > > (owner, >>> > > > > > > > > >> renwer, >>> > > > > > > > > >> >> > > > > expiration) >>> > > > > > > > > >> >> > > > > > > and >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker >>> > won't >>> > > > > store >>> > > > > > > this >>> > > > > > > > > in >>> > > > > > > > > >> >> > > zookeeper. >>> > > > > > > > > >> >> > > > > > From >>> > > > > > > > > >> >> > > > > > > > this point a client can contact any >>> > broker >>> > > > with >>> > > > > > all >>> > > > > > > > the >>> > > > > > > > > >> >> > > delegation >>> > > > > > > > > >> >> > > > > > token >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac, >>> > > uuid) >>> > > > > the >>> > > > > > > > borker >>> > > > > > > > > >> will >>> > > > > > > > > >> >> > > > > regenerate >>> > > > > > > > > >> >> > > > > > > the >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches with >>> hmac >>> > > > > > presented >>> > > > > > > by >>> > > > > > > > > >> client , >>> > > > > > > > > >> >> > > > broker >>> > > > > > > > > >> >> > > > > > > will >>> > > > > > > > > >> >> > > > > > > > allow the request to authenticate. >>> Only >>> > > > > problem >>> > > > > > > with >>> > > > > > > > > >> this >>> > > > > > > > > >> >> > > approach >>> > > > > > > > > >> >> > > > > is >>> > > > > > > > > >> >> > > > > > if >>> > > > > > > > > >> >> > > > > > > > the secret key is compromised any >>> client >>> > > can >>> > > > > now >>> > > > > > > > > generate >>> > > > > > > > > >> >> > random >>> > > > > > > > > >> >> > > > > tokens >>> > > > > > > > > >> >> > > > > > > and >>> > > > > > > > > >> >> > > > > > > > they will still be able to >>> authenticate >>> > as >>> > > > any >>> > > > > > user >>> > > > > > > > > they >>> > > > > > > > > >> like. >>> > > > > > > > > >> >> > > with >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only >>> tokens >>> > > > > acquired >>> > > > > > > via >>> > > > > > > > a >>> > > > > > > > > >> broker >>> > > > > > > > > >> >> > > > (using >>> > > > > > > > > >> >> > > > > > some >>> > > > > > > > > >> >> > > > > > > > auth scheme other than delegation >>> token) >>> > > will >>> > > > > be >>> > > > > > > > > >> accepted. We >>> > > > > > > > > >> >> > > need >>> > > > > > > > > >> >> > > > to >>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes more >>> sense >>> > and >>> > > > we >>> > > > > > can >>> > > > > > > go >>> > > > > > > > > >> over it >>> > > > > > > > > >> >> > in >>> > > > > > > > > >> >> > > > > > > tomorrow's >>> > > > > > > > > >> >> > > > > > > > meeting. >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the invite to >>> me? >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > Thanks >>> > > > > > > > > >> >> > > > > > > > Parth >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun >>> > Rao < >>> > > > > > > > > >> j...@confluent.io> >>> > > > > > > > > >> >> > > > wrote: >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few comments. >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be useful >>> for >>> > > > Kafka >>> > > > > > > > Connect >>> > > > > > > > > >> and >>> > > > > > > > > >> >> > Kafka >>> > > > > > > > > >> >> > > > > rest >>> > > > > > > > > >> >> > > > > > > > proxy >>> > > > > > > > > >> >> > > > > > > > > where a worker agent will need to >>> run a >>> > > > task >>> > > > > on >>> > > > > > > > > behalf >>> > > > > > > > > >> of a >>> > > > > > > > > >> >> > > > client. >>> > > > > > > > > >> >> > > > > > We >>> > > > > > > > > >> >> > > > > > > > will >>> > > > > > > > > >> >> > > > > > > > > likely need to change how those >>> > services >>> > > > use >>> > > > > > > Kafka >>> > > > > > > > > >> clients >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a >>> > shared >>> > > > > client >>> > > > > > > per >>> > > > > > > > > >> worker, >>> > > > > > > > > >> >> > we >>> > > > > > > > > >> >> > > > will >>> > > > > > > > > >> >> > > > > > > need >>> > > > > > > > > >> >> > > > > > > > a >>> > > > > > > > > >> >> > > > > > > > > client per user task since the >>> > > > authentication >>> > > > > > > > happens >>> > > > > > > > > >> at the >>> > > > > > > > > >> >> > > > > > connection >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the >>> renewer >>> > > will >>> > > > be >>> > > > > > the >>> > > > > > > > > >> workers. >>> > > > > > > > > >> >> > So, >>> > > > > > > > > >> >> > > we >>> > > > > > > > > >> >> > > > > > > > probably >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple renewers. For >>> > > Kafka >>> > > > > rest >>> > > > > > > > > proxy, >>> > > > > > > > > >> the >>> > > > > > > > > >> >> > > > renewer >>> > > > > > > > > >> >> > > > > > can >>> > > > > > > > > >> >> > > > > > > > > probably just be the creator of the >>> > > token. >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who can >>> > > request >>> > > > > > > > delegation >>> > > > > > > > > >> tokens? >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to send >>> > > > > delegation >>> > > > > > > > tokens >>> > > > > > > > > >> in an >>> > > > > > > > > >> >> > > > > encrypted >>> > > > > > > > > >> >> > > > > > > > > channel? >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for expiring >>> > > > tokens, >>> > > > > > > every >>> > > > > > > > > >> broker? >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens, would >>> it >>> > be >>> > > > > > better >>> > > > > > > to >>> > > > > > > > > do >>> > > > > > > > > >> it in >>> > > > > > > > > >> >> > a >>> > > > > > > > > >> >> > > > > > request >>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK directly? >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of client in >>> the >>> > > wiki >>> > > > > > > > sometimes >>> > > > > > > > > >> refers >>> > > > > > > > > >> >> > to >>> > > > > > > > > >> >> > > > the >>> > > > > > > > > >> >> > > > > > end >>> > > > > > > > > >> >> > > > > > > > > client and some other times refers >>> to >>> > the >>> > > > > > client >>> > > > > > > > > using >>> > > > > > > > > >> the >>> > > > > > > > > >> >> > > > > delegation >>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful to >>> > distinguish >>> > > > > > between >>> > > > > > > > the >>> > > > > > > > > >> two. >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the sentence >>> > > "Broker >>> > > > > > also >>> > > > > > > > > >> stores the >>> > > > > > > > > >> >> > > > > > > > DelegationToken >>> > > > > > > > > >> >> > > > > > > > > without the hmac in the zookeeper." >>> a >>> > bit >>> > > > > > more? I >>> > > > > > > > > >> thought the >>> > > > > > > > > >> >> > > > > > > delegation >>> > > > > > > > > >> >> > > > > > > > > token is the hmac. >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > Thanks, >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > Jun >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun >>> > Rao >>> > > < >>> > > > > > > > > >> j...@confluent.io> >>> > > > > > > > > >> >> > > > wrote: >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha, >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting >>> invite. >>> > We >>> > > > can >>> > > > > > > > discuss >>> > > > > > > > > >> this in >>> > > > > > > > > >> >> > > the >>> > > > > > > > > >> >> > > > > > > meeting >>> > > > > > > > > >> >> > > > > > > > > > tomorrow. >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > Thanks, >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > Jun >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, >>> > > Harsha < >>> > > > > > > > > >> ka...@harsha.io> >>> > > > > > > > > >> >> > > > wrote: >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> Hi All, >>> > > > > > > > > >> >> > > > > > > > > >> Can we have a KIP >>> meeting >>> > > > > around >>> > > > > > > > this. >>> > > > > > > > > >> The KIP >>> > > > > > > > > >> >> > is >>> > > > > > > > > >> >> > > > up >>> > > > > > > > > >> >> > > > > > for >>> > > > > > > > > >> >> > > > > > > > > >> sometime and if there >>> are >>> > > any >>> > > > > > > > questions >>> > > > > > > > > >> lets >>> > > > > > > > > >> >> > > > quickly >>> > > > > > > > > >> >> > > > > > hash >>> > > > > > > > > >> >> > > > > > > > out >>> > > > > > > > > >> >> > > > > > > > > >> details. >>> > > > > > > > > >> >> > > > > > > > > >> >>> > > > > > > > > >> >> > > > > > > > > >> Thanks, >>> > > > > > > > > >> >> > > > > > > > > >> Harsha >>> > > > > > > > > >> >> > > > > > > > > >> >>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 >>> AM, >>> > > parth >>> > > > > > > > > brahmbhatt >>> > > > > > > > > >> wrote: >>> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo >>> > system >>> > > > uses >>> > > > > > so >>> > > > > > > no >>> > > > > > > > > >> good >>> > > > > > > > > >> >> > reason >>> > > > > > > > > >> >> > > > > > really. >>> > > > > > > > > >> >> > > > > > > > We >>> > > > > > > > > >> >> > > > > > > > > >> > could >>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever is the >>> > newest >>> > > > > > > > recommended >>> > > > > > > > > >> standard >>> > > > > > > > > >> >> > > is. >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > Thanks >>> > > > > > > > > >> >> > > > > > > > > >> > Parth >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 >>> AM, >>> > > > Ismael >>> > > > > > > Juma < >>> > > > > > > > > >> >> > > > > ism...@juma.me.uk >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > > > > > > wrote: >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth, >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only >>> > started >>> > > > > > > reviewing >>> > > > > > > > > >> this and >>> > > > > > > > > >> >> > > may >>> > > > > > > > > >> >> > > > > have >>> > > > > > > > > >> >> > > > > > > > > >> additional >>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The >>> immediate >>> > > > > question >>> > > > > > > that >>> > > > > > > > > >> came to >>> > > > > > > > > >> >> > > mind >>> > > > > > > > > >> >> > > > is >>> > > > > > > > > >> >> > > > > > our >>> > > > > > > > > >> >> > > > > > > > > >> choice of >>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's >>> > > marked >>> > > > > as >>> > > > > > > > > >> OBSOLETE in >>> > > > > > > > > >> >> > the >>> > > > > > > > > >> >> > > > IANA >>> > > > > > > > > >> >> > > > > > > > > Registry >>> > > > > > > > > >> >> > > > > > > > > >> of >>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the >>> original >>> > > RFC >>> > > > > > > (2831) >>> > > > > > > > > has >>> > > > > > > > > >> been >>> > > > > > > > > >> >> > > moved >>> > > > > > > > > >> >> > > > > to >>> > > > > > > > > >> >> > > > > > > > > Historic >>> > > > > > > > > >> >> > > > > > > > > >> > > status: >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/ >>> > > rfc6331 >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >>> > > > > > > >>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl- >>> mechanisms.xhtml >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning behind >>> > that >>> > > > > > choice? >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks, >>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 >>> > PM, >>> > > > Gwen >>> > > > > > > > > Shapira < >>> > > > > > > > > >> >> > > > > > > g...@confluent.io >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> wrote: >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :) >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize >>> that >>> > > even >>> > > > > > though >>> > > > > > > > > >> delegation >>> > > > > > > > > >> >> > > > tokens >>> > > > > > > > > >> >> > > > > > > are a >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop >>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel very >>> > > strongly >>> > > > > > about >>> > > > > > > > not >>> > > > > > > > > >> adding >>> > > > > > > > > >> >> > > > > > dependency >>> > > > > > > > > >> >> > > > > > > > on >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop >>> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing >>> delegation >>> > > > > tokens >>> > > > > > > for >>> > > > > > > > > >> Kafka. The >>> > > > > > > > > >> >> > > KIP >>> > > > > > > > > >> >> > > > > > > doesn't >>> > > > > > > > > >> >> > > > > > > > > >> imply >>> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but if >>> you >>> > > can >>> > > > > > > > clarify... >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.* >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the >>> KIP >>> > so >>> > > > no >>> > > > > > one >>> > > > > > > > will >>> > > > > > > > > >> read >>> > > > > > > > > >> >> > the >>> > > > > > > > > >> >> > > > KIP >>> > > > > > > > > >> >> > > > > > and >>> > > > > > > > > >> >> > > > > > > > > panic >>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the next >>> > > > > release... >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get delegation >>> > token >>> > > at >>> > > > > any >>> > > > > > > > time >>> > > > > > > > > >> after >>> > > > > > > > > >> >> > > > > > > > authenticating? >>> > > > > > > > > >> >> > > > > > > > > >> only >>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately after? >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are >>> > > > authenticated >>> > > > > > you >>> > > > > > > > can >>> > > > > > > > > >> get >>> > > > > > > > > >> >> > > > delegation >>> > > > > > > > > >> >> > > > > > > > tokens. >>> > > > > > > > > >> >> > > > > > > > > >> We >>> > > > > > > > > >> >> > > > > > > > > >> > > need >>> > > > > > > > > >> >> > > > > > > > > >> > > > to >>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client >>> > > > authenticated >>> > > > > > > using >>> > > > > > > > > >> delegation >>> > > > > > > > > >> >> > > > > token, >>> > > > > > > > > >> >> > > > > > > can >>> > > > > > > > > >> >> > > > > > > > > also >>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token again or >>> > not. >>> > > > > Also >>> > > > > > > > there >>> > > > > > > > > >> is the >>> > > > > > > > > >> >> > > > > question >>> > > > > > > > > >> >> > > > > > of >>> > > > > > > > > >> >> > > > > > > > do >>> > > > > > > > > >> >> > > > > > > > > we >>> > > > > > > > > >> >> > > > > > > > > >> > > allow >>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire >>> delegation >>> > > > token >>> > > > > > or >>> > > > > > > we >>> > > > > > > > > >> want >>> > > > > > > > > >> >> > > specific >>> > > > > > > > > >> >> > > > > > ACLs >>> > > > > > > > > >> >> > > > > > > (I >>> > > > > > > > > >> >> > > > > > > > > >> think >>> > > > > > > > > >> >> > > > > > > > > >> > > its >>> > > > > > > > > >> >> > > > > > > > > >> > > > an >>> > > > > > > > > >> >> > > > > > > > > >> > > > > overkill.)* >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an >>> > > overkill. >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > I think we are debating two >>> > > > options: >>> > > > > > > > Either >>> > > > > > > > > >> require >>> > > > > > > > > >> >> > > > > Kerberos >>> > > > > > > > > >> >> > > > > > > > auth >>> > > > > > > > > >> >> > > > > > > > > >> for >>> > > > > > > > > >> >> > > > > > > > > >> > > > renewal or require >>> non-owners >>> > to >>> > > > > > renew. >>> > > > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is >>> > simpler >>> > > > (it >>> > > > > > > > > basically >>> > > > > > > > > >> >> > require >>> > > > > > > > > >> >> > > a >>> > > > > > > > > >> >> > > > > "job >>> > > > > > > > > >> >> > > > > > > > > master" >>> > > > > > > > > >> >> > > > > > > > > >> > > > to take responsibility for >>> the >>> > > > > > renewal, >>> > > > > > > it >>> > > > > > > > > >> will have >>> > > > > > > > > >> >> > > its >>> > > > > > > > > >> >> > > > > own >>> > > > > > > > > >> >> > > > > > > > > >> identity >>> > > > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this is >>> the >>> > > > > correct >>> > > > > > > > > design >>> > > > > > > > > >> >> > pattern >>> > > > > > > > > >> >> > > > > > anyway. >>> > > > > > > > > >> >> > > > > > > > For >>> > > > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to >>> > > > > coordinate >>> > > > > > > > > >> renewals?), >>> > > > > > > > > >> >> > but >>> > > > > > > > > >> >> > > > it >>> > > > > > > > > >> >> > > > > is >>> > > > > > > > > >> >> > > > > > > > hard >>> > > > > > > > > >> >> > > > > > > > > to >>> > > > > > > > > >> >> > > > > > > > > >> > > > debate simplicity without >>> > > looking >>> > > > at >>> > > > > > the >>> > > > > > > > > code >>> > > > > > > > > >> >> > changes >>> > > > > > > > > >> >> > > > > > > required. >>> > > > > > > > > >> >> > > > > > > > If >>> > > > > > > > > >> >> > > > > > > > > >> you >>> > > > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the >>> > "require >>> > > > > > > Kerberos" >>> > > > > > > > > >> will look >>> > > > > > > > > >> >> > > in >>> > > > > > > > > >> >> > > > > > Kafka >>> > > > > > > > > >> >> > > > > > > > > code, >>> > > > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a >>> look. >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is >>> that >>> > > > tokens >>> > > > > > will >>> > > > > > > > > >> propagate >>> > > > > > > > > >> >> > via >>> > > > > > > > > >> >> > > > ZK >>> > > > > > > > > >> >> > > > > > but >>> > > > > > > > > >> >> > > > > > > > > >> without >>> > > > > > > > > >> >> > > > > > > > > >> > > > > additional changes to >>> > > > > UpdateMetadata >>> > > > > > > > > >> protocol, >>> > > > > > > > > >> >> > > > correct? >>> > > > > > > > > >> >> > > > > > > > Clients >>> > > > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry on >>> > SASL >>> > > > auth >>> > > > > > > > failure >>> > > > > > > > > >> (IIRC), >>> > > > > > > > > >> >> > > but >>> > > > > > > > > >> >> > > > > > since >>> > > > > > > > > >> >> > > > > > > > the >>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate between >>> > > brokers >>> > > > > > > asynch, >>> > > > > > > > > we >>> > > > > > > > > >> will >>> > > > > > > > > >> >> > > need >>> > > > > > > > > >> >> > > > to >>> > > > > > > > > >> >> > > > > > > > retry a >>> > > > > > > > > >> >> > > > > > > > > >> bit >>> > > > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients failing >>> > auth >>> > > > due >>> > > > > to >>> > > > > > > > > timing >>> > > > > > > > > >> >> > issues. >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2 >>> > > alternatives >>> > > > > > right >>> > > > > > > > > now. >>> > > > > > > > > >> The >>> > > > > > > > > >> >> > > > current >>> > > > > > > > > >> >> > > > > > > > > documented >>> > > > > > > > > >> >> > > > > > > > > >> > > > approach >>> > > > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it >>> > does >>> > > > not >>> > > > > > > > require >>> > > > > > > > > >> any >>> > > > > > > > > >> >> > > changes >>> > > > > > > > > >> >> > > > > to >>> > > > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata >>> > > > > > > > > >> >> > > > > > > > > >> > > > > protocol. An alternative >>> > > > approach >>> > > > > > can >>> > > > > > > > > remove >>> > > > > > > > > >> >> > > zookeeper >>> > > > > > > > > >> >> > > > > > > > > dependency >>> > > > > > > > > >> >> > > > > > > > > >> as >>> > > > > > > > > >> >> > > > > > > > > >> > > well >>> > > > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss that >>> in >>> > KIP >>> > > > > > > > discussion >>> > > > > > > > > >> call.* >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. >>> Do >>> > you >>> > > > > want >>> > > > > > to >>> > > > > > > > > ping >>> > > > > > > > > >> Jun to >>> > > > > > > > > >> >> > > > > > arrange a >>> > > > > > > > > >> >> > > > > > > > > call? >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's >>> > suggestion >>> > > of >>> > > > > > > having >>> > > > > > > > > >> just the >>> > > > > > > > > >> >> > > > > > controller >>> > > > > > > > > >> >> > > > > > > > > issue >>> > > > > > > > > >> >> > > > > > > > > >> the >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to >>> avoid >>> > > > > syncing >>> > > > > > a >>> > > > > > > > > shared >>> > > > > > > > > >> >> > secret. >>> > > > > > > > > >> >> > > > Not >>> > > > > > > > > >> >> > > > > > > sure >>> > > > > > > > > >> >> > > > > > > > if >>> > > > > > > > > >> >> > > > > > > > > >> we >>> > > > > > > > > >> >> > > > > > > > > >> > > > > want to continue the >>> > > discussion >>> > > > > here >>> > > > > > > or >>> > > > > > > > on >>> > > > > > > > > >> the >>> > > > > > > > > >> >> > > wiki. I >>> > > > > > > > > >> >> > > > > > think >>> > > > > > > > > >> >> > > > > > > > > that >>> > > > > > > > > >> >> > > > > > > > > >> we >>> > > > > > > > > >> >> > > > > > > > > >> > > > > can decouple the problem >>> of >>> > > > "token >>> > > > > > > > > >> distribution" >>> > > > > > > > > >> >> > > from >>> > > > > > > > > >> >> > > > > > > "shared >>> > > > > > > > > >> >> > > > > > > > > >> secret >>> > > > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use the >>> > > > > controller >>> > > > > > > as >>> > > > > > > > > the >>> > > > > > > > > >> only >>> > > > > > > > > >> >> > > token >>> > > > > > > > > >> >> > > > > > > > generator >>> > > > > > > > > >> >> > > > > > > > > >> to >>> > > > > > > > > >> >> > > > > > > > > >> > > > > solve the second issue, >>> > while >>> > > > > still >>> > > > > > > > using >>> > > > > > > > > >> ZK async >>> > > > > > > > > >> >> > > to >>> > > > > > > > > >> >> > > > > > > > distribute >>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens. >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the >>> > previous >>> > > > > Email >>> > > > > > I >>> > > > > > > am >>> > > > > > > > > >> fine with >>> > > > > > > > > >> >> > > > that >>> > > > > > > > > >> >> > > > > > > > approach >>> > > > > > > > > >> >> > > > > > > > > >> as >>> > > > > > > > > >> >> > > > > > > > > >> > > long >>> > > > > > > > > >> >> > > > > > > > > >> > > > as >>> > > > > > > > > >> >> > > > > > > > > >> > > > > we agree that the extra >>> > > > complexity >>> > > > > > of >>> > > > > > > > > >> >> > > adding/updating >>> > > > > > > > > >> >> > > > > APIS >>> > > > > > > > > >> >> > > > > > > > adds >>> > > > > > > > > >> >> > > > > > > > > >> enough >>> > > > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage with >>> > the >>> > > > > > > controller >>> > > > > > > > > >> approach >>> > > > > > > > > >> >> > is >>> > > > > > > > > >> >> > > > > > secret >>> > > > > > > > > >> >> > > > > > > > > >> rotation >>> > > > > > > > > >> >> > > > > > > > > >> > > can >>> > > > > > > > > >> >> > > > > > > > > >> > > > be >>> > > > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and >>> would >>> > > not >>> > > > > > > require >>> > > > > > > > > >> >> > > deployment. * >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra >>> > > > complexity >>> > > > > > (or >>> > > > > > > > > point >>> > > > > > > > > >> me to >>> > > > > > > > > >> >> > > the >>> > > > > > > > > >> >> > > > > > email >>> > > > > > > > > >> >> > > > > > > I >>> > > > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are >>> > > > required? >>> > > > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell, >>> clients >>> > > can >>> > > > > > > already >>> > > > > > > > > >> find the >>> > > > > > > > > >> >> > > > > > controller >>> > > > > > > > > >> >> > > > > > > > from >>> > > > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more >>> > > concerned >>> > > > > > about >>> > > > > > > > > >> controller >>> > > > > > > > > >> >> > > > load. >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * While I like the idea >>> of >>> > > > forcing >>> > > > > > > > > kerberos >>> > > > > > > > > >> auth >>> > > > > > > > > >> >> > for >>> > > > > > > > > >> >> > > > > > > renwal, I >>> > > > > > > > > >> >> > > > > > > > > >> think >>> > > > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport >>> layer >>> > > the >>> > > > > the >>> > > > > > > > > request >>> > > > > > > > > >> >> > content >>> > > > > > > > > >> >> > > > in >>> > > > > > > > > >> >> > > > > a >>> > > > > > > > > >> >> > > > > > > > pretty >>> > > > > > > > > >> >> > > > > > > > > >> ugly >>> > > > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting >>> > renewer >>> > > to >>> > > > > > > > non-owner >>> > > > > > > > > >> is >>> > > > > > > > > >> >> > > better. >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a >>> necessary >>> > > > evil. >>> > > > > > > While >>> > > > > > > > > >> this will >>> > > > > > > > > >> >> > > make >>> > > > > > > > > >> >> > > > > the >>> > > > > > > > > >> >> > > > > > > > kafka >>> > > > > > > > > >> >> > > > > > > > > >> code >>> > > > > > > > > >> >> > > > > > > > > >> > > > > pretty straight forward , >>> > > > forcing >>> > > > > > > > renewer >>> > > > > > > > > >> to >>> > > > > > > > > >> >> > > > non-owner >>> > > > > > > > > >> >> > > > > > > pushes >>> > > > > > > > > >> >> > > > > > > > > >> the code >>> > > > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and >>> makes >>> > > it >>> > > > > even >>> > > > > > > > > harder >>> > > > > > > > > >> to >>> > > > > > > > > >> >> > > > > > integrate. * >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I >>> don't >>> > > think >>> > > > > the >>> > > > > > > > > >> "renewal by >>> > > > > > > > > >> >> > > > other" >>> > > > > > > > > >> >> > > > > > > > approach >>> > > > > > > > > >> >> > > > > > > > > >> is >>> > > > > > > > > >> >> > > > > > > > > >> > > > that ugly for the clients >>> we >>> > > > expect >>> > > > > to >>> > > > > > > use >>> > > > > > > > > >> >> > delegation >>> > > > > > > > > >> >> > > > > tokens >>> > > > > > > > > >> >> > > > > > > > since >>> > > > > > > > > >> >> > > > > > > > > >> > > > they will have an >>> app-master >>> > of >>> > > > some >>> > > > > > > sort >>> > > > > > > > > who >>> > > > > > > > > >> >> > > requested >>> > > > > > > > > >> >> > > > > the >>> > > > > > > > > >> >> > > > > > > > token >>> > > > > > > > > >> >> > > > > > > > > to >>> > > > > > > > > >> >> > > > > > > > > >> > > > begin with. >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > The response for my >>> question >>> > > on >>> > > > > how >>> > > > > > > > > multiple >>> > > > > > > > > >> >> > > > identities >>> > > > > > > > > >> >> > > > > > will >>> > > > > > > > > >> >> > > > > > > > be >>> > > > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super >>> clear >>> > to >>> > > > me - >>> > > > > > > > AFAIK, >>> > > > > > > > > >> we don't >>> > > > > > > > > >> >> > > > > > > > authenticate >>> > > > > > > > > >> >> > > > > > > > > >> each >>> > > > > > > > > >> >> > > > > > > > > >> > > > > request, we authenticate >>> > > > > > connections. >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate >>> > connections, >>> > > > and >>> > > > > > only >>> > > > > > > > > when >>> > > > > > > > > >> they >>> > > > > > > > > >> >> > are >>> > > > > > > > > >> >> > > > > being >>> > > > > > > > > >> >> > > > > > > > > >> established. >>> > > > > > > > > >> >> > > > > > > > > >> > > > Let >>> > > > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this as >>> a >>> > > > > question, >>> > > > > > > in >>> > > > > > > > > >> absence of >>> > > > > > > > > >> >> > > > > > > delegation >>> > > > > > > > > >> >> > > > > > > > > >> tokens if >>> > > > > > > > > >> >> > > > > > > > > >> > > > we >>> > > > > > > > > >> >> > > > > > > > > >> > > > > had to support the use >>> case >>> > > > using >>> > > > > > user >>> > > > > > > > > >> TGT's how >>> > > > > > > > > >> >> > > would >>> > > > > > > > > >> >> > > > > we >>> > > > > > > > > >> >> > > > > > do >>> > > > > > > > > >> >> > > > > > > > it? >>> > > > > > > > > >> >> > > > > > > > > >> My >>> > > > > > > > > >> >> > > > > > > > > >> > > point >>> > > > > > > > > >> >> > > > > > > > > >> > > > > was it would be no >>> different >>> > > > with >>> > > > > > > > > delegation >>> > > > > > > > > >> >> > tokens. >>> > > > > > > > > >> >> > > > The >>> > > > > > > > > >> >> > > > > > use >>> > > > > > > > > >> >> > > > > > > > > case >>> > > > > > > > > >> >> > > > > > > > > >> you >>> > > > > > > > > >> >> > > > > > > > > >> > > are >>> > > > > > > > > >> >> > > > > > > > > >> > > > > describing seems more >>> like >>> > > > > > > > impersonation.* >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of >>> > the >>> > > > > things >>> > > > > > > > that >>> > > > > > > > > >> >> > del >>> >> >> >> >> -- >> >> Regards, >> Ashish >> -- Gwen Shapira Product Manager | Confluent 650.450.2760 | @gwenshap Follow us: Twitter | blog
