According to the meeting minutes of KIP hangout on 8/30, it seems the KIP wiki needs some update?
KIP48 (delegation tokens): Harsha will update the wiki with more details on how to use delegation tokens and how to configure it. Not sure if that has been done or not. On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira <g...@confluent.io> wrote: > Hi Guys, > > This discussion was dead for a while. Are there still contentious > points? If not, why are there no votes? > > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao <j...@confluent.io> wrote: > > Ashish, > > > > Yes, I will send out a KIP invite for next week to discuss KIP-48 and > other > > remaining KIPs. > > > > Thanks, > > > > Jun > > > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh <asi...@cloudera.com> > wrote: > > > >> Thanks Harsha! > >> > >> Jun, can we add KIP-48 to next KIP hangout's agenda. Also, we did not > >> actually make a call on when we should have next KIP call. As there are > a > >> few outstanding KIPs that could not be discussed this week, can we have > a > >> KIP hangout call next week? > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha Chintalapani <ka...@harsha.io> > >> wrote: > >> > >>> Ashish, > >>> Yes we are working on it. Lets discuss in the next KIP meeting. > >>> I'll join. > >>> -Harsha > >>> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish Singh <asi...@cloudera.com> > >>> wrote: > >>> > >>> > Hello Harsha, > >>> > > >>> > Are you still working on this? Wondering if we can discuss this in > next > >>> KIP > >>> > meeting, if you can join. > >>> > > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani < > ka...@harsha.io> > >>> > wrote: > >>> > > >>> > > Hi Grant, > >>> > > We are working on it. Will add the details to KIP about > the > >>> > > request protocol. > >>> > > > >>> > > Thanks, > >>> > > Harsha > >>> > > > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <ghe...@cloudera.com> > >>> wrote: > >>> > > > >>> > > > Hi Parth, > >>> > > > > >>> > > > Are you still working on this? If you need any help please don't > >>> > hesitate > >>> > > > to ask. > >>> > > > > >>> > > > Thanks, > >>> > > > Grant > >>> > > > > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <j...@confluent.io> > wrote: > >>> > > > > >>> > > > > Parth, > >>> > > > > > >>> > > > > Thanks for the reply. > >>> > > > > > >>> > > > > It makes sense to only allow the renewal by users that > >>> authenticated > >>> > > > using > >>> > > > > *non* delegation token mechanism. Then, should we make the > >>> renewal a > >>> > > > list? > >>> > > > > For example, in the case of rest proxy, it will be useful for > >>> every > >>> > > > > instance of rest proxy to be able to renew the tokens. > >>> > > > > > >>> > > > > It would be clearer if we can document the request protocol > like > >>> > > > > > >>> > > > > > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > >>> > > 4+-+Command+line+and+centralized+administrative+operations#KIP-4- > >>> > > Commandlineandcentralizedadministrativeoperations- > >>> > > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0) > >>> > > > > . > >>> > > > > > >>> > > > > It would also be useful to document the client APIs. > >>> > > > > > >>> > > > > Thanks, > >>> > > > > > >>> > > > > Jun > >>> > > > > > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt < > >>> > > > > brahmbhatt.pa...@gmail.com> wrote: > >>> > > > > > >>> > > > > > Hi, > >>> > > > > > > >>> > > > > > I am suggesting that we will only allow the renewal by users > >>> that > >>> > > > > > authenticated using *non* delegation token mechanism. For > >>> example, > >>> > If > >>> > > > > user > >>> > > > > > Alice authenticated using kerberos and requested delegation > >>> tokens, > >>> > > > only > >>> > > > > > user Alice authenticated via non delegation token mechanism > can > >>> > > renew. > >>> > > > > > Clients that have access to delegation tokens can not issue > >>> > renewal > >>> > > > > > request for renewing their own token and this is primarily > >>> > important > >>> > > to > >>> > > > > > reduce the time window for which a compromised token will be > >>> valid. > >>> > > > > > > >>> > > > > > To clarify, Yes any authenticated user can request delegation > >>> > tokens > >>> > > > but > >>> > > > > > even here I would recommend to avoid creating a chain where a > >>> > client > >>> > > > > > authenticated via delegation token request for more > delegation > >>> > > tokens. > >>> > > > > > Basically anyone can request delegation token, as long as > they > >>> > > > > authenticate > >>> > > > > > via a non delegation token mechanism. > >>> > > > > > > >>> > > > > > Aren't classes listed here > >>> > > > > > < > >>> > > > > > > >>> > > > > > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > >>> > > 48+Delegation+token+support+for+Kafka#KIP-48Delegationtokens > >>> upportforKaf > >>> > > ka-PublicInterfaces > >>> > > > > > > > >>> > > > > > sufficient? > >>> > > > > > > >>> > > > > > Thanks > >>> > > > > > Parth > >>> > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <j...@confluent.io> > >>> wrote: > >>> > > > > > > >>> > > > > > > Parth, > >>> > > > > > > > >>> > > > > > > Thanks for the reply. A couple of comments inline below. > >>> > > > > > > > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt < > >>> > > > > > > brahmbhatt.pa...@gmail.com> wrote: > >>> > > > > > > > >>> > > > > > > > 1. Who / how are tokens renewed? By original requester > >>> only? or > >>> > > > using > >>> > > > > > > > Kerberos > >>> > > > > > > > auth only? > >>> > > > > > > > My recommendation is to do this only using Kerberos auth > and > >>> > only > >>> > > > > threw > >>> > > > > > > the > >>> > > > > > > > renewer specified during the acquisition request. > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > Hmm, not sure that I follow this. Are you saying that any > >>> client > >>> > > > > > > authenticated with the delegation token can renew, i.e. > there > >>> is > >>> > no > >>> > > > > > renewer > >>> > > > > > > needed? > >>> > > > > > > > >>> > > > > > > Also, just to be clear, any authenticated client (either > >>> through > >>> > > SASL > >>> > > > > or > >>> > > > > > > SSL) can request a delegation token for the authenticated > >>> user, > >>> > > > right? > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > 2. Are tokens stored on each broker or in ZK? > >>> > > > > > > > My recommendation is still to store in ZK or not store > them > >>> at > >>> > > all. > >>> > > > > The > >>> > > > > > > > whole controller based distribution is too much overhead > >>> with > >>> > not > >>> > > > > much > >>> > > > > > to > >>> > > > > > > > achieve. > >>> > > > > > > > > >>> > > > > > > > 3. How are tokens invalidated / expired? > >>> > > > > > > > Either by expiration time out or through an explicit > >>> request to > >>> > > > > > > invalidate. > >>> > > > > > > > > >>> > > > > > > > 4. Which encryption algorithm is used? > >>> > > > > > > > SCRAM > >>> > > > > > > > > >>> > > > > > > > 5. What is the impersonation proposal (it wasn't in the > KIP > >>> but > >>> > > was > >>> > > > > > > > discussed > >>> > > > > > > > in this thread)? > >>> > > > > > > > There is no imperonation proposal. I tried and explained > how > >>> > its > >>> > > a > >>> > > > > > > > different problem and why its not really necessary to > >>> discuss > >>> > > that > >>> > > > as > >>> > > > > > > part > >>> > > > > > > > of this KIP. This KIP will not support any > impersonation, > >>> it > >>> > > will > >>> > > > > just > >>> > > > > > > be > >>> > > > > > > > another way to authenticate. > >>> > > > > > > > > >>> > > > > > > > 6. Do we need new ACLs, if so - for what actions? > >>> > > > > > > > We do not need new ACLs. > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > Could we document the format of the new request/response > and > >>> > their > >>> > > > > > > associated Resource and Operation for ACL? > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > 7. How would the delegation token be configured in the > >>> client? > >>> > > > > > > > Should be through config. I wasn't planning on supporting > >>> JAAS > >>> > > for > >>> > > > > > > tokens. > >>> > > > > > > > I don't believe hadoop does this either. > >>> > > > > > > > > >>> > > > > > > > Thanks > >>> > > > > > > > Parth > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao < > j...@confluent.io> > >>> > > wrote: > >>> > > > > > > > > >>> > > > > > > > > Harsha, > >>> > > > > > > > > > >>> > > > > > > > > Another question. > >>> > > > > > > > > > >>> > > > > > > > > 9. How would the delegation token be configured in the > >>> > client? > >>> > > > The > >>> > > > > > > > standard > >>> > > > > > > > > way is to do this through JAAS. However, we will need > to > >>> > think > >>> > > > > > through > >>> > > > > > > if > >>> > > > > > > > > this is convenient in a shared environment. For > example, > >>> > when a > >>> > > > new > >>> > > > > > > task > >>> > > > > > > > is > >>> > > > > > > > > added to a Storm worker node, do we need to dynamically > >>> add a > >>> > > new > >>> > > > > > > section > >>> > > > > > > > > in the JAAS file? It may be more convenient if we can > >>> pass in > >>> > > the > >>> > > > > > token > >>> > > > > > > > > through the config directly w/o going through JAAS. > >>> > > > > > > > > > >>> > > > > > > > > Are you or Parth still actively working on this KIP? > >>> > > > > > > > > > >>> > > > > > > > > Thanks, > >>> > > > > > > > > > >>> > > > > > > > > Jun > >>> > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao < > >>> j...@confluent.io> > >>> > > > wrote: > >>> > > > > > > > > > >>> > > > > > > > > > Just to add on that list. > >>> > > > > > > > > > > >>> > > > > > > > > > 2. It would be good to document the format of the > data > >>> > stored > >>> > > > in > >>> > > > > > ZK. > >>> > > > > > > > > > 7. Earlier, there was a discussion on whether the > tokens > >>> > > should > >>> > > > > be > >>> > > > > > > > > > propagated through ZK like config/acl/quota, or > through > >>> the > >>> > > > > > > controller. > >>> > > > > > > > > > Currently, the controller is only designed for > >>> propagating > >>> > > > topic > >>> > > > > > > > > metadata, > >>> > > > > > > > > > but not other data. > >>> > > > > > > > > > 8. Should we use SCRAM to send the token instead of > >>> > > DIGEST-MD5 > >>> > > > > > since > >>> > > > > > > > it's > >>> > > > > > > > > > deprecated? > >>> > > > > > > > > > > >>> > > > > > > > > > Also, the images in the wiki seem broken. > >>> > > > > > > > > > > >>> > > > > > > > > > Thanks, > >>> > > > > > > > > > > >>> > > > > > > > > > Jun > >>> > > > > > > > > > > >>> > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira < > >>> > > > > g...@confluent.io> > >>> > > > > > > > > wrote: > >>> > > > > > > > > > > >>> > > > > > > > > >> From what I can see, remaining questions are: > >>> > > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens renewed? By original > requester > >>> > only? > >>> > > > or > >>> > > > > > > using > >>> > > > > > > > > >> Kerberos auth only? > >>> > > > > > > > > >> 2. Are tokens stored on each broker or in ZK? > >>> > > > > > > > > >> 3. How are tokens invalidated / expired? > >>> > > > > > > > > >> 4. Which encryption algorithm is used? > >>> > > > > > > > > >> 5. What is the impersonation proposal (it wasn't in > the > >>> > KIP > >>> > > > but > >>> > > > > > was > >>> > > > > > > > > >> discussed in this thread)? > >>> > > > > > > > > >> 6. Do we need new ACLs, if so - for what actions? > >>> > > > > > > > > >> > >>> > > > > > > > > >> Gwen > >>> > > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha < > >>> ka...@harsha.io> > >>> > > > wrote: > >>> > > > > > > > > >> > Jun & Ismael, > >>> > > > > > > > > >> > Unfortunately I couldn't > >>> attend > >>> > > the > >>> > > > > KIP > >>> > > > > > > > > meeting > >>> > > > > > > > > >> > when delegation tokens > >>> > discussed. > >>> > > > > > > > Appreciate > >>> > > > > > > > > if > >>> > > > > > > > > >> > you can update the > thread if > >>> > you > >>> > > > have > >>> > > > > > any > >>> > > > > > > > > >> > further questions. > >>> > > > > > > > > >> > Thanks, > >>> > > > > > > > > >> > Harsha > >>> > > > > > > > > >> > > >>> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei > wrote: > >>> > > > > > > > > >> >> It seems that the links to images in the KIP are > >>> > broken. > >>> > > > > > > > > >> >> > >>> > > > > > > > > >> >> Liquan > >>> > > > > > > > > >> >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth > brahmbhatt < > >>> > > > > > > > > >> >> brahmbhatt.pa...@gmail.com> wrote: > >>> > > > > > > > > >> >> > >>> > > > > > > > > >> >> > 110. What does getDelegationTokenAs mean? > >>> > > > > > > > > >> >> > In the current proposal we only allow a user to > >>> get > >>> > > > > > delegation > >>> > > > > > > > > token > >>> > > > > > > > > >> for > >>> > > > > > > > > >> >> > the identity that it authenticated as using > >>> another > >>> > > > > > mechanism, > >>> > > > > > > > i.e. > >>> > > > > > > > > >> A user > >>> > > > > > > > > >> >> > that authenticate using a keytab for principal > >>> > > > > > > us...@example.com > >>> > > > > > > > > >> will get > >>> > > > > > > > > >> >> > delegation tokens for that user only. In > future I > >>> > think > >>> > > > we > >>> > > > > > will > >>> > > > > > > > > have > >>> > > > > > > > > >> to > >>> > > > > > > > > >> >> > extend support such that we allow some set of > >>> users ( > >>> > > > > > > > > >> >> > kafka-rest-u...@example.com, > >>> > storm-nim...@example.com) > >>> > > > to > >>> > > > > > > > acquire > >>> > > > > > > > > >> >> > delegation tokens on behalf of other users > whose > >>> > > identity > >>> > > > > > they > >>> > > > > > > > have > >>> > > > > > > > > >> >> > verified independently. Kafka brokers will > have > >>> ACLs > >>> > > to > >>> > > > > > > control > >>> > > > > > > > > >> which > >>> > > > > > > > > >> >> > users are allowed to impersonate other users > and > >>> get > >>> > > > tokens > >>> > > > > > on > >>> > > > > > > > > >> behalf of > >>> > > > > > > > > >> >> > them. Overall Impersonation is a whole > different > >>> > > problem > >>> > > > in > >>> > > > > > my > >>> > > > > > > > > >> opinion and > >>> > > > > > > > > >> >> > I think we can tackle it in separate KIP. > >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > 111. What's the typical rate of getting and > >>> renewing > >>> > > > > > delegation > >>> > > > > > > > > >> tokens? > >>> > > > > > > > > >> >> > Typically this should be very very low, 1 > request > >>> per > >>> > > > > minute > >>> > > > > > > is a > >>> > > > > > > > > >> >> > relatively high estimate. However it depends on > >>> the > >>> > > token > >>> > > > > > > > > >> expiration. I am > >>> > > > > > > > > >> >> > less worried about the extra load it puts on > >>> > controller > >>> > > > vs > >>> > > > > > the > >>> > > > > > > > > added > >>> > > > > > > > > >> >> > complexity and the value it offers. > >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > Thanks > >>> > > > > > > > > >> >> > Parth > >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma < > >>> > > > > > > ism...@juma.me.uk> > >>> > > > > > > > > >> wrote: > >>> > > > > > > > > >> >> > > >>> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a > >>> separate > >>> > > KIP > >>> > > > > as > >>> > > > > > it > >>> > > > > > > > > will > >>> > > > > > > > > >> >> > > introduce user visible changes. We could also > >>> > update > >>> > > > > KIP-48 > >>> > > > > > > to > >>> > > > > > > > > >> have this > >>> > > > > > > > > >> >> > > information, but it seems cleaner to do it > >>> > > separately. > >>> > > > We > >>> > > > > > can > >>> > > > > > > > > >> discuss > >>> > > > > > > > > >> >> > that > >>> > > > > > > > > >> >> > > in the KIP call today. > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > Ismael > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini > Sivaram > >>> < > >>> > > > > > > > > >> >> > > rajinisiva...@googlemail.com> wrote: > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > Ismael, > >>> > > > > > > > > >> >> > > > > >>> > > > > > > > > >> >> > > > I have created a JIRA ( > >>> > > > > > > > > >> >> > https://issues.apache.org/ > jira/browse/KAFKA-3751) > >>> > > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would > >>> that > >>> > > need > >>> > > > > > > another > >>> > > > > > > > > >> KIP? If > >>> > > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this > just > >>> be > >>> > a > >>> > > > JIRA > >>> > > > > > > that > >>> > > > > > > > > gets > >>> > > > > > > > > >> >> > > reviewed > >>> > > > > > > > > >> >> > > > when the PR is ready? > >>> > > > > > > > > >> >> > > > > >>> > > > > > > > > >> >> > > > Thank you, > >>> > > > > > > > > >> >> > > > > >>> > > > > > > > > >> >> > > > Rajini > >>> > > > > > > > > >> >> > > > > >>> > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael > Juma < > >>> > > > > > > > > ism...@juma.me.uk> > >>> > > > > > > > > >> >> > wrote: > >>> > > > > > > > > >> >> > > > > >>> > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good > >>> > candidate. > >>> > > > > > > > > >> >> > > > > > >>> > > > > > > > > >> >> > > > > Gwen had independently mentioned this as > a > >>> SASL > >>> > > > > > mechanism > >>> > > > > > > > > that > >>> > > > > > > > > >> might > >>> > > > > > > > > >> >> > be > >>> > > > > > > > > >> >> > > > > useful for Kafka and I have been meaning > to > >>> > check > >>> > > > it > >>> > > > > in > >>> > > > > > > > more > >>> > > > > > > > > >> detail. > >>> > > > > > > > > >> >> > > Good > >>> > > > > > > > > >> >> > > > > to know that you are willing to > contribute > >>> an > >>> > > > > > > > implementation. > >>> > > > > > > > > >> Maybe > >>> > > > > > > > > >> >> > we > >>> > > > > > > > > >> >> > > > > should file a separate JIRA for this? > >>> > > > > > > > > >> >> > > > > > >>> > > > > > > > > >> >> > > > > Ismael > >>> > > > > > > > > >> >> > > > > > >>> > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini > >>> > Sivaram < > >>> > > > > > > > > >> >> > > > > rajinisiva...@googlemail.com> wrote: > >>> > > > > > > > > >> >> > > > > > >>> > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response > >>> > Authentication > >>> > > > > > > > Mechanism) > >>> > > > > > > > > >> is a > >>> > > > > > > > > >> >> > > better > >>> > > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't > >>> come > >>> > > > with a > >>> > > > > > > > > built-in > >>> > > > > > > > > >> SCRAM > >>> > > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be > >>> happy > >>> > > to > >>> > > > > add > >>> > > > > > > > > support > >>> > > > > > > > > >> in > >>> > > > > > > > > >> >> > Kafka > >>> > > > > > > > > >> >> > > > > since > >>> > > > > > > > > >> >> > > > > > it would be a useful mechanism to > support > >>> > > anyway. > >>> > > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677 > >>> > describes > >>> > > > the > >>> > > > > > > > protocol > >>> > > > > > > > > >> for > >>> > > > > > > > > >> >> > > > > > SCRAM-SHA-256. > >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun > Rao < > >>> > > > > > > > j...@confluent.io > >>> > > > > > > > > > > >>> > > > > > > > > >> wrote: > >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > > > > Parth, > >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > Thanks for the explanation. A couple > of > >>> > more > >>> > > > > > > questions. > >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs > >>> mean? > >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > 111. What's the typical rate of > getting > >>> and > >>> > > > > > renewing > >>> > > > > > > > > >> delegation > >>> > > > > > > > > >> >> > > > tokens? > >>> > > > > > > > > >> >> > > > > > > That may have an impact on whether > they > >>> > > should > >>> > > > be > >>> > > > > > > > > directed > >>> > > > > > > > > >> to the > >>> > > > > > > > > >> >> > > > > > > controller. > >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > Jun > >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, > parth > >>> > > > > brahmbhatt < > >>> > > > > > > > > >> >> > > > > > > brahmbhatt.pa...@gmail.com> wrote: > >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > Hi Jun, > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > Thanks for reviewing. > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > * We could add a Cluster action to > add > >>> > acls > >>> > > > on > >>> > > > > > who > >>> > > > > > > > can > >>> > > > > > > > > >> request > >>> > > > > > > > > >> >> > > > > > delegation > >>> > > > > > > > > >> >> > > > > > > > tokens. I don't see the use case > for > >>> that > >>> > > yet > >>> > > > > but > >>> > > > > > > > down > >>> > > > > > > > > >> the line > >>> > > > > > > > > >> >> > > > when > >>> > > > > > > > > >> >> > > > > we > >>> > > > > > > > > >> >> > > > > > > > start supporting > getDelegationTokenAs > >>> it > >>> > > will > >>> > > > > be > >>> > > > > > > > > >> necessary. > >>> > > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be > only > >>> > > > > > > used/distributed > >>> > > > > > > > > >> over > >>> > > > > > > > > >> >> > secure > >>> > > > > > > > > >> >> > > > > > > channels. > >>> > > > > > > > > >> >> > > > > > > > * Depending on what design we end > up > >>> > > choosing > >>> > > > > > > > > >> Invalidation will > >>> > > > > > > > > >> >> > > be > >>> > > > > > > > > >> >> > > > > > > > responsibility of every broker or > >>> > > controller. > >>> > > > > > > > > >> >> > > > > > > > * I am not sure if I documented > >>> somewhere > >>> > > > that > >>> > > > > > > > > >> invalidation > >>> > > > > > > > > >> >> > will > >>> > > > > > > > > >> >> > > > > > directly > >>> > > > > > > > > >> >> > > > > > > > go through zookeeper but that is > not > >>> the > >>> > > > > intent. > >>> > > > > > > > > >> Invalidation > >>> > > > > > > > > >> >> > > will > >>> > > > > > > > > >> >> > > > > > either > >>> > > > > > > > > >> >> > > > > > > > be request based or due to > >>> expiration. No > >>> > > > > direct > >>> > > > > > > > > >> zookeeper > >>> > > > > > > > > >> >> > > > > interaction > >>> > > > > > > > > >> >> > > > > > > from > >>> > > > > > > > > >> >> > > > > > > > any client. > >>> > > > > > > > > >> >> > > > > > > > * "Broker also stores the > >>> DelegationToken > >>> > > > > without > >>> > > > > > > the > >>> > > > > > > > > >> hmac in > >>> > > > > > > > > >> >> > the > >>> > > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the > >>> confusion. > >>> > > The > >>> > > > > sole > >>> > > > > > > > > >> purpose of > >>> > > > > > > > > >> >> > > > > zookeeper > >>> > > > > > > > > >> >> > > > > > in > >>> > > > > > > > > >> >> > > > > > > > this design is as distribution > channel > >>> > for > >>> > > > > tokens > >>> > > > > > > > > >> between all > >>> > > > > > > > > >> >> > > > brokers > >>> > > > > > > > > >> >> > > > > > > and a > >>> > > > > > > > > >> >> > > > > > > > layer that ensures only tokens that > >>> were > >>> > > > > > generated > >>> > > > > > > by > >>> > > > > > > > > >> making a > >>> > > > > > > > > >> >> > > > > request > >>> > > > > > > > > >> >> > > > > > > to a > >>> > > > > > > > > >> >> > > > > > > > broker will be accepted (more on > this > >>> in > >>> > > > second > >>> > > > > > > > > >> paragraph). The > >>> > > > > > > > > >> >> > > > token > >>> > > > > > > > > >> >> > > > > > > > consists of few elements (owner, > >>> renewer, > >>> > > > uuid > >>> > > > > , > >>> > > > > > > > > >> expiration, > >>> > > > > > > > > >> >> > > hmac) > >>> > > > > > > > > >> >> > > > , > >>> > > > > > > > > >> >> > > > > > one > >>> > > > > > > > > >> >> > > > > > > of > >>> > > > > > > > > >> >> > > > > > > > which is the finally generated hmac > >>> but > >>> > > hmac > >>> > > > it > >>> > > > > > > self > >>> > > > > > > > is > >>> > > > > > > > > >> >> > derivable > >>> > > > > > > > > >> >> > > > if > >>> > > > > > > > > >> >> > > > > > you > >>> > > > > > > > > >> >> > > > > > > > have all the other elements of the > >>> token > >>> > + > >>> > > > > secret > >>> > > > > > > key > >>> > > > > > > > > to > >>> > > > > > > > > >> >> > generate > >>> > > > > > > > > >> >> > > > > hmac. > >>> > > > > > > > > >> >> > > > > > > > Given zookeeper does not provide > SSL > >>> > > support > >>> > > > we > >>> > > > > > do > >>> > > > > > > > not > >>> > > > > > > > > >> want the > >>> > > > > > > > > >> >> > > > > entire > >>> > > > > > > > > >> >> > > > > > > > token to be wire transferred to > >>> zookeeper > >>> > > as > >>> > > > > that > >>> > > > > > > > will > >>> > > > > > > > > >> be an > >>> > > > > > > > > >> >> > > > insecure > >>> > > > > > > > > >> >> > > > > > > wire > >>> > > > > > > > > >> >> > > > > > > > transfer. Instead we only store all > >>> the > >>> > > other > >>> > > > > > > > elements > >>> > > > > > > > > >> of a > >>> > > > > > > > > >> >> > > > > delegation > >>> > > > > > > > > >> >> > > > > > > > tokens. Brokers can read these > >>> elements > >>> > and > >>> > > > > > because > >>> > > > > > > > > they > >>> > > > > > > > > >> also > >>> > > > > > > > > >> >> > > have > >>> > > > > > > > > >> >> > > > > > access > >>> > > > > > > > > >> >> > > > > > > > to secret key they will be able to > >>> > generate > >>> > > > > hmac > >>> > > > > > on > >>> > > > > > > > > >> their end. > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > One of the alternative proposed is > to > >>> > avoid > >>> > > > > > > zookeeper > >>> > > > > > > > > >> >> > > altogether. A > >>> > > > > > > > > >> >> > > > > > > Client > >>> > > > > > > > > >> >> > > > > > > > will call broker with required > >>> > information > >>> > > > > > (owner, > >>> > > > > > > > > >> renwer, > >>> > > > > > > > > >> >> > > > > expiration) > >>> > > > > > > > > >> >> > > > > > > and > >>> > > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid). > Broker > >>> > won't > >>> > > > > store > >>> > > > > > > this > >>> > > > > > > > > in > >>> > > > > > > > > >> >> > > zookeeper. > >>> > > > > > > > > >> >> > > > > > From > >>> > > > > > > > > >> >> > > > > > > > this point a client can contact any > >>> > broker > >>> > > > with > >>> > > > > > all > >>> > > > > > > > the > >>> > > > > > > > > >> >> > > delegation > >>> > > > > > > > > >> >> > > > > > token > >>> > > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration, > hmac, > >>> > > uuid) > >>> > > > > the > >>> > > > > > > > borker > >>> > > > > > > > > >> will > >>> > > > > > > > > >> >> > > > > regenerate > >>> > > > > > > > > >> >> > > > > > > the > >>> > > > > > > > > >> >> > > > > > > > hmac and as long as it matches with > >>> hmac > >>> > > > > > presented > >>> > > > > > > by > >>> > > > > > > > > >> client , > >>> > > > > > > > > >> >> > > > broker > >>> > > > > > > > > >> >> > > > > > > will > >>> > > > > > > > > >> >> > > > > > > > allow the request to authenticate. > >>> Only > >>> > > > > problem > >>> > > > > > > with > >>> > > > > > > > > >> this > >>> > > > > > > > > >> >> > > approach > >>> > > > > > > > > >> >> > > > > is > >>> > > > > > > > > >> >> > > > > > if > >>> > > > > > > > > >> >> > > > > > > > the secret key is compromised any > >>> client > >>> > > can > >>> > > > > now > >>> > > > > > > > > generate > >>> > > > > > > > > >> >> > random > >>> > > > > > > > > >> >> > > > > tokens > >>> > > > > > > > > >> >> > > > > > > and > >>> > > > > > > > > >> >> > > > > > > > they will still be able to > >>> authenticate > >>> > as > >>> > > > any > >>> > > > > > user > >>> > > > > > > > > they > >>> > > > > > > > > >> like. > >>> > > > > > > > > >> >> > > with > >>> > > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only > >>> tokens > >>> > > > > acquired > >>> > > > > > > via > >>> > > > > > > > a > >>> > > > > > > > > >> broker > >>> > > > > > > > > >> >> > > > (using > >>> > > > > > > > > >> >> > > > > > some > >>> > > > > > > > > >> >> > > > > > > > auth scheme other than delegation > >>> token) > >>> > > will > >>> > > > > be > >>> > > > > > > > > >> accepted. We > >>> > > > > > > > > >> >> > > need > >>> > > > > > > > > >> >> > > > to > >>> > > > > > > > > >> >> > > > > > > > discuss which proposal makes more > >>> sense > >>> > and > >>> > > > we > >>> > > > > > can > >>> > > > > > > go > >>> > > > > > > > > >> over it > >>> > > > > > > > > >> >> > in > >>> > > > > > > > > >> >> > > > > > > tomorrow's > >>> > > > > > > > > >> >> > > > > > > > meeting. > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > Also, can you forward the invite to > >>> me? > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > Thanks > >>> > > > > > > > > >> >> > > > > > > > Parth > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, > Jun > >>> > Rao < > >>> > > > > > > > > >> j...@confluent.io> > >>> > > > > > > > > >> >> > > > wrote: > >>> > > > > > > > > >> >> > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few > comments. > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 100. This potentially can be > useful > >>> for > >>> > > > Kafka > >>> > > > > > > > Connect > >>> > > > > > > > > >> and > >>> > > > > > > > > >> >> > Kafka > >>> > > > > > > > > >> >> > > > > rest > >>> > > > > > > > > >> >> > > > > > > > proxy > >>> > > > > > > > > >> >> > > > > > > > > where a worker agent will need to > >>> run a > >>> > > > task > >>> > > > > on > >>> > > > > > > > > behalf > >>> > > > > > > > > >> of a > >>> > > > > > > > > >> >> > > > client. > >>> > > > > > > > > >> >> > > > > > We > >>> > > > > > > > > >> >> > > > > > > > will > >>> > > > > > > > > >> >> > > > > > > > > likely need to change how those > >>> > services > >>> > > > use > >>> > > > > > > Kafka > >>> > > > > > > > > >> clients > >>> > > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a > >>> > shared > >>> > > > > client > >>> > > > > > > per > >>> > > > > > > > > >> worker, > >>> > > > > > > > > >> >> > we > >>> > > > > > > > > >> >> > > > will > >>> > > > > > > > > >> >> > > > > > > need > >>> > > > > > > > > >> >> > > > > > > > a > >>> > > > > > > > > >> >> > > > > > > > > client per user task since the > >>> > > > authentication > >>> > > > > > > > happens > >>> > > > > > > > > >> at the > >>> > > > > > > > > >> >> > > > > > connection > >>> > > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the > >>> renewer > >>> > > will > >>> > > > be > >>> > > > > > the > >>> > > > > > > > > >> workers. > >>> > > > > > > > > >> >> > So, > >>> > > > > > > > > >> >> > > we > >>> > > > > > > > > >> >> > > > > > > > probably > >>> > > > > > > > > >> >> > > > > > > > > need to allow multiple renewers. > For > >>> > > Kafka > >>> > > > > rest > >>> > > > > > > > > proxy, > >>> > > > > > > > > >> the > >>> > > > > > > > > >> >> > > > renewer > >>> > > > > > > > > >> >> > > > > > can > >>> > > > > > > > > >> >> > > > > > > > > probably just be the creator of > the > >>> > > token. > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who > can > >>> > > request > >>> > > > > > > > delegation > >>> > > > > > > > > >> tokens? > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to > send > >>> > > > > delegation > >>> > > > > > > > tokens > >>> > > > > > > > > >> in an > >>> > > > > > > > > >> >> > > > > encrypted > >>> > > > > > > > > >> >> > > > > > > > > channel? > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for > expiring > >>> > > > tokens, > >>> > > > > > > every > >>> > > > > > > > > >> broker? > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens, > would > >>> it > >>> > be > >>> > > > > > better > >>> > > > > > > to > >>> > > > > > > > > do > >>> > > > > > > > > >> it in > >>> > > > > > > > > >> >> > a > >>> > > > > > > > > >> >> > > > > > request > >>> > > > > > > > > >> >> > > > > > > > > instead of going to ZK directly? > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 105. The terminology of client in > >>> the > >>> > > wiki > >>> > > > > > > > sometimes > >>> > > > > > > > > >> refers > >>> > > > > > > > > >> >> > to > >>> > > > > > > > > >> >> > > > the > >>> > > > > > > > > >> >> > > > > > end > >>> > > > > > > > > >> >> > > > > > > > > client and some other times > refers > >>> to > >>> > the > >>> > > > > > client > >>> > > > > > > > > using > >>> > > > > > > > > >> the > >>> > > > > > > > > >> >> > > > > delegation > >>> > > > > > > > > >> >> > > > > > > > > tokens. It would be useful to > >>> > distinguish > >>> > > > > > between > >>> > > > > > > > the > >>> > > > > > > > > >> two. > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > 106. Could you explain the > sentence > >>> > > "Broker > >>> > > > > > also > >>> > > > > > > > > >> stores the > >>> > > > > > > > > >> >> > > > > > > > DelegationToken > >>> > > > > > > > > >> >> > > > > > > > > without the hmac in the > zookeeper." > >>> a > >>> > bit > >>> > > > > > more? I > >>> > > > > > > > > >> thought the > >>> > > > > > > > > >> >> > > > > > > delegation > >>> > > > > > > > > >> >> > > > > > > > > token is the hmac. > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > Thanks, > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > Jun > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, > Jun > >>> > Rao > >>> > > < > >>> > > > > > > > > >> j...@confluent.io> > >>> > > > > > > > > >> >> > > > wrote: > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > Hi, Harsha, > >>> > > > > > > > > >> >> > > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting > >>> invite. > >>> > We > >>> > > > can > >>> > > > > > > > discuss > >>> > > > > > > > > >> this in > >>> > > > > > > > > >> >> > > the > >>> > > > > > > > > >> >> > > > > > > meeting > >>> > > > > > > > > >> >> > > > > > > > > > tomorrow. > >>> > > > > > > > > >> >> > > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > Thanks, > >>> > > > > > > > > >> >> > > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > Jun > >>> > > > > > > > > >> >> > > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 > AM, > >>> > > Harsha < > >>> > > > > > > > > >> ka...@harsha.io> > >>> > > > > > > > > >> >> > > > wrote: > >>> > > > > > > > > >> >> > > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> Hi All, > >>> > > > > > > > > >> >> > > > > > > > > >> Can we have a KIP > >>> meeting > >>> > > > > around > >>> > > > > > > > this. > >>> > > > > > > > > >> The KIP > >>> > > > > > > > > >> >> > is > >>> > > > > > > > > >> >> > > > up > >>> > > > > > > > > >> >> > > > > > for > >>> > > > > > > > > >> >> > > > > > > > > >> sometime and if > there > >>> are > >>> > > any > >>> > > > > > > > questions > >>> > > > > > > > > >> lets > >>> > > > > > > > > >> >> > > > quickly > >>> > > > > > > > > >> >> > > > > > hash > >>> > > > > > > > > >> >> > > > > > > > out > >>> > > > > > > > > >> >> > > > > > > > > >> details. > >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> Thanks, > >>> > > > > > > > > >> >> > > > > > > > > >> Harsha > >>> > > > > > > > > >> >> > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 > >>> AM, > >>> > > parth > >>> > > > > > > > > brahmbhatt > >>> > > > > > > > > >> wrote: > >>> > > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo > >>> > system > >>> > > > uses > >>> > > > > > so > >>> > > > > > > no > >>> > > > > > > > > >> good > >>> > > > > > > > > >> >> > reason > >>> > > > > > > > > >> >> > > > > > really. > >>> > > > > > > > > >> >> > > > > > > > We > >>> > > > > > > > > >> >> > > > > > > > > >> > could > >>> > > > > > > > > >> >> > > > > > > > > >> > change it to whatever is the > >>> > newest > >>> > > > > > > > recommended > >>> > > > > > > > > >> standard > >>> > > > > > > > > >> >> > > is. > >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >> > Thanks > >>> > > > > > > > > >> >> > > > > > > > > >> > Parth > >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 > >>> AM, > >>> > > > Ismael > >>> > > > > > > Juma < > >>> > > > > > > > > >> >> > > > > ism...@juma.me.uk > >>> > > > > > > > > >> >> > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > wrote: > >>> > > > > > > > > >> >> > > > > > > > > >> > > >>> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth, > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only > >>> > started > >>> > > > > > > reviewing > >>> > > > > > > > > >> this and > >>> > > > > > > > > >> >> > > may > >>> > > > > > > > > >> >> > > > > have > >>> > > > > > > > > >> >> > > > > > > > > >> additional > >>> > > > > > > > > >> >> > > > > > > > > >> > > questions later. The > >>> immediate > >>> > > > > question > >>> > > > > > > that > >>> > > > > > > > > >> came to > >>> > > > > > > > > >> >> > > mind > >>> > > > > > > > > >> >> > > > is > >>> > > > > > > > > >> >> > > > > > our > >>> > > > > > > > > >> >> > > > > > > > > >> choice of > >>> > > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though > it's > >>> > > marked > >>> > > > > as > >>> > > > > > > > > >> OBSOLETE in > >>> > > > > > > > > >> >> > the > >>> > > > > > > > > >> >> > > > IANA > >>> > > > > > > > > >> >> > > > > > > > > Registry > >>> > > > > > > > > >> >> > > > > > > > > >> of > >>> > > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the > >>> original > >>> > > RFC > >>> > > > > > > (2831) > >>> > > > > > > > > has > >>> > > > > > > > > >> been > >>> > > > > > > > > >> >> > > moved > >>> > > > > > > > > >> >> > > > > to > >>> > > > > > > > > >> >> > > > > > > > > Historic > >>> > > > > > > > > >> >> > > > > > > > > >> > > status: > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > https://tools.ietf.org/html/ > >>> > > rfc6331 > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> > >>> > > > > > > > >>> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl- > >>> mechanisms.xhtml > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning > behind > >>> > that > >>> > > > > > choice? > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > Thanks, > >>> > > > > > > > > >> >> > > > > > > > > >> > > Ismael > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at > 11:29 > >>> > PM, > >>> > > > Gwen > >>> > > > > > > > > Shapira < > >>> > > > > > > > > >> >> > > > > > > g...@confluent.io > >>> > > > > > > > > >> >> > > > > > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> wrote: > >>> > > > > > > > > >> >> > > > > > > > > >> > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :) > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize > >>> that > >>> > > even > >>> > > > > > though > >>> > > > > > > > > >> delegation > >>> > > > > > > > > >> >> > > > tokens > >>> > > > > > > > > >> >> > > > > > > are a > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel > very > >>> > > strongly > >>> > > > > > about > >>> > > > > > > > not > >>> > > > > > > > > >> adding > >>> > > > > > > > > >> >> > > > > > dependency > >>> > > > > > > > > >> >> > > > > > > > on > >>> > > > > > > > > >> >> > > > > > > > > >> Hadoop > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > when implementing > >>> delegation > >>> > > > > tokens > >>> > > > > > > for > >>> > > > > > > > > >> Kafka. The > >>> > > > > > > > > >> >> > > KIP > >>> > > > > > > > > >> >> > > > > > > doesn't > >>> > > > > > > > > >> >> > > > > > > > > >> imply > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but > if > >>> you > >>> > > can > >>> > > > > > > > clarify... > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop > dependency.* > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to > the > >>> KIP > >>> > so > >>> > > > no > >>> > > > > > one > >>> > > > > > > > will > >>> > > > > > > > > >> read > >>> > > > > > > > > >> >> > the > >>> > > > > > > > > >> >> > > > KIP > >>> > > > > > > > > >> >> > > > > > and > >>> > > > > > > > > >> >> > > > > > > > > panic > >>> > > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the > next > >>> > > > > release... > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get > delegation > >>> > token > >>> > > at > >>> > > > > any > >>> > > > > > > > time > >>> > > > > > > > > >> after > >>> > > > > > > > > >> >> > > > > > > > authenticating? > >>> > > > > > > > > >> >> > > > > > > > > >> only > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > immediately after? > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are > >>> > > > authenticated > >>> > > > > > you > >>> > > > > > > > can > >>> > > > > > > > > >> get > >>> > > > > > > > > >> >> > > > delegation > >>> > > > > > > > > >> >> > > > > > > > tokens. > >>> > > > > > > > > >> >> > > > > > > > > >> We > >>> > > > > > > > > >> >> > > > > > > > > >> > > need > >>> > > > > > > > > >> >> > > > > > > > > >> > > > to > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client > >>> > > > authenticated > >>> > > > > > > using > >>> > > > > > > > > >> delegation > >>> > > > > > > > > >> >> > > > > token, > >>> > > > > > > > > >> >> > > > > > > can > >>> > > > > > > > > >> >> > > > > > > > > also > >>> > > > > > > > > >> >> > > > > > > > > >> > > > acquire > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation token > again or > >>> > not. > >>> > > > > Also > >>> > > > > > > > there > >>> > > > > > > > > >> is the > >>> > > > > > > > > >> >> > > > > question > >>> > > > > > > > > >> >> > > > > > of > >>> > > > > > > > > >> >> > > > > > > > do > >>> > > > > > > > > >> >> > > > > > > > > we > >>> > > > > > > > > >> >> > > > > > > > > >> > > allow > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire > >>> delegation > >>> > > > token > >>> > > > > > or > >>> > > > > > > we > >>> > > > > > > > > >> want > >>> > > > > > > > > >> >> > > specific > >>> > > > > > > > > >> >> > > > > > ACLs > >>> > > > > > > > > >> >> > > > > > > (I > >>> > > > > > > > > >> >> > > > > > > > > >> think > >>> > > > > > > > > >> >> > > > > > > > > >> > > its > >>> > > > > > > > > >> >> > > > > > > > > >> > > > an > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > overkill.)* > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an > >>> > > overkill. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > I think we are debating > two > >>> > > > options: > >>> > > > > > > > Either > >>> > > > > > > > > >> require > >>> > > > > > > > > >> >> > > > > Kerberos > >>> > > > > > > > > >> >> > > > > > > > auth > >>> > > > > > > > > >> >> > > > > > > > > >> for > >>> > > > > > > > > >> >> > > > > > > > > >> > > > renewal or require > >>> non-owners > >>> > to > >>> > > > > > renew. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is > >>> > simpler > >>> > > > (it > >>> > > > > > > > > basically > >>> > > > > > > > > >> >> > require > >>> > > > > > > > > >> >> > > a > >>> > > > > > > > > >> >> > > > > "job > >>> > > > > > > > > >> >> > > > > > > > > master" > >>> > > > > > > > > >> >> > > > > > > > > >> > > > to take responsibility > for > >>> the > >>> > > > > > renewal, > >>> > > > > > > it > >>> > > > > > > > > >> will have > >>> > > > > > > > > >> >> > > its > >>> > > > > > > > > >> >> > > > > own > >>> > > > > > > > > >> >> > > > > > > > > >> identity > >>> > > > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this > is > >>> the > >>> > > > > correct > >>> > > > > > > > > design > >>> > > > > > > > > >> >> > pattern > >>> > > > > > > > > >> >> > > > > > anyway. > >>> > > > > > > > > >> >> > > > > > > > For > >>> > > > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect > Nimbus to > >>> > > > > coordinate > >>> > > > > > > > > >> renewals?), > >>> > > > > > > > > >> >> > but > >>> > > > > > > > > >> >> > > > it > >>> > > > > > > > > >> >> > > > > is > >>> > > > > > > > > >> >> > > > > > > > hard > >>> > > > > > > > > >> >> > > > > > > > > to > >>> > > > > > > > > >> >> > > > > > > > > >> > > > debate simplicity > without > >>> > > looking > >>> > > > at > >>> > > > > > the > >>> > > > > > > > > code > >>> > > > > > > > > >> >> > changes > >>> > > > > > > > > >> >> > > > > > > required. > >>> > > > > > > > > >> >> > > > > > > > If > >>> > > > > > > > > >> >> > > > > > > > > >> you > >>> > > > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the > >>> > "require > >>> > > > > > > Kerberos" > >>> > > > > > > > > >> will look > >>> > > > > > > > > >> >> > > in > >>> > > > > > > > > >> >> > > > > > Kafka > >>> > > > > > > > > >> >> > > > > > > > > code, > >>> > > > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a > >>> look. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is > >>> that > >>> > > > tokens > >>> > > > > > will > >>> > > > > > > > > >> propagate > >>> > > > > > > > > >> >> > via > >>> > > > > > > > > >> >> > > > ZK > >>> > > > > > > > > >> >> > > > > > but > >>> > > > > > > > > >> >> > > > > > > > > >> without > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > additional changes to > >>> > > > > UpdateMetadata > >>> > > > > > > > > >> protocol, > >>> > > > > > > > > >> >> > > > correct? > >>> > > > > > > > > >> >> > > > > > > > Clients > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry > on > >>> > SASL > >>> > > > auth > >>> > > > > > > > failure > >>> > > > > > > > > >> (IIRC), > >>> > > > > > > > > >> >> > > but > >>> > > > > > > > > >> >> > > > > > since > >>> > > > > > > > > >> >> > > > > > > > the > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate > between > >>> > > brokers > >>> > > > > > > asynch, > >>> > > > > > > > > we > >>> > > > > > > > > >> will > >>> > > > > > > > > >> >> > > need > >>> > > > > > > > > >> >> > > > to > >>> > > > > > > > > >> >> > > > > > > > retry a > >>> > > > > > > > > >> >> > > > > > > > > >> bit > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients > failing > >>> > auth > >>> > > > due > >>> > > > > to > >>> > > > > > > > > timing > >>> > > > > > > > > >> >> > issues. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2 > >>> > > alternatives > >>> > > > > > right > >>> > > > > > > > > now. > >>> > > > > > > > > >> The > >>> > > > > > > > > >> >> > > > current > >>> > > > > > > > > >> >> > > > > > > > > documented > >>> > > > > > > > > >> >> > > > > > > > > >> > > > approach > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based > and it > >>> > does > >>> > > > not > >>> > > > > > > > require > >>> > > > > > > > > >> any > >>> > > > > > > > > >> >> > > changes > >>> > > > > > > > > >> >> > > > > to > >>> > > > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > protocol. An > alternative > >>> > > > approach > >>> > > > > > can > >>> > > > > > > > > remove > >>> > > > > > > > > >> >> > > zookeeper > >>> > > > > > > > > >> >> > > > > > > > > dependency > >>> > > > > > > > > >> >> > > > > > > > > >> as > >>> > > > > > > > > >> >> > > > > > > > > >> > > well > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss > that > >>> in > >>> > KIP > >>> > > > > > > > discussion > >>> > > > > > > > > >> call.* > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds > interesting. > >>> Do > >>> > you > >>> > > > > want > >>> > > > > > to > >>> > > > > > > > > ping > >>> > > > > > > > > >> Jun to > >>> > > > > > > > > >> >> > > > > > arrange a > >>> > > > > > > > > >> >> > > > > > > > > call? > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's > >>> > suggestion > >>> > > of > >>> > > > > > > having > >>> > > > > > > > > >> just the > >>> > > > > > > > > >> >> > > > > > controller > >>> > > > > > > > > >> >> > > > > > > > > issue > >>> > > > > > > > > >> >> > > > > > > > > >> the > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to > >>> avoid > >>> > > > > syncing > >>> > > > > > a > >>> > > > > > > > > shared > >>> > > > > > > > > >> >> > secret. > >>> > > > > > > > > >> >> > > > Not > >>> > > > > > > > > >> >> > > > > > > sure > >>> > > > > > > > > >> >> > > > > > > > if > >>> > > > > > > > > >> >> > > > > > > > > >> we > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > want to continue the > >>> > > discussion > >>> > > > > here > >>> > > > > > > or > >>> > > > > > > > on > >>> > > > > > > > > >> the > >>> > > > > > > > > >> >> > > wiki. I > >>> > > > > > > > > >> >> > > > > > think > >>> > > > > > > > > >> >> > > > > > > > > that > >>> > > > > > > > > >> >> > > > > > > > > >> we > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > can decouple the > problem > >>> of > >>> > > > "token > >>> > > > > > > > > >> distribution" > >>> > > > > > > > > >> >> > > from > >>> > > > > > > > > >> >> > > > > > > "shared > >>> > > > > > > > > >> >> > > > > > > > > >> secret > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use > the > >>> > > > > controller > >>> > > > > > > as > >>> > > > > > > > > the > >>> > > > > > > > > >> only > >>> > > > > > > > > >> >> > > token > >>> > > > > > > > > >> >> > > > > > > > generator > >>> > > > > > > > > >> >> > > > > > > > > >> to > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > solve the second > issue, > >>> > while > >>> > > > > still > >>> > > > > > > > using > >>> > > > > > > > > >> ZK async > >>> > > > > > > > > >> >> > > to > >>> > > > > > > > > >> >> > > > > > > > distribute > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > tokens. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the > >>> > previous > >>> > > > > Email > >>> > > > > > I > >>> > > > > > > am > >>> > > > > > > > > >> fine with > >>> > > > > > > > > >> >> > > > that > >>> > > > > > > > > >> >> > > > > > > > approach > >>> > > > > > > > > >> >> > > > > > > > > >> as > >>> > > > > > > > > >> >> > > > > > > > > >> > > long > >>> > > > > > > > > >> >> > > > > > > > > >> > > > as > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > we agree that the > extra > >>> > > > complexity > >>> > > > > > of > >>> > > > > > > > > >> >> > > adding/updating > >>> > > > > > > > > >> >> > > > > APIS > >>> > > > > > > > > >> >> > > > > > > > adds > >>> > > > > > > > > >> >> > > > > > > > > >> enough > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage > with > >>> > the > >>> > > > > > > controller > >>> > > > > > > > > >> approach > >>> > > > > > > > > >> >> > is > >>> > > > > > > > > >> >> > > > > > secret > >>> > > > > > > > > >> >> > > > > > > > > >> rotation > >>> > > > > > > > > >> >> > > > > > > > > >> > > can > >>> > > > > > > > > >> >> > > > > > > > > >> > > > be > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and > >>> would > >>> > > not > >>> > > > > > > require > >>> > > > > > > > > >> >> > > deployment. * > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra > >>> > > > complexity > >>> > > > > > (or > >>> > > > > > > > > point > >>> > > > > > > > > >> me to > >>> > > > > > > > > >> >> > > the > >>> > > > > > > > > >> >> > > > > > email > >>> > > > > > > > > >> >> > > > > > > I > >>> > > > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis > are > >>> > > > required? > >>> > > > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell, > >>> clients > >>> > > can > >>> > > > > > > already > >>> > > > > > > > > >> find the > >>> > > > > > > > > >> >> > > > > > controller > >>> > > > > > > > > >> >> > > > > > > > from > >>> > > > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more > >>> > > concerned > >>> > > > > > about > >>> > > > > > > > > >> controller > >>> > > > > > > > > >> >> > > > load. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > * While I like the > idea > >>> of > >>> > > > forcing > >>> > > > > > > > > kerberos > >>> > > > > > > > > >> auth > >>> > > > > > > > > >> >> > for > >>> > > > > > > > > >> >> > > > > > > renwal, I > >>> > > > > > > > > >> >> > > > > > > > > >> think > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport > >>> layer > >>> > > the > >>> > > > > the > >>> > > > > > > > > request > >>> > > > > > > > > >> >> > content > >>> > > > > > > > > >> >> > > > in > >>> > > > > > > > > >> >> > > > > a > >>> > > > > > > > > >> >> > > > > > > > pretty > >>> > > > > > > > > >> >> > > > > > > > > >> ugly > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting > >>> > renewer > >>> > > to > >>> > > > > > > > non-owner > >>> > > > > > > > > >> is > >>> > > > > > > > > >> >> > > better. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a > >>> necessary > >>> > > > evil. > >>> > > > > > > While > >>> > > > > > > > > >> this will > >>> > > > > > > > > >> >> > > make > >>> > > > > > > > > >> >> > > > > the > >>> > > > > > > > > >> >> > > > > > > > kafka > >>> > > > > > > > > >> >> > > > > > > > > >> code > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > pretty straight > forward , > >>> > > > forcing > >>> > > > > > > > renewer > >>> > > > > > > > > >> to > >>> > > > > > > > > >> >> > > > non-owner > >>> > > > > > > > > >> >> > > > > > > pushes > >>> > > > > > > > > >> >> > > > > > > > > >> the code > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and > >>> makes > >>> > > it > >>> > > > > even > >>> > > > > > > > > harder > >>> > > > > > > > > >> to > >>> > > > > > > > > >> >> > > > > > integrate. * > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I > >>> don't > >>> > > think > >>> > > > > the > >>> > > > > > > > > >> "renewal by > >>> > > > > > > > > >> >> > > > other" > >>> > > > > > > > > >> >> > > > > > > > approach > >>> > > > > > > > > >> >> > > > > > > > > >> is > >>> > > > > > > > > >> >> > > > > > > > > >> > > > that ugly for the > clients > >>> we > >>> > > > expect > >>> > > > > to > >>> > > > > > > use > >>> > > > > > > > > >> >> > delegation > >>> > > > > > > > > >> >> > > > > tokens > >>> > > > > > > > > >> >> > > > > > > > since > >>> > > > > > > > > >> >> > > > > > > > > >> > > > they will have an > >>> app-master > >>> > of > >>> > > > some > >>> > > > > > > sort > >>> > > > > > > > > who > >>> > > > > > > > > >> >> > > requested > >>> > > > > > > > > >> >> > > > > the > >>> > > > > > > > > >> >> > > > > > > > token > >>> > > > > > > > > >> >> > > > > > > > > to > >>> > > > > > > > > >> >> > > > > > > > > >> > > > begin with. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > The response for my > >>> question > >>> > > on > >>> > > > > how > >>> > > > > > > > > multiple > >>> > > > > > > > > >> >> > > > identities > >>> > > > > > > > > >> >> > > > > > will > >>> > > > > > > > > >> >> > > > > > > > be > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super > >>> clear > >>> > to > >>> > > > me - > >>> > > > > > > > AFAIK, > >>> > > > > > > > > >> we don't > >>> > > > > > > > > >> >> > > > > > > > authenticate > >>> > > > > > > > > >> >> > > > > > > > > >> each > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > request, we > authenticate > >>> > > > > > connections. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate > >>> > connections, > >>> > > > and > >>> > > > > > only > >>> > > > > > > > > when > >>> > > > > > > > > >> they > >>> > > > > > > > > >> >> > are > >>> > > > > > > > > >> >> > > > > being > >>> > > > > > > > > >> >> > > > > > > > > >> established. > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Let > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this > as > >>> a > >>> > > > > question, > >>> > > > > > > in > >>> > > > > > > > > >> absence of > >>> > > > > > > > > >> >> > > > > > > delegation > >>> > > > > > > > > >> >> > > > > > > > > >> tokens if > >>> > > > > > > > > >> >> > > > > > > > > >> > > > we > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > had to support the use > >>> case > >>> > > > using > >>> > > > > > user > >>> > > > > > > > > >> TGT's how > >>> > > > > > > > > >> >> > > would > >>> > > > > > > > > >> >> > > > > we > >>> > > > > > > > > >> >> > > > > > do > >>> > > > > > > > > >> >> > > > > > > > it? > >>> > > > > > > > > >> >> > > > > > > > > >> My > >>> > > > > > > > > >> >> > > > > > > > > >> > > point > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > was it would be no > >>> different > >>> > > > with > >>> > > > > > > > > delegation > >>> > > > > > > > > >> >> > tokens. > >>> > > > > > > > > >> >> > > > The > >>> > > > > > > > > >> >> > > > > > use > >>> > > > > > > > > >> >> > > > > > > > > case > >>> > > > > > > > > >> >> > > > > > > > > >> you > >>> > > > > > > > > >> >> > > > > > > > > >> > > are > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > describing seems more > >>> like > >>> > > > > > > > impersonation.* > >>> > > > > > > > > >> >> > > > > > > > > >> > > > > >>> > > > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that > one of > >>> > the > >>> > > > > things > >>> > > > > > > > that > >>> > > > > > > > > >> >> > del > >>> > >> > >> > >> > >> -- > >> > >> Regards, > >> Ashish > >> > > > > -- > Gwen Shapira > Product Manager | Confluent > 650.450.2760 | @gwenshap > Follow us: Twitter | blog >
