Hello Harsha, Are you still working on this? Wondering if we can discuss this in next KIP meeting, if you can join.
On Mon, Jul 18, 2016 at 9:51 AM, Harsha Chintalapani <ka...@harsha.io> wrote: > Hi Grant, > We are working on it. Will add the details to KIP about the > request protocol. > > Thanks, > Harsha > > On Mon, Jul 18, 2016 at 6:50 AM Grant Henke <ghe...@cloudera.com> wrote: > > > Hi Parth, > > > > Are you still working on this? If you need any help please don't hesitate > > to ask. > > > > Thanks, > > Grant > > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun Rao <j...@confluent.io> wrote: > > > > > Parth, > > > > > > Thanks for the reply. > > > > > > It makes sense to only allow the renewal by users that authenticated > > using > > > *non* delegation token mechanism. Then, should we make the renewal a > > list? > > > For example, in the case of rest proxy, it will be useful for every > > > instance of rest proxy to be able to renew the tokens. > > > > > > It would be clearer if we can document the request protocol like > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 4+-+Command+line+and+centralized+administrative+operations#KIP-4- > Commandlineandcentralizedadministrativeoperations- > CreateTopicsRequest(KAFKA-2945):(VotedandPlannedforin0.10.1.0) > > > . > > > > > > It would also be useful to document the client APIs. > > > > > > Thanks, > > > > > > Jun > > > > > > On Tue, Jun 28, 2016 at 2:55 PM, parth brahmbhatt < > > > brahmbhatt.pa...@gmail.com> wrote: > > > > > > > Hi, > > > > > > > > I am suggesting that we will only allow the renewal by users that > > > > authenticated using *non* delegation token mechanism. For example, If > > > user > > > > Alice authenticated using kerberos and requested delegation tokens, > > only > > > > user Alice authenticated via non delegation token mechanism can > renew. > > > > Clients that have access to delegation tokens can not issue renewal > > > > request for renewing their own token and this is primarily important > to > > > > reduce the time window for which a compromised token will be valid. > > > > > > > > To clarify, Yes any authenticated user can request delegation tokens > > but > > > > even here I would recommend to avoid creating a chain where a client > > > > authenticated via delegation token request for more delegation > tokens. > > > > Basically anyone can request delegation token, as long as they > > > authenticate > > > > via a non delegation token mechanism. > > > > > > > > Aren't classes listed here > > > > < > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKaf > ka-PublicInterfaces > > > > > > > > > sufficient? > > > > > > > > Thanks > > > > Parth > > > > > > > > > > > > > > > > On Tue, Jun 21, 2016 at 4:33 PM, Jun Rao <j...@confluent.io> wrote: > > > > > > > > > Parth, > > > > > > > > > > Thanks for the reply. A couple of comments inline below. > > > > > > > > > > On Tue, Jun 21, 2016 at 10:36 AM, parth brahmbhatt < > > > > > brahmbhatt.pa...@gmail.com> wrote: > > > > > > > > > > > 1. Who / how are tokens renewed? By original requester only? or > > using > > > > > > Kerberos > > > > > > auth only? > > > > > > My recommendation is to do this only using Kerberos auth and only > > > threw > > > > > the > > > > > > renewer specified during the acquisition request. > > > > > > > > > > > > > > > > > Hmm, not sure that I follow this. Are you saying that any client > > > > > authenticated with the delegation token can renew, i.e. there is no > > > > renewer > > > > > needed? > > > > > > > > > > Also, just to be clear, any authenticated client (either through > SASL > > > or > > > > > SSL) can request a delegation token for the authenticated user, > > right? > > > > > > > > > > > > > > > > 2. Are tokens stored on each broker or in ZK? > > > > > > My recommendation is still to store in ZK or not store them at > all. > > > The > > > > > > whole controller based distribution is too much overhead with not > > > much > > > > to > > > > > > achieve. > > > > > > > > > > > > 3. How are tokens invalidated / expired? > > > > > > Either by expiration time out or through an explicit request to > > > > > invalidate. > > > > > > > > > > > > 4. Which encryption algorithm is used? > > > > > > SCRAM > > > > > > > > > > > > 5. What is the impersonation proposal (it wasn't in the KIP but > was > > > > > > discussed > > > > > > in this thread)? > > > > > > There is no imperonation proposal. I tried and explained how its > a > > > > > > different problem and why its not really necessary to discuss > that > > as > > > > > part > > > > > > of this KIP. This KIP will not support any impersonation, it > will > > > just > > > > > be > > > > > > another way to authenticate. > > > > > > > > > > > > 6. Do we need new ACLs, if so - for what actions? > > > > > > We do not need new ACLs. > > > > > > > > > > > > > > > > > Could we document the format of the new request/response and their > > > > > associated Resource and Operation for ACL? > > > > > > > > > > > > > > > > 7. How would the delegation token be configured in the client? > > > > > > Should be through config. I wasn't planning on supporting JAAS > for > > > > > tokens. > > > > > > I don't believe hadoop does this either. > > > > > > > > > > > > Thanks > > > > > > Parth > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <j...@confluent.io> > wrote: > > > > > > > > > > > > > Harsha, > > > > > > > > > > > > > > Another question. > > > > > > > > > > > > > > 9. How would the delegation token be configured in the client? > > The > > > > > > standard > > > > > > > way is to do this through JAAS. However, we will need to think > > > > through > > > > > if > > > > > > > this is convenient in a shared environment. For example, when a > > new > > > > > task > > > > > > is > > > > > > > added to a Storm worker node, do we need to dynamically add a > new > > > > > section > > > > > > > in the JAAS file? It may be more convenient if we can pass in > the > > > > token > > > > > > > through the config directly w/o going through JAAS. > > > > > > > > > > > > > > Are you or Parth still actively working on this KIP? > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <j...@confluent.io> > > wrote: > > > > > > > > > > > > > > > Just to add on that list. > > > > > > > > > > > > > > > > 2. It would be good to document the format of the data stored > > in > > > > ZK. > > > > > > > > 7. Earlier, there was a discussion on whether the tokens > should > > > be > > > > > > > > propagated through ZK like config/acl/quota, or through the > > > > > controller. > > > > > > > > Currently, the controller is only designed for propagating > > topic > > > > > > > metadata, > > > > > > > > but not other data. > > > > > > > > 8. Should we use SCRAM to send the token instead of > DIGEST-MD5 > > > > since > > > > > > it's > > > > > > > > deprecated? > > > > > > > > > > > > > > > > Also, the images in the wiki seem broken. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira < > > > g...@confluent.io> > > > > > > > wrote: > > > > > > > > > > > > > > > >> From what I can see, remaining questions are: > > > > > > > >> > > > > > > > >> 1. Who / how are tokens renewed? By original requester only? > > or > > > > > using > > > > > > > >> Kerberos auth only? > > > > > > > >> 2. Are tokens stored on each broker or in ZK? > > > > > > > >> 3. How are tokens invalidated / expired? > > > > > > > >> 4. Which encryption algorithm is used? > > > > > > > >> 5. What is the impersonation proposal (it wasn't in the KIP > > but > > > > was > > > > > > > >> discussed in this thread)? > > > > > > > >> 6. Do we need new ACLs, if so - for what actions? > > > > > > > >> > > > > > > > >> Gwen > > > > > > > >> > > > > > > > >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> > > wrote: > > > > > > > >> > Jun & Ismael, > > > > > > > >> > Unfortunately I couldn't attend > the > > > KIP > > > > > > > meeting > > > > > > > >> > when delegation tokens discussed. > > > > > > Appreciate > > > > > > > if > > > > > > > >> > you can update the thread if you > > have > > > > any > > > > > > > >> > further questions. > > > > > > > >> > Thanks, > > > > > > > >> > Harsha > > > > > > > >> > > > > > > > > >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote: > > > > > > > >> >> It seems that the links to images in the KIP are broken. > > > > > > > >> >> > > > > > > > >> >> Liquan > > > > > > > >> >> > > > > > > > >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt < > > > > > > > >> >> brahmbhatt.pa...@gmail.com> wrote: > > > > > > > >> >> > > > > > > > >> >> > 110. What does getDelegationTokenAs mean? > > > > > > > >> >> > In the current proposal we only allow a user to get > > > > delegation > > > > > > > token > > > > > > > >> for > > > > > > > >> >> > the identity that it authenticated as using another > > > > mechanism, > > > > > > i.e. > > > > > > > >> A user > > > > > > > >> >> > that authenticate using a keytab for principal > > > > > us...@example.com > > > > > > > >> will get > > > > > > > >> >> > delegation tokens for that user only. In future I think > > we > > > > will > > > > > > > have > > > > > > > >> to > > > > > > > >> >> > extend support such that we allow some set of users ( > > > > > > > >> >> > kafka-rest-u...@example.com, storm-nim...@example.com) > > to > > > > > > acquire > > > > > > > >> >> > delegation tokens on behalf of other users whose > identity > > > > they > > > > > > have > > > > > > > >> >> > verified independently. Kafka brokers will have ACLs > to > > > > > control > > > > > > > >> which > > > > > > > >> >> > users are allowed to impersonate other users and get > > tokens > > > > on > > > > > > > >> behalf of > > > > > > > >> >> > them. Overall Impersonation is a whole different > problem > > in > > > > my > > > > > > > >> opinion and > > > > > > > >> >> > I think we can tackle it in separate KIP. > > > > > > > >> >> > > > > > > > > >> >> > 111. What's the typical rate of getting and renewing > > > > delegation > > > > > > > >> tokens? > > > > > > > >> >> > Typically this should be very very low, 1 request per > > > minute > > > > > is a > > > > > > > >> >> > relatively high estimate. However it depends on the > token > > > > > > > >> expiration. I am > > > > > > > >> >> > less worried about the extra load it puts on controller > > vs > > > > the > > > > > > > added > > > > > > > >> >> > complexity and the value it offers. > > > > > > > >> >> > > > > > > > > >> >> > Thanks > > > > > > > >> >> > Parth > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > > >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma < > > > > > ism...@juma.me.uk> > > > > > > > >> wrote: > > > > > > > >> >> > > > > > > > > >> >> > > Thanks Rajini. It would probably require a separate > KIP > > > as > > > > it > > > > > > > will > > > > > > > >> >> > > introduce user visible changes. We could also update > > > KIP-48 > > > > > to > > > > > > > >> have this > > > > > > > >> >> > > information, but it seems cleaner to do it > separately. > > We > > > > can > > > > > > > >> discuss > > > > > > > >> >> > that > > > > > > > >> >> > > in the KIP call today. > > > > > > > >> >> > > > > > > > > > >> >> > > Ismael > > > > > > > >> >> > > > > > > > > > >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram < > > > > > > > >> >> > > rajinisiva...@googlemail.com> wrote: > > > > > > > >> >> > > > > > > > > > >> >> > > > Ismael, > > > > > > > >> >> > > > > > > > > > > >> >> > > > I have created a JIRA ( > > > > > > > >> >> > https://issues.apache.org/jira/browse/KAFKA-3751) > > > > > > > >> >> > > > for adding SCRAM as a SASL mechanism. Would that > need > > > > > another > > > > > > > >> KIP? If > > > > > > > >> >> > > > KIP-48 will use this mechanism, can this just be a > > JIRA > > > > > that > > > > > > > gets > > > > > > > >> >> > > reviewed > > > > > > > >> >> > > > when the PR is ready? > > > > > > > >> >> > > > > > > > > > > >> >> > > > Thank you, > > > > > > > >> >> > > > > > > > > > > >> >> > > > Rajini > > > > > > > >> >> > > > > > > > > > > >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma < > > > > > > > ism...@juma.me.uk> > > > > > > > >> >> > wrote: > > > > > > > >> >> > > > > > > > > > > >> >> > > > > Thanks Rajini, SCRAM seems like a good candidate. > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > Gwen had independently mentioned this as a SASL > > > > mechanism > > > > > > > that > > > > > > > >> might > > > > > > > >> >> > be > > > > > > > >> >> > > > > useful for Kafka and I have been meaning to check > > it > > > in > > > > > > more > > > > > > > >> detail. > > > > > > > >> >> > > Good > > > > > > > >> >> > > > > to know that you are willing to contribute an > > > > > > implementation. > > > > > > > >> Maybe > > > > > > > >> >> > we > > > > > > > >> >> > > > > should file a separate JIRA for this? > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > Ismael > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram < > > > > > > > >> >> > > > > rajinisiva...@googlemail.com> wrote: > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > SCRAM (Salted Challenge Response Authentication > > > > > > Mechanism) > > > > > > > >> is a > > > > > > > >> >> > > better > > > > > > > >> >> > > > > > mechanism than Digest-MD5. Java doesn't come > > with a > > > > > > > built-in > > > > > > > >> SCRAM > > > > > > > >> >> > > > > > SaslServer or SaslClient, but I will be happy > to > > > add > > > > > > > support > > > > > > > >> in > > > > > > > >> >> > Kafka > > > > > > > >> >> > > > > since > > > > > > > >> >> > > > > > it would be a useful mechanism to support > anyway. > > > > > > > >> >> > > > > > https://tools.ietf.org/html/rfc7677 describes > > the > > > > > > protocol > > > > > > > >> for > > > > > > > >> >> > > > > > SCRAM-SHA-256. > > > > > > > >> >> > > > > > > > > > > > > >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao < > > > > > > j...@confluent.io > > > > > > > > > > > > > > > >> wrote: > > > > > > > >> >> > > > > > > > > > > > > >> >> > > > > > > Parth, > > > > > > > >> >> > > > > > > > > > > > > > >> >> > > > > > > Thanks for the explanation. A couple of more > > > > > questions. > > > > > > > >> >> > > > > > > > > > > > > > >> >> > > > > > > 110. What does getDelegationTokenAs mean? > > > > > > > >> >> > > > > > > > > > > > > > >> >> > > > > > > 111. What's the typical rate of getting and > > > > renewing > > > > > > > >> delegation > > > > > > > >> >> > > > tokens? > > > > > > > >> >> > > > > > > That may have an impact on whether they > should > > be > > > > > > > directed > > > > > > > >> to the > > > > > > > >> >> > > > > > > controller. > > > > > > > >> >> > > > > > > > > > > > > > >> >> > > > > > > Jun > > > > > > > >> >> > > > > > > > > > > > > > >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth > > > brahmbhatt < > > > > > > > >> >> > > > > > > brahmbhatt.pa...@gmail.com> wrote: > > > > > > > >> >> > > > > > > > > > > > > > >> >> > > > > > > > Hi Jun, > > > > > > > >> >> > > > > > > > > > > > > > > >> >> > > > > > > > Thanks for reviewing. > > > > > > > >> >> > > > > > > > > > > > > > > >> >> > > > > > > > * We could add a Cluster action to add acls > > on > > > > who > > > > > > can > > > > > > > >> request > > > > > > > >> >> > > > > > delegation > > > > > > > >> >> > > > > > > > tokens. I don't see the use case for that > yet > > > but > > > > > > down > > > > > > > >> the line > > > > > > > >> >> > > > when > > > > > > > >> >> > > > > we > > > > > > > >> >> > > > > > > > start supporting getDelegationTokenAs it > will > > > be > > > > > > > >> necessary. > > > > > > > >> >> > > > > > > > * Yes we recommend tokens to be only > > > > > used/distributed > > > > > > > >> over > > > > > > > >> >> > secure > > > > > > > >> >> > > > > > > channels. > > > > > > > >> >> > > > > > > > * Depending on what design we end up > choosing > > > > > > > >> Invalidation will > > > > > > > >> >> > > be > > > > > > > >> >> > > > > > > > responsibility of every broker or > controller. > > > > > > > >> >> > > > > > > > * I am not sure if I documented somewhere > > that > > > > > > > >> invalidation > > > > > > > >> >> > will > > > > > > > >> >> > > > > > directly > > > > > > > >> >> > > > > > > > go through zookeeper but that is not the > > > intent. > > > > > > > >> Invalidation > > > > > > > >> >> > > will > > > > > > > >> >> > > > > > either > > > > > > > >> >> > > > > > > > be request based or due to expiration. No > > > direct > > > > > > > >> zookeeper > > > > > > > >> >> > > > > interaction > > > > > > > >> >> > > > > > > from > > > > > > > >> >> > > > > > > > any client. > > > > > > > >> >> > > > > > > > * "Broker also stores the DelegationToken > > > without > > > > > the > > > > > > > >> hmac in > > > > > > > >> >> > the > > > > > > > >> >> > > > > > > > zookeeper." : Sorry about the confusion. > The > > > sole > > > > > > > >> purpose of > > > > > > > >> >> > > > > zookeeper > > > > > > > >> >> > > > > > in > > > > > > > >> >> > > > > > > > this design is as distribution channel for > > > tokens > > > > > > > >> between all > > > > > > > >> >> > > > brokers > > > > > > > >> >> > > > > > > and a > > > > > > > >> >> > > > > > > > layer that ensures only tokens that were > > > > generated > > > > > by > > > > > > > >> making a > > > > > > > >> >> > > > > request > > > > > > > >> >> > > > > > > to a > > > > > > > >> >> > > > > > > > broker will be accepted (more on this in > > second > > > > > > > >> paragraph). The > > > > > > > >> >> > > > token > > > > > > > >> >> > > > > > > > consists of few elements (owner, renewer, > > uuid > > > , > > > > > > > >> expiration, > > > > > > > >> >> > > hmac) > > > > > > > >> >> > > > , > > > > > > > >> >> > > > > > one > > > > > > > >> >> > > > > > > of > > > > > > > >> >> > > > > > > > which is the finally generated hmac but > hmac > > it > > > > > self > > > > > > is > > > > > > > >> >> > derivable > > > > > > > >> >> > > > if > > > > > > > >> >> > > > > > you > > > > > > > >> >> > > > > > > > have all the other elements of the token + > > > secret > > > > > key > > > > > > > to > > > > > > > >> >> > generate > > > > > > > >> >> > > > > hmac. > > > > > > > >> >> > > > > > > > Given zookeeper does not provide SSL > support > > we > > > > do > > > > > > not > > > > > > > >> want the > > > > > > > >> >> > > > > entire > > > > > > > >> >> > > > > > > > token to be wire transferred to zookeeper > as > > > that > > > > > > will > > > > > > > >> be an > > > > > > > >> >> > > > insecure > > > > > > > >> >> > > > > > > wire > > > > > > > >> >> > > > > > > > transfer. Instead we only store all the > other > > > > > > elements > > > > > > > >> of a > > > > > > > >> >> > > > > delegation > > > > > > > >> >> > > > > > > > tokens. Brokers can read these elements and > > > > because > > > > > > > they > > > > > > > >> also > > > > > > > >> >> > > have > > > > > > > >> >> > > > > > access > > > > > > > >> >> > > > > > > > to secret key they will be able to generate > > > hmac > > > > on > > > > > > > >> their end. > > > > > > > >> >> > > > > > > > > > > > > > > >> >> > > > > > > > One of the alternative proposed is to avoid > > > > > zookeeper > > > > > > > >> >> > > altogether. A > > > > > > > >> >> > > > > > > Client > > > > > > > >> >> > > > > > > > will call broker with required information > > > > (owner, > > > > > > > >> renwer, > > > > > > > >> >> > > > > expiration) > > > > > > > >> >> > > > > > > and > > > > > > > >> >> > > > > > > > get back (signed hmac, uuid). Broker won't > > > store > > > > > this > > > > > > > in > > > > > > > >> >> > > zookeeper. > > > > > > > >> >> > > > > > From > > > > > > > >> >> > > > > > > > this point a client can contact any broker > > with > > > > all > > > > > > the > > > > > > > >> >> > > delegation > > > > > > > >> >> > > > > > token > > > > > > > >> >> > > > > > > > info (owner, rewner, expiration, hmac, > uuid) > > > the > > > > > > borker > > > > > > > >> will > > > > > > > >> >> > > > > regenerate > > > > > > > >> >> > > > > > > the > > > > > > > >> >> > > > > > > > hmac and as long as it matches with hmac > > > > presented > > > > > by > > > > > > > >> client , > > > > > > > >> >> > > > broker > > > > > > > >> >> > > > > > > will > > > > > > > >> >> > > > > > > > allow the request to authenticate. Only > > > problem > > > > > with > > > > > > > >> this > > > > > > > >> >> > > approach > > > > > > > >> >> > > > > is > > > > > > > >> >> > > > > > if > > > > > > > >> >> > > > > > > > the secret key is compromised any client > can > > > now > > > > > > > generate > > > > > > > >> >> > random > > > > > > > >> >> > > > > tokens > > > > > > > >> >> > > > > > > and > > > > > > > >> >> > > > > > > > they will still be able to authenticate as > > any > > > > user > > > > > > > they > > > > > > > >> like. > > > > > > > >> >> > > with > > > > > > > >> >> > > > > > > > zookeeper we guarantee that only tokens > > > acquired > > > > > via > > > > > > a > > > > > > > >> broker > > > > > > > >> >> > > > (using > > > > > > > >> >> > > > > > some > > > > > > > >> >> > > > > > > > auth scheme other than delegation token) > will > > > be > > > > > > > >> accepted. We > > > > > > > >> >> > > need > > > > > > > >> >> > > > to > > > > > > > >> >> > > > > > > > discuss which proposal makes more sense and > > we > > > > can > > > > > go > > > > > > > >> over it > > > > > > > >> >> > in > > > > > > > >> >> > > > > > > tomorrow's > > > > > > > >> >> > > > > > > > meeting. > > > > > > > >> >> > > > > > > > > > > > > > > >> >> > > > > > > > Also, can you forward the invite to me? > > > > > > > >> >> > > > > > > > > > > > > > > >> >> > > > > > > > Thanks > > > > > > > >> >> > > > > > > > Parth > > > > > > > >> >> > > > > > > > > > > > > > > >> >> > > > > > > > > > > > > > > >> >> > > > > > > > > > > > > > > >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao < > > > > > > > >> j...@confluent.io> > > > > > > > >> >> > > > wrote: > > > > > > > >> >> > > > > > > > > > > > > > > >> >> > > > > > > > > Thanks for the KIP. A few comments. > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > 100. This potentially can be useful for > > Kafka > > > > > > Connect > > > > > > > >> and > > > > > > > >> >> > Kafka > > > > > > > >> >> > > > > rest > > > > > > > >> >> > > > > > > > proxy > > > > > > > >> >> > > > > > > > > where a worker agent will need to run a > > task > > > on > > > > > > > behalf > > > > > > > >> of a > > > > > > > >> >> > > > client. > > > > > > > >> >> > > > > > We > > > > > > > >> >> > > > > > > > will > > > > > > > >> >> > > > > > > > > likely need to change how those services > > use > > > > > Kafka > > > > > > > >> clients > > > > > > > >> >> > > > > > > > > (producer/consumer). Instead of a shared > > > client > > > > > per > > > > > > > >> worker, > > > > > > > >> >> > we > > > > > > > >> >> > > > will > > > > > > > >> >> > > > > > > need > > > > > > > >> >> > > > > > > > a > > > > > > > >> >> > > > > > > > > client per user task since the > > authentication > > > > > > happens > > > > > > > >> at the > > > > > > > >> >> > > > > > connection > > > > > > > >> >> > > > > > > > > level. For Kafka Connect, the renewer > will > > be > > > > the > > > > > > > >> workers. > > > > > > > >> >> > So, > > > > > > > >> >> > > we > > > > > > > >> >> > > > > > > > probably > > > > > > > >> >> > > > > > > > > need to allow multiple renewers. For > Kafka > > > rest > > > > > > > proxy, > > > > > > > >> the > > > > > > > >> >> > > > renewer > > > > > > > >> >> > > > > > can > > > > > > > >> >> > > > > > > > > probably just be the creator of the > token. > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > 101. Do we need new acl on who can > request > > > > > > delegation > > > > > > > >> tokens? > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > 102. Do we recommend people to send > > > delegation > > > > > > tokens > > > > > > > >> in an > > > > > > > >> >> > > > > encrypted > > > > > > > >> >> > > > > > > > > channel? > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > 103. Who is responsible for expiring > > tokens, > > > > > every > > > > > > > >> broker? > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > 104. For invalidating tokens, would it be > > > > better > > > > > to > > > > > > > do > > > > > > > >> it in > > > > > > > >> >> > a > > > > > > > >> >> > > > > > request > > > > > > > >> >> > > > > > > > > instead of going to ZK directly? > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > 105. The terminology of client in the > wiki > > > > > > sometimes > > > > > > > >> refers > > > > > > > >> >> > to > > > > > > > >> >> > > > the > > > > > > > >> >> > > > > > end > > > > > > > >> >> > > > > > > > > client and some other times refers to the > > > > client > > > > > > > using > > > > > > > >> the > > > > > > > >> >> > > > > delegation > > > > > > > >> >> > > > > > > > > tokens. It would be useful to distinguish > > > > between > > > > > > the > > > > > > > >> two. > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > 106. Could you explain the sentence > "Broker > > > > also > > > > > > > >> stores the > > > > > > > >> >> > > > > > > > DelegationToken > > > > > > > >> >> > > > > > > > > without the hmac in the zookeeper." a bit > > > > more? I > > > > > > > >> thought the > > > > > > > >> >> > > > > > > delegation > > > > > > > >> >> > > > > > > > > token is the hmac. > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > Thanks, > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > Jun > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao > < > > > > > > > >> j...@confluent.io> > > > > > > > >> >> > > > wrote: > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > > Hi, Harsha, > > > > > > > >> >> > > > > > > > > > > > > > > > > >> >> > > > > > > > > > Just sent out a KIP meeting invite. We > > can > > > > > > discuss > > > > > > > >> this in > > > > > > > >> >> > > the > > > > > > > >> >> > > > > > > meeting > > > > > > > >> >> > > > > > > > > > tomorrow. > > > > > > > >> >> > > > > > > > > > > > > > > > > >> >> > > > > > > > > > Thanks, > > > > > > > >> >> > > > > > > > > > > > > > > > > >> >> > > > > > > > > > Jun > > > > > > > >> >> > > > > > > > > > > > > > > > > >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, > Harsha < > > > > > > > >> ka...@harsha.io> > > > > > > > >> >> > > > wrote: > > > > > > > >> >> > > > > > > > > > > > > > > > > >> >> > > > > > > > > >> Hi All, > > > > > > > >> >> > > > > > > > > >> Can we have a KIP meeting > > > around > > > > > > this. > > > > > > > >> The KIP > > > > > > > >> >> > is > > > > > > > >> >> > > > up > > > > > > > >> >> > > > > > for > > > > > > > >> >> > > > > > > > > >> sometime and if there are > any > > > > > > questions > > > > > > > >> lets > > > > > > > >> >> > > > quickly > > > > > > > >> >> > > > > > hash > > > > > > > >> >> > > > > > > > out > > > > > > > >> >> > > > > > > > > >> details. > > > > > > > >> >> > > > > > > > > >> > > > > > > > >> >> > > > > > > > > >> Thanks, > > > > > > > >> >> > > > > > > > > >> Harsha > > > > > > > >> >> > > > > > > > > >> > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, > parth > > > > > > > brahmbhatt > > > > > > > >> wrote: > > > > > > > >> >> > > > > > > > > >> > That is what the hadoop echo system > > uses > > > > so > > > > > no > > > > > > > >> good > > > > > > > >> >> > reason > > > > > > > >> >> > > > > > really. > > > > > > > >> >> > > > > > > > We > > > > > > > >> >> > > > > > > > > >> > could > > > > > > > >> >> > > > > > > > > >> > change it to whatever is the newest > > > > > > recommended > > > > > > > >> standard > > > > > > > >> >> > > is. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > >> >> > > > > > > > > >> > Thanks > > > > > > > >> >> > > > > > > > > >> > Parth > > > > > > > >> >> > > > > > > > > >> > > > > > > > > >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, > > Ismael > > > > > Juma < > > > > > > > >> >> > > > > ism...@juma.me.uk > > > > > > > >> >> > > > > > > > > > > > > > >> >> > > > > > > > > wrote: > > > > > > > >> >> > > > > > > > > >> > > > > > > > > >> >> > > > > > > > > >> > > Hi Parth, > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > >> >> > > > > > > > > >> > > Thanks for the KIP. I only started > > > > > reviewing > > > > > > > >> this and > > > > > > > >> >> > > may > > > > > > > >> >> > > > > have > > > > > > > >> >> > > > > > > > > >> additional > > > > > > > >> >> > > > > > > > > >> > > questions later. The immediate > > > question > > > > > that > > > > > > > >> came to > > > > > > > >> >> > > mind > > > > > > > >> >> > > > is > > > > > > > >> >> > > > > > our > > > > > > > >> >> > > > > > > > > >> choice of > > > > > > > >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's > marked > > > as > > > > > > > >> OBSOLETE in > > > > > > > >> >> > the > > > > > > > >> >> > > > IANA > > > > > > > >> >> > > > > > > > > Registry > > > > > > > >> >> > > > > > > > > >> of > > > > > > > >> >> > > > > > > > > >> > > SASL mechanisms and the original > RFC > > > > > (2831) > > > > > > > has > > > > > > > >> been > > > > > > > >> >> > > moved > > > > > > > >> >> > > > > to > > > > > > > >> >> > > > > > > > > Historic > > > > > > > >> >> > > > > > > > > >> > > status: > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > >> >> > > > > > > > > >> > > https://tools.ietf.org/html/ > rfc6331 > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > > > > > >> >> > > > > > > > > > >> > > > > > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > >> >> > > > > > > > > >> > > What is the reasoning behind that > > > > choice? > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > >> >> > > > > > > > > >> > > Thanks, > > > > > > > >> >> > > > > > > > > >> > > Ismael > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, > > Gwen > > > > > > > Shapira < > > > > > > > >> >> > > > > > > g...@confluent.io > > > > > > > >> >> > > > > > > > > > > > > > > > >> >> > > > > > > > > >> wrote: > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > >> >> > > > > > > > > >> > > > Also comments inline :) > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > > * I want to emphasize that > even > > > > though > > > > > > > >> delegation > > > > > > > >> >> > > > tokens > > > > > > > >> >> > > > > > > are a > > > > > > > >> >> > > > > > > > > >> Hadoop > > > > > > > >> >> > > > > > > > > >> > > > > innovation, I feel very > strongly > > > > about > > > > > > not > > > > > > > >> adding > > > > > > > >> >> > > > > > dependency > > > > > > > >> >> > > > > > > > on > > > > > > > >> >> > > > > > > > > >> Hadoop > > > > > > > >> >> > > > > > > > > >> > > > > when implementing delegation > > > tokens > > > > > for > > > > > > > >> Kafka. The > > > > > > > >> >> > > KIP > > > > > > > >> >> > > > > > > doesn't > > > > > > > >> >> > > > > > > > > >> imply > > > > > > > >> >> > > > > > > > > >> > > > > such dependency, but if you > can > > > > > > clarify... > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > *No hadoop dependency.* > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so > > no > > > > one > > > > > > will > > > > > > > >> read > > > > > > > >> >> > the > > > > > > > >> >> > > > KIP > > > > > > > >> >> > > > > > and > > > > > > > >> >> > > > > > > > > panic > > > > > > > >> >> > > > > > > > > >> > > > three weeks before the next > > > release... > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > > * Can we get delegation token > at > > > any > > > > > > time > > > > > > > >> after > > > > > > > >> >> > > > > > > > authenticating? > > > > > > > >> >> > > > > > > > > >> only > > > > > > > >> >> > > > > > > > > >> > > > > immediately after? > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > *As long as you are > > authenticated > > > > you > > > > > > can > > > > > > > >> get > > > > > > > >> >> > > > delegation > > > > > > > >> >> > > > > > > > tokens. > > > > > > > >> >> > > > > > > > > >> We > > > > > > > >> >> > > > > > > > > >> > > need > > > > > > > >> >> > > > > > > > > >> > > > to > > > > > > > >> >> > > > > > > > > >> > > > > discuss if a client > > authenticated > > > > > using > > > > > > > >> delegation > > > > > > > >> >> > > > > token, > > > > > > > >> >> > > > > > > can > > > > > > > >> >> > > > > > > > > also > > > > > > > >> >> > > > > > > > > >> > > > acquire > > > > > > > >> >> > > > > > > > > >> > > > > delegation token again or not. > > > Also > > > > > > there > > > > > > > >> is the > > > > > > > >> >> > > > > question > > > > > > > >> >> > > > > > of > > > > > > > >> >> > > > > > > > do > > > > > > > >> >> > > > > > > > > we > > > > > > > >> >> > > > > > > > > >> > > allow > > > > > > > >> >> > > > > > > > > >> > > > > anyone to acquire delegation > > token > > > > or > > > > > we > > > > > > > >> want > > > > > > > >> >> > > specific > > > > > > > >> >> > > > > > ACLs > > > > > > > >> >> > > > > > > (I > > > > > > > >> >> > > > > > > > > >> think > > > > > > > >> >> > > > > > > > > >> > > its > > > > > > > >> >> > > > > > > > > >> > > > an > > > > > > > >> >> > > > > > > > > >> > > > > overkill.)* > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > I agree that ACLs is an > overkill. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > I think we are debating two > > options: > > > > > > Either > > > > > > > >> require > > > > > > > >> >> > > > > Kerberos > > > > > > > >> >> > > > > > > > auth > > > > > > > >> >> > > > > > > > > >> for > > > > > > > >> >> > > > > > > > > >> > > > renewal or require non-owners to > > > > renew. > > > > > > > >> >> > > > > > > > > >> > > > I *think* the latter is simpler > > (it > > > > > > > basically > > > > > > > >> >> > require > > > > > > > >> >> > > a > > > > > > > >> >> > > > > "job > > > > > > > >> >> > > > > > > > > master" > > > > > > > >> >> > > > > > > > > >> > > > to take responsibility for the > > > > renewal, > > > > > it > > > > > > > >> will have > > > > > > > >> >> > > its > > > > > > > >> >> > > > > own > > > > > > > >> >> > > > > > > > > >> identity > > > > > > > >> >> > > > > > > > > >> > > > anyway and I think this is the > > > correct > > > > > > > design > > > > > > > >> >> > pattern > > > > > > > >> >> > > > > > anyway. > > > > > > > >> >> > > > > > > > For > > > > > > > >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to > > > coordinate > > > > > > > >> renewals?), > > > > > > > >> >> > but > > > > > > > >> >> > > > it > > > > > > > >> >> > > > > is > > > > > > > >> >> > > > > > > > hard > > > > > > > >> >> > > > > > > > > to > > > > > > > >> >> > > > > > > > > >> > > > debate simplicity without > looking > > at > > > > the > > > > > > > code > > > > > > > >> >> > changes > > > > > > > >> >> > > > > > > required. > > > > > > > >> >> > > > > > > > If > > > > > > > >> >> > > > > > > > > >> you > > > > > > > >> >> > > > > > > > > >> > > > have a draft of how the "require > > > > > Kerberos" > > > > > > > >> will look > > > > > > > >> >> > > in > > > > > > > >> >> > > > > > Kafka > > > > > > > >> >> > > > > > > > > code, > > > > > > > >> >> > > > > > > > > >> > > > I'll be happy to take a look. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > > * My understanding is that > > tokens > > > > will > > > > > > > >> propagate > > > > > > > >> >> > via > > > > > > > >> >> > > > ZK > > > > > > > >> >> > > > > > but > > > > > > > >> >> > > > > > > > > >> without > > > > > > > >> >> > > > > > > > > >> > > > > additional changes to > > > UpdateMetadata > > > > > > > >> protocol, > > > > > > > >> >> > > > correct? > > > > > > > >> >> > > > > > > > Clients > > > > > > > >> >> > > > > > > > > >> > > > > currently don't retry on SASL > > auth > > > > > > failure > > > > > > > >> (IIRC), > > > > > > > >> >> > > but > > > > > > > >> >> > > > > > since > > > > > > > >> >> > > > > > > > the > > > > > > > >> >> > > > > > > > > >> > > > > tokens propagate between > brokers > > > > > asynch, > > > > > > > we > > > > > > > >> will > > > > > > > >> >> > > need > > > > > > > >> >> > > > to > > > > > > > >> >> > > > > > > > retry a > > > > > > > >> >> > > > > > > > > >> bit > > > > > > > >> >> > > > > > > > > >> > > > > to avoid clients failing auth > > due > > > to > > > > > > > timing > > > > > > > >> >> > issues. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > *I am considering 2 > alternatives > > > > right > > > > > > > now. > > > > > > > >> The > > > > > > > >> >> > > > current > > > > > > > >> >> > > > > > > > > documented > > > > > > > >> >> > > > > > > > > >> > > > approach > > > > > > > >> >> > > > > > > > > >> > > > > is zookeeper based and it does > > not > > > > > > require > > > > > > > >> any > > > > > > > >> >> > > changes > > > > > > > >> >> > > > > to > > > > > > > >> >> > > > > > > > > >> > > UpdateMetadata > > > > > > > >> >> > > > > > > > > >> > > > > protocol. An alternative > > approach > > > > can > > > > > > > remove > > > > > > > >> >> > > zookeeper > > > > > > > >> >> > > > > > > > > dependency > > > > > > > >> >> > > > > > > > > >> as > > > > > > > >> >> > > > > > > > > >> > > well > > > > > > > >> >> > > > > > > > > >> > > > > but we can discuss that in KIP > > > > > > discussion > > > > > > > >> call.* > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you > > > want > > > > to > > > > > > > ping > > > > > > > >> Jun to > > > > > > > >> >> > > > > > arrange a > > > > > > > >> >> > > > > > > > > call? > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion > of > > > > > having > > > > > > > >> just the > > > > > > > >> >> > > > > > controller > > > > > > > >> >> > > > > > > > > issue > > > > > > > >> >> > > > > > > > > >> the > > > > > > > >> >> > > > > > > > > >> > > > > delegation tokens, to avoid > > > syncing > > > > a > > > > > > > shared > > > > > > > >> >> > secret. > > > > > > > >> >> > > > Not > > > > > > > >> >> > > > > > > sure > > > > > > > >> >> > > > > > > > if > > > > > > > >> >> > > > > > > > > >> we > > > > > > > >> >> > > > > > > > > >> > > > > want to continue the > discussion > > > here > > > > > or > > > > > > on > > > > > > > >> the > > > > > > > >> >> > > wiki. I > > > > > > > >> >> > > > > > think > > > > > > > >> >> > > > > > > > > that > > > > > > > >> >> > > > > > > > > >> we > > > > > > > >> >> > > > > > > > > >> > > > > can decouple the problem of > > "token > > > > > > > >> distribution" > > > > > > > >> >> > > from > > > > > > > >> >> > > > > > > "shared > > > > > > > >> >> > > > > > > > > >> secret > > > > > > > >> >> > > > > > > > > >> > > > > distribution" and use the > > > controller > > > > > as > > > > > > > the > > > > > > > >> only > > > > > > > >> >> > > token > > > > > > > >> >> > > > > > > > generator > > > > > > > >> >> > > > > > > > > >> to > > > > > > > >> >> > > > > > > > > >> > > > > solve the second issue, while > > > still > > > > > > using > > > > > > > >> ZK async > > > > > > > >> >> > > to > > > > > > > >> >> > > > > > > > distribute > > > > > > > >> >> > > > > > > > > >> > > > > tokens. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > *As mentioned in the previous > > > Email > > > > I > > > > > am > > > > > > > >> fine with > > > > > > > >> >> > > > that > > > > > > > >> >> > > > > > > > approach > > > > > > > >> >> > > > > > > > > >> as > > > > > > > >> >> > > > > > > > > >> > > long > > > > > > > >> >> > > > > > > > > >> > > > as > > > > > > > >> >> > > > > > > > > >> > > > > we agree that the extra > > complexity > > > > of > > > > > > > >> >> > > adding/updating > > > > > > > >> >> > > > > APIS > > > > > > > >> >> > > > > > > > adds > > > > > > > >> >> > > > > > > > > >> enough > > > > > > > >> >> > > > > > > > > >> > > > > value. The advantage with the > > > > > controller > > > > > > > >> approach > > > > > > > >> >> > is > > > > > > > >> >> > > > > > secret > > > > > > > >> >> > > > > > > > > >> rotation > > > > > > > >> >> > > > > > > > > >> > > can > > > > > > > >> >> > > > > > > > > >> > > > be > > > > > > > >> >> > > > > > > > > >> > > > > automated,frequent and would > not > > > > > require > > > > > > > >> >> > > deployment. * > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > Can you detail the extra > > complexity > > > > (or > > > > > > > point > > > > > > > >> me to > > > > > > > >> >> > > the > > > > > > > >> >> > > > > > email > > > > > > > >> >> > > > > > > I > > > > > > > >> >> > > > > > > > > >> > > > missed?) - which Apis are > > required? > > > > > > > >> >> > > > > > > > > >> > > > As far as I can tell, clients > can > > > > > already > > > > > > > >> find the > > > > > > > >> >> > > > > > controller > > > > > > > >> >> > > > > > > > from > > > > > > > >> >> > > > > > > > > >> > > > metadata. I'm a bit more > concerned > > > > about > > > > > > > >> controller > > > > > > > >> >> > > > load. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > * While I like the idea of > > forcing > > > > > > > kerberos > > > > > > > >> auth > > > > > > > >> >> > for > > > > > > > >> >> > > > > > > renwal, I > > > > > > > >> >> > > > > > > > > >> think > > > > > > > >> >> > > > > > > > > >> > > > > it mixes the transport layer > the > > > the > > > > > > > request > > > > > > > >> >> > content > > > > > > > >> >> > > > in > > > > > > > >> >> > > > > a > > > > > > > >> >> > > > > > > > pretty > > > > > > > >> >> > > > > > > > > >> ugly > > > > > > > >> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer > to > > > > > > non-owner > > > > > > > >> is > > > > > > > >> >> > > better. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > *I feel this is a necessary > > evil. > > > > > While > > > > > > > >> this will > > > > > > > >> >> > > make > > > > > > > >> >> > > > > the > > > > > > > >> >> > > > > > > > kafka > > > > > > > >> >> > > > > > > > > >> code > > > > > > > >> >> > > > > > > > > >> > > > > pretty straight forward , > > forcing > > > > > > renewer > > > > > > > >> to > > > > > > > >> >> > > > non-owner > > > > > > > >> >> > > > > > > pushes > > > > > > > >> >> > > > > > > > > >> the code > > > > > > > >> >> > > > > > > > > >> > > > > ugliness to client and makes > it > > > even > > > > > > > harder > > > > > > > >> to > > > > > > > >> >> > > > > > integrate. * > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > As mentioned before, I don't > think > > > the > > > > > > > >> "renewal by > > > > > > > >> >> > > > other" > > > > > > > >> >> > > > > > > > approach > > > > > > > >> >> > > > > > > > > >> is > > > > > > > >> >> > > > > > > > > >> > > > that ugly for the clients we > > expect > > > to > > > > > use > > > > > > > >> >> > delegation > > > > > > > >> >> > > > > tokens > > > > > > > >> >> > > > > > > > since > > > > > > > >> >> > > > > > > > > >> > > > they will have an app-master of > > some > > > > > sort > > > > > > > who > > > > > > > >> >> > > requested > > > > > > > >> >> > > > > the > > > > > > > >> >> > > > > > > > token > > > > > > > >> >> > > > > > > > > to > > > > > > > >> >> > > > > > > > > >> > > > begin with. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > The response for my question > on > > > how > > > > > > > multiple > > > > > > > >> >> > > > identities > > > > > > > >> >> > > > > > will > > > > > > > >> >> > > > > > > > be > > > > > > > >> >> > > > > > > > > >> > > > > handled wasn't super clear to > > me - > > > > > > AFAIK, > > > > > > > >> we don't > > > > > > > >> >> > > > > > > > authenticate > > > > > > > >> >> > > > > > > > > >> each > > > > > > > >> >> > > > > > > > > >> > > > > request, we authenticate > > > > connections. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > *We authenticate connections, > > and > > > > only > > > > > > > when > > > > > > > >> they > > > > > > > >> >> > are > > > > > > > >> >> > > > > being > > > > > > > >> >> > > > > > > > > >> established. > > > > > > > >> >> > > > > > > > > >> > > > Let > > > > > > > >> >> > > > > > > > > >> > > > > me try to phrase this as a > > > question, > > > > > in > > > > > > > >> absence of > > > > > > > >> >> > > > > > > delegation > > > > > > > >> >> > > > > > > > > >> tokens if > > > > > > > >> >> > > > > > > > > >> > > > we > > > > > > > >> >> > > > > > > > > >> > > > > had to support the use case > > using > > > > user > > > > > > > >> TGT's how > > > > > > > >> >> > > would > > > > > > > >> >> > > > > we > > > > > > > >> >> > > > > > do > > > > > > > >> >> > > > > > > > it? > > > > > > > >> >> > > > > > > > > >> My > > > > > > > >> >> > > > > > > > > >> > > point > > > > > > > >> >> > > > > > > > > >> > > > > was it would be no different > > with > > > > > > > delegation > > > > > > > >> >> > tokens. > > > > > > > >> >> > > > The > > > > > > > >> >> > > > > > use > > > > > > > >> >> > > > > > > > > case > > > > > > > >> >> > > > > > > > > >> you > > > > > > > >> >> > > > > > > > > >> > > are > > > > > > > >> >> > > > > > > > > >> > > > > describing seems more like > > > > > > impersonation.* > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > Yeah, I thought that one of the > > > things > > > > > > that > > > > > > > >> >> > delegation > > > > > > > >> >> > > > > > tokens > > > > > > > >> >> > > > > > > > > >> handled. > > > > > > > >> >> > > > > > > > > >> > > > Maybe I got it wrong :) > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > Thanks for the detailed answers. > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > Gwen > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > >> >> > > > > > > > > >> > > > > Thanks > > > > > > > >> >> > > > > > > > > >> > > > > Parth > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 > > AM, > > > > Gwen > > > > > > > >> Shapira < > > > > > > > >> >> > > > > > > > > g...@confluent.io > > > > > > > >> >> > > > > > > > > >> > > > > > > > > >> >> > > > > > > > > >> > > > wrote: > > > > > > > >> >> > > > > > > > > >> > > > > > > > > > > > >> >> > > > > > > > > >> > > > >> Hi Parth and Harsha, > > > > > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> >> > > > > > > > > >> > > > >> Few more comments: > > > > > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> >> > > > > > > > > >> > > > >> * The API / RequestResponse > > > section > > > > > > > >> doesn't seem > > > > > > > >> >> > to > > > > > > > >> >> > > > > have > > > > > > > >> >> > > > > > > good > > > > > > > >> >> > > > > > > > > >> > > > >> description of the changes to > > the > > > > > Kafka > > > > > > > >> Protocol. > > > > > > > >> >> > > > > Sounds > > > > > > > >> >> > > > > > > like > > > > > > > >> >> > > > > > > > > >> you are > > > > > > > >> >> > > > > > > > > >> > > > >> proposing new > > > > DelegationTokenRequest > > > > > > and > > > > > > > >> >> > > > > > RenewTokenRequest > > > > > > > >> >> > > > > > > > (and > > > > > > > >> >> > > > > > > > > >> > > > >> matching responses), without > > > > > detailing > > > > > > > the > > > > > > > >> >> > contents > > > > > > > >> >> > > > of > > > > > > > >> >> > > > > > the > > > > > > > >> >> > > > > > > > > >> requests > > > > > > > >> >> > > > > > > > > >> > > > >> and responses? Or rather, you > > > show > > > > > the > > > > > > > >> class > > > > > > > >> >> > > > interface, > > > > > > > >> >> > > > > > but > > > > > > > >> >> > > > > > > > not > > > > > > > >> >> > > > > > > > > >> the > > > > > > > >> >> > > > > > > > > >> > > > >> underlying protocol. This > could > > > be > > > > > seen > > > > > > > as > > > > > > > >> an > > > > > > > >> >> > > > > > > implementation > > > > > > > >> >> > > > > > > > > >> detail, > > > > > > > >> >> > > > > > > > > >> > > > >> but since the binary protocol > > is > > > > what > > > > > > we > > > > > > > >> provide > > > > > > > >> >> > to > > > > > > > >> >> > > > > > > non-Java > > > > > > > >> >> > > > > > > > > >> clients, > > > > > > > >> >> > > > > > > > > >> > > > >> we need to show the changes > > > there. > > > > > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds > > like > > > > > > > >> >> > > > > > > > delegationTokenRequestHandler? > > > > > > > >> >> > > > > > > > > >> Is it > > > > > > > >> >> > > > > > > > > >> > > > >> planned to be part of > KafkaApi? > > > or > > > > > > > Client? > > > > > > > >> Its > > > > > > > >> >> > > > > unclear... > > > > > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> >> > > > > > > > > >> > > > >> * I want to emphasize that > even > > > > > though > > > > > > > >> delegation > > > > > > > >> >> > > > > tokens > > > > > > > >> >> > > > > > > are > > > > > > > >> >> > > > > > > > a > > > > > > > >> >> > > > > > > > > >> Hadoop > > > > > > > >> >> > > > > > > > > >> > > > >> innovation, I feel very > > strongly > > > > > about > > > > > > > not > > > > > > > >> adding > > > > > > > >> >> > > > > > > dependency > > > > > > > >> >> > > > > > > > on > > > > > > > >> >> > > > > > > > > >> Hadoop > > > > > > > >> >> > > > > > > > > >> > > > >> when implementing delegation > > > tokens > > > > > for > > > > > > > >> Kafka. > > > > > > > >> >> > The > > > > > > > >> >> > > > KIP > > > > > > > >> >> > > > > > > > doesn't > > > > > > > >> >> > > > > > > > > >> imply > > > > > > > >> >> > > > > > > > > >> > > > >> such dependency, but if you > can > > > > > > > clarify... > > > > > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> >> > > > > > > > > >> > > > >> * Can we get delegation token > > at > > > > any > > > > > > time > > > > > > > >> after > > > > > > > >> >> > > > > > > > authenticating? > > > > > > > >> >> > > > > > > > > >> only > > > > > > > >> >> > > > > > > > > >> > > > >> immediately after? > > > > > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> >> > > > > > > > > >> > > > >> * My understanding is that > > tokens > > > > > will > > > > > > > >> propagate > > > > > > > >> >> > > via > > > > > > > >> >> > > > ZK > > > > > > > >> >> > > > > > but > > > > > > > >> >> > > > > > > > > >> without > > > > > > > >> >> > > > > > > > > >> > > > >> additional changes to > > > > UpdateMetadata > > > > > > > >> protocol, > > > > > > > >> >> > > > correct? > > > > > > > >> >> > > > > > > > Clients > > > > > > > >> >> > > > > > > > > >> > > > >> currently don't retry on SASL > > > auth > > > > > > > failure > > > > > > > >> >> > (IIRC), > > > > > > > >> >> > > > but > > > > > > > >> >> > > > > > > since > > > > > > > >> >> > > > > > > > > the > > > > > > > >> >> > > > > > > > > >> > > > >> tokens propagate between > > brokers > > > > > > asynch, > > > > > > > >> we will > > > > > > > >> >> > > need > > > > > > > >> >> > > > > to > > > > > > > >> >> > > > > > > > retry > > > > > > > >> >> > > > > > > > > a > > > > > > > >> >> > > > > > > > > >> bit > > > > > > > >> >> > > > > > > > > >> > > > >> to avoid clients failing auth > > due > > > > to > > > > > > > timing > > > > > > > >> >> > issues. > > > > > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> >> > > > > > > > > >> > > > >> * Strongly agreeing on > clients > > > not > > > > > > > >> touching ZK > > > > > > > >> >> > > > directly > > > > > > > >> >> > > > > > :) > > > > > > > >> >> > > > > > > > > >> > > > >> > > > > > > > >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion > > of > > > > > having > > > > > > > >> just the > > > > > > > >> >> > > > > > controller > > > > > > > >> >> > > > > > > > > >> issue the > > > > > > > >> >> > > > > > > > > >> > > > >> delegation tokens, to avoid > > > > syncing a > > > > > > > >> shared > > > > > > > >> >> > > secret. > > > > > > > >> >> > > > > Not > > > > > > > >> >> > > > > > > sure > > > > > > > >> >> > > > > > > > > if > > > > > > > >> >> > > > > > > > > >> we > > > > > > > >> >> > > > > > > > > >> > > > >> want to continue the > discussion > > > > here > > > > > or > > > > > > > on > > > > > > > >> the > > > > > > > >> >> > > wiki. > > > > > > > >> >> > > > -- Regards, Ashish
