Just to add on that list. 2. It would be good to document the format of the data stored in ZK. 7. Earlier, there was a discussion on whether the tokens should be propagated through ZK like config/acl/quota, or through the controller. Currently, the controller is only designed for propagating topic metadata, but not other data. 8. Should we use SCRAM to send the token instead of DIGEST-MD5 since it's deprecated?
Also, the images in the wiki seem broken. Thanks, Jun On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <g...@confluent.io> wrote: > From what I can see, remaining questions are: > > 1. Who / how are tokens renewed? By original requester only? or using > Kerberos auth only? > 2. Are tokens stored on each broker or in ZK? > 3. How are tokens invalidated / expired? > 4. Which encryption algorithm is used? > 5. What is the impersonation proposal (it wasn't in the KIP but was > discussed in this thread)? > 6. Do we need new ACLs, if so - for what actions? > > Gwen > > On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote: > > Jun & Ismael, > > Unfortunately I couldn't attend the KIP meeting > > when delegation tokens discussed. Appreciate if > > you can update the thread if you have any > > further questions. > > Thanks, > > Harsha > > > > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote: > >> It seems that the links to images in the KIP are broken. > >> > >> Liquan > >> > >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt < > >> brahmbhatt.pa...@gmail.com> wrote: > >> > >> > 110. What does getDelegationTokenAs mean? > >> > In the current proposal we only allow a user to get delegation token > for > >> > the identity that it authenticated as using another mechanism, i.e. A > user > >> > that authenticate using a keytab for principal us...@example.com > will get > >> > delegation tokens for that user only. In future I think we will have > to > >> > extend support such that we allow some set of users ( > >> > kafka-rest-u...@example.com, storm-nim...@example.com) to acquire > >> > delegation tokens on behalf of other users whose identity they have > >> > verified independently. Kafka brokers will have ACLs to control which > >> > users are allowed to impersonate other users and get tokens on behalf > of > >> > them. Overall Impersonation is a whole different problem in my > opinion and > >> > I think we can tackle it in separate KIP. > >> > > >> > 111. What's the typical rate of getting and renewing delegation > tokens? > >> > Typically this should be very very low, 1 request per minute is a > >> > relatively high estimate. However it depends on the token expiration. > I am > >> > less worried about the extra load it puts on controller vs the added > >> > complexity and the value it offers. > >> > > >> > Thanks > >> > Parth > >> > > >> > > >> > > >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <ism...@juma.me.uk> > wrote: > >> > > >> > > Thanks Rajini. It would probably require a separate KIP as it will > >> > > introduce user visible changes. We could also update KIP-48 to have > this > >> > > information, but it seems cleaner to do it separately. We can > discuss > >> > that > >> > > in the KIP call today. > >> > > > >> > > Ismael > >> > > > >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram < > >> > > rajinisiva...@googlemail.com> wrote: > >> > > > >> > > > Ismael, > >> > > > > >> > > > I have created a JIRA ( > >> > https://issues.apache.org/jira/browse/KAFKA-3751) > >> > > > for adding SCRAM as a SASL mechanism. Would that need another > KIP? If > >> > > > KIP-48 will use this mechanism, can this just be a JIRA that gets > >> > > reviewed > >> > > > when the PR is ready? > >> > > > > >> > > > Thank you, > >> > > > > >> > > > Rajini > >> > > > > >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <ism...@juma.me.uk> > >> > wrote: > >> > > > > >> > > > > Thanks Rajini, SCRAM seems like a good candidate. > >> > > > > > >> > > > > Gwen had independently mentioned this as a SASL mechanism that > might > >> > be > >> > > > > useful for Kafka and I have been meaning to check it in more > detail. > >> > > Good > >> > > > > to know that you are willing to contribute an implementation. > Maybe > >> > we > >> > > > > should file a separate JIRA for this? > >> > > > > > >> > > > > Ismael > >> > > > > > >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram < > >> > > > > rajinisiva...@googlemail.com> wrote: > >> > > > > > >> > > > > > SCRAM (Salted Challenge Response Authentication Mechanism) is > a > >> > > better > >> > > > > > mechanism than Digest-MD5. Java doesn't come with a built-in > SCRAM > >> > > > > > SaslServer or SaslClient, but I will be happy to add support > in > >> > Kafka > >> > > > > since > >> > > > > > it would be a useful mechanism to support anyway. > >> > > > > > https://tools.ietf.org/html/rfc7677 describes the protocol > for > >> > > > > > SCRAM-SHA-256. > >> > > > > > > >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <j...@confluent.io> > wrote: > >> > > > > > > >> > > > > > > Parth, > >> > > > > > > > >> > > > > > > Thanks for the explanation. A couple of more questions. > >> > > > > > > > >> > > > > > > 110. What does getDelegationTokenAs mean? > >> > > > > > > > >> > > > > > > 111. What's the typical rate of getting and renewing > delegation > >> > > > tokens? > >> > > > > > > That may have an impact on whether they should be directed > to the > >> > > > > > > controller. > >> > > > > > > > >> > > > > > > Jun > >> > > > > > > > >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt < > >> > > > > > > brahmbhatt.pa...@gmail.com> wrote: > >> > > > > > > > >> > > > > > > > Hi Jun, > >> > > > > > > > > >> > > > > > > > Thanks for reviewing. > >> > > > > > > > > >> > > > > > > > * We could add a Cluster action to add acls on who can > request > >> > > > > > delegation > >> > > > > > > > tokens. I don't see the use case for that yet but down > the line > >> > > > when > >> > > > > we > >> > > > > > > > start supporting getDelegationTokenAs it will be > necessary. > >> > > > > > > > * Yes we recommend tokens to be only used/distributed over > >> > secure > >> > > > > > > channels. > >> > > > > > > > * Depending on what design we end up choosing > Invalidation will > >> > > be > >> > > > > > > > responsibility of every broker or controller. > >> > > > > > > > * I am not sure if I documented somewhere that > invalidation > >> > will > >> > > > > > directly > >> > > > > > > > go through zookeeper but that is not the intent. > Invalidation > >> > > will > >> > > > > > either > >> > > > > > > > be request based or due to expiration. No direct zookeeper > >> > > > > interaction > >> > > > > > > from > >> > > > > > > > any client. > >> > > > > > > > * "Broker also stores the DelegationToken without the > hmac in > >> > the > >> > > > > > > > zookeeper." : Sorry about the confusion. The sole purpose > of > >> > > > > zookeeper > >> > > > > > in > >> > > > > > > > this design is as distribution channel for tokens between > all > >> > > > brokers > >> > > > > > > and a > >> > > > > > > > layer that ensures only tokens that were generated by > making a > >> > > > > request > >> > > > > > > to a > >> > > > > > > > broker will be accepted (more on this in second > paragraph). The > >> > > > token > >> > > > > > > > consists of few elements (owner, renewer, uuid , > expiration, > >> > > hmac) > >> > > > , > >> > > > > > one > >> > > > > > > of > >> > > > > > > > which is the finally generated hmac but hmac it self is > >> > derivable > >> > > > if > >> > > > > > you > >> > > > > > > > have all the other elements of the token + secret key to > >> > generate > >> > > > > hmac. > >> > > > > > > > Given zookeeper does not provide SSL support we do not > want the > >> > > > > entire > >> > > > > > > > token to be wire transferred to zookeeper as that will be > an > >> > > > insecure > >> > > > > > > wire > >> > > > > > > > transfer. Instead we only store all the other elements of > a > >> > > > > delegation > >> > > > > > > > tokens. Brokers can read these elements and because they > also > >> > > have > >> > > > > > access > >> > > > > > > > to secret key they will be able to generate hmac on their > end. > >> > > > > > > > > >> > > > > > > > One of the alternative proposed is to avoid zookeeper > >> > > altogether. A > >> > > > > > > Client > >> > > > > > > > will call broker with required information (owner, renwer, > >> > > > > expiration) > >> > > > > > > and > >> > > > > > > > get back (signed hmac, uuid). Broker won't store this in > >> > > zookeeper. > >> > > > > > From > >> > > > > > > > this point a client can contact any broker with all the > >> > > delegation > >> > > > > > token > >> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the borker > will > >> > > > > regenerate > >> > > > > > > the > >> > > > > > > > hmac and as long as it matches with hmac presented by > client , > >> > > > broker > >> > > > > > > will > >> > > > > > > > allow the request to authenticate. Only problem with this > >> > > approach > >> > > > > is > >> > > > > > if > >> > > > > > > > the secret key is compromised any client can now generate > >> > random > >> > > > > tokens > >> > > > > > > and > >> > > > > > > > they will still be able to authenticate as any user they > like. > >> > > with > >> > > > > > > > zookeeper we guarantee that only tokens acquired via a > broker > >> > > > (using > >> > > > > > some > >> > > > > > > > auth scheme other than delegation token) will be > accepted. We > >> > > need > >> > > > to > >> > > > > > > > discuss which proposal makes more sense and we can go > over it > >> > in > >> > > > > > > tomorrow's > >> > > > > > > > meeting. > >> > > > > > > > > >> > > > > > > > Also, can you forward the invite to me? > >> > > > > > > > > >> > > > > > > > Thanks > >> > > > > > > > Parth > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao < > j...@confluent.io> > >> > > > wrote: > >> > > > > > > > > >> > > > > > > > > Thanks for the KIP. A few comments. > >> > > > > > > > > > >> > > > > > > > > 100. This potentially can be useful for Kafka Connect > and > >> > Kafka > >> > > > > rest > >> > > > > > > > proxy > >> > > > > > > > > where a worker agent will need to run a task on behalf > of a > >> > > > client. > >> > > > > > We > >> > > > > > > > will > >> > > > > > > > > likely need to change how those services use Kafka > clients > >> > > > > > > > > (producer/consumer). Instead of a shared client per > worker, > >> > we > >> > > > will > >> > > > > > > need > >> > > > > > > > a > >> > > > > > > > > client per user task since the authentication happens > at the > >> > > > > > connection > >> > > > > > > > > level. For Kafka Connect, the renewer will be the > workers. > >> > So, > >> > > we > >> > > > > > > > probably > >> > > > > > > > > need to allow multiple renewers. For Kafka rest proxy, > the > >> > > > renewer > >> > > > > > can > >> > > > > > > > > probably just be the creator of the token. > >> > > > > > > > > > >> > > > > > > > > 101. Do we need new acl on who can request delegation > tokens? > >> > > > > > > > > > >> > > > > > > > > 102. Do we recommend people to send delegation tokens > in an > >> > > > > encrypted > >> > > > > > > > > channel? > >> > > > > > > > > > >> > > > > > > > > 103. Who is responsible for expiring tokens, every > broker? > >> > > > > > > > > > >> > > > > > > > > 104. For invalidating tokens, would it be better to do > it in > >> > a > >> > > > > > request > >> > > > > > > > > instead of going to ZK directly? > >> > > > > > > > > > >> > > > > > > > > 105. The terminology of client in the wiki sometimes > refers > >> > to > >> > > > the > >> > > > > > end > >> > > > > > > > > client and some other times refers to the client using > the > >> > > > > delegation > >> > > > > > > > > tokens. It would be useful to distinguish between the > two. > >> > > > > > > > > > >> > > > > > > > > 106. Could you explain the sentence "Broker also stores > the > >> > > > > > > > DelegationToken > >> > > > > > > > > without the hmac in the zookeeper." a bit more? I > thought the > >> > > > > > > delegation > >> > > > > > > > > token is the hmac. > >> > > > > > > > > > >> > > > > > > > > Thanks, > >> > > > > > > > > > >> > > > > > > > > Jun > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao < > j...@confluent.io> > >> > > > wrote: > >> > > > > > > > > > >> > > > > > > > > > Hi, Harsha, > >> > > > > > > > > > > >> > > > > > > > > > Just sent out a KIP meeting invite. We can discuss > this in > >> > > the > >> > > > > > > meeting > >> > > > > > > > > > tomorrow. > >> > > > > > > > > > > >> > > > > > > > > > Thanks, > >> > > > > > > > > > > >> > > > > > > > > > Jun > >> > > > > > > > > > > >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha < > ka...@harsha.io> > >> > > > wrote: > >> > > > > > > > > > > >> > > > > > > > > >> Hi All, > >> > > > > > > > > >> Can we have a KIP meeting around this. > The KIP > >> > is > >> > > > up > >> > > > > > for > >> > > > > > > > > >> sometime and if there are any questions > lets > >> > > > quickly > >> > > > > > hash > >> > > > > > > > out > >> > > > > > > > > >> details. > >> > > > > > > > > >> > >> > > > > > > > > >> Thanks, > >> > > > > > > > > >> Harsha > >> > > > > > > > > >> > >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth brahmbhatt > wrote: > >> > > > > > > > > >> > That is what the hadoop echo system uses so no good > >> > reason > >> > > > > > really. > >> > > > > > > > We > >> > > > > > > > > >> > could > >> > > > > > > > > >> > change it to whatever is the newest recommended > standard > >> > > is. > >> > > > > > > > > >> > > >> > > > > > > > > >> > Thanks > >> > > > > > > > > >> > Parth > >> > > > > > > > > >> > > >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma < > >> > > > > ism...@juma.me.uk > >> > > > > > > > >> > > > > > > > > wrote: > >> > > > > > > > > >> > > >> > > > > > > > > >> > > Hi Parth, > >> > > > > > > > > >> > > > >> > > > > > > > > >> > > Thanks for the KIP. I only started reviewing > this and > >> > > may > >> > > > > have > >> > > > > > > > > >> additional > >> > > > > > > > > >> > > questions later. The immediate question that > came to > >> > > mind > >> > > > is > >> > > > > > our > >> > > > > > > > > >> choice of > >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as OBSOLETE > in > >> > the > >> > > > IANA > >> > > > > > > > > Registry > >> > > > > > > > > >> of > >> > > > > > > > > >> > > SASL mechanisms and the original RFC (2831) has > been > >> > > moved > >> > > > > to > >> > > > > > > > > Historic > >> > > > > > > > > >> > > status: > >> > > > > > > > > >> > > > >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331 > >> > > > > > > > > >> > > > >> > > > > > > > > > >> > > > > > > >> > > > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml > >> > > > > > > > > >> > > > >> > > > > > > > > >> > > What is the reasoning behind that choice? > >> > > > > > > > > >> > > > >> > > > > > > > > >> > > Thanks, > >> > > > > > > > > >> > > Ismael > >> > > > > > > > > >> > > > >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen Shapira < > >> > > > > > > g...@confluent.io > >> > > > > > > > > > >> > > > > > > > > >> wrote: > >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > Also comments inline :) > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > * I want to emphasize that even though > delegation > >> > > > tokens > >> > > > > > > are a > >> > > > > > > > > >> Hadoop > >> > > > > > > > > >> > > > > innovation, I feel very strongly about not > adding > >> > > > > > dependency > >> > > > > > > > on > >> > > > > > > > > >> Hadoop > >> > > > > > > > > >> > > > > when implementing delegation tokens for > Kafka. The > >> > > KIP > >> > > > > > > doesn't > >> > > > > > > > > >> imply > >> > > > > > > > > >> > > > > such dependency, but if you can clarify... > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > *No hadoop dependency.* > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one will > read > >> > the > >> > > > KIP > >> > > > > > and > >> > > > > > > > > panic > >> > > > > > > > > >> > > > three weeks before the next release... > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > * Can we get delegation token at any time > after > >> > > > > > > > authenticating? > >> > > > > > > > > >> only > >> > > > > > > > > >> > > > > immediately after? > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > *As long as you are authenticated you can get > >> > > > delegation > >> > > > > > > > tokens. > >> > > > > > > > > >> We > >> > > > > > > > > >> > > need > >> > > > > > > > > >> > > > to > >> > > > > > > > > >> > > > > discuss if a client authenticated using > delegation > >> > > > > token, > >> > > > > > > can > >> > > > > > > > > also > >> > > > > > > > > >> > > > acquire > >> > > > > > > > > >> > > > > delegation token again or not. Also there is > the > >> > > > > question > >> > > > > > of > >> > > > > > > > do > >> > > > > > > > > we > >> > > > > > > > > >> > > allow > >> > > > > > > > > >> > > > > anyone to acquire delegation token or we want > >> > > specific > >> > > > > > ACLs > >> > > > > > > (I > >> > > > > > > > > >> think > >> > > > > > > > > >> > > its > >> > > > > > > > > >> > > > an > >> > > > > > > > > >> > > > > overkill.)* > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > I agree that ACLs is an overkill. > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > I think we are debating two options: Either > require > >> > > > > Kerberos > >> > > > > > > > auth > >> > > > > > > > > >> for > >> > > > > > > > > >> > > > renewal or require non-owners to renew. > >> > > > > > > > > >> > > > I *think* the latter is simpler (it basically > >> > require > >> > > a > >> > > > > "job > >> > > > > > > > > master" > >> > > > > > > > > >> > > > to take responsibility for the renewal, it > will have > >> > > its > >> > > > > own > >> > > > > > > > > >> identity > >> > > > > > > > > >> > > > anyway and I think this is the correct design > >> > pattern > >> > > > > > anyway. > >> > > > > > > > For > >> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate > renewals?), > >> > but > >> > > > it > >> > > > > is > >> > > > > > > > hard > >> > > > > > > > > to > >> > > > > > > > > >> > > > debate simplicity without looking at the code > >> > changes > >> > > > > > > required. > >> > > > > > > > If > >> > > > > > > > > >> you > >> > > > > > > > > >> > > > have a draft of how the "require Kerberos" > will look > >> > > in > >> > > > > > Kafka > >> > > > > > > > > code, > >> > > > > > > > > >> > > > I'll be happy to take a look. > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > * My understanding is that tokens will > propagate > >> > via > >> > > > ZK > >> > > > > > but > >> > > > > > > > > >> without > >> > > > > > > > > >> > > > > additional changes to UpdateMetadata > protocol, > >> > > > correct? > >> > > > > > > > Clients > >> > > > > > > > > >> > > > > currently don't retry on SASL auth failure > (IIRC), > >> > > but > >> > > > > > since > >> > > > > > > > the > >> > > > > > > > > >> > > > > tokens propagate between brokers asynch, we > will > >> > > need > >> > > > to > >> > > > > > > > retry a > >> > > > > > > > > >> bit > >> > > > > > > > > >> > > > > to avoid clients failing auth due to timing > >> > issues. > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > *I am considering 2 alternatives right now. > The > >> > > > current > >> > > > > > > > > documented > >> > > > > > > > > >> > > > approach > >> > > > > > > > > >> > > > > is zookeeper based and it does not require > any > >> > > changes > >> > > > > to > >> > > > > > > > > >> > > UpdateMetadata > >> > > > > > > > > >> > > > > protocol. An alternative approach can remove > >> > > zookeeper > >> > > > > > > > > dependency > >> > > > > > > > > >> as > >> > > > > > > > > >> > > well > >> > > > > > > > > >> > > > > but we can discuss that in KIP discussion > call.* > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to ping > Jun to > >> > > > > > arrange a > >> > > > > > > > > call? > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of having just > the > >> > > > > > controller > >> > > > > > > > > issue > >> > > > > > > > > >> the > >> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a shared > >> > secret. > >> > > > Not > >> > > > > > > sure > >> > > > > > > > if > >> > > > > > > > > >> we > >> > > > > > > > > >> > > > > want to continue the discussion here or on > the > >> > > wiki. I > >> > > > > > think > >> > > > > > > > > that > >> > > > > > > > > >> we > >> > > > > > > > > >> > > > > can decouple the problem of "token > distribution" > >> > > from > >> > > > > > > "shared > >> > > > > > > > > >> secret > >> > > > > > > > > >> > > > > distribution" and use the controller as the > only > >> > > token > >> > > > > > > > generator > >> > > > > > > > > >> to > >> > > > > > > > > >> > > > > solve the second issue, while still using ZK > async > >> > > to > >> > > > > > > > distribute > >> > > > > > > > > >> > > > > tokens. > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > *As mentioned in the previous Email I am > fine with > >> > > > that > >> > > > > > > > approach > >> > > > > > > > > >> as > >> > > > > > > > > >> > > long > >> > > > > > > > > >> > > > as > >> > > > > > > > > >> > > > > we agree that the extra complexity of > >> > > adding/updating > >> > > > > APIS > >> > > > > > > > adds > >> > > > > > > > > >> enough > >> > > > > > > > > >> > > > > value. The advantage with the controller > approach > >> > is > >> > > > > > secret > >> > > > > > > > > >> rotation > >> > > > > > > > > >> > > can > >> > > > > > > > > >> > > > be > >> > > > > > > > > >> > > > > automated,frequent and would not require > >> > > deployment. * > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > Can you detail the extra complexity (or point > me to > >> > > the > >> > > > > > email > >> > > > > > > I > >> > > > > > > > > >> > > > missed?) - which Apis are required? > >> > > > > > > > > >> > > > As far as I can tell, clients can already find > the > >> > > > > > controller > >> > > > > > > > from > >> > > > > > > > > >> > > > metadata. I'm a bit more concerned about > controller > >> > > > load. > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > * While I like the idea of forcing kerberos > auth > >> > for > >> > > > > > > renwal, I > >> > > > > > > > > >> think > >> > > > > > > > > >> > > > > it mixes the transport layer the the request > >> > content > >> > > > in > >> > > > > a > >> > > > > > > > pretty > >> > > > > > > > > >> ugly > >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to non-owner is > >> > > better. > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > *I feel this is a necessary evil. While this > will > >> > > make > >> > > > > the > >> > > > > > > > kafka > >> > > > > > > > > >> code > >> > > > > > > > > >> > > > > pretty straight forward , forcing renewer to > >> > > > non-owner > >> > > > > > > pushes > >> > > > > > > > > >> the code > >> > > > > > > > > >> > > > > ugliness to client and makes it even harder > to > >> > > > > > integrate. * > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > As mentioned before, I don't think the > "renewal by > >> > > > other" > >> > > > > > > > approach > >> > > > > > > > > >> is > >> > > > > > > > > >> > > > that ugly for the clients we expect to use > >> > delegation > >> > > > > tokens > >> > > > > > > > since > >> > > > > > > > > >> > > > they will have an app-master of some sort who > >> > > requested > >> > > > > the > >> > > > > > > > token > >> > > > > > > > > to > >> > > > > > > > > >> > > > begin with. > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > The response for my question on how multiple > >> > > > identities > >> > > > > > will > >> > > > > > > > be > >> > > > > > > > > >> > > > > handled wasn't super clear to me - AFAIK, we > don't > >> > > > > > > > authenticate > >> > > > > > > > > >> each > >> > > > > > > > > >> > > > > request, we authenticate connections. > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > *We authenticate connections, and only when > they > >> > are > >> > > > > being > >> > > > > > > > > >> established. > >> > > > > > > > > >> > > > Let > >> > > > > > > > > >> > > > > me try to phrase this as a question, in > absence of > >> > > > > > > delegation > >> > > > > > > > > >> tokens if > >> > > > > > > > > >> > > > we > >> > > > > > > > > >> > > > > had to support the use case using user TGT's > how > >> > > would > >> > > > > we > >> > > > > > do > >> > > > > > > > it? > >> > > > > > > > > >> My > >> > > > > > > > > >> > > point > >> > > > > > > > > >> > > > > was it would be no different with delegation > >> > tokens. > >> > > > The > >> > > > > > use > >> > > > > > > > > case > >> > > > > > > > > >> you > >> > > > > > > > > >> > > are > >> > > > > > > > > >> > > > > describing seems more like impersonation.* > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > Yeah, I thought that one of the things that > >> > delegation > >> > > > > > tokens > >> > > > > > > > > >> handled. > >> > > > > > > > > >> > > > Maybe I got it wrong :) > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > Thanks for the detailed answers. > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > Gwen > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > Thanks > >> > > > > > > > > >> > > > > Parth > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen > Shapira < > >> > > > > > > > > g...@confluent.io > >> > > > > > > > > >> > > >> > > > > > > > > >> > > > wrote: > >> > > > > > > > > >> > > > > > >> > > > > > > > > >> > > > >> Hi Parth and Harsha, > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> Few more comments: > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * The API / RequestResponse section doesn't > seem > >> > to > >> > > > > have > >> > > > > > > good > >> > > > > > > > > >> > > > >> description of the changes to the Kafka > Protocol. > >> > > > > Sounds > >> > > > > > > like > >> > > > > > > > > >> you are > >> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest and > >> > > > > > RenewTokenRequest > >> > > > > > > > (and > >> > > > > > > > > >> > > > >> matching responses), without detailing the > >> > contents > >> > > > of > >> > > > > > the > >> > > > > > > > > >> requests > >> > > > > > > > > >> > > > >> and responses? Or rather, you show the class > >> > > > interface, > >> > > > > > but > >> > > > > > > > not > >> > > > > > > > > >> the > >> > > > > > > > > >> > > > >> underlying protocol. This could be seen as > an > >> > > > > > > implementation > >> > > > > > > > > >> detail, > >> > > > > > > > > >> > > > >> but since the binary protocol is what we > provide > >> > to > >> > > > > > > non-Java > >> > > > > > > > > >> clients, > >> > > > > > > > > >> > > > >> we need to show the changes there. > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * getDelegationToken sounds like > >> > > > > > > > delegationTokenRequestHandler? > >> > > > > > > > > >> Is it > >> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or Client? > Its > >> > > > > unclear... > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * I want to emphasize that even though > delegation > >> > > > > tokens > >> > > > > > > are > >> > > > > > > > a > >> > > > > > > > > >> Hadoop > >> > > > > > > > > >> > > > >> innovation, I feel very strongly about not > adding > >> > > > > > > dependency > >> > > > > > > > on > >> > > > > > > > > >> Hadoop > >> > > > > > > > > >> > > > >> when implementing delegation tokens for > Kafka. > >> > The > >> > > > KIP > >> > > > > > > > doesn't > >> > > > > > > > > >> imply > >> > > > > > > > > >> > > > >> such dependency, but if you can clarify... > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * Can we get delegation token at any time > after > >> > > > > > > > authenticating? > >> > > > > > > > > >> only > >> > > > > > > > > >> > > > >> immediately after? > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * My understanding is that tokens will > propagate > >> > > via > >> > > > ZK > >> > > > > > but > >> > > > > > > > > >> without > >> > > > > > > > > >> > > > >> additional changes to UpdateMetadata > protocol, > >> > > > correct? > >> > > > > > > > Clients > >> > > > > > > > > >> > > > >> currently don't retry on SASL auth failure > >> > (IIRC), > >> > > > but > >> > > > > > > since > >> > > > > > > > > the > >> > > > > > > > > >> > > > >> tokens propagate between brokers asynch, we > will > >> > > need > >> > > > > to > >> > > > > > > > retry > >> > > > > > > > > a > >> > > > > > > > > >> bit > >> > > > > > > > > >> > > > >> to avoid clients failing auth due to timing > >> > issues. > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * Strongly agreeing on clients not touching > ZK > >> > > > directly > >> > > > > > :) > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of having > just the > >> > > > > > controller > >> > > > > > > > > >> issue the > >> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a shared > >> > > secret. > >> > > > > Not > >> > > > > > > sure > >> > > > > > > > > if > >> > > > > > > > > >> we > >> > > > > > > > > >> > > > >> want to continue the discussion here or on > the > >> > > wiki. > >> > > > I > >> > > > > > > think > >> > > > > > > > > >> that we > >> > > > > > > > > >> > > > >> can decouple the problem of "token > distribution" > >> > > from > >> > > > > > > "shared > >> > > > > > > > > >> secret > >> > > > > > > > > >> > > > >> distribution" and use the controller as the > only > >> > > > token > >> > > > > > > > > generator > >> > > > > > > > > >> to > >> > > > > > > > > >> > > > >> solve the second issue, while still using ZK > >> > async > >> > > to > >> > > > > > > > > distribute > >> > > > > > > > > >> > > > >> tokens. > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * I am also uncomfortable with infinite > lifetime > >> > of > >> > > > > > tokens > >> > > > > > > > (and > >> > > > > > > > > >> hoped > >> > > > > > > > > >> > > > >> to hear from others in the community) - but > >> > having > >> > > > the > >> > > > > > > option > >> > > > > > > > > and > >> > > > > > > > > >> > > > >> documenting it as less secure, allows users > to > >> > > > > configure > >> > > > > > > > their > >> > > > > > > > > >> system > >> > > > > > > > > >> > > > >> as the wish. > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * While I like the idea of forcing kerberos > auth > >> > > for > >> > > > > > > renwal, > >> > > > > > > > I > >> > > > > > > > > >> think > >> > > > > > > > > >> > > > >> it mixes the transport layer the the request > >> > > content > >> > > > > in a > >> > > > > > > > > pretty > >> > > > > > > > > >> ugly > >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to non-owner > is > >> > > better. > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> Things I'd still like to see: > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * More detailed explanation on what we plan > to do > >> > > for > >> > > > > the > >> > > > > > > > java > >> > > > > > > > > >> clients > >> > > > > > > > > >> > > > >> specifically - new configuration? new APIs? > >> > > > > > > > > >> > > > >> The response for my question on how multiple > >> > > > identities > >> > > > > > > will > >> > > > > > > > be > >> > > > > > > > > >> > > > >> handled wasn't super clear to me - AFAIK, we > >> > don't > >> > > > > > > > authenticate > >> > > > > > > > > >> each > >> > > > > > > > > >> > > > >> request, we authenticate connections. > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are only > used > >> > in > >> > > > the > >> > > > > > > Hadoop > >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are > >> > alternatives > >> > > in > >> > > > > > other > >> > > > > > > > > >> ecosystems > >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and whether > there > >> > are > >> > > > some > >> > > > > > > > > >> advantages > >> > > > > > > > > >> > > > >> there. > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> Gwen > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha < > >> > > > > ka...@harsha.io > >> > > > > > > > >> > > > > > > > > wrote: > >> > > > > > > > > >> > > > >> > Hi Gwen, > >> > > > > > > > > >> > > > >> > Can you look at Parth's last > reply. > >> > > Does > >> > > > > it > >> > > > > > > > answer > >> > > > > > > > > >> your > >> > > > > > > > > >> > > > >> > concerns. > >> > > > > > > > > >> > > > >> > > >> > > > > > > > > >> > > > >> > Thanks, > >> > > > > > > > > >> > > > >> > Harsha > >> > > > > > > > > >> > > > >> > > >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth > >> > > brahmbhatt > >> > > > > > wrote: > >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki > already > >> > has > >> > > > > > details > >> > > > > > > on > >> > > > > > > > > >> token > >> > > > > > > > > >> > > > >> >> expiration > >> > > > > > > > > >> > > > >> >> under token acquisition process > >> > > > > > > > > >> > > > >> >> < > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > >> > > > > > > > > >> > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition > >> > > > > > > > > >> > > > >> >. > >> > > > > > > > > >> > > > >> >> Current proposal is that tokens will > expire > >> > > based > >> > > > > on a > >> > > > > > > > > server > >> > > > > > > > > >> side > >> > > > > > > > > >> > > > >> >> configuration (default 24 hours) unless > >> > renewed. > >> > > > > > Renewal > >> > > > > > > > is > >> > > > > > > > > >> only > >> > > > > > > > > >> > > > allowed > >> > > > > > > > > >> > > > >> >> until the max life time of token. > >> > Alternatively > >> > > we > >> > > > > > could > >> > > > > > > > > also > >> > > > > > > > > >> make > >> > > > > > > > > >> > > > that > >> > > > > > > > > >> > > > >> >> an > >> > > > > > > > > >> > > > >> >> optional param and the server side > default can > >> > > > serve > >> > > > > > as > >> > > > > > > > the > >> > > > > > > > > >> upper > >> > > > > > > > > >> > > > bound. > >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> To your second point it will be done > exactly > >> > the > >> > > > > same > >> > > > > > > way > >> > > > > > > > we > >> > > > > > > > > >> would > >> > > > > > > > > >> > > > >> >> support > >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling client > will have > >> > > to > >> > > > > put > >> > > > > > > the > >> > > > > > > > > >> tokens it > >> > > > > > > > > >> > > > >> wants > >> > > > > > > > > >> > > > >> >> to use in the subject instance and call > >> > > > > > produce/consume > >> > > > > > > > > inside > >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will have to > keep > >> > > track > >> > > > of > >> > > > > > its > >> > > > > > > > own > >> > > > > > > > > >> > > > subject. I > >> > > > > > > > > >> > > > >> >> will have to look at the code to see if > we > >> > > support > >> > > > > > this > >> > > > > > > > > >> feature > >> > > > > > > > > >> > > right > >> > > > > > > > > >> > > > >> now > >> > > > > > > > > >> > > > >> >> but my understanding is delegation token > >> > > shouldn't > >> > > > > > need > >> > > > > > > > any > >> > > > > > > > > >> special > >> > > > > > > > > >> > > > >> >> treatment as its just another type of > >> > Credential > >> > > > in > >> > > > > > the > >> > > > > > > > > >> subject. > >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> I would also like to know what is your > opinion > >> > > > about > >> > > > > > > > > infinite > >> > > > > > > > > >> > > renewal > >> > > > > > > > > >> > > > >> (my > >> > > > > > > > > >> > > > >> >> recommendation is to not support this), > tokens > >> > > > > > renewing > >> > > > > > > > them > >> > > > > > > > > >> > > self(my > >> > > > > > > > > >> > > > >> >> recommendation is to not support this) > and > >> > most > >> > > > > > > > importantly > >> > > > > > > > > >> your > >> > > > > > > > > >> > > > choice > >> > > > > > > > > >> > > > >> >> between the alternatives listed on this > thread > >> > > > > > > > > >> > > > >> >> < > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > >> > > > > > > > > >> > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism > >> > > > > > > > > >> > > > >> > > >> > > > > > > > > >> > > > >> >> ( I am leaning towards the alternative-2 > minus > >> > > > > > > controller > >> > > > > > > > > >> > > > distributing > >> > > > > > > > > >> > > > >> >> secret). Thanks again for reviewing. > >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> Thanks > >> > > > > > > > > >> > > > >> >> Parth > >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen > Shapira < > >> > > > > > > > > >> g...@confluent.io> > >> > > > > > > > > >> > > > wrote: > >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > Harsha, > >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy. I > didn't > >> > see > >> > > > > your > >> > > > > > > > design > >> > > > > > > > > >> yet, > >> > > > > > > > > >> > > > but in > >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of producers, > which > >> > > > will > >> > > > > > > serve > >> > > > > > > > > >> multiple > >> > > > > > > > > >> > > > users > >> > > > > > > > > >> > > > >> >> > going through the proxy. Since these > users > >> > > will > >> > > > > have > >> > > > > > > > > >> different > >> > > > > > > > > >> > > > >> >> > privileges, they'll need to > authenticate > >> > > > > separately, > >> > > > > > > and > >> > > > > > > > > >> can't > >> > > > > > > > > >> > > > share a > >> > > > > > > > > >> > > > >> >> > token. > >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> >> > Am I missing anything? > >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> >> > Gwen > >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM, Harsha > < > >> > > > > > > ka...@harsha.io > >> > > > > > > > > > >> > > > > > > > > >> wrote: > >> > > > > > > > > >> > > > >> >> > > Gwen, > >> > > > > > > > > >> > > > >> >> > > On your second point. Can > you > >> > > > > describe > >> > > > > > a > >> > > > > > > > > >> usecase > >> > > > > > > > > >> > > where > >> > > > > > > > > >> > > > >> >> > > mutliple clients ended up > >> > > sharing a > >> > > > > > > > producer > >> > > > > > > > > >> and > >> > > > > > > > > >> > > even > >> > > > > > > > > >> > > > if > >> > > > > > > > > >> > > > >> they > >> > > > > > > > > >> > > > >> >> > > do why can't they not use > >> > single > >> > > > > token > >> > > > > > > that > >> > > > > > > > > >> producer > >> > > > > > > > > >> > > > >> >> > > captures. Why would we > need > >> > > > multiple > >> > > > > > > > clients > >> > > > > > > > > >> with > >> > > > > > > > > >> > > > >> different > >> > > > > > > > > >> > > > >> >> > > tokens sharing a single > >> > instance > >> > > of > >> > > > > > > > producer. > >> > > > > > > > > >> Also > >> > > > > > > > > >> > > in > >> > > > > > > > > >> > > > >> this > >> > > > > > > > > >> > > > >> >> > > case other clients have > access > >> > > all > >> > > > > the > >> > > > > > > > tokens > >> > > > > > > > > >> no? > >> > > > > > > > > >> > > > >> >> > > > >> > > > > > > > > >> > > > >> >> > > Thanks, > >> > > > > > > > > >> > > > >> >> > > Harsha > >> > > > > > > > > >> > > > >> >> > > > >> > > > > > > > > >> > > > >> >> > > > >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM, > Gwen > >> > > Shapira > >> > > > > > > wrote: > >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay: > >> > > > > > > > > >> > > > >> >> > >> > >> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't see in > the > >> > > wiki: > >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for > delegation > >> > > > > tokens? > >> > > > > > > > > >> Renewal? How > >> > > > > > > > > >> > > > do we > >> > > > > > > > > >> > > > >> >> > >> revoke them? > >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation > tokens > >> > for > >> > > > > > "do-as" > >> > > > > > > > > (say, > >> > > > > > > > > >> > > submit > >> > > > > > > > > >> > > > >> Storm > >> > > > > > > > > >> > > > >> >> > >> job as my user), we will need a > producer > >> > > for > >> > > > > > every > >> > > > > > > > job > >> > > > > > > > > >> (we > >> > > > > > > > > >> > > can't > >> > > > > > > > > >> > > > >> share > >> > > > > > > > > >> > > > >> >> > >> them between multiple jobs running > on > >> > same > >> > > > > node), > >> > > > > > > > since > >> > > > > > > > > >> we > >> > > > > > > > > >> > > only > >> > > > > > > > > >> > > > >> >> > >> authenticate when connecting. Is > there a > >> > > plan > >> > > > > to > >> > > > > > > > change > >> > > > > > > > > >> this > >> > > > > > > > > >> > > for > >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order to allow > >> > > multiple > >> > > > > > users > >> > > > > > > > > with > >> > > > > > > > > >> > > > different > >> > > > > > > > > >> > > > >> >> > >> tokens to share a client? > >> > > > > > > > > >> > > > >> >> > >> > >> > > > > > > > > >> > > > >> >> > >> Gwen > >> > > > > > > > > >> > > > >> >> > >> > >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM, > parth > >> > > > > brahmbhatt > >> > > > > > > > > >> > > > >> >> > >> <brahmbhatt.pa...@gmail.com> wrote: > >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more time, can > >> > other > >> > > > > > > committers > >> > > > > > > > > >> review? > >> > > > > > > > > >> > > > >> >> > >> > > >> > > > > > > > > >> > > > >> >> > >> > Thanks > >> > > > > > > > > >> > > > >> >> > >> > Parth > >> > > > > > > > > >> > > > >> >> > >> > > >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07 AM, > >> > Harsha < > >> > > > > > > > > >> ka...@harsha.io> > >> > > > > > > > > >> > > > wrote: > >> > > > > > > > > >> > > > >> >> > >> > > >> > > > > > > > > >> > > > >> >> > >> >> Parth, > >> > > > > > > > > >> > > > >> >> > >> >> Overall current design > looks > >> > > > good > >> > > > > to > >> > > > > > > > me. I > >> > > > > > > > > >> am +1 > >> > > > > > > > > >> > > on > >> > > > > > > > > >> > > > >> the > >> > > > > > > > > >> > > > >> >> > KIP. > >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review this as > >> > well. > >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> -Harsha > >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57 > AM, > >> > parth > >> > > > > > > > brahmbhatt > >> > > > > > > > > >> wrote: > >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review Jitendra. > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of > infinite > >> > > > lifetime > >> > > > > > > but I > >> > > > > > > > > >> see the > >> > > > > > > > > >> > > > >> Streaming > >> > > > > > > > > >> > > > >> >> > use > >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming use > case I > >> > > was > >> > > > > > hoping > >> > > > > > > > > >> there will > >> > > > > > > > > >> > > > be > >> > > > > > > > > >> > > > >> some > >> > > > > > > > > >> > > > >> >> > notion > >> > > > > > > > > >> > > > >> >> > >> >> > of > >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can get new > >> > > > delegation > >> > > > > > > tokens > >> > > > > > > > > at > >> > > > > > > > > >> fixed > >> > > > > > > > > >> > > > >> interval > >> > > > > > > > > >> > > > >> >> > and > >> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If that > is > >> > not > >> > > > the > >> > > > > > case > >> > > > > > > > for > >> > > > > > > > > >> we can > >> > > > > > > > > >> > > > >> discuss > >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing > them self > >> > > and > >> > > > > the > >> > > > > > > > > >> security > >> > > > > > > > > >> > > > >> implications > >> > > > > > > > > >> > > > >> >> > of the > >> > > > > > > > > >> > > > >> >> > >> >> > same. > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to fetch > >> > tokens > >> > > > > from > >> > > > > > > > > >> zookeeper, > >> > > > > > > > > >> > > > >> overall I > >> > > > > > > > > >> > > > >> >> > think > >> > > > > > > > > >> > > > >> >> > >> >> > its > >> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't rely > on our > >> > > > > metadata > >> > > > > > > > store > >> > > > > > > > > >> and I > >> > > > > > > > > >> > > > >> think we > >> > > > > > > > > >> > > > >> >> > are > >> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction with > all > >> > the > >> > > > > KIP-4 > >> > > > > > > > > >> improvements. > >> > > > > > > > > >> > > > I > >> > > > > > > > > >> > > > >> chose > >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case the > client > >> > > > will > >> > > > > > > still > >> > > > > > > > > >> just talk > >> > > > > > > > > >> > > > to > >> > > > > > > > > >> > > > >> >> > broker , its > >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will use > zookeeper > >> > > > which > >> > > > > we > >> > > > > > > > > >> already do > >> > > > > > > > > >> > > > for a > >> > > > > > > > > >> > > > >> lot > >> > > > > > > > > >> > > > >> >> > of > >> > > > > > > > > >> > > > >> >> > >> >> > other > >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of development > + and > >> > > the > >> > > > > > > ability > >> > > > > > > > > so > >> > > > > > > > > >> > > tokens > >> > > > > > > > > >> > > > >> will > >> > > > > > > > > >> > > > >> >> > survive > >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling restart/cluster > >> > > failure. > >> > > > > if a > >> > > > > > > > > >> majority > >> > > > > > > > > >> > > > agrees > >> > > > > > > > > >> > > > >> the > >> > > > > > > > > >> > > > >> >> > added > >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have controller > >> > > forwarding > >> > > > > > keys > >> > > > > > > to > >> > > > > > > > > all > >> > > > > > > > > >> > > > broker is > >> > > > > > > > > >> > > > >> >> > justified > >> > > > > > > > > >> > > > >> >> > >> >> > as > >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter security , > I am > >> > > fine > >> > > > > > with > >> > > > > > > > that > >> > > > > > > > > >> option > >> > > > > > > > > >> > > > too. > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not > support SSL > >> > > we > >> > > > > can > >> > > > > > > not > >> > > > > > > > > >> store > >> > > > > > > > > >> > > > master > >> > > > > > > > > >> > > > >> keys > >> > > > > > > > > >> > > > >> >> > in > >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys will > be > >> > > exposed > >> > > > > on > >> > > > > > > > wire. > >> > > > > > > > > To > >> > > > > > > > > >> > > > support > >> > > > > > > > > >> > > > >> >> > rotation > >> > > > > > > > > >> > > > >> >> > >> >> > without affecting current > clients is > >> > > > > > > something I > >> > > > > > > > > >> need to > >> > > > > > > > > >> > > > put > >> > > > > > > > > >> > > > >> more > >> > > > > > > > > >> > > > >> >> > thought > >> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal > assumes the > >> > > > > rotation > >> > > > > > > > will > >> > > > > > > > > >> > > > invalidate > >> > > > > > > > > >> > > > >> all > >> > > > > > > > > >> > > > >> >> > current > >> > > > > > > > > >> > > > >> >> > >> >> > tokens. > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > I request committers to also > review > >> > > and > >> > > > > post > >> > > > > > > > their > >> > > > > > > > > >> > > comments > >> > > > > > > > > >> > > > >> so we > >> > > > > > > > > >> > > > >> >> > can > >> > > > > > > > > >> > > > >> >> > >> >> > make > >> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP. > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > Thanks > >> > > > > > > > > >> > > > >> >> > >> >> > Parth > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39 > AM, > >> > > Ashish > >> > > > > > Singh > >> > > > > > > < > >> > > > > > > > > >> > > > >> asi...@cloudera.com > >> > > > > > > > > >> > > > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > wrote: > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at > 11:26 AM, > >> > > > > Harsha < > >> > > > > > > > > >> > > > ka...@harsha.io> > >> > > > > > > > > >> > > > >> >> > wrote: > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two discussion > >> > > threads > >> > > > on > >> > > > > > > this > >> > > > > > > > > KIP. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response from > >> > Jitendra > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large > number of > >> > > > > clients > >> > > > > > > that > >> > > > > > > > > are > >> > > > > > > > > >> > > > running > >> > > > > > > > > >> > > > >> all > >> > > > > > > > > >> > > > >> >> > over the > >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that authenticate > with > >> > > Kafka > >> > > > > > > > brokers, > >> > > > > > > > > >> is very > >> > > > > > > > > >> > > > >> similar > >> > > > > > > > > >> > > > >> >> > to the > >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of large > number > >> > of > >> > > > > tasks > >> > > > > > > > > running > >> > > > > > > > > >> > > across > >> > > > > > > > > >> > > > >> the > >> > > > > > > > > >> > > > >> >> > cluster > >> > > > > > > > > >> > > > >> >> > >> >> that > >> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication to Hdfs > >> > > > Namenode. > >> > > > > > > > > >> Therefore, the > >> > > > > > > > > >> > > > >> >> > delegation token > >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem like a > good > >> > fit > >> > > > for > >> > > > > > > this > >> > > > > > > > > use > >> > > > > > > > > >> case > >> > > > > > > > > >> > > > as we > >> > > > > > > > > >> > > > >> >> > have seen > >> > > > > > > > > >> > > > >> >> > >> >> it > >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large scale in > HDFS > >> > and > >> > > > > YARN. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed design is > very > >> > much > >> > > > > > inline > >> > > > > > > > with > >> > > > > > > > > >> Hadoop > >> > > > > > > > > >> > > > >> >> > approach. A few > >> > > > > > > > > >> > > > >> >> > >> >> > > > comments: > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys want to > allow > >> > > > > > infinite > >> > > > > > > > > >> renewable > >> > > > > > > > > >> > > > >> lifetime > >> > > > > > > > > >> > > > >> >> > for a > >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts a > token > >> > to a > >> > > > max > >> > > > > > > life > >> > > > > > > > > time > >> > > > > > > > > >> > > > (default > >> > > > > > > > > >> > > > >> 7 > >> > > > > > > > > >> > > > >> >> > days). A > >> > > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability is > >> > believed > >> > > to > >> > > > > > > > increase > >> > > > > > > > > >> with > >> > > > > > > > > >> > > > time. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having infinite > >> > > lifetime > >> > > > > > might > >> > > > > > > > not > >> > > > > > > > > >> be the > >> > > > > > > > > >> > > > best > >> > > > > > > > > >> > > > >> idea. > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the > tokens > >> > are > >> > > > > stored > >> > > > > > > in > >> > > > > > > > > >> zookeeper > >> > > > > > > > > >> > > > as > >> > > > > > > > > >> > > > >> well, > >> > > > > > > > > >> > > > >> >> > and > >> > > > > > > > > >> > > > >> >> > >> >> can > >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This is > clever > >> > > as > >> > > > it > >> > > > > > can > >> > > > > > > > > allow > >> > > > > > > > > >> > > > >> replacing the > >> > > > > > > > > >> > > > >> >> > tokens > >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of max > life > >> > > time, > >> > > > > and > >> > > > > > > > > clients > >> > > > > > > > > >> can > >> > > > > > > > > >> > > > >> download > >> > > > > > > > > >> > > > >> >> > new > >> > > > > > > > > >> > > > >> >> > >> >> tokens > >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It > shouldn't be > >> > a > >> > > > big > >> > > > > > load > >> > > > > > > > on > >> > > > > > > > > >> > > zookeeper > >> > > > > > > > > >> > > > >> as a > >> > > > > > > > > >> > > > >> >> > client > >> > > > > > > > > >> > > > >> >> > >> >> will > >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new token > once in > >> > > > > several > >> > > > > > > > days. > >> > > > > > > > > >> In this > >> > > > > > > > > >> > > > >> approach > >> > > > > > > > > >> > > > >> >> > you > >> > > > > > > > > >> > > > >> >> > >> >> don't > >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime on > the > >> > > token > >> > > > > even > >> > > > > > > for > >> > > > > > > > > >> long > >> > > > > > > > > >> > > > running > >> > > > > > > > > >> > > > >> >> > clients. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password are > >> > > generated > >> > > > > > using > >> > > > > > > a > >> > > > > > > > > >> master > >> > > > > > > > > >> > > key. > >> > > > > > > > > >> > > > >> The > >> > > > > > > > > >> > > > >> >> > master > >> > > > > > > > > >> > > > >> >> > >> >> key > >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be periodically > >> > > changed. > >> > > > > In > >> > > > > > > > > Hadoop, > >> > > > > > > > > >> the > >> > > > > > > > > >> > > > >> default > >> > > > > > > > > >> > > > >> >> > renewal > >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.? > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require > brokers > >> > > > > > maintaining > >> > > > > > > a > >> > > > > > > > > >> list of X > >> > > > > > > > > >> > > > most > >> > > > > > > > > >> > > > >> >> > recent > >> > > > > > > > > >> > > > >> >> > >> >> master > >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will have to > be > >> > > > > persisted > >> > > > > > > > > >> somewhere, as > >> > > > > > > > > >> > > > if a > >> > > > > > > > > >> > > > >> >> > broker > >> > > > > > > > > >> > > > >> >> > >> >> goes > >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to get > that list > >> > > > again > >> > > > > > and > >> > > > > > > > > >> storing > >> > > > > > > > > >> > > > master > >> > > > > > > > > >> > > > >> keys > >> > > > > > > > > >> > > > >> >> > on ZK > >> > > > > > > > > >> > > > >> >> > >> >> is > >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea. However, > if a > >> > > > broker > >> > > > > > goes > >> > > > > > > > > down > >> > > > > > > > > >> then > >> > > > > > > > > >> > > we > >> > > > > > > > > >> > > > >> have > >> > > > > > > > > >> > > > >> >> > much > >> > > > > > > > > >> > > > >> >> > >> >> bigger > >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and > client can > >> > > > always > >> > > > > > > > > >> > > re-authenticate > >> > > > > > > > > >> > > > is > >> > > > > > > > > >> > > > >> such > >> > > > > > > > > >> > > > >> >> > >> >> events. > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a > look at > >> > > other > >> > > > > > > > > >> alternatives > >> > > > > > > > > >> > > this > >> > > > > > > > > >> > > > >> list has > >> > > > > > > > > >> > > > >> >> > >> >> > > suggested? > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough > proposal, > >> > > > great > >> > > > > > > work!" > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at > 10:28 > >> > PM, > >> > > > Gwen > >> > > > > > > > Shapira > >> > > > > > > > > >> wrote: > >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me. > Thanks! > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at > 9:25 > >> > PM, > >> > > > > > Harsha < > >> > > > > > > > > >> > > > ka...@harsha.io > >> > > > > > > > > >> > > > >> > > >> > > > > > > > > >> > > > >> >> > wrote: > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need any > release > >> > > > > vehicle > >> > > > > > > but > >> > > > > > > > > >> still the > >> > > > > > > > > >> > > > >> work can > >> > > > > > > > > >> > > > >> >> > move > >> > > > > > > > > >> > > > >> >> > >> >> > > > forward. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is > interested in > >> > the > >> > > > KIP > >> > > > > > > > please > >> > > > > > > > > >> do the > >> > > > > > > > > >> > > > >> review and > >> > > > > > > > > >> > > > >> >> > >> >> provide > >> > > > > > > > > >> > > > >> >> > >> >> > > the > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016, at > >> > 04:59 > >> > > > PM, > >> > > > > > > Ismael > >> > > > > > > > > >> Juma > >> > > > > > > > > >> > > > wrote: > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it would > be > >> > good > >> > > > to > >> > > > > > have > >> > > > > > > > > more > >> > > > > > > > > >> time > >> > > > > > > > > >> > > to > >> > > > > > > > > >> > > > >> review > >> > > > > > > > > >> > > > >> >> > and > >> > > > > > > > > >> > > > >> >> > >> >> > > discuss > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016 at > >> > 12:55 > >> > > > AM, > >> > > > > > Gwen > >> > > > > > > > > >> Shapira < > >> > > > > > > > > >> > > > >> >> > >> >> g...@confluent.io> > >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote: > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48 > depends on > >> > > > KIP-43, > >> > > > > > > which > >> > > > > > > > > is > >> > > > > > > > > >> > > > already a > >> > > > > > > > > >> > > > >> bit > >> > > > > > > > > >> > > > >> >> > of a > >> > > > > > > > > >> > > > >> >> > >> >> risk > >> > > > > > > > > >> > > > >> >> > >> >> > > for > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next release - > any > >> > > chance > >> > > > > we > >> > > > > > > can > >> > > > > > > > > >> delay > >> > > > > > > > > >> > > > >> delegation > >> > > > > > > > > >> > > > >> >> > tokens > >> > > > > > > > > >> > > > >> >> > >> >> to > >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1? > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the community > >> > working > >> > > > on a > >> > > > > > > > release > >> > > > > > > > > >> every > >> > > > > > > > > >> > > 3 > >> > > > > > > > > >> > > > >> month, > >> > > > > > > > > >> > > > >> >> > this > >> > > > > > > > > >> > > > >> >> > >> >> is not > >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26, > 2016 at > >> > > 5:11 > >> > > > > PM, > >> > > > > > > > Ashish > >> > > > > > > > > >> Singh > >> > > > > > > > > >> > > < > >> > > > > > > > > >> > > > >> >> > >> >> > > asi...@cloudera.com> > >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote: > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again for > the > >> > > > awesome > >> > > > > > > write > >> > > > > > > > > up. > >> > > > > > > > > >> > > > Following > >> > > > > > > > > >> > > > >> our > >> > > > > > > > > >> > > > >> >> > >> >> discussion > >> > > > > > > > > >> > > > >> >> > >> >> > > > from the > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think it > will > >> > be > >> > > > > easier > >> > > > > > > to > >> > > > > > > > > >> compare > >> > > > > > > > > >> > > > >> various > >> > > > > > > > > >> > > > >> >> > >> >> alternatives > >> > > > > > > > > >> > > > >> >> > >> >> > > > if they > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed together. > I am > >> > > > stating > >> > > > > > > > below a > >> > > > > > > > > >> few > >> > > > > > > > > >> > > > >> >> > alternatives along > >> > > > > > > > > >> > > > >> >> > >> >> > > with > >> > > > > > > > > >> > > > >> >> > >> >> > > > a the > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current proposal. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current proposal) > >> > Store > >> > > > > > > Delegation > >> > > > > > > > > >> Token, > >> > > > > > > > > >> > > DT, > >> > > > > > > > > >> > > > >> on ZK. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client > >> > > authenticates > >> > > > > > with a > >> > > > > > > > > >> broker. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a > client is > >> > > > > > > > authenticated, > >> > > > > > > > > >> it > >> > > > > > > > > >> > > will > >> > > > > > > > > >> > > > >> make a > >> > > > > > > > > >> > > > >> >> > broker > >> > > > > > > > > >> > > > >> >> > >> >> side > >> > > > > > > > > >> > > > >> >> > >> >> > > > call to > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a > delegation > >> > > > token. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker > >> > > generates > >> > > > a > >> > > > > > > shared > >> > > > > > > > > >> secret > >> > > > > > > > > >> > > > based > >> > > > > > > > > >> > > > >> on > >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Password/Secret > >> > shared > >> > > > > > between > >> > > > > > > > all > >> > > > > > > > > >> > > brokers, > >> > > > > > > > > >> > > > >> >> > randomly > >> > > > > > > > > >> > > > >> >> > >> >> > > generated > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId). > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Broker > stores > >> > this > >> > > > > token > >> > > > > > in > >> > > > > > > > its > >> > > > > > > > > >> in > >> > > > > > > > > >> > > > memory > >> > > > > > > > > >> > > > >> cache. > >> > > > > > > > > >> > > > >> >> > >> >> Broker > >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the > DelegationToken > >> > > > > without > >> > > > > > > the > >> > > > > > > > > >> hmac in > >> > > > > > > > > >> > > the > >> > > > > > > > > >> > > > >> >> > zookeeper. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All brokers > will > >> > > > have a > >> > > > > > > cache > >> > > > > > > > > >> backed > >> > > > > > > > > >> > > by > >> > > > > > > > > >> > > > >> >> > zookeeper so > >> > > > > > > > > >> > > > >> >> > >> >> they > >> > > > > > > > > >> > > > >> >> > >> >> > > > will all > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get notified > >> > whenever > >> > > a > >> > > > > new > >> > > > > > > > token > >> > > > > > > > > is > >> > > > > > > > > >> > > > >> generated and > >> > > > > > > > > >> > > > >> >> > they > >> > > > > > > > > >> > > > >> >> > >> >> will > >> > > > > > > > > >> > > > >> >> > >> >> > > > update > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local cache > whenever > >> > > > token > >> > > > > > > state > >> > > > > > > > > >> changes. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Broker > returns > >> > the > >> > > > > token > >> > > > > > to > >> > > > > > > > > >> Client. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues > and > >> > fixes > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Probable > race > >> > > > > condition, > >> > > > > > > > client > >> > > > > > > > > >> tries > >> > > > > > > > > >> > > to > >> > > > > > > > > >> > > > >> >> > authenticate > >> > > > > > > > > >> > > > >> >> > >> >> with > >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is yet to > be > >> > > > updated > >> > > > > > with > >> > > > > > > > the > >> > > > > > > > > >> newly > >> > > > > > > > > >> > > > >> generated > >> > > > > > > > > >> > > > >> >> > DT. > >> > > > > > > > > >> > > > >> >> > >> >> This > >> > > > > > > > > >> > > > >> >> > >> >> > > can > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt with > making > >> > > > > dtRequest > >> > > > > > > > block > >> > > > > > > > > >> until > >> > > > > > > > > >> > > all > >> > > > > > > > > >> > > > >> >> > brokers have > >> > > > > > > > > >> > > > >> >> > >> >> > > > updated > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache. Zk > barrier or > >> > > > > similar > >> > > > > > > > > >> mechanism > >> > > > > > > > > >> > > can > >> > > > > > > > > >> > > > be > >> > > > > > > > > >> > > > >> used. > >> > > > > > > > > >> > > > >> >> > >> >> However, > >> > > > > > > > > >> > > > >> >> > >> >> > > > all such > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > mechanisms will > >> > > increase > >> > > > > > > > > complexity. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Using a > static > >> > > secret > >> > > > > key > >> > > > > > > > from > >> > > > > > > > > >> config > >> > > > > > > > > >> > > > >> file. Will > >> > > > > > > > > >> > > > >> >> > >> >> require > >> > > > > > > > > >> > > > >> >> > >> >> > > yet > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config and > uses a > >> > > static > >> > > > > > > secret > >> > > > > > > > > >> key. It > >> > > > > > > > > >> > > is > >> > > > > > > > > >> > > > >> advised > >> > > > > > > > > >> > > > >> >> > to > >> > > > > > > > > >> > > > >> >> > >> >> rotate > >> > > > > > > > > >> > > > >> >> > >> >> > > > secret > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically. > This > >> > can > >> > > > be > >> > > > > > > > avoided > >> > > > > > > > > >> with > >> > > > > > > > > >> > > > >> controller > >> > > > > > > > > >> > > > >> >> > >> >> generating > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing to > brokers > >> > > > > > > periodically. > >> > > > > > > > > >> However, > >> > > > > > > > > >> > > > >> this will > >> > > > > > > > > >> > > > >> >> > >> >> require > >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain > certain > >> > > counts > >> > > > of > >> > > > > > > > > >> secretKeys. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 1) > Have > >> > > > > controller > >> > > > > > > > > >> generate > >> > > > > > > > > >> > > > >> delegation > >> > > > > > > > > >> > > > >> >> > token. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client > >> > > authenticates > >> > > > > > with a > >> > > > > > > > > >> broker. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a > client is > >> > > > > > > > authenticated, > >> > > > > > > > > >> it > >> > > > > > > > > >> > > will > >> > > > > > > > > >> > > > >> make a > >> > > > > > > > > >> > > > >> >> > broker > >> > > > > > > > > >> > > > >> >> > >> >> side > >> > > > > > > > > >> > > > >> >> > >> >> > > > call to > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a > delegation > >> > > > token. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. Broker > forwards > >> > the > >> > > > > > request > >> > > > > > > > to > >> > > > > > > > > >> > > > controller. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Controller > >> > > generates > >> > > > a > >> > > > > DT > >> > > > > > > and > >> > > > > > > > > >> > > broadcasts > >> > > > > > > > > >> > > > >> to all > >> > > > > > > > > >> > > > >> >> > >> >> brokers. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. Broker > stores > >> > this > >> > > > > token > >> > > > > > in > >> > > > > > > > its > >> > > > > > > > > >> memory > >> > > > > > > > > >> > > > >> cache. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Controller > >> > responds > >> > > > to > >> > > > > > > > broker’s > >> > > > > > > > > >> DT > >> > > > > > > > > >> > > req. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7. Broker > returns > >> > the > >> > > > > token > >> > > > > > to > >> > > > > > > > > >> Client. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues > and > >> > fixes > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We will > have to > >> > add > >> > > > new > >> > > > > > > APIs > >> > > > > > > > to > >> > > > > > > > > >> > > support > >> > > > > > > > > >> > > > >> >> > controller > >> > > > > > > > > >> > > > >> >> > >> >> pushing > >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers on top > of > >> > the > >> > > > > > minimal > >> > > > > > > > APIs > >> > > > > > > > > >> that > >> > > > > > > > > >> > > are > >> > > > > > > > > >> > > > >> >> > currently > >> > > > > > > > > >> > > > >> >> > >> >> > > proposed. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We will > also have > >> > > to > >> > > > > add > >> > > > > > > APIs > >> > > > > > > > > to > >> > > > > > > > > >> > > support > >> > > > > > > > > >> > > > >> the > >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping > >> > > > > > > > > >> > > > >> >> > >> >> > > > case, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a new > broker > >> > > comes > >> > > > up > >> > > > > > it > >> > > > > > > > will > >> > > > > > > > > >> have > >> > > > > > > > > >> > > to > >> > > > > > > > > >> > > > >> get all > >> > > > > > > > > >> > > > >> >> > >> >> delegation > >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the controller. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In > catastrophic > >> > > > > failures > >> > > > > > > > where > >> > > > > > > > > >> all > >> > > > > > > > > >> > > > brokers > >> > > > > > > > > >> > > > >> go > >> > > > > > > > > >> > > > >> >> > down, > >> > > > > > > > > >> > > > >> >> > >> >> the > >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if > >> > > servers > >> > > > > are > >> > > > > > > > > >> restarted as > >> > > > > > > > > >> > > > >> tokens > >> > > > > > > > > >> > > > >> >> > are not > >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this > happens, > >> > then > >> > > > > there > >> > > > > > > are > >> > > > > > > > > more > >> > > > > > > > > >> > > > important > >> > > > > > > > > >> > > > >> >> > things to > >> > > > > > > > > >> > > > >> >> > >> >> > > worry > >> > > > > > > > > >> > > > >> >> > >> >> > > > about > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is > better > >> > to > >> > > > > > > > > >> re-authenticate. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 2) > Do not > >> > > > > > distribute > >> > > > > > > > DT > >> > > > > > > > > to > >> > > > > > > > > >> > > other > >> > > > > > > > > >> > > > >> brokers > >> > > > > > > > > >> > > > >> >> > at > >> > > > > > > > > >> > > > >> >> > >> >> all. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client > >> > > authenticates > >> > > > > > with a > >> > > > > > > > > >> broker. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a > client is > >> > > > > > > > authenticated, > >> > > > > > > > > >> it > >> > > > > > > > > >> > > will > >> > > > > > > > > >> > > > >> make a > >> > > > > > > > > >> > > > >> >> > broker > >> > > > > > > > > >> > > > >> >> > >> >> side > >> > > > > > > > > >> > > > >> >> > >> >> > > > call to > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a > delegation > >> > > > token. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker > >> > > generates > >> > > > DT > >> > > > > > of > >> > > > > > > > > form, > >> > > > > > > > > >> > > [hmac + > >> > > > > > > > > >> > > > >> (owner, > >> > > > > > > > > >> > > > >> >> > >> >> renewer, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maxLifeTime, > id, > >> > hmac, > >> > > > > > > > > >> expirationTime)] > >> > > > > > > > > >> > > and > >> > > > > > > > > >> > > > >> passes > >> > > > > > > > > >> > > > >> >> > back > >> > > > > > > > > >> > > > >> >> > >> >> this > >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is > generated > >> > via > >> > > > > > > > > >> {HMAC-SHA256(owner, > >> > > > > > > > > >> > > > >> renewer, > >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime) > >> > using > >> > > > > > > > SecretKey}. > >> > > > > > > > > >> Note > >> > > > > > > > > >> > > that > >> > > > > > > > > >> > > > >> all > >> > > > > > > > > >> > > > >> >> > brokers > >> > > > > > > > > >> > > > >> >> > >> >> have > >> > > > > > > > > >> > > > >> >> > >> >> > > > this > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Client then > goes > >> > to > >> > > > any > >> > > > > > > > broker > >> > > > > > > > > >> and to > >> > > > > > > > > >> > > > >> >> > authenticate > >> > > > > > > > > >> > > > >> >> > >> >> sends > >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker > recalculates > >> > > hmac > >> > > > > > using > >> > > > > > > > > >> (owner, > >> > > > > > > > > >> > > > >> renewer, > >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime, > >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > expirationTime) info > >> > > > from > >> > > > > DT > >> > > > > > > and > >> > > > > > > > > its > >> > > > > > > > > >> > > > >> SecretKey. If > >> > > > > > > > > >> > > > >> >> > it > >> > > > > > > > > >> > > > >> >> > >> >> matches > >> > > > > > > > > >> > > > >> >> > >> >> > > > with > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT, client is > >> > > > > authenticated. > >> > > > > > > > Yes, > >> > > > > > > > > >> it will > >> > > > > > > > > >> > > > do > >> > > > > > > > > >> > > > >> other > >> > > > > > > > > >> > > > >> >> > >> >> obvious > >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > timestamp > expiry and > >> > > > such. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that secret > key > >> > will > >> > > > be > >> > > > > > > > > generated > >> > > > > > > > > >> by > >> > > > > > > > > >> > > > >> controller > >> > > > > > > > > >> > > > >> >> > and > >> > > > > > > > > >> > > > >> >> > >> >> passed > >> > > > > > > > > >> > > > >> >> > >> >> > > to > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues > and > >> > fixes > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How to > delete a > >> > DT? > >> > > > > Yes, > >> > > > > > > that > >> > > > > > > > > is > >> > > > > > > > > >> a > >> > > > > > > > > >> > > > downside > >> > > > > > > > > >> > > > >> >> > here. > >> > > > > > > > > >> > > > >> >> > >> >> However, > >> > > > > > > > > >> > > > >> >> > >> >> > > > this can > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be handled with > >> > > brokers > >> > > > > > > > > maintaining > >> > > > > > > > > >> a > >> > > > > > > > > >> > > > >> blacklist of > >> > > > > > > > > >> > > > >> >> > DTs, > >> > > > > > > > > >> > > > >> >> > >> >> DTs > >> > > > > > > > > >> > > > >> >> > >> >> > > > from this > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be removed > after > >> > > > > expiry. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In > catastrophic > >> > > > > failures > >> > > > > > > > where > >> > > > > > > > > >> all > >> > > > > > > > > >> > > > brokers > >> > > > > > > > > >> > > > >> go > >> > > > > > > > > >> > > > >> >> > down, > >> > > > > > > > > >> > > > >> >> > >> >> the > >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if > >> > > servers > >> > > > > are > >> > > > > > > > > >> restarted as > >> > > > > > > > > >> > > > >> tokens > >> > > > > > > > > >> > > > >> >> > are not > >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this > happens, > >> > then > >> > > > > there > >> > > > > > > are > >> > > > > > > > > more > >> > > > > > > > > >> > > > important > >> > > > > > > > > >> > > > >> >> > things to > >> > > > > > > > > >> > > > >> >> > >> >> > > worry > >> > > > > > > > > >> > > > >> >> > >> >> > > > about > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is > better > >> > to > >> > > > > > > > > >> re-authenticate. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26, > 2016 at > >> > > > 1:58 > >> > > > > > PM, > >> > > > > > > > > Parth > >> > > > > > > > > >> > > > >> Brahmbhatt < > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > pbrahmbh...@hortonworks.com> > >> > > > > > > > wrote: > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed > KIP-48 so > >> > > we > >> > > > > can > >> > > > > > > > offer > >> > > > > > > > > >> hadoop > >> > > > > > > > > >> > > > like > >> > > > > > > > > >> > > > >> >> > delegation > >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You can > review > >> > > the > >> > > > > > design > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > >> > > > > > > > > >> > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > . > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP depends > on > >> > > KIP-43 > >> > > > > and > >> > > > > > > we > >> > > > > > > > > >> have also > >> > > > > > > > > >> > > > >> >> > discussed an > >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed design > here< > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > >> > > > > > > > > >> > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800 > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >. > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > -- > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards, > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > -- > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > Regards, > >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > >> > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > -- > >> > > > > > Regards, > >> > > > > > > >> > > > > > Rajini > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > -- > >> > > > Regards, > >> > > > > >> > > > Rajini > >> > > > > >> > > > >> > > >> > >> > >> > >> -- > >> Liquan Pei > >> Software Engineer, Confluent Inc >
