That is what the hadoop echo system uses so no good reason really. We could change it to whatever is the newest recommended standard is.
Thanks Parth On Thu, May 19, 2016 at 3:33 AM, Ismael Juma <ism...@juma.me.uk> wrote: > Hi Parth, > > Thanks for the KIP. I only started reviewing this and may have additional > questions later. The immediate question that came to mind is our choice of > "DIGEST-MD5" even though it's marked as OBSOLETE in the IANA Registry of > SASL mechanisms and the original RFC (2831) has been moved to Historic > status: > > https://tools.ietf.org/html/rfc6331 > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml > > What is the reasoning behind that choice? > > Thanks, > Ismael > > On Fri, May 13, 2016 at 11:29 PM, Gwen Shapira <g...@confluent.io> wrote: > > > Also comments inline :) > > > > > * I want to emphasize that even though delegation tokens are a Hadoop > > > innovation, I feel very strongly about not adding dependency on Hadoop > > > when implementing delegation tokens for Kafka. The KIP doesn't imply > > > such dependency, but if you can clarify... > > > > > > > > > *No hadoop dependency.* > > > > Yay! Just add this to the KIP so no one will read the KIP and panic > > three weeks before the next release... > > > > > * Can we get delegation token at any time after authenticating? only > > > immediately after? > > > > > > > > > *As long as you are authenticated you can get delegation tokens. We > need > > to > > > discuss if a client authenticated using delegation token, can also > > acquire > > > delegation token again or not. Also there is the question of do we > allow > > > anyone to acquire delegation token or we want specific ACLs (I think > its > > an > > > overkill.)* > > > > I agree that ACLs is an overkill. > > > > I think we are debating two options: Either require Kerberos auth for > > renewal or require non-owners to renew. > > I *think* the latter is simpler (it basically require a "job master" > > to take responsibility for the renewal, it will have its own identity > > anyway and I think this is the correct design pattern anyway. For > > storm, I'd expect Nimbus to coordinate renewals?), but it is hard to > > debate simplicity without looking at the code changes required. If you > > have a draft of how the "require Kerberos" will look in Kafka code, > > I'll be happy to take a look. > > > > > * My understanding is that tokens will propagate via ZK but without > > > additional changes to UpdateMetadata protocol, correct? Clients > > > currently don't retry on SASL auth failure (IIRC), but since the > > > tokens propagate between brokers asynch, we will need to retry a bit > > > to avoid clients failing auth due to timing issues. > > > > > > *I am considering 2 alternatives right now. The current documented > > approach > > > is zookeeper based and it does not require any changes to > UpdateMetadata > > > protocol. An alternative approach can remove zookeeper dependency as > well > > > but we can discuss that in KIP discussion call.* > > > > Oooh! Sounds interesting. Do you want to ping Jun to arrange a call? > > > > > * I liked Ashish's suggestion of having just the controller issue the > > > delegation tokens, to avoid syncing a shared secret. Not sure if we > > > want to continue the discussion here or on the wiki. I think that we > > > can decouple the problem of "token distribution" from "shared secret > > > distribution" and use the controller as the only token generator to > > > solve the second issue, while still using ZK async to distribute > > > tokens. > > > > > > > > > *As mentioned in the previous Email I am fine with that approach as > long > > as > > > we agree that the extra complexity of adding/updating APIS adds enough > > > value. The advantage with the controller approach is secret rotation > can > > be > > > automated,frequent and would not require deployment. * > > > > Can you detail the extra complexity (or point me to the email I > > missed?) - which Apis are required? > > As far as I can tell, clients can already find the controller from > > metadata. I'm a bit more concerned about controller load. > > > > > > > > * While I like the idea of forcing kerberos auth for renwal, I think > > > it mixes the transport layer the the request content in a pretty ugly > > > way. Perhaps limiting renewer to non-owner is better. > > > > > > *I feel this is a necessary evil. While this will make the kafka code > > > pretty straight forward , forcing renewer to non-owner pushes the code > > > ugliness to client and makes it even harder to integrate. * > > > > As mentioned before, I don't think the "renewal by other" approach is > > that ugly for the clients we expect to use delegation tokens since > > they will have an app-master of some sort who requested the token to > > begin with. > > > > > > > > The response for my question on how multiple identities will be > > > handled wasn't super clear to me - AFAIK, we don't authenticate each > > > request, we authenticate connections. > > > > > > *We authenticate connections, and only when they are being established. > > Let > > > me try to phrase this as a question, in absence of delegation tokens if > > we > > > had to support the use case using user TGT's how would we do it? My > point > > > was it would be no different with delegation tokens. The use case you > are > > > describing seems more like impersonation.* > > > > Yeah, I thought that one of the things that delegation tokens handled. > > Maybe I got it wrong :) > > > > Thanks for the detailed answers. > > > > Gwen > > > > > > > Thanks > > > Parth > > > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen Shapira <g...@confluent.io> > > wrote: > > > > > >> Hi Parth and Harsha, > > >> > > >> Few more comments: > > >> > > >> * The API / RequestResponse section doesn't seem to have good > > >> description of the changes to the Kafka Protocol. Sounds like you are > > >> proposing new DelegationTokenRequest and RenewTokenRequest (and > > >> matching responses), without detailing the contents of the requests > > >> and responses? Or rather, you show the class interface, but not the > > >> underlying protocol. This could be seen as an implementation detail, > > >> but since the binary protocol is what we provide to non-Java clients, > > >> we need to show the changes there. > > >> > > >> * getDelegationToken sounds like delegationTokenRequestHandler? Is it > > >> planned to be part of KafkaApi? or Client? Its unclear... > > >> > > >> * I want to emphasize that even though delegation tokens are a Hadoop > > >> innovation, I feel very strongly about not adding dependency on Hadoop > > >> when implementing delegation tokens for Kafka. The KIP doesn't imply > > >> such dependency, but if you can clarify... > > >> > > >> * Can we get delegation token at any time after authenticating? only > > >> immediately after? > > >> > > >> * My understanding is that tokens will propagate via ZK but without > > >> additional changes to UpdateMetadata protocol, correct? Clients > > >> currently don't retry on SASL auth failure (IIRC), but since the > > >> tokens propagate between brokers asynch, we will need to retry a bit > > >> to avoid clients failing auth due to timing issues. > > >> > > >> * Strongly agreeing on clients not touching ZK directly :) > > >> > > >> * I liked Ashish's suggestion of having just the controller issue the > > >> delegation tokens, to avoid syncing a shared secret. Not sure if we > > >> want to continue the discussion here or on the wiki. I think that we > > >> can decouple the problem of "token distribution" from "shared secret > > >> distribution" and use the controller as the only token generator to > > >> solve the second issue, while still using ZK async to distribute > > >> tokens. > > >> > > >> * I am also uncomfortable with infinite lifetime of tokens (and hoped > > >> to hear from others in the community) - but having the option and > > >> documenting it as less secure, allows users to configure their system > > >> as the wish. > > >> > > >> * While I like the idea of forcing kerberos auth for renwal, I think > > >> it mixes the transport layer the the request content in a pretty ugly > > >> way. Perhaps limiting renewer to non-owner is better. > > >> > > >> Things I'd still like to see: > > >> > > >> * More detailed explanation on what we plan to do for the java clients > > >> specifically - new configuration? new APIs? > > >> The response for my question on how multiple identities will be > > >> handled wasn't super clear to me - AFAIK, we don't authenticate each > > >> request, we authenticate connections. > > >> > > >> * Alternatives: Delegation tokens are only used in the Hadoop > > >> ecosystem. I'm wondering if there are alternatives in other ecosystems > > >> (Mesos? Tachyon? Cassandra?) and whether there are some advantages > > >> there. > > >> > > >> Gwen > > >> > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha <ka...@harsha.io> wrote: > > >> > Hi Gwen, > > >> > Can you look at Parth's last reply. Does it answer your > > >> > concerns. > > >> > > > >> > Thanks, > > >> > Harsha > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth brahmbhatt wrote: > > >> >> Thanks for reviewing Gwen. The wiki already has details on token > > >> >> expiration > > >> >> under token acquisition process > > >> >> < > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition > > >> >. > > >> >> Current proposal is that tokens will expire based on a server side > > >> >> configuration (default 24 hours) unless renewed. Renewal is only > > allowed > > >> >> until the max life time of token. Alternatively we could also make > > that > > >> >> an > > >> >> optional param and the server side default can serve as the upper > > bound. > > >> >> > > >> >> To your second point it will be done exactly the same way we would > > >> >> support > > >> >> multiple keytabs. The calling client will have to put the tokens it > > >> wants > > >> >> to use in the subject instance and call produce/consume inside > > >> >> subject.doas. Each caller will have to keep track of its own > > subject. I > > >> >> will have to look at the code to see if we support this feature > right > > >> now > > >> >> but my understanding is delegation token shouldn't need any special > > >> >> treatment as its just another type of Credential in the subject. > > >> >> > > >> >> I would also like to know what is your opinion about infinite > renewal > > >> (my > > >> >> recommendation is to not support this), tokens renewing them > self(my > > >> >> recommendation is to not support this) and most importantly your > > choice > > >> >> between the alternatives listed on this thread > > >> >> < > > >> > > > http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism > > >> > > > >> >> ( I am leaning towards the alternative-2 minus controller > > distributing > > >> >> secret). Thanks again for reviewing. > > >> >> > > >> >> Thanks > > >> >> Parth > > >> >> > > >> >> > > >> >> > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen Shapira <g...@confluent.io> > > wrote: > > >> >> > > >> >> > Harsha, > > >> >> > > > >> >> > I was thinking of the Rest Proxy. I didn't see your design yet, > > but in > > >> >> > our proxy, we have a set of producers, which will serve multiple > > users > > >> >> > going through the proxy. Since these users will have different > > >> >> > privileges, they'll need to authenticate separately, and can't > > share a > > >> >> > token. > > >> >> > > > >> >> > Am I missing anything? > > >> >> > > > >> >> > Gwen > > >> >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM, Harsha <ka...@harsha.io> wrote: > > >> >> > > Gwen, > > >> >> > > On your second point. Can you describe a usecase > where > > >> >> > > mutliple clients ended up sharing a producer and > even > > if > > >> they > > >> >> > > do why can't they not use single token that producer > > >> >> > > captures. Why would we need multiple clients with > > >> different > > >> >> > > tokens sharing a single instance of producer. Also > in > > >> this > > >> >> > > case other clients have access all the tokens no? > > >> >> > > > > >> >> > > Thanks, > > >> >> > > Harsha > > >> >> > > > > >> >> > > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM, Gwen Shapira wrote: > > >> >> > >> Sorry for the delay: > > >> >> > >> > > >> >> > >> Two questions that we didn't see in the wiki: > > >> >> > >> 1. Is there an expiration for delegation tokens? Renewal? How > > do we > > >> >> > >> revoke them? > > >> >> > >> 2. If we want to use delegation tokens for "do-as" (say, > submit > > >> Storm > > >> >> > >> job as my user), we will need a producer for every job (we > can't > > >> share > > >> >> > >> them between multiple jobs running on same node), since we > only > > >> >> > >> authenticate when connecting. Is there a plan to change this > for > > >> >> > >> delegation tokens, in order to allow multiple users with > > different > > >> >> > >> tokens to share a client? > > >> >> > >> > > >> >> > >> Gwen > > >> >> > >> > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM, parth brahmbhatt > > >> >> > >> <brahmbhatt.pa...@gmail.com> wrote: > > >> >> > >> > Bumping this up one more time, can other committers review? > > >> >> > >> > > > >> >> > >> > Thanks > > >> >> > >> > Parth > > >> >> > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07 AM, Harsha <ka...@harsha.io> > > wrote: > > >> >> > >> > > > >> >> > >> >> Parth, > > >> >> > >> >> Overall current design looks good to me. I am +1 > on > > >> the > > >> >> > KIP. > > >> >> > >> >> > > >> >> > >> >> Gwen , Jun can you review this as well. > > >> >> > >> >> > > >> >> > >> >> -Harsha > > >> >> > >> >> > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57 AM, parth brahmbhatt wrote: > > >> >> > >> >> > Thanks for review Jitendra. > > >> >> > >> >> > > > >> >> > >> >> > I don't like the idea of infinite lifetime but I see the > > >> Streaming > > >> >> > use > > >> >> > >> >> > case. Even for Streaming use case I was hoping there will > > be > > >> some > > >> >> > notion > > >> >> > >> >> > of > > >> >> > >> >> > master/driver that can get new delegation tokens at fixed > > >> interval > > >> >> > and > > >> >> > >> >> > distribute to workers. If that is not the case for we can > > >> discuss > > >> >> > >> >> > delegation tokens renewing them self and the security > > >> implications > > >> >> > of the > > >> >> > >> >> > same. > > >> >> > >> >> > > > >> >> > >> >> > I did not want clients to fetch tokens from zookeeper, > > >> overall I > > >> >> > think > > >> >> > >> >> > its > > >> >> > >> >> > better if clients don't rely on our metadata store and I > > >> think we > > >> >> > are > > >> >> > >> >> > moving in that direction with all the KIP-4 improvements. > > I > > >> chose > > >> >> > >> >> > zookeeper as in this case the client will still just talk > > to > > >> >> > broker , its > > >> >> > >> >> > the brokers that will use zookeeper which we already do > > for a > > >> lot > > >> >> > of > > >> >> > >> >> > other > > >> >> > >> >> > usecases + ease of development + and the ability so > tokens > > >> will > > >> >> > survive > > >> >> > >> >> > even a rolling restart/cluster failure. if a majority > > agrees > > >> the > > >> >> > added > > >> >> > >> >> > complexity to have controller forwarding keys to all > > broker is > > >> >> > justified > > >> >> > >> >> > as > > >> >> > >> >> > it provides tighter security , I am fine with that option > > too. > > >> >> > >> >> > > > >> >> > >> >> > Given zookeeper does not support SSL we can not store > > master > > >> keys > > >> >> > in > > >> >> > >> >> > zookeeper as master keys will be exposed on wire. To > > support > > >> >> > rotation > > >> >> > >> >> > without affecting current clients is something I need to > > put > > >> more > > >> >> > thought > > >> >> > >> >> > in. My current proposal assumes the rotation will > > invalidate > > >> all > > >> >> > current > > >> >> > >> >> > tokens. > > >> >> > >> >> > > > >> >> > >> >> > I request committers to also review and post their > comments > > >> so we > > >> >> > can > > >> >> > >> >> > make > > >> >> > >> >> > progress on this KIP. > > >> >> > >> >> > > > >> >> > >> >> > Thanks > > >> >> > >> >> > Parth > > >> >> > >> >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39 AM, Ashish Singh < > > >> asi...@cloudera.com > > >> >> > > > > >> >> > >> >> > wrote: > > >> >> > >> >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at 11:26 AM, Harsha < > > ka...@harsha.io> > > >> >> > wrote: > > >> >> > >> >> > > > > >> >> > >> >> > > > Unifying the two discussion threads on this KIP. > > >> >> > >> >> > > > > > >> >> > >> >> > > > Here is the response from Jitendra > > >> >> > >> >> > > > > > >> >> > >> >> > > > "The need for a large number of clients that are > > running > > >> all > > >> >> > over the > > >> >> > >> >> > > > cluster that authenticate with Kafka brokers, is very > > >> similar > > >> >> > to the > > >> >> > >> >> > > > Hadoop use case of large number of tasks running > across > > >> the > > >> >> > cluster > > >> >> > >> >> that > > >> >> > >> >> > > > need authentication to Hdfs Namenode. Therefore, the > > >> >> > delegation token > > >> >> > >> >> > > > approach does seem like a good fit for this use case > > as we > > >> >> > have seen > > >> >> > >> >> it > > >> >> > >> >> > > > working at large scale in HDFS and YARN. > > >> >> > >> >> > > > > > >> >> > >> >> > > > The proposed design is very much inline with Hadoop > > >> >> > approach. A few > > >> >> > >> >> > > > comments: > > >> >> > >> >> > > > > > >> >> > >> >> > > > 1) Why do you guys want to allow infinite renewable > > >> lifetime > > >> >> > for a > > >> >> > >> >> > > > token? HDFS restricts a token to a max life time > > (default > > >> 7 > > >> >> > days). A > > >> >> > >> >> > > > token's vulnerability is believed to increase with > > time. > > >> >> > >> >> > > > > > >> >> > >> >> > > I agree that having infinite lifetime might not be the > > best > > >> idea. > > >> >> > >> >> > > > > >> >> > >> >> > > > > > >> >> > >> >> > > > 2) As I understand the tokens are stored in zookeeper > > as > > >> well, > > >> >> > and > > >> >> > >> >> can > > >> >> > >> >> > > > be updated there. This is clever as it can allow > > >> replacing the > > >> >> > tokens > > >> >> > >> >> > > > once they run out of max life time, and clients can > > >> download > > >> >> > new > > >> >> > >> >> tokens > > >> >> > >> >> > > > from zookeeper. It shouldn't be a big load on > zookeeper > > >> as a > > >> >> > client > > >> >> > >> >> will > > >> >> > >> >> > > > need to get a new token once in several days. In this > > >> approach > > >> >> > you > > >> >> > >> >> don't > > >> >> > >> >> > > > need infinite lifetime on the token even for long > > running > > >> >> > clients. > > >> >> > >> >> > > > > > >> >> > >> >> > > > 3) The token password are generated using a master > key. > > >> The > > >> >> > master > > >> >> > >> >> key > > >> >> > >> >> > > > should also be periodically changed. In Hadoop, the > > >> default > > >> >> > renewal > > >> >> > >> >> > > > period is 1 day.? > > >> >> > >> >> > > > > > >> >> > >> >> > > IIUC, this will require brokers maintaining a list of X > > most > > >> >> > recent > > >> >> > >> >> master > > >> >> > >> >> > > keys. This list will have to be persisted somewhere, as > > if a > > >> >> > broker > > >> >> > >> >> goes > > >> >> > >> >> > > down it will have to get that list again and storing > > master > > >> keys > > >> >> > on ZK > > >> >> > >> >> is > > >> >> > >> >> > > not the best idea. However, if a broker goes down then > we > > >> have > > >> >> > much > > >> >> > >> >> bigger > > >> >> > >> >> > > issue to deal with and client can always > re-authenticate > > is > > >> such > > >> >> > >> >> events. > > >> >> > >> >> > > > > >> >> > >> >> > > Did you happen to take a look at other alternatives > this > > >> list has > > >> >> > >> >> > > suggested? > > >> >> > >> >> > > > > >> >> > >> >> > > > > > >> >> > >> >> > > > Thanks for a thorough proposal, great work!" > > >> >> > >> >> > > > > > >> >> > >> >> > > > > > >> >> > >> >> > > > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at 10:28 PM, Gwen Shapira wrote: > > >> >> > >> >> > > > > Makes sense to me. Thanks! > > >> >> > >> >> > > > > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at 9:25 PM, Harsha < > > ka...@harsha.io > > >> > > > >> >> > wrote: > > >> >> > >> >> > > > > > It doesn't need any release vehicle but still the > > >> work can > > >> >> > move > > >> >> > >> >> > > > forward. > > >> >> > >> >> > > > > > If anyone is interested in the KIP please do the > > >> review and > > >> >> > >> >> provide > > >> >> > >> >> > > the > > >> >> > >> >> > > > > > comments. > > >> >> > >> >> > > > > > > > >> >> > >> >> > > > > > -Harsha > > >> >> > >> >> > > > > > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016, at 04:59 PM, Ismael Juma > > wrote: > > >> >> > >> >> > > > > >> I agree that it would be good to have more time > to > > >> review > > >> >> > and > > >> >> > >> >> > > discuss > > >> >> > >> >> > > > > >> KIP-48. > > >> >> > >> >> > > > > >> > > >> >> > >> >> > > > > >> Ismael > > >> >> > >> >> > > > > >> > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016 at 12:55 AM, Gwen Shapira < > > >> >> > >> >> g...@confluent.io> > > >> >> > >> >> > > > wrote: > > >> >> > >> >> > > > > >> > > >> >> > >> >> > > > > >> > Hi Team, > > >> >> > >> >> > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48 depends on KIP-43, which is > > already a > > >> bit > > >> >> > of a > > >> >> > >> >> risk > > >> >> > >> >> > > for > > >> >> > >> >> > > > > >> > the next release - any chance we can delay > > >> delegation > > >> >> > tokens > > >> >> > >> >> to > > >> >> > >> >> > > > Kafka > > >> >> > >> >> > > > > >> > 0.10.1? > > >> >> > >> >> > > > > >> > With the community working on a release every > 3 > > >> month, > > >> >> > this > > >> >> > >> >> is not > > >> >> > >> >> > > > a huge > > >> >> > >> >> > > > > >> > delay. > > >> >> > >> >> > > > > >> > > > >> >> > >> >> > > > > >> > Gwen > > >> >> > >> >> > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26, 2016 at 5:11 PM, Ashish Singh > < > > >> >> > >> >> > > asi...@cloudera.com> > > >> >> > >> >> > > > wrote: > > >> >> > >> >> > > > > >> > > Parth, > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > Thanks again for the awesome write up. > > Following > > >> our > > >> >> > >> >> discussion > > >> >> > >> >> > > > from the > > >> >> > >> >> > > > > >> > > JIRA, I think it will be easier to compare > > >> various > > >> >> > >> >> alternatives > > >> >> > >> >> > > > if they > > >> >> > >> >> > > > > >> > are > > >> >> > >> >> > > > > >> > > listed together. I am stating below a few > > >> >> > alternatives along > > >> >> > >> >> > > with > > >> >> > >> >> > > > a the > > >> >> > >> >> > > > > >> > > current proposal. > > >> >> > >> >> > > > > >> > > (Current proposal) Store Delegation Token, > DT, > > >> on ZK. > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > 1. Client authenticates with a broker. > > >> >> > >> >> > > > > >> > > 2. Once a client is authenticated, it > will > > >> make a > > >> >> > broker > > >> >> > >> >> side > > >> >> > >> >> > > > call to > > >> >> > >> >> > > > > >> > > issue a delegation token. > > >> >> > >> >> > > > > >> > > 3. The broker generates a shared secret > > based > > >> on > > >> >> > >> >> > > HMAC-SHA256(a > > >> >> > >> >> > > > > >> > > Password/Secret shared between all > brokers, > > >> >> > randomly > > >> >> > >> >> > > generated > > >> >> > >> >> > > > > >> > tokenId). > > >> >> > >> >> > > > > >> > > 4. Broker stores this token in its in > > memory > > >> cache. > > >> >> > >> >> Broker > > >> >> > >> >> > > > also stores > > >> >> > >> >> > > > > >> > > the DelegationToken without the hmac in > the > > >> >> > zookeeper. > > >> >> > >> >> > > > > >> > > 5. All brokers will have a cache backed > by > > >> >> > zookeeper so > > >> >> > >> >> they > > >> >> > >> >> > > > will all > > >> >> > >> >> > > > > >> > > get notified whenever a new token is > > >> generated and > > >> >> > they > > >> >> > >> >> will > > >> >> > >> >> > > > update > > >> >> > >> >> > > > > >> > their > > >> >> > >> >> > > > > >> > > local cache whenever token state changes. > > >> >> > >> >> > > > > >> > > 6. Broker returns the token to Client. > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > Probable issues and fixes > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > 1. Probable race condition, client tries > to > > >> >> > authenticate > > >> >> > >> >> with > > >> >> > >> >> > > > a broker > > >> >> > >> >> > > > > >> > > that is yet to be updated with the newly > > >> generated > > >> >> > DT. > > >> >> > >> >> This > > >> >> > >> >> > > can > > >> >> > >> >> > > > > >> > probably be > > >> >> > >> >> > > > > >> > > dealt with making dtRequest block until > all > > >> >> > brokers have > > >> >> > >> >> > > > updated > > >> >> > >> >> > > > > >> > their DT > > >> >> > >> >> > > > > >> > > cache. Zk barrier or similar mechanism > can > > be > > >> used. > > >> >> > >> >> However, > > >> >> > >> >> > > > all such > > >> >> > >> >> > > > > >> > > mechanisms will increase complexity. > > >> >> > >> >> > > > > >> > > 2. Using a static secret key from config > > >> file. Will > > >> >> > >> >> require > > >> >> > >> >> > > yet > > >> >> > >> >> > > > > >> > another > > >> >> > >> >> > > > > >> > > config and uses a static secret key. It > is > > >> advised > > >> >> > to > > >> >> > >> >> rotate > > >> >> > >> >> > > > secret > > >> >> > >> >> > > > > >> > keys > > >> >> > >> >> > > > > >> > > periodically. This can be avoided with > > >> controller > > >> >> > >> >> generating > > >> >> > >> >> > > > > >> > secretKey and > > >> >> > >> >> > > > > >> > > passing to brokers periodically. However, > > >> this will > > >> >> > >> >> require > > >> >> > >> >> > > > brokers to > > >> >> > >> >> > > > > >> > > maintain certain counts of secretKeys. > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > (Alternative 1) Have controller generate > > >> delegation > > >> >> > token. > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > 1. Client authenticates with a broker. > > >> >> > >> >> > > > > >> > > 2. Once a client is authenticated, it > will > > >> make a > > >> >> > broker > > >> >> > >> >> side > > >> >> > >> >> > > > call to > > >> >> > >> >> > > > > >> > > issue a delegation token. > > >> >> > >> >> > > > > >> > > 3. Broker forwards the request to > > controller. > > >> >> > >> >> > > > > >> > > 4. Controller generates a DT and > broadcasts > > >> to all > > >> >> > >> >> brokers. > > >> >> > >> >> > > > > >> > > 5. Broker stores this token in its memory > > >> cache. > > >> >> > >> >> > > > > >> > > 6. Controller responds to broker’s DT > req. > > >> >> > >> >> > > > > >> > > 7. Broker returns the token to Client. > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > Probable issues and fixes > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > 1. We will have to add new APIs to > support > > >> >> > controller > > >> >> > >> >> pushing > > >> >> > >> >> > > > tokens > > >> >> > >> >> > > > > >> > to > > >> >> > >> >> > > > > >> > > brokers on top of the minimal APIs that > are > > >> >> > currently > > >> >> > >> >> > > proposed. > > >> >> > >> >> > > > > >> > > 2. We will also have to add APIs to > support > > >> the > > >> >> > >> >> bootstrapping > > >> >> > >> >> > > > case, > > >> >> > >> >> > > > > >> > i.e, > > >> >> > >> >> > > > > >> > > when a new broker comes up it will have > to > > >> get all > > >> >> > >> >> delegation > > >> >> > >> >> > > > tokens > > >> >> > >> >> > > > > >> > from > > >> >> > >> >> > > > > >> > > the controller. > > >> >> > >> >> > > > > >> > > 3. In catastrophic failures where all > > brokers > > >> go > > >> >> > down, > > >> >> > >> >> the > > >> >> > >> >> > > > tokens will > > >> >> > >> >> > > > > >> > > be lost even if servers are restarted as > > >> tokens > > >> >> > are not > > >> >> > >> >> > > > persisted > > >> >> > >> >> > > > > >> > anywhere. > > >> >> > >> >> > > > > >> > > If this happens, then there are more > > important > > >> >> > things to > > >> >> > >> >> > > worry > > >> >> > >> >> > > > about > > >> >> > >> >> > > > > >> > and > > >> >> > >> >> > > > > >> > > maybe it is better to re-authenticate. > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > (Alternative 2) Do not distribute DT to > other > > >> brokers > > >> >> > at > > >> >> > >> >> all. > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > 1. Client authenticates with a broker. > > >> >> > >> >> > > > > >> > > 2. Once a client is authenticated, it > will > > >> make a > > >> >> > broker > > >> >> > >> >> side > > >> >> > >> >> > > > call to > > >> >> > >> >> > > > > >> > > issue a delegation token. > > >> >> > >> >> > > > > >> > > 3. The broker generates DT of form, > [hmac + > > >> (owner, > > >> >> > >> >> renewer, > > >> >> > >> >> > > > > >> > > maxLifeTime, id, hmac, expirationTime)] > and > > >> passes > > >> >> > back > > >> >> > >> >> this > > >> >> > >> >> > > > DT to > > >> >> > >> >> > > > > >> > client. > > >> >> > >> >> > > > > >> > > hmac is generated via {HMAC-SHA256(owner, > > >> renewer, > > >> >> > >> >> > > > maxLifeTime, id, > > >> >> > >> >> > > > > >> > hmac, > > >> >> > >> >> > > > > >> > > expirationTime) using SecretKey}. Note > that > > >> all > > >> >> > brokers > > >> >> > >> >> have > > >> >> > >> >> > > > this > > >> >> > >> >> > > > > >> > SecretKey. > > >> >> > >> >> > > > > >> > > 4. Client then goes to any broker and to > > >> >> > authenticate > > >> >> > >> >> sends > > >> >> > >> >> > > > the DT. > > >> >> > >> >> > > > > >> > > Broker recalculates hmac using (owner, > > >> renewer, > > >> >> > >> >> maxLifeTime, > > >> >> > >> >> > > > id, hmac, > > >> >> > >> >> > > > > >> > > expirationTime) info from DT and its > > >> SecretKey. If > > >> >> > it > > >> >> > >> >> matches > > >> >> > >> >> > > > with > > >> >> > >> >> > > > > >> > hmac of > > >> >> > >> >> > > > > >> > > DT, client is authenticated. Yes, it will > > do > > >> other > > >> >> > >> >> obvious > > >> >> > >> >> > > > checks of > > >> >> > >> >> > > > > >> > > timestamp expiry and such. > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > Note that secret key will be generated by > > >> controller > > >> >> > and > > >> >> > >> >> passed > > >> >> > >> >> > > to > > >> >> > >> >> > > > > >> > brokers > > >> >> > >> >> > > > > >> > > periodically. > > >> >> > >> >> > > > > >> > > Probable issues and fixes > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > 1. How to delete a DT? Yes, that is a > > downside > > >> >> > here. > > >> >> > >> >> However, > > >> >> > >> >> > > > this can > > >> >> > >> >> > > > > >> > > be handled with brokers maintaining a > > >> blacklist of > > >> >> > DTs, > > >> >> > >> >> DTs > > >> >> > >> >> > > > from this > > >> >> > >> >> > > > > >> > list > > >> >> > >> >> > > > > >> > > can be removed after expiry. > > >> >> > >> >> > > > > >> > > 2. In catastrophic failures where all > > brokers > > >> go > > >> >> > down, > > >> >> > >> >> the > > >> >> > >> >> > > > tokens will > > >> >> > >> >> > > > > >> > > be lost even if servers are restarted as > > >> tokens > > >> >> > are not > > >> >> > >> >> > > > persisted > > >> >> > >> >> > > > > >> > anywhere. > > >> >> > >> >> > > > > >> > > If this happens, then there are more > > important > > >> >> > things to > > >> >> > >> >> > > worry > > >> >> > >> >> > > > about > > >> >> > >> >> > > > > >> > and > > >> >> > >> >> > > > > >> > > maybe it is better to re-authenticate. > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26, 2016 at 1:58 PM, Parth > > >> Brahmbhatt < > > >> >> > >> >> > > > > >> > > pbrahmbh...@hortonworks.com> wrote: > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > >> Hi, > > >> >> > >> >> > > > > >> > >> > > >> >> > >> >> > > > > >> > >> I have filed KIP-48 so we can offer hadoop > > like > > >> >> > delegation > > >> >> > >> >> > > > tokens in > > >> >> > >> >> > > > > >> > >> kafka. You can review the design > > >> >> > >> >> > > > > >> > >> > > >> >> > >> >> > > > > >> > > > >> >> > >> >> > > > > > >> >> > >> >> > > > > >> >> > >> >> > > >> >> > > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka > > >> >> > >> >> > > > > >> > . > > >> >> > >> >> > > > > >> > >> This KIP depends on KIP-43 and we have also > > >> >> > discussed an > > >> >> > >> >> > > > alternative to > > >> >> > >> >> > > > > >> > >> proposed design here< > > >> >> > >> >> > > > > >> > >> > > >> >> > >> >> > > > > >> > > > >> >> > >> >> > > > > > >> >> > >> >> > > > > >> >> > >> >> > > >> >> > > > >> > > > https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800 > > >> >> > >> >> > > > > >> > >> >. > > >> >> > >> >> > > > > >> > >> > > >> >> > >> >> > > > > >> > >> Thanks > > >> >> > >> >> > > > > >> > >> Parth > > >> >> > >> >> > > > > >> > >> > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > -- > > >> >> > >> >> > > > > >> > > > > >> >> > >> >> > > > > >> > > Regards, > > >> >> > >> >> > > > > >> > > Ashish > > >> >> > >> >> > > > > >> > > > >> >> > >> >> > > > > > >> >> > >> >> > > > > >> >> > >> >> > > > > >> >> > >> >> > > > > >> >> > >> >> > > -- > > >> >> > >> >> > > > > >> >> > >> >> > > Regards, > > >> >> > >> >> > > Ashish > > >> >> > >> >> > > > > >> >> > >> >> > > >> >> > > > >> > > >
