Makes sense to me. Thanks!
On Mon, Mar 7, 2016 at 9:25 PM, Harsha <ka...@harsha.io> wrote: > It doesn't need any release vehicle but still the work can move forward. > If anyone is interested in the KIP please do the review and provide the > comments. > > -Harsha > > On Mon, Mar 7, 2016, at 04:59 PM, Ismael Juma wrote: >> I agree that it would be good to have more time to review and discuss >> KIP-48. >> >> Ismael >> >> On Tue, Mar 8, 2016 at 12:55 AM, Gwen Shapira <g...@confluent.io> wrote: >> >> > Hi Team, >> > >> > Since KIP-48 depends on KIP-43, which is already a bit of a risk for >> > the next release - any chance we can delay delegation tokens to Kafka >> > 0.10.1? >> > With the community working on a release every 3 month, this is not a huge >> > delay. >> > >> > Gwen >> > >> > On Fri, Feb 26, 2016 at 5:11 PM, Ashish Singh <asi...@cloudera.com> wrote: >> > > Parth, >> > > >> > > Thanks again for the awesome write up. Following our discussion from the >> > > JIRA, I think it will be easier to compare various alternatives if they >> > are >> > > listed together. I am stating below a few alternatives along with a the >> > > current proposal. >> > > (Current proposal) Store Delegation Token, DT, on ZK. >> > > >> > > 1. Client authenticates with a broker. >> > > 2. Once a client is authenticated, it will make a broker side call to >> > > issue a delegation token. >> > > 3. The broker generates a shared secret based on HMAC-SHA256(a >> > > Password/Secret shared between all brokers, randomly generated >> > tokenId). >> > > 4. Broker stores this token in its in memory cache. Broker also stores >> > > the DelegationToken without the hmac in the zookeeper. >> > > 5. All brokers will have a cache backed by zookeeper so they will all >> > > get notified whenever a new token is generated and they will update >> > their >> > > local cache whenever token state changes. >> > > 6. Broker returns the token to Client. >> > > >> > > Probable issues and fixes >> > > >> > > 1. Probable race condition, client tries to authenticate with a broker >> > > that is yet to be updated with the newly generated DT. This can >> > probably be >> > > dealt with making dtRequest block until all brokers have updated >> > their DT >> > > cache. Zk barrier or similar mechanism can be used. However, all such >> > > mechanisms will increase complexity. >> > > 2. Using a static secret key from config file. Will require yet >> > another >> > > config and uses a static secret key. It is advised to rotate secret >> > keys >> > > periodically. This can be avoided with controller generating >> > secretKey and >> > > passing to brokers periodically. However, this will require brokers to >> > > maintain certain counts of secretKeys. >> > > >> > > (Alternative 1) Have controller generate delegation token. >> > > >> > > 1. Client authenticates with a broker. >> > > 2. Once a client is authenticated, it will make a broker side call to >> > > issue a delegation token. >> > > 3. Broker forwards the request to controller. >> > > 4. Controller generates a DT and broadcasts to all brokers. >> > > 5. Broker stores this token in its memory cache. >> > > 6. Controller responds to broker’s DT req. >> > > 7. Broker returns the token to Client. >> > > >> > > Probable issues and fixes >> > > >> > > 1. We will have to add new APIs to support controller pushing tokens >> > to >> > > brokers on top of the minimal APIs that are currently proposed. >> > > 2. We will also have to add APIs to support the bootstrapping case, >> > i.e, >> > > when a new broker comes up it will have to get all delegation tokens >> > from >> > > the controller. >> > > 3. In catastrophic failures where all brokers go down, the tokens will >> > > be lost even if servers are restarted as tokens are not persisted >> > anywhere. >> > > If this happens, then there are more important things to worry about >> > and >> > > maybe it is better to re-authenticate. >> > > >> > > (Alternative 2) Do not distribute DT to other brokers at all. >> > > >> > > 1. Client authenticates with a broker. >> > > 2. Once a client is authenticated, it will make a broker side call to >> > > issue a delegation token. >> > > 3. The broker generates DT of form, [hmac + (owner, renewer, >> > > maxLifeTime, id, hmac, expirationTime)] and passes back this DT to >> > client. >> > > hmac is generated via {HMAC-SHA256(owner, renewer, maxLifeTime, id, >> > hmac, >> > > expirationTime) using SecretKey}. Note that all brokers have this >> > SecretKey. >> > > 4. Client then goes to any broker and to authenticate sends the DT. >> > > Broker recalculates hmac using (owner, renewer, maxLifeTime, id, hmac, >> > > expirationTime) info from DT and its SecretKey. If it matches with >> > hmac of >> > > DT, client is authenticated. Yes, it will do other obvious checks of >> > > timestamp expiry and such. >> > > >> > > Note that secret key will be generated by controller and passed to >> > brokers >> > > periodically. >> > > Probable issues and fixes >> > > >> > > 1. How to delete a DT? Yes, that is a downside here. However, this can >> > > be handled with brokers maintaining a blacklist of DTs, DTs from this >> > list >> > > can be removed after expiry. >> > > 2. In catastrophic failures where all brokers go down, the tokens will >> > > be lost even if servers are restarted as tokens are not persisted >> > anywhere. >> > > If this happens, then there are more important things to worry about >> > and >> > > maybe it is better to re-authenticate. >> > > >> > > >> > > >> > > On Fri, Feb 26, 2016 at 1:58 PM, Parth Brahmbhatt < >> > > pbrahmbh...@hortonworks.com> wrote: >> > > >> > >> Hi, >> > >> >> > >> I have filed KIP-48 so we can offer hadoop like delegation tokens in >> > >> kafka. You can review the design >> > >> >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka >> > . >> > >> This KIP depends on KIP-43 and we have also discussed an alternative to >> > >> proposed design here< >> > >> >> > https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800 >> > >> >. >> > >> >> > >> Thanks >> > >> Parth >> > >> >> > > >> > > >> > > >> > > -- >> > > >> > > Regards, >> > > Ashish >> >