Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

[parth brahmbhatt]

1. Who / how are tokens renewed? By original requester only? or using Kerberos
auth only?
My recommendation is to do this only using Kerberos auth and only threw the
renewer specified during the acquisition request.

2. Are tokens stored on each broker or in ZK?
My recommendation is still to store in ZK or not store them at all. The
whole controller based distribution is too much overhead with not much to
achieve.

3. How are tokens invalidated / expired?
Either by expiration time out or through an explicit request to invalidate.

4. Which encryption algorithm is used?
SCRAM

5. What is the impersonation proposal (it wasn't in the KIP but was discussed
in this thread)?
There is no imperonation proposal. I tried and explained how its a
different problem and why its not really necessary to discuss that as part
of this KIP.  This KIP will not support any impersonation, it will just be
another way to authenticate.

6. Do we need new ACLs, if so - for what actions?
We do not need new ACLs.

7. How would the delegation token be configured in the client?
Should be through config. I wasn't planning on supporting JAAS for tokens.
I don't believe hadoop does this either.

Thanks
Parth



On Thu, Jun 16, 2016 at 4:03 PM, Jun Rao <j...@confluent.io> wrote:

> Harsha,
>
> Another question.
>
> 9. How would the delegation token be configured in the client? The standard
> way is to do this through JAAS. However, we will need to think through if
> this is convenient in a shared environment. For example, when a new task is
> added to a Storm worker node, do we need to dynamically add a new section
> in the JAAS file? It may be more convenient if we can pass in the token
> through the config directly w/o going through JAAS.
>
> Are you or Parth still actively working on this KIP?
>
> Thanks,
>
> Jun
>
>
>
> On Sun, Jun 12, 2016 at 2:18 PM, Jun Rao <j...@confluent.io> wrote:
>
> > Just to add on that list.
> >
> > 2. It would be good to document the format of the data stored in ZK.
> > 7. Earlier, there was a discussion on whether the tokens should be
> > propagated through ZK like config/acl/quota, or through the controller.
> > Currently, the controller is only designed for propagating topic
> metadata,
> > but not other data.
> > 8. Should we use SCRAM to send the token instead of DIGEST-MD5 since it's
> > deprecated?
> >
> > Also, the images in the wiki seem broken.
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Jun 10, 2016 at 10:02 AM, Gwen Shapira <g...@confluent.io>
> wrote:
> >
> >> From what I can see, remaining questions are:
> >>
> >> 1. Who / how are tokens renewed? By original requester only? or using
> >> Kerberos auth only?
> >> 2. Are tokens stored on each broker or in ZK?
> >> 3. How are tokens invalidated / expired?
> >> 4. Which encryption algorithm is used?
> >> 5. What is the impersonation proposal (it wasn't in the KIP but was
> >> discussed in this thread)?
> >> 6. Do we need new ACLs, if so - for what actions?
> >>
> >> Gwen
> >>
> >> On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote:
> >> > Jun & Ismael,
> >> >                          Unfortunately I couldn't attend the KIP
> meeting
> >> >                          when delegation tokens discussed. Appreciate
> if
> >> >                          you can update the thread if you have any
> >> >                          further questions.
> >> > Thanks,
> >> > Harsha
> >> >
> >> > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote:
> >> >> It seems that the links to images in the KIP are broken.
> >> >>
> >> >> Liquan
> >> >>
> >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt <
> >> >> brahmbhatt.pa...@gmail.com> wrote:
> >> >>
> >> >> > 110. What does getDelegationTokenAs mean?
> >> >> > In the current proposal we only allow a user to get delegation
> token
> >> for
> >> >> > the identity that it authenticated as using another mechanism, i.e.
> >> A user
> >> >> > that authenticate using a keytab for principal us...@example.com
> >> will get
> >> >> > delegation tokens for that user only. In future I think we will
> have
> >> to
> >> >> > extend support such that we allow some set of users (
> >> >> > kafka-rest-u...@example.com, storm-nim...@example.com) to acquire
> >> >> > delegation tokens on behalf of other users whose identity they have
> >> >> > verified independently.  Kafka brokers will have ACLs to control
> >> which
> >> >> > users are allowed to impersonate other users and get tokens on
> >> behalf of
> >> >> > them. Overall Impersonation is a whole different problem in my
> >> opinion and
> >> >> > I think we can tackle it in separate KIP.
> >> >> >
> >> >> > 111. What's the typical rate of getting and renewing delegation
> >> tokens?
> >> >> > Typically this should be very very low, 1 request per minute is a
> >> >> > relatively high estimate. However it depends on the token
> >> expiration. I am
> >> >> > less worried about the extra load it puts on controller vs the
> added
> >> >> > complexity and the value it offers.
> >> >> >
> >> >> > Thanks
> >> >> > Parth
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <ism...@juma.me.uk>
> >> wrote:
> >> >> >
> >> >> > > Thanks Rajini. It would probably require a separate KIP as it
> will
> >> >> > > introduce user visible changes. We could also update KIP-48 to
> >> have this
> >> >> > > information, but it seems cleaner to do it separately. We can
> >> discuss
> >> >> > that
> >> >> > > in the KIP call today.
> >> >> > >
> >> >> > > Ismael
> >> >> > >
> >> >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram <
> >> >> > > rajinisiva...@googlemail.com> wrote:
> >> >> > >
> >> >> > > > Ismael,
> >> >> > > >
> >> >> > > > I have created a JIRA (
> >> >> > https://issues.apache.org/jira/browse/KAFKA-3751)
> >> >> > > > for adding SCRAM as a SASL mechanism. Would that need another
> >> KIP? If
> >> >> > > > KIP-48 will use this mechanism, can this just be a JIRA that
> gets
> >> >> > > reviewed
> >> >> > > > when the PR is ready?
> >> >> > > >
> >> >> > > > Thank you,
> >> >> > > >
> >> >> > > > Rajini
> >> >> > > >
> >> >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <
> ism...@juma.me.uk>
> >> >> > wrote:
> >> >> > > >
> >> >> > > > > Thanks Rajini, SCRAM seems like a good candidate.
> >> >> > > > >
> >> >> > > > > Gwen had independently mentioned this as a SASL mechanism
> that
> >> might
> >> >> > be
> >> >> > > > > useful for Kafka and I have been meaning to check it in more
> >> detail.
> >> >> > > Good
> >> >> > > > > to know that you are willing to contribute an implementation.
> >> Maybe
> >> >> > we
> >> >> > > > > should file a separate JIRA for this?
> >> >> > > > >
> >> >> > > > > Ismael
> >> >> > > > >
> >> >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram <
> >> >> > > > > rajinisiva...@googlemail.com> wrote:
> >> >> > > > >
> >> >> > > > > > SCRAM (Salted Challenge Response Authentication Mechanism)
> >> is a
> >> >> > > better
> >> >> > > > > > mechanism than Digest-MD5. Java doesn't come with a
> built-in
> >> SCRAM
> >> >> > > > > > SaslServer or SaslClient, but I will be happy to add
> support
> >> in
> >> >> > Kafka
> >> >> > > > > since
> >> >> > > > > > it would be a useful mechanism to support anyway.
> >> >> > > > > > https://tools.ietf.org/html/rfc7677 describes the protocol
> >> for
> >> >> > > > > > SCRAM-SHA-256.
> >> >> > > > > >
> >> >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <j...@confluent.io
> >
> >> wrote:
> >> >> > > > > >
> >> >> > > > > > > Parth,
> >> >> > > > > > >
> >> >> > > > > > > Thanks for the explanation. A couple of more questions.
> >> >> > > > > > >
> >> >> > > > > > > 110. What does getDelegationTokenAs mean?
> >> >> > > > > > >
> >> >> > > > > > > 111. What's the typical rate of getting and renewing
> >> delegation
> >> >> > > > tokens?
> >> >> > > > > > > That may have an impact on whether they should be
> directed
> >> to the
> >> >> > > > > > > controller.
> >> >> > > > > > >
> >> >> > > > > > > Jun
> >> >> > > > > > >
> >> >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt <
> >> >> > > > > > > brahmbhatt.pa...@gmail.com> wrote:
> >> >> > > > > > >
> >> >> > > > > > > > Hi Jun,
> >> >> > > > > > > >
> >> >> > > > > > > > Thanks for reviewing.
> >> >> > > > > > > >
> >> >> > > > > > > > * We could add a Cluster action to add acls on who can
> >> request
> >> >> > > > > > delegation
> >> >> > > > > > > > tokens. I don't see the use case for that yet but down
> >> the line
> >> >> > > > when
> >> >> > > > > we
> >> >> > > > > > > > start supporting getDelegationTokenAs it will be
> >> necessary.
> >> >> > > > > > > > * Yes we recommend tokens to be only used/distributed
> >> over
> >> >> > secure
> >> >> > > > > > > channels.
> >> >> > > > > > > > * Depending on what design we end up choosing
> >> Invalidation will
> >> >> > > be
> >> >> > > > > > > > responsibility of every broker or controller.
> >> >> > > > > > > > * I am not sure if I documented somewhere that
> >> invalidation
> >> >> > will
> >> >> > > > > > directly
> >> >> > > > > > > > go through zookeeper but that is not the intent.
> >> Invalidation
> >> >> > > will
> >> >> > > > > > either
> >> >> > > > > > > > be request based or due to expiration. No direct
> >> zookeeper
> >> >> > > > > interaction
> >> >> > > > > > > from
> >> >> > > > > > > > any client.
> >> >> > > > > > > > * "Broker also stores the DelegationToken without the
> >> hmac in
> >> >> > the
> >> >> > > > > > > > zookeeper." : Sorry about the confusion. The sole
> >> purpose of
> >> >> > > > > zookeeper
> >> >> > > > > > in
> >> >> > > > > > > > this design is as distribution channel for tokens
> >> between all
> >> >> > > > brokers
> >> >> > > > > > > and a
> >> >> > > > > > > > layer that ensures only tokens that were generated by
> >> making a
> >> >> > > > > request
> >> >> > > > > > > to a
> >> >> > > > > > > > broker will be accepted (more on this in second
> >> paragraph). The
> >> >> > > > token
> >> >> > > > > > > > consists of few elements (owner, renewer, uuid ,
> >> expiration,
> >> >> > > hmac)
> >> >> > > > ,
> >> >> > > > > > one
> >> >> > > > > > > of
> >> >> > > > > > > > which is the finally generated hmac but hmac it self is
> >> >> > derivable
> >> >> > > > if
> >> >> > > > > > you
> >> >> > > > > > > > have all the other elements of the token + secret key
> to
> >> >> > generate
> >> >> > > > > hmac.
> >> >> > > > > > > > Given zookeeper does not provide SSL support we do not
> >> want the
> >> >> > > > > entire
> >> >> > > > > > > > token to be wire transferred to zookeeper as that will
> >> be an
> >> >> > > > insecure
> >> >> > > > > > > wire
> >> >> > > > > > > > transfer. Instead we only store all the other elements
> >> of a
> >> >> > > > > delegation
> >> >> > > > > > > > tokens. Brokers can read these elements and because
> they
> >> also
> >> >> > > have
> >> >> > > > > > access
> >> >> > > > > > > > to secret key they will be able to generate hmac on
> >> their end.
> >> >> > > > > > > >
> >> >> > > > > > > > One of the alternative proposed is to avoid zookeeper
> >> >> > > altogether. A
> >> >> > > > > > > Client
> >> >> > > > > > > > will call broker with required information (owner,
> >> renwer,
> >> >> > > > > expiration)
> >> >> > > > > > > and
> >> >> > > > > > > > get back (signed hmac, uuid). Broker won't store this
> in
> >> >> > > zookeeper.
> >> >> > > > > > From
> >> >> > > > > > > > this point a client can contact any broker with all the
> >> >> > > delegation
> >> >> > > > > > token
> >> >> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the borker
> >> will
> >> >> > > > > regenerate
> >> >> > > > > > > the
> >> >> > > > > > > > hmac and as long as it matches with hmac presented by
> >> client ,
> >> >> > > > broker
> >> >> > > > > > > will
> >> >> > > > > > > > allow the request to authenticate.  Only problem with
> >> this
> >> >> > > approach
> >> >> > > > > is
> >> >> > > > > > if
> >> >> > > > > > > > the secret key is compromised any client can now
> generate
> >> >> > random
> >> >> > > > > tokens
> >> >> > > > > > > and
> >> >> > > > > > > > they will still be able to authenticate as any user
> they
> >> like.
> >> >> > > with
> >> >> > > > > > > > zookeeper we guarantee that only tokens acquired via a
> >> broker
> >> >> > > > (using
> >> >> > > > > > some
> >> >> > > > > > > > auth scheme other than delegation token) will be
> >> accepted. We
> >> >> > > need
> >> >> > > > to
> >> >> > > > > > > > discuss which proposal makes more sense and we can go
> >> over it
> >> >> > in
> >> >> > > > > > > tomorrow's
> >> >> > > > > > > > meeting.
> >> >> > > > > > > >
> >> >> > > > > > > > Also, can you forward the invite to me?
> >> >> > > > > > > >
> >> >> > > > > > > > Thanks
> >> >> > > > > > > > Parth
> >> >> > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <
> >> j...@confluent.io>
> >> >> > > > wrote:
> >> >> > > > > > > >
> >> >> > > > > > > > > Thanks for the KIP. A few comments.
> >> >> > > > > > > > >
> >> >> > > > > > > > > 100. This potentially can be useful for Kafka Connect
> >> and
> >> >> > Kafka
> >> >> > > > > rest
> >> >> > > > > > > > proxy
> >> >> > > > > > > > > where a worker agent will need to run a task on
> behalf
> >> of a
> >> >> > > > client.
> >> >> > > > > > We
> >> >> > > > > > > > will
> >> >> > > > > > > > > likely need to change how those services use Kafka
> >> clients
> >> >> > > > > > > > > (producer/consumer). Instead of a shared client per
> >> worker,
> >> >> > we
> >> >> > > > will
> >> >> > > > > > > need
> >> >> > > > > > > > a
> >> >> > > > > > > > > client per user task since the authentication happens
> >> at the
> >> >> > > > > > connection
> >> >> > > > > > > > > level. For Kafka Connect, the renewer will be the
> >> workers.
> >> >> > So,
> >> >> > > we
> >> >> > > > > > > > probably
> >> >> > > > > > > > > need to allow multiple renewers. For Kafka rest
> proxy,
> >> the
> >> >> > > > renewer
> >> >> > > > > > can
> >> >> > > > > > > > > probably just be the creator of the token.
> >> >> > > > > > > > >
> >> >> > > > > > > > > 101. Do we need new acl on who can request delegation
> >> tokens?
> >> >> > > > > > > > >
> >> >> > > > > > > > > 102. Do we recommend people to send delegation tokens
> >> in an
> >> >> > > > > encrypted
> >> >> > > > > > > > > channel?
> >> >> > > > > > > > >
> >> >> > > > > > > > > 103. Who is responsible for expiring tokens, every
> >> broker?
> >> >> > > > > > > > >
> >> >> > > > > > > > > 104. For invalidating tokens, would it be better to
> do
> >> it in
> >> >> > a
> >> >> > > > > > request
> >> >> > > > > > > > > instead of going to ZK directly?
> >> >> > > > > > > > >
> >> >> > > > > > > > > 105. The terminology of client in the wiki sometimes
> >> refers
> >> >> > to
> >> >> > > > the
> >> >> > > > > > end
> >> >> > > > > > > > > client and some other times refers to the client
> using
> >> the
> >> >> > > > > delegation
> >> >> > > > > > > > > tokens. It would be useful to distinguish between the
> >> two.
> >> >> > > > > > > > >
> >> >> > > > > > > > > 106. Could you explain the sentence "Broker also
> >> stores the
> >> >> > > > > > > > DelegationToken
> >> >> > > > > > > > > without the hmac in the zookeeper." a bit more? I
> >> thought the
> >> >> > > > > > > delegation
> >> >> > > > > > > > > token is the hmac.
> >> >> > > > > > > > >
> >> >> > > > > > > > > Thanks,
> >> >> > > > > > > > >
> >> >> > > > > > > > > Jun
> >> >> > > > > > > > >
> >> >> > > > > > > > >
> >> >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <
> >> j...@confluent.io>
> >> >> > > > wrote:
> >> >> > > > > > > > >
> >> >> > > > > > > > > > Hi, Harsha,
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > Just sent out a KIP meeting invite. We can discuss
> >> this in
> >> >> > > the
> >> >> > > > > > > meeting
> >> >> > > > > > > > > > tomorrow.
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > Thanks,
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > Jun
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <
> >> ka...@harsha.io>
> >> >> > > > wrote:
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >> Hi All,
> >> >> > > > > > > > > >>            Can we have a KIP meeting around this.
> >> The KIP
> >> >> > is
> >> >> > > > up
> >> >> > > > > > for
> >> >> > > > > > > > > >>            sometime and if there are any questions
> >> lets
> >> >> > > > quickly
> >> >> > > > > > hash
> >> >> > > > > > > > out
> >> >> > > > > > > > > >>            details.
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >> Thanks,
> >> >> > > > > > > > > >> Harsha
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth
> brahmbhatt
> >> wrote:
> >> >> > > > > > > > > >> > That is what the hadoop echo system uses so no
> >> good
> >> >> > reason
> >> >> > > > > > really.
> >> >> > > > > > > > We
> >> >> > > > > > > > > >> > could
> >> >> > > > > > > > > >> > change it to whatever is the newest recommended
> >> standard
> >> >> > > is.
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > Thanks
> >> >> > > > > > > > > >> > Parth
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma <
> >> >> > > > > ism...@juma.me.uk
> >> >> > > > > > >
> >> >> > > > > > > > > wrote:
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > > Hi Parth,
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > Thanks for the KIP. I only started reviewing
> >> this and
> >> >> > > may
> >> >> > > > > have
> >> >> > > > > > > > > >> additional
> >> >> > > > > > > > > >> > > questions later. The immediate question that
> >> came to
> >> >> > > mind
> >> >> > > > is
> >> >> > > > > > our
> >> >> > > > > > > > > >> choice of
> >> >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as
> >> OBSOLETE in
> >> >> > the
> >> >> > > > IANA
> >> >> > > > > > > > > Registry
> >> >> > > > > > > > > >> of
> >> >> > > > > > > > > >> > > SASL mechanisms and the original RFC (2831)
> has
> >> been
> >> >> > > moved
> >> >> > > > > to
> >> >> > > > > > > > > Historic
> >> >> > > > > > > > > >> > > status:
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > >
> >> >> > > > > >
> >> >> > >
> >> http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > What is the reasoning behind that choice?
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > Thanks,
> >> >> > > > > > > > > >> > > Ismael
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen
> Shapira <
> >> >> > > > > > > g...@confluent.io
> >> >> > > > > > > > >
> >> >> > > > > > > > > >> wrote:
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > > Also comments inline :)
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > * I want to emphasize that even though
> >> delegation
> >> >> > > > tokens
> >> >> > > > > > > are a
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > > innovation, I feel very strongly about not
> >> adding
> >> >> > > > > > dependency
> >> >> > > > > > > > on
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > > when implementing delegation tokens for
> >> Kafka. The
> >> >> > > KIP
> >> >> > > > > > > doesn't
> >> >> > > > > > > > > >> imply
> >> >> > > > > > > > > >> > > > > such dependency, but if you can clarify...
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *No hadoop dependency.*
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one will
> >> read
> >> >> > the
> >> >> > > > KIP
> >> >> > > > > > and
> >> >> > > > > > > > > panic
> >> >> > > > > > > > > >> > > > three weeks before the next release...
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > * Can we get delegation token at any time
> >> after
> >> >> > > > > > > > authenticating?
> >> >> > > > > > > > > >> only
> >> >> > > > > > > > > >> > > > > immediately after?
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *As long as you are authenticated you can
> >> get
> >> >> > > > delegation
> >> >> > > > > > > > tokens.
> >> >> > > > > > > > > >> We
> >> >> > > > > > > > > >> > > need
> >> >> > > > > > > > > >> > > > to
> >> >> > > > > > > > > >> > > > > discuss if a client authenticated using
> >> delegation
> >> >> > > > > token,
> >> >> > > > > > > can
> >> >> > > > > > > > > also
> >> >> > > > > > > > > >> > > > acquire
> >> >> > > > > > > > > >> > > > > delegation token again or not. Also there
> >> is the
> >> >> > > > > question
> >> >> > > > > > of
> >> >> > > > > > > > do
> >> >> > > > > > > > > we
> >> >> > > > > > > > > >> > > allow
> >> >> > > > > > > > > >> > > > > anyone to acquire delegation token or we
> >> want
> >> >> > > specific
> >> >> > > > > > ACLs
> >> >> > > > > > > (I
> >> >> > > > > > > > > >> think
> >> >> > > > > > > > > >> > > its
> >> >> > > > > > > > > >> > > > an
> >> >> > > > > > > > > >> > > > > overkill.)*
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > I agree that ACLs is an overkill.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > I think we are debating two options: Either
> >> require
> >> >> > > > > Kerberos
> >> >> > > > > > > > auth
> >> >> > > > > > > > > >> for
> >> >> > > > > > > > > >> > > > renewal or require non-owners to renew.
> >> >> > > > > > > > > >> > > > I *think* the latter is simpler (it
> basically
> >> >> > require
> >> >> > > a
> >> >> > > > > "job
> >> >> > > > > > > > > master"
> >> >> > > > > > > > > >> > > > to take responsibility for the renewal, it
> >> will have
> >> >> > > its
> >> >> > > > > own
> >> >> > > > > > > > > >> identity
> >> >> > > > > > > > > >> > > > anyway and I think this is the correct
> design
> >> >> > pattern
> >> >> > > > > > anyway.
> >> >> > > > > > > > For
> >> >> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate
> >> renewals?),
> >> >> > but
> >> >> > > > it
> >> >> > > > > is
> >> >> > > > > > > > hard
> >> >> > > > > > > > > to
> >> >> > > > > > > > > >> > > > debate simplicity without looking at the
> code
> >> >> > changes
> >> >> > > > > > > required.
> >> >> > > > > > > > If
> >> >> > > > > > > > > >> you
> >> >> > > > > > > > > >> > > > have a draft of how the "require Kerberos"
> >> will look
> >> >> > > in
> >> >> > > > > > Kafka
> >> >> > > > > > > > > code,
> >> >> > > > > > > > > >> > > > I'll be happy to take a look.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > * My understanding is that tokens will
> >> propagate
> >> >> > via
> >> >> > > > ZK
> >> >> > > > > > but
> >> >> > > > > > > > > >> without
> >> >> > > > > > > > > >> > > > > additional changes to UpdateMetadata
> >> protocol,
> >> >> > > > correct?
> >> >> > > > > > > > Clients
> >> >> > > > > > > > > >> > > > > currently don't retry on SASL auth failure
> >> (IIRC),
> >> >> > > but
> >> >> > > > > > since
> >> >> > > > > > > > the
> >> >> > > > > > > > > >> > > > > tokens propagate between brokers asynch,
> we
> >> will
> >> >> > > need
> >> >> > > > to
> >> >> > > > > > > > retry a
> >> >> > > > > > > > > >> bit
> >> >> > > > > > > > > >> > > > > to avoid clients failing auth due to
> timing
> >> >> > issues.
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *I am considering 2 alternatives right
> now.
> >> The
> >> >> > > > current
> >> >> > > > > > > > > documented
> >> >> > > > > > > > > >> > > > approach
> >> >> > > > > > > > > >> > > > > is zookeeper based and it does not require
> >> any
> >> >> > > changes
> >> >> > > > > to
> >> >> > > > > > > > > >> > > UpdateMetadata
> >> >> > > > > > > > > >> > > > > protocol. An alternative approach can
> remove
> >> >> > > zookeeper
> >> >> > > > > > > > > dependency
> >> >> > > > > > > > > >> as
> >> >> > > > > > > > > >> > > well
> >> >> > > > > > > > > >> > > > > but we can discuss that in KIP discussion
> >> call.*
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to
> ping
> >> Jun to
> >> >> > > > > > arrange a
> >> >> > > > > > > > > call?
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of having
> >> just the
> >> >> > > > > > controller
> >> >> > > > > > > > > issue
> >> >> > > > > > > > > >> the
> >> >> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a
> shared
> >> >> > secret.
> >> >> > > > Not
> >> >> > > > > > > sure
> >> >> > > > > > > > if
> >> >> > > > > > > > > >> we
> >> >> > > > > > > > > >> > > > > want to continue the discussion here or on
> >> the
> >> >> > > wiki. I
> >> >> > > > > > think
> >> >> > > > > > > > > that
> >> >> > > > > > > > > >> we
> >> >> > > > > > > > > >> > > > > can decouple the problem of "token
> >> distribution"
> >> >> > > from
> >> >> > > > > > > "shared
> >> >> > > > > > > > > >> secret
> >> >> > > > > > > > > >> > > > > distribution" and use the controller as
> the
> >> only
> >> >> > > token
> >> >> > > > > > > > generator
> >> >> > > > > > > > > >> to
> >> >> > > > > > > > > >> > > > > solve the second issue, while still using
> >> ZK async
> >> >> > > to
> >> >> > > > > > > > distribute
> >> >> > > > > > > > > >> > > > > tokens.
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *As mentioned in the previous Email I am
> >> fine with
> >> >> > > > that
> >> >> > > > > > > > approach
> >> >> > > > > > > > > >> as
> >> >> > > > > > > > > >> > > long
> >> >> > > > > > > > > >> > > > as
> >> >> > > > > > > > > >> > > > > we agree that the extra complexity of
> >> >> > > adding/updating
> >> >> > > > > APIS
> >> >> > > > > > > > adds
> >> >> > > > > > > > > >> enough
> >> >> > > > > > > > > >> > > > > value. The advantage with the controller
> >> approach
> >> >> > is
> >> >> > > > > > secret
> >> >> > > > > > > > > >> rotation
> >> >> > > > > > > > > >> > > can
> >> >> > > > > > > > > >> > > > be
> >> >> > > > > > > > > >> > > > > automated,frequent and would not require
> >> >> > > deployment. *
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Can you detail the extra complexity (or
> point
> >> me to
> >> >> > > the
> >> >> > > > > > email
> >> >> > > > > > > I
> >> >> > > > > > > > > >> > > > missed?) - which Apis are required?
> >> >> > > > > > > > > >> > > > As far as I can tell, clients can already
> >> find the
> >> >> > > > > > controller
> >> >> > > > > > > > from
> >> >> > > > > > > > > >> > > > metadata. I'm a bit more concerned about
> >> controller
> >> >> > > > load.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > * While I like the idea of forcing
> kerberos
> >> auth
> >> >> > for
> >> >> > > > > > > renwal, I
> >> >> > > > > > > > > >> think
> >> >> > > > > > > > > >> > > > > it mixes the transport layer the the
> request
> >> >> > content
> >> >> > > > in
> >> >> > > > > a
> >> >> > > > > > > > pretty
> >> >> > > > > > > > > >> ugly
> >> >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to non-owner
> >> is
> >> >> > > better.
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *I feel this is a necessary evil. While
> >> this will
> >> >> > > make
> >> >> > > > > the
> >> >> > > > > > > > kafka
> >> >> > > > > > > > > >> code
> >> >> > > > > > > > > >> > > > > pretty straight forward , forcing  renewer
> >> to
> >> >> > > > non-owner
> >> >> > > > > > > pushes
> >> >> > > > > > > > > >> the code
> >> >> > > > > > > > > >> > > > > ugliness to client and makes it even
> harder
> >> to
> >> >> > > > > > integrate.  *
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > As mentioned before, I don't think the
> >> "renewal by
> >> >> > > > other"
> >> >> > > > > > > > approach
> >> >> > > > > > > > > >> is
> >> >> > > > > > > > > >> > > > that ugly for the clients we expect to use
> >> >> > delegation
> >> >> > > > > tokens
> >> >> > > > > > > > since
> >> >> > > > > > > > > >> > > > they will have an app-master of some sort
> who
> >> >> > > requested
> >> >> > > > > the
> >> >> > > > > > > > token
> >> >> > > > > > > > > to
> >> >> > > > > > > > > >> > > > begin with.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > The response for my question on how
> multiple
> >> >> > > > identities
> >> >> > > > > > will
> >> >> > > > > > > > be
> >> >> > > > > > > > > >> > > > > handled wasn't super clear to me - AFAIK,
> >> we don't
> >> >> > > > > > > > authenticate
> >> >> > > > > > > > > >> each
> >> >> > > > > > > > > >> > > > > request, we authenticate connections.
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > *We authenticate connections, and only
> when
> >> they
> >> >> > are
> >> >> > > > > being
> >> >> > > > > > > > > >> established.
> >> >> > > > > > > > > >> > > > Let
> >> >> > > > > > > > > >> > > > > me try to phrase this as a question, in
> >> absence of
> >> >> > > > > > > delegation
> >> >> > > > > > > > > >> tokens if
> >> >> > > > > > > > > >> > > > we
> >> >> > > > > > > > > >> > > > > had to support the use case using user
> >> TGT's how
> >> >> > > would
> >> >> > > > > we
> >> >> > > > > > do
> >> >> > > > > > > > it?
> >> >> > > > > > > > > >> My
> >> >> > > > > > > > > >> > > point
> >> >> > > > > > > > > >> > > > > was it would be no different with
> delegation
> >> >> > tokens.
> >> >> > > > The
> >> >> > > > > > use
> >> >> > > > > > > > > case
> >> >> > > > > > > > > >> you
> >> >> > > > > > > > > >> > > are
> >> >> > > > > > > > > >> > > > > describing seems more like impersonation.*
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Yeah, I thought that one of the things that
> >> >> > delegation
> >> >> > > > > > tokens
> >> >> > > > > > > > > >> handled.
> >> >> > > > > > > > > >> > > > Maybe I got it wrong :)
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Thanks for the detailed answers.
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > Gwen
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > > > > Thanks
> >> >> > > > > > > > > >> > > > > Parth
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen
> >> Shapira <
> >> >> > > > > > > > > g...@confluent.io
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > > > wrote:
> >> >> > > > > > > > > >> > > > >
> >> >> > > > > > > > > >> > > > >> Hi Parth and Harsha,
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> Few more comments:
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * The API / RequestResponse section
> >> doesn't seem
> >> >> > to
> >> >> > > > > have
> >> >> > > > > > > good
> >> >> > > > > > > > > >> > > > >> description of the changes to the Kafka
> >> Protocol.
> >> >> > > > > Sounds
> >> >> > > > > > > like
> >> >> > > > > > > > > >> you are
> >> >> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest and
> >> >> > > > > > RenewTokenRequest
> >> >> > > > > > > > (and
> >> >> > > > > > > > > >> > > > >> matching responses), without detailing
> the
> >> >> > contents
> >> >> > > > of
> >> >> > > > > > the
> >> >> > > > > > > > > >> requests
> >> >> > > > > > > > > >> > > > >> and responses? Or rather, you show the
> >> class
> >> >> > > > interface,
> >> >> > > > > > but
> >> >> > > > > > > > not
> >> >> > > > > > > > > >> the
> >> >> > > > > > > > > >> > > > >> underlying protocol. This could be seen
> as
> >> an
> >> >> > > > > > > implementation
> >> >> > > > > > > > > >> detail,
> >> >> > > > > > > > > >> > > > >> but since the binary protocol is what we
> >> provide
> >> >> > to
> >> >> > > > > > > non-Java
> >> >> > > > > > > > > >> clients,
> >> >> > > > > > > > > >> > > > >> we need to show the changes there.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds like
> >> >> > > > > > > > delegationTokenRequestHandler?
> >> >> > > > > > > > > >> Is it
> >> >> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or
> Client?
> >> Its
> >> >> > > > > unclear...
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * I want to emphasize that even though
> >> delegation
> >> >> > > > > tokens
> >> >> > > > > > > are
> >> >> > > > > > > > a
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > >> innovation, I feel very strongly about
> not
> >> adding
> >> >> > > > > > > dependency
> >> >> > > > > > > > on
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > >> when implementing delegation tokens for
> >> Kafka.
> >> >> > The
> >> >> > > > KIP
> >> >> > > > > > > > doesn't
> >> >> > > > > > > > > >> imply
> >> >> > > > > > > > > >> > > > >> such dependency, but if you can
> clarify...
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * Can we get delegation token at any time
> >> after
> >> >> > > > > > > > authenticating?
> >> >> > > > > > > > > >> only
> >> >> > > > > > > > > >> > > > >> immediately after?
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * My understanding is that tokens will
> >> propagate
> >> >> > > via
> >> >> > > > ZK
> >> >> > > > > > but
> >> >> > > > > > > > > >> without
> >> >> > > > > > > > > >> > > > >> additional changes to UpdateMetadata
> >> protocol,
> >> >> > > > correct?
> >> >> > > > > > > > Clients
> >> >> > > > > > > > > >> > > > >> currently don't retry on SASL auth
> failure
> >> >> > (IIRC),
> >> >> > > > but
> >> >> > > > > > > since
> >> >> > > > > > > > > the
> >> >> > > > > > > > > >> > > > >> tokens propagate between brokers asynch,
> >> we will
> >> >> > > need
> >> >> > > > > to
> >> >> > > > > > > > retry
> >> >> > > > > > > > > a
> >> >> > > > > > > > > >> bit
> >> >> > > > > > > > > >> > > > >> to avoid clients failing auth due to
> timing
> >> >> > issues.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients not
> >> touching ZK
> >> >> > > > directly
> >> >> > > > > > :)
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of having
> >> just the
> >> >> > > > > > controller
> >> >> > > > > > > > > >> issue the
> >> >> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a
> >> shared
> >> >> > > secret.
> >> >> > > > > Not
> >> >> > > > > > > sure
> >> >> > > > > > > > > if
> >> >> > > > > > > > > >> we
> >> >> > > > > > > > > >> > > > >> want to continue the discussion here or
> on
> >> the
> >> >> > > wiki.
> >> >> > > > I
> >> >> > > > > > > think
> >> >> > > > > > > > > >> that we
> >> >> > > > > > > > > >> > > > >> can decouple the problem of "token
> >> distribution"
> >> >> > > from
> >> >> > > > > > > "shared
> >> >> > > > > > > > > >> secret
> >> >> > > > > > > > > >> > > > >> distribution" and use the controller as
> >> the only
> >> >> > > > token
> >> >> > > > > > > > > generator
> >> >> > > > > > > > > >> to
> >> >> > > > > > > > > >> > > > >> solve the second issue, while still using
> >> ZK
> >> >> > async
> >> >> > > to
> >> >> > > > > > > > > distribute
> >> >> > > > > > > > > >> > > > >> tokens.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * I am also uncomfortable with infinite
> >> lifetime
> >> >> > of
> >> >> > > > > > tokens
> >> >> > > > > > > > (and
> >> >> > > > > > > > > >> hoped
> >> >> > > > > > > > > >> > > > >> to hear from others in the community) -
> but
> >> >> > having
> >> >> > > > the
> >> >> > > > > > > option
> >> >> > > > > > > > > and
> >> >> > > > > > > > > >> > > > >> documenting it as less secure, allows
> >> users to
> >> >> > > > > configure
> >> >> > > > > > > > their
> >> >> > > > > > > > > >> system
> >> >> > > > > > > > > >> > > > >> as the wish.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * While I like the idea of forcing
> >> kerberos auth
> >> >> > > for
> >> >> > > > > > > renwal,
> >> >> > > > > > > > I
> >> >> > > > > > > > > >> think
> >> >> > > > > > > > > >> > > > >> it mixes the transport layer the the
> >> request
> >> >> > > content
> >> >> > > > > in a
> >> >> > > > > > > > > pretty
> >> >> > > > > > > > > >> ugly
> >> >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to
> non-owner
> >> is
> >> >> > > better.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> Things I'd still like to see:
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * More detailed explanation on what we
> >> plan to do
> >> >> > > for
> >> >> > > > > the
> >> >> > > > > > > > java
> >> >> > > > > > > > > >> clients
> >> >> > > > > > > > > >> > > > >> specifically - new configuration? new
> APIs?
> >> >> > > > > > > > > >> > > > >> The response for my question on how
> >> multiple
> >> >> > > > identities
> >> >> > > > > > > will
> >> >> > > > > > > > be
> >> >> > > > > > > > > >> > > > >> handled wasn't super clear to me - AFAIK,
> >> we
> >> >> > don't
> >> >> > > > > > > > authenticate
> >> >> > > > > > > > > >> each
> >> >> > > > > > > > > >> > > > >> request, we authenticate connections.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are
> only
> >> used
> >> >> > in
> >> >> > > > the
> >> >> > > > > > > Hadoop
> >> >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are
> >> >> > alternatives
> >> >> > > in
> >> >> > > > > > other
> >> >> > > > > > > > > >> ecosystems
> >> >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and whether
> >> there
> >> >> > are
> >> >> > > > some
> >> >> > > > > > > > > >> advantages
> >> >> > > > > > > > > >> > > > >> there.
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> Gwen
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha <
> >> >> > > > > ka...@harsha.io
> >> >> > > > > > >
> >> >> > > > > > > > > wrote:
> >> >> > > > > > > > > >> > > > >> > Hi Gwen,
> >> >> > > > > > > > > >> > > > >> >            Can you look at Parth's last
> >> reply.
> >> >> > > Does
> >> >> > > > > it
> >> >> > > > > > > > answer
> >> >> > > > > > > > > >> your
> >> >> > > > > > > > > >> > > > >> >            concerns.
> >> >> > > > > > > > > >> > > > >> >
> >> >> > > > > > > > > >> > > > >> > Thanks,
> >> >> > > > > > > > > >> > > > >> > Harsha
> >> >> > > > > > > > > >> > > > >> >
> >> >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth
> >> >> > > brahmbhatt
> >> >> > > > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki
> >> already
> >> >> > has
> >> >> > > > > > details
> >> >> > > > > > > on
> >> >> > > > > > > > > >> token
> >> >> > > > > > > > > >> > > > >> >> expiration
> >> >> > > > > > > > > >> > > > >> >> under token acquisition process
> >> >> > > > > > > > > >> > > > >> >> <
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition
> >> >> > > > > > > > > >> > > > >> >.
> >> >> > > > > > > > > >> > > > >> >> Current proposal is that tokens will
> >> expire
> >> >> > > based
> >> >> > > > > on a
> >> >> > > > > > > > > server
> >> >> > > > > > > > > >> side
> >> >> > > > > > > > > >> > > > >> >> configuration (default 24 hours)
> unless
> >> >> > renewed.
> >> >> > > > > > Renewal
> >> >> > > > > > > > is
> >> >> > > > > > > > > >> only
> >> >> > > > > > > > > >> > > > allowed
> >> >> > > > > > > > > >> > > > >> >> until the max life time of token.
> >> >> > Alternatively
> >> >> > > we
> >> >> > > > > > could
> >> >> > > > > > > > > also
> >> >> > > > > > > > > >> make
> >> >> > > > > > > > > >> > > > that
> >> >> > > > > > > > > >> > > > >> >> an
> >> >> > > > > > > > > >> > > > >> >> optional param and the server side
> >> default can
> >> >> > > > serve
> >> >> > > > > > as
> >> >> > > > > > > > the
> >> >> > > > > > > > > >> upper
> >> >> > > > > > > > > >> > > > bound.
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> To your second point it will be done
> >> exactly
> >> >> > the
> >> >> > > > > same
> >> >> > > > > > > way
> >> >> > > > > > > > we
> >> >> > > > > > > > > >> would
> >> >> > > > > > > > > >> > > > >> >> support
> >> >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling client
> >> will have
> >> >> > > to
> >> >> > > > > put
> >> >> > > > > > > the
> >> >> > > > > > > > > >> tokens it
> >> >> > > > > > > > > >> > > > >> wants
> >> >> > > > > > > > > >> > > > >> >> to use in the subject instance and
> call
> >> >> > > > > > produce/consume
> >> >> > > > > > > > > inside
> >> >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will have to
> >> keep
> >> >> > > track
> >> >> > > > of
> >> >> > > > > > its
> >> >> > > > > > > > own
> >> >> > > > > > > > > >> > > > subject. I
> >> >> > > > > > > > > >> > > > >> >> will have to look at the code to see
> if
> >> we
> >> >> > > support
> >> >> > > > > > this
> >> >> > > > > > > > > >> feature
> >> >> > > > > > > > > >> > > right
> >> >> > > > > > > > > >> > > > >> now
> >> >> > > > > > > > > >> > > > >> >> but my understanding is delegation
> token
> >> >> > > shouldn't
> >> >> > > > > > need
> >> >> > > > > > > > any
> >> >> > > > > > > > > >> special
> >> >> > > > > > > > > >> > > > >> >> treatment as its just another type of
> >> >> > Credential
> >> >> > > > in
> >> >> > > > > > the
> >> >> > > > > > > > > >> subject.
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> I would also like to know what is your
> >> opinion
> >> >> > > > about
> >> >> > > > > > > > > infinite
> >> >> > > > > > > > > >> > > renewal
> >> >> > > > > > > > > >> > > > >> (my
> >> >> > > > > > > > > >> > > > >> >> recommendation is to not support
> this),
> >> tokens
> >> >> > > > > > renewing
> >> >> > > > > > > > them
> >> >> > > > > > > > > >> > > self(my
> >> >> > > > > > > > > >> > > > >> >> recommendation is to not support this)
> >> and
> >> >> > most
> >> >> > > > > > > > importantly
> >> >> > > > > > > > > >> your
> >> >> > > > > > > > > >> > > > choice
> >> >> > > > > > > > > >> > > > >> >> between the alternatives listed on
> this
> >> thread
> >> >> > > > > > > > > >> > > > >> >> <
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >>
> http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism
> >> >> > > > > > > > > >> > > > >> >
> >> >> > > > > > > > > >> > > > >> >> ( I am leaning towards the
> >> alternative-2 minus
> >> >> > > > > > > controller
> >> >> > > > > > > > > >> > > > distributing
> >> >> > > > > > > > > >> > > > >> >> secret). Thanks again for reviewing.
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> Thanks
> >> >> > > > > > > > > >> > > > >> >> Parth
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen
> >> Shapira <
> >> >> > > > > > > > > >> g...@confluent.io>
> >> >> > > > > > > > > >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >>
> >> >> > > > > > > > > >> > > > >> >> > Harsha,
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy. I
> >> didn't
> >> >> > see
> >> >> > > > > your
> >> >> > > > > > > > design
> >> >> > > > > > > > > >> yet,
> >> >> > > > > > > > > >> > > > but in
> >> >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of
> >> producers, which
> >> >> > > > will
> >> >> > > > > > > serve
> >> >> > > > > > > > > >> multiple
> >> >> > > > > > > > > >> > > > users
> >> >> > > > > > > > > >> > > > >> >> > going through the proxy. Since these
> >> users
> >> >> > > will
> >> >> > > > > have
> >> >> > > > > > > > > >> different
> >> >> > > > > > > > > >> > > > >> >> > privileges, they'll need to
> >> authenticate
> >> >> > > > > separately,
> >> >> > > > > > > and
> >> >> > > > > > > > > >> can't
> >> >> > > > > > > > > >> > > > share a
> >> >> > > > > > > > > >> > > > >> >> > token.
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > Am I missing anything?
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > Gwen
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM,
> >> Harsha <
> >> >> > > > > > > ka...@harsha.io
> >> >> > > > > > > > >
> >> >> > > > > > > > > >> wrote:
> >> >> > > > > > > > > >> > > > >> >> > > Gwen,
> >> >> > > > > > > > > >> > > > >> >> > >            On your second point.
> >> Can you
> >> >> > > > > describe
> >> >> > > > > > a
> >> >> > > > > > > > > >> usecase
> >> >> > > > > > > > > >> > > where
> >> >> > > > > > > > > >> > > > >> >> > >            mutliple clients ended
> up
> >> >> > > sharing a
> >> >> > > > > > > > producer
> >> >> > > > > > > > > >> and
> >> >> > > > > > > > > >> > > even
> >> >> > > > > > > > > >> > > > if
> >> >> > > > > > > > > >> > > > >> they
> >> >> > > > > > > > > >> > > > >> >> > >            do why can't they not
> use
> >> >> > single
> >> >> > > > > token
> >> >> > > > > > > that
> >> >> > > > > > > > > >> producer
> >> >> > > > > > > > > >> > > > >> >> > >            captures. Why would we
> >> need
> >> >> > > > multiple
> >> >> > > > > > > > clients
> >> >> > > > > > > > > >> with
> >> >> > > > > > > > > >> > > > >> different
> >> >> > > > > > > > > >> > > > >> >> > >            tokens sharing a single
> >> >> > instance
> >> >> > > of
> >> >> > > > > > > > producer.
> >> >> > > > > > > > > >> Also
> >> >> > > > > > > > > >> > > in
> >> >> > > > > > > > > >> > > > >> this
> >> >> > > > > > > > > >> > > > >> >> > >            case other clients have
> >> access
> >> >> > > all
> >> >> > > > > the
> >> >> > > > > > > > tokens
> >> >> > > > > > > > > >> no?
> >> >> > > > > > > > > >> > > > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > > Thanks,
> >> >> > > > > > > > > >> > > > >> >> > > Harsha
> >> >> > > > > > > > > >> > > > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM,
> >> Gwen
> >> >> > > Shapira
> >> >> > > > > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay:
> >> >> > > > > > > > > >> > > > >> >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't see
> >> in the
> >> >> > > wiki:
> >> >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for
> >> delegation
> >> >> > > > > tokens?
> >> >> > > > > > > > > >> Renewal? How
> >> >> > > > > > > > > >> > > > do we
> >> >> > > > > > > > > >> > > > >> >> > >> revoke them?
> >> >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation
> >> tokens
> >> >> > for
> >> >> > > > > > "do-as"
> >> >> > > > > > > > > (say,
> >> >> > > > > > > > > >> > > submit
> >> >> > > > > > > > > >> > > > >> Storm
> >> >> > > > > > > > > >> > > > >> >> > >> job as my user), we will need a
> >> producer
> >> >> > > for
> >> >> > > > > > every
> >> >> > > > > > > > job
> >> >> > > > > > > > > >> (we
> >> >> > > > > > > > > >> > > can't
> >> >> > > > > > > > > >> > > > >> share
> >> >> > > > > > > > > >> > > > >> >> > >> them between multiple jobs
> running
> >> on
> >> >> > same
> >> >> > > > > node),
> >> >> > > > > > > > since
> >> >> > > > > > > > > >> we
> >> >> > > > > > > > > >> > > only
> >> >> > > > > > > > > >> > > > >> >> > >> authenticate when connecting. Is
> >> there a
> >> >> > > plan
> >> >> > > > > to
> >> >> > > > > > > > change
> >> >> > > > > > > > > >> this
> >> >> > > > > > > > > >> > > for
> >> >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order to
> >> allow
> >> >> > > multiple
> >> >> > > > > > users
> >> >> > > > > > > > > with
> >> >> > > > > > > > > >> > > > different
> >> >> > > > > > > > > >> > > > >> >> > >> tokens to share a client?
> >> >> > > > > > > > > >> > > > >> >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> Gwen
> >> >> > > > > > > > > >> > > > >> >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM,
> >> parth
> >> >> > > > > brahmbhatt
> >> >> > > > > > > > > >> > > > >> >> > >> <brahmbhatt.pa...@gmail.com>
> >> wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more time,
> >> can
> >> >> > other
> >> >> > > > > > > committers
> >> >> > > > > > > > > >> review?
> >> >> > > > > > > > > >> > > > >> >> > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> > Thanks
> >> >> > > > > > > > > >> > > > >> >> > >> > Parth
> >> >> > > > > > > > > >> > > > >> >> > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07
> AM,
> >> >> > Harsha <
> >> >> > > > > > > > > >> ka...@harsha.io>
> >> >> > > > > > > > > >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> Parth,
> >> >> > > > > > > > > >> > > > >> >> > >> >>           Overall current
> >> design looks
> >> >> > > > good
> >> >> > > > > to
> >> >> > > > > > > > me. I
> >> >> > > > > > > > > >> am +1
> >> >> > > > > > > > > >> > > on
> >> >> > > > > > > > > >> > > > >> the
> >> >> > > > > > > > > >> > > > >> >> > KIP.
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review this
> >> as
> >> >> > well.
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> -Harsha
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57
> >> AM,
> >> >> > parth
> >> >> > > > > > > > brahmbhatt
> >> >> > > > > > > > > >> wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review Jitendra.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of
> >> infinite
> >> >> > > > lifetime
> >> >> > > > > > > but I
> >> >> > > > > > > > > >> see the
> >> >> > > > > > > > > >> > > > >> Streaming
> >> >> > > > > > > > > >> > > > >> >> > use
> >> >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming use
> >> case I
> >> >> > > was
> >> >> > > > > > hoping
> >> >> > > > > > > > > >> there will
> >> >> > > > > > > > > >> > > > be
> >> >> > > > > > > > > >> > > > >> some
> >> >> > > > > > > > > >> > > > >> >> > notion
> >> >> > > > > > > > > >> > > > >> >> > >> >> > of
> >> >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can get
> new
> >> >> > > > delegation
> >> >> > > > > > > tokens
> >> >> > > > > > > > > at
> >> >> > > > > > > > > >> fixed
> >> >> > > > > > > > > >> > > > >> interval
> >> >> > > > > > > > > >> > > > >> >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If
> >> that is
> >> >> > not
> >> >> > > > the
> >> >> > > > > > case
> >> >> > > > > > > > for
> >> >> > > > > > > > > >> we can
> >> >> > > > > > > > > >> > > > >> discuss
> >> >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing
> >> them self
> >> >> > > and
> >> >> > > > > the
> >> >> > > > > > > > > >> security
> >> >> > > > > > > > > >> > > > >> implications
> >> >> > > > > > > > > >> > > > >> >> > of the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > same.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to
> >> fetch
> >> >> > tokens
> >> >> > > > > from
> >> >> > > > > > > > > >> zookeeper,
> >> >> > > > > > > > > >> > > > >> overall I
> >> >> > > > > > > > > >> > > > >> >> > think
> >> >> > > > > > > > > >> > > > >> >> > >> >> > its
> >> >> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't rely
> >> on our
> >> >> > > > > metadata
> >> >> > > > > > > > store
> >> >> > > > > > > > > >> and I
> >> >> > > > > > > > > >> > > > >> think we
> >> >> > > > > > > > > >> > > > >> >> > are
> >> >> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction
> with
> >> all
> >> >> > the
> >> >> > > > > KIP-4
> >> >> > > > > > > > > >> improvements.
> >> >> > > > > > > > > >> > > > I
> >> >> > > > > > > > > >> > > > >> chose
> >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case
> the
> >> client
> >> >> > > > will
> >> >> > > > > > > still
> >> >> > > > > > > > > >> just talk
> >> >> > > > > > > > > >> > > > to
> >> >> > > > > > > > > >> > > > >> >> > broker , its
> >> >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will use
> >> zookeeper
> >> >> > > > which
> >> >> > > > > we
> >> >> > > > > > > > > >> already do
> >> >> > > > > > > > > >> > > > for a
> >> >> > > > > > > > > >> > > > >> lot
> >> >> > > > > > > > > >> > > > >> >> > of
> >> >> > > > > > > > > >> > > > >> >> > >> >> > other
> >> >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of
> >> development + and
> >> >> > > the
> >> >> > > > > > > ability
> >> >> > > > > > > > > so
> >> >> > > > > > > > > >> > > tokens
> >> >> > > > > > > > > >> > > > >> will
> >> >> > > > > > > > > >> > > > >> >> > survive
> >> >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling
> restart/cluster
> >> >> > > failure.
> >> >> > > > > if a
> >> >> > > > > > > > > >> majority
> >> >> > > > > > > > > >> > > > agrees
> >> >> > > > > > > > > >> > > > >> the
> >> >> > > > > > > > > >> > > > >> >> > added
> >> >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have
> controller
> >> >> > > forwarding
> >> >> > > > > > keys
> >> >> > > > > > > to
> >> >> > > > > > > > > all
> >> >> > > > > > > > > >> > > > broker is
> >> >> > > > > > > > > >> > > > >> >> > justified
> >> >> > > > > > > > > >> > > > >> >> > >> >> > as
> >> >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter security
> >> , I am
> >> >> > > fine
> >> >> > > > > > with
> >> >> > > > > > > > that
> >> >> > > > > > > > > >> option
> >> >> > > > > > > > > >> > > > too.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not
> >> support SSL
> >> >> > > we
> >> >> > > > > can
> >> >> > > > > > > not
> >> >> > > > > > > > > >> store
> >> >> > > > > > > > > >> > > > master
> >> >> > > > > > > > > >> > > > >> keys
> >> >> > > > > > > > > >> > > > >> >> > in
> >> >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys
> will
> >> be
> >> >> > > exposed
> >> >> > > > > on
> >> >> > > > > > > > wire.
> >> >> > > > > > > > > To
> >> >> > > > > > > > > >> > > > support
> >> >> > > > > > > > > >> > > > >> >> > rotation
> >> >> > > > > > > > > >> > > > >> >> > >> >> > without affecting current
> >> clients is
> >> >> > > > > > > something I
> >> >> > > > > > > > > >> need to
> >> >> > > > > > > > > >> > > > put
> >> >> > > > > > > > > >> > > > >> more
> >> >> > > > > > > > > >> > > > >> >> > thought
> >> >> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal
> >> assumes the
> >> >> > > > > rotation
> >> >> > > > > > > > will
> >> >> > > > > > > > > >> > > > invalidate
> >> >> > > > > > > > > >> > > > >> all
> >> >> > > > > > > > > >> > > > >> >> > current
> >> >> > > > > > > > > >> > > > >> >> > >> >> > tokens.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > I request committers to also
> >> review
> >> >> > > and
> >> >> > > > > post
> >> >> > > > > > > > their
> >> >> > > > > > > > > >> > > comments
> >> >> > > > > > > > > >> > > > >> so we
> >> >> > > > > > > > > >> > > > >> >> > can
> >> >> > > > > > > > > >> > > > >> >> > >> >> > make
> >> >> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP.
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > Thanks
> >> >> > > > > > > > > >> > > > >> >> > >> >> > Parth
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39
> >> AM,
> >> >> > > Ashish
> >> >> > > > > > Singh
> >> >> > > > > > > <
> >> >> > > > > > > > > >> > > > >> asi...@cloudera.com
> >> >> > > > > > > > > >> > > > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at
> >> 11:26 AM,
> >> >> > > > > Harsha <
> >> >> > > > > > > > > >> > > > ka...@harsha.io>
> >> >> > > > > > > > > >> > > > >> >> > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two
> >> discussion
> >> >> > > threads
> >> >> > > > on
> >> >> > > > > > > this
> >> >> > > > > > > > > KIP.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response
> from
> >> >> > Jitendra
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large
> >> number of
> >> >> > > > > clients
> >> >> > > > > > > that
> >> >> > > > > > > > > are
> >> >> > > > > > > > > >> > > > running
> >> >> > > > > > > > > >> > > > >> all
> >> >> > > > > > > > > >> > > > >> >> > over the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that
> authenticate
> >> with
> >> >> > > Kafka
> >> >> > > > > > > > brokers,
> >> >> > > > > > > > > >> is very
> >> >> > > > > > > > > >> > > > >> similar
> >> >> > > > > > > > > >> > > > >> >> > to the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of large
> >> number
> >> >> > of
> >> >> > > > > tasks
> >> >> > > > > > > > > running
> >> >> > > > > > > > > >> > > across
> >> >> > > > > > > > > >> > > > >> the
> >> >> > > > > > > > > >> > > > >> >> > cluster
> >> >> > > > > > > > > >> > > > >> >> > >> >> that
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication to
> >> Hdfs
> >> >> > > > Namenode.
> >> >> > > > > > > > > >> Therefore, the
> >> >> > > > > > > > > >> > > > >> >> > delegation token
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem like
> a
> >> good
> >> >> > fit
> >> >> > > > for
> >> >> > > > > > > this
> >> >> > > > > > > > > use
> >> >> > > > > > > > > >> case
> >> >> > > > > > > > > >> > > > as we
> >> >> > > > > > > > > >> > > > >> >> > have seen
> >> >> > > > > > > > > >> > > > >> >> > >> >> it
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large scale
> in
> >> HDFS
> >> >> > and
> >> >> > > > > YARN.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >   The proposed design is
> >> very
> >> >> > much
> >> >> > > > > > inline
> >> >> > > > > > > > with
> >> >> > > > > > > > > >> Hadoop
> >> >> > > > > > > > > >> > > > >> >> > approach. A few
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >   comments:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys want
> >> to allow
> >> >> > > > > > infinite
> >> >> > > > > > > > > >> renewable
> >> >> > > > > > > > > >> > > > >> lifetime
> >> >> > > > > > > > > >> > > > >> >> > for a
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts a
> >> token
> >> >> > to a
> >> >> > > > max
> >> >> > > > > > > life
> >> >> > > > > > > > > time
> >> >> > > > > > > > > >> > > > (default
> >> >> > > > > > > > > >> > > > >> 7
> >> >> > > > > > > > > >> > > > >> >> > days).  A
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability is
> >> >> > believed
> >> >> > > to
> >> >> > > > > > > > increase
> >> >> > > > > > > > > >> with
> >> >> > > > > > > > > >> > > > time.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having
> infinite
> >> >> > > lifetime
> >> >> > > > > > might
> >> >> > > > > > > > not
> >> >> > > > > > > > > >> be the
> >> >> > > > > > > > > >> > > > best
> >> >> > > > > > > > > >> > > > >> idea.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the
> >> tokens
> >> >> > are
> >> >> > > > > stored
> >> >> > > > > > > in
> >> >> > > > > > > > > >> zookeeper
> >> >> > > > > > > > > >> > > > as
> >> >> > > > > > > > > >> > > > >> well,
> >> >> > > > > > > > > >> > > > >> >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> can
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This
> is
> >> clever
> >> >> > > as
> >> >> > > > it
> >> >> > > > > > can
> >> >> > > > > > > > > allow
> >> >> > > > > > > > > >> > > > >> replacing the
> >> >> > > > > > > > > >> > > > >> >> > tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of max
> >> life
> >> >> > > time,
> >> >> > > > > and
> >> >> > > > > > > > > clients
> >> >> > > > > > > > > >> can
> >> >> > > > > > > > > >> > > > >> download
> >> >> > > > > > > > > >> > > > >> >> > new
> >> >> > > > > > > > > >> > > > >> >> > >> >> tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It
> >> shouldn't be
> >> >> > a
> >> >> > > > big
> >> >> > > > > > load
> >> >> > > > > > > > on
> >> >> > > > > > > > > >> > > zookeeper
> >> >> > > > > > > > > >> > > > >> as a
> >> >> > > > > > > > > >> > > > >> >> > client
> >> >> > > > > > > > > >> > > > >> >> > >> >> will
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new token
> >> once in
> >> >> > > > > several
> >> >> > > > > > > > days.
> >> >> > > > > > > > > >> In this
> >> >> > > > > > > > > >> > > > >> approach
> >> >> > > > > > > > > >> > > > >> >> > you
> >> >> > > > > > > > > >> > > > >> >> > >> >> don't
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime
> on
> >> the
> >> >> > > token
> >> >> > > > > even
> >> >> > > > > > > for
> >> >> > > > > > > > > >> long
> >> >> > > > > > > > > >> > > > running
> >> >> > > > > > > > > >> > > > >> >> > clients.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password
> are
> >> >> > > generated
> >> >> > > > > > using
> >> >> > > > > > > a
> >> >> > > > > > > > > >> master
> >> >> > > > > > > > > >> > > key.
> >> >> > > > > > > > > >> > > > >> The
> >> >> > > > > > > > > >> > > > >> >> > master
> >> >> > > > > > > > > >> > > > >> >> > >> >> key
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be
> >> periodically
> >> >> > > changed.
> >> >> > > > > In
> >> >> > > > > > > > > Hadoop,
> >> >> > > > > > > > > >> the
> >> >> > > > > > > > > >> > > > >> default
> >> >> > > > > > > > > >> > > > >> >> > renewal
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.?
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require
> >> brokers
> >> >> > > > > > maintaining
> >> >> > > > > > > a
> >> >> > > > > > > > > >> list of X
> >> >> > > > > > > > > >> > > > most
> >> >> > > > > > > > > >> > > > >> >> > recent
> >> >> > > > > > > > > >> > > > >> >> > >> >> master
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will have
> >> to be
> >> >> > > > > persisted
> >> >> > > > > > > > > >> somewhere, as
> >> >> > > > > > > > > >> > > > if a
> >> >> > > > > > > > > >> > > > >> >> > broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> goes
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to get
> >> that list
> >> >> > > > again
> >> >> > > > > > and
> >> >> > > > > > > > > >> storing
> >> >> > > > > > > > > >> > > > master
> >> >> > > > > > > > > >> > > > >> keys
> >> >> > > > > > > > > >> > > > >> >> > on ZK
> >> >> > > > > > > > > >> > > > >> >> > >> >> is
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea.
> However,
> >> if a
> >> >> > > > broker
> >> >> > > > > > goes
> >> >> > > > > > > > > down
> >> >> > > > > > > > > >> then
> >> >> > > > > > > > > >> > > we
> >> >> > > > > > > > > >> > > > >> have
> >> >> > > > > > > > > >> > > > >> >> > much
> >> >> > > > > > > > > >> > > > >> >> > >> >> bigger
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and
> >> client can
> >> >> > > > always
> >> >> > > > > > > > > >> > > re-authenticate
> >> >> > > > > > > > > >> > > > is
> >> >> > > > > > > > > >> > > > >> such
> >> >> > > > > > > > > >> > > > >> >> > >> >> events.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a
> >> look at
> >> >> > > other
> >> >> > > > > > > > > >> alternatives
> >> >> > > > > > > > > >> > > this
> >> >> > > > > > > > > >> > > > >> list has
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > suggested?
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough
> >> proposal,
> >> >> > > > great
> >> >> > > > > > > work!"
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at
> >> 10:28
> >> >> > PM,
> >> >> > > > Gwen
> >> >> > > > > > > > Shapira
> >> >> > > > > > > > > >> wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me.
> >> Thanks!
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at
> >> 9:25
> >> >> > PM,
> >> >> > > > > > Harsha <
> >> >> > > > > > > > > >> > > > ka...@harsha.io
> >> >> > > > > > > > > >> > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need any
> >> release
> >> >> > > > > vehicle
> >> >> > > > > > > but
> >> >> > > > > > > > > >> still the
> >> >> > > > > > > > > >> > > > >> work can
> >> >> > > > > > > > > >> > > > >> >> > move
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > forward.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is
> >> interested in
> >> >> > the
> >> >> > > > KIP
> >> >> > > > > > > > please
> >> >> > > > > > > > > >> do the
> >> >> > > > > > > > > >> > > > >> review and
> >> >> > > > > > > > > >> > > > >> >> > >> >> provide
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016,
> >> at
> >> >> > 04:59
> >> >> > > > PM,
> >> >> > > > > > > Ismael
> >> >> > > > > > > > > >> Juma
> >> >> > > > > > > > > >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it
> >> would be
> >> >> > good
> >> >> > > > to
> >> >> > > > > > have
> >> >> > > > > > > > > more
> >> >> > > > > > > > > >> time
> >> >> > > > > > > > > >> > > to
> >> >> > > > > > > > > >> > > > >> review
> >> >> > > > > > > > > >> > > > >> >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > discuss
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016
> >> at
> >> >> > 12:55
> >> >> > > > AM,
> >> >> > > > > > Gwen
> >> >> > > > > > > > > >> Shapira <
> >> >> > > > > > > > > >> > > > >> >> > >> >> g...@confluent.io>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48
> >> depends on
> >> >> > > > KIP-43,
> >> >> > > > > > > which
> >> >> > > > > > > > > is
> >> >> > > > > > > > > >> > > > already a
> >> >> > > > > > > > > >> > > > >> bit
> >> >> > > > > > > > > >> > > > >> >> > of a
> >> >> > > > > > > > > >> > > > >> >> > >> >> risk
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > for
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next release
> -
> >> any
> >> >> > > chance
> >> >> > > > > we
> >> >> > > > > > > can
> >> >> > > > > > > > > >> delay
> >> >> > > > > > > > > >> > > > >> delegation
> >> >> > > > > > > > > >> > > > >> >> > tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1?
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the
> community
> >> >> > working
> >> >> > > > on a
> >> >> > > > > > > > release
> >> >> > > > > > > > > >> every
> >> >> > > > > > > > > >> > > 3
> >> >> > > > > > > > > >> > > > >> month,
> >> >> > > > > > > > > >> > > > >> >> > this
> >> >> > > > > > > > > >> > > > >> >> > >> >> is not
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26,
> >> 2016 at
> >> >> > > 5:11
> >> >> > > > > PM,
> >> >> > > > > > > > Ashish
> >> >> > > > > > > > > >> Singh
> >> >> > > > > > > > > >> > > <
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > asi...@cloudera.com>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again
> for
> >> the
> >> >> > > > awesome
> >> >> > > > > > > write
> >> >> > > > > > > > > up.
> >> >> > > > > > > > > >> > > > Following
> >> >> > > > > > > > > >> > > > >> our
> >> >> > > > > > > > > >> > > > >> >> > >> >> discussion
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think
> it
> >> will
> >> >> > be
> >> >> > > > > easier
> >> >> > > > > > > to
> >> >> > > > > > > > > >> compare
> >> >> > > > > > > > > >> > > > >> various
> >> >> > > > > > > > > >> > > > >> >> > >> >> alternatives
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > if they
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed
> together.
> >> I am
> >> >> > > > stating
> >> >> > > > > > > > below a
> >> >> > > > > > > > > >> few
> >> >> > > > > > > > > >> > > > >> >> > alternatives along
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > with
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current
> proposal.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current
> >> proposal)
> >> >> > Store
> >> >> > > > > > > Delegation
> >> >> > > > > > > > > >> Token,
> >> >> > > > > > > > > >> > > DT,
> >> >> > > > > > > > > >> > > > >> on ZK.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    1. Client
> >> >> > > authenticates
> >> >> > > > > > with a
> >> >> > > > > > > > > >> broker.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    2. Once a
> >> client is
> >> >> > > > > > > > authenticated,
> >> >> > > > > > > > > >> it
> >> >> > > > > > > > > >> > > will
> >> >> > > > > > > > > >> > > > >> make a
> >> >> > > > > > > > > >> > > > >> >> > broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> side
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    issue a
> >> delegation
> >> >> > > > token.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    3. The
> broker
> >> >> > > generates
> >> >> > > > a
> >> >> > > > > > > shared
> >> >> > > > > > > > > >> secret
> >> >> > > > > > > > > >> > > > based
> >> >> > > > > > > > > >> > > > >> on
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> Password/Secret
> >> >> > shared
> >> >> > > > > > between
> >> >> > > > > > > > all
> >> >> > > > > > > > > >> > > brokers,
> >> >> > > > > > > > > >> > > > >> >> > randomly
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > generated
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId).
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    4. Broker
> >> stores
> >> >> > this
> >> >> > > > > token
> >> >> > > > > > in
> >> >> > > > > > > > its
> >> >> > > > > > > > > >> in
> >> >> > > > > > > > > >> > > > memory
> >> >> > > > > > > > > >> > > > >> cache.
> >> >> > > > > > > > > >> > > > >> >> > >> >> Broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    the
> >> DelegationToken
> >> >> > > > > without
> >> >> > > > > > > the
> >> >> > > > > > > > > >> hmac in
> >> >> > > > > > > > > >> > > the
> >> >> > > > > > > > > >> > > > >> >> > zookeeper.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    5. All
> >> brokers will
> >> >> > > > have a
> >> >> > > > > > > cache
> >> >> > > > > > > > > >> backed
> >> >> > > > > > > > > >> > > by
> >> >> > > > > > > > > >> > > > >> >> > zookeeper so
> >> >> > > > > > > > > >> > > > >> >> > >> >> they
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > will all
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    get notified
> >> >> > whenever
> >> >> > > a
> >> >> > > > > new
> >> >> > > > > > > > token
> >> >> > > > > > > > > is
> >> >> > > > > > > > > >> > > > >> generated and
> >> >> > > > > > > > > >> > > > >> >> > they
> >> >> > > > > > > > > >> > > > >> >> > >> >> will
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > update
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    local cache
> >> whenever
> >> >> > > > token
> >> >> > > > > > > state
> >> >> > > > > > > > > >> changes.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    6. Broker
> >> returns
> >> >> > the
> >> >> > > > > token
> >> >> > > > > > to
> >> >> > > > > > > > > >> Client.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
> >> and
> >> >> > fixes
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    1. Probable
> >> race
> >> >> > > > > condition,
> >> >> > > > > > > > client
> >> >> > > > > > > > > >> tries
> >> >> > > > > > > > > >> > > to
> >> >> > > > > > > > > >> > > > >> >> > authenticate
> >> >> > > > > > > > > >> > > > >> >> > >> >> with
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    that is yet
> >> to be
> >> >> > > > updated
> >> >> > > > > > with
> >> >> > > > > > > > the
> >> >> > > > > > > > > >> newly
> >> >> > > > > > > > > >> > > > >> generated
> >> >> > > > > > > > > >> > > > >> >> > DT.
> >> >> > > > > > > > > >> > > > >> >> > >> >> This
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > can
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    dealt with
> >> making
> >> >> > > > > dtRequest
> >> >> > > > > > > > block
> >> >> > > > > > > > > >> until
> >> >> > > > > > > > > >> > > all
> >> >> > > > > > > > > >> > > > >> >> > brokers have
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > updated
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    cache. Zk
> >> barrier or
> >> >> > > > > similar
> >> >> > > > > > > > > >> mechanism
> >> >> > > > > > > > > >> > > can
> >> >> > > > > > > > > >> > > > be
> >> >> > > > > > > > > >> > > > >> used.
> >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > all such
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    mechanisms
> >> will
> >> >> > > increase
> >> >> > > > > > > > > complexity.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    2. Using a
> >> static
> >> >> > > secret
> >> >> > > > > key
> >> >> > > > > > > > from
> >> >> > > > > > > > > >> config
> >> >> > > > > > > > > >> > > > >> file. Will
> >> >> > > > > > > > > >> > > > >> >> > >> >> require
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > yet
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    config and
> >> uses a
> >> >> > > static
> >> >> > > > > > > secret
> >> >> > > > > > > > > >> key. It
> >> >> > > > > > > > > >> > > is
> >> >> > > > > > > > > >> > > > >> advised
> >> >> > > > > > > > > >> > > > >> >> > to
> >> >> > > > > > > > > >> > > > >> >> > >> >> rotate
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > secret
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> periodically.
> >> This
> >> >> > can
> >> >> > > > be
> >> >> > > > > > > > avoided
> >> >> > > > > > > > > >> with
> >> >> > > > > > > > > >> > > > >> controller
> >> >> > > > > > > > > >> > > > >> >> > >> >> generating
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    passing to
> >> brokers
> >> >> > > > > > > periodically.
> >> >> > > > > > > > > >> However,
> >> >> > > > > > > > > >> > > > >> this will
> >> >> > > > > > > > > >> > > > >> >> > >> >> require
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    maintain
> >> certain
> >> >> > > counts
> >> >> > > > of
> >> >> > > > > > > > > >> secretKeys.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 1)
> >> Have
> >> >> > > > > controller
> >> >> > > > > > > > > >> generate
> >> >> > > > > > > > > >> > > > >> delegation
> >> >> > > > > > > > > >> > > > >> >> > token.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    1. Client
> >> >> > > authenticates
> >> >> > > > > > with a
> >> >> > > > > > > > > >> broker.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    2. Once a
> >> client is
> >> >> > > > > > > > authenticated,
> >> >> > > > > > > > > >> it
> >> >> > > > > > > > > >> > > will
> >> >> > > > > > > > > >> > > > >> make a
> >> >> > > > > > > > > >> > > > >> >> > broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> side
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    issue a
> >> delegation
> >> >> > > > token.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    3. Broker
> >> forwards
> >> >> > the
> >> >> > > > > > request
> >> >> > > > > > > > to
> >> >> > > > > > > > > >> > > > controller.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    4.
> Controller
> >> >> > > generates
> >> >> > > > a
> >> >> > > > > DT
> >> >> > > > > > > and
> >> >> > > > > > > > > >> > > broadcasts
> >> >> > > > > > > > > >> > > > >> to all
> >> >> > > > > > > > > >> > > > >> >> > >> >> brokers.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    5. Broker
> >> stores
> >> >> > this
> >> >> > > > > token
> >> >> > > > > > in
> >> >> > > > > > > > its
> >> >> > > > > > > > > >> memory
> >> >> > > > > > > > > >> > > > >> cache.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    6.
> Controller
> >> >> > responds
> >> >> > > > to
> >> >> > > > > > > > broker’s
> >> >> > > > > > > > > >> DT
> >> >> > > > > > > > > >> > > req.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    7. Broker
> >> returns
> >> >> > the
> >> >> > > > > token
> >> >> > > > > > to
> >> >> > > > > > > > > >> Client.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
> >> and
> >> >> > fixes
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    1. We will
> >> have to
> >> >> > add
> >> >> > > > new
> >> >> > > > > > > APIs
> >> >> > > > > > > > to
> >> >> > > > > > > > > >> > > support
> >> >> > > > > > > > > >> > > > >> >> > controller
> >> >> > > > > > > > > >> > > > >> >> > >> >> pushing
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    brokers on
> >> top of
> >> >> > the
> >> >> > > > > > minimal
> >> >> > > > > > > > APIs
> >> >> > > > > > > > > >> that
> >> >> > > > > > > > > >> > > are
> >> >> > > > > > > > > >> > > > >> >> > currently
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > proposed.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    2. We will
> >> also have
> >> >> > > to
> >> >> > > > > add
> >> >> > > > > > > APIs
> >> >> > > > > > > > > to
> >> >> > > > > > > > > >> > > support
> >> >> > > > > > > > > >> > > > >> the
> >> >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > case,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    when a new
> >> broker
> >> >> > > comes
> >> >> > > > up
> >> >> > > > > > it
> >> >> > > > > > > > will
> >> >> > > > > > > > > >> have
> >> >> > > > > > > > > >> > > to
> >> >> > > > > > > > > >> > > > >> get all
> >> >> > > > > > > > > >> > > > >> >> > >> >> delegation
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    the
> >> controller.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    3. In
> >> catastrophic
> >> >> > > > > failures
> >> >> > > > > > > > where
> >> >> > > > > > > > > >> all
> >> >> > > > > > > > > >> > > > brokers
> >> >> > > > > > > > > >> > > > >> go
> >> >> > > > > > > > > >> > > > >> >> > down,
> >> >> > > > > > > > > >> > > > >> >> > >> >> the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    be lost even
> >> if
> >> >> > > servers
> >> >> > > > > are
> >> >> > > > > > > > > >> restarted as
> >> >> > > > > > > > > >> > > > >> tokens
> >> >> > > > > > > > > >> > > > >> >> > are not
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    If this
> >> happens,
> >> >> > then
> >> >> > > > > there
> >> >> > > > > > > are
> >> >> > > > > > > > > more
> >> >> > > > > > > > > >> > > > important
> >> >> > > > > > > > > >> > > > >> >> > things to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    maybe it is
> >> better
> >> >> > to
> >> >> > > > > > > > > >> re-authenticate.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 2)
> >> Do not
> >> >> > > > > > distribute
> >> >> > > > > > > > DT
> >> >> > > > > > > > > to
> >> >> > > > > > > > > >> > > other
> >> >> > > > > > > > > >> > > > >> brokers
> >> >> > > > > > > > > >> > > > >> >> > at
> >> >> > > > > > > > > >> > > > >> >> > >> >> all.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    1. Client
> >> >> > > authenticates
> >> >> > > > > > with a
> >> >> > > > > > > > > >> broker.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    2. Once a
> >> client is
> >> >> > > > > > > > authenticated,
> >> >> > > > > > > > > >> it
> >> >> > > > > > > > > >> > > will
> >> >> > > > > > > > > >> > > > >> make a
> >> >> > > > > > > > > >> > > > >> >> > broker
> >> >> > > > > > > > > >> > > > >> >> > >> >> side
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > call to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    issue a
> >> delegation
> >> >> > > > token.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    3. The
> broker
> >> >> > > generates
> >> >> > > > DT
> >> >> > > > > > of
> >> >> > > > > > > > > form,
> >> >> > > > > > > > > >> > > [hmac +
> >> >> > > > > > > > > >> > > > >> (owner,
> >> >> > > > > > > > > >> > > > >> >> > >> >> renewer,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    maxLifeTime,
> >> id,
> >> >> > hmac,
> >> >> > > > > > > > > >> expirationTime)]
> >> >> > > > > > > > > >> > > and
> >> >> > > > > > > > > >> > > > >> passes
> >> >> > > > > > > > > >> > > > >> >> > back
> >> >> > > > > > > > > >> > > > >> >> > >> >> this
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    hmac is
> >> generated
> >> >> > via
> >> >> > > > > > > > > >> {HMAC-SHA256(owner,
> >> >> > > > > > > > > >> > > > >> renewer,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> expirationTime)
> >> >> > using
> >> >> > > > > > > > SecretKey}.
> >> >> > > > > > > > > >> Note
> >> >> > > > > > > > > >> > > that
> >> >> > > > > > > > > >> > > > >> all
> >> >> > > > > > > > > >> > > > >> >> > brokers
> >> >> > > > > > > > > >> > > > >> >> > >> >> have
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    4. Client
> >> then goes
> >> >> > to
> >> >> > > > any
> >> >> > > > > > > > broker
> >> >> > > > > > > > > >> and to
> >> >> > > > > > > > > >> > > > >> >> > authenticate
> >> >> > > > > > > > > >> > > > >> >> > >> >> sends
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    Broker
> >> recalculates
> >> >> > > hmac
> >> >> > > > > > using
> >> >> > > > > > > > > >> (owner,
> >> >> > > > > > > > > >> > > > >> renewer,
> >> >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> expirationTime) info
> >> >> > > > from
> >> >> > > > > DT
> >> >> > > > > > > and
> >> >> > > > > > > > > its
> >> >> > > > > > > > > >> > > > >> SecretKey. If
> >> >> > > > > > > > > >> > > > >> >> > it
> >> >> > > > > > > > > >> > > > >> >> > >> >> matches
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > with
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    DT, client
> is
> >> >> > > > > authenticated.
> >> >> > > > > > > > Yes,
> >> >> > > > > > > > > >> it will
> >> >> > > > > > > > > >> > > > do
> >> >> > > > > > > > > >> > > > >> other
> >> >> > > > > > > > > >> > > > >> >> > >> >> obvious
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    timestamp
> >> expiry and
> >> >> > > > such.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that
> secret
> >> key
> >> >> > will
> >> >> > > > be
> >> >> > > > > > > > > generated
> >> >> > > > > > > > > >> by
> >> >> > > > > > > > > >> > > > >> controller
> >> >> > > > > > > > > >> > > > >> >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> passed
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues
> >> and
> >> >> > fixes
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    1. How to
> >> delete a
> >> >> > DT?
> >> >> > > > > Yes,
> >> >> > > > > > > that
> >> >> > > > > > > > > is
> >> >> > > > > > > > > >> a
> >> >> > > > > > > > > >> > > > downside
> >> >> > > > > > > > > >> > > > >> >> > here.
> >> >> > > > > > > > > >> > > > >> >> > >> >> However,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > this can
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    be handled
> >> with
> >> >> > > brokers
> >> >> > > > > > > > > maintaining
> >> >> > > > > > > > > >> a
> >> >> > > > > > > > > >> > > > >> blacklist of
> >> >> > > > > > > > > >> > > > >> >> > DTs,
> >> >> > > > > > > > > >> > > > >> >> > >> >> DTs
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > from this
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    can be
> >> removed after
> >> >> > > > > expiry.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    2. In
> >> catastrophic
> >> >> > > > > failures
> >> >> > > > > > > > where
> >> >> > > > > > > > > >> all
> >> >> > > > > > > > > >> > > > brokers
> >> >> > > > > > > > > >> > > > >> go
> >> >> > > > > > > > > >> > > > >> >> > down,
> >> >> > > > > > > > > >> > > > >> >> > >> >> the
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    be lost even
> >> if
> >> >> > > servers
> >> >> > > > > are
> >> >> > > > > > > > > >> restarted as
> >> >> > > > > > > > > >> > > > >> tokens
> >> >> > > > > > > > > >> > > > >> >> > are not
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    If this
> >> happens,
> >> >> > then
> >> >> > > > > there
> >> >> > > > > > > are
> >> >> > > > > > > > > more
> >> >> > > > > > > > > >> > > > important
> >> >> > > > > > > > > >> > > > >> >> > things to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > worry
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > about
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >    maybe it is
> >> better
> >> >> > to
> >> >> > > > > > > > > >> re-authenticate.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26,
> >> 2016 at
> >> >> > > > 1:58
> >> >> > > > > > PM,
> >> >> > > > > > > > > Parth
> >> >> > > > > > > > > >> > > > >> Brahmbhatt <
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > pbrahmbh...@hortonworks.com>
> >> >> > > > > > > > wrote:
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed
> >> KIP-48 so
> >> >> > > we
> >> >> > > > > can
> >> >> > > > > > > > offer
> >> >> > > > > > > > > >> hadoop
> >> >> > > > > > > > > >> > > > like
> >> >> > > > > > > > > >> > > > >> >> > delegation
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You can
> >> review
> >> >> > > the
> >> >> > > > > > design
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > .
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP
> >> depends on
> >> >> > > KIP-43
> >> >> > > > > and
> >> >> > > > > > > we
> >> >> > > > > > > > > >> have also
> >> >> > > > > > > > > >> > > > >> >> > discussed an
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed
> design
> >> here<
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >>
> https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >.
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >>
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > --
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > --
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > Regards,
> >> >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish
> >> >> > > > > > > > > >> > > > >> >> > >> >> > >
> >> >> > > > > > > > > >> > > > >> >> > >> >>
> >> >> > > > > > > > > >> > > > >> >> >
> >> >> > > > > > > > > >> > > > >>
> >> >> > > > > > > > > >> > > >
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > --
> >> >> > > > > > Regards,
> >> >> > > > > >
> >> >> > > > > > Rajini
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > --
> >> >> > > > Regards,
> >> >> > > >
> >> >> > > > Rajini
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Liquan Pei
> >> >> Software Engineer, Confluent Inc
> >>
> >
> >
>