This sounds promising and really fantastic news. We look forward to this feature, and let us know what we can do to help. Thanks
regards, Eric On Tue, May 26, 2020 at 10:55 AM Daryn Sharp <da...@verizonmedia.com.invalid> wrote: > There’s a few too many issues being mixed here. > > > We aren’t very far from having OIDC support. The pre-requisite RPC/TLS & > RPC/mTLS recently completed rollout to our entire production grid. > Majority of the past year was spent shaking out bugs and ensuring 100% > compatibility. There are a few rough edges I need to clean up for a > community release. > > > A few weeks ago I created a rough POC to leverage RPC/mTLS with OIDC access > tokens. Goal is a mTLS cert may be blessed to impersonate with an access > token. A compromised service may only be abused to impersonate users that > have recently accessed said service. > > > Kerberos, mTLs, and OIDC may all be simultaneously supported. Part of the > simplicity is regardless of the client’s authn/authz, delegation tokens are > still acquired by jobs to avoid short-lived identity credential expiration. > > > Credential refreshing is a bigger can of worms that requires careful > thought and a separate discussion. > > On Thu, May 21, 2020 at 12:32 PM Vipin Rathor <v.rat...@gmail.com> wrote: > > > Hi Eric, > > > > Thanks for starting this discussion. > > > > Kerberos was developed decade before web development becomes popular. > > > There are some Kerberos limitations which does not work well in Hadoop. > > > > > Sure, Kerberos was developed long before the web but it was selected as > de > > facto authentication mechanism in Hadoop after the internet boom. And it > > was selected for a reason - it is one of the strongest symmetric key > based > > authentication mechanism out there which doesn't transmit the password in > > the plain text. Kerberos has been around since long and has stood the > test > > of time. > > > > Microsoft Active Directory, which is extensively used in many > > > organizations, is based on Kerberos. > > > > > +1 to this. > > And the fact that Microsoft has put Active Directory in Azure too, tells > me > > that AD (and thereof Kerberos) is not going away any time soon. > > > > Overall, I agree with Rajive and Craig on this topic. Paving way for the > > OpenID Connect in Hadoop is a good idea but seeing it as a replacement to > > Kerberos, needs to be carefully thought out. All the problems, that are > > described in the original mail, are not really Kerberos issues. > > Yes, we do understand that making Kerberos work *in a right way* is > always > > an uphill task (I'm a long time Kerberos+Hadoop Support Engineer) but > that > > shouldn't be the reason to replace it. > > > > Hint: CVE-2020-9492 > > > > > Btw, the CVE-2020-9492 is not accessible right now in the CVE database, > > maybe it is not yet public. > > > > On Thu, May 21, 2020 at 9:22 AM Steve Loughran > <ste...@cloudera.com.invalid > > > > > wrote: > > > > > On Wed, 6 May 2020 at 23:32, Eric Yang <eric...@gmail.com> wrote: > > > > > > > Hi all, > > > > > > > > > > > > 4. Passing different form of tokens does not work well with cloud > > > provider > > > > security mechanism. For example, passing AWS sts token for S3 > bucket. > > > > There is no renewal mechanism, nor good way to identify when the > token > > > > would expire. > > > > > > > > > > > well, HADOOP-14556 does it fairly well, supporting session and role > > tokens. > > > We even know when they expire because we ask for a duration when we > > request > > > the session/role creds. > > > See org.apache.hadoop.fs.s3a.auth.delegation.AbstractS3ATokenIdentifier > > for > > > the core of what we marshall, including encryption secrets. > > > > > > The main issue there is that Yarn can't refresh those tokens because a > > new > > > triple of session credentials are required; currently token renewal > > assumes > > > the token is unchanged and a request is made to the service to update > > their > > > table of issued tokens. But even if the RM could get back a new token > > from > > > a refresh call, we are left with the problem of "how to get an updated > > set > > > of creds to each process" > > > > > >
