Hi Cheng Pan >> then we decided to keep them independent
> I see that, but if we decide to accept and implement both SPIPs, then we are going to provide two approaches for users that enable cloud credentials refresh, this is functionality redundant, and as you know, this part usually involve private data and 3rd party services dependencies, when user report issues, they are likely limited to share the related part of logs and environment information to provide a minimal reproducible cases, this makes diagnosis extremely difficult. Offering two distinct cloud credential refresh mechanisms undoubtedly increases system complexity. > I would lean towards to the OIDC Credential Propagation approach unless it does not cover the functionality (user perspective) provided by this SPIP. There are a few cases covered by this not covered by the OIDC credential propagation approach. For instance, - 1. Kafka delegation tokens over SCRAM — the existing KafkaDelegationTokenProvider is currently blocked by the Kerberos gates. It targets brokers, not URIs, and doesn't need a UserContext. However, the OIDC approach depends on a UserContext (an OIDC JWT with principal/issuer/rawToken). It cannot be reimplemented as a CredentialProvider.resolve(UserContext, URI). 2. Existing S3A/ABFS delegation token bindings (Steve's point in this thread) — these are already implemented as HadoopDelegationTokenProvider and work today in Kerberos environments. They just need the activation gates removed to work without Kerberos. The OIDC SPIP's parallel manager does not unblock them. 3. There can also be proprietary IdP systems which have non JWT tokens (which is what prompted this SPIP in the first place) >> The DirectProviderPath proposed in this SPIP does not go through Subject.doAs() or UserGroupInformation.doAs() and will be unaffected. The existing Kerberos path will have to be updated. > Sorry, I overlook this reply. I think this is also affected. The JDK change breaks the Subject propagation between threads, that means you can not get the same Subject (UGI) instance from the task thread as the updateTokensTask thread, so you can not access any kind of the credential you offered from the task thread. On executor task threads, there is no active doAs()/callAs() scope. getCurrentUser() sees null from Subject.current() (or Subject.getSubject() on older JDKs) and falls back to getLoginUser() — which is a static field and is not based on Subject propagation. The credentials added via addCredentials() on the RPC handler thread are added to this same static login user instance so task threads reading from getLoginUser() should see them However, broadly speaking, we do need HADOOP-19906 I've updated the SPIP document to include these points Parth
