ChenSammi commented on code in PR #10338: URL: https://github.com/apache/ozone/pull/10338#discussion_r4279418996
########## hadoop-hdds/docs/content/design/oidc-assume-role-with-web-identity.md: ########## @@ -0,0 +1,528 @@ +--- +title: OIDC AssumeRoleWithWebIdentity for Ozone STS +summary: Web identity support for Ozone STS using OIDC and Ranger authorization +date: 2026-05-13 +status: proposed +--- +<!-- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +# OIDC AssumeRoleWithWebIdentity for Ozone STS + +## Status + +Proposed staged implementation. + +This document narrows the previous broad OIDC direction to an upstream-friendly +MVP: extend the Ozone STS temporary S3 credential model with an AWS-compatible +`AssumeRoleWithWebIdentity` action. + +## Problem + +Secure Ozone S3 deployments currently depend on Kerberos-backed identities for +S3 credential issuance. Kubernetes workloads commonly already have OIDC tokens +from Keycloak or another IdP, but do not have an easy Kerberos bootstrap path. + +The target is not a Kerberos-free Ozone cluster. The target is a narrow STS +exchange: + +1. Keycloak authenticates the caller and issues a signed OIDC JWT. +2. Ozone STS validates the JWT locally using JWKS. +3. Ranger or the configured Ozone authorizer decides whether the identity may + assume the requested role. +4. Ozone STS issues temporary S3 credentials. +5. S3 Gateway and OM validate those temporary credentials and authorize object + operations using the assumed identity and session context. + +Ranger remains the authorization source of truth. Keycloak roles and groups are +identity attributes only. + +## Current STS And S3 Security Path + +This design is based on the `origin/HDDS-13323-sts` branch at commit +`37a224b217`, which contains the STS runtime that was missing from earlier +base branches. + +The current STS runtime contains: + +- `/sts` HTTP endpoint in `org.apache.hadoop.ozone.s3sts.S3STSEndpoint`; +- endpoint authentication setup in `S3STSEndpointBase`; +- AWS STS `AssumeRole` XML response model in `S3AssumeRoleResponseXml`; +- S3G to OM client path through `ObjectStore`, `ClientProtocol`, `RpcClient`, + `OzoneManagerProtocol`, and + `OzoneManagerProtocolClientSideTranslatorPB.assumeRole()`; +- OM request handling in + `org.apache.hadoop.ozone.om.request.s3.security.S3AssumeRoleRequest`; +- OM response handling in `S3AssumeRoleResponse`; +- session token identifier and secret manager in `STSTokenIdentifier`, + `STSTokenSecretManager`, and `STSSecurityUtil`; +- revoked STS token metadata and cleanup through `S3RevokeSTSTokenRequest`, + `S3DeleteRevokedSTSTokensRequest`, and + `RevokedSTSTokenCleanupService`; +- authorization extension points in `AssumeRoleRequest`, + `IAccessAuthorizer.generateAssumeRoleSessionPolicy()`, and + `RequestContext.sessionPolicy`. + +The current S3 request authentication path is: + +- S3G parses AWS SigV4 in `AuthorizationFilter`, + `SignatureProcessor`, `AuthorizationV4HeaderParser`, + `AuthorizationV4QueryParser`, and `StringToSignProducer`. +- `EndpointBase` creates `S3Auth` from the parsed access key, signature, and + string-to-sign, then stores it in the `ClientProtocol` thread-local. +- `OzoneManagerProtocolClientSideTranslatorPB` copies `S3Auth` into + `OMRequest.s3Authentication`. +- `AWSSignatureProcessor` extracts `x-amz-security-token` for temporary + credentials into `SignatureInfo.sessionToken`. +- `EndpointBase` and `S3STSEndpointBase` propagate the session token into + `S3Auth`. +- `OzoneManagerProtocolClientSideTranslatorPB` copies `S3Auth` into + `OMRequest.s3Authentication`. +- `S3SecurityUtil.validateS3Credential()` validates either permanent S3 + credentials or STS temporary credentials. +- For STS credentials, `STSSecurityUtil` decodes, validates, and decrypts the + session token, then OM validates the SigV4 signature with the temporary + secret access key. +- `OmMetadataReader` attaches STS session policy from the OM thread-local + `STSTokenIdentifier` to `RequestContext` for subsequent authorization. + +The current permanent S3 credential storage path is: + +- `S3SecretManager` and `S3SecretManagerImpl`. +- `S3SecretValue`. +- OM metadata S3 secret table via `OmMetadataManagerImpl`. +- `ozone s3 getsecret`, `setsecret`, and `revokesecret` client paths. + +## Dependency On Existing Ozone STS Runtime + +`AssumeRoleWithWebIdentity` must be an incremental extension of the existing +`AssumeRole` runtime in `origin/HDDS-13323-sts`. It must not introduce a second +STS endpoint, a separate S3 authentication system, or local S3G-only temporary +credential state. + +Existing runtime: + +```text +AssumeRole + -> S3 SigV4-authenticated /sts request + -> OM S3AssumeRoleRequest + -> temporary access key / secret / session token + -> STSSecurityUtil validation on later S3 requests + -> RequestContext.sessionPolicy + -> Ranger or configured authorizer +``` + +New runtime: + +```text +AssumeRoleWithWebIdentity + -> unauthenticated /sts bootstrap request only for this action + -> OM validates Keycloak/OIDC JWT + -> OM authorizes role assumption through Ranger or configured authorizer + -> existing temporary credential issuer / session token path + -> existing STSSecurityUtil validation on later S3 requests + -> RequestContext.sessionPolicy and assumed identity + -> Ranger or configured authorizer +``` + +The OM runtime slice adds the WebIdentity request/protobuf path while preserving +the existing `AssumeRole` flow. S3G parses and routes +`Action=AssumeRoleWithWebIdentity`, but OM remains the authoritative validator +and issuer. + +The raw `WebIdentityToken` is accepted only in the external OM RPC request. The +OM leader validates it in `S3AssumeRoleWithWebIdentityRequest.preExecute()`, +maps claims into a sanitized identity/session request, authorizes role +assumption, generates temporary credential material using the existing STS +helpers, and returns an `UpdateAssumeRoleWithWebIdentityRequest` for Ratis +replication. The replicated request must not contain the raw JWT. + +`validateAndUpdateCache()` consumes only the sanitized update request. It must +not call Keycloak, refresh JWKS, revalidate JWTs, or otherwise depend on current +external IdP state during Ratis apply or replay. Credential expiration is +computed by the leader before replication and stored as +`credentialExpirationEpochSeconds` so replay does not depend on the apply-time +clock. + +Temporary credentials must not be stored only in S3G memory. S3G can have +multiple replicas and can restart. The issuing and validation authority must be +OM-backed, persisted in Ozone metadata, or based on self-contained signed tokens +whose signing keys are rotation-safe and available to all validating components. + +## Endpoint Placement + +The existing STS runtime places `/sts` on the S3 Gateway HTTP/HTTPS port. +WebIdentity follows that placement: S3G exposes the AWS-compatible STS API +surface, while OM remains authoritative for JWT validation, identity mapping, +role-assumption authorization, credential issuance, revocation, and later +temporary credential validation. + +Because existing `/sts` `AssumeRole` is protected by the normal S3 SigV4 +`AuthorizationFilter`, `AssumeRoleWithWebIdentity` needs a narrow bootstrap +exception: + +- only for the STS application path; +- only for `Action=AssumeRoleWithWebIdentity`; +- only when `ozone.sts.web.identity.enabled=true`; Review Comment: I think they can under the same ozone.s3g.sts.http.enabled control, instead of introducing a new ozone.s3g.sts.web.identity.enabled property. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
