[
https://issues.apache.org/jira/browse/HDDS-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Theodosius reassigned HDDS-15299:
---------------------------------------
Assignee: Simon Theodosius
> [S3] Add managed local S3 access keys with expiration for Ozone S3 Gateway
> --------------------------------------------------------------------------
>
> Key: HDDS-15299
> URL: https://issues.apache.org/jira/browse/HDDS-15299
> Project: Apache Ozone
> Issue Type: New Feature
> Components: documentation, Ozone Manager, S3, Security
> Environment: Feature proposal for Apache Ozone S3 Gateway and Ozone
> Manager.
> Target environment:
> - Apache Ozone S3 Gateway
> - Ozone Manager
> - legacy S3/MinIO-compatible clients using AWS SigV4 accessKey/secretKey
> - deployments that need S3-compatible access without requiring every client
> to use Kerberos, OIDC WebIdentity, or AWS session tokens
> This is not environment-specific, but it is especially useful for deployments
> where only S3 Gateway is exposed to clients and backend Ozone endpoints are
> restricted.
> Reporter: Simon Theodosius
> Assignee: Simon Theodosius
> Priority: Minor
> Labels: access-keys, s3, security, sigv4
>
> Apache Ozone supports S3 access through existing S3 secret flows and STS
> temporary credentials. However, there is no dedicated managed local S3
> access-key lifecycle for legacy S3 clients that only support static
> accessKey/secretKey credentials and cannot use OIDC WebIdentity or AWS
> session tokens.
> This issue proposes managed local S3 access keys for Ozone S3 Gateway.
> The feature targets legacy S3/MinIO-compatible clients that support normal
> AWS SigV4 with accessKey/secretKey, but do not support:
> - OIDC WebIdentity;
> - AWS_SESSION_TOKEN;
> - AWS STS temporary credentials;
> - credential_process;
> - web_identity_token_file.
> This feature is separate from HDDS-15273. HDDS-15273 handles modern clients
> through:
> OIDC JWT -> STS temporary credentials -> SigV4 + x-amz-security-token
> This issue handles legacy clients through:
> managed accessKey/secretKey -> SigV4 without session token
> High-level goals:
> - Allow Ozone admins to create managed S3 access keys with expiration.
> - Support legacy S3/MinIO clients using normal AWS SigV4 without
> AWS_SESSION_TOKEN.
> - Keep OM as the source of truth for credential metadata and authorization
> context.
> - Keep S3G stateless.
> - Preserve existing STS, WebIdentity, and legacy getsecret behavior.
> - Add optional local JSON policy evaluation for managed access keys.
> - Fail closed on unsafe or ambiguous configurations.
> Non-goals:
> - This does not replace HDDS-15273 WebIdentity STS.
> - This does not replace Kerberos daemon authentication.
> - This does not implement OFS OIDC login.
> - This does not implement CLI device-code login.
> - This does not implement official Ozone Multi-Tenancy.
> - This does not replace Ranger-based multi-tenancy.
> - This does not store credentials in S3G.
> - This does not make S3G the final authorization decision point.
> Proposed approach:
> - Add a new OM metadata table for managed S3 access keys.
> - Store access-key metadata separately from the existing legacy
> S3SecretManager/getsecret table.
> - Support expiration, disabled state, effective user, group snapshot,
> description, encrypted secret material, and optional local JSON policy.
> - Keep S3G stateless; credential validation remains OM-authoritative.
> - Preserve STS precedence when x-amz-security-token is present.
> - Preserve legacy S3SecretManager/getsecret fallback where applicable.
> - Prevent namespace collisions between managed access keys and legacy S3
> secrets.
> - Keep the feature disabled by default.
> Security notes:
> - Plaintext secrets must not be stored in Ratis, RocksDB, checkpoints, audit
> logs, server logs, debug/TRACE output, toString(), or error messages.
> - Managed access-key secret material should be encrypted before the
> Ratis-applied request is created.
> - Create/rotate should not place plaintext secrets into the normal Ratis
> OMResponse path.
> - The design uses a two-step retrieval-handle model for one-time plaintext
> retrieval.
> - Local JSON policy protects only S3 requests through S3G. It does not
> protect OFS, Ozone CLI, OM RPC, SCM, DNs, or direct backend access.
> - Operators relying on local JSON policy must expose only S3G to clients and
> restrict backend Ozone endpoints.
> Design document:
> hadoop-hdds/docs/content/design/managed-local-s3-access-keys.md
> The design document covers the detailed data model, configuration, startup
> gates, KMS/envelope secret encryption, retrieval-handle plaintext transport,
> secure vs non-secure credential resolution, local JSON policy semantics,
> namespace collision handling, and the implementation/test plan.
> Acceptance criteria:
> - Feature is disabled by default.
> - Existing STS and WebIdentity behavior is unchanged.
> - Existing S3SecretManager/getsecret behavior is unchanged except for
> collision prevention during new secret creation.
> - Managed access-key configuration defaults match the design.
> - Unsafe non-secure startup combinations fail closed.
> - Managed access-key metadata is stored in a separate OM table.
> - Plaintext secrets are not stored in Ratis, RocksDB, checkpoints, audit
> logs, server logs, debug/TRACE output, toString(), or errors.
> - Create/rotate use a retrieval-handle model instead of returning plaintext
> through the normal Ratis OMResponse path.
> - Missing, expired, already-used, wrong-caller, or failover-lost retrieval
> handles fail closed.
> - KMS provider/key/version failures fail closed.
> - Managed/legacy accessKeyId collisions are rejected.
> - Disabled or expired managed keys fail closed and do not fall through to
> legacy S3SecretManager.
> - STS session-token credentials take precedence when x-amz-security-token is
> present.
> - Local JSON policies reject malformed JSON, unknown actions, invalid
> resources, unsupported IAM constructs, and oversized policies.
> - Deny overrides Allow.
> - S3G remains stateless and does not make final authorization decisions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
