rohankmr414 opened a new pull request, #7166:
URL: https://github.com/apache/opendal/pull/7166
Add support for AWS container credentials to enable authentication in
containerized environments:
- ECS Task IAM roles
- EKS Pod Identity
- AWS Fargate
- Custom container credential endpoints
Changes:
- Add 5 new configuration fields to S3Config with AWS-prefixed aliases
- Add 5 builder methods to S3Builder for container credentials
- Integrate ECSCredentialProvider into credential chain
- Add comprehensive tests for configuration parsing
- Add detailed documentation for ECS and EKS Pod Identity usage
The implementation uses reqsign-aws-v4's ECSCredentialProvider which
supports both ECS and EKS Pod Identity through standard AWS container
credential environment variables.
Related to PR #6525
# Rationale for this change
This PR adds support for AWS container credentials to enable S3
authentication in containerized environments, specifically for ECS Task IAM
roles and EKS Pod Identity. This was originally proposed in PR #6525 but was
blocked waiting for reqsign v0.17.0+ which included the
`ECSCredentialProvider`. Since the codebase now uses reqsign-aws-v4 v2.0.2
(which includes this provider), we can now implement this feature.
Container credentials are the recommended authentication method for
containerized workloads on AWS, providing better security through:
- Automatic credential rotation
- No need to embed static credentials in containers
- Fine-grained IAM permissions per task/pod
- Better auditability
# What changes are included in this PR?
1. **Configuration fields**: Added 5 new fields to `S3Config` for container
credentials:
- `container_credentials_relative_uri` - For ECS Task IAM roles
- `container_credentials_endpoint` - For EKS Pod Identity and Fargate
- `container_authorization_token` - Authorization token for credential
requests
- `container_authorization_token_file` - Path to authorization token file
(required for EKS Pod Identity)
- `container_metadata_uri_override` - Override for metadata endpoint (for
testing)
2. **Builder methods**: Added 5 corresponding builder methods to `S3Builder`
for programmatic configuration
3. **Credential provider integration**: Integrated `ECSCredentialProvider`
from reqsign-aws-v4 into the credential chain, positioned after static
credentials but before assume role provider
4. **Tests**: Added comprehensive test cases to verify configuration parsing
with both standard and AWS-prefixed field name aliases
5. **Documentation**: Added detailed doc strings explaining usage for both
ECS and EKS Pod Identity scenarios
# Are there any user-facing changes?
**Yes** - This PR adds new public API methods and configuration options:
**New builder methods:**
- `S3Builder::container_credentials_relative_uri()`
- `S3Builder::container_credentials_endpoint()`
- `S3Builder::container_authorization_token()`
- `S3Builder::container_authorization_token_file()`
- `S3Builder::container_metadata_uri_override()`
**New configuration fields** (can be set via config files, environment
variables, or builder methods):
- `container_credentials_relative_uri` (alias:
`aws_container_credentials_relative_uri`)
- `container_credentials_endpoint` (aliases:
`container_credentials_full_uri`, `aws_container_credentials_full_uri`)
- `container_authorization_token` (alias:
`aws_container_authorization_token`)
- `container_authorization_token_file` (alias:
`aws_container_authorization_token_file`)
- `container_metadata_uri_override` (aliases:
`aws_container_metadata_uri_override`, `aws_metadata_endpoint`,
`metadata_endpoint`)
**No breaking changes** - All new fields are optional and the credential
provider chain maintains backward compatibility.
# AI Usage Statement
This PR was developed with assistance from Augment Agent (Claude Sonnet 4.5
by Anthropic) for code implementation, testing, and documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
