jackye1995 opened a new pull request, #6525:
URL: https://github.com/apache/opendal/pull/6525
## Summary
This PR adds support for AWS container credentials used in ECS tasks and EKS
pods, enabling OpenDAL to seamlessly work in containerized AWS environments.
## Background
Container credentials are a key feature for AWS workloads running in ECS and
EKS:
- **ECS Task IAM Roles**: Allow containers to assume IAM roles without
embedding credentials
- **EKS Pod Identity**: Enable pods to authenticate using service account
tokens
This feature addresses issue #6456 by implementing the container credentials
portion of the AWS configuration compatibility improvements.
## Changes
### New Configuration Fields
| Field | Aliases | Purpose |
|-------|---------|---------|
| `container_credentials_relative_uri` |
`aws_container_credentials_relative_uri` | ECS task IAM role endpoint |
| `container_credentials_full_uri` | `aws_container_credentials_full_uri` |
EKS pod identity endpoint |
| `container_authorization_token_file` |
`aws_container_authorization_token_file` | EKS service account token file |
### Builder Methods
Added corresponding builder methods for programmatic configuration:
- `container_credentials_relative_uri(uri: &str)`
- `container_credentials_full_uri(uri: &str)`
- `container_authorization_token_file(file: &str)`
### Example Usage
```rust
// ECS Task IAM Role
let s3 = S3::default()
.bucket("my-bucket")
.container_credentials_relative_uri("/v2/credentials/12345678-1234-1234-1234-123456789012")
.build()?;
// EKS Pod Identity
let s3 = S3::default()
.bucket("my-bucket")
.container_credentials_full_uri("https://localhost:1234/token";)
.container_authorization_token_file("/var/run/secrets/eks.amazonaws.com/serviceaccount/token")
.build()?;
```
### JSON Configuration
```json
{
"bucket": "my-bucket",
"aws_container_credentials_relative_uri":
"/v2/credentials/12345678-1234-1234-1234-123456789012"
}
```
## Implementation Details
### Current State
- ✅ Configuration fields with serde aliases
- ✅ Builder methods for programmatic setup
- ✅ Comprehensive test coverage
- ✅ Debug logging when container credentials are configured
- 🔄 Placeholder implementation (credential loading will be added in
follow-up PR)
### Design Decisions
1. **Serde Aliases**: Following the same pattern as the previous PR (#6524)
to support both OpenDAL and AWS SDK naming conventions
2. **Credential Provider Pattern**: Implementation follows Apache Arrow's
object_store approach:
- ECS: HTTP GET to `http://169.254.170.2{relative_uri}`
- EKS: HTTP GET to `{full_uri}` with `Authorization: {token}` header
- Both return JSON with `AccessKeyId`, `SecretAccessKey`, `Token`,
`Expiration`
3. **Placeholder Implementation**: The current implementation detects and
logs container credential configuration but uses a TODO placeholder for the
actual credential loading. This allows:
- Configuration validation and testing
- Integration with existing customized credential loaders
- Future implementation of automatic credential loading
## Testing
- ✅ Unit tests for configuration deserialization with aliases
- ✅ All existing S3 tests pass (16/16)
- ✅ Clippy and formatting checks pass
- ✅ No breaking changes to existing functionality
## Future Work
The next PR will implement the actual credential loading logic by:
1. Creating proper `AwsCredentialLoad` implementations
2. Adding HTTP-based credential fetching
3. Implementing credential caching and refresh logic
4. Adding integration tests with mock endpoints
## Compatibility
This change is fully backward compatible. Existing configurations continue
to work unchanged, with new fields providing additional flexibility for
containerized environments.
Related to #6456
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@opendal.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
