andreasbros opened a new pull request, #333:
URL: https://github.com/apache/arrow-rs-object-store/pull/333
# Which issue does this PR close?
Closes #282
# Rationale for this change
This PR extends the AmazonS3Builder so that it recognises and supports EKS
Pod Identity credentials using the two environment variables:
- `AWS_CONTAINER_CREDENTIALS_FULL_URI`
- `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE`
Previously, the builder only considered ECS task credentials
(`AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`), instance metadata, static
credentials, or web identity tokens. Adding EKS Pod Identity support aligns it
with modern Kubernetes IRSA setups, allowing pods to retrieve AWS credentials
from an EKS endpoint without needing to mount AWS credentials directly.
# What changes are included in this PR?
1. **New Config Keys**
Adds `AmazonS3ConfigKey::ContainerCredentialsFullUri` and
`AmazonS3ConfigKey::ContainerAuthorizationTokenFile` to the config-based
approach, for parsing EKS Pod Identity settings. The builder picks these keys
from environment variables `AWS_CONTAINER_CREDENTIALS_FULL_URI` and
`AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE`.
2. **EKSPodCredentialProvider**
Introduces an `EKSPodCredentialProvider`, which is constructed when both
config keys are set. It uses a bearer token (read from the specified file) to
fetch short-lived credentials from the EKS credential endpoint.
3. **Builder Logic**
Adjusts `AmazonS3Builder::build` to give priority to EKS credentials if both
the full URI and token file are specified. It checks environment variables in
from_env or direct calls to `with_config`, falling back to ECS or instance
metadata if EKS variables are absent.
4. **Tests**
Adds tests to confirm EKS credentials build and provider.
# Are there any user-facing changes?
- **New EKS Credential Support**
Users in EKS can now set `AWS_CONTAINER_CREDENTIALS_FULL_URI` and
`AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE`, and the builder automatically fetches
credentials.
- **Configuration Keys**
Two new config keys are recognised by the builder:
- `AmazonS3ConfigKey::ContainerCredentialsFullUri`
- `AmazonS3ConfigKey::ContainerAuthorizationTokenFile`
These changes are backwards-compatible: existing ECS, static credentials, or
IMDS-based setups continue to work unchanged. No additional user steps are
required unless they specifically opt to use EKS Pod Identity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
