I-am-aman opened a new pull request, #17468:
URL: https://github.com/apache/iceberg/pull/17468
## What
Adds a pluggable storage credential-provider SPI to the GCS FileIO in the
`iceberg-gcp` module:
- `GcsTokenCredentialProvider` - interface returning a
`com.google.auth.oauth2.GoogleCredentials`,
with an `initialize(Map<String,String>)` hook for provider-specific config.
- `GcsTokenCredentialProviders` - factory that loads a custom implementation
via `DynConstructors`
from the new `gcs.token-credential-provider` property, with a
`DefaultGcsTokenCredentialProvider`
backed by Application Default Credentials.
- New `GCPProperties` constants (`gcs.token-credential-provider`,
`gcs.token-credential-provider.` prefix), an accessor, and a precondition
preventing it from being
combined with `gcs.no-auth`.
- A new branch in `PrefixedStorage#credentials(...)` that uses the
configured provider when present.
- Unit tests.
This is the GCS analogue of the existing Azure `AdlsTokenCredentialProvider`
(#14136) and AWS's
`client.credentials-provider`.
## Why
The GCS FileIO today supports a static `gcs.oauth2.token`, `gcs.no-auth`,
native impersonation, and
the vended refresh endpoint - but there is **no pluggable way to supply a
caller-provided,
self-refreshing source credential** for non-vended setups. None of the
existing paths covers this:
- `gcs.oauth2.token` is static - it never refreshes, so long-running jobs
fail at token expiry.
- The vended refresh endpoint only refreshes REST-catalog-vended
credentials, not a
caller-supplied source.
- Native impersonation (`gcs.impersonate.service-account`) structurally
starts from
`GoogleCredentials.getApplicationDefault()` as its source credential -
there is no property to
inject an arbitrary caller-supplied source, so it does not cover the
bring-your-own-credentials
case.
This SPI lets integrators plug in a credential source that refreshes,
without Iceberg taking on any
specific credential implementation. The default remains Application Default
Credentials.
## Compatibility
- Purely additive. Default behaviour is unchanged: with no
`gcs.token-credential-provider` set,
credential resolution is identical to today (oauth2Token / no-auth /
impersonation / ADC).
- No changes outside `iceberg-gcp`.
## Scope: this is storage-plane auth
GCP has two independent auth planes, and this PR touches only the first:
- **Storage plane** - how `GCSFileIO` / `PrefixedStorage` authenticate to
GCS to read/write data
files. This is where the new `gcs.token-credential-provider` SPI lives
(properties are `gcs.*`).
- **Catalog plane** - how a REST catalog session authenticates, handled by
`GoogleAuthManager`
(properties are `gcp.auth.*`). This PR does **not** change it.
## Testing
- `TestGcsTokenCredentialProviders` - default factory, empty/blank provider,
custom provider,
missing no-arg ctor, non-implementing class, prefixed-property extraction.
- `TestGCPProperties` - provider property is read; mutual-exclusion with
`gcs.no-auth` is enforced;
provider + `gcs.oauth2.token` is allowed.
- `./gradlew :iceberg-gcp:spotlessCheck :iceberg-gcp:test` passes locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
