roryqi opened a new pull request, #11229:
URL: https://github.com/apache/gravitino/pull/11229
### What changes were proposed in this pull request?
Make the vended S3 `ListBucket` `s3:prefix` condition include the bare
location
prefix per catalog type, controlled by an internal (non-user-configurable)
flag
`s3-credential-list-location-prefix` (default `false`):
- Fileset catalogs force it `true`: `s3:prefix = [loc, loc/, loc/*]`.
- Iceberg catalogs force it `false` (security-first): `s3:prefix = [loc/,
loc/*]`.
Touched: `CredentialConstants`, `S3CredentialConfig`, `S3TokenGenerator`,
`AwsIrsaCredentialGenerator`, `FilesetCatalogImpl`, `IcebergCatalog` (+
tests).
### Why are the changes needed?
For fileset access (GVFS/S3A), `getFileStatus` issues a HEAD on the bare
directory-root key. S3 returns 404 (instead of 403) only if `ListBucket`
allows
the bare prefix, so without it `getFileStatus` fails with 403 AccessDenied.
The
previously-secure prefix (`[loc/, loc/*]`) dropped the bare prefix and broke
this.
Restoring it globally would let a credential enumerate sibling keys sharing
the
prefix, so the behavior is now scoped by catalog type: Fileset enables it
(needs
the Hadoop FS API), Iceberg keeps it disabled (uses S3FileIO, no such HEAD).
### Does this PR introduce _any_ user-facing change?
No. The new property key is internal and set by the catalog type; users
cannot
configure it.
### How was this patch tested?
- Unit: `TestS3PolicyPrefix`, `TestFilesetCatalogCredential` (2/0),
`TestIcebergCatalog` (11/0).
- Integration (real S3): `GravitinoVirtualFileSystemS3CredentialIT` 9/0 (was
7
failures), `FileSystemS3CredentialAuthorizationIT` 2/0,
`IcebergRESTS3TokenIT`
28/0, `IcebergRESTS3TokenAuthorizationIT` 3/0.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
