Hi all,
Thank you again for the discussion during the 7/1 catalog community sync.
I wanted to summarize two alternative directions that came up to pre-signed
URLs:
1. scoped credential vending
2. remote signing with caching and bulk-signing
I think scoped credential vending is worth considering in cases where the
authorization boundary maps cleanly to a small number of stable storage
prefixes.
Scoped credential vending seems less practical when access needs to be
granted for many specific files rather than a small number of prefixes.
In AWS, session policies are passed as parameters during AssumeRole-style
issuance when the temporary session is created. [1]
This means that scoping down an already-issued credential cannot be done
locally by simply altering the existing token.
Instead, it requires minting another STS session whose policy encodes the
new scope.
This turns authorization into an STS issuance problem with limitations on
STS request frequency [2] and policy size. [3]
By contrast, pre-signed URLs and remote signing can authorize exact object
requests using already-held credentials, without creating a new STS session
for each file set.
Based on that discussion, I am exploring a POC around bulk remote signing +
caching to evaluate whether it could be a practical alternative to
pre-signed URLs for these finer-grained access patterns.
Best,
William
[1] https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html
[2]
https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-quotas.html#reference_iam-quotas-sts-requests
[3] https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html
On Tue, Jun 30, 2026 at 7:43 PM William Hyun <[email protected]> wrote:
> Hi everyone,
>
> I'd like to open a discussion on an extension to the REST Catalog spec to
> enable file-level access delegation during scan planning.
> Today, delegated access is table-scoped, which can force over-provisioning
> or table fragmentation when consumers should only see a subset of
> partitions or records.
> This proposal adds a path using pre-signed URLs to make partition-scoped
> sharing practical in the REST Catalog model.
>
> Here is my proposal: https://s.apache.org/n16st
>
> I look forward to hearing your thoughts.
>
> Relevant discussion:
> - https://lists.apache.org/thread/ko9kp0gvzhx85n7cvoxqnpw4vwnhmdg6
>
> Best,
> William
>
>
