Hi Austen, Just to confirm my understanding: “local signing” happens on the Polaris server, which generates scoped, temporary credentials without calling Cloudflare’s credential API. The client then uses those credentials to sign its own S3 requests, and R2 enforces the scope and expiry. This sounds like STS-style credential vending, where Polaris performs the issuance without talking to an object storage server. If this is the case, I'd lean toward considering R2 an S3-compatible storage, which may not need a separate storage config type. However, we will still need to check all configuration details to ensure no incompatible changes are introduced.
Does this work out of the box with existing Iceberg REST Catalog clients that support credential vending and S3FileIO, or does it require a special client impl. or changes? You mentioned validation with your internal Python client. Have you also tested unmodified OSS clients(e.g., Spark with Iceberg plugins, Trino)? > Would you prefer the shared FileIO validation change to land separately, and how should we sequence the management API and R2 integration changes? Yes, landing the changes separately would be preferable. Small and well-scoped PRs usually merge faster. Yufei On Wed, Sep 9, 2026 at 11:18 AM Austen Tomek < [email protected]> wrote: > Hi all, > > I’m Austen Tomek from Chicago Trading Company. We use Polaris for our > Iceberg catalogs and we're evaluating Cloudflare R2 as an object store (the > no egress fees make it a very compelling product). I’d like to discuss > contributing R2 support and credential vending to Polaris with the goal of > getting feedback before opening a ready-for-review upstream PR. This is my > first contribution, so trying to do this right. > > R2 exposes S3-compatible APIs but it does not provide AWS STS. Cloudflare > instead supports these short-lived, scoped credentials generated by locally > signed JWT with the server holding the parent API token [1]. This allows us > to retain Polaris’s authorization and credential-vending model while > storing our table data in R2. > > I have a prototype with the following design: > > > * > An opt-in R2 storage type, with catalog configuration identifying the > Cloudflare account and optional jurisdiction. Parent credentials remain in > server configuration and can be selected through storageName. > * > Local credential generation using Cloudflare’s documented format. > Credentials expire, are restricted to one bucket, and carry prefix and > access scopes derived from Polaris’s location grants. The integration uses > Polaris’s existing credential cache. > * > Credentials returned through the existing Iceberg REST vending flow as > s3.* properties, including the endpoint, session token, and expiry. > Compatible clients use their existing S3 FileIO. > * > Conservative handling of scopes the implementation cannot represent: > cross-bucket and mixed read/write grants are rejected rather than > broadened. R2 is excluded from the default supported-storage-types list. > > The main design questions I feel this runs into: > > * > Should this be a distinct storage type or a credential-vending mechanism > selected through the existing S3 configuration? > * > I chose a separate type because the identity, credential generation, and > scoping model differ from AWS STS, as this seemed more appropriate > > There is a draft preview PR against my fork [2], including implementation, > management API changes, CLI support, tests, and documentation. It also > discloses the AI assistance used during implementation. I leaned heavily on > Fable 5.1 for this implementation so I want to be upfront about it. > > We are running the implementation in a non-production environment right > now as a POC. Validation includes live R2 reads/writes, multipart > operations, negative scope-boundary tests, our internal Python client > regression matrix, and credential rotation/recovery exercises. The upstream > CI workflow has passed on the fork. > > The change also encounters the existing assumption that a FileIO > implementation identifies exactly one storage type: both S3 and R2 use > S3FileIO. This overlaps with #4486 [3]. > > I’d particularly appreciate feedback on: > > > 1. > Should R2 have its own storage configuration/type, or should non-STS > vending be modeled within the existing S3 integration? > 2. > Does the local-signing approach fit Polaris’s credential-vending model? > Any additional constraints or validations necessary? > 3. > Would you prefer the shared FileIO validation change to land separately, > and how should we sequence the management API and R2 integration changes? > > Happy to adapt the design and split the contribution into more focused > PRs. Wanted to start the discussion > > Thanks, > Austen > > [1] https://developers.cloudflare.com/r2/api/s3/temporary-credentials/ > [2] https://github.com/deepdishgary/polaris/pull/1 > [3] https://github.com/apache/polaris/issues/4486 > > Get Outlook for Mac<https://aka.ms/GetOutlookForMac> > > > > > > > This electronic mail message and any attached files contain information > intended for the exclusive use of the individual or entity to whom it is > addressed and may contain information that is proprietary, confidential > and/or exempt from disclosure under applicable law. If you are not the > intended recipient, you are hereby notified that any viewing, copying, > disclosure or distribution of this information may be subject to legal > restriction or sanction. Please notify the sender, by electronic mail or > telephone, of any unintended recipients and delete the original message > without making any copies. >
