Hi All, Recap from today's community sync call:
* Actual set of values acceptable for "credentialVendingMechanism" is a finite list. Initially "STS" and "R2" will be supported by Polaris (exact names TBD during R2 implementation). * Servers validate values in "credentialVendingMechanism" in runtime after parsing the request. * The Open API doc defines "credentialVendingMechanism" as a string to allow for flexibility in auto-generated code. Clients / parsers do not have to re-build when a server adds a new option for "credentialVendingMechanism". Please correct me if I missed anything. Does this sound acceptable? Thanks, Dmitri. On Wed, Sep 30, 2026 at 1:30 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Yufei, > > Thanks for providing the details! Replying inline. > > > When a client creates a catalog with credentialVendingMechanism: "m1", > > what is the expected behavior? > > This depends on the meaning of "m1" and the specific server build. > > In Apache Polaris "m1" is not meaningful and is not supported. > The server will reject this value on the corresponding catalog > update request. This is handled in PolarisAdminService in the current PR > state. > > > 1. The server supports it: This is the ideal scenario (happy path). > > Indeed. > > > 2. The server does not support it: This creates inconsistency if a > client > > that works with Server A fails on Server B. > > In my view this is not an inconsistency as different servers do not have > to provide the same capabilities. > > Creating a catalog with a specific credentialVendingMechanism is an > administrative task. The admin user must be aware of what the server > supports. This is a deployment / configuration concern. Not a runtime > compatibility concern. > > > 3. The server supports "m1" differently: This is the worst-case > scenario, > > where the client's expectations conflict with the server's actual > behavior. > > This is a good point! > > My take is that if "m1" is defined in Apache Polaris, then all server > builds must implement it the same way. > > The definition will be expressed by means of S3CredentialVendingMechanism > classes included in standard Apache Polaris releases (and hopefully docs > too). > > Currently only "STS" and "DEFAULT" (falling back on STS) are defined now. > > If "m1" is not defined in Apache Polaris, custom builds are free to assign > any meaning to it and do not have to be mutually compatible. This is the > main point for extensibility. > > Note that custom builds are not "Apache Polaris" anymore and cannot > claim to be the "same" as Apache Polaris. This is a matter of ASF > trademarks. > Normally, custom builds say something like "Powered by Apache Polaris". > > Again, the admin user setting "m1" as a vending mechanism in a particular > server must be aware of what that server provides and understand > the meaning of "m1" in that context. The onus to describe the meaning of > "m1" to users in this case rests with the custom server developers. > > We can talk about preventing name overlaps, but I do not think establishing > any kind of name registry is worth the trouble. In this day and age, I > believe > all downstream project developers are aware of name overlap issues. > and will follow normal practices to avoid name collisions. > > I hope this clears your concerns. > > Cheers, > Dmitri. > > On Wed, Sep 30, 2026 at 1:02 PM Yufei Gu <[email protected]> wrote: > >> Hi Dmitri, >> >> When a client creates a catalog with credentialVendingMechanism: "m1", >> what >> is the expected behavior? >> >> Here is a suitaion if we allow a freeform string, the client might >> encounter three different scenarios depending on the server: >> 1. The server supports it: This is the ideal scenario (happy path). >> 2. The server does not support it: This creates inconsistency if a client >> that works with Server A fails on Server B. >> 3. The server supports "m1" differently: This is the worst-case scenario, >> where the client's expectations conflict with the server's actual >> behavior. >> >> I don't think this is the situation we want users to run into. The fix is >> quite simple, using a specified values instead of a freefrom string. >> >> Yufei >> >> >> On Tue, Sep 29, 2026 at 9:48 PM Jean-Baptiste Onofré <[email protected]> >> wrote: >> >> > Hi all, >> > >> > Since concerns were raised, it's important to take a moment to align >> > rather than rushing. That's the way we drive consensus :) >> > >> > On the technical question of string vs enum for >> > credentialVendingMechanism, here is my take: >> > >> > 1. This field is part of the Management API for catalog >> > administrators. Iceberg REST clients (Spark, Trino, ...) never see or >> > interact with credentialVendingMechanism. They only consume standard >> > vended credentials (s3.* session tokens/keys) during table >> > load/commit. There is no risk of breaking query engine clients across >> > deployments. >> > 2. One of our primary goals when discussing S3-compatible stores (R2, >> > MinIO, ...) was to keep credential vending pluggable via SPI rather >> > than hardcoding vendor-specific paths in core. If we use a closed enum >> > in the OpenAPI spec, it means: >> > * any new mechanism (including R2 in Phase 2 or MinIO) requires a spec >> > change. >> > * any downstream deployment or plugin providing a custom mechanism >> > would break OpenAPI-generated client SDKs due to strict enum >> > validation. >> > * runtime validation on the server (returning HTTP 400 with a clear >> > message) combined with the realm allowlist provides the right >> > server-side guardrail without restricting extensibility. >> > 3. Yufei's concerns about a clear contract and avoiding confusion is >> > valid. We can address contract clarity without closing the enum: >> > * we keep type: string, but document the well-known/standard values >> > (STS, DEFAULT) explicitly in the description and examples in the >> > OpenAPI spec (as PR #5513 does). >> > * we ensure the server validates cleanly with case-insensitivity (or >> > canonical normalization) so administrators get immediate, actionable >> > feedback on typos. >> > >> > PR #5513 does a solid job laying this foundation so Austen can move >> > forward with Phase 2. If we keep it as a string with well-documented >> > standard values and server-siide validation, I think it's the right >> > balance between spec clarify and pluggability. >> > >> > Thoughts? >> > >> > Regards >> > JB >> > >> > On Wed, Sep 30, 2026 at 12:50 AM Dmitri Bourlatchkov <[email protected]> >> > wrote: >> > > >> > > Hi Yufei, >> > > >> > > > > Conversely, an enum will require defining all possible vending >> > > mechanisms in >> > > > > the Polaris API spec. >> > > >> > > > Yes. Otherwise, a client may failed in certain deployments. >> > > >> > > Could you describe this failure mode in terms of API request/response >> > flow? >> > > >> > > Thanks, >> > > Dmitri. >> > > >> > > On Tue, Sep 29, 2026 at 2:06 PM Yufei Gu <[email protected]> >> wrote: >> > > >> > > > > Conversely, an enum will require defining all possible vending >> > > > mechanisms in >> > > > the Polaris API spec. >> > > > >> > > > Yes. Otherwise, a client may failed in certain deployments. >> > > > >> > > > > This will prevent downstream projects from using custom vending >> > > > mechanisms. >> > > > >> > > > No. if downstream needs custom vending mechanisms, they can always >> > build >> > > > their own Polaris. >> > > > >> > > > Yufei >> > > > >> > > > >> > > > On Tue, Sep 29, 2026 at 11:01 AM Dmitri Bourlatchkov < >> [email protected] >> > > >> > > > wrote: >> > > > >> > > > > Hi Yufei, >> > > > > >> > > > > > Using an enum will clarify the REST spec contract. >> > > > > >> > > > > I'm not sure I understand your point. Current PR clearly describes >> > the >> > > > > meaning of "credentialVendingMechanism" and lists options >> available >> > > > > out-of-the-box. >> > > > > >> > > > > What does an enum achieve, that (current) runtime validation does >> > not? >> > > > > >> > > > > Conversely, an enum will require defining all possible vending >> > mechanisms >> > > > > in the Polaris API spec. This will prevent downstream projects >> from >> > using >> > > > > custom vending mechanisms. >> > > > > >> > > > > I believe Polaris as a project currently promotes downstream >> > > > > customizability. >> > > > > >> > > > > Cheers, >> > > > > Dmitri. >> > > > > >> > > > > On Tue, Sep 29, 2026 at 1:21 PM Yufei Gu <[email protected]> >> > wrote: >> > > > > >> > > > > > I don't think we have consensus on the PR, esp. a client should >> NOT >> > > > > depend >> > > > > > on a specific deployment. Using an enum will clarify the REST >> spec >> > > > > > contract. >> > > > > > >> > > > > > Yufei >> > > > > > >> > > > > > >> > > > > > On Tue, Sep 29, 2026 at 7:46 AM Dmitri Bourlatchkov < >> > [email protected]> >> > > > > > wrote: >> > > > > > >> > > > > > > Hi All, >> > > > > > > >> > > > > > > PR [5513] has been in review for quite some time. I believe we >> > have >> > > > > > general >> > > > > > > consensus on the Management API changes and java code changes. >> > > > > > > >> > > > > > > The only point that may still have divergent opinions is >> whether >> > to >> > > > use >> > > > > > > free text or enum for "credentialVendingMechanism". I >> personally >> > > > > support >> > > > > > > free text (rationale explained in previous comments). This >> > > > conversation >> > > > > > has >> > > > > > > been dormant for a few days, so I assume lazy consensus on the >> > > > current >> > > > > > > state of the PR (free text). In any case switching from free >> > text to >> > > > > enum >> > > > > > > is a non-breaking API change before the next release in case >> the >> > > > > > consensus >> > > > > > > shifts and we need to adjust. We have about one month for >> > followup >> > > > API >> > > > > > > changes. >> > > > > > > >> > > > > > > The java code introduced in this PR is not rigid and can >> change >> > at >> > > > any >> > > > > > time >> > > > > > > per our evolution guidelines [1]. >> > > > > > > >> > > > > > > I propose merging PR [5513] on Sept 30 unless fresh blocking >> > concerns >> > > > > are >> > > > > > > raised. >> > > > > > > >> > > > > > > [1] https://polaris.apache.org/releases/1.8.0/evolution/ >> > > > > > > >> > > > > > > [5513] https://github.com/apache/polaris/pull/5513 >> > > > > > > >> > > > > > > Cheers, >> > > > > > > Dmitri. >> > > > > > > >> > > > > > > On Tue, Sep 29, 2026 at 9:30 AM Austen Tomek < >> > > > > > > [email protected]> wrote: >> > > > > > > >> > > > > > > > Hey all, >> > > > > > > > >> > > > > > > > Any chance I can get further reviews this week on the PR? >> I’d >> > like >> > > > to >> > > > > > get >> > > > > > > > moving onto Phase 2 here if there are no further issues to >> > address. >> > > > > > > > >> > > > > > > > Get Outlook for Mac<https://aka.ms/GetOutlookForMac> >> > > > > > > > >> > > > > > > > From: Dmitri Bourlatchkov <[email protected]> >> > > > > > > > Date: Thursday, September 24, 2026 at 17:33 >> > > > > > > > To: [email protected] <[email protected]> >> > > > > > > > Subject: Re: [DISCUSS] Cloudflare R2 support with scoped >> > credential >> > > > > > > vending >> > > > > > > > >> > > > > > > > Hi Yufei, >> > > > > > > > >> > > > > > > > > The REST spec is a contract between Polaris servers and >> > clients >> > > > > [...] >> > > > > > > > >> > > > > > > > I tend to think that this statement is a bit too strong. The >> > > > > Management >> > > > > > > API >> > > > > > > > spec defines a protocol for clients and servers to >> communicate. >> > > > > > > > >> > > > > > > > > [...] This would help avoid a situation where a Polaris >> > client >> > > > > works >> > > > > > > > > only with certain deployments. >> > > > > > > > >> > > > > > > > Since Polaris has been developed as a customizable system, >> a >> > REST >> > > > > API >> > > > > > > > specification defined by Apache Polaris cannot control the >> > > > behaviour >> > > > > of >> > > > > > > all >> > > > > > > > possible downstream environments. The purpose of the spec >> is to >> > > > > ensure >> > > > > > > > protocol compatibility. >> > > > > > > > >> > > > > > > > Note that the value of `credentialVendingMechanism` has no >> > bearing >> > > > on >> > > > > > > > any logic clients need to execute. It is an opaque parameter >> > > > > controlled >> > > > > > > > by the Polaris administrator. >> > > > > > > > >> > > > > > > > > The new credentialVendingMechanism field in the REST spec >> > looks >> > > > > > > > > reasonable. My only suggestion is to use an enum instead >> of a >> > > > > string. >> > > > > > > > [...] >> > > > > > > > >> > > > > > > > The above points are exactly the reason why the Management >> API >> > > > cannot >> > > > > > > > enumerate all possible vending mechanisms. >> > > > > > > > >> > > > > > > > The same applies to storage types too. However, we do not >> have >> > any >> > > > > > > > practical >> > > > > > > > examples of 3rd party storage systems that go beyond >> > S3/GCS/ADLS >> > > > > (yet). >> > > > > > > > >> > > > > > > > Cheers, >> > > > > > > > Dmitri. >> > > > > > > > >> > > > > > > > On Thu, Sep 24, 2026 at 4:38 PM Yufei Gu < >> [email protected] >> > > >> > > > > wrote: >> > > > > > > > >> > > > > > > > > Thanks Austen. It's fair that endpoints are not reliable >> > given >> > > > that >> > > > > > > some >> > > > > > > > > s3-compatible can use arbitrary hostnames. >> > > > > > > > > I also think stsUnavailable is fine to keep as-is. I don’t >> > think >> > > > we >> > > > > > > need >> > > > > > > > to >> > > > > > > > > change its scope or repurpose it. >> > > > > > > > > >> > > > > > > > > The new credentialVendingMechanism field in the REST spec >> > looks >> > > > > > > > reasonable. >> > > > > > > > > My only suggestion is to use an enum instead of a string. >> The >> > > > REST >> > > > > > spec >> > > > > > > > is >> > > > > > > > > a contract between Polaris servers and clients, so I’d >> > expect it >> > > > to >> > > > > > > > define >> > > > > > > > > the valid values and their meanings rather than leaving >> them >> > up >> > > > to >> > > > > > each >> > > > > > > > > deployment. This would help avoid a situation where a >> Polaris >> > > > > client >> > > > > > > > works >> > > > > > > > > only with certain deployments. Deployment-specific >> mechanisms >> > > > > > shouldn’t >> > > > > > > > be >> > > > > > > > > part of the standard REST contract; users who need a >> custom >> > > > > mechanism >> > > > > > > can >> > > > > > > > > always extend the server in their own deployment. >> > > > > > > > > >> > > > > > > > > > Either way the operator needs a per-realm switch like >> > > > > > > > > SUPPORTED_S3_CREDENTIAL_VENDING_MECHANISMS to control >> which >> > realm >> > > > > can >> > > > > > > > mint >> > > > > > > > > R2 credentials from the operators Cloudflare account. >> > > > > > > > > >> > > > > > > > > Could you clarify why we need a separate per-realm switch? >> > Once >> > > > the >> > > > > > > > storage >> > > > > > > > > configuration is set at the catalog level to select a >> > specific >> > > > > > > mechanism, >> > > > > > > > > the catalog will have no choice but to support it. Do we >> need >> > > > > another >> > > > > > > > layer >> > > > > > > > > of gating? >> > > > > > > > > >> > > > > > > > > Yufei >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > On Wed, Sep 23, 2026 at 5:53 AM Austen Tomek < >> > > > > > > > > [email protected]> wrote: >> > > > > > > > > >> > > > > > > > > > Hey Yufei, >> > > > > > > > > > >> > > > > > > > > > Apologies for missing your question. >> > > > > > > > > > >> > > > > > > > > > I don’t think Endpoint is sufficient. >> > > > > > > > > > >> > > > > > > > > > Endpoint matching only works where the hostname >> identifies >> > the >> > > > > > vendor >> > > > > > > > > like >> > > > > > > > > > R2’s public hostnames that are recognizable, MinIO and >> > other >> > > > > > > > self-hosted >> > > > > > > > > > S3-compatible stores use arbitrary hostnames so there >> > would be >> > > > > > > nothing >> > > > > > > > to >> > > > > > > > > > match on. On top of that, if a match fails it would just >> > > > silently >> > > > > > > fall >> > > > > > > > > back >> > > > > > > > > > to STS. >> > > > > > > > > > >> > > > > > > > > > Right now the endpoint just says how you’ll connect not >> > how the >> > > > > > > > > > credentials are vended. There's endpoint and also >> > > > > endpointInternal >> > > > > > > for >> > > > > > > > > > server-side access, so it would be unclear which of the >> two >> > > > > > decides. >> > > > > > > > > > >> > > > > > > > > > For your other question, there is no second mechanism >> for >> > R2 >> > > > > today, >> > > > > > > but >> > > > > > > > > > for other self-hosted stores the same endpoint could be >> > served >> > > > by >> > > > > > STS >> > > > > > > > or >> > > > > > > > > by >> > > > > > > > > > a different token mechanism chosen per catalog. >> > > > > > > > > > >> > > > > > > > > > stsUnavailable keeps it’s current behavior in the PR >> (that >> > was >> > > > > > > > Prithvi’s >> > > > > > > > > > point earlier in the thread) >> > > > > > > > > > >> > > > > > > > > > Either way the operator needs a per-realm switch like >> > > > > > > > > > SUPPORTED_S3_CREDENTIAL_VENDING_MECHANISMS to control >> which >> > > > realm >> > > > > > can >> > > > > > > > > mint >> > > > > > > > > > R2 credentials from the operators Cloudflare account. >> > > > > > > > > > >> > > > > > > > > > Thanks, >> > > > > > > > > > Austen >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > Get Outlook for Mac<https://aka.ms/GetOutlookForMac> >> > > > > > > > > > >> > > > > > > > > > From: Yufei Gu <[email protected]> >> > > > > > > > > > Date: Monday, September 21, 2026 at 17:53 >> > > > > > > > > > To: [email protected] <[email protected]> >> > > > > > > > > > Subject: Re: [DISCUSS] Cloudflare R2 support with scoped >> > > > > credential >> > > > > > > > > vending >> > > > > > > > > > >> > > > > > > > > > Hi Austen, >> > > > > > > > > > >> > > > > > > > > > Thanks for the update. After looking at PR #5513, I >> think >> > we >> > > > > should >> > > > > > > > first >> > > > > > > > > > clarify whether we need a new credentialVendingMechanism >> > field >> > > > in >> > > > > > the >> > > > > > > > > > spec/config. >> > > > > > > > > > >> > > > > > > > > > Austen, you mentioned that the R2 account ID and >> > jurisdiction >> > > > > > already >> > > > > > > > > come >> > > > > > > > > > from the endpoint. Could we also use the existing field >> > like >> > > > > > > > `endpoint`, >> > > > > > > > > to >> > > > > > > > > > recognize R2 and select its credential-vending >> mechanism, >> > while >> > > > > > > keeping >> > > > > > > > > the >> > > > > > > > > > current stsUnavailable behavior unchanged? >> > > > > > > > > > >> > > > > > > > > > Are there cases where the endpoint isn’t enough, or >> where >> > users >> > > > > > need >> > > > > > > to >> > > > > > > > > > choose different vending mechanisms for the same >> > endpoint(e.g., >> > > > > R2) >> > > > > > > per >> > > > > > > > > > catalog? >> > > > > > > > > > >> > > > > > > > > > On Wed, Sep 16, 2026 at 7:24 AM Austen Tomek < >> > > > > > > > > > [email protected]> wrote: >> > > > > > > > > > >> > > > > > > > > > > Hi Yufei, Dimitri, >> > > > > > > > > > > >> > > > > > > > > > > Thanks, both. PR 1 is ready for review: >> > > > > > > > > > > https://github.com/apache/polaris/pull/5513 >> > > > > > > > > > > >> > > > > > > > > > > I pushed a revision last night that takes in Dimitri’s >> > > > > comments. >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > * >> > > > > > > > > > > I changed from issuer to credentialVendingMechanism. >> It’s >> > > > also >> > > > > > now >> > > > > > > a >> > > > > > > > > > plain >> > > > > > > > > > > string and the realm allowlist is >> > > > > > > > > > > SUPPORTED_S3_CREDENTIAL_VENDING_MECHANISMS, which >> still >> > > > > defaults >> > > > > > to >> > > > > > > > > [STS] >> > > > > > > > > > > * >> > > > > > > > > > > Mechanisms are CDI beans found by their identifier, as >> > > > Dimitri >> > > > > > > > > proposed. >> > > > > > > > > > > * >> > > > > > > > > > > The CLOUDFLARE_R2 bean and the R2 vending code will >> > follow in >> > > > > the >> > > > > > > > next >> > > > > > > > > PR >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > CI workflow is waiting approval to run. >> > > > > > > > > > > >> > > > > > > > > > > Thanks, >> > > > > > > > > > > Austen >> > > > > > > > > > > >> > > > > > > > > > > Get Outlook for Mac<https://aka.ms/GetOutlookForMac> >> > > > > > > > > > > >> > > > > > > > > > > From: Dmitri Bourlatchkov <[email protected]> >> > > > > > > > > > > Date: Tuesday, September 15, 2026 at 20:00 >> > > > > > > > > > > To: [email protected] <[email protected]> >> > > > > > > > > > > Subject: Re: [DISCUSS] Cloudflare R2 support with >> scoped >> > > > > > credential >> > > > > > > > > > vending >> > > > > > > > > > > >> > > > > > > > > > > Hi All, >> > > > > > > > > > > >> > > > > > > > > > > > Having getCloudflareR2Endpoint on >> > > > AwsStorageConfigurationInfo >> > > > > > > > > > > >> > > > > > > > > > > Yes, this does feel awkward. >> > > > > > > > > > > >> > > > > > > > > > > From my POV pursuing the option with the >> > "credentialIssuers" >> > > > > > > property >> > > > > > > > > > looks >> > > > > > > > > > > promising. This property is fairly generic and fits >> > naturally >> > > > > > into >> > > > > > > > the >> > > > > > > > > > > config, given that we already target multiple >> backends. >> > > > > > > > > > > >> > > > > > > > > > > Cheers, >> > > > > > > > > > > Dmitri. >> > > > > > > > > > > >> > > > > > > > > > > On Tue, Sep 15, 2026 at 8:48 PM Yufei Gu < >> > > > [email protected] >> > > > > > >> > > > > > > > wrote: >> > > > > > > > > > > >> > > > > > > > > > > > Hi Austen, Dmitri, >> > > > > > > > > > > > >> > > > > > > > > > > > > Having getCloudflareR2Endpoint on >> > > > > AwsStorageConfigurationInfo >> > > > > > > > > > > > >> > > > > > > > > > > > That does feel awkward. However, >> > > > AwsStorageConfigurationInfo >> > > > > > has >> > > > > > > > > > > supported >> > > > > > > > > > > > S3-compatible backends beyond AWS for a while, so >> > there is >> > > > > > > > precedent >> > > > > > > > > > > (e.g., >> > > > > > > > > > > > MinIO, etc.). The underlying mismatch is that we >> call >> > the >> > > > > > storage >> > > > > > > > > type >> > > > > > > > > > > S3, >> > > > > > > > > > > > but its configuration mixes shared S3 settings with >> > > > > > AWS-specific >> > > > > > > > > ones: >> > > > > > > > > > > > >> > > > > > > > > > > > AwsStorageConfigurationInfo → storage type: S3 >> > > > > > > > > > > > ├── Common S3 settings >> > > > > > > > > > > > │ endpoint, region, pathStyleAccess, >> > allowedLocations >> > > > > > > > > > > > └── AWS-specific settings >> > > > > > > > > > > > roleArn, externalId, STS, KMS >> > > > > > > > > > > > >> > > > > > > > > > > > I’m open to separating the shared S3 settings from >> > > > > > > > provider-specific >> > > > > > > > > > > > configuration if that helps, but I don’t think we >> need >> > to >> > > > > > settle >> > > > > > > > on a >> > > > > > > > > > > > broader refactor now. >> > > > > > > > > > > > >> > > > > > > > > > > > Let me know when the PR is ready for review. I’d be >> > happy >> > > > to >> > > > > > > take a >> > > > > > > > > > > closer >> > > > > > > > > > > > look. >> > > > > > > > > > > > >> > > > > > > > > > > > Yufei >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > Yufei >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > On Mon, Sep 14, 2026 at 3:18 PM Dmitri Bourlatchkov >> < >> > > > > > > > > [email protected]> >> > > > > > > > > > > > wrote: >> > > > > > > > > > > > >> > > > > > > > > > > > > Hi Austen, >> > > > > > > > > > > > > >> > > > > > > > > > > > > +1 to adding "credentialIssues" or similar >> (renamed) >> > > > > property >> > > > > > > to >> > > > > > > > S3 >> > > > > > > > > > > > config. >> > > > > > > > > > > > > In fact, I was looking exactly for something like >> > that >> > > > last >> > > > > > > week >> > > > > > > > > but >> > > > > > > > > > > the >> > > > > > > > > > > > > idea did not quite condense in my mind :) >> > > > > > > > > > > > > >> > > > > > > > > > > > > I posted some preliminary comments on the draft >> PR. >> > > > > > > > > > > > > >> > > > > > > > > > > > > Just for community awareness, I'd like to propose >> > here >> > > > that >> > > > > > we >> > > > > > > > use >> > > > > > > > > > CDI >> > > > > > > > > > > > for >> > > > > > > > > > > > > finding actual "issuer" / "mechanism" >> > implementations. >> > > > The >> > > > > > > > codebase >> > > > > > > > > > has >> > > > > > > > > > > > > many examples of this. The closest match in this >> > case is >> > > > > > > > > > > > > probably PolarisStorageIntegrationProvider. >> > > > > > > > > > > > > >> > > > > > > > > > > > > This way the API does not have to enumerate all >> > possible >> > > > > > > values. >> > > > > > > > > > > > Downstream >> > > > > > > > > > > > > builds are free to add different implementations >> > without >> > > > > > having >> > > > > > > > to >> > > > > > > > > > > change >> > > > > > > > > > > > > the API spec. The "allowed" list still makes >> sense. >> > > > > > > > > > > > > >> > > > > > > > > > > > > In general, I like the direction this feature is >> > taking, >> > > > > > > > although I >> > > > > > > > > > > > expect >> > > > > > > > > > > > > PR reviews may take a few rounds :) >> > > > > > > > > > > > > >> > > > > > > > > > > > > Cheers, >> > > > > > > > > > > > > Dmitri. >> > > > > > > > > > > > > >> > > > > > > > > > > > > On Mon, Sep 14, 2026 at 3:38 PM Austen Tomek < >> > > > > > > > > > > > > [email protected]> wrote: >> > > > > > > > > > > > > >> > > > > > > > > > > > > > Hi all, >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Closing the loop since my last note and >> answering >> > the >> > > > > > > questions >> > > > > > > > > > that >> > > > > > > > > > > > have >> > > > > > > > > > > > > > come up. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Dmitri's question on role ARNs: R2 doesn’t have >> > > > AWS-style >> > > > > > IAM >> > > > > > > > > roles >> > > > > > > > > > > or >> > > > > > > > > > > > > > ARNs. Our implementation signs temporary >> > credentials >> > > > > with a >> > > > > > > > > parent >> > > > > > > > > > > > > token’s >> > > > > > > > > > > > > > secret, scoped to a bucket and a set of >> prefixes. >> > The >> > > > > > account >> > > > > > > > ID >> > > > > > > > > > and >> > > > > > > > > > > > > > jurisdiction come from the R2 endpoint. Putting >> the >> > > > > account >> > > > > > > ID >> > > > > > > > in >> > > > > > > > > > > > roleArn >> > > > > > > > > > > > > > or userArn would give those fields a different >> > meaning, >> > > > > so >> > > > > > > I’ve >> > > > > > > > > > left >> > > > > > > > > > > > them >> > > > > > > > > > > > > > absent for CLOUDFLARE_R2. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > I pursued Option B. The S3 configuration gains >> one >> > new >> > > > > > field, >> > > > > > > > > > > > > > credentialIssuer, with values STS >> > > > > > > > > > > > > > (default) and CLOUDFLARE_R2 and a realm >> allowlist >> > > > > > > > > > > > > > SUPPORTED_S3_CREDENTIAL_ISSUERS that defaults >> > > > > > > > > > > > > > to [STS]. I left stsUnavailable untouched >> > (Prithvi) so >> > > > it >> > > > > > > still >> > > > > > > > > > means >> > > > > > > > > > > > > > "vend nothing". The >> > > > > > > > > > > > > > credentialIssuer is the explicit sign. The R2 >> > config >> > > > > > rejects >> > > > > > > > > > > > > > stsUnavailable along with a host of other STS >> and >> > KMS >> > > > > > fields. >> > > > > > > > I'm >> > > > > > > > > > > > running >> > > > > > > > > > > > > > this in our non-production environment at the >> > moment. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Draft PR for stage 1 (config field, API, >> allowlist, >> > > > > gates): >> > > > > > > > > > > > > > https://github.com/apache/polaris/pull/5513. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > I still see the appeal of a separate R2 >> > configuration, >> > > > as >> > > > > > > > Dmitri >> > > > > > > > > > > > > > suggested. My original reason for separating it >> > was to >> > > > > keep >> > > > > > > the >> > > > > > > > > > > > > R2-specific >> > > > > > > > > > > > > > configuration and validation together. Having >> > > > > > > > > > getCloudflareR2Endpoint >> > > > > > > > > > > > on >> > > > > > > > > > > > > > AwsStorageConfigurationInfo is one part I’d >> still >> > like >> > > > > > > feedback >> > > > > > > > > on. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > I wanted to put a working version of Option B in >> > front >> > > > of >> > > > > > > > > everyone >> > > > > > > > > > so >> > > > > > > > > > > > we >> > > > > > > > > > > > > > could discuss the actual changes. In my earlier >> > > > > > > implementation, >> > > > > > > > > the >> > > > > > > > > > > > > > separate config also meant a separate storage >> type >> > > > > sharing >> > > > > > > > > > S3FileIO. >> > > > > > > > > > > > That >> > > > > > > > > > > > > > was the tradeoff that led me to try Option B. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Do the diffs change anyone’s lean here? And are >> the >> > > > names >> > > > > > > > > > > > > credentialIssuer >> > > > > > > > > > > > > > / STS / CLOUDFLARE_R2 / >> > SUPPORTED_S3_CREDENTIAL_ISSUERS >> > > > > > > > > acceptable? >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Thanks, >> > > > > > > > > > > > > > Austen >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Get Outlook for Mac< >> > https://aka.ms/GetOutlookForMac> >> > > > > > > > > > > > > > From: Dmitri Bourlatchkov <[email protected]> >> > > > > > > > > > > > > > Date: Sunday, September 13, 2026 at 19:50 >> > > > > > > > > > > > > > To: [email protected] < >> [email protected] >> > > >> > > > > > > > > > > > > > Subject: Re: [DISCUSS] Cloudflare R2 support >> with >> > > > scoped >> > > > > > > > > credential >> > > > > > > > > > > > > vending >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Building on top of Ayush's and JB's suggestion, >> I >> > agree >> > > > > > that >> > > > > > > > > > > > jurisdiction >> > > > > > > > > > > > > > can live inside the existing endpoint config >> field. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > That leaves only Account ID, so potentially we >> > could >> > > > > store >> > > > > > it >> > > > > > > > in >> > > > > > > > > > user >> > > > > > > > > > > > or >> > > > > > > > > > > > > > role ARN. Note that userArn is currently unused >> in >> > OSS >> > > > > > code. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > That should allow us to keep existing >> > > > > > > > AwsStorageConfigurationInfo >> > > > > > > > > > for >> > > > > > > > > > > > all >> > > > > > > > > > > > > > S3 storage backend, but will require special >> > parsing of >> > > > > > > > user/role >> > > > > > > > > > ARN >> > > > > > > > > > > > > field >> > > > > > > > > > > > > > to distinguish R2 from the other systems. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > I'm still leaning towards the separate >> > > > > > > > > R2StorageConfigurationInfo, >> > > > > > > > > > > but >> > > > > > > > > > > > > I'd >> > > > > > > > > > > > > > be ok with overloading user/role ARN too if >> other >> > > > people >> > > > > > > prefer >> > > > > > > > > > > that. I >> > > > > > > > > > > > > > guess a key question in this context is whether >> R2 >> > has >> > > > > the >> > > > > > > > > concept >> > > > > > > > > > of >> > > > > > > > > > > > > role >> > > > > > > > > > > > > > ARN at all.... but TBH I have no idea about >> that. I >> > > > hope >> > > > > > > Austen >> > > > > > > > > > might >> > > > > > > > > > > > be >> > > > > > > > > > > > > > able to clarify that. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Cheers, >> > > > > > > > > > > > > > Dmitri. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > On Sun, Sep 13, 2026 at 9:18 AM Prithvi S < >> > > > > > > > > > > [email protected] >> > > > > > > > > > > > > >> > > > > > > > > > > > > > wrote: >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Hi Austen, >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Welcome, and thanks for starting with a >> > [DISCUSS]. >> > > > Best >> > > > > > way >> > > > > > > > to >> > > > > > > > > > > make a >> > > > > > > > > > > > > > first >> > > > > > > > > > > > > > > contribution :) >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > I think the thread is talking about two >> layers, >> > and >> > > > we >> > > > > > can >> > > > > > > > keep >> > > > > > > > > > > both >> > > > > > > > > > > > > > > answers. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On the client side this should stay S3: s3:// >> > > > > locations, >> > > > > > > s3.* >> > > > > > > > > > > > > properties, >> > > > > > > > > > > > > > > S3FileIO. Your PyIceberg / DuckDB / Iceberg >> Java >> > > > > results >> > > > > > > > > already >> > > > > > > > > > > show >> > > > > > > > > > > > > > that >> > > > > > > > > > > > > > > unmodified clients work, so a dedicated >> R2FileIO >> > does >> > > > > not >> > > > > > > > seem >> > > > > > > > > > > worth >> > > > > > > > > > > > > it. >> > > > > > > > > > > > > > > On the server side I lean toward Dmitri: a >> > distinct >> > > > > > > > > > > > > > > R2StorageConfigurationInfo and storage >> > integration. >> > > > > > > Issuance >> > > > > > > > is >> > > > > > > > > > not >> > > > > > > > > > > > > STS, >> > > > > > > > > > > > > > > and folding Cloudflare fields into >> > > > > > > > AwsStorageConfigurationInfo >> > > > > > > > > > > mixes >> > > > > > > > > > > > > > > unrelated concepts. The middle ground to avoid >> > is a >> > > > new >> > > > > > > type >> > > > > > > > > that >> > > > > > > > > > > is >> > > > > > > > > > > > > > still >> > > > > > > > > > > > > > > AWS-shaped underneath. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > One concrete caution if the Option B prototype >> > stays >> > > > on >> > > > > > the >> > > > > > > > S3 >> > > > > > > > > > > > config: >> > > > > > > > > > > > > > > please do not overload stsUnavailable. Today >> that >> > > > flag >> > > > > > > skips >> > > > > > > > > > > > AssumeRole >> > > > > > > > > > > > > > and >> > > > > > > > > > > > > > > disables vending. R2 wants the opposite. vend, >> > just >> > > > not >> > > > > > via >> > > > > > > > > STS. >> > > > > > > > > > so >> > > > > > > > > > > > > that >> > > > > > > > > > > > > > > flag would change meaning for existing MinIO / >> > NetApp >> > > > > > > > catalogs. >> > > > > > > > > > If >> > > > > > > > > > > we >> > > > > > > > > > > > > > stay >> > > > > > > > > > > > > > > on S3 config, we need an explicit vending >> > signal; a >> > > > > > > dedicated >> > > > > > > > > R2 >> > > > > > > > > > > type >> > > > > > > > > > > > > > > avoids that trap. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On config surface, jurisdiction can live in >> > endpoint >> > > > > > > > > (<account>. >> > > > > > > > > > > > > > > r2.cloudflarestorage.com vs >> <account>.eu.r2...). >> > > > > Account >> > > > > > > ID >> > > > > > > > is >> > > > > > > > > > > still >> > > > > > > > > > > > > > > needed >> > > > > > > > > > > > > > > for the JWT sub, so keeping an explicit >> > accountId and >> > > > > > > > deriving >> > > > > > > > > > > > audience >> > > > > > > > > > > > > > > from the endpoint seems enough. Parent tokens >> > should >> > > > > stay >> > > > > > > in >> > > > > > > > > > server >> > > > > > > > > > > > > > config >> > > > > > > > > > > > > > > via storageName. >> > > > > > > > > > > > > > > Local signing fits the vending model. >> > Fail-closed on >> > > > > > > > > cross-bucket >> > > > > > > > > > > and >> > > > > > > > > > > > > > mixed >> > > > > > > > > > > > > > > read/write grants is the right call. Prefix >> > claims >> > > > > should >> > > > > > > > > follow >> > > > > > > > > > > the >> > > > > > > > > > > > > same >> > > > > > > > > > > > > > > path comparison Polaris already uses, so we do >> > not >> > > > > mint a >> > > > > > > > > > > credential >> > > > > > > > > > > > > > wider >> > > > > > > > > > > > > > > than the grant. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > +1 to small PRs: FileIO discriminator (#4486) >> > > > > separately >> > > > > > if >> > > > > > > > R2 >> > > > > > > > > > > needs >> > > > > > > > > > > > > it, >> > > > > > > > > > > > > > > then management API + Java config, then >> vending, >> > > > Python >> > > > > > CLI >> > > > > > > > on >> > > > > > > > > > its >> > > > > > > > > > > > own. >> > > > > > > > > > > > > > > Sketching MinIO / Backblaze against the same >> > fields >> > > > is >> > > > > a >> > > > > > > > useful >> > > > > > > > > > > > design >> > > > > > > > > > > > > > > check; a generic non-STS SPI can wait for a >> > second >> > > > > > backend. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Regards, >> > > > > > > > > > > > > > > Prithvi S >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On Fri, Sep 11, 2026 at 4:31 AM Dmitri >> > Bourlatchkov < >> > > > > > > > > > > > [email protected]> >> > > > > > > > > > > > > > > wrote: >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Hi Austen, >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Thanks for starting this contribution! >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > After briefly looking through your PR [1] I >> > think >> > > > > > > > > > > > > > > > adding R2StorageConfigInfo makes sense. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Even though "aws" in >> > AwsStorageConfigurationInfo is >> > > > > > > longer >> > > > > > > > > > > implies >> > > > > > > > > > > > > AWS >> > > > > > > > > > > > > > > > services, and MinIO, Rust, Ozone are >> supported >> > just >> > > > > as >> > > > > > > > well, >> > > > > > > > > R2 >> > > > > > > > > > > > > appears >> > > > > > > > > > > > > > > to >> > > > > > > > > > > > > > > > be significantly different from STS-based S3 >> > > > systems >> > > > > > > that a >> > > > > > > > > > > > separate >> > > > > > > > > > > > > > > config >> > > > > > > > > > > > > > > > is probably the most natural way to support >> it >> > in >> > > > the >> > > > > > > > Polaris >> > > > > > > > > > > > > codebase. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > The alternative, I guess, is to add R2 >> > accountId >> > > > and >> > > > > > > > > > jurisdiction >> > > > > > > > > > > > as >> > > > > > > > > > > > > > > > optional properties to >> > AwsStorageConfigurationInfo >> > > > > (or >> > > > > > > > > overload >> > > > > > > > > > > > > > existing >> > > > > > > > > > > > > > > > properties), but that looks like mixing >> > unrelated >> > > > > > > concepts >> > > > > > > > > > > > together. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Adding a generic property bag to >> > > > > > > > AwsStorageConfigurationInfo >> > > > > > > > > is >> > > > > > > > > > > not >> > > > > > > > > > > > > > > > convenient because >> > > > > > PolarisStorageIntegrationProviderImpl >> > > > > > > > will >> > > > > > > > > > > > > probably >> > > > > > > > > > > > > > > have >> > > > > > > > > > > > > > > > to know how to interpret it in order to >> > redirect >> > > > > > > credential >> > > > > > > > > > > vending >> > > > > > > > > > > > > > > > requests to R2-specific code. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > I wonder whether R2StorageConfigurationInfo >> and >> > > > > > > > > > > > > > > AwsStorageConfigurationInfo >> > > > > > > > > > > > > > > > (and corresponding "storage integration" >> > classes) >> > > > > could >> > > > > > > > > share a >> > > > > > > > > > > > > common >> > > > > > > > > > > > > > > base >> > > > > > > > > > > > > > > > class for common properties (e.g. endpoint). >> > Even >> > > > > > though >> > > > > > > > > > > "endpoint" >> > > > > > > > > > > > > > might >> > > > > > > > > > > > > > > > not be relevant to R2 in the cloud, the fact >> > that >> > > > > > > S3FileIO >> > > > > > > > > uses >> > > > > > > > > > > it, >> > > > > > > > > > > > > and >> > > > > > > > > > > > > > > R2 >> > > > > > > > > > > > > > > > is accessed via S3FileIO means we should >> > probably >> > > > > allow >> > > > > > > > users >> > > > > > > > > > to >> > > > > > > > > > > > > > > configure >> > > > > > > > > > > > > > > > the full set of FileIO properties just in >> case. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > I think it is fine for different >> > > > > > > PolarisStorageIntegration >> > > > > > > > > > > > > > > implementations >> > > > > > > > > > > > > > > > to produce the same kind of >> > StorageAccessConfig (S3 >> > > > > in >> > > > > > > this >> > > > > > > > > > case) >> > > > > > > > > > > > and >> > > > > > > > > > > > > > > thus >> > > > > > > > > > > > > > > > cause S3FileIO to be used by clients >> (including >> > > > > > Polaris' >> > > > > > > > own >> > > > > > > > > > > > storage >> > > > > > > > > > > > > > > access >> > > > > > > > > > > > > > > > paths). >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Changes in StorageTypeFileIO in [1] look a >> bit >> > > > > > concerning >> > > > > > > > to >> > > > > > > > > > me. >> > > > > > > > > > > I >> > > > > > > > > > > > > > wonder >> > > > > > > > > > > > > > > > if this code could be refactored to avoid >> > > > > dependencies >> > > > > > on >> > > > > > > > > > storage >> > > > > > > > > > > > > > config >> > > > > > > > > > > > > > > > types completely.... but TBH, I did not look >> > too >> > > > > deeply >> > > > > > > > into >> > > > > > > > > > this >> > > > > > > > > > > > > > today. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Anticipating future PRs for this in the >> Polaris >> > > > repo, >> > > > > > I'd >> > > > > > > > > like >> > > > > > > > > > to >> > > > > > > > > > > > ask >> > > > > > > > > > > > > > to >> > > > > > > > > > > > > > > > separate Python code changes from java >> changes >> > > > since >> > > > > > > these >> > > > > > > > > > areas >> > > > > > > > > > > > > > usually >> > > > > > > > > > > > > > > > attract different reviewers :) >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > From my POV, management API and matching >> java >> > > > changes >> > > > > > can >> > > > > > > > be >> > > > > > > > > in >> > > > > > > > > > > the >> > > > > > > > > > > > > > same >> > > > > > > > > > > > > > > > PR. I'd say docs should come later (to keep >> PRs >> > > > > small). >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > All in all, thank you again and I hope this >> > > > > > contribution >> > > > > > > > > lands >> > > > > > > > > > in >> > > > > > > > > > > > > > Polaris >> > > > > > > > > > > > > > > > :) >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > [1] >> > https://github.com/deepdishgary/polaris/pull/1 >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Cheers, >> > > > > > > > > > > > > > > > Dmitri. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > On Thu, Sep 10, 2026 at 1:52 PM >> Jean-Baptiste >> > > > Onofré >> > > > > < >> > > > > > > > > > > > > [email protected]> >> > > > > > > > > > > > > > > > wrote: >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Hi Austen, >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Nice discussion! Thanks for that, and >> welcome >> > > > > aboard >> > > > > > :) >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > I think it makes sense to have R2 built >> > under the >> > > > > > > > existing >> > > > > > > > > S3 >> > > > > > > > > > > > > storage >> > > > > > > > > > > > > > > > > configuration. It matches how Polaris is >> > already >> > > > > > > > > structured: >> > > > > > > > > > > > > > > > > StorageType is keyed on the URI scheme >> > (s3://, >> > > > > > s3a://), >> > > > > > > > and >> > > > > > > > > > > > > > > > > AwsStorageConfigurationInfo already >> carries >> > > > > endpoint, >> > > > > > > > > > > > stsEndpoint, >> > > > > > > > > > > > > > > > > pathStyleAccess and, importantly, an >> > > > stsUnavailable >> > > > > > > flag. >> > > > > > > > > > > > > > > > > So, what you need is largely there >> already: >> > > > correct >> > > > > > me >> > > > > > > if >> > > > > > > > > I'm >> > > > > > > > > > > > > wrong, >> > > > > > > > > > > > > > > > > R2 is essentially "S3 compatible store >> where >> > STS >> > > > is >> > > > > > > > > > > unavailable, >> > > > > > > > > > > > > plus >> > > > > > > > > > > > > > > > > a way to actually vend credentials in that >> > case". >> > > > > > Going >> > > > > > > > > this >> > > > > > > > > > > > route >> > > > > > > > > > > > > > > > > also keeps this work "isolated" from the >> > > > > > > > > > > FileIO-per-storage-type >> > > > > > > > > > > > > > > > > discussion. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > The interesting design question is a >> > pluggable >> > > > > > > credential >> > > > > > > > > > > vending >> > > > > > > > > > > > > > path >> > > > > > > > > > > > > > > > > for S3 compatible stores when STS is not >> > > > available, >> > > > > > > with >> > > > > > > > > > > > > Cloudflare's >> > > > > > > > > > > > > > > > > local signing as the first concrete >> > > > implementation, >> > > > > > > > rather >> > > > > > > > > > than >> > > > > > > > > > > > R2 >> > > > > > > > > > > > > > > > > specific code. It would be good to sketch >> how >> > > > MinIO >> > > > > > > would >> > > > > > > > > > slot >> > > > > > > > > > > > into >> > > > > > > > > > > > > > > > > the same config fields before the vending >> > path >> > > > > > hardens >> > > > > > > > > around >> > > > > > > > > > > R2. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > I believe we have to check if Cloudflare >> > account >> > > > ID >> > > > > > and >> > > > > > > > > > > > > jurisdiction >> > > > > > > > > > > > > > > > > really need dedicated config fields, or >> can >> > we >> > > > use >> > > > > > the >> > > > > > > > > > endpoint >> > > > > > > > > > > > > value >> > > > > > > > > > > > > > > > > (with documentation)? Keeping the >> > configuration >> > > > > > surface >> > > > > > > > > > minimal >> > > > > > > > > > > > > will >> > > > > > > > > > > > > > > > > help the design generalize. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > I would love to discuss two things: >> > > > > > > > > > > > > > > > > 1. Polaris shipping a Cloudflare API token >> > and >> > > > > > locally >> > > > > > > > > > minting >> > > > > > > > > > > > > scoped >> > > > > > > > > > > > > > > > > credentials makes it effectively an STS >> for >> > R2, >> > > > > which >> > > > > > > > > > increases >> > > > > > > > > > > > the >> > > > > > > > > > > > > > > > > blast radius of a server compromise. Your >> > "reject >> > > > > > > rather >> > > > > > > > > than >> > > > > > > > > > > > > > boarden" >> > > > > > > > > > > > > > > > > handling of unrepresentable scopes is the >> > right >> > > > > call. >> > > > > > > > > > > > > > > > > 2. The PyIceberg/DuckDB/Java results are >> > > > promising. >> > > > > > > Just >> > > > > > > > > > > curious >> > > > > > > > > > > > > > about >> > > > > > > > > > > > > > > > > Spark or Trino. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > We love small, well-scoped PRs. I would >> > suggest >> > > > to >> > > > > > > start >> > > > > > > > > with >> > > > > > > > > > > the >> > > > > > > > > > > > > S3 >> > > > > > > > > > > > > > > > > config and the management API as first >> > shoot, and >> > > > > we >> > > > > > > can >> > > > > > > > > > follow >> > > > > > > > > > > > > with >> > > > > > > > > > > > > > > > > the credential vending implementation. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Regards >> > > > > > > > > > > > > > > > > JB >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > On Wed, Sep 9, 2026 at 8:18 PM Austen >> Tomek >> > > > > > > > > > > > > > > > > <[email protected]> wrote: >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Hi all, >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > I’m Austen Tomek from Chicago Trading >> > Company. >> > > > We >> > > > > > use >> > > > > > > > > > Polaris >> > > > > > > > > > > > for >> > > > > > > > > > > > > > our >> > > > > > > > > > > > > > > > > Iceberg catalogs and we're evaluating >> > Cloudflare >> > > > R2 >> > > > > > as >> > > > > > > an >> > > > > > > > > > > object >> > > > > > > > > > > > > > store >> > > > > > > > > > > > > > > > (the >> > > > > > > > > > > > > > > > > no egress fees make it a very compelling >> > > > product). >> > > > > > I’d >> > > > > > > > like >> > > > > > > > > > to >> > > > > > > > > > > > > > discuss >> > > > > > > > > > > > > > > > > contributing R2 support and credential >> > vending to >> > > > > > > Polaris >> > > > > > > > > > with >> > > > > > > > > > > > the >> > > > > > > > > > > > > > goal >> > > > > > > > > > > > > > > > of >> > > > > > > > > > > > > > > > > getting feedback before opening a >> > > > ready-for-review >> > > > > > > > upstream >> > > > > > > > > > PR. >> > > > > > > > > > > > > This >> > > > > > > > > > > > > > is >> > > > > > > > > > > > > > > > my >> > > > > > > > > > > > > > > > > first contribution, so trying to do this >> > right. >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > R2 exposes S3-compatible APIs but it >> does >> > not >> > > > > > provide >> > > > > > > > AWS >> > > > > > > > > > > STS. >> > > > > > > > > > > > > > > > > Cloudflare instead supports these >> > short-lived, >> > > > > scoped >> > > > > > > > > > > credentials >> > > > > > > > > > > > > > > > generated >> > > > > > > > > > > > > > > > > by locally signed JWT with the server >> > holding the >> > > > > > > parent >> > > > > > > > > API >> > > > > > > > > > > > token >> > > > > > > > > > > > > > [1]. >> > > > > > > > > > > > > > > > > This allows us to retain Polaris’s >> > authorization >> > > > > and >> > > > > > > > > > > > > > credential-vending >> > > > > > > > > > > > > > > > > model while storing our table data in R2. >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > I have a prototype with the following >> > design: >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > * >> > > > > > > > > > > > > > > > > > An opt-in R2 storage type, with catalog >> > > > > > configuration >> > > > > > > > > > > > identifying >> > > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > > Cloudflare account and optional >> jurisdiction. >> > > > > Parent >> > > > > > > > > > > credentials >> > > > > > > > > > > > > > remain >> > > > > > > > > > > > > > > > in >> > > > > > > > > > > > > > > > > server configuration and can be selected >> > through >> > > > > > > > > storageName. >> > > > > > > > > > > > > > > > > > * >> > > > > > > > > > > > > > > > > > Local credential generation using >> > Cloudflare’s >> > > > > > > > documented >> > > > > > > > > > > > format. >> > > > > > > > > > > > > > > > > Credentials expire, are restricted to one >> > bucket, >> > > > > and >> > > > > > > > carry >> > > > > > > > > > > > prefix >> > > > > > > > > > > > > > and >> > > > > > > > > > > > > > > > > access scopes derived from Polaris’s >> location >> > > > > grants. >> > > > > > > The >> > > > > > > > > > > > > integration >> > > > > > > > > > > > > > > > uses >> > > > > > > > > > > > > > > > > Polaris’s existing credential cache. >> > > > > > > > > > > > > > > > > > * >> > > > > > > > > > > > > > > > > > Credentials returned through the >> existing >> > > > Iceberg >> > > > > > > REST >> > > > > > > > > > > vending >> > > > > > > > > > > > > flow >> > > > > > > > > > > > > > > as >> > > > > > > > > > > > > > > > > s3.* properties, including the endpoint, >> > session >> > > > > > token, >> > > > > > > > and >> > > > > > > > > > > > expiry. >> > > > > > > > > > > > > > > > > Compatible clients use their existing S3 >> > FileIO. >> > > > > > > > > > > > > > > > > > * >> > > > > > > > > > > > > > > > > > Conservative handling of scopes the >> > > > > implementation >> > > > > > > > cannot >> > > > > > > > > > > > > > represent: >> > > > > > > > > > > > > > > > > cross-bucket and mixed read/write grants >> are >> > > > > rejected >> > > > > > > > > rather >> > > > > > > > > > > than >> > > > > > > > > > > > > > > > > broadened. R2 is excluded from the default >> > > > > > > > > > > > supported-storage-types >> > > > > > > > > > > > > > > list. >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > The main design questions I feel this >> runs >> > > > into: >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > * >> > > > > > > > > > > > > > > > > > Should this be a distinct storage type >> or a >> > > > > > > > > > > credential-vending >> > > > > > > > > > > > > > > > mechanism >> > > > > > > > > > > > > > > > > selected through the existing S3 >> > configuration? >> > > > > > > > > > > > > > > > > > * >> > > > > > > > > > > > > > > > > > I chose a separate type because the >> > identity, >> > > > > > > > credential >> > > > > > > > > > > > > > generation, >> > > > > > > > > > > > > > > > and >> > > > > > > > > > > > > > > > > scoping model differ from AWS STS, as this >> > seemed >> > > > > > more >> > > > > > > > > > > > appropriate >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > There is a draft preview PR against my >> fork >> > > > [2], >> > > > > > > > > including >> > > > > > > > > > > > > > > > > implementation, management API changes, >> CLI >> > > > > support, >> > > > > > > > tests, >> > > > > > > > > > and >> > > > > > > > > > > > > > > > > documentation. It also discloses the AI >> > > > assistance >> > > > > > used >> > > > > > > > > > during >> > > > > > > > > > > > > > > > > implementation. I leaned heavily on Fable >> > 5.1 for >> > > > > > this >> > > > > > > > > > > > > implementation >> > > > > > > > > > > > > > > so >> > > > > > > > > > > > > > > > I >> > > > > > > > > > > > > > > > > want to be upfront about it. >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > We are running the implementation in a >> > > > > > non-production >> > > > > > > > > > > > environment >> > > > > > > > > > > > > > > right >> > > > > > > > > > > > > > > > > now as a POC. Validation includes live R2 >> > > > > > reads/writes, >> > > > > > > > > > > multipart >> > > > > > > > > > > > > > > > > operations, negative scope-boundary tests, >> > our >> > > > > > internal >> > > > > > > > > > Python >> > > > > > > > > > > > > client >> > > > > > > > > > > > > > > > > regression matrix, and credential >> > > > rotation/recovery >> > > > > > > > > > exercises. >> > > > > > > > > > > > The >> > > > > > > > > > > > > > > > upstream >> > > > > > > > > > > > > > > > > CI workflow has passed on the fork. >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > The change also encounters the existing >> > > > > assumption >> > > > > > > > that a >> > > > > > > > > > > > FileIO >> > > > > > > > > > > > > > > > > implementation identifies exactly one >> storage >> > > > type: >> > > > > > > both >> > > > > > > > S3 >> > > > > > > > > > and >> > > > > > > > > > > > R2 >> > > > > > > > > > > > > > use >> > > > > > > > > > > > > > > > > S3FileIO. This overlaps with #4486 [3]. >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > I’d particularly appreciate feedback on: >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > 1. >> > > > > > > > > > > > > > > > > > Should R2 have its own storage >> > > > > configuration/type, >> > > > > > or >> > > > > > > > > > should >> > > > > > > > > > > > > > non-STS >> > > > > > > > > > > > > > > > > vending be modeled within the existing S3 >> > > > > > integration? >> > > > > > > > > > > > > > > > > > 2. >> > > > > > > > > > > > > > > > > > Does the local-signing approach fit >> > Polaris’s >> > > > > > > > > > > > credential-vending >> > > > > > > > > > > > > > > model? >> > > > > > > > > > > > > > > > > Any additional constraints or validations >> > > > > necessary? >> > > > > > > > > > > > > > > > > > 3. >> > > > > > > > > > > > > > > > > > Would you prefer the shared FileIO >> > validation >> > > > > > change >> > > > > > > to >> > > > > > > > > > land >> > > > > > > > > > > > > > > > separately, >> > > > > > > > > > > > > > > > > and how should we sequence the management >> > API and >> > > > > R2 >> > > > > > > > > > > integration >> > > > > > > > > > > > > > > changes? >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Happy to adapt the design and split >> the >> > > > > > > contribution >> > > > > > > > > into >> > > > > > > > > > > > more >> > > > > > > > > > > > > > > > focused >> > > > > > > > > > > > > > > > > PRs. Wanted to start the discussion >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Thanks, >> > > > > > > > > > > > > > > > > > Austen >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > [1] >> > > > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > >> https://developers.cloudflare.com/r2/api/s3/temporary-credentials/ >> > > > > > > > > > > > > > > > > > [2] >> > > > > > https://github.com/deepdishgary/polaris/pull/1 >> > > > > > > > > > > > > > > > > > [3] >> > > > > > https://github.com/apache/polaris/issues/4486 >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > Get Outlook for Mac< >> > > > > > https://aka.ms/GetOutlookForMac> >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > This electronic mail message and any >> > attached >> > > > > files >> > > > > > > > > contain >> > > > > > > > > > > > > > > information >> > > > > > > > > > > > > > > > > intended for the exclusive use of the >> > individual >> > > > or >> > > > > > > > entity >> > > > > > > > > to >> > > > > > > > > > > > whom >> > > > > > > > > > > > > it >> > > > > > > > > > > > > > > is >> > > > > > > > > > > > > > > > > addressed and may contain information >> that is >> > > > > > > > proprietary, >> > > > > > > > > > > > > > confidential >> > > > > > > > > > > > > > > > > and/or exempt from disclosure under >> > applicable >> > > > law. >> > > > > > If >> > > > > > > > you >> > > > > > > > > > are >> > > > > > > > > > > > not >> > > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > > intended recipient, you are hereby >> notified >> > that >> > > > > any >> > > > > > > > > viewing, >> > > > > > > > > > > > > > copying, >> > > > > > > > > > > > > > > > > disclosure or distribution of this >> > information >> > > > may >> > > > > be >> > > > > > > > > subject >> > > > > > > > > > > to >> > > > > > > > > > > > > > legal >> > > > > > > > > > > > > > > > > restriction or sanction. Please notify the >> > > > sender, >> > > > > by >> > > > > > > > > > > electronic >> > > > > > > > > > > > > mail >> > > > > > > > > > > > > > > or >> > > > > > > > > > > > > > > > > telephone, of any unintended recipients >> and >> > > > delete >> > > > > > the >> > > > > > > > > > original >> > > > > > > > > > > > > > message >> > > > > > > > > > > > > > > > > without making any copies. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > This electronic mail message and any attached >> files >> > > > > contain >> > > > > > > > > > > information >> > > > > > > > > > > > > > intended for the exclusive use of the >> individual or >> > > > > entity >> > > > > > to >> > > > > > > > > whom >> > > > > > > > > > it >> > > > > > > > > > > > is >> > > > > > > > > > > > > > addressed and may contain information that is >> > > > > proprietary, >> > > > > > > > > > > confidential >> > > > > > > > > > > > > > and/or exempt from disclosure under applicable >> > law. If >> > > > > you >> > > > > > > are >> > > > > > > > > not >> > > > > > > > > > > the >> > > > > > > > > > > > > > intended recipient, you are hereby notified that >> > any >> > > > > > viewing, >> > > > > > > > > > > copying, >> > > > > > > > > > > > > > disclosure or distribution of this information >> may >> > be >> > > > > > subject >> > > > > > > > to >> > > > > > > > > > > legal >> > > > > > > > > > > > > > restriction or sanction. Please notify the >> sender, >> > by >> > > > > > > > electronic >> > > > > > > > > > mail >> > > > > > > > > > > > or >> > > > > > > > > > > > > > telephone, of any unintended recipients and >> delete >> > the >> > > > > > > original >> > > > > > > > > > > message >> > > > > > > > > > > > > > without making any copies. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > This electronic mail message and any attached files >> > contain >> > > > > > > > information >> > > > > > > > > > > intended for the exclusive use of the individual or >> > entity to >> > > > > > whom >> > > > > > > it >> > > > > > > > > is >> > > > > > > > > > > addressed and may contain information that is >> > proprietary, >> > > > > > > > confidential >> > > > > > > > > > > and/or exempt from disclosure under applicable law. If >> > you >> > > > are >> > > > > > not >> > > > > > > > the >> > > > > > > > > > > intended recipient, you are hereby notified that any >> > viewing, >> > > > > > > > copying, >> > > > > > > > > > > disclosure or distribution of this information may be >> > subject >> > > > > to >> > > > > > > > legal >> > > > > > > > > > > restriction or sanction. Please notify the sender, by >> > > > > electronic >> > > > > > > mail >> > > > > > > > > or >> > > > > > > > > > > telephone, of any unintended recipients and delete the >> > > > original >> > > > > > > > message >> > > > > > > > > > > without making any copies. >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > This electronic mail message and any attached files >> contain >> > > > > > > information >> > > > > > > > > > intended for the exclusive use of the individual or >> entity >> > to >> > > > > whom >> > > > > > it >> > > > > > > > is >> > > > > > > > > > addressed and may contain information that is >> proprietary, >> > > > > > > confidential >> > > > > > > > > > and/or exempt from disclosure under applicable law. If >> you >> > are >> > > > > not >> > > > > > > the >> > > > > > > > > > intended recipient, you are hereby notified that any >> > viewing, >> > > > > > > copying, >> > > > > > > > > > disclosure or distribution of this information may be >> > subject >> > > > to >> > > > > > > legal >> > > > > > > > > > restriction or sanction. Please notify the sender, by >> > > > electronic >> > > > > > mail >> > > > > > > > or >> > > > > > > > > > telephone, of any unintended recipients and delete the >> > original >> > > > > > > message >> > > > > > > > > > without making any copies. >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > This electronic mail message and any attached files contain >> > > > > information >> > > > > > > > intended for the exclusive use of the individual or entity >> to >> > whom >> > > > it >> > > > > > is >> > > > > > > > addressed and may contain information that is proprietary, >> > > > > confidential >> > > > > > > > and/or exempt from disclosure under applicable law. If you >> are >> > not >> > > > > the >> > > > > > > > intended recipient, you are hereby notified that any >> viewing, >> > > > > copying, >> > > > > > > > disclosure or distribution of this information may be >> subject >> > to >> > > > > legal >> > > > > > > > restriction or sanction. Please notify the sender, by >> > electronic >> > > > mail >> > > > > > or >> > > > > > > > telephone, of any unintended recipients and delete the >> original >> > > > > message >> > > > > > > > without making any copies. >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > >> >
