Hi all, Closing the loop on the Iceberg side: the REST spec change that adds `none` to the X-Iceberg-Access-Delegation header values was merged today (https://github.com/apache/iceberg/pull/18480), after a vote on dev@iceberg (https://lists.apache.org/thread/c5ldjw25fplxjfplfx8fnvbkv598yy3p).
With that in place, Polaris can treat `none` as "the client does not need delegated access" and return the table without credentials, instead of the 400 we return today when no requested mechanism can be satisfied. I would like to pick that up as a follow-up: accept `none` in DefaultAccessDelegationModeResolver, skip credential vending for it, and update the vended-credentials documentation. It also pairs well with the new skip flags from #5884, which Dmitri mentioned in the Typed Storage Config thread. If nobody objects, I will open an issue with the proposed behaviour and a PR after that. Thanks, Youngrae Kim 2026년 10월 12일 (월) 오후 6:40, youngrae kim <[email protected]>님이 작성: > Thanks JB. > > On the documentation: the "When delegation cannot be satisfied" section of > the vended-credentials page (added in #5591) already lists both 400 > responses, > explains why Polaris fails fast, and tells clients to drop the header and > configure storage credentials through the usual FileIO properties, > with a link to the S3-compatible setup. If you see anything missing there, > let me know and I will extend it. > > On the wording: Yufei's proposal is "The server may choose to supply > access via any or none of the requested mechanisms, > or fail the request", so it stays permissive; I will keep it that way in > the spec PR. > > Agreed that the vended-credentials + remote-signing combination with > Iceberg 1.12 deserves its own thread. > > Youngrae Kim > > 2026년 10월 12일 (월) 오후 5:20, Jean-Baptiste Onofré <[email protected]>님이 작성: > >> Hi all, >> >> I agree on the consensus here. +1 to keep fail-fast (HTTP 400) in Polaris. >> >> I think we have to update the documentation "cleanly", with clear >> statement >> about the header and/or configure their storage credentials. >> About the wording, I support adding "or fail the request", but I would >> keep >> it permissive rather than a SHOULD. >> >> With Iceberg 1.12 (adding remoteSigningConfig()) we probably should >> revisit >> how combined requests like vended-credentials, remote-signing are handled. >> I don't ask to discuss that in this thread, but probably a follow-up thead >> :) >> >> Regards >> JB >> >> On Mon, Oct 12, 2026 at 8:56 AM youngrae kim <[email protected]> >> wrote: >> >> > Thanks Yufei, Alex, Dmitri. Agreed on keeping the fail-fast behavior in >> > Polaris for now. >> > >> > The Iceberg thread converged on adding "none" and clarifying that >> failing >> > the request is allowed. >> > I have offered there to open the spec PR; once that lands, I am happy to >> > follow up here on supporting "none" in Polaris. >> > >> > Youngrae Kim >> > >> > 2026년 10월 8일 (목) 오전 6:36, Yufei Gu <[email protected]>님이 작성: >> > >> > > I followed up on the Iceberg thread today. Dan suggested "none" >> > > instead of "local", meaning the client does not need delegated access >> > > from the catalog. That works for me and addresses the missing signal >> > > Dmitri pointed out. Thanks Alex and Youngrae for your involvement in >> > > the Iceberg community. >> > > >> > > I also proposed explicitly adding “or fail the request” to the >> > > existing spec wording. This would clarify that Polaris’ behavior is >> > > permitted without requiring all catalogs to behave the same way. >> > > >> > > For Polaris, I still support keeping the current fail-fast behavior, >> > > which has been documented in #5591. We can revisit support for none >> > > once its semantics are agreed upstream. >> > > >> > > Yufei >> > > >> > > On Mon, Oct 5, 2026 at 12:53 AM Alex Dutra <[email protected]> wrote: >> > > > >> > > > Hi Dmitri, >> > > > >> > > > > Do you think this is worth discussing with the Iceberg community? >> > > > >> > > > I did already, see this part of my reply [1] where I was advocating >> > > > for your "local" capability suggestion: >> > > > >> > > > > [The header] fails to represent a key capability: that of using >> the >> > > client's own storage credentials to bypass access delegation entirely. >> > > Right now, the only way to imply this capability is by omitting the >> > header >> > > altogether, which introduces ambiguity into the specification. >> > > > >> > > > That being said, I just sent another message to the same Iceberg >> > > > topic, relaying your suggestion. >> > > > >> > > > Thanks, >> > > > Alex >> > > > >> > > > [1]: >> > > https://lists.apache.org/thread.html/3r6wrblr0l7xs7cb2f7qs07ovh792vyo >> > > > >> > > > On Fri, Oct 2, 2026 at 9:30 PM Dmitri Bourlatchkov < >> [email protected]> >> > > wrote: >> > > > > >> > > > > Hi Alex, >> > > > > >> > > > > I've been thinking about the perspective using the header as a >> client >> > > > > capability. >> > > > > >> > > > > I think it has merit; however, I'd expect the client to declare >> its >> > > full >> > > > > set of capabilities in that case. >> > > > > >> > > > > For example, if a client has local storage credentials, I'd >> expect a >> > > header >> > > > > value like this: >> > > > > "local,remote-signing,vended-credentials". >> > > > > >> > > > > If the client lacks local credentials and does not support remote >> > > signing, >> > > > > it would send: >> > > > > "vended-credentials". >> > > > > >> > > > > With this approach the server (Polaris) would be able to make >> precise >> > > > > access decisions and fail requests only when no viable access >> option >> > > > > remains. >> > > > > >> > > > > Do you think this is worth discussing with the Iceberg community? >> > > > > >> > > > > However, since "local" is not something that existing clients >> > provide, >> > > I >> > > > > think Polaris has to resort to heuristic assumptions that a client >> > > flagging >> > > > > _some_ access delegation mechanisms cannot work without them. >> > > > > >> > > > > Cheers, >> > > > > Dmitri. >> > > > > >> > > > > On Fri, Oct 2, 2026 at 5:01 AM Alex Dutra <[email protected]> >> wrote: >> > > > > >> > > > > > Hi all, >> > > > > > >> > > > > > The recent discussion on the Iceberg mailing list highlighted an >> > > > > > interesting perspective: some view the delegation header >> primarily >> > as >> > > > > > a capability statement rather than a strict requirement. >> > > > > > >> > > > > > Under that approach, delegation decisions rely mainly on >> internal >> > > > > > catalog configurations (such as warehouse-level allowed modes). >> The >> > > > > > header merely serves as an optional hint, making error responses >> > > > > > unnecessary. >> > > > > > >> > > > > > By contrast, Polaris has treated the header as a mandatory >> > > requirement >> > > > > > from the start. The server attempts to fulfill it by finding the >> > > > > > intersection between supported capabilities and client >> preferences, >> > > > > > returning an error if no common mode is available. >> > > > > > >> > > > > > Daniel Weeks confirmed that the Polaris model is perfectly >> valid. >> > > > > > Consequently, no changes are required on our end, though we >> should >> > > > > > keep in mind that Polaris may be taking a distinct path with its >> > > > > > interpretation of the spec, relative to what other catalogs may >> be >> > > > > > doing. >> > > > > > >> > > > > > Thanks, >> > > > > > Alex >> > > > > > >> > > > > > >> > > > > > On Tue, Sep 29, 2026 at 4:11 AM youngrae kim < >> > [email protected] >> > > > >> > > > > > wrote: >> > > > > > > >> > > > > > > Thanks Prithvi. >> > > > > > > >> > > > > > > Closing the loop on the Iceberg side [1]: Dan Weeks (spec >> author) >> > > prefers >> > > > > > > to keep the header >> > > > > > > wording intentionally loose so that the catalog stays the sole >> > > authority >> > > > > > on >> > > > > > > access delegation. >> > > > > > > He considers the Polaris fail-fast behavior fine as an >> > > implementation >> > > > > > > choice but does not want the spec >> > > > > > > to say catalogs SHOULD fail; a catalog may also provide an >> > > alternative >> > > > > > > mechanism or return metadata only. >> > > > > > > Alex asked for a narrower clarification (capability statement >> vs >> > > > > > > requirement, meaning of an absent header), >> > > > > > > which is still open there. >> > > > > > > >> > > > > > > For Polaris nothing changes: we keep the 400 with the two >> > messages, >> > > > > > > documented in #5591 as our behavior. >> > > > > > > If the Iceberg thread later lands a clarification I will >> follow >> > up >> > > here. >> > > > > > > >> > > > > > > [1] >> > > https://lists.apache.org/thread/6zspb6cpb6j2po5lm57mq1sgcp2hppkf >> > > > > > > >> > > > > > > Thanks, >> > > > > > > Youngrae Kim >> > > > > > > >> > > > > > > 2026년 9월 25일 (금) 오전 6:25, Prithvi S < >> [email protected] >> > >님이 >> > > 작성: >> > > > > > > >> > > > > > > > Hi all, >> > > > > > > > >> > > > > > > > I agree with failing the request when the server can supply >> > none >> > > of the >> > > > > > > > requested mechanisms, and with Alex's SHOULD. >> > > > > > > > >> > > > > > > > In Polaris the two cases we discussed are static properties >> of >> > > the >> > > > > > catalog. >> > > > > > > > stsUnavailable will not start working on a later load, and >> > remote >> > > > > > signing >> > > > > > > > is not implemented. A 200 with no delegated access only >> moves >> > the >> > > > > > failure >> > > > > > > > to the first data access, where it is much harder to >> diagnose. >> > > We kept >> > > > > > the >> > > > > > > > 400 and wrote down both messages, and what the client should >> > > change, in >> > > > > > > > https://github.com/apache/polaris/pull/5591. >> > > > > > > > >> > > > > > > > SHOULD is the right strength for the spec. It records the >> > > behavior >> > > > > > > > operators can diagnose, and a server that knows its clients >> > carry >> > > > > > storage >> > > > > > > > credentials can still return the table. Polaris does not >> take >> > > that >> > > > > > path. >> > > > > > > > The header is often a shared client setting: PyIceberg >> > documents >> > > > > > > > vended-credentials as the default ( >> > > > > > > > https://py.iceberg.apache.org/configuration/), so a client >> can >> > > send it >> > > > > > > > without anyone having decided that this catalog should vend. >> > > > > > > > >> > > > > > > > The 400 sentence is the clarification this thread needs. >> > Whether >> > > one >> > > > > > > > response may carry more than one mechanism is a separate >> > change. >> > > > > > Polaris >> > > > > > > > selects a single mechanism, and that can stay as it is >> until a >> > > server >> > > > > > > > returns both. >> > > > > > > > >> > > > > > > > HTTP 400 with the existing Bad Request error is enough. >> Clients >> > > already >> > > > > > > > fail the load on it. The message is what tells the operator >> > which >> > > > > > mechanism >> > > > > > > > was missing, and to drop the header or configure storage >> > > credentials >> > > > > > on the >> > > > > > > > client. A distinct error type can wait until a client needs >> to >> > > branch >> > > > > > on >> > > > > > > > this case in code. >> > > > > > > > >> > > > > > > > Thanks, >> > > > > > > > Prithvi S >> > > > > > > > >> > > > > > > > On Wed, Sep 23, 2026 at 2:11 PM youngrae kim < >> > > [email protected]> >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > Thanks Alex, that's helpful context. The >> > > backwards-compatibility >> > > > > > reading >> > > > > > > > of >> > > > > > > > > "must first check" >> > > > > > > > > makes sense, and the upcoming remote-signing-config field >> in >> > > Iceberg >> > > > > > 1.12 >> > > > > > > > > following the >> > > > > > > > > same dual-location pattern is a good reason to get the >> > > "delegation >> > > > > > not >> > > > > > > > > provided" semantics >> > > > > > > > > written down before more mechanisms are added. >> > > > > > > > > >> > > > > > > > > Since Ayush, Yufei and you all favor asking upstream, I'll >> > > open the >> > > > > > > > > dev@iceberg thread now >> > > > > > > > > and link it here. I'll frame it as a clarification >> request: >> > > whether >> > > > > > "any >> > > > > > > > or >> > > > > > > > > none" permits >> > > > > > > > > rejecting the request, and whether an empty >> > storage-credentials >> > > > > > (and, in >> > > > > > > > > 1.12, >> > > > > > > > > remote-signing-config) is the intended signal when the >> server >> > > > > > returns 200 >> > > > > > > > > without delegation. >> > > > > > > > > >> > > > > > > > > The Polaris documentation side is done: >> > > > > > > > > - issue: https://github.com/apache/polaris/issues/5579 >> > > > > > > > > - PR: https://github.com/apache/polaris/pull/5591 >> > > > > > > > > >> > > > > > > > > Thanks, >> > > > > > > > > Youngrae Kim >> > > > > > > > > >> > > > > > > > > 2026년 9월 23일 (수) 오전 1:53, Alex Dutra <[email protected] >> >님이 >> > 작성: >> > > > > > > > > >> > > > > > > > > > And to complicate things further, there is a new field >> > > coming in >> > > > > > > > > > Iceberg 1.12: LoadTableResponse.remoteSigningConfig(). >> > > > > > > > > > >> > > > > > > > > > The idea is roughly the same as for the >> storage-credentials >> > > field: >> > > > > > > > > > clients should search for remote signing properties >> either >> > > in the >> > > > > > new >> > > > > > > > > > remote-signing-config field, or in the config field for >> > > backwards >> > > > > > > > > > compatibility with older servers. If the client >> explicitly >> > > > > > requested >> > > > > > > > > > remote signing, the absence of remote signing >> properties in >> > > both >> > > > > > > > > > places would in theory mean that the server didn't allow >> > the >> > > > > > client to >> > > > > > > > > > perform remote signing. >> > > > > > > > > > >> > > > > > > > > > But that doesn't change the overall feeling that a >> > fail-fast >> > > > > > solution >> > > > > > > > > > would be more appropriate, I think. >> > > > > > > > > > >> > > > > > > > > > Thanks, >> > > > > > > > > > Alex >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > On Tue, Sep 22, 2026 at 6:27 PM Alex Dutra < >> > > [email protected]> >> > > > > > wrote: >> > > > > > > > > > > >> > > > > > > > > > > Hi all, >> > > > > > > > > > > >> > > > > > > > > > > I'm also supportive of Option 1 (fail-fast), and also >> > > supportive >> > > > > > of >> > > > > > > > > > > asking the Iceberg community to provide a spec >> > > clarification. >> > > > > > > > > > > >> > > > > > > > > > > @Ayush: >> > > > > > > > > > > >> > > > > > > > > > > > The spec does have a signal for "delegation not >> > > applied": [...] >> > > > > > > > > > "Clients must first check whether the respective >> > credentials >> > > exist >> > > > > > in >> > > > > > > > the >> > > > > > > > > > storage-credentials field before checking the config for >> > > > > > credentials". >> > > > > > > > > > > >> > > > > > > > > > > I think this sentence is there for backwards >> > compatibility >> > > with >> > > > > > older >> > > > > > > > > > > clients that do not know about the (relatively recent) >> > > > > > > > > > > storage-credentials field. That's why Polaris vends >> > > credentials >> > > > > > in >> > > > > > > > > > > *both* places [1]; the "must first check" imperative >> is >> > > there >> > > > > > only to >> > > > > > > > > > > force clients to prefer the ones in the >> > storage-credentials >> > > > > > field. >> > > > > > > > > > > >> > > > > > > > > > > In any case, the absence of credentials in *both* >> places >> > > *could* >> > > > > > be >> > > > > > > > > > > interpreted as a signal that the server wasn't able to >> > > perform >> > > > > > access >> > > > > > > > > > > delegation – but again, failing fast appears to me as >> a >> > > much >> > > > > > cleaner >> > > > > > > > > > > behavior. >> > > > > > > > > > > >> > > > > > > > > > > Thanks, >> > > > > > > > > > > Alex >> > > > > > > > > > > >> > > > > > > > > > > [1]: >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > >> > >> https://github.com/apache/polaris/blob/e6aacf350fb7464f001f9778b9b678ed80709dcb/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java#L1246-L1250 >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > On Tue, Sep 22, 2026 at 8:00 AM youngrae kim < >> > > > > > [email protected] >> > > > > > > > > >> > > > > > > > > > wrote: >> > > > > > > > > > > > >> > > > > > > > > > > > Thanks Yufei. >> > > > > > > > > > > > >> > > > > > > > > > > > Agreed that the spec text is ambiguous and worth >> > > clarifying >> > > > > > > > upstream. >> > > > > > > > > > I'll >> > > > > > > > > > > > focus on the Polaris documentation first; if nobody >> > else >> > > picks >> > > > > > up >> > > > > > > > the >> > > > > > > > > > > > Iceberg-side question, I can raise it on dev@iceberg >> > > > > > afterwards >> > > > > > > > and >> > > > > > > > > > link >> > > > > > > > > > > > back to this thread. >> > > > > > > > > > > > >> > > > > > > > > > > > For the documentation, I'll open the Polaris issue >> as >> > > planned: >> > > > > > the >> > > > > > > > > > > > vended-credentials page and the S3 stsUnavailable >> > > section will >> > > > > > > > > describe >> > > > > > > > > > > > what a client sees when delegation cannot be >> satisfied >> > > (HTTP >> > > > > > 400 >> > > > > > > > and >> > > > > > > > > > the >> > > > > > > > > > > > two messages), and recommend omitting the header, or >> > > > > > configuring >> > > > > > > > > > storage >> > > > > > > > > > > > credentials on the client, for catalogs that cannot >> > vend. >> > > > > > > > > > > > >> > > > > > > > > > > > I'll post the link to the issue here once it is up. >> > > > > > > > > > > > >> > > > > > > > > > > > Thanks, >> > > > > > > > > > > > Youngrae Kim >> > > > > > > > > > > > >> > > > > > > > > > > > 2026년 9월 22일 (화) 오전 2:59, Yufei Gu < >> > [email protected] >> > > >님이 >> > > > > > 작성: >> > > > > > > > > > > > >> > > > > > > > > > > > > I support keeping fail-fast in Polaris and >> > documenting >> > > it. >> > > > > > > > > > > > > >> > > > > > > > > > > > > I’d still suggest a discussion in the Iceberg >> > > community. The >> > > > > > spec >> > > > > > > > > > > > > < >> > > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > >> > >> https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L2151-L2161 >> > > > > > > > > > > > > > >> > > > > > > > > > > > > says the server may provide “any or none” of the >> > > requested >> > > > > > access >> > > > > > > > > > > > > mechanisms, but it’s unclear whether that includes >> > > rejecting >> > > > > > the >> > > > > > > > > > request. >> > > > > > > > > > > > > Clarifying this would help clients work >> consistently >> > > across >> > > > > > > > > catalogs. >> > > > > > > > > > > > > >> > > > > > > > > > > > > This can happen alongside the Polaris >> documentation >> > > update. >> > > > > > > > > > > > > Yufei >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > On Fri, Sep 18, 2026 at 5:01 PM youngrae kim < >> > > > > > > > > [email protected] >> > > > > > > > > > > >> > > > > > > > > > > > > wrote: >> > > > > > > > > > > > > >> > > > > > > > > > > > > > Thanks Ayush and Dmitri. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Ayush, you're right about storage-credentials: >> the >> > > signal >> > > > > > is >> > > > > > > > > there, >> > > > > > > > > > > > > > and the real objection to option 2 is that >> clients >> > > aren't >> > > > > > > > > reliably >> > > > > > > > > > > > > checking >> > > > > > > > > > > > > > it. >> > > > > > > > > > > > > > And the point that both failure cases are static >> > > > > > properties of >> > > > > > > > > the >> > > > > > > > > > > > > catalog >> > > > > > > > > > > > > > is what settles it for me as well: >> > > > > > > > > > > > > > there is nothing a later request could do >> > > differently, so a >> > > > > > > > clear >> > > > > > > > > > error >> > > > > > > > > > > > > at >> > > > > > > > > > > > > > load time loses nothing. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > So the direction is option 1: keep fail-fast, >> and >> > > document >> > > > > > it. >> > > > > > > > > > > > > > Unless there are objections in the next couple >> of >> > > days, >> > > > > > I'll >> > > > > > > > open >> > > > > > > > > > an >> > > > > > > > > > > > > issue >> > > > > > > > > > > > > > to update the docs, >> > > > > > > > > > > > > > covering the vended-credentials-only and >> > > remote-signing >> > > > > > cases, >> > > > > > > > > > what the >> > > > > > > > > > > > > > client sees, >> > > > > > > > > > > > > > and the recommendation to omit the header (or >> > > configure >> > > > > > storage >> > > > > > > > > > > > > > credentials) for catalogs that cannot vend. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > On the Iceberg spec: I'll leave it aside for >> now, >> > > > > > > > > > > > > > given Dmitri's point that the current text >> already >> > > permits >> > > > > > > > > > refusing, >> > > > > > > > > > > > > > and revisit only if someone hits the ambiguity >> in >> > > practice. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Thanks, >> > > > > > > > > > > > > > Youngrae Kim >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > 2026년 9월 19일 (토) 오전 12:05, Dmitri Bourlatchkov < >> > > > > > > > [email protected] >> > > > > > > > > >님이 >> > > > > > > > > > 작성: >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Hi All, >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > I agree with option 1. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > The basic question is whether the client >> should >> > > use local >> > > > > > > > > > credentials >> > > > > > > > > > > > > or >> > > > > > > > > > > > > > > get vended credentials from Polaris. I do not >> > > think it >> > > > > > is a >> > > > > > > > > > runtime >> > > > > > > > > > > > > > choice. >> > > > > > > > > > > > > > > I believe it is a fundamental design decision >> > that >> > > > > > > > > administrators >> > > > > > > > > > > > > should >> > > > > > > > > > > > > > > resolve at deployment time. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Consequently, the client should either >> delegate >> > > access >> > > > > > > > controls >> > > > > > > > > > to >> > > > > > > > > > > > > > Polaris >> > > > > > > > > > > > > > > via X-Iceberg-Access-Delegation or not use >> that >> > > header at >> > > > > > > > all. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > From another angle, if the client already has >> > > access >> > > > > > > > > > credentials, what >> > > > > > > > > > > > > > > could be the rationale for also requesting >> > > different >> > > > > > access >> > > > > > > > > > channels >> > > > > > > > > > > > > > > via X-Iceberg-Access-Delegation? I do not see >> any >> > > :) >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > With that in mind, I think Polaris can >> interpret >> > > the >> > > > > > presence >> > > > > > > > > > > > > > > of X-Iceberg-Access-Delegation as a strong >> > request >> > > for >> > > > > > > > > > > > > server-controlled >> > > > > > > > > > > > > > > access methods (request signing or cred. >> > vending). >> > > If the >> > > > > > > > > server >> > > > > > > > > > cannot >> > > > > > > > > > > > > > > provide any, it is effectively a deployment / >> > > > > > configuration >> > > > > > > > > > mistake, >> > > > > > > > > > > > > so a >> > > > > > > > > > > > > > > clear error response is quite appropriate >> from my >> > > POV. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > General non-vending Polaris configuration is >> > > considered >> > > > > > > > > > deprecated per >> > > > > > > > > > > > > > > earlier discussion [1]. The only case when >> > Polaris >> > > cannot >> > > > > > > > vend >> > > > > > > > > > > > > > credentials >> > > > > > > > > > > > > > > is when the Storage System does not support >> it. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Re: Iceberg spec discussion, I'm not sure it >> is >> > > worth the >> > > > > > > > > > trouble. The >> > > > > > > > > > > > > > IRC >> > > > > > > > > > > > > > > spec clearly opted for the most lenient >> > > interpretation >> > > > > > > > > > > > > > > of X-Iceberg-Access-Delegation that does not >> > > impose any >> > > > > > > > strong >> > > > > > > > > > > > > > > protocol-level requirements on either the >> client >> > > or the >> > > > > > > > server. >> > > > > > > > > > I do >> > > > > > > > > > > > > not >> > > > > > > > > > > > > > > think it would be wise for the IRC spec to >> > venture >> > > into >> > > > > > > > > > > > > deployment-level >> > > > > > > > > > > > > > > recommendations because it defines a protocol, >> > not >> > > an >> > > > > > > > > end-to-end >> > > > > > > > > > > > > system. >> > > > > > > > > > > > > > I >> > > > > > > > > > > > > > > think the current IRC spec allows the >> behaviour >> > > proposed >> > > > > > by >> > > > > > > > > > option 1 in >> > > > > > > > > > > > > > > Polaris. This is just my personal opinion, >> feel >> > > free to >> > > > > > open >> > > > > > > > an >> > > > > > > > > > Iceberg >> > > > > > > > > > > > > > > discussion on this if you prefer. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > [1] >> > > > > > > > > > >> > > https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Cheers, >> > > > > > > > > > > > > > > Dmitri. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On Fri, Sep 18, 2026 at 7:40 AM Ayush Saxena < >> > > > > > > > > > [email protected]> >> > > > > > > > > > > > > > > wrote: >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Hi Youngrae, >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > +1 to option 1, with one correction to the >> > > premise. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > The spec does have a signal for "delegation >> not >> > > > > > applied": >> > > > > > > > > > > > > > LoadTableResult >> > > > > > > > > > > > > > > > carries storage-credentials, and clients are >> > > obliged to >> > > > > > > > look >> > > > > > > > > > at it — >> > > > > > > > > > > > > > > > "Clients must first check whether the >> > respective >> > > > > > > > credentials >> > > > > > > > > > exist in >> > > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > storage-credentials field before checking >> the >> > > config >> > > > > > for >> > > > > > > > > > credentials" >> > > > > > > > > > > > > > > [1]. >> > > > > > > > > > > > > > > > An empty storage-credentials on a 200 is >> > > observable. >> > > > > > The >> > > > > > > > > > practical >> > > > > > > > > > > > > > > > objection to option 2 is that "must check" >> > isn't >> > > "does >> > > > > > > > > check", >> > > > > > > > > > not >> > > > > > > > > > > > > that >> > > > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > information is missing. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > What decides it for me is that in Polaris >> both >> > > failure >> > > > > > > > cases >> > > > > > > > > > are >> > > > > > > > > > > > > static >> > > > > > > > > > > > > > > > properties of the catalog: stsUnavailable is >> > > > > > configuration >> > > > > > > > > and >> > > > > > > > > > remote >> > > > > > > > > > > > > > > > signing is unimplemented, so neither will >> start >> > > > > > working on >> > > > > > > > a >> > > > > > > > > > later >> > > > > > > > > > > > > > > request >> > > > > > > > > > > > > > > > or for a different table. The usual case for >> > > returning >> > > > > > 200 >> > > > > > > > > > without >> > > > > > > > > > > > > > > > delegation is that the server might satisfy >> it >> > > some >> > > > > > other >> > > > > > > > > time >> > > > > > > > > > — that >> > > > > > > > > > > > > > > > doesn't apply here, so failing at load with >> a >> > > message >> > > > > > > > naming >> > > > > > > > > > the >> > > > > > > > > > > > > reason >> > > > > > > > > > > > > > > is >> > > > > > > > > > > > > > > > strictly more diagnosable and nothing is >> lost. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Raising the ambiguity upstream sounds >> > worthwhile >> > > either >> > > > > > > > way — >> > > > > > > > > > "any or >> > > > > > > > > > > > > > > > none" doesn't say whether refusing the >> request >> > > is a >> > > > > > > > permitted >> > > > > > > > > > choice, >> > > > > > > > > > > > > > and >> > > > > > > > > > > > > > > > that's the real gap. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > -Ayush >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > [1] >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > >> > >> https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L4246-L4249 >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > On 2026/09/18 08:44:49 김영래 wrote: >> > > > > > > > > > > > > > > > > Hi all, >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > This came out of the review of PR #5473 >> > > (merged), >> > > > > > where >> > > > > > > > > > Dmitri >> > > > > > > > > > > > > > > > > suggested moving the spec-alignment >> question >> > > to the >> > > > > > list. >> > > > > > > > > > Short >> > > > > > > > > > > > > > > version: >> > > > > > > > > > > > > > > > > today Polaris fails a request whose >> > > > > > > > > > X-Iceberg-Access-Delegation >> > > > > > > > > > > > > > header >> > > > > > > > > > > > > > > > > cannot be satisfied, while the Iceberg >> REST >> > > spec >> > > > > > reads as >> > > > > > > > > if >> > > > > > > > > > the >> > > > > > > > > > > > > > server >> > > > > > > > > > > > > > > > may >> > > > > > > > > > > > > > > > > instead return the table without delegated >> > > access. >> > > > > > I'd >> > > > > > > > like >> > > > > > > > > > to >> > > > > > > > > > > > > agree >> > > > > > > > > > > > > > on >> > > > > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > > intended behavior and document it. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Current behavior (main, after #5473): >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > - "vended-credentials" against a catalog >> that >> > > cannot >> > > > > > vend >> > > > > > > > > > > > > credentials >> > > > > > > > > > > > > > > > (e.g. >> > > > > > > > > > > > > > > > > S3 with stsUnavailable=true) fails with >> 400 >> > > > > > "Credential >> > > > > > > > > > vending was >> > > > > > > > > > > > > > > > > requested ... but no credentials are >> > > available". >> > > > > > > > > > > > > > > > > - Anything that resolves to remote signing >> > (not >> > > > > > > > > implemented) >> > > > > > > > > > fails >> > > > > > > > > > > > > > with >> > > > > > > > > > > > > > > > 400 >> > > > > > > > > > > > > > > > > "This catalog cannot vend credentials or >> sign >> > > > > > requests; >> > > > > > > > > > request >> > > > > > > > > > > > > > without >> > > > > > > > > > > > > > > > > X-Iceberg-Access-Delegation and configure >> > > storage >> > > > > > > > > > credentials on >> > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > > client". This covers "remote-signing" >> alone >> > > > > > > > > > > > > > > > > and "vended-credentials,remote-signing" >> when >> > > vending >> > > > > > is >> > > > > > > > not >> > > > > > > > > > > > > possible. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > What the spec says >> > > > > > (iceberg-rest-catalog-open-api.yaml, >> > > > > > > > > > parameter >> > > > > > > > > > > > > > > > > X-Iceberg-Access-Delegation): >> > > > > > > > > > > > > > > > > "Optional signal to the server that >> the >> > > client >> > > > > > > > supports >> > > > > > > > > > > > > > > > > delegated access via a comma-separated >> list >> > of >> > > access >> > > > > > > > > > mechanisms. >> > > > > > > > > > > > > The >> > > > > > > > > > > > > > > > > server may choose to supply access via >> any or >> > > none >> > > > > > of the >> > > > > > > > > > requested >> > > > > > > > > > > > > > > > > mechanisms." >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Arguments raised in the PR review: >> > > > > > > > > > > > > > > > > - For fail-fast (current): a client that >> asks >> > > for >> > > > > > > > > delegation >> > > > > > > > > > > > > usually >> > > > > > > > > > > > > > > has >> > > > > > > > > > > > > > > > no >> > > > > > > > > > > > > > > > > storage credentials of its own, so a 200 >> > > without >> > > > > > > > > credentials >> > > > > > > > > > only >> > > > > > > > > > > > > > moves >> > > > > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > > failure to the first data access, where >> it is >> > > much >> > > > > > harder >> > > > > > > > > to >> > > > > > > > > > > > > > diagnose. >> > > > > > > > > > > > > > > It >> > > > > > > > > > > > > > > > > is also consistent with how >> > > "vended-credentials" >> > > > > > alone >> > > > > > > > has >> > > > > > > > > > always >> > > > > > > > > > > > > > > > behaved. >> > > > > > > > > > > > > > > > > - For following the spec literally: the >> > header >> > > is a >> > > > > > hint, >> > > > > > > > > and >> > > > > > > > > > > > > clients >> > > > > > > > > > > > > > > > that >> > > > > > > > > > > > > > > > > do have their own storage credentials >> > (instance >> > > > > > profile, >> > > > > > > > > > > > > environment >> > > > > > > > > > > > > > > > > variables) but send the header as a shared >> > > setting >> > > > > > would >> > > > > > > > > > simply >> > > > > > > > > > > > > work. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Options: >> > > > > > > > > > > > > > > > > 1. Keep fail-fast and document it >> explicitly. >> > > The >> > > > > > docs >> > > > > > > > > > currently >> > > > > > > > > > > > > only >> > > > > > > > > > > > > > > say >> > > > > > > > > > > > > > > > > to omit the header for stsUnavailable >> > catalogs. >> > > > > > > > > > > > > > > > > 2. Follow the spec literally and return >> the >> > > table >> > > > > > without >> > > > > > > > > > delegated >> > > > > > > > > > > > > > > > access. >> > > > > > > > > > > > > > > > > The spec has no field to signal >> "delegation >> > not >> > > > > > applied", >> > > > > > > > > so >> > > > > > > > > > > > > clients >> > > > > > > > > > > > > > > > would >> > > > > > > > > > > > > > > > > only find out at data access time. >> > > > > > > > > > > > > > > > > 3. Make it configurable per realm or >> catalog >> > > (feature >> > > > > > > > flag, >> > > > > > > > > > default >> > > > > > > > > > > > > > > > > fail-fast) for deployments that know their >> > > clients >> > > > > > carry >> > > > > > > > > > storage >> > > > > > > > > > > > > > > > > credentials. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > I lean towards option 1 unless someone >> has a >> > > concrete >> > > > > > > > > client >> > > > > > > > > > that >> > > > > > > > > > > > > > > depends >> > > > > > > > > > > > > > > > > on option 2. If there is interest, I can >> also >> > > raise >> > > > > > the >> > > > > > > > > > ambiguity >> > > > > > > > > > > > > on >> > > > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > > Iceberg side, since the spec text leaves >> the >> > > choice >> > > > > > to >> > > > > > > > the >> > > > > > > > > > server. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Context: issue #5472, PR #5473 (see the >> > > CHANGELOG >> > > > > > review >> > > > > > > > > > thread), >> > > > > > > > > > > > > and >> > > > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > > earlier threads "[DISCUSS] S3 Credential >> > > vending >> > > > > > without >> > > > > > > > > > STS" (July >> > > > > > > > > > > > > > > 2025) >> > > > > > > > > > > > > > > > > and "[DISCUSS] S3 Remote Signing" (August >> > > 2025). >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Thanks, >> > > > > > > > > > > > > > > > > Youngrae Kim >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > >> > >> >
