Hi all, I agree with failing the request when the server can supply none of the requested mechanisms, and with Alex's SHOULD.
In Polaris the two cases we discussed are static properties of the catalog. stsUnavailable will not start working on a later load, and remote signing is not implemented. A 200 with no delegated access only moves the failure to the first data access, where it is much harder to diagnose. We kept the 400 and wrote down both messages, and what the client should change, in https://github.com/apache/polaris/pull/5591. SHOULD is the right strength for the spec. It records the behavior operators can diagnose, and a server that knows its clients carry storage credentials can still return the table. Polaris does not take that path. The header is often a shared client setting: PyIceberg documents vended-credentials as the default ( https://py.iceberg.apache.org/configuration/), so a client can send it without anyone having decided that this catalog should vend. The 400 sentence is the clarification this thread needs. Whether one response may carry more than one mechanism is a separate change. Polaris selects a single mechanism, and that can stay as it is until a server returns both. HTTP 400 with the existing Bad Request error is enough. Clients already fail the load on it. The message is what tells the operator which mechanism was missing, and to drop the header or configure storage credentials on the client. A distinct error type can wait until a client needs to branch on this case in code. Thanks, Prithvi S On Wed, Sep 23, 2026 at 2:11 PM youngrae kim <[email protected]> wrote: > Thanks Alex, that's helpful context. The backwards-compatibility reading of > "must first check" > makes sense, and the upcoming remote-signing-config field in Iceberg 1.12 > following the > same dual-location pattern is a good reason to get the "delegation not > provided" semantics > written down before more mechanisms are added. > > Since Ayush, Yufei and you all favor asking upstream, I'll open the > dev@iceberg thread now > and link it here. I'll frame it as a clarification request: whether "any or > none" permits > rejecting the request, and whether an empty storage-credentials (and, in > 1.12, > remote-signing-config) is the intended signal when the server returns 200 > without delegation. > > The Polaris documentation side is done: > - issue: https://github.com/apache/polaris/issues/5579 > - PR: https://github.com/apache/polaris/pull/5591 > > Thanks, > Youngrae Kim > > 2026년 9월 23일 (수) 오전 1:53, Alex Dutra <[email protected]>님이 작성: > > > And to complicate things further, there is a new field coming in > > Iceberg 1.12: LoadTableResponse.remoteSigningConfig(). > > > > The idea is roughly the same as for the storage-credentials field: > > clients should search for remote signing properties either in the new > > remote-signing-config field, or in the config field for backwards > > compatibility with older servers. If the client explicitly requested > > remote signing, the absence of remote signing properties in both > > places would in theory mean that the server didn't allow the client to > > perform remote signing. > > > > But that doesn't change the overall feeling that a fail-fast solution > > would be more appropriate, I think. > > > > Thanks, > > Alex > > > > > > On Tue, Sep 22, 2026 at 6:27 PM Alex Dutra <[email protected]> wrote: > > > > > > Hi all, > > > > > > I'm also supportive of Option 1 (fail-fast), and also supportive of > > > asking the Iceberg community to provide a spec clarification. > > > > > > @Ayush: > > > > > > > The spec does have a signal for "delegation not applied": [...] > > "Clients must first check whether the respective credentials exist in the > > storage-credentials field before checking the config for credentials". > > > > > > I think this sentence is there for backwards compatibility with older > > > clients that do not know about the (relatively recent) > > > storage-credentials field. That's why Polaris vends credentials in > > > *both* places [1]; the "must first check" imperative is there only to > > > force clients to prefer the ones in the storage-credentials field. > > > > > > In any case, the absence of credentials in *both* places *could* be > > > interpreted as a signal that the server wasn't able to perform access > > > delegation – but again, failing fast appears to me as a much cleaner > > > behavior. > > > > > > Thanks, > > > Alex > > > > > > [1]: > > > https://github.com/apache/polaris/blob/e6aacf350fb7464f001f9778b9b678ed80709dcb/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java#L1246-L1250 > > > > > > > > > On Tue, Sep 22, 2026 at 8:00 AM youngrae kim <[email protected]> > > wrote: > > > > > > > > Thanks Yufei. > > > > > > > > Agreed that the spec text is ambiguous and worth clarifying upstream. > > I'll > > > > focus on the Polaris documentation first; if nobody else picks up the > > > > Iceberg-side question, I can raise it on dev@iceberg afterwards and > > link > > > > back to this thread. > > > > > > > > For the documentation, I'll open the Polaris issue as planned: the > > > > vended-credentials page and the S3 stsUnavailable section will > describe > > > > what a client sees when delegation cannot be satisfied (HTTP 400 and > > the > > > > two messages), and recommend omitting the header, or configuring > > storage > > > > credentials on the client, for catalogs that cannot vend. > > > > > > > > I'll post the link to the issue here once it is up. > > > > > > > > Thanks, > > > > Youngrae Kim > > > > > > > > 2026년 9월 22일 (화) 오전 2:59, Yufei Gu <[email protected]>님이 작성: > > > > > > > > > I support keeping fail-fast in Polaris and documenting it. > > > > > > > > > > I’d still suggest a discussion in the Iceberg community. The spec > > > > > < > > > > > > > > https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L2151-L2161 > > > > > > > > > > > says the server may provide “any or none” of the requested access > > > > > mechanisms, but it’s unclear whether that includes rejecting the > > request. > > > > > Clarifying this would help clients work consistently across > catalogs. > > > > > > > > > > This can happen alongside the Polaris documentation update. > > > > > Yufei > > > > > > > > > > > > > > > On Fri, Sep 18, 2026 at 5:01 PM youngrae kim < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > Thanks Ayush and Dmitri. > > > > > > > > > > > > Ayush, you're right about storage-credentials: the signal is > there, > > > > > > and the real objection to option 2 is that clients aren't > reliably > > > > > checking > > > > > > it. > > > > > > And the point that both failure cases are static properties of > the > > > > > catalog > > > > > > is what settles it for me as well: > > > > > > there is nothing a later request could do differently, so a clear > > error > > > > > at > > > > > > load time loses nothing. > > > > > > > > > > > > So the direction is option 1: keep fail-fast, and document it. > > > > > > Unless there are objections in the next couple of days, I'll open > > an > > > > > issue > > > > > > to update the docs, > > > > > > covering the vended-credentials-only and remote-signing cases, > > what the > > > > > > client sees, > > > > > > and the recommendation to omit the header (or configure storage > > > > > > credentials) for catalogs that cannot vend. > > > > > > > > > > > > On the Iceberg spec: I'll leave it aside for now, > > > > > > given Dmitri's point that the current text already permits > > refusing, > > > > > > and revisit only if someone hits the ambiguity in practice. > > > > > > > > > > > > Thanks, > > > > > > Youngrae Kim > > > > > > > > > > > > 2026년 9월 19일 (토) 오전 12:05, Dmitri Bourlatchkov <[email protected] > >님이 > > 작성: > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > I agree with option 1. > > > > > > > > > > > > > > The basic question is whether the client should use local > > credentials > > > > > or > > > > > > > get vended credentials from Polaris. I do not think it is a > > runtime > > > > > > choice. > > > > > > > I believe it is a fundamental design decision that > administrators > > > > > should > > > > > > > resolve at deployment time. > > > > > > > > > > > > > > Consequently, the client should either delegate access controls > > to > > > > > > Polaris > > > > > > > via X-Iceberg-Access-Delegation or not use that header at all. > > > > > > > > > > > > > > From another angle, if the client already has access > > credentials, what > > > > > > > could be the rationale for also requesting different access > > channels > > > > > > > via X-Iceberg-Access-Delegation? I do not see any :) > > > > > > > > > > > > > > With that in mind, I think Polaris can interpret the presence > > > > > > > of X-Iceberg-Access-Delegation as a strong request for > > > > > server-controlled > > > > > > > access methods (request signing or cred. vending). If the > server > > cannot > > > > > > > provide any, it is effectively a deployment / configuration > > mistake, > > > > > so a > > > > > > > clear error response is quite appropriate from my POV. > > > > > > > > > > > > > > General non-vending Polaris configuration is considered > > deprecated per > > > > > > > earlier discussion [1]. The only case when Polaris cannot vend > > > > > > credentials > > > > > > > is when the Storage System does not support it. > > > > > > > > > > > > > > Re: Iceberg spec discussion, I'm not sure it is worth the > > trouble. The > > > > > > IRC > > > > > > > spec clearly opted for the most lenient interpretation > > > > > > > of X-Iceberg-Access-Delegation that does not impose any strong > > > > > > > protocol-level requirements on either the client or the server. > > I do > > > > > not > > > > > > > think it would be wise for the IRC spec to venture into > > > > > deployment-level > > > > > > > recommendations because it defines a protocol, not an > end-to-end > > > > > system. > > > > > > I > > > > > > > think the current IRC spec allows the behaviour proposed by > > option 1 in > > > > > > > Polaris. This is just my personal opinion, feel free to open an > > Iceberg > > > > > > > discussion on this if you prefer. > > > > > > > > > > > > > > [1] > > https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg > > > > > > > > > > > > > > Cheers, > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 18, 2026 at 7:40 AM Ayush Saxena < > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Youngrae, > > > > > > > > > > > > > > > > +1 to option 1, with one correction to the premise. > > > > > > > > > > > > > > > > The spec does have a signal for "delegation not applied": > > > > > > LoadTableResult > > > > > > > > carries storage-credentials, and clients are obliged to look > > at it — > > > > > > > > "Clients must first check whether the respective credentials > > exist in > > > > > > the > > > > > > > > storage-credentials field before checking the config for > > credentials" > > > > > > > [1]. > > > > > > > > An empty storage-credentials on a 200 is observable. The > > practical > > > > > > > > objection to option 2 is that "must check" isn't "does > check", > > not > > > > > that > > > > > > > the > > > > > > > > information is missing. > > > > > > > > > > > > > > > > What decides it for me is that in Polaris both failure cases > > are > > > > > static > > > > > > > > properties of the catalog: stsUnavailable is configuration > and > > remote > > > > > > > > signing is unimplemented, so neither will start working on a > > later > > > > > > > request > > > > > > > > or for a different table. The usual case for returning 200 > > without > > > > > > > > delegation is that the server might satisfy it some other > time > > — that > > > > > > > > doesn't apply here, so failing at load with a message naming > > the > > > > > reason > > > > > > > is > > > > > > > > strictly more diagnosable and nothing is lost. > > > > > > > > > > > > > > > > Raising the ambiguity upstream sounds worthwhile either way — > > "any or > > > > > > > > none" doesn't say whether refusing the request is a permitted > > choice, > > > > > > and > > > > > > > > that's the real gap. > > > > > > > > > > > > > > > > -Ayush > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L4246-L4249 > > > > > > > > > > > > > > > > > > > > > > > > On 2026/09/18 08:44:49 김영래 wrote: > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > This came out of the review of PR #5473 (merged), where > > Dmitri > > > > > > > > > suggested moving the spec-alignment question to the list. > > Short > > > > > > > version: > > > > > > > > > today Polaris fails a request whose > > X-Iceberg-Access-Delegation > > > > > > header > > > > > > > > > cannot be satisfied, while the Iceberg REST spec reads as > if > > the > > > > > > server > > > > > > > > may > > > > > > > > > instead return the table without delegated access. I'd like > > to > > > > > agree > > > > > > on > > > > > > > > the > > > > > > > > > intended behavior and document it. > > > > > > > > > > > > > > > > > > Current behavior (main, after #5473): > > > > > > > > > > > > > > > > > > - "vended-credentials" against a catalog that cannot vend > > > > > credentials > > > > > > > > (e.g. > > > > > > > > > S3 with stsUnavailable=true) fails with 400 "Credential > > vending was > > > > > > > > > requested ... but no credentials are available". > > > > > > > > > - Anything that resolves to remote signing (not > implemented) > > fails > > > > > > with > > > > > > > > 400 > > > > > > > > > "This catalog cannot vend credentials or sign requests; > > request > > > > > > without > > > > > > > > > X-Iceberg-Access-Delegation and configure storage > > credentials on > > > > > the > > > > > > > > > client". This covers "remote-signing" alone > > > > > > > > > and "vended-credentials,remote-signing" when vending is not > > > > > possible. > > > > > > > > > > > > > > > > > > What the spec says (iceberg-rest-catalog-open-api.yaml, > > parameter > > > > > > > > > X-Iceberg-Access-Delegation): > > > > > > > > > "Optional signal to the server that the client supports > > > > > > > > > delegated access via a comma-separated list of access > > mechanisms. > > > > > The > > > > > > > > > server may choose to supply access via any or none of the > > requested > > > > > > > > > mechanisms." > > > > > > > > > > > > > > > > > > Arguments raised in the PR review: > > > > > > > > > - For fail-fast (current): a client that asks for > delegation > > > > > usually > > > > > > > has > > > > > > > > no > > > > > > > > > storage credentials of its own, so a 200 without > credentials > > only > > > > > > moves > > > > > > > > the > > > > > > > > > failure to the first data access, where it is much harder > to > > > > > > diagnose. > > > > > > > It > > > > > > > > > is also consistent with how "vended-credentials" alone has > > always > > > > > > > > behaved. > > > > > > > > > - For following the spec literally: the header is a hint, > and > > > > > clients > > > > > > > > that > > > > > > > > > do have their own storage credentials (instance profile, > > > > > environment > > > > > > > > > variables) but send the header as a shared setting would > > simply > > > > > work. > > > > > > > > > > > > > > > > > > Options: > > > > > > > > > 1. Keep fail-fast and document it explicitly. The docs > > currently > > > > > only > > > > > > > say > > > > > > > > > to omit the header for stsUnavailable catalogs. > > > > > > > > > 2. Follow the spec literally and return the table without > > delegated > > > > > > > > access. > > > > > > > > > The spec has no field to signal "delegation not applied", > so > > > > > clients > > > > > > > > would > > > > > > > > > only find out at data access time. > > > > > > > > > 3. Make it configurable per realm or catalog (feature flag, > > default > > > > > > > > > fail-fast) for deployments that know their clients carry > > storage > > > > > > > > > credentials. > > > > > > > > > > > > > > > > > > I lean towards option 1 unless someone has a concrete > client > > that > > > > > > > depends > > > > > > > > > on option 2. If there is interest, I can also raise the > > ambiguity > > > > > on > > > > > > > the > > > > > > > > > Iceberg side, since the spec text leaves the choice to the > > server. > > > > > > > > > > > > > > > > > > Context: issue #5472, PR #5473 (see the CHANGELOG review > > thread), > > > > > and > > > > > > > the > > > > > > > > > earlier threads "[DISCUSS] S3 Credential vending without > > STS" (July > > > > > > > 2025) > > > > > > > > > and "[DISCUSS] S3 Remote Signing" (August 2025). > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Youngrae Kim > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
