Thanks Dan and Alex. Dan, that background is helpful, and I take the point: with storage-credentials and remote-signing-config the catalog can state what it provides, so the spec does not need to mandate a failure when the requested mechanisms cannot be provided. Polaris will keep its current behavior as an implementation choice, documented as such, and I will report that outcome on the Polaris list.
The one thing I would still find useful is Alex's narrower point: a sentence saying whether the header is a capability statement or a requirement, and what an absent header means. That does not constrain what the catalog decides to provide; it only tells servers and clients how to read the header. If there is appetite for that I am happy to draft it; otherwise I am fine leaving the wording as is and letting the catalog-driven model evolve with the File type work. Thanks, Youngrae Kim 2026년 9월 29일 (화) 오전 6:28, Daniel Weeks <[email protected]>님이 작성: > I'm not sure I agree with the model of a client bringing its own > credentials if the catalog is also responsible for controlling access > (can't really stop people from using that model, but as I said above, it > feels like an anti-pattern). > > The current wording in the spec was intentionally loose so that the > catalog can make decisions without being forced into specific > security/access decisions. I still don't think that updating the wording > would be flexible enough. > > I'm not sure what all catalogs do, but I can say that it's always safe for > the catalog to return credentials/remote config even if the client was > externally configured to use its own. The current reference implementation > bypasses that, I believe, but conceptually a client does not need to use > the mechanism provided by the catalog. This again feels like a bit of an > anti-pattern: if there's a reason to bring sidecar credentials, it implies > a gap in the delegation model that we should be looking into addressing. > > Dan > > On Mon, Sep 28, 2026 at 10:00 AM Alex Dutra <[email protected]> wrote: > >> Hi all, >> >> > The delegation header intended to allow the client to communicate its >> capabilities, but we've aligned more clearly on catalogs controlling access >> as opposed to clients defining behavior. >> >> IMO, this header isn't ideal for communicating client capabilities. >> Notably, it fails to represent a key capability: that of using the >> client's own storage credentials to bypass access delegation entirely. >> Right now, the only way to imply this capability is by omitting the >> header altogether, which introduces ambiguity into the specification. >> Put simply, a client cannot explicitly communicate e.g.: "I can >> function either without access delegation or using vended credentials, >> but I do not support remote signing." >> >> Therefore, Apache Polaris today treats this header as an explicit >> requirement rather than just a capability statement. IOW, it treats >> the header presence as an indication that the client cannot function >> without access delegation, and that at least one specified mode must >> be satisfied. >> >> I would be interested to hear how other catalog implementations are >> approaching this. >> >> In any case, I still think that a minor spec clarification would be >> welcome, if for nothing else, at least to lift the ambiguity around >> what it means when the header is absent and clarify whether the header >> is a capability statement, or a requirement. >> >> Thanks, >> Alex >> >> >> On Mon, Sep 28, 2026 at 5:03 PM Daniel Weeks <[email protected]> wrote: >> > >> > Youngrae, >> > >> > I think the issue with #2 above is that it's a bit of an anti-pattern >> for the client to concurrently own access. Either the catalog should be >> the authority on access or it shouldn't. Security around the physical data >> shouldn't really be a negotiation between the client and the server. If >> the catalog is delegating access to the data, it should be the sole source >> of truth. >> > >> > It feels like we're trying to make this a negotiation between the >> catalog and client, but the catalog should be responsible for access >> delegation in a secure environment. >> > >> > The background on this header is important: The original access >> control mechanism was remote signing only, which was not originally part of >> the REST protocol (it was separate and S3 specific). We later introduced >> vended credentials, but only through the config payload. Since then, a lot >> has changed: we introduced the Storage Credentials and now the Remote >> Signing Config, which means that we're in a much better position for the >> catalog to clearly indicate which mechanism the client should use. The >> mechanisms are more standardized, and clients should be able to report if >> they cannot support what the catalog has provided. >> > >> > While there may be some residual value in the client signaling its >> capabilities, I'm not convinced we need to change the wording in the spec >> and we need to improve the other mechanisms as we evolve to support new >> functionality like File type, which may mix different modes of access. >> > >> > The behavior you describe in Polaris is perfectly fine as an >> implementation choice, but I don't think catalogs SHOULD fail if they >> cannot provide any requested mechanism. The catalog can provide an >> alternative (e.g. you requested signing, but I'm only providing >> credentials), which the client can error on if it does not understand. >> Alternatively, the catalog could reject all requested forms and return only >> the metadata, effectively signaling that the client can only see the >> metadata, but has no access to the physical data. >> > >> > -Dan >> > >> > >> > On Fri, Sep 25, 2026 at 8:18 PM youngrae kim <[email protected]> >> wrote: >> >> >> >> Thanks Alex, Yufei and Dan. >> >> >> >> Dan, I think the catalog-controlled direction and Alex's wording are >> compatible, and it may help to separate the two roles the header plays >> today: >> >> >> >> 1. Which mechanism to use. >> >> That is already the catalog's decision in Polaris: the server picks >> among the mechanisms >> >> it supports for that catalog and storage, and the header only tells >> it which ones the client can consume. >> >> Nothing in the proposed sentence changes that. >> >> >> >> 2. Whether the client can access the data at all without delegation. >> Today the header is the >> >> only signal a client has for "I have no storage credentials of my own". >> >> If the catalog cannot provide any mechanism the client can consume, >> the request cannot succeed in any >> >> useful sense, and the question is only where the failure surfaces: at >> load time with a >> >> message naming the cause, or at the first data access with a storage >> error. >> >> >> >> The proposed SHOULD covers exactly the empty-intersection case in (2); >> a server that knows its >> >> clients carry their own credentials can still return the table, which >> is why SHOULD rather than MUST seems right. >> >> >> >> If the longer-term direction is to rely less on the header, I would be >> interested in how a >> >> catalog-driven model would express (2), for example the client >> learning from /v1/config which >> >> mechanisms the catalog offers, so it can decide before loading a >> table. That seems like a >> >> larger change than the clarification this thread started with, so I >> would suggest landing the >> >> one-sentence clarification for the current header and taking the >> catalog-driven model as a >> >> separate discussion, unless you see a conflict between the two. >> >> >> >> Thanks, >> >> Youngrae Kim >> >> >> >> 2026년 9월 26일 (토) 오전 8:52, Daniel Weeks <[email protected]>님이 작성: >> >>> >> >>> I'm not convinced that we're thinking about this the right way. >> >>> >> >>> The delegation header intended to allow the client to communicate its >> capabilities, but we've aligned more clearly on catalogs controlling access >> as opposed to clients defining behavior. >> >>> >> >>> If we leaned more into that direction, then access would be >> determined by the catalog based on the availability of vented credentials >> or the new remote signing configuration. >> >>> >> >>> There may be an opportunity here to rely less on the header and give >> responsibility back on th e catalog to determine the access model. >> >>> >> >>> -Dan >> >>> >> >>> On Fri, Sep 25, 2026, 4:39 PM Yufei Gu <[email protected]> wrote: >> >>>> >> >>>> I tend to agree with Alex and Youngrae. A fail-fast approach >> provides a clear behavioral contract between the server and clients, while >> the current spec introduces ambiguity. >> >>>> >> >>>> I am not sure if this ambiguity was intentional. Please chime in if >> you can provide more context; otherwise, I suggest we update the spec to >> eliminate it. >> >>>> >> >>>> Yufei >> >>>> >> >>>> >> >>>> On Thu, Sep 24, 2026 at 10:12 AM Alex Dutra <[email protected]> >> wrote: >> >>>>> >> >>>>> Hi Youngrae, >> >>>>> >> >>>>> Thanks for starting this thread! >> >>>>> >> >>>>> I believe the phrase "via any or none of the requested mechanisms" >> >>>>> requires clarification. >> >>>>> >> >>>>> If a client requests modes A and B, a server returning only mode B >> is >> >>>>> perfectly acceptable. However, returning a 200 OK response with >> >>>>> neither mode A nor B represents an ambiguous outcome. >> >>>>> >> >>>>> The current wording likely assumes the client could theoretically >> >>>>> proceed using its own fallback storage credentials. In practice, >> >>>>> operators configure clients intentionally; requesting access >> >>>>> delegation "just in case" is unrealistic. A fail-fast error is a >> >>>>> better outcome, as opposed to letting the client proceed, only to >> see >> >>>>> it fail later on. >> >>>>> >> >>>>> I propose updating the specification to mandate an error response >> when >> >>>>> no requested mechanism is available, while also clarifying whether >> >>>>> providing multiple delegation modes is supported: >> >>>>> >> >>>>> "The server MAY supply access via any or all of the requested >> >>>>> mechanisms. If none of the requested mechanisms is available, the >> >>>>> server SHOULD return an HTTP 400 (Bad Request) response." >> >>>>> >> >>>>> Thanks, >> >>>>> Alex >> >>>>> >> >>>>> On Wed, Sep 23, 2026 at 11:14 AM youngrae kim < >> [email protected]> wrote: >> >>>>> > >> >>>>> > Hi all, >> >>>>> > The REST catalog spec describes X-Iceberg-Access-Delegation as an >> "optional signal to the server that the client supports delegated access >> via a comma-separated list of access mechanisms. The server may choose to >> supply access via any or none of the requested mechanisms." [1] >> >>>>> > >> >>>>> > In Apache Polaris we recently discussed what a server should do >> when the header is present but it can satisfy none of the requested >> mechanisms, for example vended-credentials against an S3-compatible store >> without STS, or remote-signing on a server that does not implement it [2]. >> >>>>> > >> >>>>> > Polaris fails such requests with HTTP 400 and a message naming >> the reason, because in these cases the inability is a static property of >> the catalog and a 200 without credentials would only move the failure to >> the first data access. >> >>>>> > >> >>>>> > Two readings of "any or none" came up and the text does not >> settle which is intended: >> >>>>> > (a) "none" means the server returns 200 with the table and no >> delegated access, and the client detects this from the response: an empty >> storage-credentials (and, we understand, an empty remote-signing-config >> once that field lands in 1.12), falling back to the config map for older >> servers [3]. >> >>>>> > >> >>>>> > (b) "none" also permits the server to reject the request when >> it cannot provide any of the requested mechanisms. >> >>>>> > >> >>>>> > Questions for the community: >> >>>>> > 1. Is rejecting the request a permitted server choice under the >> current text, or is (a) the only intended behavior? >> >>>>> > 2. If rejection is permitted, is there a preferred status code >> and error type, so clients can distinguish "delegation unavailable" from >> other 400s? >> >>>>> > 3. If (a) is the intended behavior, should the spec state >> explicitly that the absence of credentials in both storage-credentials and >> config (and of remote-signing-config) is the signal that no delegation was >> applied? >> >>>>> > >> >>>>> > I'm happy to draft a clarification to the parameter description >> once there is agreement. >> >>>>> > >> >>>>> > [1] >> https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L2151-L2161 >> >>>>> > [2] >> https://lists.apache.org/thread/9oyv4v0vk3t8stv9832npl8t0yyp4j8r >> >>>>> > [3] >> https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L4246-L4249 >> >>>>> > >> >>>>> > Thanks, >> >>>>> > Youngrae Kim >> >
