Hi Yufei, Thank you for raising this again. I agree it would be valuable for future readers to find the outcome of this discussion in the spec itself rather than having to search the mailing list.
To keep #18480 focused on the part we have already reached agreement on, I would prefer to leave it limited to adding "none". If Dan and Dmitri are comfortable with stating explicitly that a server may reject a request it cannot satisfy, for a specific reason, I would be glad to open a separate PR with that wording so it can be reviewed on its own. Thanks again for helping move this forward. Best regards, Youngrae Kim 2026년 10월 22일 (목) 오전 4:36, Yufei Gu <[email protected]>님이 작성: > A server may fail a request, but it should have a specific reason to > do so rather than failing randomly. It would be much clearer if the > specification explicitly stated this. > > Since this discussion started from a question about how to interpret > the spec, documenting our consensus directly in the spec would benefit > future readers. Otherwise, people will have to search the mailing list > or reach out for help to get clarity. WDYT? > > Yufei > > On Tue, Oct 13, 2026 at 12:43 AM youngrae kim <[email protected]> > wrote: > > > > The spec PR is up: https://github.com/apache/iceberg/pull/18480 > > > > It adds "none" to the enum and one sentence defining it; the rest of the > header description is unchanged. > > > > Youngrae Kim > > > > 2026년 10월 13일 (화) 오후 4:16, youngrae kim <[email protected]>님이 작성: > >> > >> Sounds good. > >> I will open a spec PR that only adds "none" to the > X-Iceberg-Access-Delegation enum and leaves the rest of the description as > is. > >> > >> Youngrae Kim > >> > >> 2026년 10월 13일 (화) 오전 7:39, Dmitri Bourlatchkov <[email protected]>님이 작성: > >>> > >>> Hi All, > >>> > >>> I agree that we do not need any specific statement on failing requests > in this case. > >>> > >>> Cheers, > >>> Dmitri. > >>> > >>> On Mon, Oct 12, 2026 at 4:12 PM Daniel Weeks <[email protected]> > wrote: > >>>> > >>>> I think we only need to add #1 above. > >>>> > >>>> For #2, a server may always reject/fail a request. I don't think we > need to clarify that point. > >>>> > >>>> On Mon, Oct 12, 2026 at 12:46 AM youngrae kim <[email protected]> > wrote: > >>>>> > >>>>> Thanks everyone. Summarizing where the thread landed: > >>>>> > >>>>> 1. Add "none" to X-Iceberg-Access-Delegation, meaning the client > does not need delegated access from the catalog (Dan's suggestion, +1 from > Yufei and Dmitri). > >>>>> 2. Clarify the header description so that failing the request is > explicitly allowed: "The server may choose to supply access via any or none > of the requested mechanisms, or fail the request." > >>>>> > >>>>> Unless someone objects, I am happy to open a spec PR with these two > changes so the discussion can continue on concrete wording. > >>>>> > >>>>> Youngrae Kim > >>>>> > >>>>> 2026년 10월 8일 (목) 오전 8:03, Dmitri Bourlatchkov <[email protected]>님이 > 작성: > >>>>>> > >>>>>> Hi Alex, Dan, > >>>>>> > >>>>>> Thanks for proposing this change. "None" sounds like a good idea to > me. It is certainly clearer than "local" in the context of > X-Iceberg-Access-Delegation. > >>>>>> > >>>>>> Cheers, > >>>>>> Dmitri. > >>>>>> > >>>>>> On Mon, Oct 5, 2026 at 4:18 PM Daniel Weeks <[email protected]> > wrote: > >>>>>>> > >>>>>>> Alex, > >>>>>>> > >>>>>>> Rather than 'local', I think I would prefer 'none'. Rather than > saying what the client is intending to do, express that the client isn't > asking for or doesn't need credentials/access from the catalog. > >>>>>>> > >>>>>>> At least then, the catalog could skip the generation. > >>>>>>> > >>>>>>> -Dan > >>>>>>> > >>>>>>> On Mon, Oct 5, 2026 at 12:51 AM Alex Dutra <[email protected]> > wrote: > >>>>>>>> > >>>>>>>> Hi all, > >>>>>>>> > >>>>>>>> How about at least introducing a new enum constant "local" in the > >>>>>>>> X-Iceberg-Access-Delegation header? If the intent of that header > was > >>>>>>>> to express capabilities, this new constant would convey the > notion of > >>>>>>>> "I can work with my own credentials" more firmly than letting the > >>>>>>>> server second-guess. E.g.: > "local,remote-signing,vended-credentials" > >>>>>>>> would mean the client can either work with vended credentials, > remote > >>>>>>>> signing, or its own credentials. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Alex > >>>>>>>> > >>>>>>>> On Tue, Sep 29, 2026 at 4:08 AM youngrae kim < > [email protected]> wrote: > >>>>>>>> > > >>>>>>>> > Thanks Dan and Alex. > >>>>>>>> > > >>>>>>>> > Dan, that background is helpful, and I take the point: with > storage-credentials and remote-signing-config > >>>>>>>> > the catalog can state what it provides, so the spec does not > need to mandate a failure when the requested > >>>>>>>> > mechanisms cannot be provided. Polaris will keep its current > behavior as an implementation choice, > >>>>>>>> > documented as such, and I will report that outcome on the > Polaris list. > >>>>>>>> > > >>>>>>>> > The one thing I would still find useful is Alex's narrower > point: a sentence saying whether the > >>>>>>>> > header is a capability statement or a requirement, and what an > absent header means. > >>>>>>>> > That does not constrain what the catalog decides to provide; it > only tells servers and clients how to read the header. > >>>>>>>> > If there is appetite for that I am happy to draft it; otherwise > I am fine leaving the wording as is and > >>>>>>>> > letting the catalog-driven model evolve with the File type work. > >>>>>>>> > > >>>>>>>> > Thanks, > >>>>>>>> > Youngrae Kim > >>>>>>>> > > >>>>>>>> > 2026년 9월 29일 (화) 오전 6:28, Daniel Weeks <[email protected]>님이 > 작성: > >>>>>>>> >> > >>>>>>>> >> I'm not sure I agree with the model of a client bringing its > own credentials if the catalog is also responsible for controlling access > (can't really stop people from using that model, but as I said above, it > feels like an anti-pattern). > >>>>>>>> >> > >>>>>>>> >> The current wording in the spec was intentionally loose so > that the catalog can make decisions without being forced into specific > security/access decisions. I still don't think that updating the wording > would be flexible enough. > >>>>>>>> >> > >>>>>>>> >> I'm not sure what all catalogs do, but I can say that it's > always safe for the catalog to return credentials/remote config even if the > client was externally configured to use its own. The current reference > implementation bypasses that, I believe, but conceptually a client does not > need to use the mechanism provided by the catalog. This again feels like a > bit of an anti-pattern: if there's a reason to bring sidecar credentials, > it implies a gap in the delegation model that we should be looking into > addressing. > >>>>>>>> >> > >>>>>>>> >> Dan > >>>>>>>> >> > >>>>>>>> >> On Mon, Sep 28, 2026 at 10:00 AM Alex Dutra <[email protected]> > wrote: > >>>>>>>> >>> > >>>>>>>> >>> Hi all, > >>>>>>>> >>> > >>>>>>>> >>> > The delegation header intended to allow the client to > communicate its capabilities, but we've aligned more clearly on catalogs > controlling access as opposed to clients defining behavior. > >>>>>>>> >>> > >>>>>>>> >>> IMO, this header isn't ideal for communicating client > capabilities. > >>>>>>>> >>> Notably, it fails to represent a key capability: that of > using the > >>>>>>>> >>> client's own storage credentials to bypass access delegation > entirely. > >>>>>>>> >>> Right now, the only way to imply this capability is by > omitting the > >>>>>>>> >>> header altogether, which introduces ambiguity into the > specification. > >>>>>>>> >>> Put simply, a client cannot explicitly communicate e.g.: "I > can > >>>>>>>> >>> function either without access delegation or using vended > credentials, > >>>>>>>> >>> but I do not support remote signing." > >>>>>>>> >>> > >>>>>>>> >>> Therefore, Apache Polaris today treats this header as an > explicit > >>>>>>>> >>> requirement rather than just a capability statement. IOW, it > treats > >>>>>>>> >>> the header presence as an indication that the client cannot > function > >>>>>>>> >>> without access delegation, and that at least one specified > mode must > >>>>>>>> >>> be satisfied. > >>>>>>>> >>> > >>>>>>>> >>> I would be interested to hear how other catalog > implementations are > >>>>>>>> >>> approaching this. > >>>>>>>> >>> > >>>>>>>> >>> In any case, I still think that a minor spec clarification > would be > >>>>>>>> >>> welcome, if for nothing else, at least to lift the ambiguity > around > >>>>>>>> >>> what it means when the header is absent and clarify whether > the header > >>>>>>>> >>> is a capability statement, or a requirement. > >>>>>>>> >>> > >>>>>>>> >>> Thanks, > >>>>>>>> >>> Alex > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> >>> On Mon, Sep 28, 2026 at 5:03 PM Daniel Weeks < > [email protected]> wrote: > >>>>>>>> >>> > > >>>>>>>> >>> > Youngrae, > >>>>>>>> >>> > > >>>>>>>> >>> > I think the issue with #2 above is that it's a bit of an > anti-pattern for the client to concurrently own access. Either the catalog > should be the authority on access or it shouldn't. Security around the > physical data shouldn't really be a negotiation between the client and the > server. If the catalog is delegating access to the data, it should be the > sole source of truth. > >>>>>>>> >>> > > >>>>>>>> >>> > It feels like we're trying to make this a negotiation > between the catalog and client, but the catalog should be responsible for > access delegation in a secure environment. > >>>>>>>> >>> > > >>>>>>>> >>> > The background on this header is important: The original > access control mechanism was remote signing only, which was not originally > part of the REST protocol (it was separate and S3 specific). We later > introduced vended credentials, but only through the config payload. Since > then, a lot has changed: we introduced the Storage Credentials and now the > Remote Signing Config, which means that we're in a much better position for > the catalog to clearly indicate which mechanism the client should use. The > mechanisms are more standardized, and clients should be able to report if > they cannot support what the catalog has provided. > >>>>>>>> >>> > > >>>>>>>> >>> > While there may be some residual value in the client > signaling its capabilities, I'm not convinced we need to change the wording > in the spec and we need to improve the other mechanisms as we evolve to > support new functionality like File type, which may mix different modes of > access. > >>>>>>>> >>> > > >>>>>>>> >>> > The behavior you describe in Polaris is perfectly fine as > an implementation choice, but I don't think catalogs SHOULD fail if they > cannot provide any requested mechanism. The catalog can provide an > alternative (e.g. you requested signing, but I'm only providing > credentials), which the client can error on if it does not understand. > Alternatively, the catalog could reject all requested forms and return only > the metadata, effectively signaling that the client can only see the > metadata, but has no access to the physical data. > >>>>>>>> >>> > > >>>>>>>> >>> > -Dan > >>>>>>>> >>> > > >>>>>>>> >>> > > >>>>>>>> >>> > On Fri, Sep 25, 2026 at 8:18 PM youngrae kim < > [email protected]> wrote: > >>>>>>>> >>> >> > >>>>>>>> >>> >> Thanks Alex, Yufei and Dan. > >>>>>>>> >>> >> > >>>>>>>> >>> >> Dan, I think the catalog-controlled direction and Alex's > wording are compatible, and it may help to separate the two roles the > header plays today: > >>>>>>>> >>> >> > >>>>>>>> >>> >> 1. Which mechanism to use. > >>>>>>>> >>> >> That is already the catalog's decision in Polaris: the > server picks among the mechanisms > >>>>>>>> >>> >> it supports for that catalog and storage, and the header > only tells it which ones the client can consume. > >>>>>>>> >>> >> Nothing in the proposed sentence changes that. > >>>>>>>> >>> >> > >>>>>>>> >>> >> 2. Whether the client can access the data at all without > delegation. Today the header is the > >>>>>>>> >>> >> only signal a client has for "I have no storage > credentials of my own". > >>>>>>>> >>> >> If the catalog cannot provide any mechanism the client can > consume, the request cannot succeed in any > >>>>>>>> >>> >> useful sense, and the question is only where the failure > surfaces: at load time with a > >>>>>>>> >>> >> message naming the cause, or at the first data access with > a storage error. > >>>>>>>> >>> >> > >>>>>>>> >>> >> The proposed SHOULD covers exactly the empty-intersection > case in (2); a server that knows its > >>>>>>>> >>> >> clients carry their own credentials can still return the > table, which is why SHOULD rather than MUST seems right. > >>>>>>>> >>> >> > >>>>>>>> >>> >> If the longer-term direction is to rely less on the > header, I would be interested in how a > >>>>>>>> >>> >> catalog-driven model would express (2), for example the > client learning from /v1/config which > >>>>>>>> >>> >> mechanisms the catalog offers, so it can decide before > loading a table. That seems like a > >>>>>>>> >>> >> larger change than the clarification this thread started > with, so I would suggest landing the > >>>>>>>> >>> >> one-sentence clarification for the current header and > taking the catalog-driven model as a > >>>>>>>> >>> >> separate discussion, unless you see a conflict between the > two. > >>>>>>>> >>> >> > >>>>>>>> >>> >> Thanks, > >>>>>>>> >>> >> Youngrae Kim > >>>>>>>> >>> >> > >>>>>>>> >>> >> 2026년 9월 26일 (토) 오전 8:52, Daniel Weeks <[email protected]>님이 > 작성: > >>>>>>>> >>> >>> > >>>>>>>> >>> >>> I'm not convinced that we're thinking about this the > right way. > >>>>>>>> >>> >>> > >>>>>>>> >>> >>> The delegation header intended to allow the client to > communicate its capabilities, but we've aligned more clearly on catalogs > controlling access as opposed to clients defining behavior. > >>>>>>>> >>> >>> > >>>>>>>> >>> >>> If we leaned more into that direction, then access would > be determined by the catalog based on the availability of vented > credentials or the new remote signing configuration. > >>>>>>>> >>> >>> > >>>>>>>> >>> >>> There may be an opportunity here to rely less on the > header and give responsibility back on th e catalog to determine the access > model. > >>>>>>>> >>> >>> > >>>>>>>> >>> >>> -Dan > >>>>>>>> >>> >>> > >>>>>>>> >>> >>> On Fri, Sep 25, 2026, 4:39 PM Yufei Gu < > [email protected]> wrote: > >>>>>>>> >>> >>>> > >>>>>>>> >>> >>>> I tend to agree with Alex and Youngrae. A fail-fast > approach provides a clear behavioral contract between the server and > clients, while the current spec introduces ambiguity. > >>>>>>>> >>> >>>> > >>>>>>>> >>> >>>> I am not sure if this ambiguity was intentional. Please > chime in if you can provide more context; otherwise, I suggest we update > the spec to eliminate it. > >>>>>>>> >>> >>>> > >>>>>>>> >>> >>>> Yufei > >>>>>>>> >>> >>>> > >>>>>>>> >>> >>>> > >>>>>>>> >>> >>>> On Thu, Sep 24, 2026 at 10:12 AM Alex Dutra < > [email protected]> wrote: > >>>>>>>> >>> >>>>> > >>>>>>>> >>> >>>>> Hi Youngrae, > >>>>>>>> >>> >>>>> > >>>>>>>> >>> >>>>> Thanks for starting this thread! > >>>>>>>> >>> >>>>> > >>>>>>>> >>> >>>>> I believe the phrase "via any or none of the requested > mechanisms" > >>>>>>>> >>> >>>>> requires clarification. > >>>>>>>> >>> >>>>> > >>>>>>>> >>> >>>>> If a client requests modes A and B, a server returning > only mode B is > >>>>>>>> >>> >>>>> perfectly acceptable. However, returning a 200 OK > response with > >>>>>>>> >>> >>>>> neither mode A nor B represents an ambiguous outcome. > >>>>>>>> >>> >>>>> > >>>>>>>> >>> >>>>> The current wording likely assumes the client could > theoretically > >>>>>>>> >>> >>>>> proceed using its own fallback storage credentials. In > practice, > >>>>>>>> >>> >>>>> operators configure clients intentionally; requesting > access > >>>>>>>> >>> >>>>> delegation "just in case" is unrealistic. A fail-fast > error is a > >>>>>>>> >>> >>>>> better outcome, as opposed to letting the client > proceed, only to see > >>>>>>>> >>> >>>>> it fail later on. > >>>>>>>> >>> >>>>> > >>>>>>>> >>> >>>>> I propose updating the specification to mandate an > error response when > >>>>>>>> >>> >>>>> no requested mechanism is available, while also > clarifying whether > >>>>>>>> >>> >>>>> providing multiple delegation modes is supported: > >>>>>>>> >>> >>>>> > >>>>>>>> >>> >>>>> "The server MAY supply access via any or all of the > requested > >>>>>>>> >>> >>>>> mechanisms. If none of the requested mechanisms is > available, the > >>>>>>>> >>> >>>>> server SHOULD return an HTTP 400 (Bad Request) > response." > >>>>>>>> >>> >>>>> > >>>>>>>> >>> >>>>> Thanks, > >>>>>>>> >>> >>>>> Alex > >>>>>>>> >>> >>>>> > >>>>>>>> >>> >>>>> On Wed, Sep 23, 2026 at 11:14 AM youngrae kim < > [email protected]> wrote: > >>>>>>>> >>> >>>>> > > >>>>>>>> >>> >>>>> > Hi all, > >>>>>>>> >>> >>>>> > The REST catalog spec describes > X-Iceberg-Access-Delegation as an "optional signal to the server that the > client supports delegated access via a comma-separated list of access > mechanisms. The server may choose to supply access via any or none of the > requested mechanisms." [1] > >>>>>>>> >>> >>>>> > > >>>>>>>> >>> >>>>> > In Apache Polaris we recently discussed what a server > should do when the header is present but it can satisfy none of the > requested mechanisms, for example vended-credentials against an > S3-compatible store without STS, or remote-signing on a server that does > not implement it [2]. > >>>>>>>> >>> >>>>> > > >>>>>>>> >>> >>>>> > Polaris fails such requests with HTTP 400 and a > message naming the reason, because in these cases the inability is a static > property of the catalog and a 200 without credentials would only move the > failure to the first data access. > >>>>>>>> >>> >>>>> > > >>>>>>>> >>> >>>>> > Two readings of "any or none" came up and the text > does not settle which is intended: > >>>>>>>> >>> >>>>> > (a) "none" means the server returns 200 with the > table and no delegated access, and the client detects this from the > response: an empty storage-credentials (and, we understand, an empty > remote-signing-config once that field lands in 1.12), falling back to the > config map for older servers [3]. > >>>>>>>> >>> >>>>> > > >>>>>>>> >>> >>>>> > (b) "none" also permits the server to reject the > request when it cannot provide any of the requested mechanisms. > >>>>>>>> >>> >>>>> > > >>>>>>>> >>> >>>>> > Questions for the community: > >>>>>>>> >>> >>>>> > 1. Is rejecting the request a permitted server > choice under the current text, or is (a) the only intended behavior? > >>>>>>>> >>> >>>>> > 2. If rejection is permitted, is there a preferred > status code and error type, so clients can distinguish "delegation > unavailable" from other 400s? > >>>>>>>> >>> >>>>> > 3. If (a) is the intended behavior, should the spec > state explicitly that the absence of credentials in both > storage-credentials and config (and of remote-signing-config) is the signal > that no delegation was applied? > >>>>>>>> >>> >>>>> > > >>>>>>>> >>> >>>>> > I'm happy to draft a clarification to the parameter > description once there is agreement. > >>>>>>>> >>> >>>>> > > >>>>>>>> >>> >>>>> > [1] > https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L2151-L2161 > >>>>>>>> >>> >>>>> > [2] > https://lists.apache.org/thread/9oyv4v0vk3t8stv9832npl8t0yyp4j8r > >>>>>>>> >>> >>>>> > [3] > https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L4246-L4249 > >>>>>>>> >>> >>>>> > > >>>>>>>> >>> >>>>> > Thanks, > >>>>>>>> >>> >>>>> > Youngrae Kim >
