I'm not sure I agree with the model of a client bringing its own credentials if the catalog is also responsible for controlling access (can't really stop people from using that model, but as I said above, it feels like an anti-pattern).
The current wording in the spec was intentionally loose so that the catalog can make decisions without being forced into specific security/access decisions. I still don't think that updating the wording would be flexible enough. I'm not sure what all catalogs do, but I can say that it's always safe for the catalog to return credentials/remote config even if the client was externally configured to use its own. The current reference implementation bypasses that, I believe, but conceptually a client does not need to use the mechanism provided by the catalog. This again feels like a bit of an anti-pattern: if there's a reason to bring sidecar credentials, it implies a gap in the delegation model that we should be looking into addressing. Dan On Mon, Sep 28, 2026 at 10:00 AM Alex Dutra <[email protected]> wrote: > Hi all, > > > The delegation header intended to allow the client to communicate its > capabilities, but we've aligned more clearly on catalogs controlling access > as opposed to clients defining behavior. > > IMO, this header isn't ideal for communicating client capabilities. > Notably, it fails to represent a key capability: that of using the > client's own storage credentials to bypass access delegation entirely. > Right now, the only way to imply this capability is by omitting the > header altogether, which introduces ambiguity into the specification. > Put simply, a client cannot explicitly communicate e.g.: "I can > function either without access delegation or using vended credentials, > but I do not support remote signing." > > Therefore, Apache Polaris today treats this header as an explicit > requirement rather than just a capability statement. IOW, it treats > the header presence as an indication that the client cannot function > without access delegation, and that at least one specified mode must > be satisfied. > > I would be interested to hear how other catalog implementations are > approaching this. > > In any case, I still think that a minor spec clarification would be > welcome, if for nothing else, at least to lift the ambiguity around > what it means when the header is absent and clarify whether the header > is a capability statement, or a requirement. > > Thanks, > Alex > > > On Mon, Sep 28, 2026 at 5:03 PM Daniel Weeks <[email protected]> wrote: > > > > Youngrae, > > > > I think the issue with #2 above is that it's a bit of an anti-pattern > for the client to concurrently own access. Either the catalog should be > the authority on access or it shouldn't. Security around the physical data > shouldn't really be a negotiation between the client and the server. If > the catalog is delegating access to the data, it should be the sole source > of truth. > > > > It feels like we're trying to make this a negotiation between the > catalog and client, but the catalog should be responsible for access > delegation in a secure environment. > > > > The background on this header is important: The original access control > mechanism was remote signing only, which was not originally part of the > REST protocol (it was separate and S3 specific). We later introduced > vended credentials, but only through the config payload. Since then, a lot > has changed: we introduced the Storage Credentials and now the Remote > Signing Config, which means that we're in a much better position for the > catalog to clearly indicate which mechanism the client should use. The > mechanisms are more standardized, and clients should be able to report if > they cannot support what the catalog has provided. > > > > While there may be some residual value in the client signaling its > capabilities, I'm not convinced we need to change the wording in the spec > and we need to improve the other mechanisms as we evolve to support new > functionality like File type, which may mix different modes of access. > > > > The behavior you describe in Polaris is perfectly fine as an > implementation choice, but I don't think catalogs SHOULD fail if they > cannot provide any requested mechanism. The catalog can provide an > alternative (e.g. you requested signing, but I'm only providing > credentials), which the client can error on if it does not understand. > Alternatively, the catalog could reject all requested forms and return only > the metadata, effectively signaling that the client can only see the > metadata, but has no access to the physical data. > > > > -Dan > > > > > > On Fri, Sep 25, 2026 at 8:18 PM youngrae kim <[email protected]> > wrote: > >> > >> Thanks Alex, Yufei and Dan. > >> > >> Dan, I think the catalog-controlled direction and Alex's wording are > compatible, and it may help to separate the two roles the header plays > today: > >> > >> 1. Which mechanism to use. > >> That is already the catalog's decision in Polaris: the server picks > among the mechanisms > >> it supports for that catalog and storage, and the header only tells it > which ones the client can consume. > >> Nothing in the proposed sentence changes that. > >> > >> 2. Whether the client can access the data at all without delegation. > Today the header is the > >> only signal a client has for "I have no storage credentials of my own". > >> If the catalog cannot provide any mechanism the client can consume, the > request cannot succeed in any > >> useful sense, and the question is only where the failure surfaces: at > load time with a > >> message naming the cause, or at the first data access with a storage > error. > >> > >> The proposed SHOULD covers exactly the empty-intersection case in (2); > a server that knows its > >> clients carry their own credentials can still return the table, which > is why SHOULD rather than MUST seems right. > >> > >> If the longer-term direction is to rely less on the header, I would be > interested in how a > >> catalog-driven model would express (2), for example the client learning > from /v1/config which > >> mechanisms the catalog offers, so it can decide before loading a table. > That seems like a > >> larger change than the clarification this thread started with, so I > would suggest landing the > >> one-sentence clarification for the current header and taking the > catalog-driven model as a > >> separate discussion, unless you see a conflict between the two. > >> > >> Thanks, > >> Youngrae Kim > >> > >> 2026년 9월 26일 (토) 오전 8:52, Daniel Weeks <[email protected]>님이 작성: > >>> > >>> I'm not convinced that we're thinking about this the right way. > >>> > >>> The delegation header intended to allow the client to communicate its > capabilities, but we've aligned more clearly on catalogs controlling access > as opposed to clients defining behavior. > >>> > >>> If we leaned more into that direction, then access would be determined > by the catalog based on the availability of vented credentials or the new > remote signing configuration. > >>> > >>> There may be an opportunity here to rely less on the header and give > responsibility back on th e catalog to determine the access model. > >>> > >>> -Dan > >>> > >>> On Fri, Sep 25, 2026, 4:39 PM Yufei Gu <[email protected]> wrote: > >>>> > >>>> I tend to agree with Alex and Youngrae. A fail-fast approach provides > a clear behavioral contract between the server and clients, while the > current spec introduces ambiguity. > >>>> > >>>> I am not sure if this ambiguity was intentional. Please chime in if > you can provide more context; otherwise, I suggest we update the spec to > eliminate it. > >>>> > >>>> Yufei > >>>> > >>>> > >>>> On Thu, Sep 24, 2026 at 10:12 AM Alex Dutra <[email protected]> > wrote: > >>>>> > >>>>> Hi Youngrae, > >>>>> > >>>>> Thanks for starting this thread! > >>>>> > >>>>> I believe the phrase "via any or none of the requested mechanisms" > >>>>> requires clarification. > >>>>> > >>>>> If a client requests modes A and B, a server returning only mode B is > >>>>> perfectly acceptable. However, returning a 200 OK response with > >>>>> neither mode A nor B represents an ambiguous outcome. > >>>>> > >>>>> The current wording likely assumes the client could theoretically > >>>>> proceed using its own fallback storage credentials. In practice, > >>>>> operators configure clients intentionally; requesting access > >>>>> delegation "just in case" is unrealistic. A fail-fast error is a > >>>>> better outcome, as opposed to letting the client proceed, only to see > >>>>> it fail later on. > >>>>> > >>>>> I propose updating the specification to mandate an error response > when > >>>>> no requested mechanism is available, while also clarifying whether > >>>>> providing multiple delegation modes is supported: > >>>>> > >>>>> "The server MAY supply access via any or all of the requested > >>>>> mechanisms. If none of the requested mechanisms is available, the > >>>>> server SHOULD return an HTTP 400 (Bad Request) response." > >>>>> > >>>>> Thanks, > >>>>> Alex > >>>>> > >>>>> On Wed, Sep 23, 2026 at 11:14 AM youngrae kim < > [email protected]> wrote: > >>>>> > > >>>>> > Hi all, > >>>>> > The REST catalog spec describes X-Iceberg-Access-Delegation as an > "optional signal to the server that the client supports delegated access > via a comma-separated list of access mechanisms. The server may choose to > supply access via any or none of the requested mechanisms." [1] > >>>>> > > >>>>> > In Apache Polaris we recently discussed what a server should do > when the header is present but it can satisfy none of the requested > mechanisms, for example vended-credentials against an S3-compatible store > without STS, or remote-signing on a server that does not implement it [2]. > >>>>> > > >>>>> > Polaris fails such requests with HTTP 400 and a message naming the > reason, because in these cases the inability is a static property of the > catalog and a 200 without credentials would only move the failure to the > first data access. > >>>>> > > >>>>> > Two readings of "any or none" came up and the text does not settle > which is intended: > >>>>> > (a) "none" means the server returns 200 with the table and no > delegated access, and the client detects this from the response: an empty > storage-credentials (and, we understand, an empty remote-signing-config > once that field lands in 1.12), falling back to the config map for older > servers [3]. > >>>>> > > >>>>> > (b) "none" also permits the server to reject the request when it > cannot provide any of the requested mechanisms. > >>>>> > > >>>>> > Questions for the community: > >>>>> > 1. Is rejecting the request a permitted server choice under the > current text, or is (a) the only intended behavior? > >>>>> > 2. If rejection is permitted, is there a preferred status code > and error type, so clients can distinguish "delegation unavailable" from > other 400s? > >>>>> > 3. If (a) is the intended behavior, should the spec state > explicitly that the absence of credentials in both storage-credentials and > config (and of remote-signing-config) is the signal that no delegation was > applied? > >>>>> > > >>>>> > I'm happy to draft a clarification to the parameter description > once there is agreement. > >>>>> > > >>>>> > [1] > https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L2151-L2161 > >>>>> > [2] > https://lists.apache.org/thread/9oyv4v0vk3t8stv9832npl8t0yyp4j8r > >>>>> > [3] > https://github.com/apache/iceberg/blob/778d103c0fa802be5e85e4cb2e4e4b4bcd8ac891/open-api/rest-catalog-open-api.yaml#L4246-L4249 > >>>>> > > >>>>> > Thanks, > >>>>> > Youngrae Kim >
