The issue and overlap between table config and storage credentials directly relate to how these fields evolved.
Table config is intended for configuration properties that the catalog wants the client to use including default values and overrides sent to configure a client. Configs can override various implementation specific settings like FileIO configurations, some Auth related configs, etc. This initially included credentials used to access the underlying storage, but as storage configuration complexity increased, we separated the storage credentials into a structure that can represent more complicated configurations. As Eduard and Alex have noted, older clients still look for credential information in the config bundle instead of respecting the new storage credentials payload. Older clients won't be able to take advantage of more complicated layouts (like multi cloud or multi policy based tables). However, most tables store data under a single location that can be supported even with the older property-based configuration. It's up to the catalog implementation to decide whether it supports both old and new configurations and can make decisions about what to send back depending on the information it has about the client (e.g. headers or user agent sent by the client). For the best compatibility, the catalog should include both representations where reasonable. However, it's entirely reasonable to block older clients if the old property based representation isn't sufficient. -Dan On Tue, Mar 3, 2026 at 10:50 AM Yufei Gu <[email protected]> wrote: > Hi Eduard, thanks for the clarification! Just to better understand the > intent, could you share examples of non-storage-related configurations > currently sent through the "config"? > > I understand that credentials are still included in "config" for backward > compatibility, which is what Polaris does today. Looking ahead, if "config" > carries more than storage related settings, we could consider moving all > storage related configuration, including credentials, into > "storage-credentials". That would simplify the contract so clients only > need to look in one place for any storage-related configuration. > > Curious to hear your thoughts. > > Yufei > > > On Mon, Mar 2, 2026 at 10:03 PM Eduard Tudenhöfner < > [email protected]> wrote: > >> I don't think we can deprecate the *config* because other >> non-storage-related configurations are being sent through the *config *field. >> *storage-credentials* is purely for storage-related things but not every >> Iceberg client will support this field, so as a server you must also send >> credentials through the *config* until all clients are updated. >> >> >> On Tue, Mar 3, 2026 at 2:29 AM Yufei Gu <[email protected]> wrote: >> >>> Given that the boundary between credentials and other storage configs is >>> not clear, I am not convinced that we need both "config" and >>> "storage-credentials". It might be simpler to keep only one field. However, >>> neither name fully captures the combined intent. We could introduce a new >>> field such as storage-config/storage-access-config, but that creates >>> another problem: consolidating two fields by adding a new one effectively >>> leaves us with three fields :-). Jokes aside. In reality, there are two >>> possible options: >>> 1. Just use "storage-credentials" and deprecate "config" >>> 2. Create a new field named "storage-config" and deprecate the other two. >>> >>> Yufei >>> >>> >>> On Mon, Mar 2, 2026 at 2:57 AM Alexandre Dutra <[email protected]> >>> wrote: >>> >>>> Hi Yun, >>>> >>>> > This means there is currently some overlap between the config and >>>> storage-credentials fields. Is my understanding correct? >>>> >>>> That is my understanding too, and this will be the case until we >>>> consider that we can drop support for credentials in config for old >>>> clients. >>>> >>>> Thanks, >>>> Alex >>>> >>>> On Sat, Feb 28, 2026 at 12:27 AM yun zou <[email protected]> >>>> wrote: >>>> > >>>> > Hi Eduard and Alexandre, >>>> > >>>> > Thank you both for chiming in on this — I really appreciate the >>>> clarification. >>>> > >>>> > If I understand correctly, the storage-credentials field contains only >>>> > credential-related configurations and was introduced mainly for the >>>> > following reasons: >>>> > >>>> > There was a lack of clear documentation around credential-related >>>> > entries within the existing config field in the LoadTable response. >>>> > >>>> > It is used by the credential endpoint to support credential refresh. >>>> > >>>> > As Alexandre mentioned, all credential configurations are still >>>> > included in the config field of LoadTableResult for backward >>>> > compatibility. This means there is currently some overlap between the >>>> > config and storage-credentials fields. Is my understanding correct? >>>> > >>>> > Best regards, >>>> > Yun >>>> > >>>> > On Fri, Feb 27, 2026 at 5:40 AM Alexandre Dutra <[email protected]> >>>> wrote: >>>> > > >>>> > > Hi Yun, >>>> > > >>>> > > In addition to what Eduard said, I would like to stress this >>>> sentence >>>> > > from the design doc: "In order to maintain backward compatibility >>>> with >>>> > > older clients, credentials still need to be passed as properties via >>>> > > the config field for LoadTableResult and LoadViewResult." >>>> > > >>>> > > IOW, servers must send credentials in both places: config and >>>> > > storage-credentials, to stay compatible with older clients; this is >>>> > > what Apache Polaris does [1]. >>>> > > >>>> > > Thanks, >>>> > > Alex >>>> > > >>>> > > [1]: >>>> https://github.com/apache/polaris/blob/579b271b8c41cd26339bccfd771c1a591ad27039/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java#L856-L861 >>>> > > >>>> > > On Fri, Feb 27, 2026 at 7:52 AM Eduard Tudenhöfner >>>> > > <[email protected]> wrote: >>>> > > >> >>>> > > >> 1) What is the intended boundary between storage-credentials and >>>> > > >> config in LoadTableResult? >>>> > > > >>>> > > > >>>> > > > We used to send back all sorts of configurations in the config of >>>> a LoadTableResponse (examples are shown here) but those configurations >>>> weren't well documented for different storage providers. Later we >>>> introduced the concept of storage credentials which are sent back in the >>>> storage-credentials field. Those will only contain storage credentials, >>>> such as access-key-id / secret-access-key / session-token / >>>> session-token-expires-at-ms in the case of S3 and no other configurations. >>>> Also the storage-credentials field takes precedence over whatever is sent >>>> in the config. >>>> > > > >>>> > > > >>>> > > >> 2) What is the motivation behind this design choice? >>>> > > > >>>> > > > >>>> > > > Storage credentials offer more flexibility because they let you >>>> use different credentials for different storage prefixes. Additionally, >>>> those credentials are being refreshed automatically, which is not the case >>>> with the vended credentials that are sent as part of the config. >>>> > > > >>>> > > > Here's the original design that outlines why we standardized >>>> credentials into the storage-credentials field. >>>> > > > >>>> > > > >>>> > > > On Thu, Feb 26, 2026 at 9:22 PM yun zou < >>>> [email protected]> wrote: >>>> > > >> >>>> > > >> Hi All, >>>> > > >> >>>> > > >> I’m looking into the credential vending support for Iceberg and >>>> got a >>>> > > >> bit confused about the config and storage-credentials fields in >>>> > > >> LoadTableResult. >>>> > > >> >>>> > > >> From what I understand: >>>> > > >> >>>> > > >> storage-credentials can contain configurations that are not >>>> strictly >>>> > > >> credential-related, such as client.region. >>>> > > >> >>>> > > >> The description for "config" states that it is "table-specific >>>> > > >> configuration for the table's resources, including its HTTP >>>> client and >>>> > > >> FileIO. For example, config may contain a specific FileIO >>>> > > >> implementation class for the table depending on its underlying >>>> > > >> storage." >>>> > > >> >>>> > > >> In our current implementation, we kind of merge both fields when >>>> > > >> initializing the client, which makes me wonder: >>>> > > >> >>>> > > >> 1) What is the intended boundary between storage-credentials and >>>> > > >> config in LoadTableResult? >>>> > > >> >>>> > > >> 2) What is the motivation behind this design choice? >>>> > > >> >>>> > > >> Thanks in advance for clarifying! >>>> > > >> >>>> > > >> Best regards, >>>> > > >> Yun >>>> >>>
