Thanks everyone for chiming in on this, *Regarding the PR : * The approach we are taking here where we pick encryption creds from the client side (catalog level) and storage creds from server vended creds, is fairly concerning to me. if the REST server is giving one access because someone has *SELECT* grant, *SERVER* should mint them everything that required for the client to successfully read the table, sending storage creds but not KMS creds feels incorrect to me, how is the server gonna know the client has creds for *decryption* for this already ? This is entirely different from what other catalogs expect i.e entirely a client side configuration for example Hive Catalog / Glue catalog etc....
Regarding backward compatibility, if we do this, server vending KMS credentials eventually, these clients will simply not work since they always expect the credentials to come from the client side (Note : I am strictly talking of table level) *Regarding the Spec Change : * It makes sense to me as well, to decouple KMS & storage creds ! All in all I believe we should conclude this discussion and not rush into things (we can make incremental progress for sure), if we have open questions and an awaiting spec change. Best, Prashant Singh On Tue, May 5, 2026 at 12:11 AM Gidon Gershinsky <[email protected]> wrote: > Hi all, > > The built-in encryption is designed to protect tables in any storage > backend, including untrusted storage. So I also think the storage > credentials and KMS credentials are best treated separately, as they don't > have the same purpose/scope. > > Regarding the current PR (13225) - it doesn't handle credentials directly; > instead, it focuses on building the scaffolding for table encryption in the > REST catalog client - similarly to the Hive catalog client (and other > catalogs in the future). > > Cheers, Gidon > > > On Tue, May 5, 2026 at 3:40 AM Chris Lu <[email protected]> wrote: > >> Hi Prashant, >> >> My own preference would be to keep storage credentials and encryption/KMS >> credentials separate in the REST contract. >> >> Storage credentials and KMS/Vault credentials have different scopes, >> lifetimes, providers, and failure modes. >> >> >> - Storage credentials authorize access to object locations. >> - >> >> KMS/Vault credentials authorize key wrap/unwrap operations. >> >> Object-store SSE-KMS is a storage-layer feature, but Iceberg table >> encryption is not necessarily tied to the object store. Vault or >> external KMS support would not naturally fit inside a storage credential >> object. So mixing KMS credentials into the storage credentials object >> may work for some S3/SSE cases, but it makes the REST model harder to >> extend. >> >> A cleaner model may be to have separate sections for storage credentials >> and encryption credentials. Another option is a generic typed credential >> model, where each credential has a type such as storage or encryption, and >> a provider such as S3, Vault, or a cloud KMS. >> >> The typed model may be easier to extend later if we expect more >> credential types. >> >> For backward compatibility, I think the current PR can still move >> forward, but the client should not assume that storage credentials are the >> permanent place for KMS credentials. If we need an interim implementation, >> maybe it should be documented as catalog-level KMS configuration support, >> not table-level encryption credential vending. >> >> I also think the catalog should have a clear capability or failure >> signal. If a table requires Iceberg encryption and the catalog supports >> vended credentials, then the catalog should either return the required >> encryption credentials or fail early during table load or credential >> refresh with a clear error. Failing later during file IO or key unwrap >> would be much harder to debug. >> >> So my suggested direction would be: >> >> >> - Keep object-store SSE credentials/config separate from Iceberg >> table encryption credentials. >> - >> >> Add a dedicated encryption credential model, or a generic typed >> credential model. >> - >> >> Add a capability or requirement signal so clients can fail early when >> required credentials are missing. >> >> Avoid baking in the assumption that KMS credentials always live under >> storage credentials. This should leave room for catalogs that vend >> table-scoped KMS/Vault credentials in the future, instead of only >> supporting catalog-level KMS client configuration. >> >> Thanks, >> Chris >> > > On Tue, May 5, 2026 at 3:22 AM Sreesh Maheshwar <[email protected]> > wrote: > >> Hey Prashant, >> >> Thank you for kicking off this discussion! Some thoughts: >> >> > The current PR advocates for including KMS credentials within the >> storage credentials object. >> >> This isn't true - the KMS client is currently configured with just the >> catalog config client-side (though that includes the server's `getConfig` >> response) [1]; there is no per-table vending of KMS credentials as such. >> >> > Separation of KMS/Vault Creds from Storage Creds >> >> I agree, if/when it supports this, the AWS KMS client could choose to >> introduce `kms.access-key-id` and co. instead of reusing `s3.access-key-id` >> and co. (similar to how we have `rest.access-key-id` and co. [2]) to >> differentiate from storage credentials. And similarly for vending: if we'd >> like to support the vending of KMS credentials [3], we can discuss its >> format to enable separation. >> >> (Specifically on the current PR [4], I don't think that the concerns >> described affect it due to my first point above. I think >> catalog-initialised KMS clients, as implemented, are useful to support now >> and in the future.) >> >> Thanks, >> Sreesh Maheshwar >> >> [1] >> https://github.com/apache/iceberg/pull/13225/files#diff-86450612dbe323d6d06cbc3846aa1913f042eaedadc0ca027c36bfbe08d3a46cR284 >> [2] >> https://github.com/apache/iceberg/blob/2d54125734ddc9b9fb87db147ff255918108fa2c/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L193 >> [3] https://github.com/apache/iceberg/issues/16194 >> > [4] https://github.com/apache/iceberg/pull/13225 > >> >> On Monday, May 4th, 2026 at 9:03 AM, Prashant Singh < >> [email protected]> wrote: >> >> Hi everyone, >> >> I’d like to start a discussion regarding how we handle credentials for >> encryption (like KMS or Vault (recently being discussed)) in the REST >> catalog. >> >> As we know, unlike other catalogs, the REST catalog mints credentials at >> the table level for the client to use in subsequent operations, and we >> already have the dedicated /credentials endpoint in place to handle >> refreshing these. >> >> While reviewing the recent encryption PR, a few architectural concerns >> came up that I believe we need to conclude on before we mark the REST >> catalog as "ready" for supporting encryption: >> >> - >> >> *Separation of KMS/Vault Creds from Storage Creds:* How should we >> handle external key managers like Vault? The current PR [1] advocates for >> including KMS credentials within the storage credentials object. If we end >> up supporting Vault, this can't be mixed with storage cred. *(Side >> note: catalogs have historically mixed KMS creds with this object for >> things like SSE, but that is entirely an object-store-level concept).* >> We need a clear path forward for how REST will return per-table >> credentials >> specifically for Vault/KMS stores. >> - >> >> *Catalog Awareness & Client-Side Assertions:* If the catalog returns >> credentials, it overrides the client-side credentials. This means a naive >> catalog that is unaware of encryption and just treats metadata as-is will >> have no clue it needs to vend these specific KMS credentials and if it >> forgerts to do that there is no way for the client to know this (for >> object >> store cases) except to fail during runtime ? Should the catalog fail such >> requests *as part of their contract of supporting v3* (we don't need >> this in spec), should the catalog send some signal, hey i send you creds >> for encryption too, and if the client doesn't find it fails early ? >> - >> >> *Backward Compatibility Risks:* If we release the client now with the >> expectation that "storage credentials" will always contain KMS >> credentials. >> If we later introduce a dedicated field for encryption credentials in the >> loadTable response, we will be forced to maintain backward >> compatibility to support both ways of returning credentials. >> >> To be clear, I do not want to block the progress on the current PR. I >> really appreciate all the hard work that has gone into it! However, I think >> it is crucial that we align on these design points for the REST catalog's >> encryption architecture before finalizing it. >> >> I would appreciate any thoughts or feedback on how we should structure >> this. >> >> [1] https://github.com/apache/iceberg/pull/13225 >> >> Thanks, >> >> Prashant Singh >> >> >>
