Hi all, Thanks everyone for the discussion and comments on the KMS credential vending spec proposal doc.
I opened a draft PR to move the current proposal into the REST OpenAPI spec: https://github.com/apache/iceberg/pull/17155 Please take a look and continue review on the PR. Cheers, Adam On Mon, 8 Jun 2026 at 12:22, Adam Szita <[email protected]> wrote: > Hi folks, > > I wrote up a spec proposal for the KMS credential vending part of this > discussion: > > https://docs.google.com/document/d/1VSewbVmjukU5eTiZruCJVmUZcRZvcAsd4JOeC6Lt3fo > > The intent is to separate this from the current key-metadata / > encryption-metadata work, so that the existing PR can hopefully move > forward independently while we discuss the REST credential-vending shape > here. > At a high level, the proposal mirrors storage credential vending: the REST > catalog can return short-lived, scoped key-management-credentials, and > clients match those credentials to KMS key IDs referenced by Iceberg > encryption metadata. > Any feedback welcome, especially on whether this is the right REST-level > direction and if the proposed response shape is enough for current > table-level encryption while also leaving room for future column-level > encryption. > > Cheers, > Adam > > On Thu, 7 May 2026 at 13:56, Ádám Szita <[email protected]> wrote: > >> Hi all, >> >> I think there are two related but separate topics here: >> >> 1. supporting Iceberg table encryption with a REST catalog client, and >> 2. defining a REST mechanism for vending/refreshing KMS or Vault >> credentials. >> >> For the current PR, I don’t think we necessarily need to solve #2. >> Yes, REST catalog deployments may use credential vending for storage >> access, >> but that is still a catalog feature/configuration choice rather than a >> requirement >> for REST catalog usage. So, similarly, I don’t think supporting table >> encryption >> with a REST catalog should require the catalog to also implement KMS >> credential vending. >> >> I do think it's a good idea to have the KMS creds provided by the catalog >> side, >> especially for catalogs that want to enforce least privilege at table >> scope. >> But I see that as a follow-up design/API discussion. I also think it >> should be >> general enough to avoid coupling certain cloud storage types with their >> respective KMS offerings. (e.g. S3 storage doesn't necessarily imply AWS >> KMS >> usage). >> >> We have KeyManagementClient interface that users can implement >> for their own KMS needs. Large enterprise deployments may already use an >> internal KMS, I don’t think we should require them to implement both >> this interface and KMS credential vending before they can use table >> encryption with a REST catalog. >> >> Let me know what you all think. >> >> Cheers, >> Adam >> >> On 2026/05/06 04:59:23 Prashant Singh wrote: >> > Thanks everyone for chiming in on this, >> > >> > *Regarding the PR : * >> > The approach we are taking here where we pick encryption creds from the >> > client side (catalog level) and storage creds from server vended creds, >> is >> > fairly concerning to me. >> > if the REST server is giving one access because someone has *SELECT* >> > grant, *SERVER* should mint them everything that required for the >> client to >> > successfully read the table, sending storage creds but not KMS creds >> > feels incorrect to me, how is the server gonna know the client has creds >> > for *decryption* for this already ? This is entirely different from what >> > other catalogs expect i.e entirely a client side configuration for >> example >> > Hive Catalog / Glue catalog etc.... >> > >> > Regarding backward compatibility, if we do this, server vending KMS >> > credentials eventually, these clients will simply not work since they >> > always expect the credentials to come from the client side (Note : I am >> > strictly talking of table level) >> > >> > *Regarding the Spec Change : * >> > It makes sense to me as well, to decouple KMS & storage creds ! >> > >> > All in all I believe we should conclude this discussion and not rush >> into >> > things (we can make incremental progress for sure), if we have open >> > questions and an awaiting spec change. >> > >> > Best, >> > Prashant Singh >> > >> > On Tue, May 5, 2026 at 12:11 AM Gidon Gershinsky <[email protected]> >> wrote: >> > >> > > Hi all, >> > > >> > > The built-in encryption is designed to protect tables in any storage >> > > backend, including untrusted storage. So I also think the storage >> > > credentials and KMS credentials are best treated separately, as they >> don't >> > > have the same purpose/scope. >> > > >> > > Regarding the current PR (13225) - it doesn't handle credentials >> directly; >> > > instead, it focuses on building the scaffolding for table encryption >> in the >> > > REST catalog client - similarly to the Hive catalog client (and other >> > > catalogs in the future). >> > > >> > > Cheers, Gidon >> > > >> > > >> > > On Tue, May 5, 2026 at 3:40 AM Chris Lu <[email protected]> wrote: >> > > >> > >> Hi Prashant, >> > >> >> > >> My own preference would be to keep storage credentials and >> encryption/KMS >> > >> credentials separate in the REST contract. >> > >> >> > >> Storage credentials and KMS/Vault credentials have different scopes, >> > >> lifetimes, providers, and failure modes. >> > >> >> > >> >> > >> - Storage credentials authorize access to object locations. >> > >> - >> > >> >> > >> KMS/Vault credentials authorize key wrap/unwrap operations. >> > >> >> > >> Object-store SSE-KMS is a storage-layer feature, but Iceberg table >> > >> encryption is not necessarily tied to the object store. Vault or >> > >> external KMS support would not naturally fit inside a storage >> credential >> > >> object. So mixing KMS credentials into the storage credentials object >> > >> may work for some S3/SSE cases, but it makes the REST model harder to >> > >> extend. >> > >> >> > >> A cleaner model may be to have separate sections for storage >> credentials >> > >> and encryption credentials. Another option is a generic typed >> credential >> > >> model, where each credential has a type such as storage or >> encryption, and >> > >> a provider such as S3, Vault, or a cloud KMS. >> > >> >> > >> The typed model may be easier to extend later if we expect more >> > >> credential types. >> > >> >> > >> For backward compatibility, I think the current PR can still move >> > >> forward, but the client should not assume that storage credentials >> are the >> > >> permanent place for KMS credentials. If we need an interim >> implementation, >> > >> maybe it should be documented as catalog-level KMS configuration >> support, >> > >> not table-level encryption credential vending. >> > >> >> > >> I also think the catalog should have a clear capability or failure >> > >> signal. If a table requires Iceberg encryption and the catalog >> supports >> > >> vended credentials, then the catalog should either return the >> required >> > >> encryption credentials or fail early during table load or credential >> > >> refresh with a clear error. Failing later during file IO or key >> unwrap >> > >> would be much harder to debug. >> > >> >> > >> So my suggested direction would be: >> > >> >> > >> >> > >> - Keep object-store SSE credentials/config separate from Iceberg >> > >> table encryption credentials. >> > >> - >> > >> >> > >> Add a dedicated encryption credential model, or a generic typed >> > >> credential model. >> > >> - >> > >> >> > >> Add a capability or requirement signal so clients can fail early >> when >> > >> required credentials are missing. >> > >> >> > >> Avoid baking in the assumption that KMS credentials always live under >> > >> storage credentials. This should leave room for catalogs that vend >> > >> table-scoped KMS/Vault credentials in the future, instead of only >> > >> supporting catalog-level KMS client configuration. >> > >> >> > >> Thanks, >> > >> Chris >> > >> >> > > >> > > On Tue, May 5, 2026 at 3:22 AM Sreesh Maheshwar < >> [email protected]> >> > > wrote: >> > > >> > >> Hey Prashant, >> > >> >> > >> Thank you for kicking off this discussion! Some thoughts: >> > >> >> > >> > The current PR advocates for including KMS credentials within the >> > >> storage credentials object. >> > >> >> > >> This isn't true - the KMS client is currently configured with just >> the >> > >> catalog config client-side (though that includes the server's >> `getConfig` >> > >> response) [1]; there is no per-table vending of KMS credentials as >> such. >> > >> >> > >> > Separation of KMS/Vault Creds from Storage Creds >> > >> >> > >> I agree, if/when it supports this, the AWS KMS client could choose to >> > >> introduce `kms.access-key-id` and co. instead of reusing >> `s3.access-key-id` >> > >> and co. (similar to how we have `rest.access-key-id` and co. [2]) to >> > >> differentiate from storage credentials. And similarly for vending: >> if we'd >> > >> like to support the vending of KMS credentials [3], we can discuss >> its >> > >> format to enable separation. >> > >> >> > >> (Specifically on the current PR [4], I don't think that the concerns >> > >> described affect it due to my first point above. I think >> > >> catalog-initialised KMS clients, as implemented, are useful to >> support now >> > >> and in the future.) >> > >> >> > >> Thanks, >> > >> Sreesh Maheshwar >> > >> >> > >> [1] >> > >> >> https://github.com/apache/iceberg/pull/13225/files#diff-86450612dbe323d6d06cbc3846aa1913f042eaedadc0ca027c36bfbe08d3a46cR284 >> > >> [2] >> > >> >> https://github.com/apache/iceberg/blob/2d54125734ddc9b9fb87db147ff255918108fa2c/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L193 >> > >> [3] https://github.com/apache/iceberg/issues/16194 >> > >> >> > > [4] https://github.com/apache/iceberg/pull/13225 >> > > >> > >> >> > >> On Monday, May 4th, 2026 at 9:03 AM, Prashant Singh < >> > >> [email protected]> wrote: >> > >> >> > >> Hi everyone, >> > >> >> > >> I’d like to start a discussion regarding how we handle credentials >> for >> > >> encryption (like KMS or Vault (recently being discussed)) in the REST >> > >> catalog. >> > >> >> > >> As we know, unlike other catalogs, the REST catalog mints >> credentials at >> > >> the table level for the client to use in subsequent operations, and >> we >> > >> already have the dedicated /credentials endpoint in place to handle >> > >> refreshing these. >> > >> >> > >> While reviewing the recent encryption PR, a few architectural >> concerns >> > >> came up that I believe we need to conclude on before we mark the REST >> > >> catalog as "ready" for supporting encryption: >> > >> >> > >> - >> > >> >> > >> *Separation of KMS/Vault Creds from Storage Creds:* How should we >> > >> handle external key managers like Vault? The current PR [1] >> advocates for >> > >> including KMS credentials within the storage credentials object. >> If we end >> > >> up supporting Vault, this can't be mixed with storage cred. *(Side >> > >> note: catalogs have historically mixed KMS creds with this object >> for >> > >> things like SSE, but that is entirely an object-store-level >> concept).* >> > >> We need a clear path forward for how REST will return per-table >> credentials >> > >> specifically for Vault/KMS stores. >> > >> - >> > >> >> > >> *Catalog Awareness & Client-Side Assertions:* If the catalog >> returns >> > >> credentials, it overrides the client-side credentials. This means >> a naive >> > >> catalog that is unaware of encryption and just treats metadata >> as-is will >> > >> have no clue it needs to vend these specific KMS credentials and >> if it >> > >> forgerts to do that there is no way for the client to know this >> (for object >> > >> store cases) except to fail during runtime ? Should the catalog >> fail such >> > >> requests *as part of their contract of supporting v3* (we don't >> need >> > >> this in spec), should the catalog send some signal, hey i send >> you creds >> > >> for encryption too, and if the client doesn't find it fails early >> ? >> > >> - >> > >> >> > >> *Backward Compatibility Risks:* If we release the client now with >> the >> > >> expectation that "storage credentials" will always contain KMS >> credentials. >> > >> If we later introduce a dedicated field for encryption >> credentials in the >> > >> loadTable response, we will be forced to maintain backward >> > >> compatibility to support both ways of returning credentials. >> > >> >> > >> To be clear, I do not want to block the progress on the current PR. I >> > >> really appreciate all the hard work that has gone into it! However, >> I think >> > >> it is crucial that we align on these design points for the REST >> catalog's >> > >> encryption architecture before finalizing it. >> > >> >> > >> I would appreciate any thoughts or feedback on how we should >> structure >> > >> this. >> > >> >> > >> [1] https://github.com/apache/iceberg/pull/13225 >> > >> >> > >> Thanks, >> > >> >> > >> Prashant Singh >> > >> >> > >> >> > >> >> > >> >
