Hi Adam, Thanks for this proposal!
Polaris should already support storage-side KMS in AWS (and compatible systems) via [2802] (cf. [1]). I guess the new features you mention relate to client-side encryption, right? [1] https://polaris.apache.org/blog/2025/12/24/securing-s3-data-with-aws-kms/ [2802] https://github.com/apache/polaris/pull/2802 Cheers, Dmitri. On Tue, May 26, 2026 at 11:06 AM Adam Szita <[email protected]> wrote: > Hi all, > > Iceberg 1.11 shipped the base implementation for table encryption, > including KMS-based key wrapping/unwrapping and encrypted data/delete, > manifest, and manifest-list files. REST catalog support is also being > worked on in Iceberg (see https://github.com/apache/iceberg/pull/13225). > > I have been testing Polaris with Iceberg REST client-side encryption > enabled. Basic catalog operations such as loadTable, commit/drop without > purge, list, etc. work without Polaris changes because Polaris only needs > the table metadata JSON for those paths, and metadata.json is not > encrypted. > > The places where Polaris does need encryption awareness are the server-side > paths that read encrypted Iceberg artifacts. The first concrete example is > drop table with purge: TableCleanupTask reads snapshot manifest lists and > manifests to enumerate files for deletion, so it needs to use an > EncryptingFileIO. The same would apply to any Polaris-side table > maintenance/optimization, orphan/snapshot cleanup logic, or any future > remote scan/planning capability that reads manifests or data/delete files. > > There is also a related but separate topic around vending KMS credentials > to clients. That likely needs Iceberg REST spec work first, similar in > spirit to current storage credential vending, so I think it should be > designed for but not required as the first Polaris step. > > The first Polaris-side building block I would propose is to allow Iceberg > catalogs to carry KMS configuration, similarly to how catalogs currently > carry StorageConfigurationInfo. This should be separate from storage > configuration because the storage backend and KMS provider may differ, for > example GCS storage with AWS KMS. AWS KMS would be a reasonable first > implementation target, using Iceberg’s existing KeyManagementClient/AWS KMS > support, while leaving the model extensible for Azure and GCP. > > I have already been experimenting with this locally and would be happy to > work on the Polaris changes. A possible first PR could be limited to: > > 1. Add catalog-level KMS configuration model/API support. > 2. Add AWS KMS server-side configuration wiring. > > Any feedback is welcome. > > Cheers, > Adam >
