Hi all, Iceberg 1.11 shipped the base implementation for table encryption, including KMS-based key wrapping/unwrapping and encrypted data/delete, manifest, and manifest-list files. REST catalog support is also being worked on in Iceberg (see https://github.com/apache/iceberg/pull/13225).
I have been testing Polaris with Iceberg REST client-side encryption enabled. Basic catalog operations such as loadTable, commit/drop without purge, list, etc. work without Polaris changes because Polaris only needs the table metadata JSON for those paths, and metadata.json is not encrypted. The places where Polaris does need encryption awareness are the server-side paths that read encrypted Iceberg artifacts. The first concrete example is drop table with purge: TableCleanupTask reads snapshot manifest lists and manifests to enumerate files for deletion, so it needs to use an EncryptingFileIO. The same would apply to any Polaris-side table maintenance/optimization, orphan/snapshot cleanup logic, or any future remote scan/planning capability that reads manifests or data/delete files. There is also a related but separate topic around vending KMS credentials to clients. That likely needs Iceberg REST spec work first, similar in spirit to current storage credential vending, so I think it should be designed for but not required as the first Polaris step. The first Polaris-side building block I would propose is to allow Iceberg catalogs to carry KMS configuration, similarly to how catalogs currently carry StorageConfigurationInfo. This should be separate from storage configuration because the storage backend and KMS provider may differ, for example GCS storage with AWS KMS. AWS KMS would be a reasonable first implementation target, using Iceberg’s existing KeyManagementClient/AWS KMS support, while leaving the model extensible for Azure and GCP. I have already been experimenting with this locally and would be happy to work on the Polaris changes. A possible first PR could be limited to: 1. Add catalog-level KMS configuration model/API support. 2. Add AWS KMS server-side configuration wiring. Any feedback is welcome. Cheers, Adam
