Hi Alexander, Thank you for preparing the spec change! This is a great addition to help cross-implementation interworking in this area. I went through the PR and left some comments that might be worth discussing. In general I feel that there is too much coupling between the table spec and the encryption spec, or in other words the encryption spec makes too many assumptions on which table metadata struct field uses what encryption method and how. We might want to be more generic with that in the encryption spec.
I'm wondering what others think. Thanks, Gabor Alexander Bailey <[email protected]> ezt írta (időpont: 2026. szept. 2., Sze, 22:55): > Hi everyone, > > Based on the conversation in > https://lists.apache.org/thread/4zm5jhwp42z10pfz2sths7q59r9dlf8l and on > the PR https://github.com/apache/iceberg/pull/16527, I’d like to raise a > vote to add a new encryption spec which documents the binary formats used > by the standard encryption scheme. > > Today the spec describes the key_metadata field in manifest entries and > the encrypted-key-metadata field in table metadata as > “implementation-specific.” In practice these formats are fixed by the Java > implementation, and any other implementation has to reverse-engineer them > from the Java source to read or write encrypted tables. This came out of > implementing table encryption in iceberg-rust. > > The PR adds a new encryption spec document covering: > > - > > The standard key metadata format: a version byte followed by an Avro > binary-encoded record holding the DEK, the AAD prefix, and the file length. > - > > The two-tier key hierarchy in the encryption-keys list: KEK entries > wrapped by the KMS, and manifest list key entries encrypted by a KEK. > > This documents existing behaviour rather than changing it, so no > implementation needs to change. > > > Vote > > [ ] +1 Adopt the spec change > > [ ] 0 No opinion > > [ ] -1 Do not adopt, because… > > Thanks, > > Xander >
