rdblue commented on code in PR #12162:
URL: https://github.com/apache/iceberg/pull/12162#discussion_r1940051614
##########
format/spec.md:
##########
@@ -889,6 +892,9 @@ Table metadata consists of the following fields:
| _optional_ | _optional_ | _optional_ | **`partition-statistics`** | A list
(optional) of [partition statistics](#partition-statistics).
|
| | | _optional_ | **`row-lineage`** | A
boolean, defaulting to false, setting whether or not to track the creation and
updates to rows in the table. See [Row Lineage](#row-lineage).
|
| | | _optional_ | **`next-row-id`** | A
`long` higher than all assigned row IDs; the next snapshot's `first-row-id`.
See [Row Lineage](#row-lineage).
|
+| | | _optional_ | **`key-cache`** | A list
of encryption keys (key-id/key-wrap pairs), used to encrypt the manifest list
file key metadata. See [Snapshot](#key-metadata-key-id).
|
Review Comment:
I talked a little about this idea with Russell and I think the trade-off is
whether to re-encrypt all of the snapshot keys when rotating the table key. If
we re-encrypt the snapshot keys, that could take a while when there are
potentially thousands of snapshots in the table. On the other hand, in practice
it would be a small set and rotating the table key would be rare.
I think I like the idea of a single table key that is used to decrypt all of
the snapshot keys. It would require a structure that allows us to change the
content of the snapshot keys (a separate list of them that is mutable) but that
wouldn't be too difficult.
@ggershinsky, what do you think about that idea?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
