smaheshwar-pltr opened a new pull request, #17404: URL: https://github.com/apache/iceberg/pull/17404
… loss under concurrent commits A single Table (and its shared TableOperations + EncryptionManager, e.g. via a caching catalog) can be used concurrently. On encrypted tables the mutable, shared StandardEncryptionManager could drop a snapshot's manifest-list key from encryption-keys, leaving a committed snapshot undecryptable. Make encryption keys live only in table metadata and the manager immutable and metadata-sourced: - StandardEncryptionManager holds an immutable key snapshot. Minting a manifest-list key is a pure operation (mintManifestListKey) that returns the new key(s) instead of storing them; removes the mutating key methods. - SnapshotProducer captures the keys minted while writing the manifest list and persists them into the metadata it commits. - HiveTableOperations builds the manager on demand from a metadata's keys (memoized by metadata identity); drops the doCommit/doRefresh key merges and the mutable manager/key fields. temp(uncommittedMetadata) sources keys from the uncommitted metadata, and BaseTransaction re-points temp ops after a rebase, so staged operations resolve earlier ops' (and concurrently committed) keys. The immutable manager serializes to Spark executors with no concurrent-mutation hazard. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
