smaheshwar-pltr commented on code in PR #17404:
URL: https://github.com/apache/iceberg/pull/17404#discussion_r3670139661
##########
spark/v4.1/spark/src/test/java/org/apache/iceberg/spark/sql/TestTableEncryption.java:
##########
@@ -162,6 +162,29 @@ public void testConcurrentAppendTransactions() {
assertThat(currentDataFiles(table)).hasSize(dataFiles.size() + 2);
}
+ @TestTemplate
+ public void testSharedTableTransactionInterleavedWithDirectCommit() {
+ validationCatalog.initialize(catalogName, catalogConfig);
+ // A single shared Table (and its EncryptionManager) drives both a staged
transaction and a
+ // direct commit. With a mutable shared manager, the direct commit's key
could be dropped from
+ // metadata, leaving its snapshot undecryptable. The metadata-sourced
manager keeps every key.
+ Table table = validationCatalog.loadTable(tableIdent);
+ List<DataFile> dataFiles = currentDataFiles(table);
+ DataFile dataFile = dataFiles.get(0);
+
+ Transaction transaction = table.newTransaction();
+ transaction.newAppend().appendFile(dataFile).commit();
+
+ // Direct commit on the same shared Table, interleaved before the
transaction commits.
+ table.newFastAppend().appendFile(dataFile).commit();
+
+ transaction.commitTransaction();
+
+ // Reading forces decryption of every snapshot's manifest list, including
the direct commit's.
Review Comment:
this is just completely wrong. reading only reads the current manifest list.
this is unacceptable correctness bug in your code. i've lost trust in your
iteration process so you must now review everything fact checking everything.
that's what happens when i catch a huge oversight like this. not acceptable at
all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
