ajantha-bhat commented on a change in pull request #3757:
URL: https://github.com/apache/iceberg/pull/3757#discussion_r772795362
##########
File path:
spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewriteDataFilesProcedure.java
##########
@@ -174,6 +174,82 @@ public void
testRewriteDataFilesWithFilterOnPartitionTable() {
assertEquals("Data after compaction should not change", expectedRecords,
actualRecords);
}
+ @Test
+ public void testRewriteDataFilesWithInFilterOnPartitionTable() {
+ createPartitionTable();
+ // create 5 files for each partition (c2 = 'foo' and c2 = 'bar')
+ insertData(10);
+ List<Object[]> expectedRecords = currentData();
+
+ // select only 5 files for compaction (files in the partition c2 in
('bar'))
+ List<Object[]> output = sql(
+ "CALL %s.system.rewrite_data_files(table => '%s'," +
+ " where => 'c2 in (\"bar\")')", catalogName, tableIdent);
+
+ assertEquals("Action should rewrite 5 data files from single matching
partition" +
+ "(containing c2 = bar) and add 1 data files",
+ ImmutableList.of(row(5, 1)),
+ output);
+
+ List<Object[]> actualRecords = currentData();
+ assertEquals("Data after compaction should not change", expectedRecords,
actualRecords);
+ }
+
+ @Test
+ public void testRewriteDataFilesWithAllPossibleFilters() {
+ createPartitionTable();
+ // create 5 files for each partition (c2 = 'foo' and c2 = 'bar')
+ insertData(10);
+
+ // Pass the literal value which is not present in the data files.
+ // So that parsing can be tested on a same dataset without actually
compacting the files.
+
+ // EqualTo
Review comment:
I think it is ok. Because it is a text representation and when multiple
filtres exist we will know which one we are focusing on, for example and / or
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
