zheliu2 commented on code in PR #15567:
URL: https://github.com/apache/iceberg/pull/15567#discussion_r2908658624
##########
spark/v4.1/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMigrateTableProcedure.java:
##########
@@ -211,6 +211,36 @@ public void testMigratePartitionWithSpecialCharacter()
throws IOException {
sql("SELECT * FROM %s ORDER BY id", tableName));
}
+ @TestTemplate
+ public void testMigratePartitionedTableWithNullPartition() throws
IOException {
Review Comment:
Good catch, thanks! The existing `SELECT *` assertions pass without the fix
because Spark reads the actual Parquet data (which contains real nulls), not
the partition metadata.
Added an assertion that queries the `.files` metadata table to verify the
partition value is stored as null:
```java
assertEquals(
"Should have one file with null partition in metadata",
ImmutableList.of(row((String) null)),
sql("SELECT partition.data FROM %s.files WHERE partition.data IS NULL",
tableName));
```
Without the `TableMigrationUtil` fix, this fails because `partition.data`
would be the literal string `"__HIVE_DEFAULT_PARTITION__"` instead of null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
