LuciferYang opened a new pull request, #14295: URL: https://github.com/apache/gravitino/pull/14295
### What changes were proposed in this pull request? - `HiveGravitinoOperationOperator` writes a null partition value as `__HIVE_DEFAULT_PARTITION__` in the partition names it builds for drop, exists, load and list, and in the values it sends for `ADD PARTITION`. The read side already maps this name back to null. - The `names` variant of `getHivePartitionName` looks each column up by name instead of by position, since Spark passes only the columns of a partial spec. ### Why are the changes needed? Partitions with a null value could not be dropped or found, and `ADD PARTITION` with a null value stored the string `null`. A partial spec on any column but the first read the value with the wrong column's type, which failed with a `ClassCastException` or matched nothing. Fix: #14291 ### Does this PR introduce _any_ user-facing change? `DROP PARTITION (x = null)` and `ADD PARTITION (x = null)` now act on the Hive default partition. Partitions added by an earlier connector with `ADD PARTITION (x = null)` hold the string `null`; on a string column they are now dropped with `x = 'null'`. This conflicts with #13997, which edits the same lines to escape partition names. When the second of the two is merged, the escaping needs the marker substituted first, for example `partitionNamePart(name, value == null ? HIVE_DEFAULT_PARTITION_NAME : value)`, otherwise `String.valueOf(null)` brings `dt=null` back. ### How was this patch tested? - New `TestHiveGravitinoOperationOperator` covers the null value on drop and on add, and a partial spec on the second column of an `INT, STRING` partitioned table. - `SparkHiveCatalogIT.testManagePartitionTable` adds `SHOW PARTITIONS` with a spec on the second column, and drops, re-adds and drops the null partition written by an `INSERT`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
