SEPURI-SAI-KRISHNA opened a new issue, #19668:
URL: https://github.com/apache/hudi/issues/19668
## Bug Description
**What happened:**
`HoodieTableConfigUtils` auto-infers the hive-sync partition value
extractor, and for a
slash-partitioned table it returns `SlashEncodedDayPartitionValueExtractor`:
```java
} else if
(cfg.contains(KeyGeneratorOptions.SLASH_SEPARATED_DATE_PARTITIONING)
&&
cfg.getString(KeyGeneratorOptions.SLASH_SEPARATED_DATE_PARTITIONING).equals("true"))
{
return
Option.of("org.apache.hudi.hive.SlashEncodedDayPartitionValueExtractor");
```
That extractor throws on any partition path that is not exactly three
segments. A slash table with
a null partition value contains a `__HIVE_DEFAULT_PARTITION__` directory --
one segment -- so hive
sync of such a table fails.
A null partition value is ordinary: it is what the writers produce for a
null or empty date
column, and #19648 adds coverage for exactly that directory.
**What you expected:**
Hive sync of a slash-partitioned table containing the default-partition
directory should succeed,
mapping `__HIVE_DEFAULT_PARTITION__` to a null partition value.
**Steps to reproduce:**
1. Create a slash-partitioned table
(`hoodie.datasource.write.slash.separated.date.partitioning=true`).
2. Insert a row whose partition column is null, producing a
`__HIVE_DEFAULT_PARTITION__`
directory alongside the `yyyy/MM/dd` ones.
3. Run hive sync -- `SlashEncodedDayPartitionValueExtractor` throws on the
one-segment path.
**Suggested fix:**
Have `SlashEncodedDayPartitionValueExtractor` pass
`__HIVE_DEFAULT_PARTITION__` through as the
default partition value instead of requiring three segments.
Raised during review of #19648.
## Environment
**Hudi version:** master (1.3.0-SNAPSHOT)
**Query engine:** Spark + hive sync
**Relevant configs:**
`hoodie.datasource.write.slash.separated.date.partitioning=true`, hive sync
enabled with the auto-inferred partition value extractor
## Logs and Stack Trace
Thrown from
`SlashEncodedDayPartitionValueExtractor#extractPartitionValuesInPath` on the
`__HIVE_DEFAULT_PARTITION__` path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
