SEPURI-SAI-KRISHNA opened a new issue, #19667:
URL: https://github.com/apache/hudi/issues/19667
## Bug Description
**What happened:**
On a table with
`hoodie.datasource.write.slash.separated.date.partitioning=true` and a
`TIMESTAMP` partition column, the Avro write path and the row-writer path
derive different
partition values.
`SqlKeyGenerator`'s slash-partitioning arm sits ahead of the `TimestampType`
normalization in the
same `dataType` match, so it shadows it. With the default
`hoodie.datasource.write.keygenerator.consistent.logical.timestamp.enabled=false`,
the Avro path
writes the raw micros value (for example `1767607200000000`) while the
row-writer path writes the
normalized rendering (`2026/01/05 18:00:00.0`).
Two write paths on the same table therefore place records for the same
logical value into
different directories.
**What you expected:**
Either both paths agree on the partition value for a TIMESTAMP column, or a
TIMESTAMP partition
column is rejected under slash-separated date partitioning -- the feature is
documented for
`yyyy-MM-dd` date values.
**Steps to reproduce:**
1. Create a table with a `TIMESTAMP` partition column and
`hoodie.datasource.write.slash.separated.date.partitioning=true`.
2. Write one batch through the Avro path and one through the row writer
(`hoodie.spark.sql.insert.into.operation=bulk_insert`).
3. Compare `_hoodie_partition_path` and the directories on disk between the
two batches.
**Suggested fix:**
Move the slash arm inside the `case _ =>` branch of the `dataType` match in
`SqlKeyGenerator`, so
the `TimestampType` normalization keeps precedence; or reject
`TimestampType` partition columns
when slash-separated date partitioning is enabled.
Raised during review of #19648.
## Environment
**Hudi version:** master (1.3.0-SNAPSHOT)
**Query engine:** Spark
**Relevant configs:**
`hoodie.datasource.write.slash.separated.date.partitioning=true`,
`hoodie.datasource.write.keygenerator.consistent.logical.timestamp.enabled=false`
(default),
TIMESTAMP partition column
## Logs and Stack Trace
No failure -- the two write paths just disagree on the directory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
