SEPURI-SAI-KRISHNA opened a new issue, #19705:
URL: https://github.com/apache/hudi/issues/19705
## Bug Description
**What happened:**
`PartitionPathParser#getPartitionValues` splits the partition path on `/`
and consumes exactly one
segment per partition field, unless the field is time-based:
```java
String[] parts = partitionPath.split("/");
...
if (isTimeBasedType(fieldSchema.getType())) {
...
int numDateDirs = parts.length - partitionFields.length + 1;
partitionValues[i] = inferDateValue(partitionPath, parts, pathSegment,
numDateDirs, fieldSchema);
pathSegment += numDateDirs;
} else {
String segment = parts[pathSegment];
String[] segmentParts = segment.split(EQUALS_SIGN);
partitionValues[i] = parseValue(segmentParts[segmentParts.length - 1],
fieldSchema);
pathSegment++;
}
```
(`PartitionPathParser.java:55-78`)
The multi-directory case is handled only for `DATE`, `TIMESTAMP` and `TIME`
(`isTimeBasedType`, `PartitionPathParser.java:164-166`). Under
`hoodie.datasource.write.slash.separated.date.partitioning=true` a partition
column declared
`STRING` — which is a perfectly ordinary way to hold a `yyyy-MM-dd` value,
and what the feature's
own test table uses — produces the directory `2026/01/05`, so `parts` is
`["2026", "01", "05"]` while `partitionFields.length` is 1. The `else`
branch takes `parts[0]` and
the partition value is read back as **`"2026"`**. The remaining two segments
are silently dropped.
This is not the Spark read path:
`HoodieSparkUtils#doParsePartitionColumnValues` handles the slash
layout, which is why the datasource tests pass. `PartitionPathParser` is the
engine-agnostic
parser, reached from `HoodieFileGroupReader`
(`HoodieFileGroupReader.java:284-286`) when a
bootstrap table merges skeleton and data files. The wrong value is then
handed to
`convertValueToEngineType` and materialized into the returned records.
So: a bootstrap table with slash-separated date partitioning and a STRING
partition column returns
`2026` in the partition column for every row, rather than `2026-01-05`.
Calling `getPartitionFieldVals` directly against a schema with two string
fields and one date field
shows the behavior:
| partition path | partition fields | result |
|---|---|---|
| `2026/01/05` | `[string_field]` | `[2026]` — truncated |
| `2026/01/05` | `[date_field]` | `[2026-01-05]` — the time-based branch
handles it |
| `2026-01-05` | `[string_field]` | `[2026-01-05]` — unaffected without
slash partitioning |
| `2026/01` | `[string_field]` | `[2026]` — truncated |
| `2026/01/05/us` | `[string_field, other_field]` | `[2026, 01]` — shifted,
`us` dropped |
The last row is the same `else` branch failing a second way: consuming one
segment per field means
a slash-partitioned field mis-aligns every field after it, so the values are
not merely truncated
but land on the wrong columns. That case is a subset of #19666, which tracks
multi-field slash
partitioning being broken generally; this issue is about the single-field
truncation.
**What you expected:**
`PartitionPathParser` should reconstruct `2026-01-05` from `2026/01/05` for
a non-time-typed
partition column when the table config has slash separated date partitioning
enabled, matching what
`HoodieSparkUtils#doParsePartitionColumnValues` already does on the Spark
side.
**Steps to reproduce:**
1. Create a bootstrap table with one `STRING` partition column and
`hoodie.datasource.write.slash.separated.date.partitioning=true`.
2. Write a row with partition value `2026-01-05`; the directory
`<base>/2026/01/05/` is created.
3. Read the table back through the file group reader's bootstrap merge path
and inspect the
partition column: it holds `2026`.
**Suggested fix:**
Make the non-time-typed branch slash-aware rather than special-casing dates
only: when the table
config enables slash separated date partitioning, consume `parts.length -
partitionFields.length + 1`
segments for the field the same way the time-based branch already does, and
rejoin them with `-`.
The config is not currently threaded into `PartitionPathParser`, so it would
have to be passed in
alongside the schema.
Raised during review of #19648.
## Environment
**Hudi version:** master (1.3.0-SNAPSHOT)
**Query engine:** engine-agnostic (`HoodieFileGroupReader`); reproduced
against Spark, but the
parser is shared
**Relevant configs:**
`hoodie.datasource.write.slash.separated.date.partitioning=true` with a
`STRING` (or any non-`DATE`/`TIMESTAMP`/`TIME`) partition column
## Logs and Stack Trace
No exception: the partition value is silently truncated to the first path
segment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
