voonhous commented on PR #19648: URL: https://github.com/apache/hudi/pull/19648#issuecomment-5394513162
Pushed a commit to this branch (maintainer edit) settling the multi-field question the last round left open. Restoring the substitution in the multi-part branch fixed the `CustomKeyGenerator` prefix miss but opened a write-side hole: `ComplexKeyGenerator#getPartitionPath(InternalRow, StructType)` lands in that branch too, while `KeyGenUtils#getRecordPartitionPath` still guards on `size() == 1`. On a `(datestr, city)` slash table, `insert` writes `2026-01-05/san-francisco` and `bulk_insert` writes `2026/01/05/san/francisco` -- and one such directory makes `BaseHoodieTableFileIndex:476` throw for the whole table under the default lazy listing. On master that leg throws `ClassCastException`, so no table can reach the state; with this PR as it stood, the write commits cleanly and the failure surfaces whenever a read next lists partitions, possibly days later. Rather than trade the CCE for a silent landmine, the commit pulls #19666's suggested fix into this PR and fails fast at the write: * `HoodieWriterUtils#validateTableConfig` rejects slash partitioning with more than one partition field (covers `df.write` and SQL writes, new and existing tables) * `HoodieCatalogTable#extraTableConfig` rejects it at `CREATE TABLE`, next to the existing hive-style check * the multi-part substitution stays exactly as you restored it -- new writes can no longer reach it, and it keeps `composeRelativePartitionPath` naming the right directory for legacy tables * the multi-field `CustomKeyGenerator` test becomes two rejection asserts (create and `df.write`); comments and javadoc reworded from "tracked in #19666" to the legacy-read framing One behavior change a committer should sign off on: registering a legacy multi-field slash table via `CREATE TABLE ... location` now also fails, since that path runs `validateTableConfig` against the on-disk config. I kept it deliberately -- such tables already return silently wrong (zero) rows on any partition-filtered query, so a loud error at registration is protection; tables already registered keep reading, as no read path runs the validation. #19666 stays open for the read side of legacy layouts. PR description updated to match. `TestSlashSeparatedPartitionValue` is green locally with the change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
