rangareddy commented on issue #16461: URL: https://github.com/apache/hudi/issues/16461#issuecomment-5264826506
Worked this one up. The documentation gap is real, but **the specific claim in the title is not true of 0.14.0**, so I have documented the behaviour where it actually holds rather than writing an incorrect sentence into a release-note page: PR **#19592**. The switch is in `MultipleSparkJobExecutionStrategy`, which picks the row-writer path from `hoodie.datasource.write.row.writer.enable`. Reading that line at each tag: | ref | fallback default | | --- | --- | | `release-0.13.0`, `release-0.13.1` | `false` | | **`release-0.14.0`** | **`false`** | | `release-0.14.1` | `true` | | `release-0.15.0`, `release-0.15.1` | `false` (flipped back) | | `release-1.0.0` … `release-1.2.0` | `true` | So in 0.14.0 — the release this ticket names — it was **not** enabled by default via that fallback, and 0.15.0 reverted it again. A "row writer w/ clustering is enabled by default" note in the 0.14.0 release notes would misinform anyone reading them. There is a genuine subtlety underneath, and it is probably where the original claim came from: `getBooleanOrDefault(key, fallback)` only falls back when the key is **absent**, and `HoodieWriterUtils` calls `setDefaultValue(ENABLE_ROW_WRITER)` (default `true`) on every Spark datasource write. So even in 0.14.0, clustering running inline as part of a datasource write did take the row-writer path — while a standalone or async clustering job, which does not carry that key, hit the `false` fallback. Two different answers for the same release, which is more than a release-note bullet can carry. What #19592 does instead: adds a **Row writer** subsection under Execution Strategy in `clustering.md`, which mentioned none of this in any version. It states the config, both halves of the default (datasource write vs standalone job), the fallback history, and how to force the RDD path. Applied to the current docs plus the five supported versioned copies — and the text differs between them, because 1.0.0/1.0.1/1.0.2 additionally gate the row path on `HoodieDataTypeUtils.canUseRowWriter(schema, conf)` (refuses when `parquet.avro.write-old-list-structure` is `false` and the schema has both a small-precision decimal and a list/map field), a check that is gone from 1.1.1 onwards. 0.15.x and 0.14.x doc copies are deliberately untouched: their fallback was `false`, so the text would not be true there. If a committer would still like something in the 0.14.0 release notes, I am happy to add it — but it would need to read closer to "clustering triggered from a Spark datasource write uses the row writer; standalone clustering jobs do not unless the config is set", and someone should confirm that is the intended message before it goes into a historical release page. One related note for whoever picks this up: **#17341 (HUDI-8749) "Deprecate clustering with row writer"** is open and asks whether this path is needed at all now. If it lands, the new subsection is the natural home for the deprecation notice. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
