nsivabalan commented on PR #19205: URL: https://github.com/apache/hudi/pull/19205#issuecomment-5222940939
**Running `TestHoodieSparkSqlWriter` for the first time found real fallout from this branch.** Worth flagging because it is the breaking change landing in the existing suite, and because of where it was hiding. ### What failed Four cases of `testDatasourceInsertForTableTypeBaseFileMetaFields`: ``` hoodie.meta.fields.mode mismatch: table is ALL but the writer requests NONE ``` ### Why — and why it is a fixture bug, not a product bug `initializeMetaClientForBootstrap` creates the table **without forwarding `POPULATE_META_FIELDS`**, even though `fooTableParams` carries it. So the table was always created as `ALL` while the writer asked for `NONE`. That disagreement predates this branch entirely. The old code narrowed the write silently, so nothing noticed. Making meta-field population a table property turns it into a loud failure — which is the gate doing its job, on a table/writer pair that had genuinely disagreed all along. Fixed by forwarding the property so the fixture creates the table it is about to write to (`85c45087350f`), and applied the same fix to the `WithTestFormat` variant, which carries its own copy of the helper (`3588955936f3`). Failures for that test go 4 → 0. The fifth error in that run, `testNonPartitionTableWithMetatableSupport`, is a marker-file/timeline-service flake — `Failed to create marker file ...marker.CREATE`. It passes in isolation and has nothing to do with meta fields. ### Correcting myself, again I had twice written this suite off as having a "pre-existing classpath issue." It does not. The whole chain of local build failures I reported — `hudi-utilities` not compiling, the stale `hudi-utilities-bundle`, `hudi-cli` unable to resolve `MetaFieldsMode.parse` — traced to one missing flag: the root pom defaults `kafka.version` to `2.0.0`, and the `spark3` profile sets it to `3.4.1`. Everything builds and runs with `-Pspark3`. The pattern worth naming: **every suite I had verified up to this point was one I wrote myself.** Those show the feature works; they say nothing about what it breaks. The one suite most likely to catch a breaking change was the one I had classified as environmentally broken — and that classification conveniently removed an obstacle, which is exactly when it deserved more scrutiny rather than less. If reviewers see other suites failing on this mismatch, the fix shape is the same: the table and the writer must agree, because a writer can no longer narrow the mode. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
