lihanyu1715446406-afk commented on PR #20561:
URL: https://github.com/apache/hudi/pull/20561#issuecomment-6822547650
Thanks for the detailed analysis — you are right, the pass-through skipped
the recursive rewrite for record-valued branches. Addressed in dfccba3:
- `HoodieArrayWritableSchemaUtils#rewriteUnionMemberGroup` now rewrites each
populated `memberN` recursively against the corresponding target branch (using
the member's nullable schema via `getNonNullType()`, so the null branch is
skipped when numbering members, matching
`AvroSchemaConverterWithTimestampNTZ#convertUnion`). Absent members stay null.
- Branch correspondence is validated positionally: a target union whose
non-null branches no longer line up (e.g. reordered branches, or a different
branch count) throws `SchemaCompatibilityException` instead of silently reading
the wrong branch.
- Added `testRewriteUnionRecordBranchWithReorderedFields` — your `[long,
Member]` case with `Member` evolving `{a, b}` -> `{b, a}`; the populated member
now comes back as `[B, A]`. Also added
`testRewriteUnionBranchOrderMismatchThrows` for the reordered-branches
rejection.
`TestHoodieArrayWritableSchemaUtils`: 33/33 pass. Checkstyle and RAT are
clean.
This is still utility-level, matching the narrow fix you suggested rather
than an end-to-end Hive query; happy to extend it to a reader-path test if you
think that is needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]