yihua opened a new pull request, #982:
URL: https://github.com/apache/hudi-rs/pull/982

   ## Description
   
   Merges resolved the ordering field only among top-level columns, so a nested 
ordering field such as `meta.ts` gave no ordering value. A merge-on-read 
snapshot then let the log record replace the base row whatever its `meta.ts`, 
and a copy-on-write upsert failed with a missing-column error. A shared 
dotted-path resolver in `util::arrow` (split as 
`HoodieUnsafeRowUtils.composeNestedFieldPath` does, with a null parent struct 
reading as a null value) is now used by both file group readers and by the 
write side (incoming dedupe, the cross-partition event-time check, and partial 
upserts that set the enclosing struct).
   
   Closes #919
   
   ## How are the changes test-covered
   
   - [ ] N/A
   - [x] Automated tests (unit and/or integration tests)
   - [ ] Manual tests
     - [ ] Details are described below
   
   A new Spark-written MOR fixture (`v9_nested_ordering_field`, Hudi 1.1.1, 
Avro data and delete blocks) reproduces the issue's writes plus a stale and an 
applied delete, and matches Spark on both reader versions, by key and by record 
position. Native-write tests cover the MOR snapshot with a delete, COW upsert, 
partial upsert of the ordering struct, and a stale cross-partition move; unit 
tests cover the resolver and ordering values.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to