nada-attia opened a new pull request, #20763:
URL: https://github.com/apache/hudi/pull/20763

   ### Describe the issue this Pull Request addresses
   
   A `MERGE INTO ... WHEN MATCHED THEN UPDATE` that goes through 
`HoodieWriteMergeHandle` can fail with `HoodieIOException: Failed to parse 
record into provided schema` (caused by an `EOFException` or `Invalid int 
encoding`), or silently write shifted values. This happens when the MERGE 
source columns are `NOT NULL` (for example a subquery of constants) and the 
table columns are nullable.
   
   ### Summary and Changelog
   
   Fixes MERGE INTO updates through the payload-based merge handle when source 
and table nullability differ.
   
   - `HoodieWriteMergeHandle#write` wrapped the merge result, which is already 
in the table schema, in the configured payload (`ExpressionPayload` for MERGE 
INTO) via `constructHoodieRecord`. `shouldIgnore` and `writeUpdateRecord` then 
called `ExpressionPayload#getInsertValue` on it, which decodes the bytes with 
the MERGE source schema (`hoodie.payload.record.schema`) instead of the table 
schema, so the nullable-union bytes were read as plain fields.
   - The combined record is now built with `constructFinalHoodieRecord`, so it 
is a plain record. The record merger already resolves a payload `SENTINEL` 
result by returning the stored record, so no payload evaluation is needed on 
the merge result.
   
   No code was copied.
   
   ### Impact
   
   No public API change. MERGE INTO updates through `HoodieWriteMergeHandle` no 
longer fail or write shifted values for the scenario above. The `numUpdates` 
stat may differ when the stored row is kept, since the combined record is now a 
plain record rather than a payload-wrapped one.
   
   ### Risk Level
   
   low. This is a one-line change limited to the payload-based merge handle 
(`HoodieWriteMergeHandle`), not the default `FileGroupReaderBasedMergeHandle`. 
The same fix was verified on a Hudi 1.2 branch: a MERGE INTO with a NOT NULL 
constants source, a nullable source, and a conditional update whose condition 
does not match (the stored row is kept) all passed. This PR itself has not been 
built or tested against `master`, and no new test is added here.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [ ] Adequate tests were added if applicable


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [PR] fix(client): build the com... nada-attia (via GitHub) via commits

Reply via email to