[
https://issues.apache.org/jira/browse/HUDI-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375617#comment-17375617
]
ASF GitHub Bot commented on HUDI-2139:
--------------------------------------
pengzhiwei2018 opened a new pull request #3230:
URL: https://github.com/apache/hudi/pull/3230
## What is the purpose of the pull request
Currently we process all the update-action and inert-action in the
ExpressionPayload#
getInsertValue without know whether the record is matched or not matched for
MOR table. This may result in incorrect merge result. e.g.
> Merge into h0
> using (select 2 as id, 'a1' as name, 10 as price from s) s0
> on h0.id = s0.id
> when matched then s0.id = 1 the update set id = s0.id, name = s0.name,
price = 10
> when not matched then s0.id = 2 the insert (id,name,price) values(id,name,
20)
If the id = 2 can matched the target table h0, but it cannot match the
udpate-condition ( s0.id = 1), It should not update the table. However,
currently we cannot know the matched state of the input record, it will goes to
the not-matched actions and update the price to 20 finally. This is incorrect.
So we need to know the update flag of the record and process the
matched-action if it is update record, else process the not-matched action.
Currently we've mixed the two actions together which result the bug.
## Brief change log
- Pass the update flag to the `ExpressionPlay#getInsertValue` from
`HoodieAppendHandle`.
- `ExpressionPlay#getInsertValue` call `processMatchedRecord` or
`processNotMatchedRecord` according to the update flag to distinguish the
matched-action and not-matched action for MOR table.
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test
coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please
describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> MergeInto MOR Table May Result InCorrect Result
> -----------------------------------------------
>
> Key: HUDI-2139
> URL: https://issues.apache.org/jira/browse/HUDI-2139
> Project: Apache Hudi
> Issue Type: Bug
> Components: Spark Integration
> Reporter: pengzhiwei
> Assignee: pengzhiwei
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Currently we process all the update-action and inert-action in the
> ExpressionPayload#
> getInsertValue without know whether the record is matched or not matched for
> MOR table. This may result in incorrect merge result. e.g.
> {code:java}
> Merge into h0
> using (select 2 as id, 'a1' as name, 10 as price from s) s0
> on h0.id = s0.id
> when matched then s0.id = 1 the update set id = s0.id, name = s0.name, price
> = 10
> when not matched then s0.id = 2 the insert (id,name,price) values(id,name,
> 20){code}
> If the id = 2 can matched the target table h0, but it cannot match the
> udpate-condition ( s0.id = 1), It should not update the table. However,
> currently we cannot know the matched state of the input record, it will goes
> to the not-matched actions and update the price to 20 finally. This is
> incorrect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
