linliu-code commented on issue #17372: URL: https://github.com/apache/hudi/issues/17372#issuecomment-5704927470
Closing this issue: the behaviour reported here is confirmed, but we are resolving it in the opposite direction, so there is no fix coming that makes COW honour event time ordering for MIT deletes. The report is accurate — on a COW table, `MERGE INTO ... WHEN MATCHED THEN DELETE` with a source ordering value older than the stored record deletes it anyway. What the report does not capture is that MOR behaves differently: the delete is written into a log delete block carrying its ordering value, and the reader discards it when the stored record is newer. So the two table types disagree. We decided to align them the other way — **MOR will be changed to ignore the ordering value for SQL merge-into deletes**, matching COW — rather than making COW honour it. The reasoning: - `DELETE FROM` already ignores ordering values on both table types and both merge modes (`TestDeleteFromTable` parameterises over all four and asserts the row is gone in every one, with the ordering field populated). Making MIT honour ordering would leave MIT as the only SQL delete surface where a delete can be silently dropped. - A SQL-issued delete is a statement of intent by the query, not a late-arriving event; commit-time semantics match what a user writing SQL expects. - Ingestion paths (Deltastreamer, CDC) are unaffected and continue to respect ordering values, which is where event-time semantics for deletes actually matter. PR #19934, which implemented the direction described in this issue, has been closed unmerged. A replacement PR going the other way will follow. Note for anyone tracking HUDI-8915 in JIRA: this GitHub issue is a mirror, so the JIRA ticket needs the same update separately. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
