deniskuzZ commented on PR #6924: URL: https://github.com/apache/hive/pull/6924#issuecomment-6918556145
> [@deniskuzZ](https://github.com/deniskuzZ) I have two questions regarding this point: > > > MERGE with UPDATE and DELETE clauses on V3: one delete branch (SplitMergeRewriter ORs the clauses' conditions); the update's insert branch is not counted, rows affected unchanged > > 1. Can merging into native ACID and Iceberg v2 tables with both delete and delete of update branches work the same way? > The delete branches (one for update and one for delete) never overlap. The combined branch could have a predicate like A OR B, so it would delete the exact same rows as the two separate branches. On disk, native ACID would write one delete delta per statement instead of two, and Iceberg v2 would write one set of position-delete files instead of two. This would mean fewer small files and a less complex `SplitMergeRewriter`, since only the single-delete-branch implementation would be needed. > 2. If `1.` is a no-go, can the special code for handling update-deletes and delete-deletes in the same branch be extracted into another `SplitMergeRewriter`? > The decision about which one to instantiate could be made in `MergeRewriterFactory`: > > ``` > } else if (splitUpdate) { > if (hasDistributeKeys(table)) { // storageHandler != null && !acidDistributeColumns(table, DELETE).isEmpty() > return new SingleDeleteBranchMergeRewriter(db, conf, sqlGeneratorFactory) > } else { > return new SplitMergeRewriter(db, conf, sqlGeneratorFactory); > } > } > ``` Thanks! 1. Yes, it would work for native ACID and V2 too: the update's delete predicate and the delete clause's predicate never overlap, so a single branch with (A) OR (B) deletes the same rows, and writes one delete delta (or one set of position delete files) instead of two. I kept this PR to V3, where one delete branch is needed for correctness (one DV per data file), because the same change rewrites the plan of every native ACID and Iceberg V2 MERGE that has both clauses (`hive.split.update` is on by default). I'd propose doing it in a follow-up: make the single delete branch the default for every table, which also drops the V3-only condition from SplitMergeRewriter. I can open the JIRA if you agree. 2. With that follow-up there would be one implementation, so a second rewriter wouldn't be needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
