[
https://issues.apache.org/jira/browse/HIVE-30160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi reassigned HIVE-30160:
------------------------------------
Assignee: Ryu Kobayashi
> Add cardinality violation handling to Iceberg copy-on-write MERGE
> -----------------------------------------------------------------
>
> Key: HIVE-30160
> URL: https://issues.apache.org/jira/browse/HIVE-30160
> Project: Hive
> Issue Type: Bug
> Reporter: Ryu Kobayashi
> Assignee: Ryu Kobayashi
> Priority: Major
>
> h2. Summary
> Iceberg copy-on-write MERGE does not currently perform the standard
> cardinality-violation validation used by native MERGE.
> When multiple source rows match the same target row, the copy-on-write
> rewrite can apply multiple actions to the same target row instead of raising
> the standard cardinality violation error. This behavior should be aligned
> with native MERGE.
> h2. Root cause
> `CopyOnWriteMergeRewriter` currently does not add the standard
> cardinality-validation branch implemented by `MergeRewriter`. The native
> MERGE rewrite uses the `cardinality_violation` UDF and a dedicated branch
> writing to the session-scoped `merge_tmp_table`.
> The copy-on-write rewrite instead emits action and replacement-marker
> branches directly into the target INSERT/UNION plan. Without a dedicated
> cardinality-validation branch, duplicate source matches can flow through the
> normal update or delete branches.
> The cardinality validation must also account for:
> - Iceberg spec ID and partition identity
> - Data-file path and row position
> - Partitioned and partition-evolved tables
> - Row-lineage projections
> - CBO-enabled and CBO-disabled execution
> - NOT NULL and CHECK constraint processing
> The validation branch must not be implemented as a full-width target row
> containing NULL values if that would cause constraint enforcement to mask the
> intended cardinality error.
> h2. Reproduction steps
> # Create an Iceberg v2 table configured for copy-on-write MERGE:
> {code:java}
> CREATE TABLE target (a int, b string) STORED BY ICEBERG TBLPROPERTIES
> ('format-version'='2', 'write.merge.mode'='copy-on-write'); {code}
> # Insert one target row and duplicate source keys:
> {code:java}
> INSERT INTO target VALUES (1, 'original'); CREATE TABLE source (a int, b
> string); INSERT INTO source VALUES (1, 'first'), (1, 'second');{code}
> # Execute the MERGE:
> {code:java}
> MERGE INTO target t
> USING source s
> ON t.a = s.a
> WHEN MATCHED THEN UPDATE SET b = s.b;{code}
>
> h2. Impact / exposure
> Without cardinality validation:
> - A target row can be updated more than once by a single MERGE statement.
> - The final value may depend on execution order.
> - The reported affected-row count can be inflated.
> - Copy-on-write and native MERGE can produce different results for the same
> input.
> - Target constraints may mask the intended cardinality error if validation
> is added at the wrong point in the plan.
> The issue affects Iceberg copy-on-write MERGE statements with duplicate
> source matches. Insert-only MERGE statements should retain their existing
> behavior because they do not modify matched target rows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
