Ryu Kobayashi created HIVE-30160:
------------------------------------
Summary: Add cardinality violation handling to Iceberg
copy-on-write MERGE
Key: HIVE-30160
URL: https://issues.apache.org/jira/browse/HIVE-30160
Project: Hive
Issue Type: Bug
Reporter: Ryu Kobayashi
h2. Summary
Iceberg copy-on-write MERGE does not currently perform the standard
cardinality-violation validation used by native MERGE.
When multiple source rows match the same target row, the copy-on-write rewrite
can apply multiple actions to the same target row instead of raising the
standard cardinality violation error. This behavior should be aligned with
native MERGE.
h2. Root cause
`CopyOnWriteMergeRewriter` currently does not add the standard
cardinality-validation branch implemented by `MergeRewriter`. The native MERGE
rewrite uses the `cardinality_violation` UDF and a dedicated branch writing to
the session-scoped `merge_tmp_table`.
The copy-on-write rewrite instead emits action and replacement-marker branches
directly into the target INSERT/UNION plan. Without a dedicated
cardinality-validation branch, duplicate source matches can flow through the
normal update or delete branches.
The cardinality validation must also account for:
- Iceberg spec ID and partition identity
- Data-file path and row position
- Partitioned and partition-evolved tables
- Row-lineage projections
- CBO-enabled and CBO-disabled execution
- NOT NULL and CHECK constraint processing
The validation branch must not be implemented as a full-width target row
containing NULL values if that would cause constraint enforcement to mask the
intended cardinality error.
h2. Reproduction steps
# Create an Iceberg v2 table configured for copy-on-write MERGE:
{code:java}
CREATE TABLE target (a int, b string) STORED BY ICEBERG TBLPROPERTIES
('format-version'='2', 'write.merge.mode'='copy-on-write'); {code}
# Insert one target row and duplicate source keys:
{code:java}
INSERT INTO target VALUES (1, 'original'); CREATE TABLE source (a int, b
string); INSERT INTO source VALUES (1, 'first'), (1, 'second');{code}
# Execute the MERGE:
{code:java}
MERGE INTO target t
USING source s
ON t.a = s.a
WHEN MATCHED THEN UPDATE SET b = s.b;{code}
h2. Impact / exposure
Without cardinality validation:
- A target row can be updated more than once by a single MERGE statement.
- The final value may depend on execution order.
- The reported affected-row count can be inflated.
- Copy-on-write and native MERGE can produce different results for the same
input.
- Target constraints may mask the intended cardinality error if validation is
added at the wrong point in the plan.
The issue affects Iceberg copy-on-write MERGE statements with duplicate source
matches. Insert-only MERGE statements should retain their existing behavior
because they do not modify matched target rows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
