[
https://issues.apache.org/jira/browse/HIVE-30147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi reassigned HIVE-30147:
------------------------------------
Assignee: Ryu Kobayashi
> Align Iceberg copy-on-write MERGE semantics with native MERGE
> -------------------------------------------------------------
>
> Key: HIVE-30147
> URL: https://issues.apache.org/jira/browse/HIVE-30147
> Project: Hive
> Issue Type: Bug
> Reporter: Ryu Kobayashi
> Assignee: Ryu Kobayashi
> Priority: Major
>
> h2. Summary
> The Iceberg copy-on-write MERGE rewrite does not fully preserve the semantics
> of the native MERGE implementation.
> The affected areas include NULL-key handling, source-only row detection, SQL
> three-valued predicate evaluation, `WHEN MATCHED` clause ordering, and
> cardinality-violation handling. These issues should be corrected so that
> copy-on-write MERGE behaves consistently with native MERGE.
> h2. Root cause
> The current implementation in `CopyOnWriteMergeRewriter` rewrites MERGE
> statements into a source CTE and multiple `UNION ALL` branches.
> Several parts of the rewrite rely on derived predicates and string-level
> predicate composition:
> - `MergeSemanticAnalyzer.OnClauseAnalyzer` derives source-only predicates
> from NULL checks on target join columns.
> - `CopyOnWriteMergeRewriter` reuses those predicates for the INSERT and
> carry-over branches.
> - Target/source presence is not represented by an explicit join-side marker
> such as the target file path.
> - Multiple predicates are concatenated with `AND` and `OR`, which can change
> their effective precedence when the original expressions contain OR operators.
> - Later `WHEN MATCHED` clauses are gated with `NOT(previousPredicate)`,
> which does not implement SQL `IS NOT TRUE` semantics when the previous
> predicate evaluates to `UNKNOWN`.
> - The copy-on-write rewrite does not currently use the standard
> cardinality-violation branch implemented by `MergeRewriter`.
> h2. Confirmed / observed behavior
> The following cases require regression coverage and correction:
> 1. A target row whose join key is NULL can be mistaken for a source-only row
> when another row in the same data file is matched and the file is rewritten.
> 2. A target-only row with NULL join keys can satisfy the derived source-only
> predicate and produce an unintended INSERT.
> 3. A matched predicate that evaluates to `UNKNOWN` can prevent a later `WHEN
> MATCHED` clause from executing.
> 4. Unparenthesized combinations of `ON` predicates and clause predicates can
> cause source-only rows or unrelated target rows to enter the
> replacement-marker path.
> 5. Multiple source rows matching the same target row can produce duplicate
> CoW updates or incorrect affected-row behavior instead of the standard
> cardinality violation.
> 6. On constrained targets, the full-width CoW cardinality branch may interact
> with NOT NULL/CHECK enforcement before the intended cardinality error is
> reported. This requires explicit validation with both CBO enabled and
> disabled.
> h2. Impact / exposure
> These issues affect the correctness and compatibility of Iceberg
> copy-on-write MERGE statements.
> Potential effects include:
> - Silent loss of legitimate target rows.
> - Unintended insertion of NULL or source-derived rows.
> - Different results from native MERGE for predicates involving NULL or OR
> expressions.
> - Duplicate or nondeterministic updates when source cardinality is invalid.
> - Cardinality errors being masked by constraint violations.
> - Inconsistent behavior depending on CBO configuration.
> This issue is separate from HIVE-30064's affected-row reporting scope.
> HIVE-30064 tracks logical affected-row counts for CoW DML; this issue tracks
> the broader correctness and semantic alignment of CoW MERGE rewriting.
> The implementation should preserve native MERGE semantics and include
> regression coverage for the affected cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
