cloud-fan commented on code in PR #58244:
URL: https://github.com/apache/spark/pull/58244#discussion_r4063743836
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##########
@@ -1881,11 +1882,11 @@ class Analyzer(
val newNotMatchedBySourceActions = m.notMatchedBySourceActions.map
{
case DeleteAction(deleteCondition) =>
val resolvedDeleteCondition = deleteCondition.map(
- resolveExpressionByPlanOutput(_, targetTable))
+ resolveExpressionByPlanOutput(_, targetTable,
includeLastResort = true))
Review Comment:
**Blocking (P1):** Please defer last-resort resolution for MERGE conditions
until schema evolution has reached its final target schema. On the first
analyzer pass, assignments have not yet made pending schema changes visible, so
a condition reference such as `new_col` can bind to a same-named SQL variable
here. On the next pass, `ResolveSchemaEvolution` reloads the target with
`new_col`, but the resolved `VariableReference` is not reconsidered; the
variable then wins over the target column and an update/delete predicate can
affect the wrong rows. Please gate the last-resort step on schema evolution
being disabled or being ready with no pending changes (across all MERGE
condition forms), and add a collision test that distinguishes per-row
evolved-column values from the variable value.
**Recommended change:** Delay last-resort resolution for MERGE conditions
until schema evolution is disabled or the command is ready and has no pending
schema changes, then add focused MERGE schema-evolution coverage where a newly
added target column collides with a SQL variable.
**Why this works:** Use the command's existing final-schema readiness facts
to keep condition attributes unresolved while assignments establish and apply
pending schema changes. On the fixed-point pass after the target relation is
reloaded, resolve ordinary target/source columns first and only then run
outer-reference and SQL-variable fallback.
**Scope:** Coordinate MERGE condition fallback with final target-schema
readiness and verify variable/column precedence across the schema-evolution
iteration boundary.
**Compatibility:** Ordinary target/source columns continue to win over outer
references and SQL variables, and missing names still fall back to variables
only after normal column resolution fails.
**Risks:** The readiness gate must still allow assignment resolution to make
schema evolution ready and must not prevent fixed-point convergence. Deferring
fallback must not suppress source-only condition variables once the final
target schema is established.
**Constraints:** Preserve table/source column precedence over outer
references and SQL variables. Keep assignment-value resolution outside this
PR's condition-only scope. Retain existing behavior for MERGE statements
without schema evolution and for schema-evolution commands with no pending
changes.
**Success:** A name that becomes a target column through the current MERGE
schema evolution resolves to that column before a same-named SQL variable in
every target-visible MERGE condition. SQL variables still resolve in all MERGE
condition families after the command reaches its final target schema. MERGE
behavior without schema evolution and assignment-value resolution remain
unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
