[GitHub] [spark] johanl-db commented on a diff in pull request #38400: [SPARK-40921][SQL] Add WHEN NOT MATCHED BY SOURCE clause to MERGE INTO
johanl-db commented on code in PR #38400:
URL: https://github.com/apache/spark/pull/38400#discussion_r1009246467
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##
@@ -1583,21 +1599,20 @@ class Analyzer(override val catalogManager:
CatalogManager)
def resolveAssignments(
assignments: Seq[Assignment],
mergeInto: MergeIntoTable,
-resolveValuesWithSourceOnly: Boolean): Seq[Assignment] = {
+resolveValuesFrom: LogicalPlan): Seq[Assignment] = {
assignments.map { assign =>
val resolvedKey = assign.key match {
case c if !c.resolved =>
resolveMergeExprOrFail(c, Project(Nil, mergeInto.targetTable))
case o => o
}
val resolvedValue = assign.value match {
- // The update values may contain target and/or source references.
case c if !c.resolved =>
-if (resolveValuesWithSourceOnly) {
- resolveMergeExprOrFail(c, Project(Nil, mergeInto.sourceTable))
-} else {
- resolveMergeExprOrFail(c, mergeInto)
+val resolveFromChildren = resolveValuesFrom match {
+ case m: MergeIntoTable => m
+ case p => Project(Nil, p)
Review Comment:
I went the enum route, introduced MergeResolvePolicy.
##
core/src/main/resources/error/error-classes.json:
##
@@ -589,6 +589,24 @@
"More than one row returned by a subquery used as an expression."
]
},
+ "NON_LAST_MATCHED_CLAUSE_OMIT_CONDITION" : {
+"message" : [
+ "When there are more than one MATCHED clauses in a MERGE statement, only
the last MATCHED clause can omit the condition."
+],
+"sqlState" : "42000"
+ },
+ "NON_LAST_NOT_MATCHED_BY_SOURCE_CLAUSE_OMIT_CONDITION" : {
+"message" : [
+ "When there are more than one NOT MATCHED BY SOURCE clauses in a MERGE
statement, only the last NOT MATCHED BY SOURCE clause can omit the condition."
+],
+"sqlState" : "42000"
+ },
+ "NON_LAST_NOT_MATCHED_BY_TARGET_CLAUSE_OMIT_CONDITION" : {
+"message" : [
+ "When there are more than one NOT MATCHED clauses in a MERGE statement,
only the last NOT MATCHED clause can omit the condition."
Review Comment:
Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] johanl-db commented on a diff in pull request #38400: [SPARK-40921][SQL] Add WHEN NOT MATCHED BY SOURCE clause to MERGE INTO
johanl-db commented on code in PR #38400:
URL: https://github.com/apache/spark/pull/38400#discussion_r1008270419
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##
@@ -451,7 +451,22 @@ class AstBuilder extends SqlBaseParserBaseVisitor[AnyRef]
with SQLConfHelper wit
}
}
}
-if (matchedActions.isEmpty && notMatchedActions.isEmpty) {
+val notMatchedBySourceActions = ctx.notMatchedBySourceClause().asScala.map
{
+ clause => {
+val notMatchedBySourceAction = clause.notMatchedBySourceAction()
+if (notMatchedBySourceAction.DELETE() != null) {
+ DeleteAction(Option(clause.notMatchedBySourceCond).map(expression))
+} else if (notMatchedBySourceAction.UPDATE() != null) {
+ val condition = Option(clause.notMatchedBySourceCond).map(expression)
+ UpdateAction(condition,
+
withAssignments(clause.notMatchedBySourceAction().assignmentList()))
+} else {
+ // It should not be here.
+ throw
QueryParsingErrors.unrecognizedNotMatchedBySourceActionError(clause)
Review Comment:
Parsing will fail if a user tries to use an invalid action. We don't cover
this error in tests since it can't surface unless there's a bug. I replaced it
(and similar error for matched / not matched) with internal errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] johanl-db commented on a diff in pull request #38400: [SPARK-40921][SQL] Add WHEN NOT MATCHED BY SOURCE clause to MERGE INTO
johanl-db commented on code in PR #38400:
URL: https://github.com/apache/spark/pull/38400#discussion_r1008270419
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##
@@ -451,7 +451,22 @@ class AstBuilder extends SqlBaseParserBaseVisitor[AnyRef]
with SQLConfHelper wit
}
}
}
-if (matchedActions.isEmpty && notMatchedActions.isEmpty) {
+val notMatchedBySourceActions = ctx.notMatchedBySourceClause().asScala.map
{
+ clause => {
+val notMatchedBySourceAction = clause.notMatchedBySourceAction()
+if (notMatchedBySourceAction.DELETE() != null) {
+ DeleteAction(Option(clause.notMatchedBySourceCond).map(expression))
+} else if (notMatchedBySourceAction.UPDATE() != null) {
+ val condition = Option(clause.notMatchedBySourceCond).map(expression)
+ UpdateAction(condition,
+
withAssignments(clause.notMatchedBySourceAction().assignmentList()))
+} else {
+ // It should not be here.
+ throw
QueryParsingErrors.unrecognizedNotMatchedBySourceActionError(clause)
Review Comment:
Parsing will fail if a user tries to use an invalid action. We don't cover
this error in tests since it can't surface unless there's a but. I replaced it
(and similar error for matched / not matched) by internal errors.
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##
@@ -1563,13 +1563,29 @@ class Analyzer(override val catalogManager:
CatalogManager)
}
InsertAction(
resolvedInsertCondition,
- resolveAssignments(assignments, m,
resolveValuesWithSourceOnly = true))
+ resolveAssignments(assignments, m, resolveValuesFrom =
sourceTable))
case o => o
}
+val newNotMatchedBySourceActions = m.notMatchedBySourceActions.map
{
+ case DeleteAction(deleteCondition) =>
+val resolvedDeleteCondition = deleteCondition.map(
+ resolveExpressionByPlanChildren(_, Project(Nil,
targetTable)))
Review Comment:
Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
