andreaschat-db commented on code in PR #55623:
URL: https://github.com/apache/spark/pull/55623#discussion_r3208335419
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##########
@@ -1064,6 +1064,8 @@ class Analyzer(
table: NamedRelation,
withNewTable: NamedRelation => LogicalPlan): LogicalPlan = {
table match {
+ // Streaming write targets are constructed with isStreaming=false even
inside a streaming
+ // query, because the sink is a regular batch write destination.
case u: UnresolvedRelation if !u.isStreaming =>
Review Comment:
Removed stale comment and reverted refactor.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##########
@@ -1124,8 +1123,17 @@ class Analyzer(
case other => i.copy(table = other)
}
- case write: StreamingV2WriteCommand =>
- resolveWriteTarget(write, write.table, write.withNewTable)
+ case write: V2StreamingWriteCommand =>
+ write.table match {
+ case ref: V2TableReference =>
+ relationResolution.resolveReference(ref) match {
+ case r: NamedRelation => write.withNewTable(r)
+ case other => throw SparkException.internalError(
Review Comment:
All codepaths in `resolveReference` return `DataSourceV2Relation` which is a
`NamedRelation`. Other call sites return `LogicalPlan`, so they do not need to
explicitly validate the return type. I would keep the validation to be
foolproof against future changes.
The alternative is to tighten the signature of `resolveReference` and return
`NamedRelation` instead. Is there a particular reason, similar functions such
as `loadRelation`/`getOrLoadRelation`/`adaptCachedRelation` return
`LogicalPlan`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
