srielau commented on code in PR #58317:
URL: https://github.com/apache/spark/pull/58317#discussion_r4027800923
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala:
##########
@@ -115,7 +115,10 @@ case class DataSourceV2Relation(
catalog: Option[CatalogPlugin],
identifier: Option[Identifier],
options: CaseInsensitiveStringMap,
- timeTravelSpec: Option[TimeTravelSpec] = None)
+ timeTravelSpec: Option[TimeTravelSpec] = None,
+ // Bound at analysis so sameResult / cache reuse distinguish preserve-only
vs standard
+ // CHAR/VARCHAR scans. None means the relation was not analyzed under
first-class types.
+ charVarcharScanMode: Option[CharVarcharScanMode] = None)
Review Comment:
Done: `WriteToDataSourceV2` now recaches through the same catalog-name /
catalog-less `refreshCache` path as batch V2 writes after a successful commit,
so PreserveNative and SparkStandard caches are rebuilt even when the
micro-batch target is unbound. Covered by `micro-batch V2 write recaches all
bound CHAR/VARCHAR scan modes`.
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala:
##########
@@ -41,7 +41,10 @@ case class LogicalRelation(
output: Seq[AttributeReference],
catalogTable: Option[CatalogTable],
override val isStreaming: Boolean,
- @transient stream: Option[SparkDataStream])
+ @transient stream: Option[SparkDataStream],
+ // Bound at analysis so sameResult / cache reuse distinguish preserve-only
vs standard
+ // CHAR/VARCHAR scans. None means the relation was not analyzed under
first-class types.
+ charVarcharScanMode: Option[CharVarcharScanMode])
Review Comment:
Done: `CommandUtils.recacheTableOrView` recaches V1 tables by catalog
identity (and catalog-less `BaseRelation` via `recacheByV1Relation`) and Hive
tables by qualified name before `recacheByPlan`, so one `refreshTable` rebuilds
every bound CHAR/VARCHAR variant. Covered by `refreshTable recaches all bound
CHAR/VARCHAR V1 scan modes`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
