Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16613#discussion_r96560936
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ---
@@ -207,29 +209,26 @@ case class CreateViewCommand(
}
/**
- * Returns a [[CatalogTable]] that can be used to save in the catalog.
This comment canonicalize
- * SQL based on the analyzed plan, and also creates the proper schema
for the view.
+ * Returns a [[CatalogTable]] that can be used to save in the catalog.
Generate the view-specific
+ * properties(e.g. view default database, view query output column
names) and store them as
+ * properties in the CatalogTable, and also creates the proper schema
for the view.
*/
- private def prepareTable(sparkSession: SparkSession, aliasedPlan:
LogicalPlan): CatalogTable = {
- val viewSQL: String = new SQLBuilder(aliasedPlan).toSQL
-
- // Validate the view SQL - make sure we can parse it and analyze it.
- // If we cannot analyze the generated query, there is probably a bug
in SQL generation.
- try {
- sparkSession.sql(viewSQL).queryExecution.assertAnalyzed()
- } catch {
- case NonFatal(e) =>
- throw new RuntimeException(s"Failed to analyze the canonicalized
SQL: $viewSQL", e)
+ private def prepareTable(session: SparkSession, aliasedPlan:
LogicalPlan): CatalogTable = {
+ if (originalText.isEmpty) {
+ throw new AnalysisException(
+ "It is not allowed to create a persisted view from the Dataset
API")
}
+ val newProperties = generateViewProperties(properties, session,
originalText.get)
+
CatalogTable(
identifier = name,
tableType = CatalogTableType.VIEW,
storage = CatalogStorageFormat.empty,
schema = aliasedPlan.schema,
- properties = properties,
+ properties = newProperties,
viewOriginalText = originalText,
- viewText = Some(viewSQL),
+ viewText = originalText,
--- End diff --
something we can clean up: Hive will expand the view text, so it needs 2
fields: originalText and viewText. Since we don't expand the view text, but
only add table properties, I think we only need a single field `viewText` in
CatalogTable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
