Ryan Blue created SPARK-27108:
---------------------------------
Summary: Add parsed CreateTable plans to Catalyst
Key: SPARK-27108
URL: https://issues.apache.org/jira/browse/SPARK-27108
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.4.1
Reporter: Ryan Blue
The abstract Catalyst SQL AST builder cannot currently parse {{CREATE TABLE}}
commands. Creates are handled only by {{SparkSqlParser}} because the logical
plans are defined in the v1 datasource package
(org.apache.spark.sql.execution.datasources).
The {{SparkSqlParser}} mixes parsing with logic that is specific to v1, like
converting {{IF NOT EXISTS}} into a {{SaveMode}}. This makes it difficult (and
error-prone) to produce v2 plans because it requires converting the AST to v1
and the converting v1 to v2.
Instead, the catalyst parser should create plans that represent exactly what
was parsed, after validation like ensuring no duplicate clauses. Then those
plans should be converted to v1 or v2 plans in the analyzer. This structure
will avoid errors caused by multiple layers of translation and keeps v1 and v2
plans separate to ensure that v1 has no behavior changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
