[
https://issues.apache.org/jira/browse/SPARK-48579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Milicevic updated SPARK-48579:
------------------------------------
Description:
OSS has single parser for SQL statements - SparkSqlParser.
However, in runtime we have additional parser - DatabricksSqlParser. This
parser is used for edge SQL rules, but there is no clear separation because
folks keep adding edge rules/features to the SparkSqlParser as well.
More details can be found in [this design doc
comment|[https://docs.google.com/document/d/1DIsMf2LQJvD4UC5JR1-YBH6Rv3UvGK3dD2QTOf_0SK4/edit?disco=AAABNCpDpdo]].
It seems like there is no reason not to merge these two parsers into one, but
it needs to be investigated first before refactoring.
was:
OSS has single parser for SQL statements - SparkSqlParser.
However, in runtime we have additional parser - DatabricksSqlParser. This
parser is used for edge SQL rules, but there is no clear separation because
folks keep adding edge rules/features to the SparkSqlParser as well.
More details can be found in [this design doc
comment|[https://docs.google.com/document/d/1DIsMf2LQJvD4UC5JR1-YBH6Rv3UvGK3dD2QTOf_0SK4/edit?disco=AAABNCpDpdo].]
It seems like there is no reason not to merge these two parsers into one, but
it needs to be investigated first before refactoring.
> [M1] Merge DatabricksSqlParser with SparkSqlParser
> --------------------------------------------------
>
> Key: SPARK-48579
> URL: https://issues.apache.org/jira/browse/SPARK-48579
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 4.0.0
> Reporter: David Milicevic
> Priority: Major
>
> OSS has single parser for SQL statements - SparkSqlParser.
> However, in runtime we have additional parser - DatabricksSqlParser. This
> parser is used for edge SQL rules, but there is no clear separation because
> folks keep adding edge rules/features to the SparkSqlParser as well.
> More details can be found in [this design doc
> comment|[https://docs.google.com/document/d/1DIsMf2LQJvD4UC5JR1-YBH6Rv3UvGK3dD2QTOf_0SK4/edit?disco=AAABNCpDpdo]].
>
> It seems like there is no reason not to merge these two parsers into one, but
> it needs to be investigated first before refactoring.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
