[
https://issues.apache.org/jira/browse/SPARK-12855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207765#comment-15207765
]
Joseph Levin commented on SPARK-12855:
--------------------------------------
Reynold - We would except grudgingly. Of course we don't want it to break each
build but some churn we could live with. I guess part of my push back is I
believe it is closing off one of the most powerful aspects of SQL on Spark.
Writing an extensible parser is in itself a large undertaking. (I can only
think of 2 others that have similar flexibility, Antlr, which can be
implemented to be extensible but isn't fully out of the box and MS's Roslyn.)
Marrying an extensible parser to Spark's distributed cross platform
functionality is, as far as I have been able find, unique. For this project's
initial work we didn't even need to be in the hadoop/big data space; our
initial set of data sources all support jdbc. We did require a query an engine
that could handle a single request to multiple data sources and give us the
ability to rewrite the request on the fly. Spark is the only toolset we found
that met both those needs.
As a side note, it was the data bricks Deep Dive article on Catalyst that, I
believe, you cowrote that led us to try Spark for this problem.
> Remove parser pluggability
> --------------------------
>
> Key: SPARK-12855
> URL: https://issues.apache.org/jira/browse/SPARK-12855
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Reynold Xin
> Assignee: Reynold Xin
> Fix For: 2.0.0
>
>
> This pull request removes the public developer parser API for external
> parsers. Given everything a parser depends on (e.g. logical plans and
> expressions) are internal and not stable, external parsers will break with
> every release of Spark. It is a bad idea to create the illusion that Spark
> actually supports pluggable parsers. In addition, this also reduces
> incentives for 3rd party projects to contribute parse improvements back to
> Spark.
> The number of applications that are using this feature is small (as far as I
> know it came down from two to one as of Jan 2016, and will be 0 once we have
> better ansi SQL support).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
