[
https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543733#comment-15543733
]
Reynold Xin edited comment on SPARK-17767 at 10/3/16 11:22 PM:
---------------------------------------------------------------
Guys - while I think this is very useful to do, I'm going to mark this as
"later" for now. The reason is that there are a lot of things to consider
before making this switch, including:
- The ExternalCatalog API is currently internal, and we can't just make it
public without thinking about the consequences and whether this API is
maintainable in the long run.
- SPARK-15777 We need to design this in the context of catalog federation and
persistence.
- SPARK-15691 Refactoring of how we integrate with Hive.
This is not as simple as just submitting a PR to make it pluggable.
was (Author: rxin):
Guys - while I think this is very useful to do, I'm going to mark this as
"later" for now. The reason is that there are a lot of things to consider
before making this switch, including:
- The ExternalCatalog API is currently internal, and we can't just make it
public without thinking about the consequences.
- SPARK-15777 We need to design this in the context of catalog federation and
persistence.
- SPARK-15691 Refactoring of how we integrate with Hive.
> Spark SQL ExternalCatalog API custom implementation support
> -----------------------------------------------------------
>
> Key: SPARK-17767
> URL: https://issues.apache.org/jira/browse/SPARK-17767
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Alex Liu
>
> There is no way/easy way to configure Spark to use customized
> ExternalCatalog. Internal source code is hardcoded to use either hive or
> in-memory metastore. Spark SQL thriftserver is hardcoded to use
> HiveExternalCatalog. We should be able to create a custom external catalog
> and thriftserver should be able to use it. Potentially Spark SQL thriftserver
> shouldn't depend on Hive thriftserer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
