[
https://issues.apache.org/jira/browse/SPARK-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15233474#comment-15233474
]
Steve Loughran commented on SPARK-10793:
----------------------------------------
I've not gone near it for a while; what's left is setting up hive thrift for
service subclass, rather than the abuse of reflection that's needed today.
It's easily enough added, to the code, there's just the risk of the fork
diverging more from ASF hive.
> Make spark's use/subclassing of hive more maintainable
> ------------------------------------------------------
>
> Key: SPARK-10793
> URL: https://issues.apache.org/jira/browse/SPARK-10793
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.5.0
> Reporter: Steve Loughran
>
> The latest spark/hive integration round has closed the gap with Hive
> versions, but the integration is still pretty complex
> # SparkSQL has deep hooks into the parser
> # hivethriftserver uses "aggressive reflection" to inject spark classes into
> the Hive base classes.
> # there's a separate org.sparkproject.hive JAR to isolate Kryo versions while
> avoiding the hive uberjar with all its dependencies getting into the spark
> uberjar.
> We can improve this with some assistance from the other projects, even though
> no guarantees of stability of things like the parser and thrift server APIs
> are likely in the near future
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
