[
https://issues.apache.org/jira/browse/SPARK-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15233108#comment-15233108
]
Josh Rosen commented on SPARK-10793:
------------------------------------
The Kryo part of this seems to be addressed by my reintroduction of the
shading. What's the status of the rest of these subtasks / this JIRA as a
whole? I don't think we want to run vanilla Hive 1.2.1 for our execution Hive
due to the protobuf shading requirement. What's left to do here / which parts
are still relevant?
> Make spark's use/subclassing of hive more maintainable
> ------------------------------------------------------
>
> Key: SPARK-10793
> URL: https://issues.apache.org/jira/browse/SPARK-10793
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.5.0
> Reporter: Steve Loughran
>
> The latest spark/hive integration round has closed the gap with Hive
> versions, but the integration is still pretty complex
> # SparkSQL has deep hooks into the parser
> # hivethriftserver uses "aggressive reflection" to inject spark classes into
> the Hive base classes.
> # there's a separate org.sparkproject.hive JAR to isolate Kryo versions while
> avoiding the hive uberjar with all its dependencies getting into the spark
> uberjar.
> We can improve this with some assistance from the other projects, even though
> no guarantees of stability of things like the parser and thrift server APIs
> are likely in the near future
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
