Hive uses spark-submit to launch the spark jobs - that's why the
assembly is needed.
On 03/16/2017 11:54 PM, Reynold Xin wrote:
Why do you need an assembly? Is there something preventing Hive from
depending on normal jars like all other applications?
On Thu, Mar 16, 2017 at 3:42 PM, Zoltan Haindrich <k...@rxd.hu
<mailto:k...@rxd.hu>> wrote:
Hello,
Hive needs a spark assembly to execute the HoS tests.
Until now…this assembly have been downloaded from an S3 bucket -
because this is not the best solution available, it sometimes
causes troubles and inconveniences...
We had a discussion about improving this; and the best option
would be to download spark-without-hive assembly from the maven
repository...but that opens up a few questions:
1) which project should publish it: Hive or Spark?
2) what should be the group-id? this artifact is only needed for
Hive...but it contains Spark code! :) so its in some kind of grey
zone...
3) how will we be able to get a spark-without-hive artifact for
2.0.0 - since that version is already released?
for more details:
https://issues.apache.org/jira/browse/HIVE-14735
<https://issues.apache.org/jira/browse/HIVE-14735>
What do you guys think?
cheers,
Zoltan
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
<mailto:dev-unsubscr...@spark.apache.org>
