Re: spark-without-hive assembly for hive build/development purposes

2017-03-16 Thread Marcelo Vanzin 
The solution to this is being tracked in
https://issues.apache.org/jira/browse/HIVE-15302, although I haven't
seen activity in a while.

On Thu, Mar 16, 2017 at 3:42 PM, Zoltan Haindrich  wrote:
> Hello,
>
> Hive needs a spark assembly to execute the HoS tests.
> Until now…this assembly have been downloaded from an S3 bucket - because
> this is not the best solution available, it sometimes causes troubles and
> inconveniences...
>
> We had a discussion about improving this; and the best option would be to
> download spark-without-hive assembly from the maven repository...but that
> opens up a few questions:
>
> 1) which project should publish it: Hive or Spark?
> 2) what should be the group-id? this artifact is only needed for Hive...but
> it contains Spark code! :) so its in some kind of grey zone...
> 3) how will we be able to get a spark-without-hive artifact for 2.0.0 -
> since that version is already released?
>
> for more details:
> https://issues.apache.org/jira/browse/HIVE-14735
>
> What do you guys think?
>
> cheers,
> Zoltan
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>



-- 
Marcelo

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: spark-without-hive assembly for hive build/development purposes

2017-03-16 Thread Zoltan Haindrich 
Hive uses spark-submit to launch the spark jobs - that's why the 
assembly is needed.



On 03/16/2017 11:54 PM, Reynold Xin wrote:
Why do you need an assembly? Is there something preventing Hive from 
depending on normal jars like all other applications?


On Thu, Mar 16, 2017 at 3:42 PM, Zoltan Haindrich > wrote:


Hello,

Hive needs a spark assembly to execute the HoS tests.
Until now…this assembly have been downloaded from an S3 bucket -
because this is not the best solution available, it sometimes
causes troubles and inconveniences...

We had a discussion about improving this; and the best option
would be to download spark-without-hive assembly from the maven
repository...but that opens up a few questions:

1) which project should publish it: Hive or Spark?
2) what should be the group-id? this artifact is only needed for
Hive...but it contains Spark code! :) so its in some kind of grey
zone...
3) how will we be able to get a spark-without-hive artifact for
2.0.0 - since that version is already released?

for more details:
https://issues.apache.org/jira/browse/HIVE-14735


What do you guys think?

cheers,
Zoltan

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: spark-without-hive assembly for hive build/development purposes

2017-03-16 Thread Reynold Xin 
Why do you need an assembly? Is there something preventing Hive from
depending on normal jars like all other applications?

On Thu, Mar 16, 2017 at 3:42 PM, Zoltan Haindrich  wrote:

> Hello,
>
> Hive needs a spark assembly to execute the HoS tests.
> Until now…this assembly have been downloaded from an S3 bucket - because
> this is not the best solution available, it sometimes causes troubles and
> inconveniences...
>
> We had a discussion about improving this; and the best option would be to
> download spark-without-hive assembly from the maven repository...but that
> opens up a few questions:
>
> 1) which project should publish it: Hive or Spark?
> 2) what should be the group-id? this artifact is only needed for
> Hive...but it contains Spark code! :) so its in some kind of grey zone...
> 3) how will we be able to get a spark-without-hive artifact for 2.0.0 -
> since that version is already released?
>
> for more details:
> https://issues.apache.org/jira/browse/HIVE-14735
>
> What do you guys think?
>
> cheers,
> Zoltan
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>