Re: spark-without-hive assembly for hive build/development purposes

2017-03-16 Thread Marcelo Vanzin
The solution to this is being tracked in https://issues.apache.org/jira/browse/HIVE-15302, although I haven't seen activity in a while. On Thu, Mar 16, 2017 at 3:42 PM, Zoltan Haindrich wrote: > Hello, > > Hive needs a spark assembly to execute the HoS tests. > Until now…this

Re: spark-without-hive assembly for hive build/development purposes

2017-03-16 Thread Zoltan Haindrich
Hive uses spark-submit to launch the spark jobs - that's why the assembly is needed. On 03/16/2017 11:54 PM, Reynold Xin wrote: Why do you need an assembly? Is there something preventing Hive from depending on normal jars like all other applications? On Thu, Mar 16, 2017 at 3:42 PM, Zoltan

Re: spark-without-hive assembly for hive build/development purposes

2017-03-16 Thread Reynold Xin
Why do you need an assembly? Is there something preventing Hive from depending on normal jars like all other applications? On Thu, Mar 16, 2017 at 3:42 PM, Zoltan Haindrich wrote: > Hello, > > Hive needs a spark assembly to execute the HoS tests. > Until now…this assembly have been

spark-without-hive assembly for hive build/development purposes

2017-03-16 Thread Zoltan Haindrich
Hello, Hive needs a spark assembly to execute the HoS tests. Until now…this assembly have been downloaded from an S3 bucket - because this is not the best solution available, it sometimes causes troubles and inconveniences... We had a discussion about improving this; and the best option

Re: SPIP docs are live

2017-03-16 Thread Matei Zaharia
Yup, thanks everyone and Cody in particular for putting this together. I think it will help a lot. Matei > On Mar 16, 2017, at 1:57 PM, Joseph Bradley wrote: > > Awesome! Thanks for pushing this through, Cody. > Joseph > > On Sun, Mar 12, 2017 at 1:18 AM, Sean Owen

Re: SPIP docs are live

2017-03-16 Thread Joseph Bradley
Awesome! Thanks for pushing this through, Cody. Joseph On Sun, Mar 12, 2017 at 1:18 AM, Sean Owen wrote: > http://spark.apache.org/improvement-proposals.html > > (Thanks Cody!) > > We should use this process where appropriate now, and we can refine it > further if needed. >

Re: Should we consider a Spark 2.1.1 release?

2017-03-16 Thread Nick Pentreath
Spark 1.5.1 had 87 issues fix version 1 month after 1.5.0. Spark 1.6.1 had 123 issues 2 months after 1.6.0 2.0.1 was larger (317 issues) at 3 months after 2.0.0 - makes sense due to how large a release it was. We are at 185 for 2.1.1 and 3 months after (and not released yet so it could slip