Re: RFC: packaging Spark without assemblies

> On 25 Sep 2015, at 19:11, Marcelo Vanzin wrote: > > - People who ship the assembly with their application. As Matei > suggested (and I agree), that is kinda weird. But currently that is > the easiest way to embed Spark and get, for example, the YARN backend > working.

Re: RFC: packaging Spark without assemblies

On Wed, Sep 23, 2015 at 4:43 PM, Patrick Wendell wrote: > For me a key step in moving away would be to fully audit/understand > all compatibility implications of removing it. If other people are > supportive of this plan I can offer to help spend some time thinking > about any

RFC: packaging Spark without assemblies

Hey all, This is something that we've discussed several times internally, but never really had much time to look into; but as time passes by, it's increasingly becoming an issue for us and I'd like to throw some ideas around about how to fix it. So, without further ado:

Re: RFC: packaging Spark without assemblies

I think it would be a big improvement to get rid of it. It's not how jars are supposed to be packaged and it has caused problems in many different context over the years. For me a key step in moving away would be to fully audit/understand all compatibility implications of removing it. If other