On Wed, Sep 23, 2015 at 4:43 PM, Patrick Wendell <pwend...@gmail.com> wrote: > For me a key step in moving away would be to fully audit/understand > all compatibility implications of removing it. If other people are > supportive of this plan I can offer to help spend some time thinking > about any potential corner cases, etc.
Thanks Patrick (and all the others) who commented on the document. For BC, I think there are two main cases: - People who ship the assembly with their application. As Matei suggested (and I agree), that is kinda weird. But currently that is the easiest way to embed Spark and get, for example, the YARN backend working. There are ways around that but they are tricky. The code changes I propose would make that much easier to do without the need for an assembly. - People who somehow depend on the layout of the Spark distribution. Meaning they expect a "lib/" directory with an assembly in there matching a specific file name pattern. Although I kinda consider that to be an invalid use case (as in "you're doing it wrong"). One potential way to avoid it is to do the work to make the assemblies unnecessary, but not get rid of them, at least at first. Maybe a build profile or an argument in make-distribution.sh to enable or disable them as desired. -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
