Hi Matt, I'm not sure whether we have documented compatibility guidelines here. However, a strong goal is to keep the external shuffle service compatible so that many versions of Spark can run against the same shuffle service.
-Sandy On Wed, Mar 25, 2015 at 6:44 PM, Matt Cheah <mch...@palantir.com> wrote: > Hi everyone, > > I am considering moving from Spark-Standalone to YARN. The context is that > there are multiple Spark applications that are using different versions of > Spark that all want to use the same YARN cluster. > > My question is: if I use a single Spark YARN shuffle service jar on the > Node Manager, will the service work properly with all of the Spark > applications, regardless of the specific versions of the applications? Or, > is it it the case that, if I want to use the external shuffle service, I > need to have all of my applications using the same version of Spark? > > Thanks, > > -Matt Cheah >
