Get it. Thanks Reynold and Andrew! Jianshi
On Thu, Feb 12, 2015 at 12:25 AM, Andrew Or <and...@databricks.com> wrote: > Hi Jianshi, > > For YARN, there may be an issue with how a recently patch changes the > accessibility of the shuffle files by the external shuffle service: > https://issues.apache.org/jira/browse/SPARK-5655. It is likely that you > will hit this with 1.2.1, actually. For this reason I would have to > recommend that you use 1.2.2 when it is released, but for now you should > use 1.2.0 for this specific use case. > > -Andrew > > 2015-02-10 23:38 GMT-08:00 Reynold Xin <r...@databricks.com>: > > I think we made the binary protocol compatible across all versions, so you >> should be fine with using any one of them. 1.2.1 is probably the best since >> it is the most recent stable release. >> >> On Tue, Feb 10, 2015 at 8:43 PM, Jianshi Huang <jianshi.hu...@gmail.com> >> wrote: >> >>> Hi, >>> >>> I need to use branch-1.2 and sometimes master builds of Spark for my >>> project. However the officially supported Spark version by our Hadoop admin >>> is only 1.2.0. >>> >>> So, my question is which version/build of spark-yarn-shuffle.jar should >>> I use that works for all four versions? (1.2.0, 1.2.1, 1.2,2, 1.3.0) >>> >>> Thanks, >>> -- >>> Jianshi Huang >>> >>> LinkedIn: jianshi >>> Twitter: @jshuang >>> Github & Blog: http://huangjs.github.com/ >>> >> >> > -- Jianshi Huang LinkedIn: jianshi Twitter: @jshuang Github & Blog: http://huangjs.github.com/
