The question is really: will Spark 1.1 work with a particular version of YARN? many, but not all versions of YARN are supported. The "stable" versions are (2.2.x+). Before that, support is patchier, and in fact has been removed in Spark 1.3.
The "yarn" profile supports "YARN stable" which is about 2.2.x and onwards. The "yarn-alpha" profile should work for YARN about 0.23.x. 2.0.x and 2.1.x were a sort of "beta" period and I recall that yarn-alpha works with some of it, but not all, and there is no yarn-beta profile. I believe early CDH 4.x has basically "YARN beta". Later 4.x has stable. I think I'd try the yarn-alpha profile and see if it compiles. But the version of YARN in that release may well be among those that fall in the gap between "alpha" and "stable" support. Thankfully things got a lot more stable past Hadoop / YARN 2.2 or so, so it far more just works without version issues. And CDH 5 is based on Hadoop 2.3 and then 2.5, so you should be much more able to build whatever versions together that you want. CDH 5.1.x ships Spark 1.0.x. There should be no problem using 1.1.x, 1.2.x, etc. with it; you just need to make and support your own binaries. 5.2.x has 1.1.x; 5.3.x will have 1.2.x. On Thu, Dec 18, 2014 at 9:18 AM, Canoe <canoe...@gmail.com> wrote: > I did not compile spark 1.1.0 source code on CDH4.3.0 with yarn successfully. > Does it support CDH4.3.0 with yarn ? > And will spark 1.2.0 support CDH5.1.2? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Can-Spark-1-0-2-run-on-CDH-4-3-0-with-yarn-And-Will-Spark-1-2-0-support-CDH5-1-2-with-yarn-tp20760.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
