Re: spark-0.8.0 and hadoop-2.1.0-beta

2013-10-29 Thread Matei Zaharia
I’m curious, Viren, do you have a patch you could post to build this against YARN 2.1 / 2.2? It would be nice to see how big the changes are. Matei On Sep 30, 2013, at 10:14 AM, viren kumar vire...@gmail.com wrote: I was able to get Spark 0.8.0 to compile with Hadoop/Yarn 2.1.0-beta, by

RE: spark-0.8.0 and hadoop-2.1.0-beta

2013-10-29 Thread Liu, Raymond
don't find the replacement or they are not need anymore. Best Regards, Raymond Liu From: Matei Zaharia [mailto:matei.zaha...@gmail.com] Sent: Wednesday, October 30, 2013 2:35 AM To: user@spark.incubator.apache.org Subject: Re: spark-0.8.0 and hadoop-2.1.0-beta I'm curious, Viren, do you have

Re: spark-0.8.0 and hadoop-2.1.0-beta

2013-09-29 Thread Matei Zaharia
Hi Terence, YARN's API changed in an incompatible way in Hadoop 2.1.0, so I'd suggest sticking with 2.0.x for now. We may create a different branch for this version. Unfortunately due to the API change it may not be possible to support this version while also supporting other widely-used