Are you setting -Pyarn-alpha? ./sbt/sbt -Pyarn-alpha, followed by
projects, shows it as a module. You should only build yarn-stable
*or* yarn-alpha at any given time.
I don't remember the modules changing in a while. 'yarn-alpha' is for
YARN before it stabilized, circa early Hadoop 2.0.x.
To add, we've made some effort to yarn-alpha to work with the 2.0.x line,
but this was a time when YARN went through wild API changes. The only line
that the yarn-alpha profile is guaranteed to work against is the 0.23 line.
On Thu, Jul 17, 2014 at 12:40 AM, Sean Owen so...@cloudera.com wrote:
@Sean and @Sandy
Thanks for the reply. I used to be able to see yarn-alpha and yarn
directories which corresponding to the modules.
I guess due to the recent SparkBuild.scala changes, I did not see
yarn-alpha (by default) and I thought yarn-alpha is renamed to yarn and
yarn-stable is the
Looks like a real problem. I see it too. I think the same workaround
found in ClientBase.scala needs to be used here. There, the fact that
this field can be a String or String[] is handled explicitly. In fact
I think you can just call to ClientBase for this? PR it, I say.
On Thu, Jul 17, 2014 at
OK I will create PR.
thanks
On Thu, Jul 17, 2014 at 7:58 AM, Sean Owen so...@cloudera.com wrote:
Looks like a real problem. I see it too. I think the same workaround
found in ClientBase.scala needs to be used here. There, the fact that
this field can be a String or String[] is handled
Hi, Sandy
We do have some issue with this. The difference is in Yarn-Alpha and
Yarn Stable ( I noticed that in the latest build, the module name has
changed,
yarn-alpha -- yarn
yarn -- yarn-stable
)
For example: MRJobConfig.class
the field:
DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH
Hmm
looks like a Build script issue:
I run the command with :
sbt/sbt clean *yarn/*test:compile
but errors came from
[error] 40 errors found
[error] (*yarn-stable*/compile:compile) Compilation failed
Chester
On Wed, Jul 16, 2014 at 5:18 PM, Chester Chen ches...@alpinenow.com wrote:
Hi,
Looking further, the yarn and yarn-stable are both for the stable version
of Yarn, that explains the compilation errors when using 2.0.5-alpha
version of hadoop.
the module yarn-alpha ( although is still on SparkBuild.scala), is no
longer there in sbt console.
projects
[info] In
