Great - I think that should work, but if there are any issues we can
definitely fix them up.
On Sat, Nov 8, 2014 at 12:47 AM, Sean Owen wrote:
> Oops, that was my mistake. I moved network/shuffle into yarn, when
> it's just that network/yarn should be removed from yarn-alpha. That
> makes yarn-al
Oops, that was my mistake. I moved network/shuffle into yarn, when
it's just that network/yarn should be removed from yarn-alpha. That
makes yarn-alpha work. I'll run tests and open a quick JIRA / PR for
the change.
On Sat, Nov 8, 2014 at 8:23 AM, Patrick Wendell wrote:
> This second error is som
I think you might be conflating two things. The first error you posted
was because YARN didn't standardize the shuffle API in alpha versions
so our spark-network-yarn module won't compile. We should just disable
that module if yarn alpha is used. spark-network-yarn is a leaf in the
intra-module dep
Hm. Problem is, core depends directly on it:
[error]
/Users/srowen/Documents/spark/core/src/main/scala/org/apache/spark/SecurityManager.scala:25:
object sasl is not a member of package org.apache.spark.network
[error] import org.apache.spark.network.sasl.SecretKeyHolder
[error]
I bet it doesn't work. +1 on isolating it's inclusion to only the
newer YARN API's.
- Patrick
On Fri, Nov 7, 2014 at 11:43 PM, Sean Owen wrote:
> I noticed that this doesn't compile:
>
> mvn -Pyarn-alpha -Phadoop-0.23 -Dhadoop.version=0.23.7 -DskipTests clean
> package
>
> [error] warning: [opt
I noticed that this doesn't compile:
mvn -Pyarn-alpha -Phadoop-0.23 -Dhadoop.version=0.23.7 -DskipTests clean package
[error] warning: [options] bootstrap class path not set in conjunction
with -source 1.6
[error]
/Users/srowen/Documents/spark/network/yarn/src/main/java/org/apache/spark/network/
