[
https://issues.apache.org/jira/browse/SPARK-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056447#comment-14056447
]
Pete MacKinnon commented on SPARK-2338:
---------------------------------------
Patrick, it appears https://github.com/apache/spark/pull/1343 fixes/addresses
those mllib DecisionTreeSuite failures. I just got all green in my local tests
with it.
> Jenkins Spark-Master-Maven-with-YARN builds failing due to test
> misconfiguration
> --------------------------------------------------------------------------------
>
> Key: SPARK-2338
> URL: https://issues.apache.org/jira/browse/SPARK-2338
> Project: Spark
> Issue Type: Bug
> Components: Build, Project Infra, YARN
> Affects Versions: 1.0.0
> Environment: https://amplab.cs.berkeley.edu/jenkins
> Reporter: Pete MacKinnon
> Assignee: Pete MacKinnon
> Labels: hadoop2, jenkins, maven, protobuf, yarn
>
> https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-Maven-with-YARN/hadoop.version=2.2.0,label=centos/
> https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-Maven-with-YARN/hadoop.version=2.3.0,label=centos/
> These builds are currently failing due to the builder configuration being
> incomplete. After building, they specify the test command as:
> {noformat}
> /home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.0.5/bin/mvn
> -Dhadoop.version=2.3.0 -Dlabel=centos test -Pyarn -Phive
> {noformat}
> However, it is not enough to specify the hadoop.version, the tests should
> instead be run using the hadoop-2.2 and hadoop-2.3 profiles respectively.
> For example:
> {noformat}
> /home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.0.5/bin/mvn
> -Phadoop2.2 -Dlabel=centos test -Pyarn -Phive
> {noformat}
> These profiles will not only set the appropriate hadoop.version but also set
> the version of protobuf-java required by yarn (2.5.0). Without the correct
> profile set, the test run fails at:
> {noformat}
> *** RUN ABORTED ***
> java.lang.VerifyError: class
> org.apache.hadoop.yarn.proto.YarnProtos$LocalResourceProto overrides final
> method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
> {noformat}
> since it is getting the default version of protobuf-java (2.4.1) which has
> the old incompatible version of getUnknownFields.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
