[ https://issues.apache.org/jira/browse/SPARK-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Wendell resolved SPARK-2338. ------------------------------------ Resolution: Fixed Assignee: Pete MacKinnon Thanks a ton for getting to the bottom of this. I was super confused why the tests were so messed up even though this seems totally obvious in retrospect. I went ahead and updated the build configuration. There are some failing tests in MLLib in maven, I'll try to track those down as well to get this all green. > Jenkins Spark-Master-Maven-with-YARN builds failing due to test > misconfiguration > -------------------------------------------------------------------------------- > > Key: SPARK-2338 > URL: https://issues.apache.org/jira/browse/SPARK-2338 > Project: Spark > Issue Type: Bug > Components: Build, Project Infra, YARN > Affects Versions: 1.0.0 > Environment: https://amplab.cs.berkeley.edu/jenkins > Reporter: Pete MacKinnon > Assignee: Pete MacKinnon > Labels: hadoop2, jenkins, maven, protobuf, yarn > > https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-Maven-with-YARN/hadoop.version=2.2.0,label=centos/ > https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-Maven-with-YARN/hadoop.version=2.3.0,label=centos/ > These builds are currently failing due to the builder configuration being > incomplete. After building, they specify the test command as: > {noformat} > /home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.0.5/bin/mvn > -Dhadoop.version=2.3.0 -Dlabel=centos test -Pyarn -Phive > {noformat} > However, it is not enough to specify the hadoop.version, the tests should > instead be run using the hadoop-2.2 and hadoop-2.3 profiles respectively. > For example: > {noformat} > /home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.0.5/bin/mvn > -Phadoop2.2 -Dlabel=centos test -Pyarn -Phive > {noformat} > These profiles will not only set the appropriate hadoop.version but also set > the version of protobuf-java required by yarn (2.5.0). Without the correct > profile set, the test run fails at: > {noformat} > *** RUN ABORTED *** > java.lang.VerifyError: class > org.apache.hadoop.yarn.proto.YarnProtos$LocalResourceProto overrides final > method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet; > {noformat} > since it is getting the default version of protobuf-java (2.4.1) which has > the old incompatible version of getUnknownFields. -- This message was sent by Atlassian JIRA (v6.2#6252)