[ https://issues.apache.org/jira/browse/SPARK-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216672#comment-15216672 ]
Sean Owen commented on SPARK-14247: ----------------------------------- hadoop-core is the old artifact name from pre-YARN Hadoop 1 days. CDH still publishes a build that works with these different sets of artifacts -- commonly called the "MapReduce 1" artifacts. You want to refer to hadoop-common instead and use the artifacts without "-mr1-". This alone may solve it, since this is actually a build set up to support an old set of dependencies. > Spark does not compile with CDH-5.4.x due to the possible bug of ivy..... > ------------------------------------------------------------------------- > > Key: SPARK-14247 > URL: https://issues.apache.org/jira/browse/SPARK-14247 > Project: Spark > Issue Type: Bug > Components: Build > Affects Versions: 1.6.0 > Reporter: Nan Zhu > Priority: Minor > > I recently tried to compile Spark with CDH 5.4.x by the following command: > sbt -Phadoop-2.6 -Dhadoop.version=2.6.0-mr1-cdh5.4.5 -DskipTests assembly > I cannot finish the building due to an error saying that [error] impossible > to get artifacts when data has not been loaded. IvyNode = > org.slf4j#slf4j-api;1.6.1 > It seems that CDH depends on slf4j 1.6.x while Spark has upgraded to 1.7.x > long time ago and during the compilation, slf4j 1.6.x is evicted unexpectedly > (see the related discussion in https://github.com/sbt/sbt/issues/1598) > I currently work around this by downgrade to slf4j 1.6.1.... > What surprises me is that I was upgrading from Spark 1.5.0 to 1.6.x, with > 1.5.0, I can successfully compile Spark with the same version of CDH.... > -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org