[
https://issues.apache.org/jira/browse/SPARK-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216661#comment-15216661
]
Nan Zhu commented on SPARK-14247:
---------------------------------
[~srowen] I always blindly copied "CDH.*" string from Spark building doc since
I only use HDFS/HBase and Cloud manager contained in CDH...I think it is just
fine...and I have been working with it since 1.5 (but what's the name of the
normal packages?
https://repository.cloudera.com/artifactory/public/org/apache/hadoop/hadoop-core/
)
yeah, it might be the problem of the compilation tools, but I'm not 100% sure
about it....maybe someone who is more familiar about how maven/sbt decides
which package to be evicted can give some explanations and solutions
I do not mind closing it....just keep a record here in case someone has the
similar problem and they know what happened....
> Spark does not compile with CDH-5.4.x due to the possible bug of ivy.....
> -------------------------------------------------------------------------
>
> Key: SPARK-14247
> URL: https://issues.apache.org/jira/browse/SPARK-14247
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 1.6.0
> Reporter: Nan Zhu
>
> I recently tried to compile Spark with CDH 5.4.x by the following command:
> sbt -Phadoop-2.6 -Dhadoop.version=2.6.0-mr1-cdh5.4.5 -DskipTests assembly
> I cannot finish the building due to an error saying that [error] impossible
> to get artifacts when data has not been loaded. IvyNode =
> org.slf4j#slf4j-api;1.6.1
> It seems that CDH depends on slf4j 1.6.x while Spark has upgraded to 1.7.x
> long time ago and during the compilation, slf4j 1.6.x is evicted unexpectedly
> (see the related discussion in https://github.com/sbt/sbt/issues/1598)
> I currently work around this by downgrade to slf4j 1.6.1....
> What surprises me is that I was upgrading from Spark 1.5.0 to 1.6.x, with
> 1.5.0, I can successfully compile Spark with the same version of CDH....
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
