https://github.com/apache/spark/blob/master/project/SparkBuild.scala#L350
On Sun, Jun 1, 2014 at 11:03 AM, Patrick Wendell <pwend...@gmail.com> wrote: > One potential issue here is that mesos is using classifiers now to > publish there jars. It might be that sbt-pack has trouble with > dependencies that are published using classifiers. I'm pretty sure > mesos is the only dependency in Spark that is using classifiers, so > that's why I mention it. > > On Sun, Jun 1, 2014 at 2:34 AM, Pierre B > <pierre.borckm...@realimpactanalytics.com> wrote: >> Hi all! >> >> We'be been using the sbt-pack sbt plugin >> (https://github.com/xerial/sbt-pack) for building our standalone Spark >> application for a while now. Until version 1.0.0, that worked nicely. >> >> For those who don't know the sbt-pack plugin, it basically copies all the >> dependencies JARs from your local ivy/maven cache to a your target folder >> (in target/pack/lib), and creates launch scripts (in target/pack/bin) for >> your application (notably setting all these jars on the classpath). >> >> Now, since Spark 1.0.0 was released, we are encountering a weird error where >> running our project with "sbt run" is fine but running our app with the >> launch scripts generated by sbt-pack fails. >> >> After a (quite painful) investigation, it turns out some JARs are NOT copied >> from the local ivy2 cache to the lib folder. I noticed that all the missing >> jars contain "shaded" in their file name (but all not all jars with such >> name are missing). >> One of the missing JARs is explicitly from the Spark definition >> (SparkBuild.scala, line 350): ``mesos-0.18.1-shaded-protobuf.jar``. >> >> This file is clearly present in my local ivy cache, but is not copied by >> sbt-pack. >> >> Is there an evident reason for that? >> >> I don't know much about the shading mechanism, maybe I'm missing something >> here? >> >> >> Any help would be appreciated! >> >> Cheers >> >> Pierre >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Using-sbt-pack-with-Spark-1-0-0-tp6649.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com.
