Changed to sbt.0.14.3 and it gave :
[info] Packaging
/Users/jshetty/sparkApplications/MainTemplate/target/scala-2.11/maintemplate_2.11-1.0.jar
...
java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
at java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:233)
Do we need to create assembly.sbt file inside project directory if so what
will the the contents of it for this config ?
On Fri, Jul 22, 2016 at 5:42 AM, janardhan shetty <janardhan...@gmail.com>
wrote:
> Is scala version also the culprit? 2.10 and 2.11.8
>
> Also Can you give the steps to create sbt package command just like maven
> install from within intellij to create jar file in target directory ?
> On Jul 22, 2016 5:16 AM, "Jacek Laskowski" <ja...@japila.pl> wrote:
>
>> Hi,
>>
>> There has never been 0.13.8 for sbt-assembly AFAIK. Use 0.14.3 and
>> start over. See
>>
>> https://github.com/jaceklaskowski/spark-workshop/tree/master/solutions/spark-external-cluster-manager
>> for a sample Scala/sbt project with Spark 2.0 RC5.
>>
>> Pozdrawiam,
>> Jacek Laskowski
>> ----
>> https://medium.com/@jaceklaskowski/
>> Mastering Apache Spark http://bit.ly/mastering-apache-spark
>> Follow me at https://twitter.com/jaceklaskowski
>>
>>
>> On Fri, Jul 22, 2016 at 2:08 PM, janardhan shetty
>> <janardhan...@gmail.com> wrote:
>> > Hi,
>> >
>> > I was setting up my development environment.
>> >
>> > Local Mac laptop setup
>> > IntelliJ IDEA 14CE
>> > Scala
>> > Sbt (Not maven)
>> >
>> > Error:
>> > $ sbt package
>> > [warn] ::::::::::::::::::::::::::::::::::::::::::::::
>> > [warn] :: UNRESOLVED DEPENDENCIES ::
>> > [warn] ::::::::::::::::::::::::::::::::::::::::::::::
>> > [warn] :: com.eed3si9n#sbt-assembly;0.13.8: not found
>> > [warn] ::::::::::::::::::::::::::::::::::::::::::::::
>> > [warn]
>> > [warn] Note: Some unresolved dependencies have extra attributes.
>> Check
>> > that these dependencies exist with the requested attributes.
>> > [warn] com.eed3si9n:sbt-assembly:0.13.8 (scalaVersion=2.10,
>> > sbtVersion=0.13)
>> > [warn]
>> > [warn] Note: Unresolved dependencies path:
>> > [warn] com.eed3si9n:sbt-assembly:0.13.8 (scalaVersion=2.10,
>> > sbtVersion=0.13)
>> > (/Users/jshetty/sparkApplications/MainTemplate/project/plugins.sbt#L2-3)
>> > [warn] +- default:maintemplate-build:0.1-SNAPSHOT
>> > (scalaVersion=2.10, sbtVersion=0.13)
>> > sbt.ResolveException: unresolved dependency:
>> > com.eed3si9n#sbt-assembly;0.13.8: not found
>> > sbt.ResolveException: unresolved dependency:
>> > com.eed3si9n#sbt-assembly;0.13.8: not found
>> > at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:291)
>> > at
>> sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:188)
>> > at
>> sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:165)
>> > at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155)
>> > at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155)
>> > at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:132)
>> > at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57)
>> > at sbt.IvySbt$$anon$4.call(Ivy.scala:65)
>> > at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
>> >
>> >
>> >
>> > build.sbt:
>> >
>> > name := "MainTemplate"
>> > version := "1.0"
>> > scalaVersion := "2.11.8"
>> > libraryDependencies ++= {
>> > val sparkVersion = "2.0.0-preview"
>> > Seq(
>> > "org.apache.spark" %% "spark-core" % sparkVersion,
>> > "org.apache.spark" %% "spark-sql" % sparkVersion,
>> > "org.apache.spark" %% "spark-streaming" % sparkVersion,
>> > "org.apache.spark" %% "spark-mllib" % sparkVersion
>> > )
>> > }
>> >
>> > assemblyMergeStrategy in assembly := {
>> > case PathList("META-INF", xs @ _*) => MergeStrategy.discard
>> > case x => MergeStrategy.first
>> > }
>> >
>> >
>> > plugins.sbt
>> >
>> > logLevel := Level.Warn
>> > addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.13.8")
>> >
>>
>
