Hi, I'm developing an application with spark-streaming-kafka, which depends on spark-streaming and kafka. Since spark-streaming is provided in runtime, I want to exclude the jars from the assembly. I tried the following configuration:
libraryDependencies ++= { val sparkVersion = "1.0.2" Seq( "org.apache.spark" %% "spark-streaming-kafka" % sparkVersion, "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided" ) } Not working. I also tried: libraryDependencies ++= { val sparkVersion = "1.0.2" Seq( ("org.apache.spark" %% "spark-streaming-kafka" % sparkVersion). exclude("org.apache.spark", "spark-streaming"), "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided" ) } It still package all the jars of spark-streaming. Finally I come up with: libraryDependencies ++= { val sparkVersion = "1.0.2" Seq( "org.apache.spark" %% "spark-streaming-kafka" % sparkVersion intransitive(), "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided", ("org.apache.kafka" %% "kafka" % "0.8.0"). exclude("com.sun.jmx", "jmxri"). exclude("com.sun.jdmk", "jmxtools"). exclude("net.sf.jopt-simple", "jopt-simple"). exclude("org.slf4j", "slf4j-simple"). exclude("org.apache.zookeeper", "zookeeper") ) } Wordy, but works. So I'm wondering whether there's a better way. Thanks. -- Jerry --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org