Like before I get a java.lang.NoClassDefFoundError:
akka/stream/FlowMaterializer$
This can be solved using assembly plugin. you need to enable assembly plugin
in global plugins
C:\Users\infoshore\.sbt\0.13\plugins
add a line in plugins.sbt addSbtPlugin("com.eed3si9n" % "sbt-assembly" %
"0.11.0")
and then add the following lines in build.sbt
import AssemblyKeys._ // put this at the top of the file
seq(assemblySettings: _*)
Also in the bottom dont forget to add
assemblySettings
mergeStrategy in assembly := {
case m if m.toLowerCase.endsWith("manifest.mf") =>
MergeStrategy.discard
case m if m.toLowerCase.matches("meta-inf.*\\.sf$") =>
MergeStrategy.discard
case "log4j.properties" =>
MergeStrategy.discard
case m if m.toLowerCase.startsWith("meta-inf/services/") =>
MergeStrategy.filterDistinctLines
case "reference.conf" =>
MergeStrategy.concat
case _ =>
MergeStrategy.first
}
Now in your sbt run sbt assembly that will create the jar which can be run
without --jars options
as this will be a uber jar containing all jars
Also nosuchmethod exception is thrown when there is difference in versions
of complied and runtime versions.
What is the version of spark you are using ? You need to use same version in
build.sbt
Here is your build.sbt
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.1"
//exclude("com.typesafe", "config")
libraryDependencies += "org.apache.spark" %% "spark-sql" % "1.1.1"
libraryDependencies += "com.datastax.cassandra" % "cassandra-driver-core" %
"2.1.3"
libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector" %
"1.1.0" withSources() withJavadoc()
libraryDependencies += "org.apache.cassandra" % "cassandra-thrift" % "2.0.5"
libraryDependencies += "joda-time" % "joda-time" % "2.6"
and your error is Exception in thread "main" java.lang.NoSuchMethodError:
com.typesafe.config.Config.getDuration(Ljava/lang/String;Ljava/util/concurrent/TimeUnit;)J
at
akka.stream.StreamSubscriptionTimeoutSettings$.apply(FlowMaterializer.scala:256)
I think there is version mismatch on the jars you use at runtime
If you need more help add me on skype pankaj.narang
---Pankaj
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/NoSuchMethodError-com-typesafe-config-Config-getDuration-with-akka-http-akka-stream-tp20926p20950.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
