Hi Harold, It seems like, based on your previous post, you are using one version of the connector as a dependency yet building the assembly jar from master? You were using 1.1.0-alpha3 (you can upgrade to alpha4, beta coming this week) yet your assembly is spark-cassandra-connector-assembly-1.2.0-SNAPSHOT.
libraryDependencies ++= Seq( "org.apache.spark" % "spark-streaming_2.10" % "1.1.0", "com.datastax.spark" %% "spark-cassandra-connector" % "1.1.0-alpha3" withSources() withJavadoc(), "org.apache.spark" %% "spark-sql" % "1.1.0" ) - Helena On Oct 28, 2014, at 2:08 PM, Harold Nguyen <har...@nexgate.com> wrote: > Hi all, > > The following works fine when submitting dependency jars through Spark-Shell: > > ./bin/spark-shell --master spark://ip-172-31-38-112:7077 --jars > /home/ubuntu/spark-cassandra-connector/spark-cassandra-connector/spark-cassandra-connector/target/scala-2.10/spark-cassandra-connector-assembly-1.2.0-SNAPSHOT.jar > > But not through spark-submit: > > ./bin/spark-submit --class > "org.apache.spark.examples.streaming.CassandraSave" --master > spark://ip-172-31-38-112:7077 > streaming-test/target/scala-2.10/simple-streaming_2.10-1.0.jar --jars > local:///home/ubuntu/spark-cassandra-connector/spark-cassandra-connector/target/scala-2.10/spark-cassandra-connector-assembly-1.2.0-SNAPSHOT.jar > > Am I issuing the spark-submit command incorrectly ? Each of the workers has > that built jar in their respective directories > (spark-cassandra-connector-assembly-1.2.0-SNAPSHOT.jar) > > Thanks, > > Harold --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
