Hi all, The short answer is that standalone-cluster mode through spark-submit is broken (and in fact not officially supported). Please use standalone-client mode instead. The long answer is provided here: http://mail-archives.apache.org/mod_mbox/spark-user/201406.mbox/%3cCAMJOb8m6gF9B3W=p12hi88mexkoon15-1jkvs8pblmuwh9r...@mail.gmail.com%3e
Andrew 2014-06-19 12:00 GMT-07:00 lannyripple <lanny.rip...@gmail.com>: > Gino, > > I can confirm that your solution of assembling with spark-streaming-kafka > but excluding spark-core and spark-streaming has me working with basic > spark-submit. As mentioned you must specify the assembly jar in the > SparkConfig as well as to spark-submit. > > When I see the error you are now experiencing I just restart my cluster > (sbin/stop-all.sh; sleep 6; sbin/start-all.sh). My thought is a resource > leak somewhere but I haven't tried to chase it down since restarting is > nice > and quick. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/spark-streaming-kafka-SPARK-CLASSPATH-tp7356p7941.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
