Try to use --packages to include the jars. From error it seems it's looking for main class in jars but u r running a python script.......
On 25 Feb 2017 10:36 pm, "Raymond Xie" <xie3208...@gmail.com> wrote: That's right Anahita, however, the class name is not indicated in the original github project so I don't know what class should be used here. The github only says: and then run the example `$ bin/spark-submit --jars \ external/kafka-assembly/target/scala-*/spark-streaming-kafka-assembly-*.jar \ examples/src/main/python/streaming/kafka_wordcount.py \ localhost:2181 test` """ Can anyone give any thought on how to find out? Thank you very much in advance. *------------------------------------------------* *Sincerely yours,* *Raymond* On Sat, Feb 25, 2017 at 5:27 PM, Anahita Talebi <anahita.t.am...@gmail.com> wrote: > You're welcome. > You need to specify the class. I meant like that: > > spark-submit /usr/hdp/2.5.0.0-1245/spark/lib/spark-assembly-1.6.2.2.5.0. > 0-1245-hadoop2.7.3.2.5.0.0-1245.jar --class "give the name of the class" > > > > On Saturday, February 25, 2017, Raymond Xie <xie3208...@gmail.com> wrote: > >> Thank you, it is still not working: >> >> [image: Inline image 1] >> >> By the way, here is the original source: >> >> https://github.com/apache/spark/blob/master/examples/src/mai >> n/python/streaming/kafka_wordcount.py >> >> >> *------------------------------------------------* >> *Sincerely yours,* >> >> >> *Raymond* >> >> On Sat, Feb 25, 2017 at 4:48 PM, Anahita Talebi < >> anahita.t.am...@gmail.com> wrote: >> >>> Hi, >>> >>> I think if you remove --jars, it will work. Like: >>> >>> spark-submit /usr/hdp/2.5.0.0-1245/spark/l >>> ib/spark-assembly-1.6.2.2.5.0.0-1245-hadoop2.7.3.2.5.0.0-1245.jar >>> >>> I had the same problem before and solved it by removing --jars. >>> >>> Cheers, >>> Anahita >>> >>> On Saturday, February 25, 2017, Raymond Xie <xie3208...@gmail.com> >>> wrote: >>> >>>> I am doing a spark streaming on a hortonworks sandbox and am stuck here >>>> now, can anyone tell me what's wrong with the following code and the >>>> exception it causes and how do I fix it? Thank you very much in advance. >>>> >>>> spark-submit --jars /usr/hdp/2.5.0.0-1245/spark/li >>>> b/spark-assembly-1.6.2.2.5.0.0-1245-hadoop2.7.3.2.5.0.0-1245.jar >>>> /usr/hdp/2.5.0.0-1245/kafka/libs/kafka-streams-0.10.0.2.5.0.0-1245.jar >>>> /root/hdp/kafka_wordcount.py 192.168.128.119:2181 test >>>> >>>> Error: >>>> No main class set in JAR; please specify one with --class >>>> >>>> >>>> spark-submit --class /usr/hdp/2.5.0.0-1245/spark/li >>>> b/spark-assembly-1.6.2.2.5.0.0-1245-hadoop2.7.3.2.5.0.0-1245.jar >>>> /usr/hdp/2.5.0.0-1245/kafka/libs/kafka-streams-0.10.0.2.5.0.0-1245.jar >>>> /root/hdp/kafka_wordcount.py 192.168.128.119:2181 test >>>> >>>> Error: >>>> java.lang.ClassNotFoundException: /usr/hdp/2.5.0.0-1245/spark/li >>>> b/spark-assembly-1.6.2.2.5.0.0-1245-hadoop2.7.3.2.5.0.0-1245.jar >>>> >>>> spark-submit --class /usr/hdp/2.5.0.0-1245/kafka/l >>>> ibs/kafka-streams-0.10.0.2.5.0.0-1245.jar >>>> /usr/hdp/2.5.0.0-1245/spark/lib/spark-assembly-1.6.2.2.5.0.0 >>>> -1245-hadoop2.7.3.2.5.0.0-1245.jar /root/hdp/kafka_wordcount.py >>>> 192.168.128.119:2181 test >>>> >>>> Error: >>>> java.lang.ClassNotFoundException: /usr/hdp/2.5.0.0-1245/kafka/li >>>> bs/kafka-streams-0.10.0.2.5.0.0-1245.jar >>>> >>>> *------------------------------------------------* >>>> *Sincerely yours,* >>>> >>>> >>>> *Raymond* >>>> >>> >>
