Hi Tathagata, I got rid of parts of this problem today. It was embarassingly simple: somehow, my synchronisation script had failed and one of the two jars wasn't sent out to all slave nodes.
The java error disappears, however, I still don't receive any tweets. This might well be firewall related. As I have set export SPARK_MASTER_PORT=10000 export SPARK_MASTER_WEBUI_PORT=10001 export SPARK_WORKER_PORT=11000 export SPARK_WORKER_WEBUI_PORT=11001 I monitored the port range 10000-11002 with tcpdump and the destination host api.twitter.com. I could not see any traffic there, so I was wondering how can I get a closer look at this? I.e. how is Spark trying to communicate with the twitter API? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-fails-where-is-the-problem-tp11355p11533.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
