What happens when you do:
sc.textFile(hdfs://path/to/the_file.txt)
Thanks
Best Regards
On Mon, Mar 30, 2015 at 11:04 AM, Nick Travers n.e.trav...@gmail.com
wrote:
Hi List,
I'm following this example here
https://github.com/databricks/learning-spark/tree/master/mini-complete-example
I think the jar file has to be local. In HDFS is not supported yet in Spark.
See this answer:
http://stackoverflow.com/questions/28739729/spark-submit-not-working-when-application-jar-is-in-hdfs
Date: Sun, 29 Mar 2015 22:34:46 -0700
From: n.e.trav...@gmail.com
To: user@spark.apache.org
Try running it like this:
sudo -u hdfs spark-submit --class org.apache.spark.examples.SparkPi
--deploy-mode cluster --master yarn
hdfs:///user/spark/spark-examples-1.2.0-cdh5.3.2-hadoop2.5.0-cdh5.3.2.jar 10
Caveats:
1) Make sure the permissions of /user/nick is 775 or 777.
2) No need for
