What happens when you do:
sc.textFile(hdfs://path/to/the_file.txt)
Thanks
Best Regards
On Mon, Mar 30, 2015 at 11:04 AM, Nick Travers n.e.trav...@gmail.com
wrote:
Hi List,
I'm following this example here
https://github.com/databricks/learning-spark/tree/master/mini-complete-example
Subject: java.io.FileNotFoundException when using HDFS in cluster mode
Hi List,
I'm following this example here
https://github.com/databricks/learning-spark/tree/master/mini-complete-example
with the following:
$SPARK_HOME/bin/spark-submit \
--deploy-mode cluster \
--master spark
Try running it like this:
sudo -u hdfs spark-submit --class org.apache.spark.examples.SparkPi
--deploy-mode cluster --master yarn
hdfs:///user/spark/spark-examples-1.2.0-cdh5.3.2-hadoop2.5.0-cdh5.3.2.jar 10
Caveats:
1) Make sure the permissions of /user/nick is 775 or 777.
2) No need for
Hi List,
I'm following this example here
https://github.com/databricks/learning-spark/tree/master/mini-complete-example
with the following:
$SPARK_HOME/bin/spark-submit \
--deploy-mode cluster \
--master spark://host.domain.ex:7077 \
--class