Re: Errors accessing hdfs while in local mode

2014-07-17 Thread Akhil Das
You can try the following in the spark-shell: 1. Run it in *Clustermode* by going inside the spark directory: $ SPARK_MASTER=spark://masterip:7077 ./bin/spark-shell val textFile = sc.textFile(hdfs://masterip/data/blah.csv) textFile.take(10).foreach(println) 2. Now try running in *Localmode:*

Re: Errors accessing hdfs while in local mode

2014-07-17 Thread Andrew Or
Hi Chris, Did you ever figure this out? It should just work provided that your HDFS is set up correctly. If you don't call setMaster, it actually uses the spark://[master-node-ip]:7077 by default (this is configured in your conf/spark-env.sh). However, even if you use a local master, it should