Re: Spark 2.x Core: .setMaster(local[*]) output is different from spark-submit

2018-03-17 Thread klrmowse
for clarification... .saveAsTextFile(rdd) writes to local fs, but not hdfs anyone? -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Spark 2.x Core: .setMaster(local[*]) output is different from spark-submit

2018-03-16 Thread klrmowse
when i run a job with .setMaster(local[*]), the output is as expected... but when i run it using YARN (single node, pseudo-distributed hdfs) via spark-submit, the output is fudged - instead of key-value pairs, it only shows one value preceded by a comma, and the rest are blank what am i missing