Hello everyone, I wanted to do something like this: Given a JavaPairRDD(let's say with 10 rows), I want to store each of the rows separately with following requirements: a) Each of them should be a map(Can not use saveAsTextFile) b) The file name should have the key in it(Eg: If the key is 0,1.. The file name should be something like path0_1.
Inorder to do that, I created a class which extends MultipleTextOutputFormat, and I tweaked it a bit to generate separate files for each row of rdd(using saveAsHadoopFile to save them). It works fine on single machine, but for some reason it does not create the file when I try to run it on cluster: (works fine with --> .setMaster("local") and doesn't create that individual file when I change Master to cluster). Any clues.