Thanks for your reply！ According to your hint, the code should be like this:
// i want to save data in rdd to mongodb and hdfs
rdd.saveAsNewAPIHadoopFile() rdd.saveAsTextFile()
but will the application read hdfs twice?
qinwei
From: Akhil DasDate: 2014-11-07
Thanks for your reply! As you mentioned , the insert clause is not executed as
the results of args.map are never used anywhere, and after i modified the code
, it works.
qinwei
From: Tobias PfeifferDate: 2014-11-07 18:04To: qinweiCC: userSubject: Re:
about write mongodb in
Hi, everyone
I come across with a prolem about writing data to mongodb in mapPartitions,
my code is as below: val sourceRDD =
sc.textFile(hdfs://host:port/sourcePath) // some transformations
val rdd= sourceRDD .map(mapFunc).filter(filterFunc) val
Why not saveAsNewAPIHadoopFile?
//Define your mongoDB confs
val config = new Configuration()
config.set(mongo.output.uri, mongodb://
127.0.0.1:27017/sigmoid.output)
//Write everything to mongo
rdd.saveAsNewAPIHadoopFile(file:///some/random, classOf[Any],
classOf[Any],
