Re: Can we write output directly to HDFS from Mapper

2012-09-27 Thread Harsh J
Anand, You may read this in the FAQ: http://wiki.apache.org/hadoop/FAQ#Can_I_write_create.2BAC8-write-to_hdfs_files_directly_from_map.2BAC8-reduce_tasks.3F On Fri, Sep 28, 2012 at 9:45 AM, Balaraman, Anand anand_balara...@syntelinc.com wrote: Hi In Map-Reduce, is it appropriate to write the

Re: Can we write output directly to HDFS from Mapper

2012-09-27 Thread Hemanth Yamijala
Can certainly do that. Indeed, if you set the number of reducers to 0, the map output will be directly written to HDFS by the framework itself. You may also want to look at http://hadoop.apache.org/docs/stable/mapred_tutorial.html#Task+Side-Effect+Files to see some things that need to be taken