multiple map tasks writing in same hdfs file -issue

2014-07-10 Thread rab ra
Hello I have one use-case that spans multiple map tasks in hadoop environment. I use hadoop 1.2.1 and with 6 task nodes. Each map task writes their output into a file stored in hdfs. This file is shared across all the map tasks. Though, they all computes thier output but some of them are missing

Re: multiple map tasks writing in same hdfs file -issue

2014-07-10 Thread Arpit Agarwal
HDFS is single-writer, multiple-reader (see sec 8.3.1 of http://aosabook.org/en/hdfs.html). You cannot have multiple writers for a single file at a time. On Thu, Jul 10, 2014 at 2:55 AM, rab ra wrote: > Hello > > > I have one use-case that spans multiple map tasks in hadoop environment. I > use