Is the hadoop version Hadoop 0.20.203.0 API That means still the hadoop files in HDFS version 0.20.20 are immutable? And there is no means we can append to an existing file in HDFS?
We need to do this urgently as we have do set up the pipeline accordingly in production? Regards, Jagaran ________________________________ From: Xiaobo Gu <guxiaobo1...@gmail.com> To: common-user@hadoop.apache.org Sent: Thu, 16 June, 2011 6:26:45 PM Subject: Re: HDFS File Appending please refer to FileUtil.CopyMerge On Fri, Jun 17, 2011 at 8:33 AM, jagaran das <jagaran_...@yahoo.co.in> wrote: > Hi, > > We have a requirement where > > There would be huge number of small files to be pushed to hdfs and then use >pig > to do analysis. > To get around the classic "Small File Issue" we merge the files and push a > bigger file in to HDFS. > But we are loosing time in this merging process of our pipeline. > > But If we can directly append to an existing file in HDFS we can save this > "Merging Files" time. > > Can you please suggest if there a newer stable version of Hadoop where can go > for appending ? > > Thanks and Regards, > Jagaran
