HDFS files are write once so you cannot append to them (at the moment). What you can do is copy your local file to HDFS dir containing the same file you want to append to. Once that is done you can run a simple (Identity Mapper & Identity Reducer) mapreduce job with input as this directory and number of reducers = 1.
----- Original Message ----- From: "Turner Kunkel" <thkun...@gmail.com> To: core-u...@hadoop.apache.org Sent: Wednesday, August 26, 2009 10:02:41 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Concatenating files on HDFS Is there any way to concatenate/append a local file to a file on HDFS without copying down the HDFS file locally first? I tried: bin/hadoop dfs -cat file:///[local file] >> hdfs://[hdfs file] But it just tries to look for hdfs://[hdfs file] as a local file, since I suppose the dfs -cat command doesn't support the >> operator. Thanks. -- -Turner Kunkel
