Re: How to concatenate hadoop files to a single hadoop file

2008-10-02 Thread Michael Andrews 

You might be able to use hars:

http://hadoop.apache.org/core/docs/current/hadoop_archives.html

On 10/2/08 2:51 PM, "Steve Gao" <[EMAIL PROTECTED]> wrote:

Anybody knows? Thanks a lot.

--- On Thu, 10/2/08, Steve Gao <[EMAIL PROTECTED]> wrote:
From: Steve Gao <[EMAIL PROTECTED]>
Subject: How to concatenate hadoop files to a single hadoop file
To: core-user@hadoop.apache.org
Cc: [EMAIL PROTECTED]
Date: Thursday, October 2, 2008, 3:17 PM

Suppose I have 3 files in Hadoop that I want to "cat" them to a single
file. I know it can be done by "hadoop dfs -cat" to a local file and
updating it to Hadoop. But it's very expensive for large files. Is there an
internal way to do this in Hadoop itself? Thanks

Re: How to concatenate hadoop files to a single hadoop file

2008-10-02 Thread Steve Gao 
Anybody knows? Thanks a lot.

--- On Thu, 10/2/08, Steve Gao <[EMAIL PROTECTED]> wrote:
From: Steve Gao <[EMAIL PROTECTED]>
Subject: How to concatenate hadoop files to a single hadoop file
To: core-user@hadoop.apache.org
Cc: [EMAIL PROTECTED]
Date: Thursday, October 2, 2008, 3:17 PM

Suppose I have 3 files in Hadoop that I want to "cat" them to a single
file. I know it can be done by "hadoop dfs -cat" to a local file and
updating it to Hadoop. But it's very expensive for large files. Is there an
internal way to do this in Hadoop itself? Thanks

How to concatenate hadoop files to a single hadoop file

2008-10-02 Thread Steve Gao 
Suppose I have 3 files in Hadoop that I want to "cat" them to a single file. I 
know it can be done by "hadoop dfs -cat" to a local file and updating it to 
Hadoop. But it's very expensive for large files. Is there an internal way to do 
this in Hadoop itself? Thanks