Re: Re: Compression using Hadoop...

2007-08-31 Thread Arun C Murthy
--Original Message- >From: Stu Hood [mailto:[EMAIL PROTECTED] >Sent: Friday, August 31, 2007 2:23 PM >To: hadoop-user@lucene.apache.org >Subject: RE: Re: Compression using Hadoop... > >Isn't that what the distcp script does? > >Thanks, >Stu > > >-Original Mess

RE: Re: Compression using Hadoop...

2007-08-31 Thread Joydeep Sen Sarma
Ah - very similar .. wish I had known :-) Support for Globs and logic to split/parallelize large text files would help .. -Original Message- From: Stu Hood [mailto:[EMAIL PROTECTED] Sent: Friday, August 31, 2007 2:23 PM To: hadoop-user@lucene.apache.org Subject: RE: Re: Compression

RE: Re: Compression using Hadoop...

2007-08-31 Thread Stu Hood
Isn't that what the distcp script does? Thanks, Stu -Original Message- From: Joydeep Sen Sarma Sent: Friday, August 31, 2007 3:58pm To: hadoop-user@lucene.apache.org Subject: Re: Compression using Hadoop... One thing I had done to speed up copy/put speeds was write a simple map-reduce