Re: Gzip progress during map phase.

2011-12-27 Thread Niels Basjes
Yes, this is what i was looking for. Thanks -- Met vriendelijke groet, Niels Basjes (Verstuurd vanaf mobiel ) Op 27 dec. 2011 12:08 schreef "Koji Noguchi" het volgende: > Assuming you're using TextInputFormat, it sounds like > https://issues.apache.org/jira/browse/MAPREDUCE-773 > In 0.21. Don'

Re: Gzip progress during map phase.

2011-12-27 Thread Niels Basjes
I would not expect this. I would expect behaviour that is independent of the way the splits are created. -- Met vriendelijke groet, Niels Basjes (Verstuurd vanaf mobiel ) Op 26 dec. 2011 07:57 schreef "Anthony Urso" het volgende: > Gzip files (unlike uncompressed files) are not splittable, whic

Writing output in multiple files in Hadoop

2011-12-27 Thread Bhavesh Shah
Hello all, I want to write the files according to months using map-reduce in hadoop. If data is from January month then the data should be wriiten in jan-file and likewise for each month there should be seperate file. How Can I create such file in hadoop mapredude. I am trying for recursive map-re

Re: Gzip progress during map phase.

2011-12-27 Thread Koji Noguchi
Assuming you're using TextInputFormat, it sounds like https://issues.apache.org/jira/browse/MAPREDUCE-773 In 0.21. Don't know about CDH. Koji On 12/27/11 2:00 AM, "Niels Basjes" wrote: > I would not expect this. I would expect behaviour that is independent of > the way the splits are created.

Re: Writing output in multiple files in Hadoop

2011-12-27 Thread Harsh J
Use the class 'MultipleOutputs': hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/mapred/lib/MultipleOutputs.html On Tue, Dec 27, 2011 at 3:29 PM, Bhavesh Shah wrote: > Hello all, > I want to write the files according to months using map-reduce in > hadoop. If data is from January mont

RE: Writing output in multiple files in Hadoop

2011-12-27 Thread Devaraj K
Can you check the MultipleOutputs class. http://hadoop.apache.org/common/docs/r0.20.205.0/api/org/apache/hadoop/mapre d/lib/MultipleOutputs.html Devaraj K -Original Message- From: Bhavesh Shah [mailto:bhavesh25s...@gmail.com] Sent: Tuesday, December 27, 2011 3:29 PM To: hadoop-...@luc