Re: Error while building Hadoop-Yarn

2011-08-19 Thread rajesh putta
Thanks Arun, Now its working fine Thanks & Regards Rajesh Putta Development Engineer Pramati Technologies On Fri, Aug 19, 2011 at 12:25 PM, Arun Murthy wrote: > That means you don't have the autotool chain necessary for build the > native code. > > For now pass -P-cbuild to skip them. >

RE: Wonky reduce progress

2011-08-19 Thread Vinayakumar B
Please check the defect in MAPREDUCE jira https://issues.apache.org/jira/browse/MAPREDUCE-2264 This is because the compression is enabled for map outputs and statistics are taken on compressed data instead of original data. -Original Message- From: Joey Echeverria [mailto:j...@cloudera.c

SequenceFileInputFormat doesn't return whole records

2011-08-19 Thread Tim Fletcher
Hi all, I am having issues using SequenceFileInputFormat to retrieve whole records I have 1 job that is used to write to a SequenceFile SequenceFileOutputFormat.setOutputPath(job, new Path("out/data")); SequenceFileOutputFormat.setOutputCompressionType(job, SequenceFile.CompressionType.NONE);

Re: SequenceFileInputFormat doesn't return whole records

2011-08-19 Thread Harsh J
Tim, Do you also set your I/O formats explicitly to SequenceFileInputFormat and SequenceFileOutputFormat? Via job.setInputFormat/setOutputFormat I mean. Hadoop should not be splitting records across maps/mappers. There are specific test cases that ensure this does not happen, so it would seem str

Re: SequenceFileInputFormat doesn't return whole records

2011-08-19 Thread Tim Fletcher
Harsh, that was exactly the issue! Thanks very much for your help Tim On 19 August 2011 15:15, Harsh J wrote: > Tim, > > Do you also set your I/O formats explicitly to SequenceFileInputFormat > and SequenceFileOutputFormat? Via job.setInputFormat/setOutputFormat I > mean. > > Hadoop should not

Re: Wonky reduce progress

2011-08-19 Thread Joey Echeverria
Yup, that would do it. Thanks. -Joey On Fri, Aug 19, 2011 at 4:14 AM, Vinayakumar B wrote: > > Please check the defect in MAPREDUCE jira > https://issues.apache.org/jira/browse/MAPREDUCE-2264 > > This is because the compression is enabled for map outputs and statistics > are taken on compressed