Re: When does Reduce job start

Tht is wht I was looking for Thanks a mil harsh Kool , now tht I have a start point, I will check it in hadoop 18 -Sagar On Tue, Jan 4, 2011 at 7:23 PM, Harsh J wrote: > Hello Sagar, > > On Wed, Jan 5, 2011 at 6:44 AM, sagar naik wrote: >>>> Wht is the configuration

Re: When does Reduce job start

e there is no way to gaurantee you have all the values for any key > until all the mappers are done. reduce() requires a key and all the values > for that key in order to execute. > > Jeff > > > On Tue, Jan 4, 2011 at 10:53 AM, sagar naik wrote: > >> Hi All, >> &

When does Reduce job start

Hi All, number  of map task: 1000s number of reduce task:single digit In such cases the reduce task wont  started even when few map task are completed. Example: In my observation of a sample run of bin/hadoop jar hadoop-*examples*.jar pi 1 10, the reduce did not start untill 90% of map task w

Force Delete

Hi, I knw tht blocks are not deleted immediately. However, Whenever I restart a datanode , it deletes all invalid blocks. Is there any way tht I can force delete blocks invalid blocks -Sagar

Re: Hadoop DFS IO Performance measurement

> What hardware did you run it on ? > > Edson Ramiro > > > On 30 March 2010 19:41, sagar naik wrote: > >> Hi All, >> >> I am trying to get DFS IO performance. >> I used TestDFSIO from hadoop jars. >> The results were abt 100Mbps read and write

Hadoop DFS IO Performance measurement

Hi All, I am trying to get DFS IO performance. I used TestDFSIO from hadoop jars. The results were abt 100Mbps read and write . I think it should be more than this Pl share some stats to compare Either I am missing something like config params or something else -Sagar

NPE in Datanode

Hi All, I got an NPE on hadoop-18.1 datanode Exception in thread "org.apache.hadoop.dfs.datanod...@107f7f7" java.lang.NullPointerException at org.apache.hadoop.dfs.FSDataset.getMetaFile(FSDataset.java:571) at org.apache.hadoop.dfs.FSDataset.updateBlock(FSDataset.java:801)

Reporter - incrCounter

public void incrCounter(Enum key, long amount) I am using a multi-threaded Mapper. I m not sure, but should this be synchronized ? I log the counter call incrCounter. log stmt counters are fine and Reporter counters are not matching m i missing something ? -Sagar