Re: NameNode per-block memory usage?

2012-01-30 Thread ke yuan
how much blocks a file has depends on your data status,please consider it according to your data status 2012/1/18 prasenjit mukherjee > Does it mean that on an average 1 file has only 2 blocks ( with > replication=1 ) ? > > > On 1/18/12, M. C. Srivas wrote: > > Konstantin's paper > > http:

Re: reducing mappers for a job

2011-11-16 Thread ke yuan
configuration. > > If you change the block size for your Hadoop cluster configureation (in the > hdfs-site.xml file), this method may bring some side-effects. > > 1) waste of disk space; > 2) difficulty to balance HDFS; > 3) low Map stage data locality; > > Bests! > >

Re: reducing mappers for a job

2011-11-16 Thread ke yuan
just the blocksize 128M or 256M,it may reduce the number of mappers per job 2011/11/17 Jay Vyas > Hi guys : In a shared cluster environment, whats the best way to reduce the > number of mappers per job ? Should you do it with inputSplits ? Or simply > toggle the values in the JobConf (i.e. inc