Thanks Ravi. The number of nodes isn't a lot but the size is rather large. Each data node has about 14-16T (560-640T).
For the datanode block scanner, how can increase its "Current scan rate limit KBps" ? On Sun, Oct 6, 2013 at 11:09 PM, Ravi Prakash <ravi...@ymail.com> wrote: > Please look at dfs.heartbeat.interval and > dfs.namenode.heartbeat.recheck-interval > > 40 datanodes is not a large cluster IMHO and the Namenode is capable of > managing 100 times more datanodes. > > > > > ________________________________ > From: Rita <rmorgan...@gmail.com> > To: "<common-user@hadoop.apache.org>" <common-user@hadoop.apache.org> > Sent: Sunday, October 6, 2013 9:49 AM > Subject: datanode tuning > > > I would like my 40 data nodes to aggressively report to namenode if they > are alive or not therefore I think I need to change these params > > dfs.block.access.token.lifetime : Default is 600 seconds. Can I decrease > this to 60? > > > dfs.block.access.key.update.interval: Default is 600 seconds. Can I > decrease this to 60? > > Also, what are some other turnings people do for datanodes in a relatively > large cluster? > > > > -- > --- Get your facts first, then you can distort them as you please.-- > -- --- Get your facts first, then you can distort them as you please.--
