Hi Doris! I'm not sure what the difference between lab / production use is. All configuration affects some behavior of the Hadoop system. Usually the defaults are good for small clusters. For larger clusters, it becomes worthwhile to tune the configuration.
1. dfs.namenode.heartbeat.recheck-interval : This is more a function of how busy your datanodes are (sometimes they are too busy to heartbeat) and how robust is your network (dropping heartbeat packets). It doesn't really take too long to *check* the last heartbeat time of datanodes, but its a lot of work to order re-replications, so I would err on the side of keeping it long. 2. The clients gets an *ordered* list of datanodes from the namenode. It has its own timeouts and mechanism for finding which one it wants to get / send data from / to. Are live datanodes becoming stale too often in your cluster? What's your concern? Usually if your cluster is large enough, you *want* to spend time tuning it. And that usually means, you will have to spend lots of time analyzing the workload, finding the bottlenecks / wasted work and seeing what configurations can help you remove that. HTH Ravi On Wed, Aug 23, 2017 at 6:15 AM, <gu.yiz...@zte.com.cn> wrote: > Hi All, > > There are default values of configs in hdfs-default.xml and > core-default.xml, and I am wondering which situation are they for? Are they > closer to lab use, or closer to real production environment? > > > Maybe it depends on different configs, then I have questions to these > certain configs as follows: > > > > Hadoop 2.7.3 > > 1.dfs.namenode.heartbeat.recheck-interval: the default value is 5min > which makes the datanode to be marked as dead by the namenode after 10:30 > minutes. I set it to 30s, and gain a lot of removing and registering in > namanode's log. Is 5min too long, maybe 2.5min? > > > 2.avoid stale: I notice there is a stale state of datanode but is off > by default. I feel it's good, is it advised to set on? > > > Thanks in advance, > > Doris > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org > For additional commands, e-mail: user-h...@hadoop.apache.org >
