Worst case if it puts too much strain: randomly failing clients and missing DN heartbeats leading to unnecessary dead node appearances. Worth using a release with HDFS-1457 in or patching your NN/SNN for that - helps shape those graphs.
On Mon, Mar 25, 2013 at 9:59 PM, Ivan Tretyakov <itretya...@griddynamics.com> wrote: > Thanks Harsh! > > My image size is about 3.1 Gb. > Yes, I think feature from HDFS-1457 is what I need, but unfortunately it is > not available in version of hadoop we use. > > What kind of risks pose by these peaks. > > > On Mon, Mar 25, 2013 at 7:31 PM, Harsh J <ha...@cloudera.com> wrote: >> >> What's your fsimage size? If its too high you would want to control >> the checkpoint transfer bandwidth to not affect the load at the NN. >> This is available via the JIRA HDFS-1457. >> >> >> On Mon, Mar 25, 2013 at 8:13 PM, Ivan Tretyakov >> <itretya...@griddynamics.com> wrote: >> > Hi! >> > >> > We see DataNode heartbeat average time peaks in Ganglia up to 20-70 >> > seconds >> > while SecondaryNameNode performs checkpointing. >> > See attached screenshots please. >> > >> > I would like to clarify if it is Ok, or not. And what kind of >> > consequences >> > and risks it could bring up. >> > >> > -- >> > Best Regards >> > Ivan Tretyakov >> >> >> >> -- >> Harsh J > > > > > -- > Best Regards > Ivan Tretyakov > -- Harsh J
