bq. then namenode start to copy block replicates on DN-2 to another DN, supposed DN-2.
sorry for typo. Correct for it: then namenode start to copy block replicates on DN-1 to another DN, supposed DN-2. On Wed, Apr 3, 2013 at 9:51 AM, Azuryy Yu <azury...@gmail.com> wrote: > It's different. > If you just want to stop DN-1 a short time, just kill the DataNode process > on DN-1. then do what you want. during this time, Namenode cannot receive > the heart beat from DN-1, then namenode start to copy block replicates on > DN-2 to another DN, supposed DN-2. > > But when you start DN-1 again, Namenode receive the DN-1 registration, > then namenode stop to copy the DN-1's block replicates even if NN doesn't > finish coping. > > Am I explain clearly? > > > > On Wed, Apr 3, 2013 at 9:43 AM, Henry Junyoung Kim > <henry.jy...@gmail.com>wrote: > >> @Harsh >> >> What's the reasons to make big gaps for removing nodes between >> decommission and just down nodes? >> In my understanding, both are necessary to copy un-replicated blocks to >> another alive nodes. >> If main costs of them are this one, total elapsed time couldn't be big >> different. >> >> Could you share some articles or documents to understand about >> decommissioning procedures? >> - explaining is always thanks ;) >> >> >> 2013. 4. 2., 오후 5:37, Harsh J <ha...@cloudera.com> 작성: >> >> > Yes, you can do the downtime work in steps of 2 DNs at a time, >> > especially since you mentioned the total work would be only ~30mins at >> > most. >> > >> > On Tue, Apr 2, 2013 at 1:46 PM, Henry Junyoung Kim >> > <henry.jy...@gmail.com> wrote: >> >> the rest of nodes to be alive has enough size to store. >> >> >> >> for this one that you've mentioned. >> >>> its easier to do so in a rolling manner without need of a >> >>> decommission. >> >> >> >> to check my understanding, just shutting down 2 of them and then 2 >> more and then 2 more without decommissions. >> >> >> >> is this correct? >> >> >> >> >> >> 2013. 4. 2., 오후 4:54, Harsh J <ha...@cloudera.com> 작성: >> >> >> >>> Note though that its only possible to decommission 7 nodes at the same >> >>> time and expect it to finish iff the remaining 8 nodes have adequate >> >>> free space for the excess replicas. >> >>> >> >>> If you're just going to take them down for a short while (few mins >> >>> each), its easier to do so in a rolling manner without need of a >> >>> decommission. You can take upto two down at a time on a replication >> >>> average of 3 or 3+, and put it back in later without too much data >> >>> movement impact. >> >>> >> >>> On Tue, Apr 2, 2013 at 1:06 PM, Yanbo Liang <yanboha...@gmail.com> >> wrote: >> >>>> It's reasonable to decommission 7 nodes at the same time. >> >>>> But may be it also takes long time to finish it. >> >>>> Because all the replicas in these 7 nodes need to be copied to >> remaining 8 >> >>>> nodes. >> >>>> The size of transfer from these nodes to the remaining nodes is >> equal. >> >>>> >> >>>> >> >>>> 2013/4/2 Henry Junyoung Kim <henry.jy...@gmail.com> >> >>>>> >> >>>>> :) >> >>>>> >> >>>>> currently, I have 15 data nodes. >> >>>>> for some tests, I am trying to decommission until 8 nodes. >> >>>>> >> >>>>> Now, the total dfs used size is 52 TB which is including all >> replicated >> >>>>> blocks. >> >>>>> from 15 to 8, total spent time is almost 4 days long. ;( >> >>>>> >> >>>>> someone mentioned that I don't need to decommission node by node. >> >>>>> for this case, is there no problems if I decommissioned 7 nodes at >> the >> >>>>> same time? >> >>>>> >> >>>>> >> >>>>> 2013. 4. 2., 오후 12:14, Azuryy Yu <azury...@gmail.com> 작성: >> >>>>> >> >>>>> I can translate it to native English: how many nodes you want to >> >>>>> decommission? >> >>>>> >> >>>>> >> >>>>> On Tue, Apr 2, 2013 at 11:01 AM, Yanbo Liang <yanboha...@gmail.com> >> wrote: >> >>>>>> >> >>>>>> You want to decommission how many nodes? >> >>>>>> >> >>>>>> >> >>>>>> 2013/4/2 Henry JunYoung KIM <henry.jy...@gmail.com> >> >>>>>>> >> >>>>>>> 15 for datanodes and 3 for replication factor. >> >>>>>>> >> >>>>>>> 2013. 4. 1., 오후 3:23, varun kumar <varun....@gmail.com> 작성: >> >>>>>>> >> >>>>>>>> How many nodes do you have and replication factor for it. >> >>>>>>> >> >>>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Harsh J >> >> >> > >> > >> > >> > -- >> > Harsh J >> >> >
