It's different. If you just want to stop DN-1 a short time, just kill the DataNode process on DN-1. then do what you want. during this time, Namenode cannot receive the heart beat from DN-1, then namenode start to copy block replicates on DN-2 to another DN, supposed DN-2.
But when you start DN-1 again, Namenode receive the DN-1 registration, then namenode stop to copy the DN-1's block replicates even if NN doesn't finish coping. Am I explain clearly? On Wed, Apr 3, 2013 at 9:43 AM, Henry Junyoung Kim <henry.jy...@gmail.com>wrote: > @Harsh > > What's the reasons to make big gaps for removing nodes between > decommission and just down nodes? > In my understanding, both are necessary to copy un-replicated blocks to > another alive nodes. > If main costs of them are this one, total elapsed time couldn't be big > different. > > Could you share some articles or documents to understand about > decommissioning procedures? > - explaining is always thanks ;) > > > 2013. 4. 2., 오후 5:37, Harsh J <ha...@cloudera.com> 작성: > > > Yes, you can do the downtime work in steps of 2 DNs at a time, > > especially since you mentioned the total work would be only ~30mins at > > most. > > > > On Tue, Apr 2, 2013 at 1:46 PM, Henry Junyoung Kim > > <henry.jy...@gmail.com> wrote: > >> the rest of nodes to be alive has enough size to store. > >> > >> for this one that you've mentioned. > >>> its easier to do so in a rolling manner without need of a > >>> decommission. > >> > >> to check my understanding, just shutting down 2 of them and then 2 more > and then 2 more without decommissions. > >> > >> is this correct? > >> > >> > >> 2013. 4. 2., 오후 4:54, Harsh J <ha...@cloudera.com> 작성: > >> > >>> Note though that its only possible to decommission 7 nodes at the same > >>> time and expect it to finish iff the remaining 8 nodes have adequate > >>> free space for the excess replicas. > >>> > >>> If you're just going to take them down for a short while (few mins > >>> each), its easier to do so in a rolling manner without need of a > >>> decommission. You can take upto two down at a time on a replication > >>> average of 3 or 3+, and put it back in later without too much data > >>> movement impact. > >>> > >>> On Tue, Apr 2, 2013 at 1:06 PM, Yanbo Liang <yanboha...@gmail.com> > wrote: > >>>> It's reasonable to decommission 7 nodes at the same time. > >>>> But may be it also takes long time to finish it. > >>>> Because all the replicas in these 7 nodes need to be copied to > remaining 8 > >>>> nodes. > >>>> The size of transfer from these nodes to the remaining nodes is equal. > >>>> > >>>> > >>>> 2013/4/2 Henry Junyoung Kim <henry.jy...@gmail.com> > >>>>> > >>>>> :) > >>>>> > >>>>> currently, I have 15 data nodes. > >>>>> for some tests, I am trying to decommission until 8 nodes. > >>>>> > >>>>> Now, the total dfs used size is 52 TB which is including all > replicated > >>>>> blocks. > >>>>> from 15 to 8, total spent time is almost 4 days long. ;( > >>>>> > >>>>> someone mentioned that I don't need to decommission node by node. > >>>>> for this case, is there no problems if I decommissioned 7 nodes at > the > >>>>> same time? > >>>>> > >>>>> > >>>>> 2013. 4. 2., 오후 12:14, Azuryy Yu <azury...@gmail.com> 작성: > >>>>> > >>>>> I can translate it to native English: how many nodes you want to > >>>>> decommission? > >>>>> > >>>>> > >>>>> On Tue, Apr 2, 2013 at 11:01 AM, Yanbo Liang <yanboha...@gmail.com> > wrote: > >>>>>> > >>>>>> You want to decommission how many nodes? > >>>>>> > >>>>>> > >>>>>> 2013/4/2 Henry JunYoung KIM <henry.jy...@gmail.com> > >>>>>>> > >>>>>>> 15 for datanodes and 3 for replication factor. > >>>>>>> > >>>>>>> 2013. 4. 1., 오후 3:23, varun kumar <varun....@gmail.com> 작성: > >>>>>>> > >>>>>>>> How many nodes do you have and replication factor for it. > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>>> > >>> > >>> > >>> > >>> -- > >>> Harsh J > >> > > > > > > > > -- > > Harsh J > >
