It means that may be some replicas will be stay in under replica state?
2013/4/3 Azuryy Yu azury...@gmail.com
bq. then namenode start to copy block replicates on DN-2 to another DN,
supposed DN-2.
sorry for typo.
Correct for it:
then namenode start to copy block replicates on DN-1 to
not at all. so don't worry about that.
On Wed, Apr 3, 2013 at 2:04 PM, Yanbo Liang yanboha...@gmail.com wrote:
It means that may be some replicas will be stay in under replica state?
2013/4/3 Azuryy Yu azury...@gmail.com
bq. then namenode start to copy block replicates on DN-2 to another
thanks for all.
my strategies for removing from 15 DN to 8 DN.
1. kill two DNs at same time. : NN will detect nodes' down and he will try to
retain replication factors of lost blocks.
2. check your NN web UI. there is an info to let you know counts of
under-replicated blocks.
3. if it is
Note though that its only possible to decommission 7 nodes at the same
time and expect it to finish iff the remaining 8 nodes have adequate
free space for the excess replicas.
If you're just going to take them down for a short while (few mins
each), its easier to do so in a rolling manner without
the rest of nodes to be alive has enough size to store.
for this one that you've mentioned.
its easier to do so in a rolling manner without need of a
decommission.
to check my understanding, just shutting down 2 of them and then 2 more and
then 2 more without decommissions.
is this correct?
Yes, you can do the downtime work in steps of 2 DNs at a time,
especially since you mentioned the total work would be only ~30mins at
most.
On Tue, Apr 2, 2013 at 1:46 PM, Henry Junyoung Kim
henry.jy...@gmail.com wrote:
the rest of nodes to be alive has enough size to store.
for this one that
@Harsh
What's the reasons to make big gaps for removing nodes between decommission and
just down nodes?
In my understanding, both are necessary to copy un-replicated blocks to another
alive nodes.
If main costs of them are this one, total elapsed time couldn't be big
different.
Could you
It's different.
If you just want to stop DN-1 a short time, just kill the DataNode process
on DN-1. then do what you want. during this time, Namenode cannot receive
the heart beat from DN-1, then namenode start to copy block replicates on
DN-2 to another DN, supposed DN-2.
But when you start
bq. then namenode start to copy block replicates on DN-2 to another DN,
supposed DN-2.
sorry for typo.
Correct for it:
then namenode start to copy block replicates on DN-1 to another DN,
supposed DN-2.
On Wed, Apr 3, 2013 at 9:51 AM, Azuryy Yu azury...@gmail.com wrote:
It's different.
If
How many nodes do you have and replication factor for it.
It's alowable to decommission multi nodes at the same time.
Just write the all the hostnames which will be decommissioned to the
exclude file and run bin/hadoop dfsadmin -refreshNodes.
However you need to ensure the decommissioned DataNodes are minority of all
the DataNodes in the cluster and
15 for datanodes and 3 for replication factor.
2013. 4. 1., 오후 3:23, varun kumar varun@gmail.com 작성:
How many nodes do you have and replication factor for it.
You want to decommission how many nodes?
2013/4/2 Henry JunYoung KIM henry.jy...@gmail.com
15 for datanodes and 3 for replication factor.
2013. 4. 1., 오후 3:23, varun kumar varun@gmail.com 작성:
How many nodes do you have and replication factor for it.
I can translate it to native English: how many nodes you want to
decommission?
On Tue, Apr 2, 2013 at 11:01 AM, Yanbo Liang yanboha...@gmail.com wrote:
You want to decommission how many nodes?
2013/4/2 Henry JunYoung KIM henry.jy...@gmail.com
15 for datanodes and 3 for replication factor.
:)
currently, I have 15 data nodes.
for some tests, I am trying to decommission until 8 nodes.
Now, the total dfs used size is 52 TB which is including all replicated blocks.
from 15 to 8, total spent time is almost 4 days long. ;(
someone mentioned that I don't need to decommission node by
hi, hadoop users.
to decommission a single node, there is necessary to wait to remove a node from
a cluster for awhile. (in my case, 20 ~ 30 minutes)
for the safety, I am decommissioning a node at a time.
for the performance, am I able to remove multi nodes at same time?
16 matches
Mail list logo
