what happens when a datanode rejoins?
Hi, What happens when an existing (not new) datanode rejoins a cluster for following scenarios: 1. Some of the blocks it was managing are deleted/modified? 2. The size of the blocks are now modified say from 64MB to 128MB? 3. What if the block replication factor was one (yea not in most deployments but say incase) so does the namenode recreate a file once the datanode rejoins? Thanks, Mehul
what happens when a datanode rejoins?
Hi, What happens when an existing (not new) datanode rejoins a cluster for following scenarios: a) Some of the blocks it was managing are deleted/modified? b) The size of the blocks are now modified say from 64MB to 128MB? c) What if the block replication factor was one (yea not in most deployments but say in case) so does the namenode recreate a file once the datanode rejoins? Thanks, Mehul
Re: what happens when a datanode rejoins?
Hi Mehul Some of the blocks it was managing are deleted/modified? The namenode will asynchronously replicate the blocks to other datanodes in order to maintain the replication factor after a datanode has not been in contact for 10 minutes. The size of the blocks are now modified say from 64MB to 128MB? Block size is a per-file setting so new files will be 128MB, but the old ones will remain at 64MB. What if the block replication factor was one (yea not in most deployments but say incase) so does the namenode recreate a file once the datanode rejoins? (assuming you didn't perform a decommission) Blocks that lived only on that datanode will be declared "missing" and the files associated with those blocks will be not be able to be fully read, until the datanode rejoins. George
Re: what happens when a datanode rejoins?
Mehul, Let me make an addition. Some of the blocks it was managing are deleted/modified? Blocks that are deleted in the interim will deleted on the rejoining node as well, after it rejoins . Regarding the "modified," I'd advise against modifying blocks after they have been fully written. George
Re: what happens when a datanode rejoins?
George has answered most of these. I'll just add on: On Tue, Sep 11, 2012 at 12:44 PM, Mehul Choube wrote: > 1. Some of the blocks it was managing are deleted/modified? A DN runs a block report upon start, and sends the list of blocks to the NN. NN validates them and if it finds any files to miss block replicas post-report, it will schedule a re-replication from one of the good DNs that still carry it. The modified (out-of-HDFS) blocks fail their stored checksums so are treated as corrupt and deleted, and are re-replicated in the same manner. > 2. The size of the blocks are now modified say from 64MB to 128MB? George's got this already. Changing of block size does not impact any existing blocks. It is a per-file metadata prop. > 3. What if the block replication factor was one (yea not in most > deployments but say incase) so does the namenode recreate a file once the > datanode rejoins? Files exist at the NN metadata (its fsimage/edits persist this). Blocks pertaining to a file exists at a DN. If the file had a single replica and that replica was lost, then the file's data is lost and the NameNode will tell you as much in its metrics/fsck. -- Harsh J
Re: what happens when a datanode rejoins?
Yes the cluster will be re balanced. On Tue, Sep 11, 2012 at 2:09 PM, mehul choube wrote: > Hi, > > > What happens when an existing (not new) datanode rejoins a cluster for > following scenarios: > > > a) Some of the blocks it was managing are deleted/modified? > > b) The size of the blocks are now modified say from 64MB to 128MB? > > c) What if the block replication factor was one (yea not in most > deployments but say in case) so does the namenode recreate a file once the > datanode rejoins? > > > > > Thanks, > > Mehul > > > -- ∞ Shashwat Shriparv
Re: what happens when a datanode rejoins?
Hi Mehul, Please do not send multiple mails with the same questions. We've already answered this at your other post, follow thread at: http://mail-archives.apache.org/mod_mbox/hadoop-user/201209.mbox/%3ce884ec9cd547324b8976a5d37317ac566d11c7f...@apj1xchevspin30.symc.symantec.com%3e On Tue, Sep 11, 2012 at 2:09 PM, mehul choube wrote: > Hi, > > > What happens when an existing (not new) datanode rejoins a cluster for > following scenarios: > > > a) Some of the blocks it was managing are deleted/modified? > > b) The size of the blocks are now modified say from 64MB to 128MB? > > c) What if the block replication factor was one (yea not in most deployments > but say in case) so does the namenode recreate a file once the datanode > rejoins? > > > > > Thanks, > > Mehul > > -- Harsh J
RE: what happens when a datanode rejoins?
> The namenode will asynchronously replicate the blocks to other datanodes in > order to maintain the replication factor after a datanode has not been in > contact for 10 minutes. What happens when the datanode rejoins after namenode has already re-replicated the blocs it was managing? Will namenode ask the datanode to discard the blocks and start managing new blocks? Or will namenode discard the new blocks which were replicated due to unavailability of this datanode? Thanks, Mehul From: George Datskos [mailto:george.dats...@jp.fujitsu.com] Sent: Tuesday, September 11, 2012 12:56 PM To: user@hadoop.apache.org Subject: Re: what happens when a datanode rejoins? Hi Mehul Some of the blocks it was managing are deleted/modified? The namenode will asynchronously replicate the blocks to other datanodes in order to maintain the replication factor after a datanode has not been in contact for 10 minutes. The size of the blocks are now modified say from 64MB to 128MB? Block size is a per-file setting so new files will be 128MB, but the old ones will remain at 64MB. What if the block replication factor was one (yea not in most deployments but say incase) so does the namenode recreate a file once the datanode rejoins? (assuming you didn't perform a decommission) Blocks that lived only on that datanode will be declared "missing" and the files associated with those blocks will be not be able to be fully read, until the datanode rejoins. George
RE: what happens when a datanode rejoins?
I apologize for this :( I thought the earlier mail didn't go through -Original Message- From: Harsh J [mailto:ha...@cloudera.com] Sent: Tuesday, September 11, 2012 2:31 PM To: user@hadoop.apache.org Subject: Re: what happens when a datanode rejoins? Hi Mehul, Please do not send multiple mails with the same questions. We've already answered this at your other post, follow thread at: http://mail-archives.apache.org/mod_mbox/hadoop-user/201209.mbox/%3ce884ec9cd547324b8976a5d37317ac566d11c7f...@apj1xchevspin30.symc.symantec.com%3e On Tue, Sep 11, 2012 at 2:09 PM, mehul choube wrote: > Hi, > > > What happens when an existing (not new) datanode rejoins a cluster for > following scenarios: > > > a) Some of the blocks it was managing are deleted/modified? > > b) The size of the blocks are now modified say from 64MB to 128MB? > > c) What if the block replication factor was one (yea not in most deployments > but say in case) so does the namenode recreate a file once the datanode > rejoins? > > > > > Thanks, > > Mehul > > -- Harsh J
Re: what happens when a datanode rejoins?
Hi Mehul, DataNode rejoins take care of only NameNode. Thanks & Regards, Ramesh.Narasingu On Tue, Sep 11, 2012 at 2:36 PM, Mehul Choube wrote: > > The namenode will asynchronously replicate the blocks to other > datanodes in order to maintain the replication factor after a datanode has > not been in contact for 10 minutes. > > What happens when the datanode rejoins after namenode has already > re-replicated the blocs it was managing? > > Will namenode ask the datanode to discard the blocks and start managing > new blocks? > > Or will namenode discard the new blocks which were replicated due to > unavailability of this datanode? > > ** ** > > ** ** > > ** ** > > Thanks, > > Mehul > > ** ** > > ** ** > > *From:* George Datskos [mailto:george.dats...@jp.fujitsu.com] > *Sent:* Tuesday, September 11, 2012 12:56 PM > *To:* user@hadoop.apache.org > *Subject:* Re: what happens when a datanode rejoins? > > ** ** > > Hi Mehul > > Some of the blocks it was managing are deleted/modified? > > > The namenode will asynchronously replicate the blocks to other datanodes > in order to maintain the replication factor after a datanode has not been > in contact for 10 minutes. > > > > > The size of the blocks are now modified say from 64MB to 128MB? > > > Block size is a per-file setting so new files will be 128MB, but the old > ones will remain at 64MB. > > > > > What if the block replication factor was one (yea not in most deployments > but say incase) so does the namenode recreate a file once the datanode > rejoins? > > > (assuming you didn't perform a decommission) Blocks that lived only on > that datanode will be declared "missing" and the files associated with > those blocks will be not be able to be fully read, until the datanode > rejoins. > > > > George >
Re: what happens when a datanode rejoins?
Hi, Inline. On Tue, Sep 11, 2012 at 2:36 PM, Mehul Choube wrote: >> The namenode will asynchronously replicate the blocks to other datanodes >> in order to maintain the replication factor after a datanode has not been in >> contact for 10 minutes. > > What happens when the datanode rejoins after namenode has already > re-replicated the blocs it was managing? The block count total goes +1, and the file's block is treated as an over-replicated one. > Will namenode ask the datanode to discard the blocks and start managing new > blocks? Yes, this may happen. > Or will namenode discard the new blocks which were replicated due to > unavailability of this datanode? It deletes extra blocks while still keeping the block placement policy in mind. It may delete any block replica as long as the placement policy is not violated by doing so. -- Harsh J
RE: what happens when a datanode rejoins?
>DataNode rejoins take care of only NameNode. Sorry didn't get this From: Narasingu Ramesh [mailto:ramesh.narasi...@gmail.com] Sent: Tuesday, September 11, 2012 2:38 PM To: user@hadoop.apache.org Subject: Re: what happens when a datanode rejoins? Hi Mehul, DataNode rejoins take care of only NameNode. Thanks & Regards, Ramesh.Narasingu On Tue, Sep 11, 2012 at 2:36 PM, Mehul Choube mailto:mehul_cho...@symantec.com>> wrote: > The namenode will asynchronously replicate the blocks to other datanodes in > order to maintain the replication factor after a datanode has not been in > contact for 10 minutes. What happens when the datanode rejoins after namenode has already re-replicated the blocs it was managing? Will namenode ask the datanode to discard the blocks and start managing new blocks? Or will namenode discard the new blocks which were replicated due to unavailability of this datanode? Thanks, Mehul From: George Datskos [mailto:george.dats...@jp.fujitsu.com<mailto:george.dats...@jp.fujitsu.com>] Sent: Tuesday, September 11, 2012 12:56 PM To: user@hadoop.apache.org<mailto:user@hadoop.apache.org> Subject: Re: what happens when a datanode rejoins? Hi Mehul Some of the blocks it was managing are deleted/modified? The namenode will asynchronously replicate the blocks to other datanodes in order to maintain the replication factor after a datanode has not been in contact for 10 minutes. The size of the blocks are now modified say from 64MB to 128MB? Block size is a per-file setting so new files will be 128MB, but the old ones will remain at 64MB. What if the block replication factor was one (yea not in most deployments but say incase) so does the namenode recreate a file once the datanode rejoins? (assuming you didn't perform a decommission) Blocks that lived only on that datanode will be declared "missing" and the files associated with those blocks will be not be able to be fully read, until the datanode rejoins. George