Re: Doubt regarding Replication Factor
This exactly true. Thanks for correcting, Brian. --Konstantin Brian Bockelman wrote: On Aug 12, 2009, at 2:11 PM, Konstantin Shvachko wrote: You can try it: start a 3 node cluster and create a file with replication 5. The answer is that each data-node can store only one replica of a block. So in your case you will get an exception on close() saying the file cannot be fully replicated. I'm not sure if that's true. You only get an exception if the system cannot provide the minimum replication factor. I.e, if you ask for 5 replicas and the minimum replicas is 3, then you only get an exception if HDFS can't meet 3 replicas. Am I recalling this wrong? We have "discovered" this fact at least once :) Brian Thanks, --Konstantin Rakhi Khatwani wrote: Hi, I just wanted to know what if we have set the replication factor greater than the number of nodes in the cluster. for example, i have only 3 nodes in my cluster but i set the replication factor to 5. will it create 3 copies and save it in each node, or can it create more than one copy per node? Regards, Raakhi Khatwani
Re: Doubt regarding Replication Factor
If the number of replicas meets the minimum replication factor, no exception will be thrown. The default replication factor is 1. So as long as your cluster has at least one node with available space, file creations should be fine. Hairong On 8/12/09 12:55 PM, "Tarandeep Singh" wrote: > A similar question- > > If in an N node cluster, a file's replication is set to N (replicate on each > node) and later if a node goes down, will HDFS throw an exception since the > file's replication has gone down below the specified number ? > > Thanks, > Tarandeep > > On Wed, Aug 12, 2009 at 12:11 PM, Konstantin Shvachko > wrote: > >> You can try it: start a 3 node cluster and create a file with replication >> 5. >> >> The answer is that each data-node can store only one replica of a block. >> So in your case you will get an exception on close() saying the file cannot >> be fully replicated. >> >> Thanks, >> --Konstantin >> >> >> Rakhi Khatwani wrote: >> >>> Hi, >>> I just wanted to know what if we have set the replication factor greater >>> than the number of nodes in the cluster. >>> for example, i have only 3 nodes in my cluster but i set the replication >>> factor to 5. >>> >>> will it create 3 copies and save it in each node, or can it create more >>> than >>> one copy per node? >>> >>> Regards, >>> Raakhi Khatwani >>> >>>
Re: Doubt regarding Replication Factor
On Aug 12, 2009, at 2:11 PM, Konstantin Shvachko wrote: You can try it: start a 3 node cluster and create a file with replication 5. The answer is that each data-node can store only one replica of a block. So in your case you will get an exception on close() saying the file cannot be fully replicated. I'm not sure if that's true. You only get an exception if the system cannot provide the minimum replication factor. I.e, if you ask for 5 replicas and the minimum replicas is 3, then you only get an exception if HDFS can't meet 3 replicas. Am I recalling this wrong? We have "discovered" this fact at least once :) Brian Thanks, --Konstantin Rakhi Khatwani wrote: Hi, I just wanted to know what if we have set the replication factor greater than the number of nodes in the cluster. for example, i have only 3 nodes in my cluster but i set the replication factor to 5. will it create 3 copies and save it in each node, or can it create more than one copy per node? Regards, Raakhi Khatwani smime.p7s Description: S/MIME cryptographic signature
Re: Doubt regarding Replication Factor
A similar question- If in an N node cluster, a file's replication is set to N (replicate on each node) and later if a node goes down, will HDFS throw an exception since the file's replication has gone down below the specified number ? Thanks, Tarandeep On Wed, Aug 12, 2009 at 12:11 PM, Konstantin Shvachko wrote: > You can try it: start a 3 node cluster and create a file with replication > 5. > > The answer is that each data-node can store only one replica of a block. > So in your case you will get an exception on close() saying the file cannot > be fully replicated. > > Thanks, > --Konstantin > > > Rakhi Khatwani wrote: > >> Hi, >> I just wanted to know what if we have set the replication factor greater >> than the number of nodes in the cluster. >> for example, i have only 3 nodes in my cluster but i set the replication >> factor to 5. >> >> will it create 3 copies and save it in each node, or can it create more >> than >> one copy per node? >> >> Regards, >> Raakhi Khatwani >> >>
Re: Doubt regarding Replication Factor
You can try it: start a 3 node cluster and create a file with replication 5. The answer is that each data-node can store only one replica of a block. So in your case you will get an exception on close() saying the file cannot be fully replicated. Thanks, --Konstantin Rakhi Khatwani wrote: Hi, I just wanted to know what if we have set the replication factor greater than the number of nodes in the cluster. for example, i have only 3 nodes in my cluster but i set the replication factor to 5. will it create 3 copies and save it in each node, or can it create more than one copy per node? Regards, Raakhi Khatwani
Doubt regarding Replication Factor
Hi, I just wanted to know what if we have set the replication factor greater than the number of nodes in the cluster. for example, i have only 3 nodes in my cluster but i set the replication factor to 5. will it create 3 copies and save it in each node, or can it create more than one copy per node? Regards, Raakhi Khatwani
