Hi,
We have a requirement where we have change our Hadoop Cluster's Replication
Factor without restarting the Cluster. We are running our Cluster on Amazon EMR.
Can you please suggest the way to achieve this? Any pointer to this will be
very helpful.
Thanks And Regards
Uddipan
Hi
You can change the replication factor of an existing directory using
'-setrep'
http://hadoop.apache.org/common/docs/r0.20.0/hdfs_shell.html#setrep
The below command will recursively set the replication factor to 1 for all
files within the given directory '/user'
hadoop fs -setrep -w 1 -R
Replication factor is per file option, So, you may have to write a small
program which will iterate over all files and set the replication factor to
desired one.
API: FileSystem#setReplication
Regards,
Uma
On Wed, Sep 5, 2012 at 11:39 PM, Uddipan Mukherjee
uddipan_mukher...@infosys.com wrote:
Replication factor is per-file, and is a client-side property. So, this is
doable.
1. Change the replication factor of all existing files (or needed ones):
$ hadoop fs -setrep -R value /
2. Change the dfs.replication parameter in all client configs to the
desired value
On Wed, Sep 5, 2012 at
[mailto:ha...@cloudera.com]
Sent: Wednesday, September 05, 2012 7:17 PM
To: user@hadoop.apache.org
Subject: Re: Replication Factor Modification
Replication factor is per-file, and is a client-side property. So, this is
doable.
1. Change the replication factor of all existing files (or needed
, 2012 7:17 PM
*To:* user@hadoop.apache.org
*Subject:* Re: Replication Factor Modification
** **
Replication factor is per-file, and is a client-side property. So, this is
doable.
** **
1. Change the replication factor of all existing files (or needed ones):**
**
** **
$ hadoop fs
Hi Uddippan,
Check out the following link for setrep command in Hadoop:
http://hadoop.apache.org/common/docs/r0.20.0/hdfs_shell.html#setrep
You don't need to restart the cluster after running the command.
HTH,
Anil
On Wed, Sep 5, 2012 at 11:02 AM, Uddipan Mukherjee
