Hi Dhiraj,
AFAIK there is a mechanism to pass the set of 'favoredNodes datanodes' that
should be favored as targets while creating a file. But this is only considered
as a hint, sometimes due to cluster state(for example: given dn doesn't have
sufficient space, doesn't available etc.), namenode may not be able to place
the blocks on these datanodes then the hadoop replication works as per its
policy. Please go through the below api which can be used for achieving this
behavior and hope this will help you to get some idea.
DistributedFileSystem.java
/* Same as
* {@link #create(Path, FsPermission, boolean, int, short, long,
* Progressable)} with the addition of favoredNodes that is a hint to
* where the namenode should place the file blocks.
* The favored nodes hint is not persisted in HDFS. Hence it may be honored
* at the creation time only. HDFS could move the blocks during balancing or
* replication, to move the blocks from favored nodes. A value of null means
* no favored nodes for this create
*/
public HdfsDataOutputStream create(final Path f,
final FsPermission permission, final boolean overwrite,
final int bufferSize, final short replication, final long blockSize,
final Progressable progress, final InetSocketAddress[] favoredNodes)
Regards,
Rakesh
From: Dhiraj Kamble [mailto:dhiraj.kam...@sandisk.com]
Sent: 14 October 2014 10:38
To: user@hadoop.apache.org
Subject: Redirect Writes/Reads to a particular Node
Hi,
Is it possible to redirect writes to one particular node i.e. store the primary
replica always on the same node; and have reads served from this primary node.
If the primary node goes down; then hadoop replication works as per its policy;
but when this node comes up it should again become the primary node. I don't
see any config parameter available for core-site.xml or hdfs-site.xml to serve
this purpose.
Is there any way I can do this.
Regards,
Dhiraj
PLEASE NOTE: The information contained in this electronic mail message is
intended only for the use of the designated recipient(s) named above. If the
reader of this message is not the intended recipient, you are hereby notified
that you have received this message in error and that any review,
dissemination, distribution, or copying of this message is strictly prohibited.
If you have received this communication in error, please notify the sender by
telephone or e-mail (as shown above) immediately and destroy any and all copies
of this message in your possession (whether hard copies or electronically
stored copies).
