Re: Storing data-node content to other machine

2009-04-28 Thread Steve Loughran

Vishal Ghawate wrote:

Hi,
I want to store the contents of all the client machine(datanode)of hadoop 
cluster to centralized machine
 with high storage capacity.so that tasktracker will be on the client machine 
but the contents are stored on the
centralized machine.
Can anybody help me on this please.



set the datanode to point to the (mounted) filesystem with the 
dfs.data.dir parameter.




Re: Storing data-node content to other machine

2009-04-27 Thread jason hadoop
There is no requirement that your hdfs and mapred clusters share an
installation directory, it is just done that way because it is simple and
most people have a datanode and tasktracker on each slave node.

Simply have 2 configuration directories on your cluster machines, and us the
bin/start-dfs.sh script in one, and the bin/start-mapred.sh script in the
other, and maintain different slaves files in the two directories.

You will loose the benefit of data locality for your tasktrackers which do
not reside on the datanode machines.

On Sun, Apr 26, 2009 at 10:06 PM, Vishal Ghawate <
vishal_ghaw...@persistent.co.in> wrote:

> Hi,
> I want to store the contents of all the client machine(datanode)of hadoop
> cluster to centralized machine
>  with high storage capacity.so that tasktracker will be on the client
> machine but the contents are stored on the
> centralized machine.
>Can anybody help me on this please.
>
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>



-- 
Alpha Chapters of my book on Hadoop are available
http://www.apress.com/book/view/9781430219422


Storing data-node content to other machine

2009-04-26 Thread Vishal Ghawate
Hi,
I want to store the contents of all the client machine(datanode)of hadoop 
cluster to centralized machine
 with high storage capacity.so that tasktracker will be on the client machine 
but the contents are stored on the
centralized machine.
Can anybody help me on this please.

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.