Re: Secondary namenode fsimage concept

2011-10-11 Thread Shouguo Li
the1plum...@gmail.com Date: Tuesday, October 11, 2011 2:31 am Subject: Re: Secondary namenode fsimage concept To: common-user@hadoop.apache.org hey parick i wanted to configure my cluster to write namenode metadata to multipledirectories as well: property

Re: Secondary namenode fsimage concept

2011-10-10 Thread Shouguo Li
hey parick i wanted to configure my cluster to write namenode metadata to multiple directories as well: property namedfs.name.dir/name value/hadoop/var/name,/mnt/hadoop/var/name/value /property in my case, /hadoop/var/name is local directory, /mnt/hadoop/var/name is NFS volume. i

Re: Secondary namenode fsimage concept

2011-10-10 Thread Uma Maheswara Rao G 72686
: Shouguo Li the1plum...@gmail.com Date: Tuesday, October 11, 2011 2:31 am Subject: Re: Secondary namenode fsimage concept To: common-user@hadoop.apache.org hey parick i wanted to configure my cluster to write namenode metadata to multipledirectories as well: property namedfs.name.dir

Re: Secondary namenode fsimage concept

2011-10-10 Thread Harsh J
:31 am Subject: Re: Secondary namenode fsimage concept To: common-user@hadoop.apache.org hey parick i wanted to configure my cluster to write namenode metadata to multipledirectories as well:  property    namedfs.name.dir/name    value/hadoop/var/name,/mnt/hadoop/var/name/value  /property

Re: Secondary namenode fsimage concept

2011-10-06 Thread Kai Voigt
Hi, you're correct when saying the namenode hosts the fsimage file and the edits log file. The fsimage file contains a snapshot of the HDFS metadata (a filename to blocks list mapping). Whenever there is a change to HDFS, it will be appended to the edits file. Think of it as a database

Re: Secondary namenode fsimage concept

2011-10-06 Thread shanmuganathan.r
Hi Kai, In the Second part I meant Is the secondary namenode also contain the FSImage file or the two files(FSImage and EdiltLog) are transferred from the namenode at the checkpoint time. Thanks Shanmuganathan On Thu, 06 Oct 2011 11:37:50 +0530 Kai Voigtlt;k...@123.orggt;

Re: Secondary namenode fsimage concept

2011-10-06 Thread Kai Voigt
Hi, the secondary namenode only fetches the two files when a checkpointing is needed. Kai Am 06.10.2011 um 08:45 schrieb shanmuganathan.r: Hi Kai, In the Second part I meant Is the secondary namenode also contain the FSImage file or the two files(FSImage and EdiltLog) are

Re: Secondary namenode fsimage concept

2011-10-06 Thread shanmuganathan.r
Hi Kai, There is no datas stored in the secondarynamenode related to the Hadoop cluster . Am I correct? If it correct means If we run the secondaryname node in separate machine then fetching , merging and transferring time is increased if the cluster has large data in the namenode

Re: Secondary namenode fsimage concept

2011-10-06 Thread Kai Voigt
Hi, yes, the secondary namenode is actually a badly named piece of software, as it's not a namenode at all. And it's going to be renamed to checkpoint node. To prevent metadata loss when your namenode fails, you should write the namenode files to a local RAID and also a networked storage (NFS,

Re: Secondary namenode fsimage concept

2011-10-06 Thread patrick sang
I would say your namenode write metadata in local fs (where your secondary namenode will pull files), and NFS mount. property namedfs.name.dir/name value/hadoop/name,/hadoop/nfs_server_name/value /property my 0.02$ P On Thu, Oct 6, 2011 at 12:04 AM, shanmuganathan.r

Secondary namenode fsimage concept

2011-10-05 Thread shanmuganathan.r
Hi All, I have a doubt in hadoop secondary namenode concept . Please correct if the following statements are wrong . The namenode hosts the fsimage and edit log files. The secondary namenode hosts the fsimage file only. At the time of checkpoint the edit log file is transferred