Re: Best way to handle namespace host failures

2008-11-12 Thread Allen Wittenauer
On 11/10/08 10:42 PM, Dhruba Borthakur [EMAIL PROTECTED] wrote: 2. Create a virtual IP, say name.xx.com that points to the real machine name of the machine on which the namenode runs. Everyone doing this should be aware of the discussion happening in

Re: Best way to handle namespace host failures

2008-11-10 Thread Sharad Agarwal
Goel, Ankur wrote: Hi Folks, I am looking for some advice on some the ways / techniques that people are using to get around namenode failures (Both disk and host). We have a small cluster with several job scheduled for periodic execution on the same host where name server runs.

Re: Best way to handle namespace host failures

2008-11-10 Thread Amar Kamat
Goel, Ankur wrote: Hi Folks, I am looking for some advice on some the ways / techniques that people are using to get around namenode failures (Both disk and host). We have a small cluster with several job scheduled for periodic execution on the same host where name server runs.

RE: Best way to handle namespace host failures

2008-11-10 Thread Goel, Ankur
- From: Amar Kamat [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 3:53 PM To: core-user@hadoop.apache.org Subject: Re: Best way to handle namespace host failures Goel, Ankur wrote: Hi Folks, I am looking for some advice on some the ways / techniques that people

Re: Best way to handle namespace host failures

2008-11-10 Thread Alex Loddengaard
smoothly in production without abrupt failures. Thanks -Ankur -Original Message- From: Amar Kamat [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 3:53 PM To: core-user@hadoop.apache.org Subject: Re: Best way to handle namespace host failures Goel, Ankur wrote: Hi Folks

RE: Best way to handle namespace host failures

2008-11-10 Thread Goel, Ankur
@hadoop.apache.org Cc: Ian Holsman Subject: Re: Best way to handle namespace host failures There has been a lot of discussion on this list about handling namenode failover. Generally the most common approach is to backup the namenode to an NFS mount and manually instantiate a new namenode when your current

Re: Best way to handle namespace host failures

2008-11-10 Thread Dhruba Borthakur
Couple of things that one can do: 1. dfs.name.dir should have at least two locations, one on the local disk and one on NFS. This means that all transactions are synchronously logged into two places. 2. Create a virtual IP, say name.xx.com that points to the real machine name of the machine on