Re: HDFS Backup nodes

2011-12-14 Thread Scott Carey
-- From: M. C. Srivas mcsri...@gmail.com Date: Sun, Dec 11, 2011 at 10:47 PM Subject: Re: HDFS Backup nodes To: common-user@hadoop.apache.org You are out of luck if you don't want to use NFS, and yet want redundancy for the NN. Even the new NN HA work being done

Re: HDFS Backup nodes

2011-12-14 Thread Konstantin Boudnik
On Wed, Dec 14, 2011 at 10:09AM, Scott Carey wrote: On 12/13/11 11:28 PM, Konstantin Boudnik c...@apache.org wrote: On Tue, Dec 13, 2011 at 11:00PM, M. C. Srivas wrote: Suresh, As of today, there is no option except to use NFS. And as you yourself mention, the first HA prototype

Re: HDFS Backup nodes

2011-12-14 Thread Todd Lipcon
On Wed, Dec 14, 2011 at 10:00 AM, Scott Carey sc...@richrelevance.com wrote: As of today, there is no option except to use NFS.  And as you yourself mention, the first HA prototype when it comes out will require NFS. How will it 'require' NFS?  Won't any 'remote, high availability storage'

Re: HDFS Backup nodes

2011-12-13 Thread Suresh Srinivas
be used for HA. 2. We have a short term goal also to enable editlogs going to HDFS itself. The work is in progress. Regards, Suresh -- Forwarded message -- From: M. C. Srivas mcsri...@gmail.com Date: Sun, Dec 11, 2011 at 10:47 PM Subject: Re: HDFS Backup nodes To: common

Re: HDFS Backup nodes

2011-12-13 Thread Todd Lipcon
, randy - Original Message - From: Joey Echeverriaj...@cloudera.com To: common-user@hadoop.apache.org Sent: Wednesday, December 7, 2011 6:07:58 AM Subject: Re: HDFS Backup nodes You should also configure the Namenode to use an NFS mount for one of it's storage directories

Re: HDFS Backup nodes

2011-12-13 Thread M. C. Srivas
...@gmail.com Date: Sun, Dec 11, 2011 at 10:47 PM Subject: Re: HDFS Backup nodes To: common-user@hadoop.apache.org You are out of luck if you don't want to use NFS, and yet want redundancy for the NN. Even the new NN HA work being done by the community will require NFS ... and the NFS itself

Re: HDFS Backup nodes

2011-12-13 Thread Todd Lipcon
On Tue, Dec 13, 2011 at 10:42 PM, M. C. Srivas mcsri...@gmail.com wrote: Any simple file meta-data test will cause the NN to spiral to death with infinite GC.  For example, try create many many files. Or even simple stat a bunch of file continuously. Sure. If I run dd if=/dev/zero of=foo my

Re: HDFS Backup nodes

2011-12-13 Thread Konstantin Boudnik
to enable editlogs going to HDFS itself. The work is in progress. Regards, Suresh -- Forwarded message -- From: M. C. Srivas mcsri...@gmail.com Date: Sun, Dec 11, 2011 at 10:47 PM Subject: Re: HDFS Backup nodes To: common-user@hadoop.apache.org You

Re: HDFS Backup nodes

2011-12-13 Thread Todd Lipcon
mcsri...@gmail.com Date: Sun, Dec 11, 2011 at 10:47 PM Subject: Re: HDFS Backup nodes To: common-user@hadoop.apache.org You are out of luck if you don't want to use NFS, and yet want redundancy for the NN.  Even the new NN HA work being done by the community will require NFS

Re: HDFS Backup nodes

2011-12-11 Thread M. C. Srivas
Message - From: Joey Echeverriaj...@cloudera.com To: common-user@hadoop.apache.org Sent: Wednesday, December 7, 2011 6:07:58 AM Subject: Re: HDFS Backup nodes You should also configure the Namenode to use an NFS mount for one of it's storage directories. That will give the most up-to-date

RE: HDFS Backup nodes

2011-12-08 Thread Jorn Argelo - Ephorus
-Oorspronkelijk bericht- Van: Joey Echeverria [mailto:j...@cloudera.com] Verzonden: woensdag 7 december 2011 12:08 Aan: common-user@hadoop.apache.org Onderwerp: Re: HDFS Backup nodes You should also configure the Namenode to use an NFS mount for one of it's storage directories

RE: HDFS Backup nodes

2011-12-07 Thread Uma Maheswara Rao G
AFAIK backup node introduced in 0.21 version onwards. From: praveenesh kumar [praveen...@gmail.com] Sent: Wednesday, December 07, 2011 12:40 PM To: common-user@hadoop.apache.org Subject: HDFS Backup nodes Does hadoop 0.20.205 supports configuring HDFS

Re: HDFS Backup nodes

2011-12-07 Thread praveenesh kumar
This means still we are relying on Secondary NameNode idealogy for Namenode's backup. Can OS-mirroring of Namenode is a good alternative keep it alive all the time ? Thanks, Praveenesh On Wed, Dec 7, 2011 at 1:35 PM, Uma Maheswara Rao G mahesw...@huawei.comwrote: AFAIK backup node introduced

RE: HDFS Backup nodes

2011-12-07 Thread Sagar Shukla
[mailto:praveen...@gmail.com] Sent: Wednesday, December 07, 2011 1:47 PM To: common-user@hadoop.apache.org Subject: Re: HDFS Backup nodes This means still we are relying on Secondary NameNode idealogy for Namenode's backup. Can OS-mirroring of Namenode is a good alternative keep it alive all

Re: HDFS Backup nodes

2011-12-07 Thread Joey Echeverria
You should also configure the Namenode to use an NFS mount for one of it's storage directories. That will give the most up-to-date back of the metadata in case of total node failure. -Joey On Wed, Dec 7, 2011 at 3:17 AM, praveenesh kumar praveen...@gmail.com wrote: This means still we are

RE: HDFS Backup nodes

2011-12-07 Thread Jorn Argelo - Ephorus
moved away from NFS and I'm using DRBD instead. Not having any problems anymore whatsoever. YMMV. Jorn -Oorspronkelijk bericht- Van: Joey Echeverria [mailto:j...@cloudera.com] Verzonden: woensdag 7 december 2011 12:08 Aan: common-user@hadoop.apache.org Onderwerp: Re: HDFS Backup nodes

Re: HDFS Backup nodes

2011-12-07 Thread randysch
: HDFS Backup nodes You should also configure the Namenode to use an NFS mount for one of it's storage directories. That will give the most up-to-date back of the metadata in case of total node failure. -Joey On Wed, Dec 7, 2011 at 3:17 AM, praveenesh kumar praveen...@gmail.com wrote: This means

Re: HDFS Backup nodes

2011-12-07 Thread Joey Echeverria
@hadoop.apache.org Sent: Wednesday, December 7, 2011 6:07:58 AM Subject: Re: HDFS Backup nodes You should also configure the Namenode to use an NFS mount for one of it's storage directories. That will give the most up-to-date back of the metadata in case of total node failure. -Joey On Wed, Dec 7

Re: HDFS Backup nodes

2011-12-07 Thread randy
or isn't reachable? Does hdfs lock up? Does it gracefully ignore the nfs copy? Thanks, randy - Original Message - From: Joey Echeverriaj...@cloudera.com To: common-user@hadoop.apache.org Sent: Wednesday, December 7, 2011 6:07:58 AM Subject: Re: HDFS Backup nodes You should also configure

Re: HDFS Backup nodes

2011-12-07 Thread Harsh J
ignore the nfs copy? Thanks, randy - Original Message - From: Joey Echeverriaj...@cloudera.com To: common-user@hadoop.apache.org Sent: Wednesday, December 7, 2011 6:07:58 AM Subject: Re: HDFS Backup nodes You should also configure the Namenode to use an NFS mount for one of it's