See this regarding instructions on configuring a 2NN on a separate machine from the NN: http://www.cloudera.com/blog/2009/02/10/multi-host-secondarynamenode-configuration/
- Aaron On Thu, May 14, 2009 at 10:42 AM, Koji Noguchi <knogu...@yahoo-inc.com>wrote: > Before 0.19, fsimage/edits were on the same directory. > So whenever secondary finishes checkpointing, it copies back the fsimage > while namenode still kept on writing to the edits file. > > Usually we observed some latency on the namenode side during that time. > > HADOOP-3948 would probably help after 0.19 or later. > > Koji > > -----Original Message----- > From: Brian Bockelman [mailto:bbock...@cse.unl.edu] > Sent: Thursday, May 14, 2009 10:32 AM > To: core-user@hadoop.apache.org > Subject: Re: Setting up another machine as secondary node > > Hey Koji, > > It's an expensive operation - for the secondary namenode, not the > namenode itself, right? I don't particularly care if I stress out a > dedicated node that doesn't have to respond to queries ;) > > Locally we checkpoint+backup fairly frequently (not 5 minutes ... > maybe less than the default hour) due to sheer paranoia of losing > metadata. > > Brian > > On May 14, 2009, at 12:25 PM, Koji Noguchi wrote: > > >> The secondary namenode takes a snapshot > >> at 5 minute (configurable) intervals, > >> > > This is a bit too aggressive. > > Checkpointing is still an expensive operation. > > I'd say every hour or even every day. > > > > Isn't the default 3600 seconds? > > > > Koji > > > > -----Original Message----- > > From: jason hadoop [mailto:jason.had...@gmail.com] > > Sent: Thursday, May 14, 2009 7:46 AM > > To: core-user@hadoop.apache.org > > Subject: Re: Setting up another machine as secondary node > > > > any machine put in the conf/masters file becomes a secondary namenode. > > > > At some point there was confusion on the safety of more than one > > machine, > > which I believe was settled, as many are safe. > > > > The secondary namenode takes a snapshot at 5 minute (configurable) > > intervals, rebuilds the fsimage and sends that back to the namenode. > > There is some performance advantage of having it on the local machine, > > and > > some safety advantage of having it on an alternate machine. > > Could someone who remembers speak up on the single vrs multiple > > secondary > > namenodes? > > > > > > On Thu, May 14, 2009 at 6:07 AM, David Ritch <david.ri...@gmail.com> > > wrote: > > > >> First of all, the secondary namenode is not a what you might think a > >> secondary is - it's not failover device. It does make a copy of the > >> filesystem metadata periodically, and it integrates the edits into > >> the > >> image. It does *not* provide failover. > >> > >> Second, you specify its IP address in hadoop-site.xml. This is where > > you > >> can override the defaults set in hadoop-default.xml. > >> > >> dbr > >> > >> On Thu, May 14, 2009 at 9:03 AM, Rakhi Khatwani > > <rakhi.khatw...@gmail.com > >>> wrote: > >> > >>> Hi, > >>> I wanna set up a cluster of 5 nodes in such a way that > >>> node1 - master > >>> node2 - secondary namenode > >>> node3 - slave > >>> node4 - slave > >>> node5 - slave > >>> > >>> > >>> How do we go about that? > >>> there is no property in hadoop-env where i can set the ip-address > > for > >>> secondary name node. > >>> > >>> if i set node-1 and node-2 in masters, and when we start dfs, in > > both the > >>> m/cs, the namenode n secondary namenode processes r present. but i > > think > >>> only node1 is active. > >>> n my namenode fail over operation fails. > >>> > >>> ny suggesstions? > >>> > >>> Regards, > >>> Rakhi > >>> > >> > > > > > > > > -- > > Alpha Chapters of my book on Hadoop are available > > http://www.apress.com/book/view/9781430219422 > > www.prohadoopbook.com a community for Hadoop Professionals > >
