er... is that: Set up a DNS server, use hostnames instead of raw ips?
Config all node in the slaves file, and put this file on the namenode and secondary namenode to prevent accidents? Use a real system configuration management package to sync software in all nodes of cluster? Thanks for all, and Alex Loddengaard's creation of wiki are commendable, anyone should enrich it. 2008/9/12 Alex Loddengaard <[EMAIL PROTECTED]> > My inexperience has been revealed ;). I've taken your comments, James and > Allen, and added them to the wiki: > > <http://wiki.apache.org/hadoop/LargeClusterTips> > > Alex > > On Fri, Sep 12, 2008 at 2:01 AM, James Moore <[EMAIL PROTECTED] > >wrote: > > > On Thu, Sep 11, 2008 at 5:46 AM, Allen Wittenauer <[EMAIL PROTECTED]> > > wrote: > > > On 9/11/08 2:39 AM, "Alex Loddengaard" <[EMAIL PROTECTED]> wrote: > > >> I've never dealt with a large cluster, though I'd imagine it is > managed > > the > > >> same way as small clusters: > > > > > > Maybe. :) > > > > Add me to the "maybe :)" column. In my experience, large rarely turns > > out the same as small. > > > > What usually happens is that the developers build the small thing, > > keeping in mind that good sysadmins are going to need to do some work > > turn it into the large thing. (Never underestimate the value of good > > system admin people. Speaking as a developer, good sysadmins will > > almost always know something about "large" that you haven't thought > > about.) > > > > I think what I was doing on a small cluster (100 machines) would take > > some modifications to scale up. > > > > >> -Use hostnames or ips, whichever is more convenient for you > > > > > > Use hostnames. Seriously. Who are you people using raw IPs for > > things? > > > :) Besides, you're going to need it for the eventual support of > > Kerberos. > > > > I suspect lots of people buy arrays by the hour from Amazon, so you're > > going to have a different batch of IP addresses every > > $WHATEVER_PERIOD. Not having to worry about dynamic dns is probably > > interesting to someone. (Our plan was to spin up an array of 100 or > > so servers every N days, work for a few hours, then shut down.) > > > > Dynamic DNS sounded like a pain to me only because I'm a really bad > > system administrator - it may be that it's worth it (or trivial). > > > > > Instead, use a real system configuration management package such as > > > bcfg2, smartfrog, puppet, cfengine, etc. [Steve, you owe me for the > > plug. > > > :) ] > > > > Or on EC2 and its competitors, just build a new image whenever you > > need to update Hadoop itself. > > > > -- > > James Moore | [EMAIL PROTECTED] > > Ruby and Ruby on Rails consulting > > blog.restphone.com > > > -- Sorry for my english!! 明 Please help me to correct my english expression and error in syntax
