Also I second those JVM options, they can and do cause stability issues.
On Fri, Aug 13, 2010 at 12:17 PM, Jeremy Carroll <[email protected]> wrote: > I would second upping the NameNode RAM. Most name nodes have the most ram of > any server in the cluster. Make sure you are not storing small files and have > a very high block count. From the article that I linked below about 10 > million files = 3Gb of JVM heap for the NameNode. > > http://www.cloudera.com/blog/2009/02/the-small-files-problem/ > > ________________________________________ > From: Edward Capriolo [[email protected]] > Sent: Friday, August 13, 2010 2:16 PM > To: [email protected] > Subject: Re: Fw: namenode crash > > On Fri, Aug 13, 2010 at 3:03 PM, Ryan Rawson <[email protected]> wrote: >> We don't use centos here at Stumbleupon... your version looks quite >> old! Our uname looks like: >> >> Linux host 2.6.28-14-generic #47-Ubuntu SMP Sat Jul 25 01:19:55 UTC >> 2009 x86_64 GNU/Linux >> >> I'd consider using something newer than 2.6.18! >> >> On Fri, Aug 13, 2010 at 11:54 AM, Jean-Daniel Cryans >> <[email protected]> wrote: >>> u18 should never be used. >>> >>> You say it's crashing on both u17 and u20? How is it crashing? (it's >>> kind of a vague word) >>> >>> Here with use both u14 and u17 on 20 nodes clusters without any issue. >>> >>> J-D >>> >>> On Fri, Aug 13, 2010 at 11:27 AM, Jinsong Hu <[email protected]> wrote: >>>> >>>> >>>> Hi, There: >>>> does anybody know of a good combination of centos version and jdk version >>>> that works stably ? I am using centos version >>>> >>>> Linux 2.6.18-194.8.1.el5.centos.plus #1 SMP Wed Jul 7 11:45:38 EDT 2010 >>>> x86_64 x86_64 x86_64 GNU/Linux >>>> >>>> jdk version >>>> Java(TM) SE Runtime Environment (build 1.6.0_17-b04) >>>> Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode) >>>> >>>> and run the namenode with the following jvm config >>>> -Xmx1000m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode >>>> -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError >>>> -XX:+UseCompressedOops -XX:+DoEscapeAnalysis -XX:+AggressiveOpts -Xmx2G >>>> >>>> but it crashed silently after 16 hours. >>>> >>>> I used jdk >>>> Java(TM) SE Runtime Environment (build 1.6.0_20-b02) >>>> Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode) >>>> >>>> with the same jvm config, and the namenode crashed in about 1 week. I >>>> searched internet and people say 1.6.0_18 is not good. >>>> but does anybody can recommend a good combination of jdk and os version >>>> that can run stably ? >>>> >>>> >>>> This crashing problem doesn't happen with a small cluster of 4 datanodes. >>>> but it happens with a cluster of 17 datanodes. >>>> >>>> Jimmy. >>>> >>>> >>>> >>> >> > > RedHat/CentOS backport kernel patches and attempt to keep the minor > number relatively stable. > > Something like 2.6.18-194 is probably closer to 2.6.28 then 2.6.18. > > Do you have any more free memory? Maybe for fun raise you -Xmx4G. > > Edward >
