Re: Hmaster and HRegionServer disappearance reason to ask

2012-07-02 Thread Lars George
Hi lztaomin, > org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode > = Session expired indicates that you have experienced the "Juliet Pause" issue, which means you ran into a JVM garbage collection that lasted longer than the configured ZooKeeper timeout threshold.

Re: Hmaster and HRegionServer disappearance reason to ask

2012-07-05 Thread N Keywal
Hi, It's a ZK expiry on sunday 1st. Root cause could be the leap second bug? N. On Thu, Jul 5, 2012 at 8:59 AM, lztaomin wrote: > HI ALL > My HBase group a total of 3 machine, Hadoop HBase mounted in the same > machine, zookeeper using HBase own. Operation 3 months after the reported >

RE: Hmaster and HRegionServer disappearance reason to ask

2012-07-05 Thread Pablo Musa
part means? Each line is a GC cicle? Thanks, Pablo -Original Message- From: Lars George [mailto:lars.geo...@gmail.com] Sent: segunda-feira, 2 de julho de 2012 06:43 To: user@hbase.apache.org Subject: Re: Hmaster and HRegionServer disappearance reason to ask Hi lztaomin, > org.apach

Re: Hmaster and HRegionServer disappearance reason to ask

2012-07-05 Thread Dhaval Shah
Times: user=0.07 sys=151.48, real=151.47 secs] What does each part means? Each line is a GC cicle? Thanks, Pablo -Original Message- From: Lars George [mailto:lars.geo...@gmail.com] Sent: segunda-feira, 2 de julho de 2012 06:43 To: user@hbase.apache.org Subject: Re: Hmaster and HRegionServ

re: Hmaster and HRegionServer disappearance reason to ask

2012-07-05 Thread Gaojinchao
Did you check "http://hbase.apache.org/book.html#perf.os.swap";? -邮件原件- 发件人: Pablo Musa [mailto:pa...@psafe.com] 发送时间: 2012年7月6日 5:38 收件人: user@hbase.apache.org 主题: RE: Hmaster and HRegionServer disappearance reason to ask I am having the same problem. I tried N different th

RE: Hmaster and HRegionServer disappearance reason to ask

2012-07-10 Thread Pablo Musa
ta-feira, 5 de julho de 2012 20:55 To: user@hbase.apache.org Subject: Re: Hmaster and HRegionServer disappearance reason to ask  Pablo, instead of CMSIncrementalMode try UseParNewGC.. That seemed to be the silver bullet when I was dealing with HBase region server crashes Rega

RE: Hmaster and HRegionServer disappearance reason to ask

2012-07-10 Thread Gopinathan A
m: Pablo Musa [mailto:pa...@psafe.com] Sent: Tuesday, July 10, 2012 7:58 PM To: user@hbase.apache.org Subject: RE: Hmaster and HRegionServer disappearance reason to ask I tried to change the flag but yesterday it happened again: Application time: 0.3025790 seconds 30013.866: [GC 30013.866: [ParNew:

RE: Hmaster and HRegionServer disappearance reason to ask

2012-07-10 Thread Laxman
his HDFS-3600. Please let us know your findings. -- Regards, Laxman > -Original Message- > From: Pablo Musa [mailto:pa...@psafe.com] > Sent: Tuesday, July 10, 2012 7:58 PM > To: user@hbase.apache.org > Subject: RE: Hmaster and HRegionServer disappearance reason to ask > &

RE: Hmaster and HRegionServer disappearance reason to ask

2012-07-11 Thread Pablo Musa
RE: Hmaster and HRegionServer disappearance reason to ask IIUC, this seems to be a problem with Direct memory cleanup. I've seen this problem mostly with application using lot of NIO (RegionServer, DataNode). Symptoms of this problem 1) Allocated heap is 1 GB 2) VIRT/RES will occupy 10GB or e

RE: Hmaster and HRegionServer disappearance reason to ask

2012-07-11 Thread Laxman
> > 1) Fix the direct memory usage to a fixed value - > XX:MaxDirectMemorySize=1G > > This flag should be in RS ou DN? We need to apply for both but limit can be increased based on your load (May be 2G). Also we can to apply to all processes which are having following symptoms. 1) Allocated heap

RE: Hmaster and HRegionServer disappearance reason to ask

2012-07-16 Thread Pablo Musa
ge- From: Laxman [mailto:lakshman...@huawei.com] Sent: quinta-feira, 12 de julho de 2012 01:22 To: Pablo Musa; user@hbase.apache.org Subject: RE: Hmaster and HRegionServer disappearance reason to ask > > 1) Fix the direct memory usage to a fixed value - > XX:MaxDirectMemorySize=1G > &

RE: Hmaster and HRegionServer disappearance reason to ask

2012-07-16 Thread Sandy Pratt
To: 'Pablo Musa'; user@hbase.apache.org Subject: RE: Hmaster and HRegionServer disappearance reason to ask > > 1) Fix the direct memory usage to a fixed value - > XX:MaxDirectMemorySize=1G > > This flag should be in RS ou DN? We need to apply for both but limit can be incre