You mentioned in your email that "total data size varies between about 1 &
2K". I am guessing you meant by this that your individual record size
varies between 1 & 2K.
If that is true, there is a good chance that you might be hitting the CMS
occupancy fraction sooner than otherwise due to a varyin
Great, thanks very much for the help. I'm going to see if I can get more
memory into the servers and will also experiment with XX:ParallelGCThreads.
We already have XX:CMSInitiatingOccupancyFraction=70 in the config.
Uday, what do you mean by "a fixed size record"? Do you mean the record
that is b
On Fri, May 18, 2012 at 4:54 AM, Simon Kelly wrote:
> Cluster details
> CPUs: 4 x 2.39 GHz
> RAM: 8 GB
> Ubuntu 10.04.2 LTS
>
> 1230 - hbase logs for server1 record: We slept 13318ms instead of 3000ms and
> regionserver1 is killed by master, request_processing_time goes way up
> 1326 - hbase logs
Head over to Cloudera's site and look at a couple of blog posts from Todd
Lipcon.
Also look at MSLABs .
On a side note... you don't have a lot of memory to play with...
On May 18, 2012, at 6:54 AM, Simon Kelly wrote:
> Hi
>
> Firstly, let me complement the Hbase team on a great piece of softw
Hi
Firstly, let me complement the Hbase team on a great piece of software.
We're running a few clusters that are working well but we're really
struggling with a new one I'm trying to setup and could use a bit of help.
I have read as much as I can but just can't seem to get it right.
The differenc
Hi Friso,
Great to know! Todd was the last one to try to crash G1 and the recent
iteration seemed much more stable.
Lars
On Nov 29, 2010, at 10:49, Friso van Vollenhoven
wrote:
> On a slightly related note, we've been running with G1 with default settings
> on a 16GB heap for some weeks no
On Mon, Nov 29, 2010 at 6:33 AM, Sean Sechrist wrote:
> Just an update, in case anyone's interested in our performance numbers:
>
> With the 512MB newSize, our minor GC pauses are generally less than .05s,
> although we see a fair amount get up around .15s. We still see some
> promotion failures
Just an update, in case anyone's interested in our performance numbers:
With the 512MB newSize, our minor GC pauses are generally less than .05s,
although we see a fair amount get up around .15s. We still see some
promotion failures causing full pauses over a minute occasionally. But we
have a scr
On a slightly related note, we've been running with G1 with default settings on
a 16GB heap for some weeks now. It's never given us trouble, so I didn't do any
real analysis on the GC times, just some eye balling.
I looked at the longer GCs (everything longer than 1 second: grep -C 5 -i
real=[1
I'd love to hear the kinds of minor pauses you get... left alone to
it's devices, 1.6.0_14 or so wants to grow the new gen to 1gb if your
xmx is large enough, at that size you are looking at 800ms minor
pauses!
It's a tough subject.
-ryan
On Wed, Nov 24, 2010 at 12:52 PM, Sean Sechrist wrote:
>
Just wanted to add to Todd's explanation this link:
http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html (Java
SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning).
It gives more detailed (to some extent of course, on this deep topic)
description on what Todd mentioned.
Ale
Interesting. The settings we tried earlier today slowed jobs significantly,
but no failures (yet). We're going to try the 512MB newSize and 60%
CMSInitiatingOccupancyFraction. 1 second pauses here and there would be OK
for us we just want to avoid the long pauses right now. We'll also do
what w
On Wed, Nov 24, 2010 at 7:01 AM, Sean Sechrist wrote:
> Hey guys,
>
> I just want to get an idea about how everyone avoids these long GC pauses
> that cause regionservers to die.
>
> What kind of java heap and garbage collection settings do you use?
>
> What do you do to make sure that the HBase
Setting swappiness to 0 is one thing, but does it swap at all? If so,
then it's definitely a problem and the fact that the real was 4x
higher than the user CPU on that big GC pause strongly indicates
swapping. Setup ganglia, and watch your swap. The typical error is
setting too many tasks per node
Hey guys,
I just want to get an idea about how everyone avoids these long GC pauses
that cause regionservers to die.
What kind of java heap and garbage collection settings do you use?
What do you do to make sure that the HBase vm never uses swap? I have heard
turning off swap altogether can be d
15 matches
Mail list logo
