[
https://issues.apache.org/jira/browse/HBASE-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544437#comment-15544437
]
ramkrishna.s.vasudevan edited comment on HBASE-15561 at 10/7/16 11:29 AM:
--------------------------------------------------------------------------
Just wanted to update this JIRa with our findings and result over here. If
things are discussed we could close this JIRA.
There are two parts to this
-> testing trunk (default memstore) with *MSLAB and chunk pool ON* and without
MSLAB with G1GC configs
-> Next is to test G1GC with defaut memstore and offheap memstore
(HBASE-15179). For now this offheap is not in trunk. It is a POC work. Working
on getting things committed to trunk.
cluster setup
==========
The set up is a single node set up with one RS and the client in the same
machine. The machine has 150G of RAM.
GC config:
========
export HBASE_OPTS="-XX:+UseG1GC -XX:MaxGCPauseMillis=200
-XX:InitiatingHeapOccupancyPercent=60 -XX:G1HeapWastePercent=20
-XX:G1MixedGCCountTarget=8"
<property>
<name>hbase.regionserver.global.memstore.size</name>
<value>0.42</value>
</property>
PE command:
===========
./hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --presplit=50
--size=150 --columns=50 --valueSize=200 --oneCon=true --writeToWAL=false
--bloomFilter=NONE --inmemoryCompaction=false randomWrite 100
We think MSLAB and chunk pool is important mainly because when you move things
to offheap we need to ensure that we are managing the offheap buffers
efficiently. It is very dangerous to allocate offheap buffers and keep them
unclaimed which will lead to offheap memory leak. Whereas in onheap buffers
cases we
are able to manage it with the help of GC.
Hence it is always recommended for offheap memstore to go with MSLAB and chunk
pool.
Perf readings
==========
For details on onheap and offheap memstore comparison with MSLAB and chunk pool
pls refer to these docs and various GC configs
https://docs.google.com/document/d/1fj5P8JeutQ-Uadb29ChDscMuMaJqaMNRI86C4k5S1rQ/edit
https://docs.google.com/document/d/1bYIkwNVVyYk8bgik-C5x_yc_2hWHyi2-9KMwEFWLfzY/edit
The above experiments were done on top of a POc branch where we had our own
changes to support offheap memstore including some PB changes.
With the current trunk we performed experiments with 50 threads and 100 threads
and with 64 G heap memory.
With 100 threads and 10 cols per row pushed from a single client the setup with
MSLAB and chunk pool performs 13% better than the no MSLAB case.
With 50 threads and 10 cols per row pushed from a single client the setup with
MSLAB and chunk pool performs 11% better than the no MSLAB case.
PE completion time for 50 threads, 10 cols
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|1132.923 secs|1262 secs|
PE completion time for 100 threads, 10 cols
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|1947.330 secs|2214.201 secs|
No of GC pauses for 100 threads
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|403|475|
GC pause in secs for 100 threads
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|181.58 secs|239.9 secs|
Pls feel free to share your feedback or thoughts on this. It helps to see if
there is something to be tested or some configs needs to be changed and the
tests has to be repeated.
was (Author: ram_krish):
Just wanted to update this JIRa with our findings and result over here. If
things are discussed we could close this JIRA.
There are two parts to this
-> testing trunk (default memstore) with MSLAB and chunk pool ON and without
MSLAB with G1GC configs
-> Next is to test G1GC with defaut memstore and offheap memstore
(HBASE-15179). For now this offheap is not in trunk. It is a POC work. Working
on getting things committed to trunk.
cluster setup
==========
The set up is a single node set up with one RS and the client in the same
machine. The machine has 150G of RAM.
GC config:
========
export HBASE_OPTS="-XX:+UseG1GC -XX:MaxGCPauseMillis=200
-XX:InitiatingHeapOccupancyPercent=60 -XX:G1HeapWastePercent=20
-XX:G1MixedGCCountTarget=8"
<property>
<name>hbase.regionserver.global.memstore.size</name>
<value>0.42</value>
</property>
PE command:
===========
./hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --presplit=50
--size=150 --columns=50 --valueSize=200 --oneCon=true --writeToWAL=false
--bloomFilter=NONE --inmemoryCompaction=false randomWrite 100
We think MSLAB and chunk pool is important mainly because when you move things
to offheap we need to ensure that we are managing the offheap buffers
efficiently. It is very dangerous to allocate offheap buffers and keep them
unclaimed which will lead to offheap memory leak. Whereas in onheap buffers
cases we
are able to manage it with the help of GC.
Hence it is always recommended for offheap memstore to go with MSLAB and chunk
pool.
Perf readings
==========
For details on onheap and offheap memstore comparison with MSLAB and chunk pool
pls refer to these docs and various GC configs
https://docs.google.com/document/d/1fj5P8JeutQ-Uadb29ChDscMuMaJqaMNRI86C4k5S1rQ/edit
https://docs.google.com/document/d/1bYIkwNVVyYk8bgik-C5x_yc_2hWHyi2-9KMwEFWLfzY/edit
The above experiments were done on top of a POc branch where we had our own
changes to support offheap memstore including some PB changes.
With the current trunk we performed experiments with 50 threads and 100 threads
and with 64 G heap memory.
With 100 threads and 10 cols per row pushed from a single client the setup with
MSLAB and chunk pool performs 13% better than the no MSLAB case.
With 50 threads and 10 cols per row pushed from a single client the setup with
MSLAB and chunk pool performs 11% better than the no MSLAB case.
PE completion time for 50 threads, 10 cols
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|1132.923 secs|1262 secs|
PE completion time for 100 threads, 10 cols
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|1947.330 secs|2214.201 secs|
No of GC pauses for 100 threads
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|403|475|
GC pause in secs for 100 threads
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|181.58 secs|239.9 secs|
Pls feel free to share your feedback or thoughts on this. It helps to see if
there is something to be tested or some configs needs to be changed and the
tests has to be repeated.
> See how G1GC works with MSLAB and chunk pool
> --------------------------------------------
>
> Key: HBASE-15561
> URL: https://issues.apache.org/jira/browse/HBASE-15561
> Project: HBase
> Issue Type: Sub-task
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
>
> Based on the discussion in https://issues.apache.org/jira/browse/HBASE-14613,
> we need to identify specifically how MSLAB and G1GC work. This sub-task is
> mainly to identify how things work with G1GC and note down the observations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
