[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015737#comment-14015737 ] Hudson commented on HBASE-11238: FAILURE: Integrated in HBase-TRUNK #5166 (See [https://builds.apache.org/job/HBase-TRUNK/5166/]) HBASE-11238 Add info about SlabCache and BucketCache to Ref Guide (Misty Stanley-Jones) (stack: rev 768c4d6775797e99e2489c36d9b215d9553493b5) * src/main/docbkx/book.xml * src/main/docbkx/performance.xml > Add info about SlabCache and BucketCache to Ref Guide > - > > Key: HBASE-11238 > URL: https://issues.apache.org/jira/browse/HBASE-11238 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.98.2 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0 > > Attachments: HBASE-11238-2.patch, HBASE-11238-3.patch, > HBASE-11238-4.patch, HBASE-11238.patch > > > Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is > in these issues, the package-info.java class for instance. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[
https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015190#comment-14015190
]
Hadoop QA commented on HBASE-11238:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12647854/HBASE-11238-4.patch
against trunk revision .
ATTACHMENT ID: 12647854
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+
xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/io/hfile/LruBlockCache.html";>LruBlockCache
+Your data is not the only resident of the block cache. Here are
others that you may have to take into account:
+
xlink:href="http://blog.cloudera.com/blog/2012/01/caching-in-hbase-slabcache/";>Caching
+
xlink:href="http://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.html";>DoubleBlockCache,
+ the hbase-site.xml file on the
RegionServer. The value will be multiplied by the
+
xlink:href="http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/io/hfile/CacheConfig.html";
/>.
+
xlink:href="http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/io/hfile/CombinedBlockCache.html";>CombinedBlockCache
+
xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/wal/HLog.html";>HLog
+is the HBase WAL implementation, and there is one HLog instance
per RegionServer.
+ and
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9661//console
This message is automatically generated.
> Add info about SlabCache and BucketCache to Ref Guide
> -
>
> Key: HBASE-11238
> URL: https://issues.apache.org/jira/browse/HBASE-11238
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.98.2
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11238-2.patch, HBASE-11238-3.patch,
> HBASE-11238-4.patch, HBASE-11238.patch
>
>
> Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is
> in these issues, the package-info.java class for instance.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015169#comment-14015169 ] Dima Spivak commented on HBASE-11238: - Just as a thought: it might be worth noting on which machines particular files need to be modified (e.g. those running RegionServers) for things like this. Playing around with setting up BucketCache earlier today, I inadvertently modified hbase-env.sh on my gateway before understanding why my RS' couldn't restart without throwing OutOfMemoryError. > Add info about SlabCache and BucketCache to Ref Guide > - > > Key: HBASE-11238 > URL: https://issues.apache.org/jira/browse/HBASE-11238 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.98.2 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11238-2.patch, HBASE-11238-3.patch, > HBASE-11238.patch > > > Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is > in these issues, the package-info.java class for instance. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[
https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010840#comment-14010840
]
Hadoop QA commented on HBASE-11238:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12647051/HBASE-11238-3.patch
against trunk revision .
ATTACHMENT ID: 12647051
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 32 release
audit warnings (more than the trunk's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+
xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/io/hfile/LruBlockCache.html";>LruBlockCache
+
xlink:href="http://blog.cloudera.com/blog/2012/01/caching-in-hbase-slabcache/";>Caching
+
xlink:href="http://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.html";>DoubleBlockCache,
+
xlink:href="http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/io/hfile/CacheConfig.html";
/>.
+
xlink:href="http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/io/hfile/CombinedBlockCache.html";>CombinedBlockCache
+
xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/wal/HLog.html";>HLog
+is the HBase WAL implementation, and there is one HLog instance
per RegionServer.
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9616//console
This message is automatically generated.
> Add info about SlabCache and BucketCache to Ref Guide
> -
>
> Key: HBASE-11238
> URL: https://issues.apache.org/jira/browse/HBASE-11238
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.98.2
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11238-2.patch, HBASE-11238-3.patch,
> HBASE-11238.patch
>
>
> Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is
> in these issues, the package-info.java class for instance.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010742#comment-14010742 ] stack commented on HBASE-11238: --- Looks generally good. Thanks. Should it not be related to this section though? http://hbase.apache.org/book.html#block.cache > Add info about SlabCache and BucketCache to Ref Guide > - > > Key: HBASE-11238 > URL: https://issues.apache.org/jira/browse/HBASE-11238 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.98.2 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11238-2.patch, HBASE-11238.patch > > > Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is > in these issues, the package-info.java class for instance. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[
https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010678#comment-14010678
]
Hadoop QA commented on HBASE-11238:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12647019/HBASE-11238-2.patch
against trunk revision .
ATTACHMENT ID: 12647019
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 32 release
audit warnings (more than the trunk's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+
xlink:href="http://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.html";>DoubleBlockCache,
+
xlink:href="http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/io/hfile/CacheConfig.html";
/>.
+
xlink:href="http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/io/hfile/CombinedBlockCache.html";>CombinedBlockCache
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9609//console
This message is automatically generated.
> Add info about SlabCache and BucketCache to Ref Guide
> -
>
> Key: HBASE-11238
> URL: https://issues.apache.org/jira/browse/HBASE-11238
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.98.2
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11238-2.patch, HBASE-11238.patch
>
>
> Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is
> in these issues, the package-info.java class for instance.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010019#comment-14010019 ] stack commented on HBASE-11238: --- I buy your argument that this better belongs in the refguide than in javadoc. The features depends so much on code reading, having the doc in javadoc makes it easier linking the code internals to the explanation. The doc is still under development and configuration of this feature is likely to change in near future too. If we add doc to refguide, we need to change the package-info so it points into refguide instead so we have doc in one place only. In following, it should be fetching from BucketCache or SlabCache will be slower than from native on heap LruBlockCache "Fetching + will always be slower when fetching from BucketCache, but latencies tend to be less + erratic over time, because there is less garbage collection." Also experience seems to have it that BucketCache makes less garbage than SlabCache so should be even less erratic (than SlabCache or LruBlockCache). No hard numbers, just reports by a few of the lads. Change '(CombinedBlockCache)' to '(DoubleBlockCache vs CombinedBlockCache)' You lost the link to Nick's 'BlockCache 101'? This is not right "SlabCache is enabled by default." (and you contradict yourself by writing later 'To enable it...") There are three block cache options. The original implemenation which is in the java heap called LruBlockCache. Then there are SlabCache and BucketCache, both of which are mainly intended for keeping blockcache data offheap (but they can also keep data onheap and in files, at least in the BucketCache case). We should say what hbase.offheapcache.percentage is a multiplicand for (the JVM setting). oh you say it later so there is some repeat going on here? Otherwise, looks great. > Add info about SlabCache and BucketCache to Ref Guide > - > > Key: HBASE-11238 > URL: https://issues.apache.org/jira/browse/HBASE-11238 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.98.2 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11238.patch > > > Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is > in these issues, the package-info.java class for instance. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008457#comment-14008457 ] Misty Stanley-Jones commented on HBASE-11238: - My thinking was that this is a config setting, not a development setting. I wouldn't normally look in Javadoc for a config setting. But maybe it's different in HBase-land? If you want to keep it in Javadoc, do you like my version or have I lost some of the meaning in the rewrite? > Add info about SlabCache and BucketCache to Ref Guide > - > > Key: HBASE-11238 > URL: https://issues.apache.org/jira/browse/HBASE-11238 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.98.2 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11238.patch > > > Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is > in these issues, the package-info.java class for instance. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006819#comment-14006819 ] stack commented on HBASE-11238: --- You think that the way to go [~misty]? Now the doc is in two places? I thought I had it so the refguide pointed at the javadoc? Thanks. > Add info about SlabCache and BucketCache to Ref Guide > - > > Key: HBASE-11238 > URL: https://issues.apache.org/jira/browse/HBASE-11238 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.98.2 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11238.patch > > > Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is > in these issues, the package-info.java class for instance. -- This message was sent by Atlassian JIRA (v6.2#6252)
