[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907221#comment-14907221 ] Hudson commented on HBASE-14098: FAILURE: Integrated in HBase-0.98 #1128 (See [https://builds.apache.org/job/HBase-0.98/1128/]) Backport HBASE-14098 (Allow dropping caches behind compactions) to 0.98 (Liu Shaohui) (apurtell: rev 672d74e7ba8c6caaedf3a549c30db5e1cbb4289b) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeCompactor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java > Allow dropping caches behind compactions > > > Key: HBASE-14098 > URL: https://issues.apache.org/jira/browse/HBASE-14098 > Project: HBase > Issue Type: Bug > Components: Compaction, hadoop2, HFile >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0 > > Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, > HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, > HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch, > HBASE-14098.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907253#comment-14907253 ] Hudson commented on HBASE-14098: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1082 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1082/]) Backport HBASE-14098 (Allow dropping caches behind compactions) to 0.98 (Liu Shaohui) (apurtell: rev 672d74e7ba8c6caaedf3a549c30db5e1cbb4289b) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java > Allow dropping caches behind compactions > > > Key: HBASE-14098 > URL: https://issues.apache.org/jira/browse/HBASE-14098 > Project: HBase > Issue Type: Bug > Components: Compaction, hadoop2, HFile >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0 > > Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, > HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, > HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch, > HBASE-14098.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715291#comment-14715291
]
stack commented on HBASE-14098:
---
This change drops ugly thread dumps in our logs:
{code}
9034 2015-08-26 11:18:38,175 DEBUG [Time-limited test]
hfile.HFile$WriterFactory(308): Unable to set drop behind on
/Users/stack/checkouts/hbase.git.commit/hbase-server/target/test-data/ed67f436-1d46-43f#
9035 java.lang.UnsupportedOperationException: the wrapped stream does not
support setting the drop-behind caching setting.
9036 › at
org.apache.hadoop.fs.FSDataOutputStream.setDropBehind(FSDataOutputStream.java:150)
9037 › at
org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:306)
9038 › at
org.apache.hadoop.hbase.regionserver.StoreFile$Writer.init(StoreFile.java:787)
9039 › at
org.apache.hadoop.hbase.regionserver.StoreFile$Writer.init(StoreFile.java:742)
9040 › at
org.apache.hadoop.hbase.regionserver.StoreFile$WriterBuilder.build(StoreFile.java:682)
9041 › at
org.apache.hadoop.hbase.regionserver.HStore.createWriterInTmp(HStore.java:1029)
9042 › at
org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:66)
9043 › at
org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:932)
9044 › at
org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2069)
9045 › at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2312)
9046 › at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2048)
9047 › at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2010)
9048 › at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1901)
9049 › at
org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1827)
9050 › at
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testSelectiveFlushWhenNotEnabled(TestPerColumnFamilyFlush.java:303)
9051 › at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
9052 › at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
9053 › at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
9054 › at java.lang.reflect.Method.invoke(Method.java:606)
9055 › at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
9056 › at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
9057 › at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
9058 › at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
9059 › at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
9060 › at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
9061 › at java.util.concurrent.FutureTask.run(FutureTask.java:262)
9062 › at java.lang.Thread.run(Thread.java:744)
{code}
It looks like an issue and only after looking in code do I see it informative.
I should change this to be TRACE level with DEBUG emitting just that the
feature is not available? (This is in testing master with its version of hadoop)
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.2.0
Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch,
HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch,
HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch,
HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694817#comment-14694817 ] Hudson commented on HBASE-14098: SUCCESS: Integrated in HBase-1.3-IT #87 (See [https://builds.apache.org/job/HBase-1.3-IT/87/]) HBASE-14098 Allow dropping caches behind compactions (eclark: rev 0dbaf0ea5145b318b18b487d8d9bc2d2eb67e19d) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.2.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694807#comment-14694807 ] Hudson commented on HBASE-14098: SUCCESS: Integrated in HBase-1.2-IT #88 (See [https://builds.apache.org/job/HBase-1.2-IT/88/]) HBASE-14098 Allow dropping caches behind compactions (eclark: rev 34b706af4d44ad7dff8ac5f35eec304d7dc0ccab) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.2.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694834#comment-14694834 ] Hudson commented on HBASE-14098: FAILURE: Integrated in HBase-1.2 #106 (See [https://builds.apache.org/job/HBase-1.2/106/]) HBASE-14098 Allow dropping caches behind compactions (eclark: rev 34b706af4d44ad7dff8ac5f35eec304d7dc0ccab) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.2.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694764#comment-14694764 ] Hudson commented on HBASE-14098: SUCCESS: Integrated in HBase-1.3 #106 (See [https://builds.apache.org/job/HBase-1.3/106/]) HBASE-14098 Allow dropping caches behind compactions (eclark: rev 0dbaf0ea5145b318b18b487d8d9bc2d2eb67e19d) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.2.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694468#comment-14694468
]
Hadoop QA commented on HBASE-14098:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12750184/HBASE-14098-v7-branch-1.patch
against branch-1 branch at commit 5e5bcceb533e6e4ded65bc778f05da213c07b688.
ATTACHMENT ID: 12750184
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15074//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15074//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15074//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15074//console
This message is automatically generated.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.2.0
Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch,
HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch,
HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch,
HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694316#comment-14694316
]
Hadoop QA commented on HBASE-14098:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12750165/HBASE-14098-branch1-v2.patch
against master branch at commit beb1f1d358e7f786c510c988b09dab0f87945d1f.
ATTACHMENT ID: 12750165
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15072//console
This message is automatically generated.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.2.0
Attachments: HBASE-14098-branch1-v2.patch, HBASE-14098-v1.patch,
HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch,
HBASE-14098-v5.patch, HBASE-14098-v6.patch, HBASE-14098-v7.patch,
HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694649#comment-14694649 ] Hudson commented on HBASE-14098: FAILURE: Integrated in HBase-TRUNK #6722 (See [https://builds.apache.org/job/HBase-TRUNK/6722/]) HBASE-14098 Allow dropping caches behind compactions (eclark: rev a399ac9c4f76dccebb3476c418aac68667dee545) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreCompaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestPartitionedMobCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeCompactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.2.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694662#comment-14694662 ] Lars Hofhansl commented on HBASE-14098: --- bq. Yes this does turn on private readers by default. We've been running it for a while and I haven't seen any down sides, so I feel pretty sure that it's not too big a risk. Cool. I put it in a while back, but haven't seen anybody but us using it. Did you notice improvements in scan performance when you enabled it? (sorry off topic) Cool to have this committed. Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.2.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098-v6.patch, HBASE-14098-v7-branch-1.patch, HBASE-14098-v7.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14694020#comment-14694020 ] Elliott Clark commented on HBASE-14098: --- Committing soon unless there are any other comments. Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.2.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098-v6.patch, HBASE-14098-v7.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681925#comment-14681925
]
Hadoop QA commented on HBASE-14098:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12749862/HBASE-14098-v6.patch
against master branch at commit 7d4de20cafd6b765bd5f33df72fc0e630d1731f7.
ATTACHMENT ID: 12749862
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail with Hadoop version 2.4.0.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFile.java:[60,57]
no suitable method found for
getScannersForStoreFiles(java.util.Listorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,boolean,nulltype,long)
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFile.java:[91,59]
no suitable method found for
getScannersForStoreFiles(java.util.Listorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,boolean,nulltype,long)
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java:[80,3]
method does not override or implement a method from a supertype
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java:[549,37]
no suitable method found for
getScannersForStoreFiles(java.util.Listorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,boolean,nulltype,long)
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on
project hbase-server: Compilation failure: Compilation failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFile.java:[60,57]
no suitable method found for
getScannersForStoreFiles(java.util.Listorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,boolean,nulltype,long)
[ERROR] method
org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(java.util.Collectionorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,boolean,boolean,org.apache.hadoop.hbase.regionserver.ScanQueryMatcher,long)
is not applicable
[ERROR] (actual and formal argument lists differ in length)
[ERROR] method
org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(java.util.Collectionorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,boolean,boolean,long)
is not applicable
[ERROR] (actual argument nulltype cannot be converted to boolean by method
invocation conversion)
[ERROR] method
org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(java.util.Collectionorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,long)
is not applicable
[ERROR] (actual and formal argument lists differ in length)
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFile.java:[91,59]
no suitable method found for
getScannersForStoreFiles(java.util.Listorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,boolean,nulltype,long)
[ERROR] method
org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(java.util.Collectionorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,boolean,boolean,org.apache.hadoop.hbase.regionserver.ScanQueryMatcher,long)
is not applicable
[ERROR] (actual and formal argument lists differ in length)
[ERROR] method
org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(java.util.Collectionorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,boolean,boolean,long)
is not applicable
[ERROR] (actual argument nulltype cannot be converted to boolean by method
invocation conversion)
[ERROR] method
org.apache.hadoop.hbase.regionserver.StoreFileScanner.getScannersForStoreFiles(java.util.Collectionorg.apache.hadoop.hbase.regionserver.StoreFile,boolean,boolean,long)
is not applicable
[ERROR] (actual and formal argument lists differ in length)
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java:[80,3]
method does not override or implement a method from a supertype
[ERROR]
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682363#comment-14682363
]
Hadoop QA commented on HBASE-14098:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12749877/HBASE-14098-v7.patch
against master branch at commit d49b1f60f4cec6f04768f28f678bf4329bfa7364.
ATTACHMENT ID: 12749877
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 28 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15050//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15050//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15050//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15050//console
This message is automatically generated.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch,
HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch,
HBASE-14098-v6.patch, HBASE-14098-v7.patch, HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647263#comment-14647263 ] ramkrishna.s.vasudevan commented on HBASE-14098: bq.if (this.conf.getBoolean(hbase.regionserver.compaction.private.readers, true)) So with this change compactions will always use new readers so that the OS pages of those files are not cached based on the new setting that says dropBehind on compaction? Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647988#comment-14647988 ] Lars Hofhansl commented on HBASE-14098: --- Was just going to +1... Missed the private readers bit. I added the private.readers features and order to decouple compactions readers from operational readers (user scans, gets, etc). Defensively I defaulted this to false, but I think we can safely enable that always... But if we wanted to stay defensively we could default to the value of drop caches behind compactions. Actually, in either case +1 from me. Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646409#comment-14646409 ] Elliott Clark commented on HBASE-14098: --- Ping? This one solved kernel stalls for us. Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636224#comment-14636224 ] Elliott Clark commented on HBASE-14098: --- Yeah that test looks good. I think it's just a little flakey. Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635589#comment-14635589 ] Elliott Clark commented on HBASE-14098: --- TestIOFencing works on my box 3 times in a row. Checking master. Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch, HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14633970#comment-14633970
]
Hadoop QA commented on HBASE-14098:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12746144/HBASE-14098-v2.patch
against master branch at commit 0f614a1c44e1887ca7177b66bb6208b6e69db7e1.
ATTACHMENT ID: 12746144
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1880 checkstyle errors (more than the master's current 1871 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ .getScannersForStoreFiles(storeFilesToScan, cacheBlocks, usePread,
isCompaction, false, matcher,
+ in = new FSDataInputStreamWrapper(fs, this.link, canUseDropBehind
cacheConf.shouldDropBehindCompaction());
+ in = new FSDataInputStreamWrapper(fs, referencePath, canUseDropBehind
cacheConf.shouldDropBehindCompaction());
+ in = new FSDataInputStreamWrapper(fs, this.getPath(), canUseDropBehind
cacheConf.shouldDropBehindCompaction());
+ scanners = createFileScanners(readersToClose, smallestReadPoint,
store.throttleCompaction(request.getSize()));
+ scanners = createFileScanners(request.getFiles(), smallestReadPoint,
store.throttleCompaction(request.getSize()));
+ fd.maxMVCCReadpoint 0, fd.maxTagsLength 0,
store.throttleCompaction(request.getSize()));
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
org.apache.hadoop.hbase.regionserver.TestStripeCompactor
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14839//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14839//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14839//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14839//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14839//console
This message is automatically generated.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch,
HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634366#comment-14634366
]
Hadoop QA commented on HBASE-14098:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12746192/HBASE-14098-v4.patch
against master branch at commit 0f614a1c44e1887ca7177b66bb6208b6e69db7e1.
ATTACHMENT ID: 12746192
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1873 checkstyle errors (more than the master's current 1871 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14843//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14843//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14843//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14843//console
This message is automatically generated.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch,
HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634270#comment-14634270
]
Hadoop QA commented on HBASE-14098:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12746174/HBASE-14098-v3.patch
against master branch at commit 0f614a1c44e1887ca7177b66bb6208b6e69db7e1.
ATTACHMENT ID: 12746174
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1880 checkstyle errors (more than the master's current 1871 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ .getScannersForStoreFiles(storeFilesToScan, cacheBlocks, usePread,
isCompaction, false, matcher,
+ in = new FSDataInputStreamWrapper(fs, this.link, canUseDropBehind
cacheConf.shouldDropBehindCompaction());
+ in = new FSDataInputStreamWrapper(fs, referencePath, canUseDropBehind
cacheConf.shouldDropBehindCompaction());
+ in = new FSDataInputStreamWrapper(fs, this.getPath(), canUseDropBehind
cacheConf.shouldDropBehindCompaction());
+ scanners = createFileScanners(readersToClose, smallestReadPoint,
store.throttleCompaction(request.getSize()));
+ scanners = createFileScanners(request.getFiles(), smallestReadPoint,
store.throttleCompaction(request.getSize()));
+ fd.maxMVCCReadpoint 0, fd.maxTagsLength 0,
store.throttleCompaction(request.getSize()));
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.mapreduce.TestImportExport
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14842//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14842//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14842//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14842//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14842//console
This message is automatically generated.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch,
HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634528#comment-14634528
]
Hadoop QA commented on HBASE-14098:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12746242/HBASE-14098-v5.patch
against master branch at commit 7ce318dd3be9df0ee1c025b4792ded0161aa2c9c.
ATTACHMENT ID: 12746242
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestIOFencing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14847//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14847//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14847//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14847//console
This message is automatically generated.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-14098-v1.patch, HBASE-14098-v2.patch,
HBASE-14098-v3.patch, HBASE-14098-v4.patch, HBASE-14098-v5.patch,
HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631091#comment-14631091 ] Lars Hofhansl commented on HBASE-14098: --- That makes sense. Thanks [~eclark]. A cheap approximation might be to set it for major compactions. Agree that ideally we'd decided based on the size of the output file(s) compared to the available RAM for the OS buffer cache. Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-14098-v1.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631655#comment-14631655 ] Elliott Clark commented on HBASE-14098: --- I tested this out yesterday on a cluster with 1.1. When compacting 100+ GB storfiles I can reliably make the linux kernel pause trying to thrash and find pages to clear out. With just dropping the write's out of cache things are better and there are fewer stalls. With dropping the caches behind writes and reads from a compaction there are no noticeable stalls ( ones that atop sees that take 1 second ). So I think that we should provide the ability to drop caches behind reads and writes on large compactions ( those that would run on the large compaction threads ). However because this could adversely affect people with less data per machine I think that we should have this turned off by default. Thoughts? Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-14098-v1.patch, HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631685#comment-14631685
]
Lars Hofhansl commented on HBASE-14098:
---
Awesome!
Cache flush stalls can also be avoided by configuring the Linux disk cache.
Could you do a {{sysctl -a | grep dirty}} on one of the boxes and paste
vm.dirty_background_ratio, vm.dirty_background_bytes, and vm.dirty_ratio here?
Just curious :) Your previous comment made sense anyway. Caching anything that
is usually read sequentially and larger than the available RAM never makes
sense.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-14098-v1.patch, HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631708#comment-14631708
]
Elliott Clark commented on HBASE-14098:
---
{code}
vm.dirty_background_ratio = 3
vm.dirty_ratio = 20
{code}
The interesting part is that these stalls aren't really caused by dirty pages.
There's only 400mb of dirty pages. These stalls seem to be caused by thrashing
the cache. Too much reading/writing actually causes reads to happen on the root
disk to re-page in executables.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-14098-v1.patch, HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630189#comment-14630189
]
Hadoop QA commented on HBASE-14098:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12745644/HBASE-14098-v1.patch
against master branch at commit 97e0af001bdef74d9e0cc23e09cf7105bb8e0f64.
ATTACHMENT ID: 12745644
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 8 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels.testMROnTableWithDeletes(TestImportTSVWithVisibilityLabels.java:180)
at
org.apache.hadoop.hbase.mapreduce.TestCopyTable.testStartStopRow(TestCopyTable.java:159)
at
org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportFileSystemState(TestExportSnapshot.java:288)
at
org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportFileSystemState(TestExportSnapshot.java:262)
at
org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testSnapshotWithRefsExportFileSystemState(TestExportSnapshot.java:256)
at
org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testSnapshotWithRefsExportFileSystemState(TestExportSnapshot.java:236)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeMinorCompaction(TestHFileOutputFormat.java:977)
at
org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster.testMoveRegion(TestAssignmentManagerOnCluster.java:389)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14803//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14803//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14803//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14803//console
This message is automatically generated.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
Project: HBase
Issue Type: Bug
Components: Compaction, hadoop2, HFile
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-14098-v1.patch, HBASE-14098.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629479#comment-14629479 ] Elliott Clark commented on HBASE-14098: --- Yeah related but on the other side. Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629498#comment-14629498 ] Lars Hofhansl commented on HBASE-14098: --- Oh... This is for the output files of a compaction... Aren't we likely going to read the compacted data again (especially since we just dropped it all from the block cache)? Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[
https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629537#comment-14629537
]
Hadoop QA commented on HBASE-14098:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12745598/HBASE-14098.patch
against master branch at commit ebdac4b52e67614db70b59be8cd8143efe701911.
ATTACHMENT ID: 12745598
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.io.hfile.TestLazyDataBlockDecompression
org.apache.hadoop.hbase.regionserver.TestBlocksScanned
org.apache.hadoop.hbase.io.hfile.TestReseekTo
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
org.apache.hadoop.hbase.io.hfile.TestSeekTo
org.apache.hadoop.hbase.regionserver.TestColumnSeeking
org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
org.apache.hadoop.hbase.io.hfile.TestHFileInlineToRootChunkConversion
org.apache.hadoop.hbase.io.hfile.TestPrefetch
org.apache.hadoop.hbase.regionserver.TestStoreFile
org.apache.hadoop.hbase.io.encoding.TestPrefixTree
org.apache.hadoop.hbase.io.hfile.TestHFileWriterV3
org.apache.hadoop.hbase.regionserver.TestBulkLoad
org.apache.hadoop.hbase.io.hfile.TestHFileEncryption
org.apache.hadoop.hbase.filter.TestColumnPrefixFilter
org.apache.hadoop.hbase.regionserver.TestScanner
org.apache.hadoop.hbase.filter.TestDependentColumnFilter
org.apache.hadoop.hbase.regionserver.TestResettingCounters
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransaction
org.apache.hadoop.hbase.regionserver.TestKeepDeletes
org.apache.hadoop.hbase.regionserver.TestStoreFileRefresherChore
org.apache.hadoop.hbase.coprocessor.TestRegionObserverStacking
org.apache.hadoop.hbase.io.hfile.TestScannerSelectionUsingKeyRange
org.apache.hadoop.hbase.io.hfile.TestHFile
org.apache.hadoop.hbase.regionserver.TestScanWithBloomError
org.apache.hadoop.hbase.filter.TestFilter
org.apache.hadoop.hbase.filter.TestInvocationRecordFilter
org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter
org.apache.hadoop.hbase.regionserver.TestWideScanner
org.apache.hadoop.hbase.io.TestHalfStoreFileReader
org.apache.hadoop.hbase.regionserver.TestStoreFileScannerWithTagCompression
org.apache.hadoop.hbase.client.TestIntraRowPagination
org.apache.hadoop.hbase.regionserver.TestMinVersions
org.apache.hadoop.hbase.io.hfile.TestHFileWriterV2
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14799//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14799//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14799//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14799//console
This message is automatically generated.
Allow dropping caches behind compactions
Key: HBASE-14098
URL: https://issues.apache.org/jira/browse/HBASE-14098
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629459#comment-14629459 ] Lars Hofhansl commented on HBASE-14098: --- Related to HBASE-10052? Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Reporter: Elliott Clark Assignee: Elliott Clark -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14098) Allow dropping caches behind compactions
[ https://issues.apache.org/jira/browse/HBASE-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629949#comment-14629949 ] Elliott Clark commented on HBASE-14098: --- Yes it's possible that we will read the file again. It's also likely that on large compactions we will just be blowing the fs cache out of the water. Compacting 32gb on machine with 32gb memory free means that nothing else can be in the fs cache. No /bin/bash, no inodes, nothing. My plan is likely to set this only for large compactions; the thought being that large compactions are much more likely to have stale data in them. I'm going to test the current patch out on a cluster that's doing really large compactions right now. If I see any positive changes then we can make this smarter. Allow dropping caches behind compactions Key: HBASE-14098 URL: https://issues.apache.org/jira/browse/HBASE-14098 Project: HBase Issue Type: Bug Components: Compaction, hadoop2, HFile Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-14098.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
