[jira] [Commented] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[ https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096676#comment-14096676 ] Lars Hofhansl commented on HBASE-11733: --- Nice! Could this have performance implications, especially for hooks into hot paths like preScannerNext(...), etc. We're creating one more object upon each invocation now. > Avoid copy-paste in Master/Region CoprocessorHost > - > > Key: HBASE-11733 > URL: https://issues.apache.org/jira/browse/HBASE-11733 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-11733-v0.patch > > > all the methods of MasterCoprocessorHost and RegionCoprocessorHost are > copy-paste, and a couple of them have some missing fixes. > The body of each function is the same, the only difference is the call to the > specific coprocessor. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11738) Document improvements to LoadTestTool and PerformanceEvaluation
[
https://issues.apache.org/jira/browse/HBASE-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096668#comment-14096668
]
Hadoop QA commented on HBASE-11738:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661643/HBABASE-11738.patch
against trunk revision .
ATTACHMENT ID: 12661643
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10429//console
This message is automatically generated.
> Document improvements to LoadTestTool and PerformanceEvaluation
> ---
>
> Key: HBASE-11738
> URL: https://issues.apache.org/jira/browse/HBASE-11738
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
>Priority: Minor
> Fix For: 0.99.0, 0.98.4
>
> Attachments: HBABASE-11738.patch, HBASE-11738.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11739) Document blockCache contents report in the UI
[
https://issues.apache.org/jira/browse/HBASE-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096656#comment-14096656
]
Hadoop QA commented on HBASE-11739:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661628/HBASE-11739.patch
against trunk revision .
ATTACHMENT ID: 12661628
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testCloneAndRestoreSnapshot(TestRestoreSnapshotFromClient.java:255)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10428//console
This message is automatically generated.
> Document blockCache contents report in the UI
> -
>
> Key: HBASE-11739
> URL: https://issues.apache.org/jira/browse/HBASE-11739
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 0.99.0
>
> Attachments: HBASE-11739.patch, bc_basic.png, bc_config.png,
> bc_l1.png, bc_l2_buckets.png, bc_stats.png
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096648#comment-14096648 ] Hudson commented on HBASE-11550: SUCCESS: Integrated in HBase-0.98 #447 (See [https://builds.apache.org/job/HBase-0.98/447/]) HBASE-11550 Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted (Gustavo Anatoly) (ndimiduk: rev 7e1e818bbc0bed59c12fbd8c003eaed702cf16ad) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketAllocator.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java > Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted > -- > > Key: HBASE-11550 > URL: https://issues.apache.org/jira/browse/HBASE-11550 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, > HBASE-11550-v3.patch, HBASE-11550-v4-0.98.patch, HBASE-11550-v4.patch, > HBASE-11550.patch > > > User can pass bucket sizes through "hbase.bucketcache.bucket.sizes" config > entry. > The sizes are supposed to be in increasing order. Validation should be added > in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11736) Document SKIP_FLUSH snapshot option
[
https://issues.apache.org/jira/browse/HBASE-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096643#comment-14096643
]
Hadoop QA commented on HBASE-11736:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661613/HBASE-11736.patch
against trunk revision .
ATTACHMENT ID: 12661613
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
org.apache.hadoop.hbase.client.TestReplicaWithCluster
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10423//console
This message is automatically generated.
> Document SKIP_FLUSH snapshot option
> ---
>
> Key: HBASE-11736
> URL: https://issues.apache.org/jira/browse/HBASE-11736
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, snapshots
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 0.99.0, 0.94.21, 0.98.4
>
> Attachments: HBASE-11736.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096609#comment-14096609
]
stack commented on HBASE-11657:
---
Yeah, we've been trying to shutdown notions of 'region' showing through in API.
We don't want to break API for 1.0. Mark new Interface Private audience too
instead of evolving as it is currently (IIRC)?
> Put HTable region methods in an interface
> -
>
> Key: HBASE-11657
> URL: https://issues.apache.org/jira/browse/HBASE-11657
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: Carter
>Assignee: Carter
> Fix For: 0.99.0
>
> Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
> HBASE_11657_v3.patch, HBASE_11657_v3.patch, HBASE_11657_v4.patch
>
>
> Most of the HTable methods are now abstracted by HTableInterface, with the
> notable exception of the following methods that pertain to region metadata:
> {code}
> HRegionLocation getRegionLocation(final String row)
> HRegionLocation getRegionLocation(final byte [] row)
> HRegionLocation getRegionLocation(final byte [] row, boolean reload)
> byte [][] getStartKeys()
> byte[][] getEndKeys()
> Pair getStartEndKeys()
> void clearRegionCache()
> {code}
> and a default scope method which maybe should be bundled with the others:
> {code}
> List listRegionLocations()
> {code}
> Since the consensus seems to be that these would muddy HTableInterface with
> non-core functionality, where should it go? MapReduce looks up the region
> boundaries, so it needs to be exposed somewhere.
> Let me throw out a straw man to start the conversation. I propose:
> {code}
> org.apache.hadoop.hbase.client.HRegionInterface
> {code}
> Have HTable implement this interface. Also add these methods to HConnection:
> {code}
> HRegionInterface getTableRegion(TableName tableName)
> HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
> {code}
> [~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
[
https://issues.apache.org/jira/browse/HBASE-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096586#comment-14096586
]
Hadoop QA commented on HBASE-11735:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661612/HBASE-11735.patch
against trunk revision .
ATTACHMENT ID: 12661612
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10425//console
This message is automatically generated.
> Document Configurable Bucket Sizes in bucketCache
> -
>
> Key: HBASE-11735
> URL: https://issues.apache.org/jira/browse/HBASE-11735
> Project: HBase
> Issue Type: Task
> Components: documentation
>Affects Versions: 0.99.0, 0.98.4, 0.98.5
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 0.99.0, 0.98.6
>
> Attachments: HBASE-11735.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
[
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096583#comment-14096583
]
wuchengzhi commented on HBASE-11728:
Does this Issue just fix in lastest version? will you provide patch for
0.96.1.1?
> Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
> --
>
> Key: HBASE-11728
> URL: https://issues.apache.org/jira/browse/HBASE-11728
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.1.1, 0.98.4
> Environment: ubuntu12
> hadoop-2.2.0
> Hbase-0.96.1.1
> SUN-JDK(1.7.0_06-b24)
>Reporter: wuchengzhi
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: TestPrefixTree.java
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> In Scan case, i prepare some data as beflow:
> Table Desc (Using the prefix-tree encoding) :
> 'prefix_tree_test', {NAME => 'cf_1', DATA_BLOCK_ENCODING => 'PREFIX_TREE',
> TTL => '15552000'}
> and i put 5 rows as:
> (RowKey , Qualifier, Value)
> 'a-b-0-0', 'qf_1', 'c1-value'
> 'a-b-A-1', 'qf_1', 'c1-value'
> 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
> 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
> 'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
> so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the
> corret result:
> Test 1:
> Scan scan = new Scan();
> scan.setStartRow("a-b-A-1".getBytes());
> scan.setStopRow("a-b-A-1:".getBytes());
> --
> 'a-b-A-1', 'qf_1', 'c1-value'
> 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
> 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
> and then i try next , scan to addColumn
> Test2:
> Scan scan = new Scan();
> scan.addColumn(Bytes.toBytes("cf_1") , Bytes.toBytes("qf_2"));
> scan.setStartRow("a-b-A-1".getBytes());
> scan.setStopRow("a-b-A-1:".getBytes());
> --
> except:
> 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
> 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
> but actually i got nonthing. Then i update the addColumn for
> scan.addColumn(Bytes.toBytes("cf_1") , Bytes.toBytes("qf_1")); and i got the
> expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
> then i do more testing... i update the case to modify the startRow greater
> than the 'a-b-A-1'
> Test3:
> Scan scan = new Scan();
> scan.setStartRow("a-b-A-1-".getBytes());
> scan.setStopRow("a-b-A-1:".getBytes());
> --
> except:
> 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
> 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
> but actually i got nothing again. i modify the start row greater than
> 'a-b-A-1-1402329600-1402396277'
> Scan scan = new Scan();
> scan.setStartRow("a-b-A-1-140239".getBytes());
> scan.setStopRow("a-b-A-1:".getBytes());
> and i got the expect row as well:
> 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
> So, i think it may be a bug in the prefix-tree encoding.It happens after the
> data flush to the storefile, and it's ok when the data in mem-store.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11610) Enhance remote meta updates
[
https://issues.apache.org/jira/browse/HBASE-11610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096562#comment-14096562
]
Hadoop QA commented on HBASE-11610:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661597/HBASE-11610.patch
against trunk revision .
ATTACHMENT ID: 12661597
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileGetting(TestHRegion.java:3808)
at
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScanBase.testScan(TestTableInputFormatScanBase.java:238)
at
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2.testScanOPPToEmpty(TestTableInputFormatScan2.java:70)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10421//console
This message is automatically generated.
> Enhance remote meta updates
> ---
>
> Key: HBASE-11610
> URL: https://issues.apache.org/jira/browse/HBASE-11610
> Project: HBase
> Issue Type: Improvement
>Reporter: Jimmy Xiang
>Assignee: Virag Kothari
> Attachments: HBASE-11610.patch
>
>
> Currently, if the meta region is on a regionserver instead of the master,
> meta update is synchronized on one HTable instance. We should be able to do
> better.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11738) Document improvements to LoadTestTool and PerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11738: Attachment: HBABASE-11738.patch Feedback assimilated. I took out the specific info about the improvements to the commands, even though that was kind of the point of this JIRA. ;) > Document improvements to LoadTestTool and PerformanceEvaluation > --- > > Key: HBASE-11738 > URL: https://issues.apache.org/jira/browse/HBASE-11738 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0, 0.98.4 > > Attachments: HBABASE-11738.patch, HBASE-11738.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11739) Document blockCache contents report in the UI
[ https://issues.apache.org/jira/browse/HBASE-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096554#comment-14096554 ] Misty Stanley-Jones commented on HBASE-11739: - I just stole Stack's PNGs, didn't generate them myself. I can try to get it done though. > Document blockCache contents report in the UI > - > > Key: HBASE-11739 > URL: https://issues.apache.org/jira/browse/HBASE-11739 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0 > > Attachments: HBASE-11739.patch, bc_basic.png, bc_config.png, > bc_l1.png, bc_l2_buckets.png, bc_stats.png > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11738) Document improvements to LoadTestTool and PerformanceEvaluation
[
https://issues.apache.org/jira/browse/HBASE-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096552#comment-14096552
]
Nick Dimiduk commented on HBASE-11738:
--
It seems strange to call out specific features of these tools. Better to just
document they exist and how to get help messages. Also, should say how to run
them before HBASE-11649. {{hbase pe}} was {{hbase
org.apache.hadoop.hbase.PerformanceEvaluation}} and {{hbase ltt}} was {{hbase
org.apache.hadoop.hbase.util.LoadTestTool}} before 0.98.5.
> Document improvements to LoadTestTool and PerformanceEvaluation
> ---
>
> Key: HBASE-11738
> URL: https://issues.apache.org/jira/browse/HBASE-11738
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
>Priority: Minor
> Fix For: 0.99.0, 0.98.4
>
> Attachments: HBASE-11738.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11729) Document HFile v3
[ https://issues.apache.org/jira/browse/HBASE-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096550#comment-14096550 ] Misty Stanley-Jones commented on HBASE-11729: - +1 from me, of course someone else should check the technical bits. > Document HFile v3 > - > > Key: HBASE-11729 > URL: https://issues.apache.org/jira/browse/HBASE-11729 > Project: HBase > Issue Type: Task > Components: documentation, HFile >Affects Versions: 0.98.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Trivial > Labels: beginner > Attachments: HBASE-11729.patch, HBASE-11729.pdf > > > 0.98 added HFile v3. There are a couple of mentions of it in the book on the > sections on cell tags, but there isn't an actual overview or design > explanation like there is for [HFile > v2|http://hbase.apache.org/book/hfilev2.html]. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-2821) Keep young storefiles at lower replication
[
https://issues.apache.org/jira/browse/HBASE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096551#comment-14096551
]
Hadoop QA commented on HBASE-2821:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661618/HBASE-2821.patch
against trunk revision .
ATTACHMENT ID: 12661618
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.regionserver.TestHRegion
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.coprocessor.TestMasterObserver.testRegionTransitionOperations(TestMasterObserver.java:1435)
at
org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:250)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10422//console
This message is automatically generated.
> Keep young storefiles at lower replication
> --
>
> Key: HBASE-2821
> URL: https://issues.apache.org/jira/browse/HBASE-2821
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Todd Lipcon
>Assignee: Andrew Purtell
> Fix For: 2.0.0
>
> Attachments: HBASE-2821.patch, HBASE-2821.patch,
> lifetime-distribution.png, storefile_age.pl
>
>
> jgray and I were brainstorming some ideas about this:
> In a typical heavy-write scenario, many store files do not last very long.
> They're flushed and then within a small number of seconds a compaction runs
> and they get deleted. For these "short lifetime" store files, it's less
> likely that a failure will occur during the window in which they're valid.
> So, I think we can consider some optimizations like the following:
> - Flush files at replication count 2. Scan once a minute for any store files
> in the region that are older than 2 minutes. If they're found, increase their
> replication to 3. (alternatively, queue them to avoid scanning)
> - More dangerous: flush files at replication count 1, but don't count them
> when figuring log expiration. So, if they get lost, we force log splitting to
> recover.
> The performance gain here is that we avoid the network and disk transfer of
> writing the third replica for a file that we're just about to delete anyway.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11739) Document blockCache contents report in the UI
[ https://issues.apache.org/jira/browse/HBASE-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096548#comment-14096548 ] Nick Dimiduk commented on HBASE-11739: -- Looks good. It's unfortunate that bc_basic.png shows a DoubleBlockCache, given that this has been removed in later versions. Any chance you can create a shot with a CombinedBlockCache instead? > Document blockCache contents report in the UI > - > > Key: HBASE-11739 > URL: https://issues.apache.org/jira/browse/HBASE-11739 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0 > > Attachments: HBASE-11739.patch, bc_basic.png, bc_config.png, > bc_l1.png, bc_l2_buckets.png, bc_stats.png > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11550: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the contribution [~gustavoanatoly]! > Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted > -- > > Key: HBASE-11550 > URL: https://issues.apache.org/jira/browse/HBASE-11550 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, > HBASE-11550-v3.patch, HBASE-11550-v4-0.98.patch, HBASE-11550-v4.patch, > HBASE-11550.patch > > > User can pass bucket sizes through "hbase.bucketcache.bucket.sizes" config > entry. > The sizes are supposed to be in increasing order. Validation should be added > in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11550: - Attachment: HBASE-11550-v4-0.98.patch Pushed to master. Cherry-picked to branch-1 with minimal conflict. Applied manually to 0.98, attaching patch. > Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted > -- > > Key: HBASE-11550 > URL: https://issues.apache.org/jira/browse/HBASE-11550 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, > HBASE-11550-v3.patch, HBASE-11550-v4-0.98.patch, HBASE-11550-v4.patch, > HBASE-11550.patch > > > User can pass bucket sizes through "hbase.bucketcache.bucket.sizes" config > entry. > The sizes are supposed to be in increasing order. Validation should be added > in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11740) RegionStates.getRegionAssignments() gets stuck on clone
[ https://issues.apache.org/jira/browse/HBASE-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-11740: -- Description: This happens in processDeadServersAndRegionsInTransition() on Master startup. RegionAssigments is a tree map and TreeMap.clone() is expensive as it builds a tree from sorted data (Order of n). There were a million entries in RegionAssigments and from jstack, thread was stuck in TreeMap.buildFromSorted() for couple of hours. Instead of this shadow clone, wrapping as an unmodifiable Map should be enough. was: This happens in processDeadServersAndRegionsInTransition() on Master startup. RegionAssigments is a tree map and TreeMap.clone() is expensive as it builds a tree from sorted data (O(n)). There were a million entries in RegionAssigments and from jstack, thread was stuck in TreeMap.buildFromSorted() for couple of hours. Instead of this shadow clone, wrapping as an unmodifiable Map should be enough. > RegionStates.getRegionAssignments() gets stuck on clone > --- > > Key: HBASE-11740 > URL: https://issues.apache.org/jira/browse/HBASE-11740 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Reporter: Virag Kothari >Assignee: Virag Kothari > > This happens in processDeadServersAndRegionsInTransition() on Master startup. > RegionAssigments is a tree map and TreeMap.clone() is expensive as it builds > a tree from sorted data (Order of n). There were a million entries in > RegionAssigments and from jstack, thread was stuck in > TreeMap.buildFromSorted() for couple of hours. > Instead of this shadow clone, wrapping as an unmodifiable Map should be > enough. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11729) Document HFile v3
[ https://issues.apache.org/jira/browse/HBASE-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11729: Status: Patch Available (was: Open) > Document HFile v3 > - > > Key: HBASE-11729 > URL: https://issues.apache.org/jira/browse/HBASE-11729 > Project: HBase > Issue Type: Task > Components: documentation, HFile >Affects Versions: 0.98.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Trivial > Labels: beginner > Attachments: HBASE-11729.patch, HBASE-11729.pdf > > > 0.98 added HFile v3. There are a couple of mentions of it in the book on the > sections on cell tags, but there isn't an actual overview or design > explanation like there is for [HFile > v2|http://hbase.apache.org/book/hfilev2.html]. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11740) RegionStates.getRegionAssignments() gets stuck on clone
Virag Kothari created HBASE-11740: - Summary: RegionStates.getRegionAssignments() gets stuck on clone Key: HBASE-11740 URL: https://issues.apache.org/jira/browse/HBASE-11740 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Virag Kothari Assignee: Virag Kothari This happens in processDeadServersAndRegionsInTransition() on Master startup. RegionAssigments is a tree map and TreeMap.clone() is expensive as it builds a tree from sorted data (O(n)). There were a million entries in RegionAssigments and from jstack, thread was stuck in TreeMap.buildFromSorted() for couple of hours. Instead of this shadow clone, wrapping as an unmodifiable Map should be enough. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11739) Document blockCache contents report in the UI
[ https://issues.apache.org/jira/browse/HBASE-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11739: Status: Patch Available (was: Open) > Document blockCache contents report in the UI > - > > Key: HBASE-11739 > URL: https://issues.apache.org/jira/browse/HBASE-11739 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0 > > Attachments: HBASE-11739.patch, bc_basic.png, bc_config.png, > bc_l1.png, bc_l2_buckets.png, bc_stats.png > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11739) Document blockCache contents report in the UI
[ https://issues.apache.org/jira/browse/HBASE-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11739: Attachment: bc_basic.png bc_config.png bc_stats.png bc_l1.png bc_l2_buckets.png HBASE-11739.patch Ready for review. Place images in src/main/site/resources/images/. > Document blockCache contents report in the UI > - > > Key: HBASE-11739 > URL: https://issues.apache.org/jira/browse/HBASE-11739 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0 > > Attachments: HBASE-11739.patch, bc_basic.png, bc_config.png, > bc_l1.png, bc_l2_buckets.png, bc_stats.png > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11729) Document HFile v3
[ https://issues.apache.org/jira/browse/HBASE-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11729: Attachment: HBASE-11729.pdf > Document HFile v3 > - > > Key: HBASE-11729 > URL: https://issues.apache.org/jira/browse/HBASE-11729 > Project: HBase > Issue Type: Task > Components: documentation, HFile >Affects Versions: 0.98.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Trivial > Labels: beginner > Attachments: HBASE-11729.patch, HBASE-11729.pdf > > > 0.98 added HFile v3. There are a couple of mentions of it in the book on the > sections on cell tags, but there isn't an actual overview or design > explanation like there is for [HFile > v2|http://hbase.apache.org/book/hfilev2.html]. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11729) Document HFile v3
[ https://issues.apache.org/jira/browse/HBASE-11729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11729: Attachment: HBASE-11729.patch Initial draft. Unfortunately, the hfile appendix was in hte main book.xml. Since that's so over burdened (6k lines), I took the opportunity to break it out into its own file. I didn't fix the formatting on parts of the text that I wasn't changing, but I could do that as well if folks would like. > Document HFile v3 > - > > Key: HBASE-11729 > URL: https://issues.apache.org/jira/browse/HBASE-11729 > Project: HBase > Issue Type: Task > Components: documentation, HFile >Affects Versions: 0.98.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Trivial > Labels: beginner > Attachments: HBASE-11729.patch > > > 0.98 added HFile v3. There are a couple of mentions of it in the book on the > sections on cell tags, but there isn't an actual overview or design > explanation like there is for [HFile > v2|http://hbase.apache.org/book/hfilev2.html]. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11610) Enhance remote meta updates
[ https://issues.apache.org/jira/browse/HBASE-11610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096520#comment-14096520 ] Virag Kothari commented on HBASE-11610: --- bq. Can we share one HConnection instead. We can create a HTable instance when needed. As long as we share the same HConnection, the performance should be ok. This is what I did initially. I had one HConnection and was creating htable instance for each put. But with more HConnections (tried with 5,10 and 15), the write throughput on meta regionserver increased and it brought the assignment time down by around 2 mins. The no of HConnection is configurable. By default, it is set to 1 as for small no of regions that is enough. Also, using threadlocal so we create only one htable instance for each thread accessing RegionStateStore. > Enhance remote meta updates > --- > > Key: HBASE-11610 > URL: https://issues.apache.org/jira/browse/HBASE-11610 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Virag Kothari > Attachments: HBASE-11610.patch > > > Currently, if the meta region is on a regionserver instead of the master, > meta update is synchronized on one HTable instance. We should be able to do > better. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11734) Document changed behavior of hbase.hstore.time.to.purge.deletes
[
https://issues.apache.org/jira/browse/HBASE-11734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096522#comment-14096522
]
Hadoop QA commented on HBASE-11734:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661601/HBASE-11734.patch
against trunk revision .
ATTACHMENT ID: 12661601
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10420//console
This message is automatically generated.
> Document changed behavior of hbase.hstore.time.to.purge.deletes
> ---
>
> Key: HBASE-11734
> URL: https://issues.apache.org/jira/browse/HBASE-11734
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 0.99.0, 0.94.19, 0.98.2, 0.96.3
>
> Attachments: HBASE-11734.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11739) Document blockCache contents report in the UI
Misty Stanley-Jones created HBASE-11739: --- Summary: Document blockCache contents report in the UI Key: HBASE-11739 URL: https://issues.apache.org/jira/browse/HBASE-11739 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[ https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096512#comment-14096512 ] Andrew Purtell commented on HBASE-11733: That's a nice refactor. Would you be willing to do this for 0.98+? Or someone else could backport... > Avoid copy-paste in Master/Region CoprocessorHost > - > > Key: HBASE-11733 > URL: https://issues.apache.org/jira/browse/HBASE-11733 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-11733-v0.patch > > > all the methods of MasterCoprocessorHost and RegionCoprocessorHost are > copy-paste, and a couple of them have some missing fixes. > The body of each function is the same, the only difference is the call to the > specific coprocessor. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11738) Document improvements to LoadTestTool and PerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11738: Status: Patch Available (was: Open) > Document improvements to LoadTestTool and PerformanceEvaluation > --- > > Key: HBASE-11738 > URL: https://issues.apache.org/jira/browse/HBASE-11738 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0, 0.98.4 > > Attachments: HBASE-11738.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11738) Document improvements to LoadTestTool and PerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11738: Attachment: HBASE-11738.patch Ready for review. > Document improvements to LoadTestTool and PerformanceEvaluation > --- > > Key: HBASE-11738 > URL: https://issues.apache.org/jira/browse/HBASE-11738 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0, 0.98.4 > > Attachments: HBASE-11738.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11610) Enhance remote meta updates
[ https://issues.apache.org/jira/browse/HBASE-11610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096505#comment-14096505 ] Jimmy Xiang commented on HBASE-11610: - The performance number looks great. Can we share one HConnection instead. We can create a HTable instance when needed. As long as we share the same HConnection, the performance should be ok. Probably we can use some methods in MetaTableAccessor? > Enhance remote meta updates > --- > > Key: HBASE-11610 > URL: https://issues.apache.org/jira/browse/HBASE-11610 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Virag Kothari > Attachments: HBASE-11610.patch > > > Currently, if the meta region is on a regionserver instead of the master, > meta update is synchronized on one HTable instance. We should be able to do > better. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11738) Document improvements to LoadTestTool and PerformanceEvaluation
Misty Stanley-Jones created HBASE-11738: --- Summary: Document improvements to LoadTestTool and PerformanceEvaluation Key: HBASE-11738 URL: https://issues.apache.org/jira/browse/HBASE-11738 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Priority: Minor -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11737) Document callQueue improvements from HBASE-11355
[ https://issues.apache.org/jira/browse/HBASE-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11737: Attachment: HBASE-11737.patch Ready for review. > Document callQueue improvements from HBASE-11355 > > > Key: HBASE-11737 > URL: https://issues.apache.org/jira/browse/HBASE-11737 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.4 > > Attachments: HBASE-11737.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11737) Document callQueue improvements from HBASE-11355
[ https://issues.apache.org/jira/browse/HBASE-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11737: Status: Patch Available (was: Open) > Document callQueue improvements from HBASE-11355 > > > Key: HBASE-11737 > URL: https://issues.apache.org/jira/browse/HBASE-11737 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.4 > > Attachments: HBASE-11737.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[
https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096481#comment-14096481
]
Hadoop QA commented on HBASE-11733:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661579/HBASE-11733-v0.patch
against trunk revision .
ATTACHMENT ID: 12661579
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat
org.apache.hadoop.hbase.master.TestMasterFailover
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10418//console
This message is automatically generated.
> Avoid copy-paste in Master/Region CoprocessorHost
> -
>
> Key: HBASE-11733
> URL: https://issues.apache.org/jira/browse/HBASE-11733
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-11733-v0.patch
>
>
> all the methods of MasterCoprocessorHost and RegionCoprocessorHost are
> copy-paste, and a couple of them have some missing fixes.
> The body of each function is the same, the only difference is the call to the
> specific coprocessor.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096479#comment-14096479 ] Gustavo Anatoly commented on HBASE-11550: - I don't have objections [~ndimiduk]. Thanks everyone. I get happy to help a little bit. :) > Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted > -- > > Key: HBASE-11550 > URL: https://issues.apache.org/jira/browse/HBASE-11550 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, > HBASE-11550-v3.patch, HBASE-11550-v4.patch, HBASE-11550.patch > > > User can pass bucket sizes through "hbase.bucketcache.bucket.sizes" config > entry. > The sizes are supposed to be in increasing order. Validation should be added > in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-2821) Keep young storefiles at lower replication
[ https://issues.apache.org/jira/browse/HBASE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-2821: -- Attachment: HBASE-2821.patch Updated patch fixes where StorefileReplicationWatcher code was incomplete or wrong. > Keep young storefiles at lower replication > -- > > Key: HBASE-2821 > URL: https://issues.apache.org/jira/browse/HBASE-2821 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Todd Lipcon >Assignee: Andrew Purtell > Fix For: 2.0.0 > > Attachments: HBASE-2821.patch, HBASE-2821.patch, > lifetime-distribution.png, storefile_age.pl > > > jgray and I were brainstorming some ideas about this: > In a typical heavy-write scenario, many store files do not last very long. > They're flushed and then within a small number of seconds a compaction runs > and they get deleted. For these "short lifetime" store files, it's less > likely that a failure will occur during the window in which they're valid. > So, I think we can consider some optimizations like the following: > - Flush files at replication count 2. Scan once a minute for any store files > in the region that are older than 2 minutes. If they're found, increase their > replication to 3. (alternatively, queue them to avoid scanning) > - More dangerous: flush files at replication count 1, but don't count them > when figuring log expiration. So, if they get lost, we force log splitting to > recover. > The performance gain here is that we avoid the network and disk transfer of > writing the third replica for a file that we're just about to delete anyway. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11737) Document callQueue improvements from HBASE-11355
Misty Stanley-Jones created HBASE-11737: --- Summary: Document callQueue improvements from HBASE-11355 Key: HBASE-11737 URL: https://issues.apache.org/jira/browse/HBASE-11737 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11610) Enhance remote meta updates
[ https://issues.apache.org/jira/browse/HBASE-11610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-11610: -- Status: Patch Available (was: Open) > Enhance remote meta updates > --- > > Key: HBASE-11610 > URL: https://issues.apache.org/jira/browse/HBASE-11610 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Virag Kothari > Attachments: HBASE-11610.patch > > > Currently, if the meta region is on a regionserver instead of the master, > meta update is synchronized on one HTable instance. We should be able to do > better. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[
https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096474#comment-14096474
]
Hadoop QA commented on HBASE-10202:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661586/HBASE-10202.patch
against trunk revision .
ATTACHMENT ID: 12661586
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+Usage: rolling-restart.sh [--config ] [--rs-only]
[--master-only] [--graceful] [--maxthreads xx]
+$ for i in `cat conf/regionservers|sort`; do ./bin/graceful_stop.sh --restart
--reload --debug $i; done &> /tmp/log.txt &;
+Use the following guidelines if you want to create your own
rolling restart script.
+$ for i in `cat conf/regionservers|sort`; do ./bin/graceful_stop.sh --restart
--reload --debug $i; done &> /tmp/log.txt &
+$ for i in `cat conf/regionservers|sort`; do ./bin/graceful_stop.sh --restart
--reload --debug $i & sleep 5m; done &> /tmp/log.txt &
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10419//console
This message is automatically generated.
> Documentation is lacking information about rolling-restart.sh script.
> -
>
> Key: HBASE-10202
> URL: https://issues.apache.org/jira/browse/HBASE-10202
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Affects Versions: 0.98.0, 0.94.14, 0.99.0, 0.96.1.1
>Reporter: Jean-Marc Spaggiari
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-10202-1.patch, HBASE-10202-2.patch,
> HBASE-10202.patch, HBASE-10202.patch
>
>
> Current documentation is talking about graceful_stop.sh and how to do a
> rolling restart but is not talking about the rolling-restart.sh script. We
> need to document that.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11732) No need to force unassign a region
[ https://issues.apache.org/jira/browse/HBASE-11732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11732: Attachment: hbase-11732.patch > No need to force unassign a region > -- > > Key: HBASE-11732 > URL: https://issues.apache.org/jira/browse/HBASE-11732 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: hbase-11732.patch > > > Clean up force region unassignment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11732) No need to force unassign a region
[ https://issues.apache.org/jira/browse/HBASE-11732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11732: Status: Patch Available (was: Open) > No need to force unassign a region > -- > > Key: HBASE-11732 > URL: https://issues.apache.org/jira/browse/HBASE-11732 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: hbase-11732.patch > > > Clean up force region unassignment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11736) Document SKIP_FLUSH snapshot option
[ https://issues.apache.org/jira/browse/HBASE-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11736: Attachment: HBASE-11736.patch > Document SKIP_FLUSH snapshot option > --- > > Key: HBASE-11736 > URL: https://issues.apache.org/jira/browse/HBASE-11736 > Project: HBase > Issue Type: Sub-task > Components: documentation, snapshots >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.94.21, 0.98.4 > > Attachments: HBASE-11736.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11736) Document SKIP_FLUSH snapshot option
[ https://issues.apache.org/jira/browse/HBASE-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11736: Status: Patch Available (was: Open) Ready for review. > Document SKIP_FLUSH snapshot option > --- > > Key: HBASE-11736 > URL: https://issues.apache.org/jira/browse/HBASE-11736 > Project: HBase > Issue Type: Sub-task > Components: documentation, snapshots >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.4, 0.94.21 > > Attachments: HBASE-11736.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11736) Document SKIP_FLUSH snapshot option
Misty Stanley-Jones created HBASE-11736: --- Summary: Document SKIP_FLUSH snapshot option Key: HBASE-11736 URL: https://issues.apache.org/jira/browse/HBASE-11736 Project: HBase Issue Type: Sub-task Components: documentation, snapshots Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter
[ https://issues.apache.org/jira/browse/HBASE-11476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096443#comment-14096443 ] Hudson commented on HBASE-11476: FAILURE: Integrated in HBase-1.0 #99 (See [https://builds.apache.org/job/HBase-1.0/99/]) HBASE-11476 Expand 'Conceptual View' section of Data Model chapter (Misty Stanley-Jones) (jmhsieh: rev 65375f8258095f0a0fac16d870ee134d43a7910b) * src/main/docbkx/book.xml > Expand 'Conceptual View' section of Data Model chapter > --- > > Key: HBASE-11476 > URL: https://issues.apache.org/jira/browse/HBASE-11476 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 2.0.0 > > Attachments: HBASE-11476.patch, HBASE-11476.patch > > > Could use some updating and expansion to emphasize the differences between > HBase and an RDBMS. I found > http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable > which is just excellent and we should link to it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11735: Status: Patch Available (was: Open) > Document Configurable Bucket Sizes in bucketCache > - > > Key: HBASE-11735 > URL: https://issues.apache.org/jira/browse/HBASE-11735 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 0.98.5, 0.98.4, 0.99.0 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.6 > > Attachments: HBASE-11735.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11735: Attachment: HBASE-11735.patch > Document Configurable Bucket Sizes in bucketCache > - > > Key: HBASE-11735 > URL: https://issues.apache.org/jira/browse/HBASE-11735 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.6 > > Attachments: HBASE-11735.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10877) HBase non-retriable exception list should be expanded
[
https://issues.apache.org/jira/browse/HBASE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096439#comment-14096439
]
Nick Dimiduk commented on HBASE-10877:
--
[~gmedasani] Glad to hear things are working :)
> HBase non-retriable exception list should be expanded
> -
>
> Key: HBASE-10877
> URL: https://issues.apache.org/jira/browse/HBASE-10877
> Project: HBase
> Issue Type: Improvement
>Reporter: Sergey Shelukhin
>Priority: Minor
>
> Example where retries do not make sense:
> {noformat}
> 2014-03-31 20:54:27,765 WARN [InputInitializer [Map 1] #0]
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation:
> Encountered problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=35, exceptions:
> Mon Mar 31 20:45:17 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: class
> com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass
> com.google.protobuf.LiteralByteString
> Mon Mar 31 20:45:17 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:45:17 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:45:18 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:45:20 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:45:24 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:45:34 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:45:45 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:45:55 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:46:05 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:46:25 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:46:45 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:47:05 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:47:25 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:47:45 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:48:05 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:48:25 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:48:46 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:49:06 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:49:26 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:49:46 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:50:06 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
> Mon Mar 31 20:50:26 UTC 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
> java.lang.IllegalAccessError: com/google/proto
[jira] [Commented] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096434#comment-14096434 ] Nick Dimiduk commented on HBASE-11735: -- Converted to task as parent issue is already closed and shipped. Linked to parent instead. > Document Configurable Bucket Sizes in bucketCache > - > > Key: HBASE-11735 > URL: https://issues.apache.org/jira/browse/HBASE-11735 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.6 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11735: - Issue Type: Task (was: Sub-task) Parent: (was: HBASE-10641) > Document Configurable Bucket Sizes in bucketCache > - > > Key: HBASE-11735 > URL: https://issues.apache.org/jira/browse/HBASE-11735 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.6 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11735: - Affects Version/s: 0.99.0 0.98.4 0.98.5 > Document Configurable Bucket Sizes in bucketCache > - > > Key: HBASE-11735 > URL: https://issues.apache.org/jira/browse/HBASE-11735 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.6 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11735: - Fix Version/s: (was: 0.98.4) 0.98.6 > Document Configurable Bucket Sizes in bucketCache > - > > Key: HBASE-11735 > URL: https://issues.apache.org/jira/browse/HBASE-11735 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.6 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096427#comment-14096427 ] Nick Dimiduk commented on HBASE-11735: -- Until 11550 is in, i think the string must be comma-delimited only; no spaces. > Document Configurable Bucket Sizes in bucketCache > - > > Key: HBASE-11735 > URL: https://issues.apache.org/jira/browse/HBASE-11735 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.4 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-2821) Keep young storefiles at lower replication
[ https://issues.apache.org/jira/browse/HBASE-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096416#comment-14096416 ] Nick Dimiduk commented on HBASE-2821: - This is an interesting idea. It probably looks a bit different in a world with HDFS storage tiers. In the mean time, this patch is reasonable to emulate the Colossus style (non-chained) writes. bq. So eliminating the third replica without changing block placement policy doesn't buy us much (if anything) Even if going off-rack, it probably buys us a good bit on the long tail. I'd like to try it out and see what we see. cc [~nkeywal]. > Keep young storefiles at lower replication > -- > > Key: HBASE-2821 > URL: https://issues.apache.org/jira/browse/HBASE-2821 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Todd Lipcon >Assignee: Andrew Purtell > Fix For: 2.0.0 > > Attachments: HBASE-2821.patch, lifetime-distribution.png, > storefile_age.pl > > > jgray and I were brainstorming some ideas about this: > In a typical heavy-write scenario, many store files do not last very long. > They're flushed and then within a small number of seconds a compaction runs > and they get deleted. For these "short lifetime" store files, it's less > likely that a failure will occur during the window in which they're valid. > So, I think we can consider some optimizations like the following: > - Flush files at replication count 2. Scan once a minute for any store files > in the region that are older than 2 minutes. If they're found, increase their > replication to 3. (alternatively, queue them to avoid scanning) > - More dangerous: flush files at replication count 1, but don't count them > when figuring log expiration. So, if they get lost, we force log splitting to > recover. > The performance gain here is that we avoid the network and disk transfer of > writing the third replica for a file that we're just about to delete anyway. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10017) HRegionPartitioner, rows directed to last partition are wrongly mapped.
[ https://issues.apache.org/jira/browse/HBASE-10017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096415#comment-14096415 ] Alex Newman commented on HBASE-10017: - amazing! > HRegionPartitioner, rows directed to last partition are wrongly mapped. > --- > > Key: HBASE-10017 > URL: https://issues.apache.org/jira/browse/HBASE-10017 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 0.94.6 >Reporter: Roman Nikitchenko >Priority: Critical > Attachments: HBASE-10017-r1544633.patch, HBASE-10017-r1544633.patch, > TEST-org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.xml.gz, > TestHRegionServerBulkLoad-more-splits.txt, > TestHRegionServerBulkLoad-more-splits.txt, patchSiteOutput.txt > > > Inside HRegionPartitioner class there is getPartition() method which should > map first numPartitions regions to appropriate partitions 1:1. But based on > condition last region is hashed which could lead to last reducer not having > any data. This is considered serious issue. > I reproduced this only starting from 16 regions per table. Original defect > was found in 0.94.6 but at least today's trunk and 0.91 branch head have the > same HRegionPartitioner code in this part which means the same issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11702) Better introspection of long running compactions
[
https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096418#comment-14096418
]
Hadoop QA commented on HBASE-11702:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661560/HBASE-11702_v3.patch
against trunk revision .
ATTACHMENT ID: 12661560
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10417//console
This message is automatically generated.
> Better introspection of long running compactions
>
>
> Key: HBASE-11702
> URL: https://issues.apache.org/jira/browse/HBASE-11702
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11702.patch, HBASE-11702_v2-0.98.patch,
> HBASE-11702_v2.patch, HBASE-11702_v3-0.98.patch, HBASE-11702_v3.patch
>
>
> For better introspection of long running compactions, periodically print
> compaction progress for a file at DEBUG level (thread name, file path, total
> compacted KVs, total compacted bytes, completion percent, rate).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10017) HRegionPartitioner, rows directed to last partition are wrongly mapped.
[
https://issues.apache.org/jira/browse/HBASE-10017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096414#comment-14096414
]
Hadoop QA commented on HBASE-10017:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12617251/TestHRegionServerBulkLoad-more-splits.txt
against trunk revision .
ATTACHMENT ID: 12617251
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10416//console
This message is automatically generated.
> HRegionPartitioner, rows directed to last partition are wrongly mapped.
> ---
>
> Key: HBASE-10017
> URL: https://issues.apache.org/jira/browse/HBASE-10017
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 0.94.6
>Reporter: Roman Nikitchenko
>Priority: Critical
> Attachments: HBASE-10017-r1544633.patch, HBASE-10017-r1544633.patch,
> TEST-org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.xml.gz,
> TestHRegionServerBulkLoad-more-splits.txt,
> TestHRegionServerBulkLoad-more-splits.txt, patchSiteOutput.txt
>
>
> Inside HRegionPartitioner class there is getPartition() method which should
> map first numPartitions regions to appropriate partitions 1:1. But based on
> condition last region is hashed which could lead to last reducer not having
> any data. This is considered serious issue.
> I reproduced this only starting from 16 regions per table. Original defect
> was found in 0.94.6 but at least today's trunk and 0.91 branch head have the
> same HRegionPartitioner code in this part which means the same issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096388#comment-14096388 ] Misty Stanley-Jones commented on HBASE-11735: - Configurable sizes via hbase.bucketcache.bucket.sizes, should be a comma-separated list of block sizes, like 4096, 8192 Should be in increasing order, HBASE-11550 will check for that. For only one size of bucket, use hbase.bucketcache.bucket.size instead. Goal is to optimize buckets based on data access patterns. > Document Configurable Bucket Sizes in bucketCache > - > > Key: HBASE-11735 > URL: https://issues.apache.org/jira/browse/HBASE-11735 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.98.4 > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10834) Better error messaging on issuing grant commands in non-authz mode
[
https://issues.apache.org/jira/browse/HBASE-10834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096380#comment-14096380
]
Srikanth Srungarapu commented on HBASE-10834:
-
Tested the patch on the secure cluster. The output:
{code}
hbase(main):003:0> grant "test", "RWCXA"
0 row(s) in 0.3920 seconds
{code}
The output on non-secure cluster is as follows:
{code}
hbase(main):002:0> grant "test", "RWXCA"
ERROR: Command not supported as authorization is turned off
{code}
> Better error messaging on issuing grant commands in non-authz mode
> --
>
> Key: HBASE-10834
> URL: https://issues.apache.org/jira/browse/HBASE-10834
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.94.17
>Reporter: Srikanth Srungarapu
>Assignee: Srikanth Srungarapu
>Priority: Trivial
> Attachments: HBASE-10834.patch, HBASE-10834_v2.patch,
> HBASE-10834_v3.patch, HBASE-10834_v4.patch
>
>
> Running the below sequence of steps should give a better error messaging
> rather than "table not found" error.
> {code}
> hbase(main):009:0> grant "test", "RWCXA"
> ERROR: Unknown table _acl_!
> Here is some help for this command:
> Grant users specific rights.
> Syntax : grant [ [ [ qualifier>]]
> permissions is either zero or more letters from the set "RWXCA".
> READ('R'), WRITE('W'), EXEC('X'), CREATE('C'), ADMIN('A')
> For example:
> hbase> grant 'bobsmith', 'RWXCA'
> hbase> grant 'bobsmith', 'RW', 't1', 'f1', 'col1'
> {code}
> Instead of ERROR: Unknown table _acl_!, hbase should give out a warning like
> "Command not supported in non-authz mode(as acl table is only created if
> authz is turned on)"
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11735) Document Configurable Bucket Sizes in bucketCache
Misty Stanley-Jones created HBASE-11735: --- Summary: Document Configurable Bucket Sizes in bucketCache Key: HBASE-11735 URL: https://issues.apache.org/jira/browse/HBASE-11735 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11734) Document changed behavior of hbase.hstore.time.to.purge.deletes
[ https://issues.apache.org/jira/browse/HBASE-11734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11734: Status: Patch Available (was: Open) > Document changed behavior of hbase.hstore.time.to.purge.deletes > --- > > Key: HBASE-11734 > URL: https://issues.apache.org/jira/browse/HBASE-11734 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.96.3, 0.98.2, 0.94.19 > > Attachments: HBASE-11734.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11734) Document changed behavior of hbase.hstore.time.to.purge.deletes
[ https://issues.apache.org/jira/browse/HBASE-11734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11734: Attachment: HBASE-11734.patch Attached patch should do the job. > Document changed behavior of hbase.hstore.time.to.purge.deletes > --- > > Key: HBASE-11734 > URL: https://issues.apache.org/jira/browse/HBASE-11734 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 0.94.19, 0.98.2, 0.96.3 > > Attachments: HBASE-11734.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11550: - Affects Version/s: 0.99.0 0.98.4 0.98.5 > Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted > -- > > Key: HBASE-11550 > URL: https://issues.apache.org/jira/browse/HBASE-11550 > Project: HBase > Issue Type: Task >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, > HBASE-11550-v3.patch, HBASE-11550-v4.patch, HBASE-11550.patch > > > User can pass bucket sizes through "hbase.bucketcache.bucket.sizes" config > entry. > The sizes are supposed to be in increasing order. Validation should be added > in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11550: - Issue Type: Bug (was: Task) > Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted > -- > > Key: HBASE-11550 > URL: https://issues.apache.org/jira/browse/HBASE-11550 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.0, 0.98.4, 0.98.5 >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, > HBASE-11550-v3.patch, HBASE-11550-v4.patch, HBASE-11550.patch > > > User can pass bucket sizes through "hbase.bucketcache.bucket.sizes" config > entry. > The sizes are supposed to be in increasing order. Validation should be added > in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096362#comment-14096362 ] Nick Dimiduk commented on HBASE-11550: -- Patch v4 is great, +1. I'll commit this in a couple hours unless there's an objection. Thanks [~gustavoanatoly]. We can leave tests for detecting wasted space as a separate ticket. > Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted > -- > > Key: HBASE-11550 > URL: https://issues.apache.org/jira/browse/HBASE-11550 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, > HBASE-11550-v3.patch, HBASE-11550-v4.patch, HBASE-11550.patch > > > User can pass bucket sizes through "hbase.bucketcache.bucket.sizes" config > entry. > The sizes are supposed to be in increasing order. Validation should be added > in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11610) Enhance remote meta updates
[ https://issues.apache.org/jira/browse/HBASE-11610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari updated HBASE-11610: -- Attachment: HBASE-11610.patch Without patch, it took around 1hr 20 mins to assign 1M region. With patch, it takes around 12 mins. This is on 0.98 > Enhance remote meta updates > --- > > Key: HBASE-11610 > URL: https://issues.apache.org/jira/browse/HBASE-11610 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Virag Kothari > Attachments: HBASE-11610.patch > > > Currently, if the meta region is on a regionserver instead of the master, > meta update is synchronized on one HTable instance. We should be able to do > better. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11734) Document changed behavior of hbase.hstore.time.to.purge.deletes
Misty Stanley-Jones created HBASE-11734: --- Summary: Document changed behavior of hbase.hstore.time.to.purge.deletes Key: HBASE-11734 URL: https://issues.apache.org/jira/browse/HBASE-11734 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter
[ https://issues.apache.org/jira/browse/HBASE-11476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096335#comment-14096335 ] Hudson commented on HBASE-11476: FAILURE: Integrated in HBase-TRUNK #5395 (See [https://builds.apache.org/job/HBase-TRUNK/5395/]) HBASE-11476 Expand 'Conceptual View' section of Data Model chapter (Misty Stanley-Jones) (jmhsieh: rev 92c3b877c0a2f1ca0fa6c791e41fbcb889f220ad) * src/main/docbkx/book.xml > Expand 'Conceptual View' section of Data Model chapter > --- > > Key: HBASE-11476 > URL: https://issues.apache.org/jira/browse/HBASE-11476 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 2.0.0 > > Attachments: HBASE-11476.patch, HBASE-11476.patch > > > Could use some updating and expansion to emphasize the differences between > HBase and an RDBMS. I found > http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable > which is just excellent and we should link to it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11591) Scanner fails to retrieve KV from bulk loaded file with highest sequence id than the cell's mvcc in a non-bulk loaded file
[
https://issues.apache.org/jira/browse/HBASE-11591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096324#comment-14096324
]
Jeffrey Zhong commented on HBASE-11591:
---
The patch looks good to me(+1)! Basically it restores old behavior for bulk
loaded files by treating all KVs involved with mvcc==0. Thanks.
> Scanner fails to retrieve KV from bulk loaded file with highest sequence id
> than the cell's mvcc in a non-bulk loaded file
> ---
>
> Key: HBASE-11591
> URL: https://issues.apache.org/jira/browse/HBASE-11591
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.99.0
>
> Attachments: HBASE-11591.patch, TestBulkload.java
>
>
> See discussion in HBASE-11339.
> When we have a case where there are same KVs in two files one produced by
> flush/compaction and the other thro the bulk load.
> Both the files have some same kvs which matches even in timestamp.
> Steps:
> Add some rows with a specific timestamp and flush the same.
> Bulk load a file with the same data.. Enusre that "assign seqnum" property is
> set.
> The bulk load should use HFileOutputFormat2 (or ensure that we write the
> bulk_time_output key).
> This would ensure that the bulk loaded file has the highest seq num.
> Assume the cell in the flushed/compacted store file is
> row1,cf,cq,ts1, value1 and the cell in the bulk loaded file is
> row1,cf,cq,ts1,value2
> (There are no parallel scans).
> Issue a scan on the table in 0.96. The retrieved value is
> row1,cf1,cq,ts1,value2
> But the same in 0.98 will retrieve row1,cf1,cq,ts2,value1.
> This is a behaviour change. This is because of this code
> {code}
> public int compare(KeyValueScanner left, KeyValueScanner right) {
> int comparison = compare(left.peek(), right.peek());
> if (comparison != 0) {
> return comparison;
> } else {
> // Since both the keys are exactly the same, we break the tie in favor
> // of the key which came latest.
> long leftSequenceID = left.getSequenceID();
> long rightSequenceID = right.getSequenceID();
> if (leftSequenceID > rightSequenceID) {
> return -1;
> } else if (leftSequenceID < rightSequenceID) {
> return 1;
> } else {
> return 0;
> }
> }
> }
> {code}
> Here in 0.96 case the mvcc of the cell in both the files will have 0 and so
> the comparison will happen from the else condition . Where the seq id of the
> bulk loaded file is greater and would sort out first ensuring that the scan
> happens from that bulk loaded file.
> In case of 0.98+ as we are retaining the mvcc+seqid we are not making the
> mvcc as 0 (remains a non zero positive value). Hence the compare() sorts out
> the cell in the flushed/compacted file. Which means though we know the
> lateset file is the bulk loaded file we don't scan the data.
> Seems to be a behaviour change. Will check on other corner cases also but we
> are trying to know the behaviour of bulk load because we are evaluating if it
> can be used for MOB design.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096314#comment-14096314 ] Andrew Purtell commented on HBASE-11702: Thanks Lars. Will commit this in a few hours unless there is an objection > Better introspection of long running compactions > > > Key: HBASE-11702 > URL: https://issues.apache.org/jira/browse/HBASE-11702 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11702.patch, HBASE-11702_v2-0.98.patch, > HBASE-11702_v2.patch, HBASE-11702_v3-0.98.patch, HBASE-11702_v3.patch > > > For better introspection of long running compactions, periodically print > compaction progress for a file at DEBUG level (thread name, file path, total > compacted KVs, total compacted bytes, completion percent, rate). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-10202: Status: Patch Available (was: Open) > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.96.1.1, 0.94.14, 0.98.0, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > Attachments: HBASE-10202-1.patch, HBASE-10202-2.patch, > HBASE-10202.patch, HBASE-10202.patch > > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-10202: Attachment: HBASE-10202.patch Thanks for the review, and especially sorry about missing that rebasing error. I hope this one addresses all the feedback. > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.98.0, 0.94.14, 0.99.0, 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > Attachments: HBASE-10202-1.patch, HBASE-10202-2.patch, > HBASE-10202.patch, HBASE-10202.patch > > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-10202: Status: Open (was: Patch Available) > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.96.1.1, 0.94.14, 0.98.0, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > Attachments: HBASE-10202-1.patch, HBASE-10202-2.patch, > HBASE-10202.patch, HBASE-10202.patch > > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[
https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-11733:
Attachment: HBASE-11733-v0.patch
I have etracted the common code in a execOperation() method, and now each one
of the pre/post method have only to implement something like this:
{code}
public boolean preCreateNamespace(final NamespaceDescriptor ns) throws
IOException {
return execOperation(new CoprocessorOperation() {
@Override
public void call(MasterObserver oserver,
ObserverContext ctx)
throws IOException {
oserver.preCreateNamespace(ctx, ns);
}
});
}
public RegionScanner preScannerOpen(final Scan scan) throws IOException {
final ObjectReference res = new
ObjectReference(null);
boolean bypass = execOperation(new RegionOperation() {
@Override
public void call(RegionObserver oserver,
ObserverContext ctx)
throws IOException {
res.set(oserver.preScannerOpen(ctx, scan, res.get()));
}
});
return bypass ? res.get() : null;
}
{code}
> Avoid copy-paste in Master/Region CoprocessorHost
> -
>
> Key: HBASE-11733
> URL: https://issues.apache.org/jira/browse/HBASE-11733
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-11733-v0.patch
>
>
> all the methods of MasterCoprocessorHost and RegionCoprocessorHost are
> copy-paste, and a couple of them have some missing fixes.
> The body of each function is the same, the only difference is the call to the
> specific coprocessor.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
[ https://issues.apache.org/jira/browse/HBASE-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-11733: Status: Patch Available (was: Open) > Avoid copy-paste in Master/Region CoprocessorHost > - > > Key: HBASE-11733 > URL: https://issues.apache.org/jira/browse/HBASE-11733 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-11733-v0.patch > > > all the methods of MasterCoprocessorHost and RegionCoprocessorHost are > copy-paste, and a couple of them have some missing fixes. > The body of each function is the same, the only difference is the call to the > specific coprocessor. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11733) Avoid copy-paste in Master/Region CoprocessorHost
Matteo Bertozzi created HBASE-11733: --- Summary: Avoid copy-paste in Master/Region CoprocessorHost Key: HBASE-11733 URL: https://issues.apache.org/jira/browse/HBASE-11733 Project: HBase Issue Type: Bug Components: Coprocessors Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Trivial Fix For: 2.0.0 all the methods of MasterCoprocessorHost and RegionCoprocessorHost are copy-paste, and a couple of them have some missing fixes. The body of each function is the same, the only difference is the call to the specific coprocessor. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11580) Failover handling for secondary region replicas
[
https://issues.apache.org/jira/browse/HBASE-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096293#comment-14096293
]
Jonathan Hsieh commented on HBASE-11580:
{quote}
As an optimization so that the secondary will sooner start serving data, the
region server opening a secondary region replica will make an RPC to the RS
serving the primary for a flush. This will result either a flush, which will
asynchronously be replayed to the secondary, or a no-op because memstores are
empty. In the latter case, we should still drop a WAL marker though.
{quote}
Would this be on by default or would this be an option? It seems that in a
failure situation we would hit a "flush amplification" problem which seems
concerning.
Let's say we have a 20 node cluster with each rs having 25 regions on them. If
those regions have 2 secondaries, each hosts would have 50 secondaries. Let's
say one rs goes down When these secondaries gets rebalanced, they'd get spread
around a bunch likely landing on every or almost every node in the cluster. If
each secondary region reopening triggers a flush request we'd end up with a lot
of flush requests, all around the same time all across the cluster, no? This
seems problematic. We could probably coalesce some of the flushes (each rs
receiving secondaries could consolidate all into one flush) but if we had 5
machines go down wouldn't we still get 5x the flushes?
> Failover handling for secondary region replicas
> ---
>
> Key: HBASE-11580
> URL: https://issues.apache.org/jira/browse/HBASE-11580
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> With the async wal approach (HBASE-11568), the edits are not persisted (to
> wal) in the secondary region replicas. However this means that we have to
> deal with secondary region replica failures.
> We can seek to re-replicate the edits from primary to the secondary when the
> secondary region is opened in another server but this would mean to setup a
> replication queue again, and holding on to the wals for longer.
> Instead, we can design it so that the edits form the secondaries are not
> persisted to wal, and if the secondary replica fails over, it will not start
> serving reads until it has guaranteed that it has all the past data.
> For guaranteeing that the secondary replica has all the edits before serving
> reads, we can use flush and region opening markers. Whenever a region open
> event is seen, it writes all the files at the time of opening to wal
> (HBASE-11512). In case of flush, the flushed file is written as well, and the
> secondary replica can do a ls for the store files and pick up all the files
> before the seqId of the flushed file. So, in this design, the secodary
> replica will wait until it sees and replays a flush or region open marker
> from wal from primary. and then start serving. For speeding up replica
> opening time, we can trigger a flush to the primary whenever the secondary
> replica opens as an optimization.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10017) HRegionPartitioner, rows directed to last partition are wrongly mapped.
[ https://issues.apache.org/jira/browse/HBASE-10017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096275#comment-14096275 ] Alex Newman commented on HBASE-10017: - I assume this has been fixed? > HRegionPartitioner, rows directed to last partition are wrongly mapped. > --- > > Key: HBASE-10017 > URL: https://issues.apache.org/jira/browse/HBASE-10017 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 0.94.6 >Reporter: Roman Nikitchenko >Priority: Critical > Attachments: HBASE-10017-r1544633.patch, HBASE-10017-r1544633.patch, > TEST-org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad.xml.gz, > TestHRegionServerBulkLoad-more-splits.txt, > TestHRegionServerBulkLoad-more-splits.txt, patchSiteOutput.txt > > > Inside HRegionPartitioner class there is getPartition() method which should > map first numPartitions regions to appropriate partitions 1:1. But based on > condition last region is hashed which could lead to last reducer not having > any data. This is considered serious issue. > I reproduced this only starting from 16 regions per table. Original defect > was found in 0.94.6 but at least today's trunk and 0.91 branch head have the > same HRegionPartitioner code in this part which means the same issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11656) Document how to script snapshots
[ https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones resolved HBASE-11656. - Resolution: Won't Fix > Document how to script snapshots > > > Key: HBASE-11656 > URL: https://issues.apache.org/jira/browse/HBASE-11656 > Project: HBase > Issue Type: Sub-task > Components: documentation, shell >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11656.patch, HBASE-11656.patch, HBASE-11656.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11656) Document how to script snapshots
[ https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096263#comment-14096263 ] Misty Stanley-Jones commented on HBASE-11656: - Pulling this out of consideration as too specific for the RefGuide. > Document how to script snapshots > > > Key: HBASE-11656 > URL: https://issues.apache.org/jira/browse/HBASE-11656 > Project: HBase > Issue Type: Sub-task > Components: documentation, shell >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11656.patch, HBASE-11656.patch, HBASE-11656.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11656) Document how to script snapshots
[ https://issues.apache.org/jira/browse/HBASE-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11656: Status: Open (was: Patch Available) > Document how to script snapshots > > > Key: HBASE-11656 > URL: https://issues.apache.org/jira/browse/HBASE-11656 > Project: HBase > Issue Type: Sub-task > Components: documentation, shell >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11656.patch, HBASE-11656.patch, HBASE-11656.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10071) Backport HBASE-6592 to 0.94 branch
[
https://issues.apache.org/jira/browse/HBASE-10071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10071:
--
Fix Version/s: 0.94.23
> Backport HBASE-6592 to 0.94 branch
> --
>
> Key: HBASE-10071
> URL: https://issues.apache.org/jira/browse/HBASE-10071
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 0.94.14
>Reporter: cuijianwei
> Fix For: 0.94.23
>
> Attachments: HBASE-10071-0.94-v1.patch
>
>
> Users tend to run hbase shell to query hbase quickly. The result will be
> shown as binary format which may not look clear enough when users write
> columns using specified types, such as long/int/short. Therefore, it may be
> helpful if the results could be shown as specified format. We make a patch to
> extend get/scan in hbase shell in which user could specify the data type in
> get/scan for each column as:
> {code}
> scan 'table', {COLUMNS=>['CF:QF:long']}
> get 'table', 'r0', {COLUMN=>'CF:QF:long'}
> {code}
> Then, the result will be shown as Long type. The result of above get will be:
> {code}
> COLUMNCELL
>
>
> CF:QFtimestamp=24311261, value=24311229
> {code}
> This extended format is compatible with previous format, if users do not
> specify the data type, the command will also work and output binary format.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11730) Document release managers for non-deprecated branches
[ https://issues.apache.org/jira/browse/HBASE-11730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096243#comment-14096243 ] Lars Hofhansl commented on HBASE-11730: --- +1 to everything Andy said. Not sure we need to document this. An RM is just somebody who volunteers to do that. In some projects there's an RM for each individual release. In HBase we happen to come to a model where the same person does all the releases for a particular major version. We're not very formal about this. > Document release managers for non-deprecated branches > - > > Key: HBASE-11730 > URL: https://issues.apache.org/jira/browse/HBASE-11730 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Sean Busbey > > New development goes against trunk and is backported as desired to existing > release branches. From what I have seen on the jira, it looks like each > branch's release manager makes the call on backporting a particular issue. > We should document both this norm and who the relevant release manager is for > each branch. > In the current docs, I'd suggest adding the RM list to the "Codelines" > section (18.11.1) and add a brief explanation of pinging the RM as a new > section after "submitting a patch again" (18.12.6). > Post HBASE-4593, the note about pinging a prior branch RM should just go as a > bullet in the "patch workflow." -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096241#comment-14096241 ] Lars Hofhansl commented on HBASE-11702: --- +1 (For large compaction we could update the metric as we go - as frequently as we emit the log lines - but maybe this is getting to be overkill) > Better introspection of long running compactions > > > Key: HBASE-11702 > URL: https://issues.apache.org/jira/browse/HBASE-11702 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11702.patch, HBASE-11702_v2-0.98.patch, > HBASE-11702_v2.patch, HBASE-11702_v3-0.98.patch, HBASE-11702_v3.patch > > > For better introspection of long running compactions, periodically print > compaction progress for a file at DEBUG level (thread name, file path, total > compacted KVs, total compacted bytes, completion percent, rate). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11702: --- Attachment: HBASE-11702_v3-0.98.patch > Better introspection of long running compactions > > > Key: HBASE-11702 > URL: https://issues.apache.org/jira/browse/HBASE-11702 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11702.patch, HBASE-11702_v2-0.98.patch, > HBASE-11702_v2.patch, HBASE-11702_v3-0.98.patch, HBASE-11702_v3.patch > > > For better introspection of long running compactions, periodically print > compaction progress for a file at DEBUG level (thread name, file path, total > compacted KVs, total compacted bytes, completion percent, rate). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11702: --- Attachment: HBASE-11702_v3.patch v3 patch adds additional requested metrics. It also fixes a problem with the previous 0.98 patch where I missed updating MetricsRegionServerSourceImpl in the hbase-hadoop1-compat module. > Better introspection of long running compactions > > > Key: HBASE-11702 > URL: https://issues.apache.org/jira/browse/HBASE-11702 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11702.patch, HBASE-11702_v2-0.98.patch, > HBASE-11702_v2.patch, HBASE-11702_v3-0.98.patch, HBASE-11702_v3.patch > > > For better introspection of long running compactions, periodically print > compaction progress for a file at DEBUG level (thread name, file path, total > compacted KVs, total compacted bytes, completion percent, rate). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11702: --- Status: Open (was: Patch Available) > Better introspection of long running compactions > > > Key: HBASE-11702 > URL: https://issues.apache.org/jira/browse/HBASE-11702 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11702.patch, HBASE-11702_v2-0.98.patch, > HBASE-11702_v2.patch, HBASE-11702_v3-0.98.patch, HBASE-11702_v3.patch > > > For better introspection of long running compactions, periodically print > compaction progress for a file at DEBUG level (thread name, file path, total > compacted KVs, total compacted bytes, completion percent, rate). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11702) Better introspection of long running compactions
[ https://issues.apache.org/jira/browse/HBASE-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11702: --- Status: Patch Available (was: Open) > Better introspection of long running compactions > > > Key: HBASE-11702 > URL: https://issues.apache.org/jira/browse/HBASE-11702 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11702.patch, HBASE-11702_v2-0.98.patch, > HBASE-11702_v2.patch, HBASE-11702_v3-0.98.patch, HBASE-11702_v3.patch > > > For better introspection of long running compactions, periodically print > compaction progress for a file at DEBUG level (thread name, file path, total > compacted KVs, total compacted bytes, completion percent, rate). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter
[ https://issues.apache.org/jira/browse/HBASE-11476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11476: --- Resolution: Fixed Fix Version/s: 2.0.0 0.99.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) committed to branch-1 and master. > Expand 'Conceptual View' section of Data Model chapter > --- > > Key: HBASE-11476 > URL: https://issues.apache.org/jira/browse/HBASE-11476 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 2.0.0 > > Attachments: HBASE-11476.patch, HBASE-11476.patch > > > Could use some updating and expansion to emphasize the differences between > HBase and an RDBMS. I found > http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable > which is just excellent and we should link to it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter
[
https://issues.apache.org/jira/browse/HBASE-11476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096184#comment-14096184
]
Jonathan Hsieh commented on HBASE-11476:
{code}
+ You can specify the maximum number of versions of a value that
HBase retains, per column
+family. When the maximum number of versions is reached, the oldest
version is deleted.
+By default, only the newest version is kept.
+
{code}
nit -- .. When the maximum number of versions is reached, the oldest *versions
are eventually* deleted. By default ...
I'm going to fix this on commit. Thanks misty.
> Expand 'Conceptual View' section of Data Model chapter
> ---
>
> Key: HBASE-11476
> URL: https://issues.apache.org/jira/browse/HBASE-11476
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-11476.patch, HBASE-11476.patch
>
>
> Could use some updating and expansion to emphasize the differences between
> HBase and an RDBMS. I found
> http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable
> which is just excellent and we should link to it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11717) Remove unused config 'hbase.offheapcache.percentage' from hbase-default.xml and book
[
https://issues.apache.org/jira/browse/HBASE-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096181#comment-14096181
]
Enis Soztutar commented on HBASE-11717:
---
+1 for branch-1+.
> Remove unused config 'hbase.offheapcache.percentage' from hbase-default.xml
> and book
>
>
> Key: HBASE-11717
> URL: https://issues.apache.org/jira/browse/HBASE-11717
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.99.0
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Minor
> Fix For: 0.99.0, 2.0.0
>
> Attachments: HBASE-11717.patch, HBASE-11717_V2.patch,
> HBASE-11717_V3.patch
>
>
> hbase.offheapcache.percentage is unused now.
> {code}
>
> hbase.offheapcache.percentage
> 0
> The percentage of the off heap space
> (-XX:MaxDirectMemorySize) to be
> allocated towards the experimental off heap "SlabCache" (This is
> different to
> the BucketCache -- see the package javadoc for
> org.apache.hadoop.hbase.io.hfile
> for more on your options). If you desire the cache to be disabled,
> simply set this
> value to 0.
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096178#comment-14096178
]
Enis Soztutar commented on HBASE-11657:
---
A couple of comments:
- RegionLocator -> I like the name.
- Should RegionLocator extend Closeable?
- clearRegionCache() seems like an overkill for exposing.
- HRegionLocation and RegionLocations have been made InterfaceAudience.Private
in 0.99. The main idea was to make start-end keys are byte[] ranges the main
public API, but not the regions / region locations themselves. I know that some
advanced users have been using HRL for some time, but I am not keen to make it
a public API. What do you guys think?
> Put HTable region methods in an interface
> -
>
> Key: HBASE-11657
> URL: https://issues.apache.org/jira/browse/HBASE-11657
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.99.0
>Reporter: Carter
>Assignee: Carter
> Fix For: 0.99.0
>
> Attachments: HBASE_11657.patch, HBASE_11657_v2.patch,
> HBASE_11657_v3.patch, HBASE_11657_v3.patch, HBASE_11657_v4.patch
>
>
> Most of the HTable methods are now abstracted by HTableInterface, with the
> notable exception of the following methods that pertain to region metadata:
> {code}
> HRegionLocation getRegionLocation(final String row)
> HRegionLocation getRegionLocation(final byte [] row)
> HRegionLocation getRegionLocation(final byte [] row, boolean reload)
> byte [][] getStartKeys()
> byte[][] getEndKeys()
> Pair getStartEndKeys()
> void clearRegionCache()
> {code}
> and a default scope method which maybe should be bundled with the others:
> {code}
> List listRegionLocations()
> {code}
> Since the consensus seems to be that these would muddy HTableInterface with
> non-core functionality, where should it go? MapReduce looks up the region
> boundaries, so it needs to be exposed somewhere.
> Let me throw out a straw man to start the conversation. I propose:
> {code}
> org.apache.hadoop.hbase.client.HRegionInterface
> {code}
> Have HTable implement this interface. Also add these methods to HConnection:
> {code}
> HRegionInterface getTableRegion(TableName tableName)
> HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
> {code}
> [~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[
https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096161#comment-14096161
]
Hadoop QA commented on HBASE-11550:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12661511/HBASE-11550-v4.patch
against trunk revision .
ATTACHMENT ID: 12661511
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestRollingRestart
org.apache.hadoop.hbase.TestRegionRebalancing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10415//console
This message is automatically generated.
> Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
> --
>
> Key: HBASE-11550
> URL: https://issues.apache.org/jira/browse/HBASE-11550
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
> Fix For: 0.99.0, 2.0.0, 0.98.6
>
> Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch,
> HBASE-11550-v3.patch, HBASE-11550-v4.patch, HBASE-11550.patch
>
>
> User can pass bucket sizes through "hbase.bucketcache.bucket.sizes" config
> entry.
> The sizes are supposed to be in increasing order. Validation should be added
> in CacheConfig#getL2().
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11580) Failover handling for secondary region replicas
[ https://issues.apache.org/jira/browse/HBASE-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096089#comment-14096089 ] Enis Soztutar commented on HBASE-11580: --- Sorry I should have missed that one. The reason we need special handling in secondary region replicas is that the edits are replayed to secondaries, but the WAL is skipped in secondary replicas. This is because we do not want to cause additional WAL writes for secondaries. However, this implies that when the secondary region replica starts serving, it cannot make sure that it has all the memstore updates because some of it may have been missed after the last flush. So, what I am thinking is a mechanism with two parts: - Whenever a secondary is opened, it will reject read requests until it makes sure (on its own) that is has all the updates up to a seqId. The seqId has to be greater than the last flush seqId of the files since a previous open of the same region replica might have replayed some in-memory data, and we do not want the seqId served from a particular replica_id to go back. At this state, the secondary will watch for flush start and commit events from WAL replay. The secondary will ignore all replayed edits before seeing a flush start record. After seeing flush start, it will apply all edits to memstores. Once the secondary sees the corresponding flush commit, it can pick up the new flushed file, and start serving from all flushed files and its memstore data. - As an optimization so that the secondary will sooner start serving data, the region server opening a secondary region replica will make an RPC to the RS serving the primary for a flush. This will result either a flush, which will asynchronously be replayed to the secondary, or a no-op because memstores are empty. In the latter case, we should still drop a WAL marker though. > Failover handling for secondary region replicas > --- > > Key: HBASE-11580 > URL: https://issues.apache.org/jira/browse/HBASE-11580 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > > With the async wal approach (HBASE-11568), the edits are not persisted (to > wal) in the secondary region replicas. However this means that we have to > deal with secondary region replica failures. > We can seek to re-replicate the edits from primary to the secondary when the > secondary region is opened in another server but this would mean to setup a > replication queue again, and holding on to the wals for longer. > Instead, we can design it so that the edits form the secondaries are not > persisted to wal, and if the secondary replica fails over, it will not start > serving reads until it has guaranteed that it has all the past data. > For guaranteeing that the secondary replica has all the edits before serving > reads, we can use flush and region opening markers. Whenever a region open > event is seen, it writes all the files at the time of opening to wal > (HBASE-11512). In case of flush, the flushed file is written as well, and the > secondary replica can do a ls for the store files and pick up all the files > before the seqId of the flushed file. So, in this design, the secodary > replica will wait until it sees and replays a flush or region open marker > from wal from primary. and then start serving. For speeding up replica > opening time, we can trigger a flush to the primary whenever the secondary > replica opens as an optimization. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11723) Document all options of bin/hbase command
[ https://issues.apache.org/jira/browse/HBASE-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096024#comment-14096024 ] Hudson commented on HBASE-11723: FAILURE: Integrated in HBase-1.0 #98 (See [https://builds.apache.org/job/HBase-1.0/98/]) HBASE-11723 Document all options of bin/hbase command (Misty Stanley-Jones) (jmhsieh: rev acf56f18dd546cd0748e3c7ab3a2130cbae2f77b) * src/main/docbkx/ops_mgt.xml > Document all options of bin/hbase command > - > > Key: HBASE-11723 > URL: https://issues.apache.org/jira/browse/HBASE-11723 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 0.99.0, 2.0.0 > > Attachments: HBASE-11723.patch > > > The bin/hbase command is not documented fully in the Ref Guide: > http://hbase.apache.org/book.html#tools > Specifically a few new options were added in HBASE-11649 and need to be > documented. Also the generic usage instructions need to be there. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11724) Add to RWQueueRpcExecutor the ability to split get and scan handlers
[
https://issues.apache.org/jira/browse/HBASE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095995#comment-14095995
]
Matteo Bertozzi commented on HBASE-11724:
-
{quote}read.share = 1.0 scan.share = .5 Does this mean that writes are 0%, gets
are 50% and scans are 50%{quote}
almost, but since 0% writes does not make sense you have at least 1 write
queue...
{quote}read.share = .5 .scan.share = .5
Does this mean that writes are 0%, gets are 50% and scans are 50%?
Or, does this mean that writes are 50%, gets are 25% and scans are 25%?
{quote}
The second one.. isn't this clear enough? "will split the READ CALL QUEUES
into small-read and long-read queues".
You split reads and writes, and then you split the scans. otherwise will result
too complicate to understand
can you wordify what you want to see in the doc? I'm a bit stuck here
> Add to RWQueueRpcExecutor the ability to split get and scan handlers
>
>
> Key: HBASE-11724
> URL: https://issues.apache.org/jira/browse/HBASE-11724
> Project: HBase
> Issue Type: New Feature
> Components: IPC/RPC
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11724-v0.patch
>
>
> RWQueueRpcExecutor has the devision between reads and writes requests, but we
> can split also small-reads and long-reads. This can be useful to force a
> deprioritization of scans on the RS.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (HBASE-11724) Add to RWQueueRpcExecutor the ability to split get and scan handlers
[
https://issues.apache.org/jira/browse/HBASE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095981#comment-14095981
]
Jonathan Hsieh edited comment on HBASE-11724 at 8/13/14 7:31 PM:
-
{quote}
Adding something like this? "A value of 0.5 means that there will be the same
number of short-read and long-read queues" and "A value > 0.5 means that there
will be more long-read queues then short-read queues" and "A value < 0.5 means
that there will be less long-read queues then short-read queues"
{quote}
Yeah. Let me give some examples that could illustrate my confusion and help
clarify.
hbase.ipc.server.callqueue.read.share = 1.0
hbase.ipc.server.callqueue.scan.share = .5
Does this mean that writes are 0%, gets are 50% and scans are 50%?
or Does this mean that writes are 0%, gets are 66% (1.0/(1.0+.5)) and scans are
33% (.5/(1.0+.5)?
hbase.ipc.server.callqueue.read.share = .5
hbase.ipc.server.callqueue.scan.share = .5
Does this mean that writes are 0%, gets are 50% and scans are 50%?
Or, does this mean that writes are 50%, gets are 25% and scans are 25%?
{quote}
Why? "share" may be interpreted as a boolean but if you see 0 as default
instead of false, it is probably not a boolean.. then you read about the 0 and
1 edge case... and you may start understanding that that probably is the
limit...
otherwise, you are just the fancy user that wants to break things and set
random crap as value...
{quote}
I'm not worried about fancy users, I'm worried about uninformed users that
unknowingly break things because they don't know or don't get an indication
that they set a crap value.
{quote}
If you want I can add "the value must be between 0.0 and 1.0" to the doc... but
you may want to add that also to all the other fields that we have in
hbase-default.xml
{quote}
Yes, I fact I strongly encourage this for other settings as well. It is even
better if the code warns when there is a violation. If we can add more
guardrails so uninformed users can't make silly mistakes (and get notified
about it), hbase becomes easier to use and we avoid bad situations
preemptively.
was (Author: jmhsieh):
{quote}
Adding something like this? "A value of 0.5 means that there will be the same
number of short-read and long-read queues" and "A value > 0.5 means that there
will be more long-read queues then short-read queues" and "A value < 0.5 means
that there will be less long-read queues then short-read queues"
{quote}
Yeah. Let me give some examples that could illustrate my confusion and help
clarify.
hbase.ipc.server.callqueue.read.share = 1.0
hbase.ipc.server.callqueue.scan.share = .5
Does this mean that writes are 0%, gets are 50% and scans are 50%?
hbase.ipc.server.callqueue.read.share = .5
hbase.ipc.server.callqueue.scan.share = .5
Does this mean that writes are 0%, gets are 50% and scans are 50%?
Or, does this mean that writes are 50%, gets are 25% and scans are 25%?
{quote}
Why? "share" may be interpreted as a boolean but if you see 0 as default
instead of false, it is probably not a boolean.. then you read about the 0 and
1 edge case... and you may start understanding that that probably is the
limit...
otherwise, you are just the fancy user that wants to break things and set
random crap as value...
{quote}
I'm not worried about fancy users, I'm worried about uninformed users that
unknowingly break things because they don't know or don't get an indication
that they set a crap value.
{quote}
If you want I can add "the value must be between 0.0 and 1.0" to the doc... but
you may want to add that also to all the other fields that we have in
hbase-default.xml
{quote}
Yes, I fact I strongly encourage this for other settings as well. It is even
better if the code warns when there is a violation. If we can add more
guardrails so uninformed users can't make silly mistakes (and get notified
about it), hbase becomes easier to use and we avoid bad situations
preemptively.
> Add to RWQueueRpcExecutor the ability to split get and scan handlers
>
>
> Key: HBASE-11724
> URL: https://issues.apache.org/jira/browse/HBASE-11724
> Project: HBase
> Issue Type: New Feature
> Components: IPC/RPC
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11724-v0.patch
>
>
> RWQueueRpcExecutor has the devision between reads and writes requests, but we
> can split also small-reads and long-reads. This can be useful to force a
> deprioritization of scans on the RS.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11724) Add to RWQueueRpcExecutor the ability to split get and scan handlers
[
https://issues.apache.org/jira/browse/HBASE-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095981#comment-14095981
]
Jonathan Hsieh commented on HBASE-11724:
{quote}
Adding something like this? "A value of 0.5 means that there will be the same
number of short-read and long-read queues" and "A value > 0.5 means that there
will be more long-read queues then short-read queues" and "A value < 0.5 means
that there will be less long-read queues then short-read queues"
{quote}
Yeah. Let me give some examples that could illustrate my confusion and help
clarify.
hbase.ipc.server.callqueue.read.share = 1.0
hbase.ipc.server.callqueue.scan.share = .5
Does this mean that writes are 0%, gets are 50% and scans are 50%?
hbase.ipc.server.callqueue.read.share = .5
hbase.ipc.server.callqueue.scan.share = .5
Does this mean that writes are 0%, gets are 50% and scans are 50%?
Or, does this mean that writes are 50%, gets are 25% and scans are 25%?
{quote}
Why? "share" may be interpreted as a boolean but if you see 0 as default
instead of false, it is probably not a boolean.. then you read about the 0 and
1 edge case... and you may start understanding that that probably is the
limit...
otherwise, you are just the fancy user that wants to break things and set
random crap as value...
{quote}
I'm not worried about fancy users, I'm worried about uninformed users that
unknowingly break things because they don't know or don't get an indication
that they set a crap value.
{quote}
If you want I can add "the value must be between 0.0 and 1.0" to the doc... but
you may want to add that also to all the other fields that we have in
hbase-default.xml
{quote}
Yes, I fact I strongly encourage this for other settings as well. It is even
better if the code warns when there is a violation. If we can add more
guardrails so uninformed users can't make silly mistakes (and get notified
about it), hbase becomes easier to use and we avoid bad situations
preemptively.
> Add to RWQueueRpcExecutor the ability to split get and scan handlers
>
>
> Key: HBASE-11724
> URL: https://issues.apache.org/jira/browse/HBASE-11724
> Project: HBase
> Issue Type: New Feature
> Components: IPC/RPC
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-11724-v0.patch
>
>
> RWQueueRpcExecutor has the devision between reads and writes requests, but we
> can split also small-reads and long-reads. This can be useful to force a
> deprioritization of scans on the RS.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11550) Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted
[ https://issues.apache.org/jira/browse/HBASE-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gustavo Anatoly updated HBASE-11550: Attachment: HBASE-11550-v4.patch Gentlemen, New patch (v4) has been attached. Thanks. > Custom value for BUCKET_CACHE_BUCKETS_KEY should be sorted > -- > > Key: HBASE-11550 > URL: https://issues.apache.org/jira/browse/HBASE-11550 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Gustavo Anatoly > Fix For: 0.99.0, 2.0.0, 0.98.6 > > Attachments: HBASE-11550-v1.patch, HBASE-11550-v2.patch, > HBASE-11550-v3.patch, HBASE-11550-v4.patch, HBASE-11550.patch > > > User can pass bucket sizes through "hbase.bucketcache.bucket.sizes" config > entry. > The sizes are supposed to be in increasing order. Validation should be added > in CacheConfig#getL2(). -- This message was sent by Atlassian JIRA (v6.2#6252)
