[jira] [Commented] (HBASE-6162) Move KeyValue to hbase-common module
[
https://issues.apache.org/jira/browse/HBASE-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294848#comment-13294848
]
Ted Yu commented on HBASE-6162:
---
Patch looks good.
{code}
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
@@ -0,0 +1,2563 @@
+/**
+ * Copyright 2009 The Apache Software Foundation
{code}
Can year be removed from the above file ?
Same for HeapSize.java and ClassSize.java
> Move KeyValue to hbase-common module
>
>
> Key: HBASE-6162
> URL: https://issues.apache.org/jira/browse/HBASE-6162
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.96.0
>Reporter: Matt Corgan
>Assignee: Matt Corgan
> Fix For: 0.96.0
>
> Attachments: HBASE-6162-v1.patch, HBASE-6162-v2.patch,
> HBASE-6162-v3.patch, HBASE-6162-v4.patch
>
>
> * pull KeyValue up to hbase-common module
> This is part of the modularization strategy in HBASE-5977, and is
> specifically necessary to modularize HBASE-4676.
> also brings these classes to hbase-common:
> * ClassSize, HeapSize
> * HTestConst
> * TestKeyValue, KeyValueTestUtil
> * LoadTestKVGenerator, TestLoadTestKVGenerator
> * MD5Hash
> moves a trivial constant (HRegionInfo.DELIMITER) from HRegionInfo to
> HConstants
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6162) Move KeyValue to hbase-common module
[ https://issues.apache.org/jira/browse/HBASE-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Corgan updated HBASE-6162: --- Attachment: HBASE-6162-v5.patch v5 patch removes the apache header years in HeapSize, ClassSize, and KeyValue > Move KeyValue to hbase-common module > > > Key: HBASE-6162 > URL: https://issues.apache.org/jira/browse/HBASE-6162 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.96.0 >Reporter: Matt Corgan >Assignee: Matt Corgan > Fix For: 0.96.0 > > Attachments: HBASE-6162-v1.patch, HBASE-6162-v2.patch, > HBASE-6162-v3.patch, HBASE-6162-v4.patch, HBASE-6162-v5.patch > > > * pull KeyValue up to hbase-common module > This is part of the modularization strategy in HBASE-5977, and is > specifically necessary to modularize HBASE-4676. > also brings these classes to hbase-common: > * ClassSize, HeapSize > * HTestConst > * TestKeyValue, KeyValueTestUtil > * LoadTestKVGenerator, TestLoadTestKVGenerator > * MD5Hash > moves a trivial constant (HRegionInfo.DELIMITER) from HRegionInfo to > HConstants -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time
[ https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294864#comment-13294864 ] chunhui shen commented on HBASE-6205: - bq.how much space is currently consumed by trash We should consume size of each file recursively, is it too expensive? bq.prompt user if he/she is creating a table with the same name as one of the tables in trash I think this feature is used for HBase administrator. It could be invisible to general users. In fact, we need use tool like add_table.rb to recover dropped table. > Support an option to keep data of dropped table for some time > - > > Key: HBASE-6205 > URL: https://issues.apache.org/jira/browse/HBASE-6205 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6205.patch, HBASE-6205v2.patch > > > User may drop table accidentally because of error code or other uncertain > reasons. > Unfortunately, it happens in our environment because one user make a mistake > between production cluster and testing cluster. > So, I just give a suggestion, do we need to support an option to keep data of > dropped table for some time, e.g. 1 day > In the patch: > We make a new dir named .trashtables in the rood dir. > In the DeleteTableHandler, we move files in dropped table's dir to trash > table dir instead of deleting them directly. > And Create new class TrashTableCleaner which will clean dropped tables if it > is time out with a period check. > Default keep time for dropped tables is 1 day, and check period is 1 hour. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6162) Move KeyValue to hbase-common module
[ https://issues.apache.org/jira/browse/HBASE-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294871#comment-13294871 ] Hadoop QA commented on HBASE-6162: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532043/HBASE-6162-v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2168//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2168//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2168//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2168//console This message is automatically generated. > Move KeyValue to hbase-common module > > > Key: HBASE-6162 > URL: https://issues.apache.org/jira/browse/HBASE-6162 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.96.0 >Reporter: Matt Corgan >Assignee: Matt Corgan > Fix For: 0.96.0 > > Attachments: HBASE-6162-v1.patch, HBASE-6162-v2.patch, > HBASE-6162-v3.patch, HBASE-6162-v4.patch, HBASE-6162-v5.patch > > > * pull KeyValue up to hbase-common module > This is part of the modularization strategy in HBASE-5977, and is > specifically necessary to modularize HBASE-4676. > also brings these classes to hbase-common: > * ClassSize, HeapSize > * HTestConst > * TestKeyValue, KeyValueTestUtil > * LoadTestKVGenerator, TestLoadTestKVGenerator > * MD5Hash > moves a trivial constant (HRegionInfo.DELIMITER) from HRegionInfo to > HConstants -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6205) Support an option to keep data of dropped table for some time
[ https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-6205: Attachment: HBASE-6205v3.patch Updating the patch v3 (change the name) > Support an option to keep data of dropped table for some time > - > > Key: HBASE-6205 > URL: https://issues.apache.org/jira/browse/HBASE-6205 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6205.patch, HBASE-6205v2.patch, HBASE-6205v3.patch > > > User may drop table accidentally because of error code or other uncertain > reasons. > Unfortunately, it happens in our environment because one user make a mistake > between production cluster and testing cluster. > So, I just give a suggestion, do we need to support an option to keep data of > dropped table for some time, e.g. 1 day > In the patch: > We make a new dir named .trashtables in the rood dir. > In the DeleteTableHandler, we move files in dropped table's dir to trash > table dir instead of deleting them directly. > And Create new class TrashTableCleaner which will clean dropped tables if it > is time out with a period check. > Default keep time for dropped tables is 1 day, and check period is 1 hour. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6205) Support an option to keep data of dropped table for some time
[ https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-6205: Description: User may drop table accidentally because of error code or other uncertain reasons. Unfortunately, it happens in our environment because one user make a mistake between production cluster and testing cluster. So, I just give a suggestion, do we need to support an option to keep data of dropped table for some time, e.g. 1 day In the patch: We make a new dir named .trashtables in the rood dir. In the DeleteTableHandler, we move files in dropped table's dir to trash table dir instead of deleting them directly. And Create new class TrashCleaner which will clean dropped tables if it is time out with a period check. Default keep time for dropped tables is 1 day, and check period is 1 hour. was: User may drop table accidentally because of error code or other uncertain reasons. Unfortunately, it happens in our environment because one user make a mistake between production cluster and testing cluster. So, I just give a suggestion, do we need to support an option to keep data of dropped table for some time, e.g. 1 day In the patch: We make a new dir named .trashtables in the rood dir. In the DeleteTableHandler, we move files in dropped table's dir to trash table dir instead of deleting them directly. And Create new class TrashTableCleaner which will clean dropped tables if it is time out with a period check. Default keep time for dropped tables is 1 day, and check period is 1 hour. > Support an option to keep data of dropped table for some time > - > > Key: HBASE-6205 > URL: https://issues.apache.org/jira/browse/HBASE-6205 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6205.patch, HBASE-6205v2.patch, HBASE-6205v3.patch > > > User may drop table accidentally because of error code or other uncertain > reasons. > Unfortunately, it happens in our environment because one user make a mistake > between production cluster and testing cluster. > So, I just give a suggestion, do we need to support an option to keep data of > dropped table for some time, e.g. 1 day > In the patch: > We make a new dir named .trashtables in the rood dir. > In the DeleteTableHandler, we move files in dropped table's dir to trash > table dir instead of deleting them directly. > And Create new class TrashCleaner which will clean dropped tables if it is > time out with a period check. > Default keep time for dropped tables is 1 day, and check period is 1 hour. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6197) HRegion's append operation may lose data
[ https://issues.apache.org/jira/browse/HBASE-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294873#comment-13294873 ] Hudson commented on HBASE-6197: --- Integrated in HBase-TRUNK #3022 (See [https://builds.apache.org/job/HBase-TRUNK/3022/]) HBASE-6197 HRegion's append operation may lose data (Xing Shi) Submitted by: Xing Shi Reviewed by:Ted Yu (Revision 1350099) Result = SUCCESS tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java > HRegion's append operation may lose data > > > Key: HBASE-6197 > URL: https://issues.apache.org/jira/browse/HBASE-6197 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Xing Shi >Assignee: ShiXing > Attachments: HBASE-6197-trunk-V1.patch > > > Like the HBASE-6195, when flushing the append thread will read out the old > value for the larger timestamp in snapshot and smaller timestamp in memstore. > We Should make the first-in-thread generates the smaller timestamp. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time
[ https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294901#comment-13294901 ] Hadoop QA commented on HBASE-6205: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532048/HBASE-6205v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2170//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2170//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2170//console This message is automatically generated. > Support an option to keep data of dropped table for some time > - > > Key: HBASE-6205 > URL: https://issues.apache.org/jira/browse/HBASE-6205 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6205.patch, HBASE-6205v2.patch, HBASE-6205v3.patch > > > User may drop table accidentally because of error code or other uncertain > reasons. > Unfortunately, it happens in our environment because one user make a mistake > between production cluster and testing cluster. > So, I just give a suggestion, do we need to support an option to keep data of > dropped table for some time, e.g. 1 day > In the patch: > We make a new dir named .trashtables in the rood dir. > In the DeleteTableHandler, we move files in dropped table's dir to trash > table dir instead of deleting them directly. > And Create new class TrashCleaner which will clean dropped tables if it is > time out with a period check. > Default keep time for dropped tables is 1 day, and check period is 1 hour. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6208) hbase use several secs to writting hlog
zhangyong created HBASE-6208: Summary: hbase use several secs to writting hlog Key: HBASE-6208 URL: https://issues.apache.org/jira/browse/HBASE-6208 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.90.4 Reporter: zhangyong 2012-06-14 15:06:54,902 WARN org.apache.hadoop.hbase.regionserver.wal.HLog: IPC Server handler 188 on 60020 took 8872 ms appending an edit to hlog; editcount=15730, len~=2.2k we have many this log , and the write request will be blocked . i find no solution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4487) The increment operation can release the rowlock before sync-ing the Hlog
[
https://issues.apache.org/jira/browse/HBASE-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294912#comment-13294912
]
dhruba borthakur commented on HBASE-4487:
-
If the hlog sync failed, you can invoke
{code}
// if the wal sync was unsuccessful, remove keys from memstore
if (!walSyncSuccessful) {
rollbackMemstore(batchOp, familyMaps, firstIndex, lastIndexExclusive);
}
{code}
Similar code exists inside doMiniBatchPut
> The increment operation can release the rowlock before sync-ing the Hlog
>
>
> Key: HBASE-4487
> URL: https://issues.apache.org/jira/browse/HBASE-4487
> Project: HBase
> Issue Type: Improvement
>Reporter: dhruba borthakur
>Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4487-v7.txt, appendNoSync4.txt, appendNoSync5.txt,
> appendNoSync6.txt
>
>
> This allows for better throughput when there are hot rows.I have seen this
> change make a single row update improve from 400 increments/sec/server to
> 4000 increments/sec/server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6192) Document ACL matrix in the book
[ https://issues.apache.org/jira/browse/HBASE-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294917#comment-13294917 ] Laxman commented on HBASE-6192: --- This matrix does not cover AccessControllerProtocol interface. I will add that too as this interface can be directly used by user. Will post the updated matrix. Please give your opinion. > Document ACL matrix in the book > --- > > Key: HBASE-6192 > URL: https://issues.apache.org/jira/browse/HBASE-6192 > Project: HBase > Issue Type: Sub-task > Components: documentation, security >Affects Versions: 0.96.0, 0.94.1 >Reporter: Enis Soztutar >Assignee: Laxman > Labels: documentaion, security > Attachments: HBase Security-ACL Matrix.pdf, HBase Security-ACL > Matrix.xls > > > We have an excellent matrix at > https://issues.apache.org/jira/secure/attachment/12531252/Security-ACL%20Matrix.pdf > for ACL. Once the changes are done, we can adapt that and put it in the > book, also add some more documentation about the new authorization features. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6209) ACL Corrections corrections for AccessControllerProtocol apis
Laxman created HBASE-6209: - Summary: ACL Corrections corrections for AccessControllerProtocol apis Key: HBASE-6209 URL: https://issues.apache.org/jira/browse/HBASE-6209 Project: HBase Issue Type: Sub-task Components: security Affects Versions: 0.94.0, 0.96.0, 0.94.1 Reporter: Laxman Assignee: Laxman Fix For: 0.96.0, 0.94.1 APIs provided in AccessController are authorized against global-admin permissions. Instead we need to check for table-admin level permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6162) Move KeyValue to hbase-common module
[ https://issues.apache.org/jira/browse/HBASE-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294918#comment-13294918 ] Hadoop QA commented on HBASE-6162: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532046/HBASE-6162-v5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2169//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2169//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2169//console This message is automatically generated. > Move KeyValue to hbase-common module > > > Key: HBASE-6162 > URL: https://issues.apache.org/jira/browse/HBASE-6162 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.96.0 >Reporter: Matt Corgan >Assignee: Matt Corgan > Fix For: 0.96.0 > > Attachments: HBASE-6162-v1.patch, HBASE-6162-v2.patch, > HBASE-6162-v3.patch, HBASE-6162-v4.patch, HBASE-6162-v5.patch > > > * pull KeyValue up to hbase-common module > This is part of the modularization strategy in HBASE-5977, and is > specifically necessary to modularize HBASE-4676. > also brings these classes to hbase-common: > * ClassSize, HeapSize > * HTestConst > * TestKeyValue, KeyValueTestUtil > * LoadTestKVGenerator, TestLoadTestKVGenerator > * MD5Hash > moves a trivial constant (HRegionInfo.DELIMITER) from HRegionInfo to > HConstants -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5924) In the client code, don't wait for all the requests to be executed before resubmitting a request in error.
[ https://issues.apache.org/jira/browse/HBASE-5924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294928#comment-13294928 ] Hudson commented on HBASE-5924: --- Integrated in HBase-TRUNK #3023 (See [https://builds.apache.org/job/HBase-TRUNK/3023/]) HBASE-5924 In the client code, don't wait for all the requests to be executed before resubmitting a request in error (N Keywal) Submitted by: N Keywal Reviewed by:Stack, Ted (Revision 1350105) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HConnection.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/Triple.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithRemove.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java > In the client code, don't wait for all the requests to be executed before > resubmitting a request in error. > -- > > Key: HBASE-5924 > URL: https://issues.apache.org/jira/browse/HBASE-5924 > Project: HBase > Issue Type: Improvement > Components: client >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 5924.v11.patch, 5924.v14.patch, 5924.v19.patch, > 5924.v5.patch, 5924.v9.patch > > > The client (in the function HConnectionManager#processBatchCallback) works in > two steps: > - make the requests > - collect the failures and successes and prepare for retry > It means that when there is an immediate error (region moved, split, dead > server, ...) we still wait for all the initial requests to be executed before > submitting again the failed request. If we have a scenario with all the > requests taking 5 seconds we have a final execution time of: 5 (initial > requests) + 1 (wait time) + 5 (final request) = 11s. > We could improve this by analyzing immediately the results. This would lead > us, for the scenario mentioned above, to 6 seconds. > So we could have a performance improvement of nearly 50% in many cases, and > much more than 50% if the request execution time is different. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6188) Remove the concept of table owner
[ https://issues.apache.org/jira/browse/HBASE-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294943#comment-13294943 ] Laxman commented on HBASE-6188: --- All tests were passing in QA bot build. https://builds.apache.org/job/PreCommit-HBASE-Build/2164//testReport/org.apache.hadoop.hbase.security.access/TestAccessController/ Please review. > Remove the concept of table owner > - > > Key: HBASE-6188 > URL: https://issues.apache.org/jira/browse/HBASE-6188 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0, 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Laxman > Labels: security > Fix For: 0.96.0, 0.94.1 > > Attachments: HBASE-6188.1.patch, HBASE-6188.2.patch, HBASE-6188.patch > > > The table owner concept was a design simplification in the initial drop. > First, the design changes under review means only a user with GLOBAL CREATE > permission can create a table, which will probably be an administrator. > Then, granting implicit permissions may lead to oversights and it adds > unnecessary conditionals to our code. So instead the administrator with > GLOBAL CREATE permission should make the appropriate grants at table create > time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5372) Table mutation operations should check table level rights, not global rights
[
https://issues.apache.org/jira/browse/HBASE-5372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laxman updated HBASE-5372:
--
Description: drop/modify/disable/enable etc table operations should not
check for global CREATE/ADMIN rights, but table CREATE/ADMIN rights. Since we
check for global permissions first for table permissions, configuring table
access using global permissions will continue to work. (was:
getUserPermissions(tableName)/grant/revoke and drop/modify table operations
should not check for global CREATE/ADMIN rights, but table CREATE/ADMIN rights.
The reasoning is that if a user is able to admin or read from a table, she
should be able to read the table's permissions. We can choose whether we want
only READ or ADMIN permissions for getUserPermission(). Since we check for
global permissions first for table permissions, configuring table access using
global permissions will continue to work. )
{quote}
getUserPermissions(tableName)/grant/revoke
The reasoning is that if a user is able to admin or read from a table, she
should be able to read the table's permissions. We can choose whether we want
only READ or ADMIN permissions for getUserPermission().
{quote}
ACL corrections for AccessController protocol apis will be addressed in
HBASE-6209.
> Table mutation operations should check table level rights, not global rights
> -
>
> Key: HBASE-5372
> URL: https://issues.apache.org/jira/browse/HBASE-5372
> Project: HBase
> Issue Type: Sub-task
> Components: security
>Affects Versions: 0.94.0, 0.96.0, 0.94.1
>Reporter: Enis Soztutar
>Assignee: Laxman
> Labels: security
> Fix For: 0.96.0, 0.94.1
>
> Attachments: HBASE-5372.patch
>
>
> drop/modify/disable/enable etc table operations should not check for global
> CREATE/ADMIN rights, but table CREATE/ADMIN rights. Since we check for global
> permissions first for table permissions, configuring table access using
> global permissions will continue to work.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6209) ACL Corrections for AccessControllerProtocol apis
[ https://issues.apache.org/jira/browse/HBASE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laxman updated HBASE-6209: -- Labels: acl security (was: security) Summary: ACL Corrections for AccessControllerProtocol apis (was: ACL Corrections corrections for AccessControllerProtocol apis) > ACL Corrections for AccessControllerProtocol apis > - > > Key: HBASE-6209 > URL: https://issues.apache.org/jira/browse/HBASE-6209 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0, 0.96.0, 0.94.1 >Reporter: Laxman >Assignee: Laxman > Labels: acl, security > Fix For: 0.96.0, 0.94.1 > > > APIs provided in AccessController are authorized against global-admin > permissions. Instead we need to check for table-admin level permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6197) HRegion's append operation may lose data
[ https://issues.apache.org/jira/browse/HBASE-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294976#comment-13294976 ] Hudson commented on HBASE-6197: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #53 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/53/]) HBASE-6197 HRegion's append operation may lose data (Xing Shi) Submitted by: Xing Shi Reviewed by:Ted Yu (Revision 1350099) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java > HRegion's append operation may lose data > > > Key: HBASE-6197 > URL: https://issues.apache.org/jira/browse/HBASE-6197 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Xing Shi >Assignee: ShiXing > Attachments: HBASE-6197-trunk-V1.patch > > > Like the HBASE-6195, when flushing the append thread will read out the old > value for the larger timestamp in snapshot and smaller timestamp in memstore. > We Should make the first-in-thread generates the smaller timestamp. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5924) In the client code, don't wait for all the requests to be executed before resubmitting a request in error.
[ https://issues.apache.org/jira/browse/HBASE-5924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294977#comment-13294977 ] Hudson commented on HBASE-5924: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #53 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/53/]) HBASE-5924 In the client code, don't wait for all the requests to be executed before resubmitting a request in error (N Keywal) Submitted by: N Keywal Reviewed by:Stack, Ted (Revision 1350105) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HConnection.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/Triple.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithRemove.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java > In the client code, don't wait for all the requests to be executed before > resubmitting a request in error. > -- > > Key: HBASE-5924 > URL: https://issues.apache.org/jira/browse/HBASE-5924 > Project: HBase > Issue Type: Improvement > Components: client >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 5924.v11.patch, 5924.v14.patch, 5924.v19.patch, > 5924.v5.patch, 5924.v9.patch > > > The client (in the function HConnectionManager#processBatchCallback) works in > two steps: > - make the requests > - collect the failures and successes and prepare for retry > It means that when there is an immediate error (region moved, split, dead > server, ...) we still wait for all the initial requests to be executed before > submitting again the failed request. If we have a scenario with all the > requests taking 5 seconds we have a final execution time of: 5 (initial > requests) + 1 (wait time) + 5 (final request) = 11s. > We could improve this by analyzing immediately the results. This would lead > us, for the scenario mentioned above, to 6 seconds. > So we could have a performance improvement of nearly 50% in many cases, and > much more than 50% if the request execution time is different. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6207) Add jitter to client retry timer
[ https://issues.apache.org/jira/browse/HBASE-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295034#comment-13295034 ] Nicolas Spiegelberg commented on HBASE-6207: jitter + exponential backoff? > Add jitter to client retry timer > > > Key: HBASE-6207 > URL: https://issues.apache.org/jira/browse/HBASE-6207 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time
[ https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295036#comment-13295036 ] Ted Yu commented on HBASE-6205: --- bq. prompt user if he/she is creating a table with the same name as one of the tables in trash I think the above is needed, especially if the table being created has different schema from the deleted table (which had the same name). > Support an option to keep data of dropped table for some time > - > > Key: HBASE-6205 > URL: https://issues.apache.org/jira/browse/HBASE-6205 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6205.patch, HBASE-6205v2.patch, HBASE-6205v3.patch > > > User may drop table accidentally because of error code or other uncertain > reasons. > Unfortunately, it happens in our environment because one user make a mistake > between production cluster and testing cluster. > So, I just give a suggestion, do we need to support an option to keep data of > dropped table for some time, e.g. 1 day > In the patch: > We make a new dir named .trashtables in the rood dir. > In the DeleteTableHandler, we move files in dropped table's dir to trash > table dir instead of deleting them directly. > And Create new class TrashCleaner which will clean dropped tables if it is > time out with a period check. > Default keep time for dropped tables is 1 day, and check period is 1 hour. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time
[ https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295198#comment-13295198 ] stack commented on HBASE-6205: -- @Chunhui Why a special hbase trash dir? Why not just use the trash facility of hdfs? It already has configurable retention period? > Support an option to keep data of dropped table for some time > - > > Key: HBASE-6205 > URL: https://issues.apache.org/jira/browse/HBASE-6205 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6205.patch, HBASE-6205v2.patch, HBASE-6205v3.patch > > > User may drop table accidentally because of error code or other uncertain > reasons. > Unfortunately, it happens in our environment because one user make a mistake > between production cluster and testing cluster. > So, I just give a suggestion, do we need to support an option to keep data of > dropped table for some time, e.g. 1 day > In the patch: > We make a new dir named .trashtables in the rood dir. > In the DeleteTableHandler, we move files in dropped table's dir to trash > table dir instead of deleting them directly. > And Create new class TrashCleaner which will clean dropped tables if it is > time out with a period check. > Default keep time for dropped tables is 1 day, and check period is 1 hour. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5924) In the client code, don't wait for all the requests to be executed before resubmitting a request in error.
[ https://issues.apache.org/jira/browse/HBASE-5924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295204#comment-13295204 ] stack commented on HBASE-5924: -- I took a look at committed patch. This looks great. > In the client code, don't wait for all the requests to be executed before > resubmitting a request in error. > -- > > Key: HBASE-5924 > URL: https://issues.apache.org/jira/browse/HBASE-5924 > Project: HBase > Issue Type: Improvement > Components: client >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 5924.v11.patch, 5924.v14.patch, 5924.v19.patch, > 5924.v5.patch, 5924.v9.patch > > > The client (in the function HConnectionManager#processBatchCallback) works in > two steps: > - make the requests > - collect the failures and successes and prepare for retry > It means that when there is an immediate error (region moved, split, dead > server, ...) we still wait for all the initial requests to be executed before > submitting again the failed request. If we have a scenario with all the > requests taking 5 seconds we have a final execution time of: 5 (initial > requests) + 1 (wait time) + 5 (final request) = 11s. > We could improve this by analyzing immediately the results. This would lead > us, for the scenario mentioned above, to 6 seconds. > So we could have a performance improvement of nearly 50% in many cases, and > much more than 50% if the request execution time is different. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6208) hbase use several secs to writting hlog
[ https://issues.apache.org/jira/browse/HBASE-6208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans resolved HBASE-6208. --- Resolution: Not A Problem Please write to the user mailing list: http://hbase.apache.org/mail-lists.html > hbase use several secs to writting hlog > --- > > Key: HBASE-6208 > URL: https://issues.apache.org/jira/browse/HBASE-6208 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.90.4 >Reporter: zhangyong > > 2012-06-14 15:06:54,902 WARN org.apache.hadoop.hbase.regionserver.wal.HLog: > IPC Server handler 188 on 60020 took 8872 ms appending an edit to hlog; > editcount=15730, len~=2.2k > we have many this log , and the write request will be blocked . i find no > solution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6195) Increment data will be lost when the memstore is flushed
[
https://issues.apache.org/jira/browse/HBASE-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295241#comment-13295241
]
stack commented on HBASE-6195:
--
Xing Nice work. When you run MultiThreadsIncrement w/ your patch in place, do
the numbers come out better?
It looks like this increment was originally added by:
{code}
r1027681 | jgray | 2010-10-26 11:50:13 -0700 (Tue, 26 Oct 2010) | 1 line
HBASE-2946 Increment multiple columns in a row at once
{code}
There is some discussion there that there may be danger around flush time.
Might be useful to review. See
https://issues.apache.org/jira/browse/HBASE-2946?focusedCommentId=12924737&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12924737
In original patch, it has our updating MemStore before we append to WAL. I
seem to recall that there was a reason why we had this unorthodoxy but I am
unable to find it now.
Thanks for fixing this Xing.
We should backport?
> Increment data will be lost when the memstore is flushed
>
>
> Key: HBASE-6195
> URL: https://issues.apache.org/jira/browse/HBASE-6195
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Xing Shi
>Assignee: ShiXing
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6195-trunk-V7.patch, 6195.addendum,
> HBASE-6195-trunk-V2.patch, HBASE-6195-trunk-V3.patch,
> HBASE-6195-trunk-V4.patch, HBASE-6195-trunk-V5.patch,
> HBASE-6195-trunk-V6.patch, HBASE-6195-trunk.patch
>
>
> There are two problems in increment() now:
> First:
> I see that the timestamp(the variable now) in HRegion's Increment() is
> generated before got the rowLock, so when there are multi-thread increment
> the same row, although it generate earlier, it may got the lock later.
> Because increment just store one version, so till now, the result will still
> be right.
> When the region is flushing, these increment will read the kv from snapshot
> and memstore with whose timestamp is larger, and write it back to memstore.
> If the snapshot's timestamp larger than the memstore, the increment will got
> the old data and then do the increment, it's wrong.
> Secondly:
> Also there is a risk in increment. Because it writes the memstore first and
> then HLog, so if it writes HLog failed, the client will also read the
> incremented value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6162) Move KeyValue to hbase-common module
[ https://issues.apache.org/jira/browse/HBASE-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6162: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Matt. I wasn't mad about the machinations where we try to fake the push test dependencies into src/main. I was thinking that rather we should refactor the unit test so its module scoped? (At least for the stuff that doesn't need minihbasecluster, etc.) > Move KeyValue to hbase-common module > > > Key: HBASE-6162 > URL: https://issues.apache.org/jira/browse/HBASE-6162 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.96.0 >Reporter: Matt Corgan >Assignee: Matt Corgan > Fix For: 0.96.0 > > Attachments: HBASE-6162-v1.patch, HBASE-6162-v2.patch, > HBASE-6162-v3.patch, HBASE-6162-v4.patch, HBASE-6162-v5.patch > > > * pull KeyValue up to hbase-common module > This is part of the modularization strategy in HBASE-5977, and is > specifically necessary to modularize HBASE-4676. > also brings these classes to hbase-common: > * ClassSize, HeapSize > * HTestConst > * TestKeyValue, KeyValueTestUtil > * LoadTestKVGenerator, TestLoadTestKVGenerator > * MD5Hash > moves a trivial constant (HRegionInfo.DELIMITER) from HRegionInfo to > HConstants -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6192) Document ACL matrix in the book
[ https://issues.apache.org/jira/browse/HBASE-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295259#comment-13295259 ] stack commented on HBASE-6192: -- @Laxman I'll take care of convertion to our doc format. Just say when you want me to do it. I'd add it under here: http://hbase.apache.org/book.html#hbase.accesscontrol.configuration ? > Document ACL matrix in the book > --- > > Key: HBASE-6192 > URL: https://issues.apache.org/jira/browse/HBASE-6192 > Project: HBase > Issue Type: Sub-task > Components: documentation, security >Affects Versions: 0.96.0, 0.94.1 >Reporter: Enis Soztutar >Assignee: Laxman > Labels: documentaion, security > Attachments: HBase Security-ACL Matrix.pdf, HBase Security-ACL > Matrix.xls > > > We have an excellent matrix at > https://issues.apache.org/jira/secure/attachment/12531252/Security-ACL%20Matrix.pdf > for ACL. Once the changes are done, we can adapt that and put it in the > book, also add some more documentation about the new authorization features. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295267#comment-13295267 ] stack commented on HBASE-3909: -- bq. My understanding is that all configuration properties will be/must be defined in hbase-default.xml with a default value. Some exceptions are possible in some corner cases but majority of properties must be defined here. IMO, on #1 in your first of two comments above Subbu, there are configs. that are do not have to be in hbase-default. Such configs. are those we think we'd never change but that we've made configurable just-in-case. This class of config can only be found by reading source code. We don't add them to hbase-default.xml because they detract from the important configs. in the flat hbase-default xml file (Maybe this is less of an issue now we 'hide' hbase-default.xml inside the jar and the book includes description). If we do add these to hbase-default.xml, I'd think we'd add an annotation saying like 'exotic' or 'not-for-the-faint-of-heart'). On #2, I think its fine that they have to read the source for esoterics. On your second set of comments, I think yes, this is an important feature to have and I agree many configs. will not be changeable without extensive refactoring server-side (and even then, some may be just plain dangerous to change in mid-flight). Perhaps we do a subset in this case and if you stray outside of the subset, you get an exception. > Add dynamic config > -- > > Key: HBASE-3909 > URL: https://issues.apache.org/jira/browse/HBASE-3909 > Project: HBase > Issue Type: Bug >Reporter: stack > Fix For: 0.96.0 > > Attachments: 3909-v1.patch, 3909.v1, HBase Cluster Config > Details.xlsx, patch-v2.patch > > > I'm sure this issue exists already, at least as part of the discussion around > making online schema edits possible, but no hard this having its own issue. > Ted started a conversation on this topic up on dev and Todd suggested we > lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295271#comment-13295271 ] stack commented on HBASE-5998: -- I took a look at backporting at Anoop's request. Its not a clean backport. Mind taking a look Nkeywal? > Bulk assignment: regionserver optimization by using a temporary cache for > table descriptors when receveing an open regions request > -- > > Key: HBASE-5998 > URL: https://issues.apache.org/jira/browse/HBASE-5998 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5998.v2.patch, 5998.v3.patch > > > During the assignment, on the regionserver, before creating the handlers we > load the table description. Even if there is a cache, we check the timestamps > for each region, while it's not necessary. The test below is just with one > node, with more nodes the benefit will improve. By limiting the time spent in > HRegion#openRegion we increase the parallelization during cluster startup, as > the master is using a pool of threads to call the RS. > -- Without the fix > 2012-05-14 11:40:52,501 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 > region(s) to localhost,11003,1336988444043 > 2012-05-14 11:41:09,947 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for > localhost,11003,1336988444043 > -- With the fix > 2012-05-14 11:34:40,444 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 > region(s) to localhost,11003,1336988444043 > 2012-05-14 11:34:40,929 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for > localhost,11003,1336988065948 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6202) Medium tests fail with jdk1.7
[ https://issues.apache.org/jira/browse/HBASE-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295272#comment-13295272 ] stack commented on HBASE-6202: -- Changes seem reasonable Jimmy. Why you think they are needed when jdk1.7? > Medium tests fail with jdk1.7 > - > > Key: HBASE-6202 > URL: https://issues.apache.org/jira/browse/HBASE-6202 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Attachments: 6202-hbase.patch > > > Failed tests: > testOrphanTaskAcquisition(org.apache.hadoop.hbase.master.TestSplitLogManager) > testCreateAndUpdate(org.apache.hadoop.hbase.util.TestFSTableDescriptors): > statuses.length=5 > testManageSingleton(org.apache.hadoop.hbase.util.TestEnvironmentEdgeManager) > Tests in error: > > testRegionTransitionOperations(org.apache.hadoop.hbase.coprocessor.TestMasterObserver): > org.apache.hadoop.hbase.UnknownRegionException: > ef07dae0851f4fbe7d550413530b8774 > > testTableOperations(org.apache.hadoop.hbase.coprocessor.TestMasterObserver): > org.apache.hadoop.hbase.TableExistsException: observed_table > > testClassLoadingFromHDFS(org.apache.hadoop.hbase.coprocessor.TestClassLoading): > org.apache.hadoop.hbase.TableNotEnabledException: TestClassLoading -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4391) Add ability to start RS as root and call mlockall
[ https://issues.apache.org/jira/browse/HBASE-4391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295276#comment-13295276 ] stack commented on HBASE-4391: -- @Matteo What about Andrew's suggested location for the src and native bits? Whats it like when we add compile of mlockall to our compile goal in mvn? Is it fast? On this question: bq. If this will be integrated in hbase, do you want to force everyone to use root or add hbase user in limits.d to start RS? or we can add some --no-swap flag to region server start script? We cannot require you be root to start hbase. It looks like this not necessary anyways? If you want the mlockall facility, then you can require folks be root (or do you security.conf trick). The Todd trick is nice as is your tidy up. Thanks. > Add ability to start RS as root and call mlockall > - > > Key: HBASE-4391 > URL: https://issues.apache.org/jira/browse/HBASE-4391 > Project: HBase > Issue Type: New Feature > Components: regionserver >Affects Versions: 0.94.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Fix For: 0.96.0 > > Attachments: HBASE-4391-v0.patch > > > A common issue we've seen in practice is that users oversubscribe their > region servers with too many MR tasks, etc. As soon as the machine starts > swapping, the RS grinds to a halt, loses ZK session, aborts, etc. > This can be combatted by starting the RS as root, calling mlockall(), and > then setuid down to the hbase user. We should not require this, but we should > provide it as an option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6197) HRegion's append operation may lose data
[ https://issues.apache.org/jira/browse/HBASE-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6197: - Resolution: Fixed Fix Version/s: 0.96.0 Status: Resolved (was: Patch Available) Ted committed this. Backport? > HRegion's append operation may lose data > > > Key: HBASE-6197 > URL: https://issues.apache.org/jira/browse/HBASE-6197 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Xing Shi >Assignee: ShiXing > Fix For: 0.96.0 > > Attachments: HBASE-6197-trunk-V1.patch > > > Like the HBASE-6195, when flushing the append thread will read out the old > value for the larger timestamp in snapshot and smaller timestamp in memstore. > We Should make the first-in-thread generates the smaller timestamp. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6210) Backport HBASE-6197 to 0.94 and 0.92?
stack created HBASE-6210: Summary: Backport HBASE-6197 to 0.94 and 0.92? Key: HBASE-6210 URL: https://issues.apache.org/jira/browse/HBASE-6210 Project: HBase Issue Type: Bug Reporter: stack Backport HBASE-6197 'HRegion's append operation may lose data' and the accompanying HBASE-6195 to 0.94 and 0.92 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6197) HRegion's append operation may lose data
[ https://issues.apache.org/jira/browse/HBASE-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295297#comment-13295297 ] stack commented on HBASE-6197: -- I made HBASE-6210 for backport. > HRegion's append operation may lose data > > > Key: HBASE-6197 > URL: https://issues.apache.org/jira/browse/HBASE-6197 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Xing Shi >Assignee: ShiXing > Fix For: 0.96.0 > > Attachments: HBASE-6197-trunk-V1.patch > > > Like the HBASE-6195, when flushing the append thread will read out the old > value for the larger timestamp in snapshot and smaller timestamp in memstore. > We Should make the first-in-thread generates the smaller timestamp. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6185) Update javadoc for ConstantSizeRegionSplitPolicy class
[
https://issues.apache.org/jira/browse/HBASE-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6185:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to trunk and 0.94
> Update javadoc for ConstantSizeRegionSplitPolicy class
> --
>
> Key: HBASE-6185
> URL: https://issues.apache.org/jira/browse/HBASE-6185
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.94.0
>Reporter: nneverwei
> Fix For: 0.94.1
>
> Attachments: HBASE-6185.patch, HBASE-6185.v2.patch,
> HBASE-6185.v3.patch
>
>
> When using hbase0.94.0 we met a strange problem.
> We config the 'hbase.hregion.max.filesize' to 100Gb (The recommed value to
> act as auto-split turn off).
> {code:xml}
>
> hbase.hregion.max.filesize
> 107374182400
>
> {code}
> Then we keep putting datas into a table.
> But when the data size far more less than 100Gb(about 500~600 uncompressed
> datas), the table auto splte to 2 regions...
> I change the log4j config to DEBUG, and saw logs below:
> {code}
> 2012-06-07 10:30:52,161 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~128.0m/134221272, currentsize=1.5m/1617744 for
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. in
> 3201ms, sequenceid=176387980, compaction requested=false
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,240 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Split requested for
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8..
> compaction_queue=(0:0), split_queue=0
> 2012-06-07 10:30:52,265 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.
> 2012-06-07 10:30:52,265 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:60020-0x137c4929efe0001 Creating ephemeral node for
> 7b229abcd0785408251a579e9bdf49c8 in SPLITTING state
> 2012-06-07 10:30:52,368 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Attempting to transition node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,382 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Successfully transitioned node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,410 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Closing FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.:
> disabling compactions & flushes
> 2012-06-07 10:30:52,410 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> 2012-06-07 10:30:52,411 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> {code}
> {color:red}IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because info
> size=138657416, sizeToCheck=134217728{color}
> I did not config splitPolicy for hbase, so it means
> *IncreasingToUpperBoundRegionSplitPolicy is the default splitPolicy of 0.94.0*
> After add
> {code:xml}
>
> hbase.regionserver.region.split.policy
>
> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy
>
> {code}
> autosplit did not happen again and everything goes well.
> But we can still see javadoc on ConstantSizeRegionSplitPolicy, it says 'This
> is the default split policy'. Or even in the
> http://hbase.apache.org/book/regions.arch.html 9.7.4.1. Custom Split
> Policies, 'default split policy: ConstantSizeRegionSplitPolicy.'.
> Those may mistaken us that if we set hbase.hregion.max.filesize to 100Gb,
> than the auto-split can be almost shutdown.
> You may change those docs, and What more, in many scenerys, we actually need
> to control split manually（As you know when spliting the table are offline,
> reads and writes will fail）
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.co
[jira] [Commented] (HBASE-6162) Move KeyValue to hbase-common module
[ https://issues.apache.org/jira/browse/HBASE-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295308#comment-13295308 ] Hudson commented on HBASE-6162: --- Integrated in HBase-TRUNK #3025 (See [https://builds.apache.org/job/HBase-TRUNK/3025/]) HBASE-6162 Move KeyValue to hbase-common module (Revision 1350378) Result = SUCCESS stack : Files : * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/io/HeapSize.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HeapSize.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/GetClosestRowBeforeTracker.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java > Move KeyValue to hbase-common module > > > Key: HBASE-6162 > URL: https://issues.apache.org/jira/browse/HBASE-6162 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.96.0 >Reporter: Matt Corgan >Assignee: Matt Corgan > Fix For: 0.96.0 > > Attachments: HBASE-6162-v1.patch, HBASE-6162-v2.patch, > HBASE-6162-v3.patch, HBASE-6162-v4.patch, HBASE-6162-v5.patch > > > * pull KeyValue up to hbase-common module > This is part of the modularization strategy in HBASE-5977, and is > specifically necessary to modularize HBASE-4676. > also brings these classes to hbase-common: > * ClassSize, HeapSize > * HTestConst > * TestKeyValue, KeyValueTestUtil > * LoadTestKVGenerator, TestLoadTestKVGenerator > * MD5Hash > moves a trivial constant (HRegionInfo.DELIMITER) from HRegionInfo to > HConstants -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6060) Regions's in OPENING state from failed regionservers takes a long time to recover
[ https://issues.apache.org/jira/browse/HBASE-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295311#comment-13295311 ] stack commented on HBASE-6060: -- bq. But i agree two systems AM and SSH take decision based on that.. Sorry, I don't follow? Are you saying we should rely on RegionState? Or on RegionPlan? (Plan is what we 'want' to happen; RegionState should be where regions actually 'are'. I'd think that we'd want to deal w/ current state making a decision rather than planned state?) bq. One more thing on RegionState in RIT is that it is reactiveEvery time Assignment starts RegionState in RIT goes thro a set of steps and may be that is why we are not sure as in what step the RIT is and who made that change. In SSH, we need the answer to one basic question only; i.e. who owns the region, master or regionserver. Unless you have a better idea, I think setting of znode to OPENING before returning from the open rpc necessary to plug the gray area you fellas identified. bq. We also tried thought of one approach, like can we remove the retry logic itself in assign? It seems like the retry has been there a long time (Originally we just recursed calling assign on exception but in HBASE-3263 we changed it to a bounded loop). It seems like the retry is ok since we'll try a different server if we fail on first plan? We also like this single-assign method because it more rigorous about state management than bulk-assign? Its only a problem if a concurrent assign when we are unsure who is responsible for the region. > Regions's in OPENING state from failed regionservers takes a long time to > recover > - > > Key: HBASE-6060 > URL: https://issues.apache.org/jira/browse/HBASE-6060 > Project: HBase > Issue Type: Bug > Components: master, regionserver >Reporter: Enis Soztutar >Assignee: rajeshbabu > Fix For: 0.96.0, 0.94.1, 0.92.3 > > Attachments: 6060-94-v3.patch, 6060-94-v4.patch, 6060-94-v4_1.patch, > 6060-94-v4_1.patch, 6060-trunk.patch, 6060-trunk.patch, 6060-trunk_2.patch, > 6060-trunk_3.patch, 6060_alternative_suggestion.txt, > 6060_suggestion2_based_off_v3.patch, 6060_suggestion_based_off_v3.patch, > 6060_suggestion_toassign_rs_wentdown_beforerequest.patch, > HBASE-6060-92.patch, HBASE-6060-94.patch, HBASE-6060-trunk_4.patch, > HBASE-6060_trunk_5.patch > > > we have seen a pattern in tests, that the regions are stuck in OPENING state > for a very long time when the region server who is opening the region fails. > My understanding of the process: > > - master calls rs to open the region. If rs is offline, a new plan is > generated (a new rs is chosen). RegionState is set to PENDING_OPEN (only in > master memory, zk still shows OFFLINE). See HRegionServer.openRegion(), > HMaster.assign() > - RegionServer, starts opening a region, changes the state in znode. But > that znode is not ephemeral. (see ZkAssign) > - Rs transitions zk node from OFFLINE to OPENING. See > OpenRegionHandler.process() > - rs then opens the region, and changes znode from OPENING to OPENED > - when rs is killed between OPENING and OPENED states, then zk shows OPENING > state, and the master just waits for rs to change the region state, but since > rs is down, that wont happen. > - There is a AssignmentManager.TimeoutMonitor, which does exactly guard > against these kind of conditions. It periodically checks (every 10 sec by > default) the regions in transition to see whether they timedout > (hbase.master.assignment.timeoutmonitor.timeout). Default timeout is 30 min, > which explains what you and I are seeing. > - ServerShutdownHandler in Master does not reassign regions in OPENING > state, although it handles other states. > Lowering that threshold from the configuration is one option, but still I > think we can do better. > Will investigate more. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[
https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295318#comment-13295318
]
Eugene Koontz commented on HBASE-6135:
--
Hi Elliott, your patch works great for me and looks very attractive in the
browser.
I had the same problem with the hadoop_logo.png.
Also, what's the most efficient workflow for developing with jamon? I've been
doing the following:
1. edit a jamon file (e.g.
hbase/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/)
2. do the following: (using a shell script):
{code}
rm
hbase-server/target/classes/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.class
\
hbase-server/target/generated-jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.java
mvn compile
bin/hbase-daemon.sh restart master
{code}
> Style the Web UI to use Twitter's Bootstrap.
>
>
> Key: HBASE-6135
> URL: https://issues.apache.org/jira/browse/HBASE-6135
> Project: HBase
> Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 0.96.0
>
> Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch,
> master-redesign.png, redesign-master-r1.png, redesign-rs-r1.png
>
>
> Our web ui has lagged a little bit behind. While it's not a huge deal, it is
> one of the first things that new people see. As such styling it a little bit
> better would put a good foot forward.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[ https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295323#comment-13295323 ] stack commented on HBASE-6135: -- Ok if I commit this? Can fine tune in other issues? > Style the Web UI to use Twitter's Bootstrap. > > > Key: HBASE-6135 > URL: https://issues.apache.org/jira/browse/HBASE-6135 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch, > master-redesign.png, redesign-master-r1.png, redesign-rs-r1.png > > > Our web ui has lagged a little bit behind. While it's not a huge deal, it is > one of the first things that new people see. As such styling it a little bit > better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[ https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6135: - Attachment: HBASE-6135-2.patch Elliott's patch only w/ the logo in src/resources dir so can be found when run in-situ > Style the Web UI to use Twitter's Bootstrap. > > > Key: HBASE-6135 > URL: https://issues.apache.org/jira/browse/HBASE-6135 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch, > HBASE-6135-2.patch, master-redesign.png, redesign-master-r1.png, > redesign-rs-r1.png > > > Our web ui has lagged a little bit behind. While it's not a huge deal, it is > one of the first things that new people see. As such styling it a little bit > better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[ https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6135: - Attachment: HBASE-6135-3.patch Moved region server metrics into a table. They are actually readable now. > Style the Web UI to use Twitter's Bootstrap. > > > Key: HBASE-6135 > URL: https://issues.apache.org/jira/browse/HBASE-6135 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch, > HBASE-6135-2.patch, HBASE-6135-3.patch, master-redesign.png, > redesign-master-r1.png, redesign-rs-r1.png > > > Our web ui has lagged a little bit behind. While it's not a huge deal, it is > one of the first things that new people see. As such styling it a little bit > better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[ https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295331#comment-13295331 ] Elliott Clark commented on HBASE-6135: -- @Eugene I've been editing a jmon then running: bin/stop-hbase.sh && mvn package -DskipTests && bin/start-hbase.sh While it's not real fast it gets the job done. @stack Things should be good from my end. > Style the Web UI to use Twitter's Bootstrap. > > > Key: HBASE-6135 > URL: https://issues.apache.org/jira/browse/HBASE-6135 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch, > HBASE-6135-2.patch, HBASE-6135-3.patch, master-redesign.png, > redesign-master-r1.png, redesign-rs-r1.png > > > Our web ui has lagged a little bit behind. While it's not a huge deal, it is > one of the first things that new people see. As such styling it a little bit > better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[ https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6135: - Attachment: HBASE-6135-4.patch > Style the Web UI to use Twitter's Bootstrap. > > > Key: HBASE-6135 > URL: https://issues.apache.org/jira/browse/HBASE-6135 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch, > HBASE-6135-2.patch, HBASE-6135-3.patch, HBASE-6135-4.patch, > master-redesign.png, redesign-master-r1.png, redesign-rs-r1.png > > > Our web ui has lagged a little bit behind. While it's not a huge deal, it is > one of the first things that new people see. As such styling it a little bit > better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[ https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6135: - Attachment: HBASE-6135-5.patch What I committed. > Style the Web UI to use Twitter's Bootstrap. > > > Key: HBASE-6135 > URL: https://issues.apache.org/jira/browse/HBASE-6135 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch, > HBASE-6135-2.patch, HBASE-6135-3.patch, HBASE-6135-4.patch, > HBASE-6135-5.patch, master-redesign.png, redesign-master-r1.png, > redesign-rs-r1.png > > > Our web ui has lagged a little bit behind. While it's not a huge deal, it is > one of the first things that new people see. As such styling it a little bit > better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[ https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6135. -- Resolution: Fixed Hadoop Flags: Reviewed Thanks for the patch Elliott. Nice. > Style the Web UI to use Twitter's Bootstrap. > > > Key: HBASE-6135 > URL: https://issues.apache.org/jira/browse/HBASE-6135 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch, > HBASE-6135-2.patch, HBASE-6135-3.patch, HBASE-6135-4.patch, > HBASE-6135-5.patch, master-redesign.png, redesign-master-r1.png, > redesign-rs-r1.png > > > Our web ui has lagged a little bit behind. While it's not a huge deal, it is > one of the first things that new people see. As such styling it a little bit > better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6211) Latencies not in jmx
Elliott Clark created HBASE-6211: Summary: Latencies not in jmx Key: HBASE-6211 URL: https://issues.apache.org/jira/browse/HBASE-6211 Project: HBase Issue Type: Bug Environment: RegionServerMetrics pushes latency histograms to hadoop metrics, but they are not getting into jmx. Reporter: Elliott Clark Assignee: Elliott Clark -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5564) Bulkload is discarding duplicate records
[ https://issues.apache.org/jira/browse/HBASE-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295345#comment-13295345 ] stack commented on HBASE-5564: -- +1 on commit. > Bulkload is discarding duplicate records > > > Key: HBASE-5564 > URL: https://issues.apache.org/jira/browse/HBASE-5564 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 0.96.0 > Environment: HBase 0.92 >Reporter: Laxman >Assignee: Laxman > Labels: bulkloader > Fix For: 0.96.0 > > Attachments: 5564.lint, 5564v5.txt, HBASE-5564.patch, > HBASE-5564_1.patch, HBASE-5564_trunk.1.patch, HBASE-5564_trunk.1.patch, > HBASE-5564_trunk.2.patch, HBASE-5564_trunk.3.patch, > HBASE-5564_trunk.4_final.patch, HBASE-5564_trunk.patch > > > Duplicate records are getting discarded when duplicate records exists in same > input file and more specifically if they exists in same split. > Duplicate records are considered if the records are from diffrent different > splits. > Version under test: HBase 0.92 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6185) Update javadoc for ConstantSizeRegionSplitPolicy class
[
https://issues.apache.org/jira/browse/HBASE-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295349#comment-13295349
]
Hudson commented on HBASE-6185:
---
Integrated in HBase-TRUNK #3026 (See
[https://builds.apache.org/job/HBase-TRUNK/3026/])
HBASE-6185 Update javadoc for ConstantSizeRegionSplitPolicy class (Revision
1350388)
Result = SUCCESS
stack :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.java
> Update javadoc for ConstantSizeRegionSplitPolicy class
> --
>
> Key: HBASE-6185
> URL: https://issues.apache.org/jira/browse/HBASE-6185
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.94.0
>Reporter: nneverwei
> Fix For: 0.94.1
>
> Attachments: HBASE-6185.patch, HBASE-6185.v2.patch,
> HBASE-6185.v3.patch
>
>
> When using hbase0.94.0 we met a strange problem.
> We config the 'hbase.hregion.max.filesize' to 100Gb (The recommed value to
> act as auto-split turn off).
> {code:xml}
>
> hbase.hregion.max.filesize
> 107374182400
>
> {code}
> Then we keep putting datas into a table.
> But when the data size far more less than 100Gb(about 500~600 uncompressed
> datas), the table auto splte to 2 regions...
> I change the log4j config to DEBUG, and saw logs below:
> {code}
> 2012-06-07 10:30:52,161 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~128.0m/134221272, currentsize=1.5m/1617744 for
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. in
> 3201ms, sequenceid=176387980, compaction requested=false
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,240 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Split requested for
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8..
> compaction_queue=(0:0), split_queue=0
> 2012-06-07 10:30:52,265 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.
> 2012-06-07 10:30:52,265 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:60020-0x137c4929efe0001 Creating ephemeral node for
> 7b229abcd0785408251a579e9bdf49c8 in SPLITTING state
> 2012-06-07 10:30:52,368 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Attempting to transition node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,382 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Successfully transitioned node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,410 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Closing FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.:
> disabling compactions & flushes
> 2012-06-07 10:30:52,410 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> 2012-06-07 10:30:52,411 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> {code}
> {color:red}IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because info
> size=138657416, sizeToCheck=134217728{color}
> I did not config splitPolicy for hbase, so it means
> *IncreasingToUpperBoundRegionSplitPolicy is the default splitPolicy of 0.94.0*
> After add
> {code:xml}
>
> hbase.regionserver.region.split.policy
>
> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy
>
> {code}
> autosplit did not happen again and everything goes well.
> But we can still see javadoc on ConstantSizeRegionSplitPolicy, it says 'This
> is the default split policy'. Or even in the
> http://hbase.apache.org/book/regions.arch.html 9.7.4.1. Custom Split
> Policies, 'default split policy: ConstantSizeRegionSplitPolicy.'.
> Those may mistaken us that if we set hbase.hregion.max.filesize to 100Gb,
> than the auto-split can be almost shutdown.
> You may change those docs, and What more, in many scenery
[jira] [Commented] (HBASE-6196) MR testcases does not run with hadoop - 2.0.
[
https://issues.apache.org/jira/browse/HBASE-6196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295348#comment-13295348
]
stack commented on HBASE-6196:
--
Want to update the JIRA subject then Ram to reflect the new narrower scope?
> MR testcases does not run with hadoop - 2.0.
>
>
> Key: HBASE-6196
> URL: https://issues.apache.org/jira/browse/HBASE-6196
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
>Reporter: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
>
> The MR related testcases are failing in hadoop-2.0. The resource manager
> scheduler is not getting spawned.
> The following fix solves the problem. If you feel it is ok, I can submit as
> patch and commit.
> {code}
> String rmSchedulerAdress =
> mrClusterJobConf.get("yarn.resourcemanager.scheduler.address");
>
> if (rmSchedulerAdress != null) {
> conf.set("yarn.resourcemanager.scheduler.address", rmSchedulerAdress);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6185) Update javadoc for ConstantSizeRegionSplitPolicy class
[
https://issues.apache.org/jira/browse/HBASE-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295350#comment-13295350
]
Hudson commented on HBASE-6185:
---
Integrated in HBase-0.94 #257 (See
[https://builds.apache.org/job/HBase-0.94/257/])
HBASE-6185 Update javadoc for ConstantSizeRegionSplitPolicy class (Revision
1350389)
Result = SUCCESS
stack :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.java
> Update javadoc for ConstantSizeRegionSplitPolicy class
> --
>
> Key: HBASE-6185
> URL: https://issues.apache.org/jira/browse/HBASE-6185
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.94.0
>Reporter: nneverwei
> Fix For: 0.94.1
>
> Attachments: HBASE-6185.patch, HBASE-6185.v2.patch,
> HBASE-6185.v3.patch
>
>
> When using hbase0.94.0 we met a strange problem.
> We config the 'hbase.hregion.max.filesize' to 100Gb (The recommed value to
> act as auto-split turn off).
> {code:xml}
>
> hbase.hregion.max.filesize
> 107374182400
>
> {code}
> Then we keep putting datas into a table.
> But when the data size far more less than 100Gb(about 500~600 uncompressed
> datas), the table auto splte to 2 regions...
> I change the log4j config to DEBUG, and saw logs below:
> {code}
> 2012-06-07 10:30:52,161 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~128.0m/134221272, currentsize=1.5m/1617744 for
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. in
> 3201ms, sequenceid=176387980, compaction requested=false
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,240 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Split requested for
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8..
> compaction_queue=(0:0), split_queue=0
> 2012-06-07 10:30:52,265 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.
> 2012-06-07 10:30:52,265 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:60020-0x137c4929efe0001 Creating ephemeral node for
> 7b229abcd0785408251a579e9bdf49c8 in SPLITTING state
> 2012-06-07 10:30:52,368 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Attempting to transition node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,382 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Successfully transitioned node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,410 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Closing FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.:
> disabling compactions & flushes
> 2012-06-07 10:30:52,410 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> 2012-06-07 10:30:52,411 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> {code}
> {color:red}IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because info
> size=138657416, sizeToCheck=134217728{color}
> I did not config splitPolicy for hbase, so it means
> *IncreasingToUpperBoundRegionSplitPolicy is the default splitPolicy of 0.94.0*
> After add
> {code:xml}
>
> hbase.regionserver.region.split.policy
>
> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy
>
> {code}
> autosplit did not happen again and everything goes well.
> But we can still see javadoc on ConstantSizeRegionSplitPolicy, it says 'This
> is the default split policy'. Or even in the
> http://hbase.apache.org/book/regions.arch.html 9.7.4.1. Custom Split
> Policies, 'default split policy: ConstantSizeRegionSplitPolicy.'.
> Those may mistaken us that if we set hbase.hregion.max.filesize to 100Gb,
> than the auto-split can be almost shutdown.
> You may change those docs, and What more, in many scenerys, we actually
[jira] [Commented] (HBASE-6201) HBase integration/system tests
[
https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295354#comment-13295354
]
stack commented on HBASE-6201:
--
Thanks for doing this Enis. +1 on making subjiras off this one.
On IT tests and mvn verify, I concluded it a lost cause. Maybe you can figure
how to make it change its behavior when you pass a -Dconf= and have it
instead run on a real cluster (I found the mvn verify targets insufficient
IIRC).
The above is a comment on the mvn verify mechanism only, not on the intent of
this issue. I'm rah-rah on your intent. Just suggesting we may need to do it
atop a different vehicle (BT would be coolio. On hearing BT perspective on
this, we may have blown that for a while. We've gotten Roman excited on
multiple occasions but have failed to come through. I'd say lets deliver
something first, then Roman might think us serious this time).
bq. We need a tool on top of MiniHBaseCluster or improve HBaseTestingUtility,
so that tests can interface with the mini cluster or the actual cluster
uniformly.
I agree. Need to make it so test is same whether on mini or real cluster.
+1 on #3 and #4 points above.
> HBase integration/system tests
> --
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.96.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the
> conclusion is that we need to unify how we do "release candidate" testing
> (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open
> subtickets for execution so that we can carry out most of the tests in
> HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454
> (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt
> like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some
> of the tests (TestAcidGuarantees) from command line). Most of the system
> tests will be client side, and interface with the cluster through public
> APIs. We need a tool on top of MiniHBaseCluster or improve
> HBaseTestingUtility, so that tests can interface with the mini cluster or the
> actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the
> candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in
> semantics. However, given an actual cluster, they should use that, instead of
> spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's
> TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6195) Increment data will be lost when the memstore is flushed
[ https://issues.apache.org/jira/browse/HBASE-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6195: - Resolution: Fixed Status: Resolved (was: Patch Available) Ted committed this a few days ago. > Increment data will be lost when the memstore is flushed > > > Key: HBASE-6195 > URL: https://issues.apache.org/jira/browse/HBASE-6195 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Xing Shi >Assignee: ShiXing > Fix For: 0.96.0, 0.94.1 > > Attachments: 6195-trunk-V7.patch, 6195.addendum, > HBASE-6195-trunk-V2.patch, HBASE-6195-trunk-V3.patch, > HBASE-6195-trunk-V4.patch, HBASE-6195-trunk-V5.patch, > HBASE-6195-trunk-V6.patch, HBASE-6195-trunk.patch > > > There are two problems in increment() now: > First: > I see that the timestamp(the variable now) in HRegion's Increment() is > generated before got the rowLock, so when there are multi-thread increment > the same row, although it generate earlier, it may got the lock later. > Because increment just store one version, so till now, the result will still > be right. > When the region is flushing, these increment will read the kv from snapshot > and memstore with whose timestamp is larger, and write it back to memstore. > If the snapshot's timestamp larger than the memstore, the increment will got > the old data and then do the increment, it's wrong. > Secondly: > Also there is a risk in increment. Because it writes the memstore first and > then HLog, so if it writes HLog failed, the client will also read the > incremented value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6210) Backport HBASE-6197 to 0.94 and 0.92?
[ https://issues.apache.org/jira/browse/HBASE-6210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6210: - Priority: Blocker (was: Major) Fix Version/s: 0.94.1 Making blocker so it gets consideration > Backport HBASE-6197 to 0.94 and 0.92? > - > > Key: HBASE-6210 > URL: https://issues.apache.org/jira/browse/HBASE-6210 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Blocker > Fix For: 0.94.1 > > > Backport HBASE-6197 'HRegion's append operation may lose data' and the > accompanying HBASE-6195 to 0.94 and 0.92 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[ https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295362#comment-13295362 ] Hudson commented on HBASE-6135: --- Integrated in HBase-TRUNK #3027 (See [https://builds.apache.org/job/HBase-TRUNK/3027/]) HBASE-6135 Style the Web UI to use Twitter's Bootstrap. (Revision 1350407) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/common/TaskMonitorTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/AssignmentManagerStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/table.jsp * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/tablesDetailed.jsp * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css/bootstrap-responsive.css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css/bootstrap-responsive.min.css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css/bootstrap.css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css/bootstrap.min.css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/hbase_logo.png * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/js * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/js/bootstrap.js * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/js/bootstrap.min.js > Style the Web UI to use Twitter's Bootstrap. > > > Key: HBASE-6135 > URL: https://issues.apache.org/jira/browse/HBASE-6135 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch, > HBASE-6135-2.patch, HBASE-6135-3.patch, HBASE-6135-4.patch, > HBASE-6135-5.patch, master-redesign.png, redesign-master-r1.png, > redesign-rs-r1.png > > > Our web ui has lagged a little bit behind. While it's not a huge deal, it is > one of the first things that new people see. As such styling it a little bit > better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6212) Allow WAL to be disabled perTable.
Amitanand Aiyer created HBASE-6212: -- Summary: Allow WAL to be disabled perTable. Key: HBASE-6212 URL: https://issues.apache.org/jira/browse/HBASE-6212 Project: HBase Issue Type: New Feature Affects Versions: 0.89-fb Reporter: Amitanand Aiyer Priority: Minor There are use cases which want to get better performance by turning off WALEdits. At the server side, the config setting currently allows us to turn on or turn off the WAL edits. But, the setting applies to all the tables. We should be able to control this setting at a per table level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6185) Update javadoc for ConstantSizeRegionSplitPolicy class
[
https://issues.apache.org/jira/browse/HBASE-6185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295378#comment-13295378
]
Hudson commented on HBASE-6185:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #54 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/54/])
HBASE-6185 Update javadoc for ConstantSizeRegionSplitPolicy class (Revision
1350388)
Result = FAILURE
stack :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.java
> Update javadoc for ConstantSizeRegionSplitPolicy class
> --
>
> Key: HBASE-6185
> URL: https://issues.apache.org/jira/browse/HBASE-6185
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 0.94.0
>Reporter: nneverwei
> Fix For: 0.94.1
>
> Attachments: HBASE-6185.patch, HBASE-6185.v2.patch,
> HBASE-6185.v3.patch
>
>
> When using hbase0.94.0 we met a strange problem.
> We config the 'hbase.hregion.max.filesize' to 100Gb (The recommed value to
> act as auto-split turn off).
> {code:xml}
>
> hbase.hregion.max.filesize
> 107374182400
>
> {code}
> Then we keep putting datas into a table.
> But when the data size far more less than 100Gb(about 500~600 uncompressed
> datas), the table auto splte to 2 regions...
> I change the log4j config to DEBUG, and saw logs below:
> {code}
> 2012-06-07 10:30:52,161 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Finished memstore flush of ~128.0m/134221272, currentsize=1.5m/1617744 for
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. in
> 3201ms, sequenceid=176387980, compaction requested=false
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,161 DEBUG
> org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy:
> ShouldSplit because info size=138657416, sizeToCheck=134217728,
> regionsWithCommonTable=1
> 2012-06-07 10:30:52,240 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Split requested for
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8..
> compaction_queue=(0:0), split_queue=0
> 2012-06-07 10:30:52,265 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.
> 2012-06-07 10:30:52,265 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:60020-0x137c4929efe0001 Creating ephemeral node for
> 7b229abcd0785408251a579e9bdf49c8 in SPLITTING state
> 2012-06-07 10:30:52,368 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Attempting to transition node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,382 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x137c4929efe0001 Successfully transitioned node
> 7b229abcd0785408251a579e9bdf49c8 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2012-06-07 10:30:52,410 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
> Closing FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8.:
> disabling compactions & flushes
> 2012-06-07 10:30:52,410 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> 2012-06-07 10:30:52,411 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> NotServingRegionException;
> FileStructIndex,,1339032525500.7b229abcd0785408251a579e9bdf49c8. is closing
> {code}
> {color:red}IncreasingToUpperBoundRegionSplitPolicy: ShouldSplit because info
> size=138657416, sizeToCheck=134217728{color}
> I did not config splitPolicy for hbase, so it means
> *IncreasingToUpperBoundRegionSplitPolicy is the default splitPolicy of 0.94.0*
> After add
> {code:xml}
>
> hbase.regionserver.region.split.policy
>
> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy
>
> {code}
> autosplit did not happen again and everything goes well.
> But we can still see javadoc on ConstantSizeRegionSplitPolicy, it says 'This
> is the default split policy'. Or even in the
> http://hbase.apache.org/book/regions.arch.html 9.7.4.1. Custom Split
> Policies, 'default split policy: ConstantSizeRegionSplitPolicy.'.
> Those may mistaken us that if we set hbase.hregion.max.filesize to 100Gb,
> than the auto-split can be almost shutdown.
> You may change those docs, an
[jira] [Commented] (HBASE-6135) Style the Web UI to use Twitter's Bootstrap.
[ https://issues.apache.org/jira/browse/HBASE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295377#comment-13295377 ] Hudson commented on HBASE-6135: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #54 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/54/]) HBASE-6135 Style the Web UI to use Twitter's Bootstrap. (Revision 1350407) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/common/TaskMonitorTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/AssignmentManagerStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/table.jsp * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/tablesDetailed.jsp * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css/bootstrap-responsive.css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css/bootstrap-responsive.min.css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css/bootstrap.css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/css/bootstrap.min.css * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/hbase_logo.png * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/js * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/js/bootstrap.js * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/static/js/bootstrap.min.js > Style the Web UI to use Twitter's Bootstrap. > > > Key: HBASE-6135 > URL: https://issues.apache.org/jira/browse/HBASE-6135 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-6135-0.patch, HBASE-6135-1.patch, > HBASE-6135-2.patch, HBASE-6135-3.patch, HBASE-6135-4.patch, > HBASE-6135-5.patch, master-redesign.png, redesign-master-r1.png, > redesign-rs-r1.png > > > Our web ui has lagged a little bit behind. While it's not a huge deal, it is > one of the first things that new people see. As such styling it a little bit > better would put a good foot forward. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6162) Move KeyValue to hbase-common module
[ https://issues.apache.org/jira/browse/HBASE-6162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295379#comment-13295379 ] Hudson commented on HBASE-6162: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #54 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/54/]) HBASE-6162 Move KeyValue to hbase-common module (Revision 1350378) Result = FAILURE stack : Files : * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/io/HeapSize.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HeapSize.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/GetClosestRowBeforeTracker.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java > Move KeyValue to hbase-common module > > > Key: HBASE-6162 > URL: https://issues.apache.org/jira/browse/HBASE-6162 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.96.0 >Reporter: Matt Corgan >Assignee: Matt Corgan > Fix For: 0.96.0 > > Attachments: HBASE-6162-v1.patch, HBASE-6162-v2.patch, > HBASE-6162-v3.patch, HBASE-6162-v4.patch, HBASE-6162-v5.patch > > > * pull KeyValue up to hbase-common module > This is part of the modularization strategy in HBASE-5977, and is > specifically necessary to modularize HBASE-4676. > also brings these classes to hbase-common: > * ClassSize, HeapSize > * HTestConst > * TestKeyValue, KeyValueTestUtil > * LoadTestKVGenerator, TestLoadTestKVGenerator > * MD5Hash > moves a trivial constant (HRegionInfo.DELIMITER) from HRegionInfo to > HConstants -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295400#comment-13295400 ] Subbu M Iyer commented on HBASE-3909: - Thanks Stack/Ted. The latest version of patch basically enables any and all of the configuration properties (that I have captured in the excel. Roughly 300 config properties) to be changed through the shell. Some config changes will take immediate effect, while others will have no effect mid-flight, but will take effect when a Master/RS newly joins the cluster or fails over. So, basically what we have now is all you can change option with no exception. Only caveat is that the user may not know which one will be immediate and which is not. > Add dynamic config > -- > > Key: HBASE-3909 > URL: https://issues.apache.org/jira/browse/HBASE-3909 > Project: HBase > Issue Type: Bug >Reporter: stack > Fix For: 0.96.0 > > Attachments: 3909-v1.patch, 3909.v1, HBase Cluster Config > Details.xlsx, patch-v2.patch > > > I'm sure this issue exists already, at least as part of the discussion around > making online schema edits possible, but no hard this having its own issue. > Ted started a conversation on this topic up on dev and Todd suggested we > lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6202) Medium tests fail with jdk1.7
[ https://issues.apache.org/jira/browse/HBASE-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295420#comment-13295420 ] Jimmy Xiang commented on HBASE-6202: Not sure if it is really jdk1.7 related, which I need to find out. The issue is that I found that tests in a same test class are executed very randomly. Without the change, these tests are ok if executed in the order they are defined in the class. So this patch basically is to make sure each test doesn't affect each other if execution order is undetermined. > Medium tests fail with jdk1.7 > - > > Key: HBASE-6202 > URL: https://issues.apache.org/jira/browse/HBASE-6202 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Attachments: 6202-hbase.patch > > > Failed tests: > testOrphanTaskAcquisition(org.apache.hadoop.hbase.master.TestSplitLogManager) > testCreateAndUpdate(org.apache.hadoop.hbase.util.TestFSTableDescriptors): > statuses.length=5 > testManageSingleton(org.apache.hadoop.hbase.util.TestEnvironmentEdgeManager) > Tests in error: > > testRegionTransitionOperations(org.apache.hadoop.hbase.coprocessor.TestMasterObserver): > org.apache.hadoop.hbase.UnknownRegionException: > ef07dae0851f4fbe7d550413530b8774 > > testTableOperations(org.apache.hadoop.hbase.coprocessor.TestMasterObserver): > org.apache.hadoop.hbase.TableExistsException: observed_table > > testClassLoadingFromHDFS(org.apache.hadoop.hbase.coprocessor.TestClassLoading): > org.apache.hadoop.hbase.TableNotEnabledException: TestClassLoading -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time
[ https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295423#comment-13295423 ] chunhui shen commented on HBASE-6205: - @stack In my memory, trash facility of hdfs is only used for shell commend, if we called fs.delete() through code, could it work also? Sorry, I'm not sure about this. Anyway, we would delete files after compaction or old logs, and needn't to reserve them. > Support an option to keep data of dropped table for some time > - > > Key: HBASE-6205 > URL: https://issues.apache.org/jira/browse/HBASE-6205 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6205.patch, HBASE-6205v2.patch, HBASE-6205v3.patch > > > User may drop table accidentally because of error code or other uncertain > reasons. > Unfortunately, it happens in our environment because one user make a mistake > between production cluster and testing cluster. > So, I just give a suggestion, do we need to support an option to keep data of > dropped table for some time, e.g. 1 day > In the patch: > We make a new dir named .trashtables in the rood dir. > In the DeleteTableHandler, we move files in dropped table's dir to trash > table dir instead of deleting them directly. > And Create new class TrashCleaner which will clean dropped tables if it is > time out with a period check. > Default keep time for dropped tables is 1 day, and check period is 1 hour. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6213) Add metrics for the number of blocks in cache per CF and block type
Mikhail Bautin created HBASE-6213: - Summary: Add metrics for the number of blocks in cache per CF and block type Key: HBASE-6213 URL: https://issues.apache.org/jira/browse/HBASE-6213 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor We need to know the number of blocks in cache per CF/block type, not just the total size of all blocks of the given type, in order to measure data block encoding efficiency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time
[ https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295428#comment-13295428 ] chunhui shen commented on HBASE-6205: - review board https://review.cloudera.org/r/2153/ > Support an option to keep data of dropped table for some time > - > > Key: HBASE-6205 > URL: https://issues.apache.org/jira/browse/HBASE-6205 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.94.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-6205.patch, HBASE-6205v2.patch, HBASE-6205v3.patch > > > User may drop table accidentally because of error code or other uncertain > reasons. > Unfortunately, it happens in our environment because one user make a mistake > between production cluster and testing cluster. > So, I just give a suggestion, do we need to support an option to keep data of > dropped table for some time, e.g. 1 day > In the patch: > We make a new dir named .trashtables in the rood dir. > In the DeleteTableHandler, we move files in dropped table's dir to trash > table dir instead of deleting them directly. > And Create new class TrashCleaner which will clean dropped tables if it is > time out with a period check. > Default keep time for dropped tables is 1 day, and check period is 1 hour. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4487) The increment operation can release the rowlock before sync-ing the Hlog
[ https://issues.apache.org/jira/browse/HBASE-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295432#comment-13295432 ] Xing Shi commented on HBASE-4487: - But now there is no code for hlog sync in Increment and Append > The increment operation can release the rowlock before sync-ing the Hlog > > > Key: HBASE-4487 > URL: https://issues.apache.org/jira/browse/HBASE-4487 > Project: HBase > Issue Type: Improvement >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.94.0 > > Attachments: 4487-v7.txt, appendNoSync4.txt, appendNoSync5.txt, > appendNoSync6.txt > > > This allows for better throughput when there are hot rows.I have seen this > change make a single row update improve from 400 increments/sec/server to > 4000 increments/sec/server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6192) Document ACL matrix in the book
[ https://issues.apache.org/jira/browse/HBASE-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295435#comment-13295435 ] Laxman commented on HBASE-6192: --- Thanks stack. I'll update the matrix considering HBASE-6209. I'll do that. > Document ACL matrix in the book > --- > > Key: HBASE-6192 > URL: https://issues.apache.org/jira/browse/HBASE-6192 > Project: HBase > Issue Type: Sub-task > Components: documentation, security >Affects Versions: 0.96.0, 0.94.1 >Reporter: Enis Soztutar >Assignee: Laxman > Labels: documentaion, security > Attachments: HBase Security-ACL Matrix.pdf, HBase Security-ACL > Matrix.xls > > > We have an excellent matrix at > https://issues.apache.org/jira/secure/attachment/12531252/Security-ACL%20Matrix.pdf > for ACL. Once the changes are done, we can adapt that and put it in the > book, also add some more documentation about the new authorization features. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6209) ACL Corrections for AccessControllerProtocol apis
[ https://issues.apache.org/jira/browse/HBASE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laxman updated HBASE-6209: -- Description: APIs provided in AccessController are authorized against global-admin permissions. Instead we need to check for table-admin level permissions. Edit: Append operation also has no authorization check. We can update it together. was:APIs provided in AccessController are authorized against global-admin permissions. Instead we need to check for table-admin level permissions. > ACL Corrections for AccessControllerProtocol apis > - > > Key: HBASE-6209 > URL: https://issues.apache.org/jira/browse/HBASE-6209 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0, 0.96.0, 0.94.1 >Reporter: Laxman >Assignee: Laxman > Labels: acl, security > Fix For: 0.96.0, 0.94.1 > > > APIs provided in AccessController are authorized against global-admin > permissions. Instead we need to check for table-admin level permissions. > Edit: Append operation also has no authorization check. We can update it > together. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6209) ACL Corrections for AccessControllerProtocol apis
[ https://issues.apache.org/jira/browse/HBASE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295436#comment-13295436 ] Laxman commented on HBASE-6209: --- Thinking of following approach. grant, revoke, getUserPermissions, checkPermissions - All these apis needs to be authorized against CQ/CF/table/global admin permissions. Append - WRITE Any corrections required? > ACL Corrections for AccessControllerProtocol apis > - > > Key: HBASE-6209 > URL: https://issues.apache.org/jira/browse/HBASE-6209 > Project: HBase > Issue Type: Sub-task > Components: security >Affects Versions: 0.94.0, 0.96.0, 0.94.1 >Reporter: Laxman >Assignee: Laxman > Labels: acl, security > Fix For: 0.96.0, 0.94.1 > > > APIs provided in AccessController are authorized against global-admin > permissions. Instead we need to check for table-admin level permissions. > Edit: Append operation also has no authorization check. We can update it > together. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295448#comment-13295448 ] ramkrishna.s.vasudevan commented on HBASE-5998: --- @Stack We have a backport version. We can submit it today. > Bulk assignment: regionserver optimization by using a temporary cache for > table descriptors when receveing an open regions request > -- > > Key: HBASE-5998 > URL: https://issues.apache.org/jira/browse/HBASE-5998 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5998.v2.patch, 5998.v3.patch > > > During the assignment, on the regionserver, before creating the handlers we > load the table description. Even if there is a cache, we check the timestamps > for each region, while it's not necessary. The test below is just with one > node, with more nodes the benefit will improve. By limiting the time spent in > HRegion#openRegion we increase the parallelization during cluster startup, as > the master is using a pool of threads to call the RS. > -- Without the fix > 2012-05-14 11:40:52,501 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 > region(s) to localhost,11003,1336988444043 > 2012-05-14 11:41:09,947 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for > localhost,11003,1336988444043 > -- With the fix > 2012-05-14 11:34:40,444 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 > region(s) to localhost,11003,1336988444043 > 2012-05-14 11:34:40,929 DEBUG > org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for > localhost,11003,1336988065948 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4821) A fully automated comprehensive distributed integration test for HBase
[ https://issues.apache.org/jira/browse/HBASE-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295453#comment-13295453 ] Andrew Purtell commented on HBASE-4821: --- This discussion was carried forward into HBASE-6201 ? > A fully automated comprehensive distributed integration test for HBase > -- > > Key: HBASE-4821 > URL: https://issues.apache.org/jira/browse/HBASE-4821 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin >Priority: Critical > > To properly verify that a particular version of HBase is good for production > deployment we need a better way to do real cluster testing after incremental > changes. Running unit tests is good, but we also need to deploy HBase to a > cluster, run integration tests, load tests, Thrift server tests, kill some > region servers, kill the master, and produce a report. All of this needs to > happen in 20-30 minutes with minimal manual intervention. I think this way we > can combine agile development with high stability of the codebase. I am > envisioning a high-level framework written in a scripting language (e.g. > Python) that would abstract external operations such as "deploy to test > cluster", "kill a particular server", "run load test A", "run load test B" > (we already have a few kinds of load tests implemented in Java, and we could > write a Thrift load test in Python). This tool should also produce > intermediate output, allowing to catch problems early and restart the test. > No implementation has yet been done. Any ideas or suggestions are welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6201) HBase integration/system tests
[
https://issues.apache.org/jira/browse/HBASE-6201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295455#comment-13295455
]
Andrew Purtell commented on HBASE-6201:
---
bq. On IT tests and mvn verify, I concluded it a lost cause.
That is disappointing.
And we should also consider the potential benefits of importing and using
iTest: http://cloudera.github.com/bigtop/iTest/userguide.html It provides some
useful functionality.
> HBase integration/system tests
> --
>
> Key: HBASE-6201
> URL: https://issues.apache.org/jira/browse/HBASE-6201
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.96.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> Integration and general system tests have been discussed previously, and the
> conclusion is that we need to unify how we do "release candidate" testing
> (HBASE-6091).
> In this issue, I would like to discuss and agree on a general plan, and open
> subtickets for execution so that we can carry out most of the tests in
> HBASE-6091 automatically.
> Initially, here is what I have in mind:
> 1. Create hbase-it (or hbase-tests) containing forward port of HBASE-4454
> (without any tests). This will allow integration test to be run with
> {code}
> mvn verify
> {code}
> 2. Add ability to run all integration/system tests on a given cluster. Smt
> like:
> {code}
> mvn verify -Dconf=/etc/hbase/conf/
> {code}
> should run the test suite on the given cluster. (Right now we can launch some
> of the tests (TestAcidGuarantees) from command line). Most of the system
> tests will be client side, and interface with the cluster through public
> APIs. We need a tool on top of MiniHBaseCluster or improve
> HBaseTestingUtility, so that tests can interface with the mini cluster or the
> actual cluster uniformly.
> 3. Port candidate unit tests to the integration tests module. Some of the
> candidates are:
> - TestAcidGuarantees / TestAtomicOperation
> - TestRegionBalancing (HBASE-6053)
> - TestFullLogReconstruction
> - TestMasterFailover
> - TestImportExport
> - TestMultiVersions / TestKeepDeletes
> - TestFromClientSide
> - TestShell and src/test/ruby
> - TestRollingRestart
> - Test**OnCluster
> - Balancer tests
> These tests should continue to be run as unit tests w/o any change in
> semantics. However, given an actual cluster, they should use that, instead of
> spinning a mini cluster.
> 4. Add more tests, especially, long running ingestion tests (goraci, BigTop's
> TestLoadAndVerify, LoadTestTool), and chaos monkey style fault tests.
> All suggestions welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6196) MR testcases TestImportExport does not run in Trunk with hadoop2.0
[
https://issues.apache.org/jira/browse/HBASE-6196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-6196:
--
Description:
TEstImportExport test cases does not run in trunk when compiled with hadoop 2.0
was:
The MR related testcases are failing in hadoop-2.0. The resource manager
scheduler is not getting spawned.
The following fix solves the problem. If you feel it is ok, I can submit as
patch and commit.
{code}
String rmSchedulerAdress =
mrClusterJobConf.get("yarn.resourcemanager.scheduler.address");
if (rmSchedulerAdress != null) {
conf.set("yarn.resourcemanager.scheduler.address", rmSchedulerAdress);
}
{code}
Summary: MR testcases TestImportExport does not run in Trunk with
hadoop2.0 (was: MR testcases does not run with hadoop - 2.0.)
> MR testcases TestImportExport does not run in Trunk with hadoop2.0
> --
>
> Key: HBASE-6196
> URL: https://issues.apache.org/jira/browse/HBASE-6196
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0, 0.96.0
>Reporter: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
>
> TEstImportExport test cases does not run in trunk when compiled with hadoop
> 2.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6196) MR testcases TestImportExport does not run in Trunk with hadoop2.0
[ https://issues.apache.org/jira/browse/HBASE-6196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295456#comment-13295456 ] ramkrishna.s.vasudevan commented on HBASE-6196: --- Update the title and description > MR testcases TestImportExport does not run in Trunk with hadoop2.0 > -- > > Key: HBASE-6196 > URL: https://issues.apache.org/jira/browse/HBASE-6196 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0, 0.96.0 >Reporter: ramkrishna.s.vasudevan > Fix For: 0.96.0 > > > TEstImportExport test cases does not run in trunk when compiled with hadoop > 2.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6214) Backport HBASE-5998 to 94.1
Anoop Sam John created HBASE-6214: - Summary: Backport HBASE-5998 to 94.1 Key: HBASE-6214 URL: https://issues.apache.org/jira/browse/HBASE-6214 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Anoop Sam John Fix For: 0.94.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
