[jira] [Commented] (HBASE-9441) Intermittent TestRegionPlacement#testRegionPlacement failure
[
https://issues.apache.org/jira/browse/HBASE-9441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13759982#comment-13759982
]
Hadoop QA commented on HBASE-9441:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12601765/9441-v5.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7066//console
This message is automatically generated.
Intermittent TestRegionPlacement#testRegionPlacement failure
Key: HBASE-9441
URL: https://issues.apache.org/jira/browse/HBASE-9441
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9441-v1.txt, 9441-v2.txt, 9441-v3.txt, 9441-v4.txt,
9441-v5.txt, org.apache.hadoop.hbase.master.TestRegionPlacement-output.txt
Occasionally TestRegionPlacement#testRegionPlacement fails due to no region
found on secondary / tertiary region server.
In the following test output, 10359edcfb1a761f94ab6d5b559c2278 was the region
from the killed region server:
{code}
2013-09-04 23:28:25,127 INFO [pool-1-thread-1] util.FSUtils(1878): Scan DFS
for locality info takes 38 ms
Region Assignment Verification for Table: testRegionAssignment
Total regions : 10
Total regions on favored nodes 10
Total regions on PRIMARY region servers: 10
Total regions on SECONDARY region servers: 0
Total regions on TERTIARY region servers: 0
...
2013-09-04 23:28:25,130 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
path=/hbase/region-in-transition/10359edcfb1a761f94ab6d5b559c2278
2013-09-04 23:28:25,130 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
zookeeper.ZKAssign(861): regionserver:49841-0x140eb4dc4ef0007 Transitioned
node 10359edcfb1a761f94ab6d5b559c2278 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
handler.OpenRegionHandler(386): Transitioned 10359edcfb1a761f94ab6d5b559c2278
to OPENED in zk on kiyo.gq1.ygridcore.net,49841,1378337278470
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
[jira] [Created] (HBASE-9451) Meta remains unassigned when the meta server crashes with the ClusterStatusListener set
Devaraj Das created HBASE-9451: -- Summary: Meta remains unassigned when the meta server crashes with the ClusterStatusListener set Key: HBASE-9451 URL: https://issues.apache.org/jira/browse/HBASE-9451 Project: HBase Issue Type: Bug Reporter: Devaraj Das While running tests described in HBASE-9338, ran into this problem. The hbase.status.listener.class was set to org.apache.hadoop.hbase.client.ClusterStatusListener$MultiCastListener. 1. I had the meta server coming down 2. The metaSSH got triggered. The call chain: 2.1 verifyAndAssignMetaWithRetries 2.2 verifyMetaRegionLocation 2.3 waitForMetaServerConnection 2.4 getMetaServerConnection 2.5 getCachedConnection 2.6 HConnectionManager.getAdmin(serverName, false) 2.7 isDeadServer(serverName) - This is hardcoded to return 'false' when the clusterStatusListener field is null. If clusterStatusListener is not null (in my test), then it could return true in certain cases (and in this case, indeed it should return true since the server is down). I am trying to understand why it's hardcoded to 'false' for former case. 3. When isDeadServer returns true, the method HConnectionManager.getAdmin(ServerName, boolean) throws RegionServerStoppedException. 4. Finally, after the retries are over verifyAndAssignMetaWithRetries gives up and the master aborts. The methods in the above call chain don't handle RegionServerStoppedException. Maybe something to look at... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation
[ https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-3484: -- Attachment: WIP_HBASE-3484.patch Doing the memstore slicing Replace memstore's ConcurrentSkipListMap with our own implementation Key: HBASE-3484 URL: https://issues.apache.org/jira/browse/HBASE-3484 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.92.0 Reporter: Todd Lipcon Priority: Critical Attachments: hierarchical-map.txt, memstore_drag.png, WIP_HBASE-3484.patch By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements to it for our use case in MemStore: - add an iterator.replace() method which should allow us to do upsert much more cheaply - implement a Set directly without having to do MapKeyValue,KeyValue to save one reference per entry It turns out CSLM is in public domain from its development as part of JSR 166, so we should be OK with licenses. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9444) EncodedScannerV2#isSeeked does not behave as described in javadoc
[ https://issues.apache.org/jira/browse/HBASE-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760092#comment-13760092 ] Anoop Sam John commented on HBASE-9444: --- will it be better to ask seeker whether it is being seeked to? Agree that with the current code, checking the seeker object to not null will work. But which one will look cleaner? EncodedScannerV2#isSeeked does not behave as described in javadoc - Key: HBASE-9444 URL: https://issues.apache.org/jira/browse/HBASE-9444 Project: HBase Issue Type: Bug Components: HFile Reporter: Chao Shi Priority: Minor Attachments: hbase-9444.patch I hit this when my tool is scanning HFiles using the scanner. I found isSeeked behaves different whether the HFiles are prefix-encoded or not. There is a test case in my patch that demonstrates the bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9451) Meta remains unassigned when the meta server crashes with the ClusterStatusListener set
[ https://issues.apache.org/jira/browse/HBASE-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760115#comment-13760115 ] Nicolas Liochon commented on HBASE-9451: bq. I am trying to understand why it's hardcoded to 'false' for former case. It's because if we don't have the status, then we don't know, so we consider the server is up. Meta remains unassigned when the meta server crashes with the ClusterStatusListener set --- Key: HBASE-9451 URL: https://issues.apache.org/jira/browse/HBASE-9451 Project: HBase Issue Type: Bug Reporter: Devaraj Das While running tests described in HBASE-9338, ran into this problem. The hbase.status.listener.class was set to org.apache.hadoop.hbase.client.ClusterStatusListener$MultiCastListener. 1. I had the meta server coming down 2. The metaSSH got triggered. The call chain: 2.1 verifyAndAssignMetaWithRetries 2.2 verifyMetaRegionLocation 2.3 waitForMetaServerConnection 2.4 getMetaServerConnection 2.5 getCachedConnection 2.6 HConnectionManager.getAdmin(serverName, false) 2.7 isDeadServer(serverName) - This is hardcoded to return 'false' when the clusterStatusListener field is null. If clusterStatusListener is not null (in my test), then it could return true in certain cases (and in this case, indeed it should return true since the server is down). I am trying to understand why it's hardcoded to 'false' for former case. 3. When isDeadServer returns true, the method HConnectionManager.getAdmin(ServerName, boolean) throws RegionServerStoppedException. 4. Finally, after the retries are over verifyAndAssignMetaWithRetries gives up and the master aborts. The methods in the above call chain don't handle RegionServerStoppedException. Maybe something to look at... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9451) Meta remains unassigned when the meta server crashes with the ClusterStatusListener set
[ https://issues.apache.org/jira/browse/HBASE-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9451: --- Attachment: 9451.v1.patch Meta remains unassigned when the meta server crashes with the ClusterStatusListener set --- Key: HBASE-9451 URL: https://issues.apache.org/jira/browse/HBASE-9451 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Nicolas Liochon Attachments: 9451.v1.patch While running tests described in HBASE-9338, ran into this problem. The hbase.status.listener.class was set to org.apache.hadoop.hbase.client.ClusterStatusListener$MultiCastListener. 1. I had the meta server coming down 2. The metaSSH got triggered. The call chain: 2.1 verifyAndAssignMetaWithRetries 2.2 verifyMetaRegionLocation 2.3 waitForMetaServerConnection 2.4 getMetaServerConnection 2.5 getCachedConnection 2.6 HConnectionManager.getAdmin(serverName, false) 2.7 isDeadServer(serverName) - This is hardcoded to return 'false' when the clusterStatusListener field is null. If clusterStatusListener is not null (in my test), then it could return true in certain cases (and in this case, indeed it should return true since the server is down). I am trying to understand why it's hardcoded to 'false' for former case. 3. When isDeadServer returns true, the method HConnectionManager.getAdmin(ServerName, boolean) throws RegionServerStoppedException. 4. Finally, after the retries are over verifyAndAssignMetaWithRetries gives up and the master aborts. The methods in the above call chain don't handle RegionServerStoppedException. Maybe something to look at... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-9451) Meta remains unassigned when the meta server crashes with the ClusterStatusListener set
[ https://issues.apache.org/jira/browse/HBASE-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon reassigned HBASE-9451: -- Assignee: Nicolas Liochon Meta remains unassigned when the meta server crashes with the ClusterStatusListener set --- Key: HBASE-9451 URL: https://issues.apache.org/jira/browse/HBASE-9451 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Nicolas Liochon While running tests described in HBASE-9338, ran into this problem. The hbase.status.listener.class was set to org.apache.hadoop.hbase.client.ClusterStatusListener$MultiCastListener. 1. I had the meta server coming down 2. The metaSSH got triggered. The call chain: 2.1 verifyAndAssignMetaWithRetries 2.2 verifyMetaRegionLocation 2.3 waitForMetaServerConnection 2.4 getMetaServerConnection 2.5 getCachedConnection 2.6 HConnectionManager.getAdmin(serverName, false) 2.7 isDeadServer(serverName) - This is hardcoded to return 'false' when the clusterStatusListener field is null. If clusterStatusListener is not null (in my test), then it could return true in certain cases (and in this case, indeed it should return true since the server is down). I am trying to understand why it's hardcoded to 'false' for former case. 3. When isDeadServer returns true, the method HConnectionManager.getAdmin(ServerName, boolean) throws RegionServerStoppedException. 4. Finally, after the retries are over verifyAndAssignMetaWithRetries gives up and the master aborts. The methods in the above call chain don't handle RegionServerStoppedException. Maybe something to look at... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9451) Meta remains unassigned when the meta server crashes with the ClusterStatusListener set
[ https://issues.apache.org/jira/browse/HBASE-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9451: --- Status: Patch Available (was: Open) Meta remains unassigned when the meta server crashes with the ClusterStatusListener set --- Key: HBASE-9451 URL: https://issues.apache.org/jira/browse/HBASE-9451 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Nicolas Liochon Attachments: 9451.v1.patch While running tests described in HBASE-9338, ran into this problem. The hbase.status.listener.class was set to org.apache.hadoop.hbase.client.ClusterStatusListener$MultiCastListener. 1. I had the meta server coming down 2. The metaSSH got triggered. The call chain: 2.1 verifyAndAssignMetaWithRetries 2.2 verifyMetaRegionLocation 2.3 waitForMetaServerConnection 2.4 getMetaServerConnection 2.5 getCachedConnection 2.6 HConnectionManager.getAdmin(serverName, false) 2.7 isDeadServer(serverName) - This is hardcoded to return 'false' when the clusterStatusListener field is null. If clusterStatusListener is not null (in my test), then it could return true in certain cases (and in this case, indeed it should return true since the server is down). I am trying to understand why it's hardcoded to 'false' for former case. 3. When isDeadServer returns true, the method HConnectionManager.getAdmin(ServerName, boolean) throws RegionServerStoppedException. 4. Finally, after the retries are over verifyAndAssignMetaWithRetries gives up and the master aborts. The methods in the above call chain don't handle RegionServerStoppedException. Maybe something to look at... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9413) WebUI says .META. table but table got renames to hbase:meta. Meed to update the UI.
[ https://issues.apache.org/jira/browse/HBASE-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760162#comment-13760162 ] Hudson commented on HBASE-9413: --- FAILURE: Integrated in hbase-0.96-hadoop2 #8 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/8/]) HBASE-9413 WebUI says .META. table but table got renames to hbase:meta. Meed to update the UI (stack: rev 1520464) * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Registry.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/executor/EventType.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaRegionTracker.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/Cell.java * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/CellComparator.java * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java * /hbase/branches/0.96/hbase-common/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java * /hbase/branches/0.96/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterFileSystemSource.java * /hbase/branches/0.96/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RestartRsHoldingMetaAction.java * /hbase/branches/0.96/hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/PrefixTreeCodec.java * /hbase/branches/0.96/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * /hbase/branches/0.96/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaMigrationConvertingToPB.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ClusterStatusPublisher.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionPlacementMaintainer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/SnapshotOfRegionAssignmentFromMeta.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ClosedRegionHandler.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java *
[jira] [Commented] (HBASE-9451) Meta remains unassigned when the meta server crashes with the ClusterStatusListener set
[
https://issues.apache.org/jira/browse/HBASE-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760163#comment-13760163
]
Hadoop QA commented on HBASE-9451:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12601807/9451.v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7067//console
This message is automatically generated.
Meta remains unassigned when the meta server crashes with the
ClusterStatusListener set
---
Key: HBASE-9451
URL: https://issues.apache.org/jira/browse/HBASE-9451
Project: HBase
Issue Type: Bug
Reporter: Devaraj Das
Assignee: Nicolas Liochon
Attachments: 9451.v1.patch
While running tests described in HBASE-9338, ran into this problem. The
hbase.status.listener.class was set to
org.apache.hadoop.hbase.client.ClusterStatusListener$MultiCastListener.
1. I had the meta server coming down
2. The metaSSH got triggered. The call chain:
2.1 verifyAndAssignMetaWithRetries
2.2 verifyMetaRegionLocation
2.3 waitForMetaServerConnection
2.4 getMetaServerConnection
2.5 getCachedConnection
2.6 HConnectionManager.getAdmin(serverName, false)
2.7 isDeadServer(serverName) - This is hardcoded to return 'false' when
the clusterStatusListener field is null. If clusterStatusListener is not null
(in my test), then it could return true in certain cases (and in this case,
indeed it should return true since the server is down). I am trying to
understand why it's hardcoded to 'false' for former case.
3. When isDeadServer returns true, the method
HConnectionManager.getAdmin(ServerName, boolean) throws
RegionServerStoppedException.
4. Finally, after the retries are over verifyAndAssignMetaWithRetries gives
up and the master aborts.
The methods in the above call chain don't handle
RegionServerStoppedException. Maybe something to look at...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA
[jira] [Updated] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation
[ https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-3484: -- Attachment: (was: WIP_HBASE-3484.patch) Replace memstore's ConcurrentSkipListMap with our own implementation Key: HBASE-3484 URL: https://issues.apache.org/jira/browse/HBASE-3484 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.92.0 Reporter: Todd Lipcon Priority: Critical Attachments: hierarchical-map.txt, memstore_drag.png By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements to it for our use case in MemStore: - add an iterator.replace() method which should allow us to do upsert much more cheaply - implement a Set directly without having to do MapKeyValue,KeyValue to save one reference per entry It turns out CSLM is in public domain from its development as part of JSR 166, so we should be OK with licenses. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation
[ https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-3484: -- Attachment: WIP_HBASE-3484.patch Replace memstore's ConcurrentSkipListMap with our own implementation Key: HBASE-3484 URL: https://issues.apache.org/jira/browse/HBASE-3484 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.92.0 Reporter: Todd Lipcon Priority: Critical Attachments: hierarchical-map.txt, memstore_drag.png, WIP_HBASE-3484.patch By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements to it for our use case in MemStore: - add an iterator.replace() method which should allow us to do upsert much more cheaply - implement a Set directly without having to do MapKeyValue,KeyValue to save one reference per entry It turns out CSLM is in public domain from its development as part of JSR 166, so we should be OK with licenses. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3484) Replace memstore's ConcurrentSkipListMap with our own implementation
[ https://issues.apache.org/jira/browse/HBASE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760178#comment-13760178 ] Anoop Sam John commented on HBASE-3484: --- In current patch the max size of the memstore slice is specified in terms of heap size. But what matters is the #entries in the CSLM. I am thinking this can be #entries in the slice. Replace memstore's ConcurrentSkipListMap with our own implementation Key: HBASE-3484 URL: https://issues.apache.org/jira/browse/HBASE-3484 Project: HBase Issue Type: Improvement Components: Performance Affects Versions: 0.92.0 Reporter: Todd Lipcon Priority: Critical Attachments: hierarchical-map.txt, memstore_drag.png, WIP_HBASE-3484.patch By copy-pasting ConcurrentSkipListMap into HBase we can make two improvements to it for our use case in MemStore: - add an iterator.replace() method which should allow us to do upsert much more cheaply - implement a Set directly without having to do MapKeyValue,KeyValue to save one reference per entry It turns out CSLM is in public domain from its development as part of JSR 166, so we should be OK with licenses. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9439) shell command list shows something not meaningful
[
https://issues.apache.org/jira/browse/HBASE-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760227#comment-13760227
]
Jonathan Hsieh commented on HBASE-9439:
---
I've gotten my snapshot rig up and working with hadoop 2.1.0 and 0.96.0rc0 now.
I doubly confirm that it works. While working on HBASE-9449, it is true that
the output is unsightly and is handled by the parts originally handled by
HBASE-5548. I'll try to clean it up.
shell command list shows something not meaningful
-
Key: HBASE-9439
URL: https://issues.apache.org/jira/browse/HBASE-9439
Project: HBase
Issue Type: Bug
Components: shell
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.0
Attachments: trunk-9439.patch
Here is a sample output:
{noformat}
hbase(main):004:0 list
TABLE
usertable
1 row(s) in 0.3240 seconds
= ##Class:0x2026c088:0x5eb8f6d
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9450) TestDistributedLogSplitting fails
[ https://issues.apache.org/jira/browse/HBASE-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760260#comment-13760260 ] stack commented on HBASE-9450: -- Thank you [~jxiang] and [~jeffreyz] for digging in on this one. TestDistributedLogSplitting fails - Key: HBASE-9450 URL: https://issues.apache.org/jira/browse/HBASE-9450 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jimmy Xiang Attachments: trunk-9450.patch Would you mind taking a look at a recent set of failures please [~jeffreyz]? It seems to be failing more recently of late: https://issues.apache.org/jira/browse/HBASE-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13759275#comment-13759275 https://builds.apache.org/job/PreCommit-HBASE-Build/7060//testReport/ https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/703/ Thank you. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9301) Default hbase.dynamic.jars.dir to hbase.rootdir/jars
[ https://issues.apache.org/jira/browse/HBASE-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760381#comment-13760381 ] Enis Soztutar commented on HBASE-9301: -- bq. Let's do .lib in 0.94 and lib in 0.96+ then. Sounds good. The only thing now is whether we want migration for the lib dir between 0.94 and 0.96. If so, we can add that to NamespaceUpgrade.migrateDotDirs(). Default hbase.dynamic.jars.dir to hbase.rootdir/jars Key: HBASE-9301 URL: https://issues.apache.org/jira/browse/HBASE-9301 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.2, 0.94.12, 0.96.0 Reporter: James Taylor Assignee: Vasu Mariyala Attachments: 0.94-HBASE-9301.patch, HBASE-9301.patch A reasonable default for hbase.dynamic.jars.dir would be hbase.rootdir/jars so that folks aren't forced to edit their hbase-sites.xml to take advantage of the new, cool feature to load coprocessor/custom filter jars out of HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9451) Meta remains unassigned when the meta server crashes with the ClusterStatusListener set
[ https://issues.apache.org/jira/browse/HBASE-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760384#comment-13760384 ] Jimmy Xiang commented on HBASE-9451: +1 Meta remains unassigned when the meta server crashes with the ClusterStatusListener set --- Key: HBASE-9451 URL: https://issues.apache.org/jira/browse/HBASE-9451 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Nicolas Liochon Attachments: 9451.v1.patch While running tests described in HBASE-9338, ran into this problem. The hbase.status.listener.class was set to org.apache.hadoop.hbase.client.ClusterStatusListener$MultiCastListener. 1. I had the meta server coming down 2. The metaSSH got triggered. The call chain: 2.1 verifyAndAssignMetaWithRetries 2.2 verifyMetaRegionLocation 2.3 waitForMetaServerConnection 2.4 getMetaServerConnection 2.5 getCachedConnection 2.6 HConnectionManager.getAdmin(serverName, false) 2.7 isDeadServer(serverName) - This is hardcoded to return 'false' when the clusterStatusListener field is null. If clusterStatusListener is not null (in my test), then it could return true in certain cases (and in this case, indeed it should return true since the server is down). I am trying to understand why it's hardcoded to 'false' for former case. 3. When isDeadServer returns true, the method HConnectionManager.getAdmin(ServerName, boolean) throws RegionServerStoppedException. 4. Finally, after the retries are over verifyAndAssignMetaWithRetries gives up and the master aborts. The methods in the above call chain don't handle RegionServerStoppedException. Maybe something to look at... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9451) Meta remains unassigned when the meta server crashes with the ClusterStatusListener set
[
https://issues.apache.org/jira/browse/HBASE-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760373#comment-13760373
]
Hadoop QA commented on HBASE-9451:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12601807/9451.v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7069//console
This message is automatically generated.
Meta remains unassigned when the meta server crashes with the
ClusterStatusListener set
---
Key: HBASE-9451
URL: https://issues.apache.org/jira/browse/HBASE-9451
Project: HBase
Issue Type: Bug
Reporter: Devaraj Das
Assignee: Nicolas Liochon
Attachments: 9451.v1.patch
While running tests described in HBASE-9338, ran into this problem. The
hbase.status.listener.class was set to
org.apache.hadoop.hbase.client.ClusterStatusListener$MultiCastListener.
1. I had the meta server coming down
2. The metaSSH got triggered. The call chain:
2.1 verifyAndAssignMetaWithRetries
2.2 verifyMetaRegionLocation
2.3 waitForMetaServerConnection
2.4 getMetaServerConnection
2.5 getCachedConnection
2.6 HConnectionManager.getAdmin(serverName, false)
2.7 isDeadServer(serverName) - This is hardcoded to return 'false' when
the clusterStatusListener field is null. If clusterStatusListener is not null
(in my test), then it could return true in certain cases (and in this case,
indeed it should return true since the server is down). I am trying to
understand why it's hardcoded to 'false' for former case.
3. When isDeadServer returns true, the method
HConnectionManager.getAdmin(ServerName, boolean) throws
RegionServerStoppedException.
4. Finally, after the retries are over verifyAndAssignMetaWithRetries gives
up and the master aborts.
The methods in the above call chain don't handle
RegionServerStoppedException. Maybe something to look at...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9437) hbase.regionserver.checksum.verify default: true or false
[ https://issues.apache.org/jira/browse/HBASE-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon resolved HBASE-9437. Resolution: Fixed Fix Version/s: 0.96.0 0.98.0 Assignee: Nicolas Liochon hbase.regionserver.checksum.verify default: true or false - Key: HBASE-9437 URL: https://issues.apache.org/jira/browse/HBASE-9437 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.0 HRegionServer: default to false hbase-common/hbase-default.xml: true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9436) hbase.regionserver.handler.count default: 5, 10, 25, 30? pick one
[ https://issues.apache.org/jira/browse/HBASE-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9436: --- Status: Patch Available (was: Open) hbase.regionserver.handler.count default: 5, 10, 25, 30? pick one - Key: HBASE-9436 URL: https://issues.apache.org/jira/browse/HBASE-9436 Project: HBase Issue Type: Bug Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Attachments: 9436.v1.patch Below what we have today. I vote for 10. configuration.xml The default of 10 is rather low common/hbase-site namehbase.regionserver.handler.count/name value30/value server/hbase-site value5/value descriptionCount of RPC Server instances spun up on RegionServers Same property is used by the HMaster for count of master handlers. Default is 10. === HMaster.java int numHandlers = conf.getInt(hbase.master.handler.count, conf.getInt(hbase.regionserver.handler.count, 25)); HRegionServer.java hbase.regionserver.handler.count: conf.getInt(hbase.regionserver.handler.count, 10), -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8810) Bring in code constants in line with default xml's
[ https://issues.apache.org/jira/browse/HBASE-8810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760395#comment-13760395 ] Nicolas Liochon commented on HBASE-8810: here is the result of a pass on the current defaults: hbase.master.logcleaner.ttl ok hbase.master.catalog.timeout NOK not used anymore hbase.master.dns.interface ok hbase.master.dns.nameserver ok hbase.regionserver.port ok hbase.regionserver.info.port.auto NOK not used hbase.regionserver.msginterval ok hbase.regionserver.optionallogflushinterval ok hbase.regionserver.regionSplitLimit ok hbase.regionserver.logroll.period ok hbase.regionserver.logroll.errors.tolerated NOK 2 or 0 hbase.regionserver.hlog.reader.impl ok hbase.regionserver.global.memstore.upperLimit ok hbase.regionserver.global.memstore.lowerLimit NOK .38 or .35 hbase.regionserver.optionalcacheflushinterval ok hbase.regionserver.catalog.timeout NOK not used hbase.regionserver.dns.interface ok zookeeper.session.timeout NOK 3 minutes, 90s, 180 * 1000 zookeeper.znode.rootserver ok hbase.zookeeper.peerport ok hbase.zookeeper.leaderport ok hbase.zookeeper.useMulti NOK true false hbase.config.read.zookeeper.config ok hbase.client.write.buffer ok hbase.client.pause NOK 100 1000 hbase.client.retries.number NOK 31 35 hbase.client.scanner.caching ok hbase.client.keyvalue.maxsize NOK -1 and other hbase.client.scanner.timeout.period ok hbase.bulkload.retries.number ok hbase.balancer.period ok hbase.regions.slop NOK 0.01 .02 hbase.server.thread.wakefrequency NOK 1 6 hbase.server.versionfile.writeattempts ok hbase.hregion.memstore.flush.size ok hbase.hregion.preclose.flush.size ok hbase.hregion.memstore.block.multiplier ok hbase.hregion.memstore.mslab.enabled ok hbase.hregion.max.filesize ok hbase.hregion.majorcompaction ok hbase.hregion.majorcompaction.jitter ok hbase.hstore.compactionThreshold ok hbase.hstore.blockingStoreFiles NOK 10 7 hbase.hstore.blockingWaitTime ok hbase.hstore.compaction.max ok hbase.hstore.compaction.kv.max ok hbase.storescanner.parallel.seek.enable ok hbase.storescanner.parallel.seek.threads ok hfile.block.cache.size NOK 0.4 0.25 hfile.block.index.cacheonwrite ok hfile.format.version ok hfile.block.bloom.cacheonwrite ok hbase.rs.cacheblocksonwrite ok hbase.ipc.client.tcpnodelay ok hbase.hstore.bytes.per.checksum ok Bring in code constants in line with default xml's -- Key: HBASE-8810 URL: https://issues.apache.org/jira/browse/HBASE-8810 Project: HBase Issue Type: Bug Components: Usability Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 8810.txt, 8810v2.txt, hbase-default_to_java_constants.xsl, HBaseDefaultXMLConstants.java After the defaults were changed in the xml some constants were left the same. DEFAULT_HBASE_CLIENT_PAUSE for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9436) hbase.regionserver.handler.count default: 5, 10, 25, 30? pick one
[ https://issues.apache.org/jira/browse/HBASE-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760394#comment-13760394 ] Jean-Daniel Cryans commented on HBASE-9436: --- We could modify this further: bq. The default of 30 is rather low in order to prevent users from killing their region servers when using large write buffers with a high number of concurrent clients Since we now have protections against it and 30 ain't that low. hbase.regionserver.handler.count default: 5, 10, 25, 30? pick one - Key: HBASE-9436 URL: https://issues.apache.org/jira/browse/HBASE-9436 Project: HBase Issue Type: Bug Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Attachments: 9436.v1.patch Below what we have today. I vote for 10. configuration.xml The default of 10 is rather low common/hbase-site namehbase.regionserver.handler.count/name value30/value server/hbase-site value5/value descriptionCount of RPC Server instances spun up on RegionServers Same property is used by the HMaster for count of master handlers. Default is 10. === HMaster.java int numHandlers = conf.getInt(hbase.master.handler.count, conf.getInt(hbase.regionserver.handler.count, 25)); HRegionServer.java hbase.regionserver.handler.count: conf.getInt(hbase.regionserver.handler.count, 10), -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9449) document how to use shell enhancements from HBASE-5548
[ https://issues.apache.org/jira/browse/HBASE-9449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9449: -- Summary: document how to use shell enhancements from HBASE-5548 (was: document how to use shell 'list' command.) document how to use shell enhancements from HBASE-5548 -- Key: HBASE-9449 URL: https://issues.apache.org/jira/browse/HBASE-9449 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh HBASE-5548 introduced new behavior for shell commands like 'list' to make them act more ruby-like. There is no documentation for this in the refguide. We should 1) have an example in the shell sectoin 2) document the that new '= #xxx.x..' line is expected We can probably lift a bunch of docs from HBASE-5548. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-9449) document how to use shell enhancements from HBASE-5548
[ https://issues.apache.org/jira/browse/HBASE-9449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-9449: - Assignee: Jonathan Hsieh document how to use shell enhancements from HBASE-5548 -- Key: HBASE-9449 URL: https://issues.apache.org/jira/browse/HBASE-9449 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh HBASE-5548 introduced new behavior for shell commands like 'list' to make them act more ruby-like. There is no documentation for this in the refguide. We should 1) have an example in the shell sectoin 2) document the that new '= #xxx.x..' line is expected We can probably lift a bunch of docs from HBASE-5548. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9452) Simplify the configuration of the multicast notifier
Nicolas Liochon created HBASE-9452: -- Summary: Simplify the configuration of the multicast notifier Key: HBASE-9452 URL: https://issues.apache.org/jira/browse/HBASE-9452 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.0 As JD pointed it out, we not consistent in the naming. As well, it could be simpler to make it run. patch for trunk, but I would like to put it in the next 0.96 rc as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9452) Simplify the configuration of the multicast notifier
[ https://issues.apache.org/jira/browse/HBASE-9452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760401#comment-13760401 ] Jean-Daniel Cryans commented on HBASE-9452: --- +1 Simplify the configuration of the multicast notifier Key: HBASE-9452 URL: https://issues.apache.org/jira/browse/HBASE-9452 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.0 Attachments: 9452.v1.patch As JD pointed it out, we not consistent in the naming. As well, it could be simpler to make it run. patch for trunk, but I would like to put it in the next 0.96 rc as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9450) TestDistributedLogSplitting fails
[ https://issues.apache.org/jira/browse/HBASE-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760400#comment-13760400 ] Jeffrey Zhong commented on HBASE-9450: -- The patch looks good to me. +1. Thanks. TestDistributedLogSplitting fails - Key: HBASE-9450 URL: https://issues.apache.org/jira/browse/HBASE-9450 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jimmy Xiang Attachments: trunk-9450.patch Would you mind taking a look at a recent set of failures please [~jeffreyz]? It seems to be failing more recently of late: https://issues.apache.org/jira/browse/HBASE-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13759275#comment-13759275 https://builds.apache.org/job/PreCommit-HBASE-Build/7060//testReport/ https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/703/ Thank you. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9449) document how to use shell enhancements from HBASE-5548
[ https://issues.apache.org/jira/browse/HBASE-9449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9449: -- Status: Patch Available (was: Open) document how to use shell enhancements from HBASE-5548 -- Key: HBASE-9449 URL: https://issues.apache.org/jira/browse/HBASE-9449 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9449.patch HBASE-5548 introduced new behavior for shell commands like 'list' to make them act more ruby-like. There is no documentation for this in the refguide. We should 1) have an example in the shell sectoin 2) document the that new '= #xxx.x..' line is expected We can probably lift a bunch of docs from HBASE-5548. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9449) document how to use shell enhancements from HBASE-5548
[ https://issues.apache.org/jira/browse/HBASE-9449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9449: -- Attachment: hbase-9449.patch tested by running 'mvn site' and then 'mvn docbkx:generate-html' Looks good. document how to use shell enhancements from HBASE-5548 -- Key: HBASE-9449 URL: https://issues.apache.org/jira/browse/HBASE-9449 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9449.patch HBASE-5548 introduced new behavior for shell commands like 'list' to make them act more ruby-like. There is no documentation for this in the refguide. We should 1) have an example in the shell sectoin 2) document the that new '= #xxx.x..' line is expected We can probably lift a bunch of docs from HBASE-5548. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9452) Simplify the configuration of the multicast notifier
[ https://issues.apache.org/jira/browse/HBASE-9452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9452: --- Attachment: 9452.v1.patch Simplify the configuration of the multicast notifier Key: HBASE-9452 URL: https://issues.apache.org/jira/browse/HBASE-9452 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.0 Attachments: 9452.v1.patch As JD pointed it out, we not consistent in the naming. As well, it could be simpler to make it run. patch for trunk, but I would like to put it in the next 0.96 rc as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9454) HBaseAdmin#unassign() has incorrect @param argument
[
https://issues.apache.org/jira/browse/HBASE-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9454:
--
Description:
Here is the javadoc warning:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
was:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
HBaseAdmin#unassign() has incorrect @param argument
---
Key: HBASE-9454
URL: https://issues.apache.org/jira/browse/HBASE-9454
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Here is the javadoc warning:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9377) Backport HBASE- 9208 ReplicationLogCleaner slow at large scale
[ https://issues.apache.org/jira/browse/HBASE-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760483#comment-13760483 ] Lars Hofhansl commented on HBASE-9377: -- Comment from Dave: bq. Perhaps rather than changing FileCleanerDelegate we could introduce a new interface BatchFileCleanerDelegate or some such. Then configured cleaners that implement that interface can use it but others could still work by wrapping them. Anyone agree with that approach or have other suggestions? It seems that is what we should do. Backport HBASE- 9208 ReplicationLogCleaner slow at large scale Key: HBASE-9377 URL: https://issues.apache.org/jira/browse/HBASE-9377 Project: HBase Issue Type: Task Components: Replication Reporter: stack Assignee: Lars Hofhansl Fix For: 0.94.12 For [~lhofhansl] to make a call on. See end where Dave Latham talks about issues w/ patch in 0.94. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-9377) Backport HBASE- 9208 ReplicationLogCleaner slow at large scale
[ https://issues.apache.org/jira/browse/HBASE-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760483#comment-13760483 ] Lars Hofhansl edited comment on HBASE-9377 at 9/6/13 6:26 PM: -- Comment from [~davelatham]: bq. Perhaps rather than changing FileCleanerDelegate we could introduce a new interface BatchFileCleanerDelegate or some such. Then configured cleaners that implement that interface can use it but others could still work by wrapping them. Anyone agree with that approach or have other suggestions? It seems that is what we should do. was (Author: lhofhansl): Comment from Dave: bq. Perhaps rather than changing FileCleanerDelegate we could introduce a new interface BatchFileCleanerDelegate or some such. Then configured cleaners that implement that interface can use it but others could still work by wrapping them. Anyone agree with that approach or have other suggestions? It seems that is what we should do. Backport HBASE- 9208 ReplicationLogCleaner slow at large scale Key: HBASE-9377 URL: https://issues.apache.org/jira/browse/HBASE-9377 Project: HBase Issue Type: Task Components: Replication Reporter: stack Assignee: Lars Hofhansl Fix For: 0.94.12 For [~lhofhansl] to make a call on. See end where Dave Latham talks about issues w/ patch in 0.94. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9286) [0.94] ageOfLastShippedOp replication metric doesn't update if the slave regionserver is stalled
[ https://issues.apache.org/jira/browse/HBASE-9286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9286: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.94. Thanks Alex! [0.94] ageOfLastShippedOp replication metric doesn't update if the slave regionserver is stalled Key: HBASE-9286 URL: https://issues.apache.org/jira/browse/HBASE-9286 Project: HBase Issue Type: Bug Reporter: Alex Newman Assignee: Alex Newman Fix For: 0.94.12 Attachments: 0001-HBASE-9286.-ageOfLastShippedOp-replication-metric-do.patch In replicationmanager HRegionInterface rrs = getRS(); rrs.replicateLogEntries(Arrays.copyOf(this.entriesArray, currentNbEntries)); this.metrics.setAgeOfLastShippedOp( this.entriesArray[currentNbEntries-1].getKey().getWriteTime()); break; which makes sense, but is wrong. The problem is that rrs.replicateLogEntries will block for a very long time if the slave server is suspended or unavailable but not down. However this is easy to fix. We just need to call refreshAgeOfLastShippedOp(); on a regular basis, in a different thread. I've attached a patch which fixed this for cdh4. I can make one for trunk and the like as well if you need me to do but it's a small change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9454) HBaseAdmin#unassign() has incorrect @param argument
[
https://issues.apache.org/jira/browse/HBASE-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9454:
--
Attachment: 9454.txt
HBaseAdmin#unassign() has incorrect @param argument
---
Key: HBASE-9454
URL: https://issues.apache.org/jira/browse/HBASE-9454
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9454.txt
Here is the javadoc warning:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9454) HBaseAdmin#unassign() has incorrect @param argument
[
https://issues.apache.org/jira/browse/HBASE-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9454:
--
Status: Patch Available (was: Open)
HBaseAdmin#unassign() has incorrect @param argument
---
Key: HBASE-9454
URL: https://issues.apache.org/jira/browse/HBASE-9454
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9454.txt
Here is the javadoc warning:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9454) HBaseAdmin#unassign() has incorrect @param argument
[
https://issues.apache.org/jira/browse/HBASE-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760489#comment-13760489
]
Jimmy Xiang commented on HBASE-9454:
Can we change the parameter name instead?
HBaseAdmin#unassign() has incorrect @param argument
---
Key: HBASE-9454
URL: https://issues.apache.org/jira/browse/HBASE-9454
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9454.txt
Here is the javadoc warning:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9438) Enhance hbase shell un/assign to take encoded region name
[ https://issues.apache.org/jira/browse/HBASE-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang resolved HBASE-9438. Resolution: Fixed Revert is not trivial for this one. Enhance hbase shell un/assign to take encoded region name - Key: HBASE-9438 URL: https://issues.apache.org/jira/browse/HBASE-9438 Project: HBase Issue Type: Improvement Components: shell Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.98.0, 0.96.0 Attachments: trunk-9438.patch, trunk-9438_v2.patch Currently assign/unassign in hbase shell takes only region name. It is hard to type the region name especially when there is some non-printable character. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9441) Intermittent TestRegionPlacement#testRegionPlacement failure
[
https://issues.apache.org/jira/browse/HBASE-9441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760443#comment-13760443
]
Devaraj Das commented on HBASE-9441:
+1 (although now that you don't kill RS with the namespace region, the wait for
RIT is not needed. The while loop will cover that).
Intermittent TestRegionPlacement#testRegionPlacement failure
Key: HBASE-9441
URL: https://issues.apache.org/jira/browse/HBASE-9441
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9441-v1.txt, 9441-v2.txt, 9441-v3.txt, 9441-v4.txt,
9441-v5.txt, org.apache.hadoop.hbase.master.TestRegionPlacement-output.txt
Occasionally TestRegionPlacement#testRegionPlacement fails due to no region
found on secondary / tertiary region server.
In the following test output, 10359edcfb1a761f94ab6d5b559c2278 was the region
from the killed region server:
{code}
2013-09-04 23:28:25,127 INFO [pool-1-thread-1] util.FSUtils(1878): Scan DFS
for locality info takes 38 ms
Region Assignment Verification for Table: testRegionAssignment
Total regions : 10
Total regions on favored nodes 10
Total regions on PRIMARY region servers: 10
Total regions on SECONDARY region servers: 0
Total regions on TERTIARY region servers: 0
...
2013-09-04 23:28:25,130 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
path=/hbase/region-in-transition/10359edcfb1a761f94ab6d5b559c2278
2013-09-04 23:28:25,130 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
zookeeper.ZKAssign(861): regionserver:49841-0x140eb4dc4ef0007 Transitioned
node 10359edcfb1a761f94ab6d5b559c2278 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
handler.OpenRegionHandler(386): Transitioned 10359edcfb1a761f94ab6d5b559c2278
to OPENED in zk on kiyo.gq1.ygridcore.net,49841,1378337278470
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
handler.OpenRegionHandler(187): Opened
hbase:namespace,,1378337284888.10359edcfb1a761f94ab6d5b559c2278. on
kiyo.gq1.ygridcore.net,49841,1378337278470
2013-09-04 23:28:25,132 DEBUG [AM.ZK.Worker-pool2-t6]
master.AssignmentManager(801): Handling transition=RS_ZK_REGION_OPENED,
server=kiyo.gq1.ygridcore.net,49841,1378337278470,
region=10359edcfb1a761f94ab6d5b559c2278, current state from region state map
={10359edcfb1a761f94ab6d5b559c2278 state=OPENING, ts=1378337305025,
server=kiyo.gq1.ygridcore.net,49841,1378337278470}
2013-09-04 23:28:25,132 INFO [AM.ZK.Worker-pool2-t6]
master.RegionStates(263): Transitioned from {10359edcfb1a761f94ab6d5b559c2278
state=OPENING, ts=1378337305025,
server=kiyo.gq1.ygridcore.net,49841,1378337278470} to
{10359edcfb1a761f94ab6d5b559c2278 state=OPEN, ts=1378337305132,
server=kiyo.gq1.ygridcore.net,49841,1378337278470}
2013-09-04 23:28:25,132 DEBUG [AM.ZK.Worker-pool2-t6]
handler.OpenedRegionHandler(150): Handling OPENED event for
10359edcfb1a761f94ab6d5b559c2278 from
kiyo.gq1.ygridcore.net,49841,1378337278470; deleting unassigned node
2013-09-04 23:28:25,132 DEBUG [AM.ZK.Worker-pool2-t6]
zookeeper.ZKAssign(405): master:37502-0x140eb4dc4ef Deleting existing
unassigned node 10359edcfb1a761f94ab6d5b559c2278 in expected state
RS_ZK_REGION_OPENED
2013-09-04 23:28:25,188 INFO [pool-1-thread-1] hbase.ResourceChecker(171):
after: master.TestRegionPlacement#testRegionPlacement Thread=873 (was 814)
Potentially hanging thread: RS_OPEN_REGION-kiyo:53373-2
...
- Thread LEAK? -, OpenFileDescriptor=1061 (was 1033) - OpenFileDescriptor
LEAK? -, MaxFileDescriptor=32768 (was 32768), SystemLoadAverage=549 (was
609), ProcessCount=489 (was 489), AvailableMemoryMB=9288 (was 9107) -
AvailableMemoryMB LEAK? -, ConnectionCount=23 (was 22) - ConnectionCount
LEAK? -
2013-09-04 23:28:25,189 WARN [pool-1-thread-1] hbase.ResourceChecker(134):
Thread=873 is superior to 500
2013-09-04 23:28:25,190 WARN [pool-1-thread-1] hbase.ResourceChecker(134):
OpenFileDescriptor=1061 is superior to 1024
2013-09-04 23:28:25,223 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeDeleted, state=SyncConnected,
path=/hbase/region-in-transition/10359edcfb1a761f94ab6d5b559c2278
2013-09-04 23:28:25,224 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected,
path=/hbase/region-in-transition
2013-09-04 23:28:25,224 DEBUG
[jira] [Updated] (HBASE-9450) TestDistributedLogSplitting fails
[ https://issues.apache.org/jira/browse/HBASE-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-9450: --- Resolution: Fixed Fix Version/s: 0.96.0 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Integrated into trunk and 0.96. These tests look ok now. Thanks. TestDistributedLogSplitting fails - Key: HBASE-9450 URL: https://issues.apache.org/jira/browse/HBASE-9450 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jimmy Xiang Fix For: 0.98.0, 0.96.0 Attachments: trunk-9450.patch Would you mind taking a look at a recent set of failures please [~jeffreyz]? It seems to be failing more recently of late: https://issues.apache.org/jira/browse/HBASE-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13759275#comment-13759275 https://builds.apache.org/job/PreCommit-HBASE-Build/7060//testReport/ https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/703/ Thank you. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9301) Default hbase.dynamic.jars.dir to hbase.rootdir/jars
[ https://issues.apache.org/jira/browse/HBASE-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760446#comment-13760446 ] Vasu Mariyala commented on HBASE-9301: -- Yes, I would post the patch soon. Default hbase.dynamic.jars.dir to hbase.rootdir/jars Key: HBASE-9301 URL: https://issues.apache.org/jira/browse/HBASE-9301 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.2, 0.94.12, 0.96.0 Reporter: James Taylor Assignee: Vasu Mariyala Fix For: 0.98.0, 0.94.12, 0.96.1 Attachments: 0.94-HBASE-9301.patch, HBASE-9301.patch A reasonable default for hbase.dynamic.jars.dir would be hbase.rootdir/jars so that folks aren't forced to edit their hbase-sites.xml to take advantage of the new, cool feature to load coprocessor/custom filter jars out of HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9441) Intermittent TestRegionPlacement#testRegionPlacement failure
[
https://issues.apache.org/jira/browse/HBASE-9441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9441:
--
Attachment: 9441-v6.txt
Patch v6 addresses Devaraj's review comments.
Intermittent TestRegionPlacement#testRegionPlacement failure
Key: HBASE-9441
URL: https://issues.apache.org/jira/browse/HBASE-9441
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9441-v1.txt, 9441-v2.txt, 9441-v3.txt, 9441-v4.txt,
9441-v5.txt, 9441-v6.txt,
org.apache.hadoop.hbase.master.TestRegionPlacement-output.txt
Occasionally TestRegionPlacement#testRegionPlacement fails due to no region
found on secondary / tertiary region server.
In the following test output, 10359edcfb1a761f94ab6d5b559c2278 was the region
from the killed region server:
{code}
2013-09-04 23:28:25,127 INFO [pool-1-thread-1] util.FSUtils(1878): Scan DFS
for locality info takes 38 ms
Region Assignment Verification for Table: testRegionAssignment
Total regions : 10
Total regions on favored nodes 10
Total regions on PRIMARY region servers: 10
Total regions on SECONDARY region servers: 0
Total regions on TERTIARY region servers: 0
...
2013-09-04 23:28:25,130 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
path=/hbase/region-in-transition/10359edcfb1a761f94ab6d5b559c2278
2013-09-04 23:28:25,130 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
zookeeper.ZKAssign(861): regionserver:49841-0x140eb4dc4ef0007 Transitioned
node 10359edcfb1a761f94ab6d5b559c2278 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
handler.OpenRegionHandler(386): Transitioned 10359edcfb1a761f94ab6d5b559c2278
to OPENED in zk on kiyo.gq1.ygridcore.net,49841,1378337278470
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
handler.OpenRegionHandler(187): Opened
hbase:namespace,,1378337284888.10359edcfb1a761f94ab6d5b559c2278. on
kiyo.gq1.ygridcore.net,49841,1378337278470
2013-09-04 23:28:25,132 DEBUG [AM.ZK.Worker-pool2-t6]
master.AssignmentManager(801): Handling transition=RS_ZK_REGION_OPENED,
server=kiyo.gq1.ygridcore.net,49841,1378337278470,
region=10359edcfb1a761f94ab6d5b559c2278, current state from region state map
={10359edcfb1a761f94ab6d5b559c2278 state=OPENING, ts=1378337305025,
server=kiyo.gq1.ygridcore.net,49841,1378337278470}
2013-09-04 23:28:25,132 INFO [AM.ZK.Worker-pool2-t6]
master.RegionStates(263): Transitioned from {10359edcfb1a761f94ab6d5b559c2278
state=OPENING, ts=1378337305025,
server=kiyo.gq1.ygridcore.net,49841,1378337278470} to
{10359edcfb1a761f94ab6d5b559c2278 state=OPEN, ts=1378337305132,
server=kiyo.gq1.ygridcore.net,49841,1378337278470}
2013-09-04 23:28:25,132 DEBUG [AM.ZK.Worker-pool2-t6]
handler.OpenedRegionHandler(150): Handling OPENED event for
10359edcfb1a761f94ab6d5b559c2278 from
kiyo.gq1.ygridcore.net,49841,1378337278470; deleting unassigned node
2013-09-04 23:28:25,132 DEBUG [AM.ZK.Worker-pool2-t6]
zookeeper.ZKAssign(405): master:37502-0x140eb4dc4ef Deleting existing
unassigned node 10359edcfb1a761f94ab6d5b559c2278 in expected state
RS_ZK_REGION_OPENED
2013-09-04 23:28:25,188 INFO [pool-1-thread-1] hbase.ResourceChecker(171):
after: master.TestRegionPlacement#testRegionPlacement Thread=873 (was 814)
Potentially hanging thread: RS_OPEN_REGION-kiyo:53373-2
...
- Thread LEAK? -, OpenFileDescriptor=1061 (was 1033) - OpenFileDescriptor
LEAK? -, MaxFileDescriptor=32768 (was 32768), SystemLoadAverage=549 (was
609), ProcessCount=489 (was 489), AvailableMemoryMB=9288 (was 9107) -
AvailableMemoryMB LEAK? -, ConnectionCount=23 (was 22) - ConnectionCount
LEAK? -
2013-09-04 23:28:25,189 WARN [pool-1-thread-1] hbase.ResourceChecker(134):
Thread=873 is superior to 500
2013-09-04 23:28:25,190 WARN [pool-1-thread-1] hbase.ResourceChecker(134):
OpenFileDescriptor=1061 is superior to 1024
2013-09-04 23:28:25,223 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeDeleted, state=SyncConnected,
path=/hbase/region-in-transition/10359edcfb1a761f94ab6d5b559c2278
2013-09-04 23:28:25,224 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected,
path=/hbase/region-in-transition
2013-09-04 23:28:25,224 DEBUG [AM.ZK.Worker-pool2-t6]
zookeeper.ZKAssign(434): master:37502-0x140eb4dc4ef Deleted unassigned
node
[jira] [Commented] (HBASE-9452) Simplify the configuration of the multicast notifier
[
https://issues.apache.org/jira/browse/HBASE-9452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760372#comment-13760372
]
Hadoop QA commented on HBASE-9452:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12601839/9452.v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7068//console
This message is automatically generated.
Simplify the configuration of the multicast notifier
Key: HBASE-9452
URL: https://issues.apache.org/jira/browse/HBASE-9452
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.98.0, 0.96.0
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Priority: Minor
Fix For: 0.98.0, 0.96.0
Attachments: 9452.v1.patch
As JD pointed it out, we not consistent in the naming. As well, it could be
simpler to make it run.
patch for trunk, but I would like to put it in the next 0.96 rc as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9454) HBaseAdmin#unassign() has incorrect @param argument
[
https://issues.apache.org/jira/browse/HBASE-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760503#comment-13760503
]
Jimmy Xiang commented on HBASE-9454:
+1
HBaseAdmin#unassign() has incorrect @param argument
---
Key: HBASE-9454
URL: https://issues.apache.org/jira/browse/HBASE-9454
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9454.txt, 9454-v2.txt
Here is the javadoc warning:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9449) document how to use shell enhancements from HBASE-5548
[ https://issues.apache.org/jira/browse/HBASE-9449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9449: -- Attachment: shell.html Attached shell.html file generated with the new documentation. document how to use shell enhancements from HBASE-5548 -- Key: HBASE-9449 URL: https://issues.apache.org/jira/browse/HBASE-9449 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9449.patch, shell.html HBASE-5548 introduced new behavior for shell commands like 'list' to make them act more ruby-like. There is no documentation for this in the refguide. We should 1) have an example in the shell sectoin 2) document the that new '= #xxx.x..' line is expected We can probably lift a bunch of docs from HBASE-5548. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9377) Backport HBASE- 9208 ReplicationLogCleaner slow at large scale
[ https://issues.apache.org/jira/browse/HBASE-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760505#comment-13760505 ] Dave Latham commented on HBASE-9377: Sorry, been pulled on to some more urgent work at the moment, so I will try to get back to this within a week or so. Feel free to punt from 0.94.12. Backport HBASE- 9208 ReplicationLogCleaner slow at large scale Key: HBASE-9377 URL: https://issues.apache.org/jira/browse/HBASE-9377 Project: HBase Issue Type: Task Components: Replication Reporter: stack Assignee: Lars Hofhansl Fix For: 0.94.12 For [~lhofhansl] to make a call on. See end where Dave Latham talks about issues w/ patch in 0.94. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9343) Implement stateless scanner for Stargate
[ https://issues.apache.org/jira/browse/HBASE-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vandana Ayyalasomayajula updated HBASE-9343: Attachment: HBASE-9343_94.01.patch Implement stateless scanner for Stargate Key: HBASE-9343 URL: https://issues.apache.org/jira/browse/HBASE-9343 Project: HBase Issue Type: Improvement Components: REST Affects Versions: 0.94.11 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-9343_94.00.patch, HBASE-9343_94.01.patch The current scanner implementation for scanner stores state and hence not very suitable for REST server failure scenarios. The current JIRA proposes to implement a stateless scanner. In the first version of the patch, a new resource class ScanResource has been added and all the scan parameters will be specified as query params. The following are the scan parameters startrow - The start row for the scan. endrow - The end row for the scan. columns - The columns to scan. starttime, endtime - To only retrieve columns within a specific range of version timestamps,both start and end time must be specified. maxversions - To limit the number of versions of each column to be returned. batchsize - To limit the maximum number of values returned for each call to next(). limit - The number of rows to return in the scan operation. More on start row, end row and limit parameters. 1. If start row, end row and limit not specified, then the whole table will be scanned. 2. If start row and limit (say N) is specified, then the scan operation will return N rows from the start row specified. 3. If only limit parameter is specified, then the scan operation will return N rows from the start of the table. 4. If limit and end row are specified, then the scan operation will return N rows from start of table till the end row. If the end row is reached before N rows ( say M and M lt; N ), then M rows will be returned to the user. 5. If start row, end row and limit (say N ) are specified and N lt; number of rows between start row and end row, then N rows from start row will be returned to the user. If N gt; (number of rows between start row and end row (say M), then M number of rows will be returned to the user. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9343) Implement stateless scanner for Stargate
[ https://issues.apache.org/jira/browse/HBASE-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vandana Ayyalasomayajula updated HBASE-9343: Attachment: (was: HBASE-9364.01.patch) Implement stateless scanner for Stargate Key: HBASE-9343 URL: https://issues.apache.org/jira/browse/HBASE-9343 Project: HBase Issue Type: Improvement Components: REST Affects Versions: 0.94.11 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-9343_94.00.patch, HBASE-9343_94.01.patch The current scanner implementation for scanner stores state and hence not very suitable for REST server failure scenarios. The current JIRA proposes to implement a stateless scanner. In the first version of the patch, a new resource class ScanResource has been added and all the scan parameters will be specified as query params. The following are the scan parameters startrow - The start row for the scan. endrow - The end row for the scan. columns - The columns to scan. starttime, endtime - To only retrieve columns within a specific range of version timestamps,both start and end time must be specified. maxversions - To limit the number of versions of each column to be returned. batchsize - To limit the maximum number of values returned for each call to next(). limit - The number of rows to return in the scan operation. More on start row, end row and limit parameters. 1. If start row, end row and limit not specified, then the whole table will be scanned. 2. If start row and limit (say N) is specified, then the scan operation will return N rows from the start row specified. 3. If only limit parameter is specified, then the scan operation will return N rows from the start of the table. 4. If limit and end row are specified, then the scan operation will return N rows from start of table till the end row. If the end row is reached before N rows ( say M and M lt; N ), then M rows will be returned to the user. 5. If start row, end row and limit (say N ) are specified and N lt; number of rows between start row and end row, then N rows from start row will be returned to the user. If N gt; (number of rows between start row and end row (say M), then M number of rows will be returned to the user. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9441) Intermittent TestRegionPlacement#testRegionPlacement failure
[
https://issues.apache.org/jira/browse/HBASE-9441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760537#comment-13760537
]
Hadoop QA commented on HBASE-9441:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12601869/9441-v6.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7072//console
This message is automatically generated.
Intermittent TestRegionPlacement#testRegionPlacement failure
Key: HBASE-9441
URL: https://issues.apache.org/jira/browse/HBASE-9441
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9441-v1.txt, 9441-v2.txt, 9441-v3.txt, 9441-v4.txt,
9441-v5.txt, 9441-v6.txt,
org.apache.hadoop.hbase.master.TestRegionPlacement-output.txt
Occasionally TestRegionPlacement#testRegionPlacement fails due to no region
found on secondary / tertiary region server.
In the following test output, 10359edcfb1a761f94ab6d5b559c2278 was the region
from the killed region server:
{code}
2013-09-04 23:28:25,127 INFO [pool-1-thread-1] util.FSUtils(1878): Scan DFS
for locality info takes 38 ms
Region Assignment Verification for Table: testRegionAssignment
Total regions : 10
Total regions on favored nodes 10
Total regions on PRIMARY region servers: 10
Total regions on SECONDARY region servers: 0
Total regions on TERTIARY region servers: 0
...
2013-09-04 23:28:25,130 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
path=/hbase/region-in-transition/10359edcfb1a761f94ab6d5b559c2278
2013-09-04 23:28:25,130 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
zookeeper.ZKAssign(861): regionserver:49841-0x140eb4dc4ef0007 Transitioned
node 10359edcfb1a761f94ab6d5b559c2278 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
handler.OpenRegionHandler(386): Transitioned 10359edcfb1a761f94ab6d5b559c2278
to OPENED in zk on kiyo.gq1.ygridcore.net,49841,1378337278470
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
handler.OpenRegionHandler(187): Opened
[jira] [Created] (HBASE-9454) HBaseAdmin#unassign() has incorrect @param argument
Ted Yu created HBASE-9454:
-
Summary: HBaseAdmin#unassign() has incorrect @param argument
Key: HBASE-9454
URL: https://issues.apache.org/jira/browse/HBASE-9454
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9343) Implement stateless scanner for Stargate
[
https://issues.apache.org/jira/browse/HBASE-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760478#comment-13760478
]
Hadoop QA commented on HBASE-9343:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12601871/HBASE-9343_94.01.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7073//console
This message is automatically generated.
Implement stateless scanner for Stargate
Key: HBASE-9343
URL: https://issues.apache.org/jira/browse/HBASE-9343
Project: HBase
Issue Type: Improvement
Components: REST
Affects Versions: 0.94.11
Reporter: Vandana Ayyalasomayajula
Assignee: Vandana Ayyalasomayajula
Priority: Minor
Attachments: HBASE-9343_94.00.patch, HBASE-9343_94.01.patch
The current scanner implementation for scanner stores state and hence not
very suitable for REST server failure scenarios. The current JIRA proposes to
implement a stateless scanner. In the first version of the patch, a new
resource class ScanResource has been added and all the scan parameters will
be specified as query params.
The following are the scan parameters
startrow - The start row for the scan.
endrow - The end row for the scan.
columns - The columns to scan.
starttime, endtime - To only retrieve columns within a specific range of
version timestamps,both start and end time must be specified.
maxversions - To limit the number of versions of each column to be returned.
batchsize - To limit the maximum number of values returned for each call to
next().
limit - The number of rows to return in the scan operation.
More on start row, end row and limit parameters.
1. If start row, end row and limit not specified, then the whole table will
be scanned.
2. If start row and limit (say N) is specified, then the scan operation will
return N rows from the start row specified.
3. If only limit parameter is specified, then the scan operation will return
N rows from the start of the table.
4. If limit and end row are specified, then the scan operation will return N
rows from start of table till the end row. If the end row is
reached before N rows ( say M and M lt; N ), then M rows will be returned to
the user.
5. If start row, end row and limit (say N ) are specified and N lt; number
of rows between start row and end row, then N rows from start row
will be returned to the user. If N gt; (number of rows between start row and
end row (say M), then M number of rows will be returned to the
user.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9301) Default hbase.dynamic.jars.dir to hbase.rootdir/jars
[ https://issues.apache.org/jira/browse/HBASE-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760411#comment-13760411 ] Lars Hofhansl commented on HBASE-9301: -- It seems external software would need to be changed anyway, so migrating this makes less sense than the .logs directory. No harm moving it over, though. [~vasu.mariy...@gmail.com], do you have time to do that? Default hbase.dynamic.jars.dir to hbase.rootdir/jars Key: HBASE-9301 URL: https://issues.apache.org/jira/browse/HBASE-9301 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.2, 0.94.12, 0.96.0 Reporter: James Taylor Assignee: Vasu Mariyala Attachments: 0.94-HBASE-9301.patch, HBASE-9301.patch A reasonable default for hbase.dynamic.jars.dir would be hbase.rootdir/jars so that folks aren't forced to edit their hbase-sites.xml to take advantage of the new, cool feature to load coprocessor/custom filter jars out of HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9453) make dev-support/generate-hadoopX-poms.sh have exec perms.
[
https://issues.apache.org/jira/browse/HBASE-9453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-9453:
--
Attachment: hbase-9453.patch
trivial patch, committing to 0.96/trunk.
make dev-support/generate-hadoopX-poms.sh have exec perms.
--
Key: HBASE-9453
URL: https://issues.apache.org/jira/browse/HBASE-9453
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.95.2
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Minor
Fix For: 0.98.0, 0.96.1
Attachments: hbase-9453.patch
currently:
{code}
jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
-rw-r--r-- 1 jon jon 5216 2013-09-06 10:45
dev-support/generate-hadoopX-poms.sh
{code}
after patch:
{code}
jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
-rwxr-xr-x 1 jon jon 5216 2013-08-07 18:05
dev-support/generate-hadoopX-poms.sh
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9286) [0.94] ageOfLastShippedOp replication metric doesn't update if the slave regionserver is stalled
[ https://issues.apache.org/jira/browse/HBASE-9286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760543#comment-13760543 ] Hudson commented on HBASE-9286: --- FAILURE: Integrated in HBase-0.94-security #285 (See [https://builds.apache.org/job/HBase-0.94-security/285/]) HBASE-9286 [0.94] ageOfLastShippedOp replication metric doesn't update if the slave regionserver is stalled (Alex Newman) (larsh: rev 1520646) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceMetrics.java [0.94] ageOfLastShippedOp replication metric doesn't update if the slave regionserver is stalled Key: HBASE-9286 URL: https://issues.apache.org/jira/browse/HBASE-9286 Project: HBase Issue Type: Bug Reporter: Alex Newman Assignee: Alex Newman Fix For: 0.94.12 Attachments: 0001-HBASE-9286.-ageOfLastShippedOp-replication-metric-do.patch In replicationmanager HRegionInterface rrs = getRS(); rrs.replicateLogEntries(Arrays.copyOf(this.entriesArray, currentNbEntries)); this.metrics.setAgeOfLastShippedOp( this.entriesArray[currentNbEntries-1].getKey().getWriteTime()); break; which makes sense, but is wrong. The problem is that rrs.replicateLogEntries will block for a very long time if the slave server is suspended or unavailable but not down. However this is easy to fix. We just need to call refreshAgeOfLastShippedOp(); on a regular basis, in a different thread. I've attached a patch which fixed this for cdh4. I can make one for trunk and the like as well if you need me to do but it's a small change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9436) hbase.regionserver.handler.count default: 5, 10, 25, 30? pick one
[ https://issues.apache.org/jira/browse/HBASE-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760415#comment-13760415 ] Nicolas Liochon commented on HBASE-9436: bq. Since we now have protections against it and 30 ain't that low. Agreed, will do. hbase.regionserver.handler.count default: 5, 10, 25, 30? pick one - Key: HBASE-9436 URL: https://issues.apache.org/jira/browse/HBASE-9436 Project: HBase Issue Type: Bug Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Attachments: 9436.v1.patch Below what we have today. I vote for 10. configuration.xml The default of 10 is rather low common/hbase-site namehbase.regionserver.handler.count/name value30/value server/hbase-site value5/value descriptionCount of RPC Server instances spun up on RegionServers Same property is used by the HMaster for count of master handlers. Default is 10. === HMaster.java int numHandlers = conf.getInt(hbase.master.handler.count, conf.getInt(hbase.regionserver.handler.count, 25)); HRegionServer.java hbase.regionserver.handler.count: conf.getInt(hbase.regionserver.handler.count, 10), -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9450) TestDistributedLogSplitting fails
[ https://issues.apache.org/jira/browse/HBASE-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760540#comment-13760540 ] Hudson commented on HBASE-9450: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #714 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/714/]) HBASE-9450 TestDistributedLogSplitting fails (jxiang: rev 1520613) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java TestDistributedLogSplitting fails - Key: HBASE-9450 URL: https://issues.apache.org/jira/browse/HBASE-9450 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jimmy Xiang Fix For: 0.98.0, 0.96.0 Attachments: trunk-9450.patch Would you mind taking a look at a recent set of failures please [~jeffreyz]? It seems to be failing more recently of late: https://issues.apache.org/jira/browse/HBASE-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13759275#comment-13759275 https://builds.apache.org/job/PreCommit-HBASE-Build/7060//testReport/ https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/703/ Thank you. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9437) hbase.regionserver.checksum.verify default: true or false
[ https://issues.apache.org/jira/browse/HBASE-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760539#comment-13760539 ] Hudson commented on HBASE-9437: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #714 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/714/]) HBASE-9437 hbase.regionserver.checksum.verify default is true (nkeywal: rev 1520634) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java hbase.regionserver.checksum.verify default: true or false - Key: HBASE-9437 URL: https://issues.apache.org/jira/browse/HBASE-9437 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.0 HRegionServer: default to false hbase-common/hbase-default.xml: true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9450) TestDistributedLogSplitting fails
[ https://issues.apache.org/jira/browse/HBASE-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760418#comment-13760418 ] Hudson commented on HBASE-9450: --- FAILURE: Integrated in HBase-TRUNK #4475 (See [https://builds.apache.org/job/HBase-TRUNK/4475/]) HBASE-9450 TestDistributedLogSplitting fails (jxiang: rev 1520613) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java TestDistributedLogSplitting fails - Key: HBASE-9450 URL: https://issues.apache.org/jira/browse/HBASE-9450 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jimmy Xiang Attachments: trunk-9450.patch Would you mind taking a look at a recent set of failures please [~jeffreyz]? It seems to be failing more recently of late: https://issues.apache.org/jira/browse/HBASE-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13759275#comment-13759275 https://builds.apache.org/job/PreCommit-HBASE-Build/7060//testReport/ https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/703/ Thank you. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9441) Intermittent TestRegionPlacement#testRegionPlacement failure
[
https://issues.apache.org/jira/browse/HBASE-9441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760542#comment-13760542
]
Ted Yu commented on HBASE-9441:
---
Integrated to trunk.
Thanks for the review.
Intermittent TestRegionPlacement#testRegionPlacement failure
Key: HBASE-9441
URL: https://issues.apache.org/jira/browse/HBASE-9441
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9441-v1.txt, 9441-v2.txt, 9441-v3.txt, 9441-v4.txt,
9441-v5.txt, 9441-v6.txt,
org.apache.hadoop.hbase.master.TestRegionPlacement-output.txt
Occasionally TestRegionPlacement#testRegionPlacement fails due to no region
found on secondary / tertiary region server.
In the following test output, 10359edcfb1a761f94ab6d5b559c2278 was the region
from the killed region server:
{code}
2013-09-04 23:28:25,127 INFO [pool-1-thread-1] util.FSUtils(1878): Scan DFS
for locality info takes 38 ms
Region Assignment Verification for Table: testRegionAssignment
Total regions : 10
Total regions on favored nodes 10
Total regions on PRIMARY region servers: 10
Total regions on SECONDARY region servers: 0
Total regions on TERTIARY region servers: 0
...
2013-09-04 23:28:25,130 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
path=/hbase/region-in-transition/10359edcfb1a761f94ab6d5b559c2278
2013-09-04 23:28:25,130 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
zookeeper.ZKAssign(861): regionserver:49841-0x140eb4dc4ef0007 Transitioned
node 10359edcfb1a761f94ab6d5b559c2278 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
handler.OpenRegionHandler(386): Transitioned 10359edcfb1a761f94ab6d5b559c2278
to OPENED in zk on kiyo.gq1.ygridcore.net,49841,1378337278470
2013-09-04 23:28:25,131 DEBUG [RS_OPEN_REGION-kiyo:49841-1]
handler.OpenRegionHandler(187): Opened
hbase:namespace,,1378337284888.10359edcfb1a761f94ab6d5b559c2278. on
kiyo.gq1.ygridcore.net,49841,1378337278470
2013-09-04 23:28:25,132 DEBUG [AM.ZK.Worker-pool2-t6]
master.AssignmentManager(801): Handling transition=RS_ZK_REGION_OPENED,
server=kiyo.gq1.ygridcore.net,49841,1378337278470,
region=10359edcfb1a761f94ab6d5b559c2278, current state from region state map
={10359edcfb1a761f94ab6d5b559c2278 state=OPENING, ts=1378337305025,
server=kiyo.gq1.ygridcore.net,49841,1378337278470}
2013-09-04 23:28:25,132 INFO [AM.ZK.Worker-pool2-t6]
master.RegionStates(263): Transitioned from {10359edcfb1a761f94ab6d5b559c2278
state=OPENING, ts=1378337305025,
server=kiyo.gq1.ygridcore.net,49841,1378337278470} to
{10359edcfb1a761f94ab6d5b559c2278 state=OPEN, ts=1378337305132,
server=kiyo.gq1.ygridcore.net,49841,1378337278470}
2013-09-04 23:28:25,132 DEBUG [AM.ZK.Worker-pool2-t6]
handler.OpenedRegionHandler(150): Handling OPENED event for
10359edcfb1a761f94ab6d5b559c2278 from
kiyo.gq1.ygridcore.net,49841,1378337278470; deleting unassigned node
2013-09-04 23:28:25,132 DEBUG [AM.ZK.Worker-pool2-t6]
zookeeper.ZKAssign(405): master:37502-0x140eb4dc4ef Deleting existing
unassigned node 10359edcfb1a761f94ab6d5b559c2278 in expected state
RS_ZK_REGION_OPENED
2013-09-04 23:28:25,188 INFO [pool-1-thread-1] hbase.ResourceChecker(171):
after: master.TestRegionPlacement#testRegionPlacement Thread=873 (was 814)
Potentially hanging thread: RS_OPEN_REGION-kiyo:53373-2
...
- Thread LEAK? -, OpenFileDescriptor=1061 (was 1033) - OpenFileDescriptor
LEAK? -, MaxFileDescriptor=32768 (was 32768), SystemLoadAverage=549 (was
609), ProcessCount=489 (was 489), AvailableMemoryMB=9288 (was 9107) -
AvailableMemoryMB LEAK? -, ConnectionCount=23 (was 22) - ConnectionCount
LEAK? -
2013-09-04 23:28:25,189 WARN [pool-1-thread-1] hbase.ResourceChecker(134):
Thread=873 is superior to 500
2013-09-04 23:28:25,190 WARN [pool-1-thread-1] hbase.ResourceChecker(134):
OpenFileDescriptor=1061 is superior to 1024
2013-09-04 23:28:25,223 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeDeleted, state=SyncConnected,
path=/hbase/region-in-transition/10359edcfb1a761f94ab6d5b559c2278
2013-09-04 23:28:25,224 DEBUG [pool-1-thread-1-EventThread]
zookeeper.ZooKeeperWatcher(312): master:37502-0x140eb4dc4ef Received
ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected,
path=/hbase/region-in-transition
2013-09-04 23:28:25,224 DEBUG [AM.ZK.Worker-pool2-t6]
zookeeper.ZKAssign(434): master:37502-0x140eb4dc4ef Deleted
[jira] [Commented] (HBASE-9453) make dev-support/generate-hadoopX-poms.sh have exec perms.
[
https://issues.apache.org/jira/browse/HBASE-9453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760426#comment-13760426
]
Jonathan Hsieh commented on HBASE-9453:
---
hm.. for svn you commit' this by executing:
{code}
svn propset svn:executable ON dev-support/generate-hadoopX-poms.sh
{code}
svn doesn't update this for you on an update but if you delete the file it
comes back with the -x perms.
make dev-support/generate-hadoopX-poms.sh have exec perms.
--
Key: HBASE-9453
URL: https://issues.apache.org/jira/browse/HBASE-9453
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.95.2
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Minor
Fix For: 0.98.0, 0.96.1
Attachments: hbase-9453.patch
currently:
{code}
jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
-rw-r--r-- 1 jon jon 5216 2013-09-06 10:45
dev-support/generate-hadoopX-poms.sh
{code}
after patch:
{code}
jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
-rwxr-xr-x 1 jon jon 5216 2013-08-07 18:05
dev-support/generate-hadoopX-poms.sh
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9453) make dev-support/generate-hadoopX-poms.sh have exec perms.
Jonathan Hsieh created HBASE-9453:
-
Summary: make dev-support/generate-hadoopX-poms.sh have exec perms.
Key: HBASE-9453
URL: https://issues.apache.org/jira/browse/HBASE-9453
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.95.2
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Minor
Fix For: 0.98.0, 0.96.1
currently:
{code}
jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
-rw-r--r-- 1 jon jon 5216 2013-09-06 10:45 dev-support/generate-hadoopX-poms.sh
{code}
after patch:
{code}
jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
-rwxr-xr-x 1 jon jon 5216 2013-08-07 18:05 dev-support/generate-hadoopX-poms.sh
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9449) document how to use shell enhancements from HBASE-5548
[ https://issues.apache.org/jira/browse/HBASE-9449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760428#comment-13760428 ] Jimmy Xiang commented on HBASE-9449: Cool, looks good. Can we also give an example of command list whose response is not plain English? So that people who is not so familiar with ruby like me can just ignore it instead of wondering what it is. Not sure if we can make the response looks better. document how to use shell enhancements from HBASE-5548 -- Key: HBASE-9449 URL: https://issues.apache.org/jira/browse/HBASE-9449 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9449.patch, shell.html HBASE-5548 introduced new behavior for shell commands like 'list' to make them act more ruby-like. There is no documentation for this in the refguide. We should 1) have an example in the shell sectoin 2) document the that new '= #xxx.x..' line is expected We can probably lift a bunch of docs from HBASE-5548. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9453) make dev-support/generate-hadoopX-poms.sh have exec perms.
[
https://issues.apache.org/jira/browse/HBASE-9453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh resolved HBASE-9453.
---
Resolution: Fixed
make dev-support/generate-hadoopX-poms.sh have exec perms.
--
Key: HBASE-9453
URL: https://issues.apache.org/jira/browse/HBASE-9453
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.95.2
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Minor
Fix For: 0.98.0, 0.96.1
Attachments: hbase-9453.patch
currently:
{code}
jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
-rw-r--r-- 1 jon jon 5216 2013-09-06 10:45
dev-support/generate-hadoopX-poms.sh
{code}
after patch:
{code}
jon@swoop:~/proj/hbase-trunk$ ls -la dev-support/generate-hadoopX-poms.sh
-rwxr-xr-x 1 jon jon 5216 2013-08-07 18:05
dev-support/generate-hadoopX-poms.sh
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9301) Default hbase.dynamic.jars.dir to hbase.rootdir/jars
[ https://issues.apache.org/jira/browse/HBASE-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9301: - Fix Version/s: 0.96.1 0.94.12 0.98.0 Default hbase.dynamic.jars.dir to hbase.rootdir/jars Key: HBASE-9301 URL: https://issues.apache.org/jira/browse/HBASE-9301 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.95.2, 0.94.12, 0.96.0 Reporter: James Taylor Assignee: Vasu Mariyala Fix For: 0.98.0, 0.94.12, 0.96.1 Attachments: 0.94-HBASE-9301.patch, HBASE-9301.patch A reasonable default for hbase.dynamic.jars.dir would be hbase.rootdir/jars so that folks aren't forced to edit their hbase-sites.xml to take advantage of the new, cool feature to load coprocessor/custom filter jars out of HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8911) Inject MTTR specific traces to get a break up of various steps
[ https://issues.apache.org/jira/browse/HBASE-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760586#comment-13760586 ] Himanshu Vashishtha commented on HBASE-8911: This got folded in 9121, right [~eclark]? Inject MTTR specific traces to get a break up of various steps -- Key: HBASE-8911 URL: https://issues.apache.org/jira/browse/HBASE-8911 Project: HBase Issue Type: Bug Components: MTTR Affects Versions: 0.95.1 Reporter: Himanshu Vashishtha Attachments: 8911-v0.patch, 8911-v1.patch, 8911-v2.patch, 8911-v3.patch There are various steps involved in a regionserver recovery process. This jira adds instrumentation at various places in order to get an idea what are the steps involved in a regionserver recovery and how much time is spent in each of these parts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8911) Inject MTTR specific traces to get a break up of various steps
[ https://issues.apache.org/jira/browse/HBASE-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha reassigned HBASE-8911: -- Assignee: Himanshu Vashishtha Inject MTTR specific traces to get a break up of various steps -- Key: HBASE-8911 URL: https://issues.apache.org/jira/browse/HBASE-8911 Project: HBase Issue Type: Bug Components: MTTR Affects Versions: 0.95.1 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Attachments: 8911-v0.patch, 8911-v1.patch, 8911-v2.patch, 8911-v3.patch There are various steps involved in a regionserver recovery process. This jira adds instrumentation at various places in order to get an idea what are the steps involved in a regionserver recovery and how much time is spent in each of these parts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9454) HBaseAdmin#unassign() has incorrect @param argument
[
https://issues.apache.org/jira/browse/HBASE-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9454:
--
Attachment: 9454-v2.txt
That would be better.
HBaseAdmin#unassign() has incorrect @param argument
---
Key: HBASE-9454
URL: https://issues.apache.org/jira/browse/HBASE-9454
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9454.txt, 9454-v2.txt
Here is the javadoc warning:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9455) Port HBASE-7113 'TestGzipFilter is flaky with jdk1.7' to 0.94
[
https://issues.apache.org/jira/browse/HBASE-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9455:
--
Fix Version/s: 0.94.12
Port HBASE-7113 'TestGzipFilter is flaky with jdk1.7' to 0.94
-
Key: HBASE-9455
URL: https://issues.apache.org/jira/browse/HBASE-9455
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 0.94.12
Attachments: 9455-0.94.txt
Test failure was the same as described in HBASE-7113:
{code}
testScannerResultCodes(org.apache.hadoop.hbase.rest.TestGzipFilter) Time
elapsed: 0.033 sec FAILURE!
java.lang.AssertionError: expected:204 but was:200
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.hadoop.hbase.rest.TestGzipFilter.testScannerResultCodes(TestGzipFilter.java:148)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9455) Port HBASE-7113 'TestGzipFilter is flaky with jdk1.7' to 0.94
[
https://issues.apache.org/jira/browse/HBASE-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9455:
--
Attachment: 9455-0.94.txt
Port HBASE-7113 'TestGzipFilter is flaky with jdk1.7' to 0.94
-
Key: HBASE-9455
URL: https://issues.apache.org/jira/browse/HBASE-9455
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9455-0.94.txt
Test failure was the same as described in HBASE-7113:
{code}
testScannerResultCodes(org.apache.hadoop.hbase.rest.TestGzipFilter) Time
elapsed: 0.033 sec FAILURE!
java.lang.AssertionError: expected:204 but was:200
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.hadoop.hbase.rest.TestGzipFilter.testScannerResultCodes(TestGzipFilter.java:148)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8410) Basic quota support for namespaces
[
https://issues.apache.org/jira/browse/HBASE-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760589#comment-13760589
]
Ted Yu commented on HBASE-8410:
---
+1, provided that the following javadoc warning is fixed:
{code}
[WARNING]
/Users/tyu/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/NamespaceDescriptorRetriever.java:38:
warning - @param argument The is not a parameter name.
{code}
Basic quota support for namespaces
--
Key: HBASE-8410
URL: https://issues.apache.org/jira/browse/HBASE-8410
Project: HBase
Issue Type: Sub-task
Reporter: Francis Liu
Assignee: Vandana Ayyalasomayajula
Attachments: HBASE_8410_1_trunk.patch, HBASE_8410.patch,
HBASE-8410_trunk_2.patch, HBASE-8410_trunk_3.patch, HBASE-8410_trunk_4.patch,
HBASE-8410_trunk_4.patch, HBASE-8410_trunk_5.patch, HBASE-8410_trunk_6.patch
This task involves creating an observer which provides basic quota support to
namespaces in terms of (1) number of tables and (2) number of regions. The
quota support can be enabled by setting:
property
namehbase.coprocessor.region.classes/name
valueorg.apache.hadoop.hbase.namespace.NamespaceController/value
/property
property
namehbase.coprocessor.master.classes/name
valueorg.apache.hadoop.hbase.namespace.NamespaceController/value
/property
in the hbase-site.xml.
To add quotas to namespace, while creating namespace properties need to be
added.
Examples:
1. namespace_create 'ns1', {'hbase.namespace.quota.maxregion'='10'}
2. 1. namespace_create 'ns2', {'hbase.namespace.quota.maxtables'='2'},
{'hbase.namespace.quota.maxregion'='5'}
The quotas can be modified/added to namespace at any point of time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9286) [0.94] ageOfLastShippedOp replication metric doesn't update if the slave regionserver is stalled
[ https://issues.apache.org/jira/browse/HBASE-9286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760570#comment-13760570 ] Hudson commented on HBASE-9286: --- SUCCESS: Integrated in HBase-0.94 #1138 (See [https://builds.apache.org/job/HBase-0.94/1138/]) HBASE-9286 [0.94] ageOfLastShippedOp replication metric doesn't update if the slave regionserver is stalled (Alex Newman) (larsh: rev 1520646) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceMetrics.java [0.94] ageOfLastShippedOp replication metric doesn't update if the slave regionserver is stalled Key: HBASE-9286 URL: https://issues.apache.org/jira/browse/HBASE-9286 Project: HBase Issue Type: Bug Reporter: Alex Newman Assignee: Alex Newman Fix For: 0.94.12 Attachments: 0001-HBASE-9286.-ageOfLastShippedOp-replication-metric-do.patch In replicationmanager HRegionInterface rrs = getRS(); rrs.replicateLogEntries(Arrays.copyOf(this.entriesArray, currentNbEntries)); this.metrics.setAgeOfLastShippedOp( this.entriesArray[currentNbEntries-1].getKey().getWriteTime()); break; which makes sense, but is wrong. The problem is that rrs.replicateLogEntries will block for a very long time if the slave server is suspended or unavailable but not down. However this is easy to fix. We just need to call refreshAgeOfLastShippedOp(); on a regular basis, in a different thread. I've attached a patch which fixed this for cdh4. I can make one for trunk and the like as well if you need me to do but it's a small change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9455) Port HBASE-7113 'TestGzipFilter is flaky with jdk1.7' to 0.94
Ted Yu created HBASE-9455:
-
Summary: Port HBASE-7113 'TestGzipFilter is flaky with jdk1.7' to
0.94
Key: HBASE-9455
URL: https://issues.apache.org/jira/browse/HBASE-9455
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Test failure was the same as described in HBASE-7113:
{code}
testScannerResultCodes(org.apache.hadoop.hbase.rest.TestGzipFilter) Time
elapsed: 0.033 sec FAILURE!
java.lang.AssertionError: expected:204 but was:200
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.hadoop.hbase.rest.TestGzipFilter.testScannerResultCodes(TestGzipFilter.java:148)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9338) Test Big Linked List fails on Hadoop 2.1.0
[
https://issues.apache.org/jira/browse/HBASE-9338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760594#comment-13760594
]
Elliott Clark commented on HBASE-9338:
--
h1.What's happened
When the new Hadoop release was in RC and we were getting close to a 0.96
release of our own I changed all of the IT tests to run for longer. When I did
this IntegrationTestBigLinkedList and IntegrationTestLoadAndVerify started
failing. I at first thought this was due to 2.1.0 but now I'm pretty sure it
was just a red herring as the tests pass with a shorter run time. So I started
creating new chaos monkeys with different actions turned on and off. Doing
that I found that Online Schema Change was what was causing Test Load and
Verify to fail. So that was disabled in 0.96.
Then I started working on Big Linked List. That one has been much harder.
h1.Observations
The calm monkey always passes. The unbalance one almost always passes. When
it doesn't, the failure is usually due to the assignment manager getting stuck.
There appears to be no data loss then.
Lots of the failures are like this one:
{code} Map-Reduce Framework
Map input records=59660
Map output records=119320
Map output bytes=3877900
Map output materialized bytes=41399980716
Input split bytes=7360
Combine input records=0
Combine output records=0
Reduce input groups=6
Reduce shuffle bytes=41399980716
Reduce input records=119320
Reduce output records=680
Spilled Records=3576554634
Shuffled Maps =696
Failed Shuffles=0
Merged Map outputs=696
GC time elapsed (ms)=96902
CPU time spent (ms)=14269000
Physical memory (bytes) snapshot=86934548480
Virtual memory (bytes) snapshot=157839667200
Total committed heap usage (bytes)=86516498432
HBase Counters
BYTES_IN_REMOTE_RESULTS=29267422740
BYTES_IN_RESULTS=3582060
MILLIS_BETWEEN_NEXTS=15450135
NOT_SERVING_REGION_EXCEPTION=27
NUM_SCANNER_RESTARTS=0
REGIONS_SCANNED=100
REMOTE_RPC_CALLS=4878206
REMOTE_RPC_RETRIES=90
RPC_CALLS=6000370
RPC_RETRIES=103
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Verify$Counts
REFERENCED=59320
UNDEFINED=340
UNREFERENCED=340
{code}
The number of map input records is too low. It should always be a multiple of
30m. The number of un-referenced rows is equal to the number of undefined rows.
If you add that number to the number of map input records then all rows are
accounted for. That suggests that for this job there are lots of places that we
are missing exactly on link at each location.
However that pattern isn't 100%. There are other tests where the number of
unreferenced nodes is not equal to the number of undefined rows. So those
tests runs have some places in the chain were more then one link is missing.
To me this seems to suggest that it's not an off by one error and that it seems
like it's on the client side.
h1.Things I've tried
* Tried to see if it's just a badly written test
** The test is hard to follow and could be cleaned up
** But it's logic seems solid.
* I thought maybe yarn is the one messing up our data. So I Changed the output
keys to match what Load Test and Verify was doing
** This didn't seem to have any real impact one Online Schema Change was turned
off.
* Tried to see if one map task was failing. So I changed the job to log more
and to use a better client column.
** It's not just one map task that has failures
** It is always the last generation job that was the writer of data that's
missing.
* Then I tried to see if it's just the beginning or the end of the linked list
that is missing data
** It's not. There is missing data everywhere in the chain.
* Running Verify step with no monkey
** This failed suggesting that we are really losing data
* Then I though it might be the same bug that was causing replication to lose
edits. So I re-ran the tests before and after JD's HLog reader change.
** Tests fail with JD's patch
* Then I tried Running the test with a Monkey that kills RS and Master
** This passed
** I even ran this with more loop iterations and it still passed
* I Thought maybe it was merge regions. So I turned off merging regions
** The tests still failed.
* So I ran a job
[jira] [Commented] (HBASE-9449) document how to use shell enhancements from HBASE-5548
[ https://issues.apache.org/jira/browse/HBASE-9449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760614#comment-13760614 ] Jimmy Xiang commented on HBASE-9449: Sure. Thanks. document how to use shell enhancements from HBASE-5548 -- Key: HBASE-9449 URL: https://issues.apache.org/jira/browse/HBASE-9449 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9449.patch, shell.html HBASE-5548 introduced new behavior for shell commands like 'list' to make them act more ruby-like. There is no documentation for this in the refguide. We should 1) have an example in the shell sectoin 2) document the that new '= #xxx.x..' line is expected We can probably lift a bunch of docs from HBASE-5548. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-5349) Automagically tweak global memstore and block cache sizes based on workload
[
https://issues.apache.org/jira/browse/HBASE-5349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760641#comment-13760641
]
Ted Yu edited comment on HBASE-5349 at 9/6/13 9:20 PM:
---
There're ^M's at the end of each line. Please generate patch on Linux.
It would be nice to put the patch on review board.
For HeapMemoryAutoTuner:
{code}
+ AutoTunerContext context = createTunerContext();^M
{code}
Do we need to create new context for each iteration of the chore ? Can one
instance be reused ?
{code}
+LOG.debug(From HeapMemoryBalancer new memstoreSize: + memstoreSize
+ %. new blockCacheSize: + blockCacheSize + %);^M
{code}
I think the percent sign is not needed.
{code}
+if (1 - (memstoreSize + blockCacheSize)
HConstants.HBASE_CLUSTER_MINIMUM_MEMORY_THRESHOLD) {^M
+ LOG.warn(Current heap configuration from HeapMemoryBalancer exceeds
^M
+ + the threshold required for successful cluster operation. ^M
{code}
Should some action be taken for the above case ? Otherwise tuning is
effectively disabled.
Since memstoreSize and blockCacheSize are local variables, I would expect some
action when result.needsTuning() returns true.
For DefaultHeapMemoryBalancerImpl, please add javadoc and audience annotation.
{code}
+ result.setMemstoreSize(context.getCurMemStoreSize() - step);^M
{code}
Should we check that the decrement would not produce negative result ?
Please add unit tests for the new classes.
was (Author: yuzhih...@gmail.com):
There're ^M's at the end of each line. Please generate patch on Linux.
For HeapMemoryAutoTuner:
It would be nice to put the patch on review board.
{code}
+ AutoTunerContext context = createTunerContext();^M
{code}
Do we need to create new context for each iteration of the chore ? Can one
instance be reused ?
{code}
+LOG.debug(From HeapMemoryBalancer new memstoreSize: + memstoreSize
+ %. new blockCacheSize: + blockCacheSize + %);^M
{code}
I think the percent sign is not needed.
{code}
+if (1 - (memstoreSize + blockCacheSize)
HConstants.HBASE_CLUSTER_MINIMUM_MEMORY_THRESHOLD) {^M
+ LOG.warn(Current heap configuration from HeapMemoryBalancer exceeds
^M
+ + the threshold required for successful cluster operation. ^M
{code}
Should some action be taken for the above case ? Otherwise tuning is
effectively disabled.
Since memstoreSize and blockCacheSize are local variables, I would expect some
action when result.needsTuning() returns true.
For DefaultHeapMemoryBalancerImpl, please add javadoc and audience annotation.
{code}
+ result.setMemstoreSize(context.getCurMemStoreSize() - step);^M
{code}
Should we check that the decrement would not produce negative result ?
Please add unit tests for the new classes.
Automagically tweak global memstore and block cache sizes based on workload
---
Key: HBASE-5349
URL: https://issues.apache.org/jira/browse/HBASE-5349
Project: HBase
Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: Anoop Sam John
Attachments: WIP_HBASE-5349.patch
Hypertable does a neat thing where it changes the size given to the CellCache
(our MemStores) and Block Cache based on the workload. If you need an image,
scroll down at the bottom of this link:
http://www.hypertable.com/documentation/architecture/
That'd be one less thing to configure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5349) Automagically tweak global memstore and block cache sizes based on workload
[
https://issues.apache.org/jira/browse/HBASE-5349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760641#comment-13760641
]
Ted Yu commented on HBASE-5349:
---
There're ^M's at the end of each line. Please generate patch on Linux.
For HeapMemoryAutoTuner:
It would be nice to put the patch on review board.
{code}
+ AutoTunerContext context = createTunerContext();^M
{code}
Do we need to create new context for each iteration of the chore ? Can one
instance be reused ?
{code}
+LOG.debug(From HeapMemoryBalancer new memstoreSize: + memstoreSize
+ %. new blockCacheSize: + blockCacheSize + %);^M
{code}
I think the percent sign is not needed.
{code}
+if (1 - (memstoreSize + blockCacheSize)
HConstants.HBASE_CLUSTER_MINIMUM_MEMORY_THRESHOLD) {^M
+ LOG.warn(Current heap configuration from HeapMemoryBalancer exceeds
^M
+ + the threshold required for successful cluster operation. ^M
{code}
Should some action be taken for the above case ? Otherwise tuning is
effectively disabled.
Since memstoreSize and blockCacheSize are local variables, I would expect some
action when result.needsTuning() returns true.
For DefaultHeapMemoryBalancerImpl, please add javadoc and audience annotation.
{code}
+ result.setMemstoreSize(context.getCurMemStoreSize() - step);^M
{code}
Should we check that the decrement would not produce negative result ?
Please add unit tests for the new classes.
Automagically tweak global memstore and block cache sizes based on workload
---
Key: HBASE-5349
URL: https://issues.apache.org/jira/browse/HBASE-5349
Project: HBase
Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: Jean-Daniel Cryans
Assignee: Anoop Sam John
Attachments: WIP_HBASE-5349.patch
Hypertable does a neat thing where it changes the size given to the CellCache
(our MemStores) and Block Cache based on the workload. If you need an image,
scroll down at the bottom of this link:
http://www.hypertable.com/documentation/architecture/
That'd be one less thing to configure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7211) Improve hbase ref guide for the testing part.
[ https://issues.apache.org/jira/browse/HBASE-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760640#comment-13760640 ] Nicolas Liochon commented on HBASE-7211: Actually, I'm waiting for 4955 to finish this one. Improve hbase ref guide for the testing part. - Key: HBASE-7211 URL: https://issues.apache.org/jira/browse/HBASE-7211 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.95.2 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Attachments: hbase-7211-partial.patch Here is some stuff I saw. I will propose a fix in a week or so, please add the comment or issues you have in mind. ??15.6.1. Apache HBase Modules?? = We should be able to use categories in all modules. The default should be small; but any test manipulating the time needs to be in a specific jvm (hence medium), so it's not always related to minicluster. ??15.6.3.6. hbasetests.sh?? = We can remove this chapter, and the script The script is not totally useless, but I think nobody actually uses it. = Add a chapter on flakiness. Some tests are, unfortunately, flaky. While there number decreases, we still have some. Rules are: - don't write flaky tests! :-) - small tests cannot be flaky, as it blocks other test execution. Corollary: if you have an issue with a small test, it's either your environment either a severe issue. - rerun the test a few time to validate, check the ports and file descriptors used. ??mvn test -P localTests -Dtest=MyTest?? = We could actually activate the localTests profile whenever -Dtest is used. If we do that, we can remove the reference from localTests in the doc. ??mvn test -P runSmallTests?? ??mvn test -P runMediumTests?? = I'm not sure it's actually used. We could remove them from the pom.xml (and the doc). ??The HBase build uses a patched version of the maven surefire plugin?? = Hopefully, we will be able to remove this soon :-) ??Integration tests are described TODO: POINTER_TO_INTEGRATION_TEST_SECTION?? = Should be documented -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9436) hbase.regionserver.handler.count default: 5, 10, 25, 30? pick one
[ https://issues.apache.org/jira/browse/HBASE-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9436: --- Attachment: 9436.v1.patch hbase.regionserver.handler.count default: 5, 10, 25, 30? pick one - Key: HBASE-9436 URL: https://issues.apache.org/jira/browse/HBASE-9436 Project: HBase Issue Type: Bug Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Attachments: 9436.v1.patch Below what we have today. I vote for 10. configuration.xml The default of 10 is rather low common/hbase-site namehbase.regionserver.handler.count/name value30/value server/hbase-site value5/value descriptionCount of RPC Server instances spun up on RegionServers Same property is used by the HMaster for count of master handlers. Default is 10. === HMaster.java int numHandlers = conf.getInt(hbase.master.handler.count, conf.getInt(hbase.regionserver.handler.count, 25)); HRegionServer.java hbase.regionserver.handler.count: conf.getInt(hbase.regionserver.handler.count, 10), -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7211) Improve hbase ref guide for the testing part.
[ https://issues.apache.org/jira/browse/HBASE-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760631#comment-13760631 ] Nick Dimiduk commented on HBASE-7211: - It looks like this issue was resolved but never closed. Ping [~nkeywal], [~jeffreyz]. Improve hbase ref guide for the testing part. - Key: HBASE-7211 URL: https://issues.apache.org/jira/browse/HBASE-7211 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.95.2 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Attachments: hbase-7211-partial.patch Here is some stuff I saw. I will propose a fix in a week or so, please add the comment or issues you have in mind. ??15.6.1. Apache HBase Modules?? = We should be able to use categories in all modules. The default should be small; but any test manipulating the time needs to be in a specific jvm (hence medium), so it's not always related to minicluster. ??15.6.3.6. hbasetests.sh?? = We can remove this chapter, and the script The script is not totally useless, but I think nobody actually uses it. = Add a chapter on flakiness. Some tests are, unfortunately, flaky. While there number decreases, we still have some. Rules are: - don't write flaky tests! :-) - small tests cannot be flaky, as it blocks other test execution. Corollary: if you have an issue with a small test, it's either your environment either a severe issue. - rerun the test a few time to validate, check the ports and file descriptors used. ??mvn test -P localTests -Dtest=MyTest?? = We could actually activate the localTests profile whenever -Dtest is used. If we do that, we can remove the reference from localTests in the doc. ??mvn test -P runSmallTests?? ??mvn test -P runMediumTests?? = I'm not sure it's actually used. We could remove them from the pom.xml (and the doc). ??The HBase build uses a patched version of the maven surefire plugin?? = Hopefully, we will be able to remove this soon :-) ??Integration tests are described TODO: POINTER_TO_INTEGRATION_TEST_SECTION?? = Should be documented -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9454) HBaseAdmin#unassign() has incorrect @param argument
[
https://issues.apache.org/jira/browse/HBASE-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9454:
--
Fix Version/s: 0.96.0
0.98.0
Hadoop Flags: Reviewed
Integrated to 0.96 and trunk.
Thanks for the review, Jimmy.
HBaseAdmin#unassign() has incorrect @param argument
---
Key: HBASE-9454
URL: https://issues.apache.org/jira/browse/HBASE-9454
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 0.98.0, 0.96.0
Attachments: 9454.txt, 9454-v2.txt
Here is the javadoc warning:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9437) hbase.regionserver.checksum.verify default: true or false
[ https://issues.apache.org/jira/browse/HBASE-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760635#comment-13760635 ] Hudson commented on HBASE-9437: --- SUCCESS: Integrated in hbase-0.96 #19 (See [https://builds.apache.org/job/hbase-0.96/19/]) HBASE-9437 hbase.regionserver.checksum.verify default is true (nkeywal: rev 1520636) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java hbase.regionserver.checksum.verify default: true or false - Key: HBASE-9437 URL: https://issues.apache.org/jira/browse/HBASE-9437 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.0 HRegionServer: default to false hbase-common/hbase-default.xml: true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9221) Provide interface for getting a User in the client
[ https://issues.apache.org/jira/browse/HBASE-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9221: - Fix Version/s: 0.96.1 0.94.12 0.98.0 Provide interface for getting a User in the client -- Key: HBASE-9221 URL: https://issues.apache.org/jira/browse/HBASE-9221 Project: HBase Issue Type: Improvement Components: Client, hadoop2, security Affects Versions: 0.98.0, 0.95.2, 0.94.12 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.98.0, 0.94.12, 0.96.1 Attachments: hbase-9221-0.94-v0.patch Sometimes, users will want to provide their own User class depending on the type of security they will support in their local environment. For instance, running Hadoop1 vs Hadoop2 vs CDH means potentially different ways of getting the UserGroupInformation. This issue abstracts out the mechanism by which we obtain an o.a.h.hbase.security.User to a UserProvider. This UserProvider can then be extented as a Hadoop 1/2 shim as well as supporting custom authentication code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9454) HBaseAdmin#unassign() has incorrect @param argument
[
https://issues.apache.org/jira/browse/HBASE-9454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760555#comment-13760555
]
Hadoop QA commented on HBASE-9454:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12601874/9454-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7075//console
This message is automatically generated.
HBaseAdmin#unassign() has incorrect @param argument
---
Key: HBASE-9454
URL: https://issues.apache.org/jira/browse/HBASE-9454
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 9454.txt, 9454-v2.txt
Here is the javadoc warning:
{code}
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:1693:
warning - @param argument regionName is not a parameter name.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9437) hbase.regionserver.checksum.verify default: true or false
[ https://issues.apache.org/jira/browse/HBASE-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760666#comment-13760666 ] Hudson commented on HBASE-9437: --- FAILURE: Integrated in HBase-TRUNK #4476 (See [https://builds.apache.org/job/HBase-TRUNK/4476/]) HBASE-9437 hbase.regionserver.checksum.verify default is true (nkeywal: rev 1520634) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java hbase.regionserver.checksum.verify default: true or false - Key: HBASE-9437 URL: https://issues.apache.org/jira/browse/HBASE-9437 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.0 HRegionServer: default to false hbase-common/hbase-default.xml: true -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-9390: - Attachment: hbase-9390-v2.patch I reverted the preWALRestore interface change part. The v2 patch is to trigger preWALRestore, postWALRestore events. As a follow up, I'll ensure preWALRestore semantics is kept in distributedLogReplay mode. Thanks. coprocessors observers are not called during a recovery with the new log replay algorithm - Key: HBASE-9390 URL: https://issues.apache.org/jira/browse/HBASE-9390 Project: HBase Issue Type: Bug Components: Coprocessors, MTTR Affects Versions: 0.95.2 Reporter: Nicolas Liochon Assignee: Jeffrey Zhong Attachments: copro.patch, hbase-9390.patch, hbase-9390-v2.patch See the patch to reproduce the issue: If we activate log replay we don't have the events on WAL restore. Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9456) Meta doesn't get assigned in a master failure scenario
[ https://issues.apache.org/jira/browse/HBASE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760695#comment-13760695 ] Elliott Clark commented on HBASE-9456: -- Yep I've seen this a few times on IT test runs. It's annoying when the whole test fails for this. Meta doesn't get assigned in a master failure scenario -- Key: HBASE-9456 URL: https://issues.apache.org/jira/browse/HBASE-9456 Project: HBase Issue Type: Bug Reporter: Devaraj Das Fix For: 0.96.0 The flow: 1. Cluster is up, meta is assigned to some server 2. Master is killed 3. Master is brought up, it is initializing. It learns about the Meta server (in assignMeta). 4. Server holding meta is killed 5. Meta never gets reassigned since the SSH wasn't enabled -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9456) Meta doesn't get assigned in a master failure scenario
Devaraj Das created HBASE-9456: -- Summary: Meta doesn't get assigned in a master failure scenario Key: HBASE-9456 URL: https://issues.apache.org/jira/browse/HBASE-9456 Project: HBase Issue Type: Bug Reporter: Devaraj Das The flow: 1. Cluster is up, meta is assigned to some server 2. Master is killed 3. Master is brought up, it is initializing. It learns about the Meta server (in assignMeta). 4. Server holding meta is killed 5. Meta never gets reassigned since the SSH wasn't enabled -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9456) Meta doesn't get assigned in a master failure scenario
[ https://issues.apache.org/jira/browse/HBASE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-9456: --- Fix Version/s: 0.96.0 Meta doesn't get assigned in a master failure scenario -- Key: HBASE-9456 URL: https://issues.apache.org/jira/browse/HBASE-9456 Project: HBase Issue Type: Bug Reporter: Devaraj Das Fix For: 0.96.0 The flow: 1. Cluster is up, meta is assigned to some server 2. Master is killed 3. Master is brought up, it is initializing. It learns about the Meta server (in assignMeta). 4. Server holding meta is killed 5. Meta never gets reassigned since the SSH wasn't enabled -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-9338) Test Big Linked List fails on Hadoop 2.1.0
[
https://issues.apache.org/jira/browse/HBASE-9338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760594#comment-13760594
]
Elliott Clark edited comment on HBASE-9338 at 9/6/13 10:21 PM:
---
h1.What's happened
When the new Hadoop release was in RC and we were getting close to a 0.96
release of our own I changed all of the IT tests to run for longer. When I did
this IntegrationTestBigLinkedList and IntegrationTestLoadAndVerify started
failing. I at first thought this was due to 2.1.0 but now I'm pretty sure it
was just a red herring as the tests pass with a shorter run time. So I started
creating new chaos monkeys with different actions turned on and off. Doing
that I found that Online Schema Change was what was causing Test Load and
Verify to fail. So that was disabled in 0.96.
Then I started working on Big Linked List. That one has been much harder.
h1.Observations
The calm monkey always passes. The unbalance one almost always passes. When
it doesn't, the failure is usually due to the assignment manager getting stuck.
There appears to be no data loss then.
Lots of the failures are like this one:
{code} Map-Reduce Framework
Map input records=59660
Map output records=119320
Map output bytes=3877900
Map output materialized bytes=41399980716
Input split bytes=7360
Combine input records=0
Combine output records=0
Reduce input groups=6
Reduce shuffle bytes=41399980716
Reduce input records=119320
Reduce output records=680
Spilled Records=3576554634
Shuffled Maps =696
Failed Shuffles=0
Merged Map outputs=696
GC time elapsed (ms)=96902
CPU time spent (ms)=14269000
Physical memory (bytes) snapshot=86934548480
Virtual memory (bytes) snapshot=157839667200
Total committed heap usage (bytes)=86516498432
HBase Counters
BYTES_IN_REMOTE_RESULTS=29267422740
BYTES_IN_RESULTS=3582060
MILLIS_BETWEEN_NEXTS=15450135
NOT_SERVING_REGION_EXCEPTION=27
NUM_SCANNER_RESTARTS=0
REGIONS_SCANNED=100
REMOTE_RPC_CALLS=4878206
REMOTE_RPC_RETRIES=90
RPC_CALLS=6000370
RPC_RETRIES=103
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Verify$Counts
REFERENCED=59320
UNDEFINED=340
UNREFERENCED=340
{code}
The number of map input records is too low. It should always be a multiple of
30m. The number of un-referenced rows is equal to the number of undefined rows.
If you add that number to the number of map input records then all rows are
accounted for. That suggests that for this job there are lots of places that we
are missing exactly on link at each location.
However that pattern isn't 100%. There are other tests where the number of
unreferenced nodes is not equal to the number of undefined rows. So those
tests runs have some places in the chain were more then one link is missing.
To me this seems to suggest that it's not an off by one error and that it seems
like it's on the client side.
h1.Things I've tried
* Tried to see if it's just a badly written test
** The test is hard to follow and could be cleaned up
** But it's logic seems solid.
* I thought maybe yarn is the one messing up our data. So I Changed the output
keys to match what Load Test and Verify was doing
** This didn't seem to have any real impact one Online Schema Change was turned
off.
* Tried to see if one map task was failing. So I changed the job to log more
and to use a better client column.
** It's not just one map task that has failures
** It is always the last generation job that was the writer of data that's
missing.
* Then I tried to see if it's just the beginning or the end of the linked list
that is missing data
** It's not. There is missing data everywhere in the chain.
* Running Verify step with no monkey
** This failed suggesting that we are really losing data
* Then I though it might be the same bug that was causing replication to lose
edits. So I re-ran the tests before and after JD's HLog reader change.
** Tests fail with JD's patch
* Then I tried Running the test with a Monkey that kills RS and Master
** This passed
** I even ran this with more loop iterations and it still passed
* I Thought maybe it was merge regions. So I turned off merging
[jira] [Updated] (HBASE-9165) Improvements to addDependencyJars
[
https://issues.apache.org/jira/browse/HBASE-9165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-9165:
Status: Open (was: Patch Available)
The patch has gone stale.
Improvements to addDependencyJars
-
Key: HBASE-9165
URL: https://issues.apache.org/jira/browse/HBASE-9165
Project: HBase
Issue Type: Sub-task
Components: mapreduce
Affects Versions: 0.95.2
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Attachments:
0001-HBASE-9165-mapreduce-Modularize-building-dependency-.patch
The way we support adding HBase dependencies to a MapReduce job in
{{TableMapReduceUtils#addDependencyJars(job)}} is a bit monolithic. Advanced
users need a way to add HBase and its dependencies to their job without us
snooping around for ouput formats and the like (see PIG-3285). We can also
benefit from a little more code reuse between our {{mapred}} and
{{mapreduce}} namespaces.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9456) Meta doesn't get assigned in a master failure scenario
[ https://issues.apache.org/jira/browse/HBASE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-9456: --- Attachment: 9456-1.txt This should fix the problem. I will try to write a test. [~eclark], yes, I haven't been able to get a good run of the IT tests. Yesterday, hit HBASE-9451 and today this one. Meta doesn't get assigned in a master failure scenario -- Key: HBASE-9456 URL: https://issues.apache.org/jira/browse/HBASE-9456 Project: HBase Issue Type: Bug Reporter: Devaraj Das Fix For: 0.96.0 Attachments: 9456-1.txt The flow: 1. Cluster is up, meta is assigned to some server 2. Master is killed 3. Master is brought up, it is initializing. It learns about the Meta server (in assignMeta). 4. Server holding meta is killed 5. Meta never gets reassigned since the SSH wasn't enabled -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9272) A simple parallel, unordered scanner
[ https://issues.apache.org/jira/browse/HBASE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9272: - Attachment: 9272-0.94.txt So here's a sample patch against 0.94. It does the following: # An API to parallelize a single Scan. # Round robin across RegionServers # Builds its own task queue in order not to rely on a specifically configured thread pool (i.e. the HTable's pool can be used) # explores ways of automated scaling. The parallelism is controlled by a scaling factor that takes the number of a region server touched by the scan into account # An alternate API where the caller can pass in a set of Splits (in form of Scans) and then those are executed on the pool # limits all thread synchronization to the a BlockingQueue, which (in theory) allows the reader and the writer to lock independently # to avoid other synchronization, marker objects are passed to indicate when the thread is done or encountered an exception # Also hooked this up with HTable (which is the only questionable - IMHO - part of this, since it changes HTableInterface and could break client application that directly implement HTableInterface). This part is not strictly needed, ParallelClientScanner can be used on its own. # Pushes a bit more common code into AbstractClientScanner. Please let me know what you think. If direction is good I'll add tests and make a trunk patch. A simple parallel, unordered scanner Key: HBASE-9272 URL: https://issues.apache.org/jira/browse/HBASE-9272 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Attachments: 9272-0.94.txt, ParallelClientScanner.java, ParallelClientScanner.java The contract of ClientScanner is to return rows in sort order. That limits the order in which region can be scanned. I propose a simple ParallelScanner that does not have this requirement and queries regions in parallel, return whatever gets returned first. This is generally useful for scans that filter a lot of data on the server, or in cases where the client can very quickly react to the returned data. I have a simple prototype (doesn't do error handling right, and might be a bit heavy on the synchronization side - it used a BlockingQueue to hand data between the client using the scanner and the threads doing the scanning, it also could potentially starve some scanners long enugh to time out at the server). On the plus side, it's only a 130 lines of code. :) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9440) Pass blocks of KVs from HFile scanner to the StoreFileScanner and up
[ https://issues.apache.org/jira/browse/HBASE-9440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760600#comment-13760600 ] Matt Corgan commented on HBASE-9440: It's a somewhat advanced optimization, but I've always hoped to see block level transfer of data like this. Both for compactions and long scans. For compactions it's probably quite often that all the cells in a block will remain contiguous, in which case you could save the decompression, decoding, heap logic, encoding, compression steps. Just hand the byte[] through to the new file. For the client case, maybe make it a setting to bring whole blocks back to the client (as soon as any part of a block is needed) and do filtering logic client-side. Pass blocks of KVs from HFile scanner to the StoreFileScanner and up Key: HBASE-9440 URL: https://issues.apache.org/jira/browse/HBASE-9440 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Currently we read KVs from an HFileScanner one-by-one and pass them up the scanner/heap tree. Many time the ranges of KVs retrieved from StoreFileScanner (by StoreScanners) and HFileScanner (by StoreFileScanner) will be non-overlapping. If chunks of KVs do not overlap we can sort entire chunks just by comparing the start/end key of the chunk. Only if chunks are overlapping do we need to sort KV by KV as we do now. I have no patch, but I wanted to float this idea. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8534) Fix coverage for org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-8534: Component/s: test mapreduce Fix coverage for org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Components: mapreduce, test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Assignee: Aleksey Gorshkov Fix For: 0.98.0 Attachments: 0001-HBASE-8534-hadoop2-addendum.patch, 8534-trunk-h.patch, HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94-g.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk-g.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8534) Fix coverage for org.apache.hadoop.hbase.mapreduce
[ https://issues.apache.org/jira/browse/HBASE-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760688#comment-13760688 ] Nick Dimiduk commented on HBASE-8534: - Ping. I think this was committed to trunk already. Should I resolve issue? What about 0.94 backport? Fix coverage for org.apache.hadoop.hbase.mapreduce -- Key: HBASE-8534 URL: https://issues.apache.org/jira/browse/HBASE-8534 Project: HBase Issue Type: Test Affects Versions: 0.94.8, 0.95.2 Reporter: Aleksey Gorshkov Assignee: Aleksey Gorshkov Fix For: 0.98.0 Attachments: 0001-HBASE-8534-hadoop2-addendum.patch, 8534-trunk-h.patch, HBASE-8534-0.94-d.patch, HBASE-8534-0.94-e.patch, HBASE-8534-0.94-f.patch, HBASE-8534-0.94-g.patch, HBASE-8534-0.94.patch, HBASE-8534-trunk-a.patch, HBASE-8534-trunk-b.patch, HBASE-8534-trunk-c.patch, HBASE-8534-trunk-d.patch, HBASE-8534-trunk-e.patch, HBASE-8534-trunk-f.patch, HBASE-8534-trunk-g.patch, HBASE-8534-trunk.patch fix coverage org.apache.hadoop.hbase.mapreduce patch HBASE-8534-0.94.patch for branch-0.94 patch HBASE-8534-trunk.patch for branch-0.95 and trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9376) TestDistributedLogSplitting creates a MiniCluster rooted at ~/hbase
[ https://issues.apache.org/jira/browse/HBASE-9376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-9376: - Resolution: Fixed Fix Version/s: 0.96.0 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) TestDistributedLogSplitting creates a MiniCluster rooted at ~/hbase --- Key: HBASE-9376 URL: https://issues.apache.org/jira/browse/HBASE-9376 Project: HBase Issue Type: Test Components: test Affects Versions: 0.95.2 Reporter: Nick Dimiduk Assignee: Jeffrey Zhong Priority: Minor Fix For: 0.98.0, 0.96.0 Attachments: hbase-9376.patch TestDistributedLogSplitting creates a MiniCluster rooted at ~/hbase. I have a machine configured such that my $HOME is NFS mounted, and this combination makes for a rather flakey test. Test cluster(s) instantiated by this test should instead root out of target (like all the rest). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9376) TestDistributedLogSplitting creates a MiniCluster rooted at ~/hbase
[ https://issues.apache.org/jira/browse/HBASE-9376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760605#comment-13760605 ] Jeffrey Zhong commented on HBASE-9376: -- Thanks [~saint@gmail.com] [~ndimiduk] for the review! I've integrated the fix into 0.96 and trunk branch. TestDistributedLogSplitting creates a MiniCluster rooted at ~/hbase --- Key: HBASE-9376 URL: https://issues.apache.org/jira/browse/HBASE-9376 Project: HBase Issue Type: Test Components: test Affects Versions: 0.95.2 Reporter: Nick Dimiduk Assignee: Jeffrey Zhong Priority: Minor Attachments: hbase-9376.patch TestDistributedLogSplitting creates a MiniCluster rooted at ~/hbase. I have a machine configured such that my $HOME is NFS mounted, and this combination makes for a rather flakey test. Test cluster(s) instantiated by this test should instead root out of target (like all the rest). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9338) Test Big Linked List fails on Hadoop 2.1.0
[ https://issues.apache.org/jira/browse/HBASE-9338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760671#comment-13760671 ] Sergey Shelukhin commented on HBASE-9338: - there might be several types of failures. Are all of these real data loss? For the last one it looks like something I saw on 94 (HBASE-8935), but then it was scanner issues during original verify, which went away on verify rerun. Test Big Linked List fails on Hadoop 2.1.0 -- Key: HBASE-9338 URL: https://issues.apache.org/jira/browse/HBASE-9338 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-HBASE-9338-TESTING.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9457) Master could fail start if region server with system table is down
Jimmy Xiang created HBASE-9457: -- Summary: Master could fail start if region server with system table is down Key: HBASE-9457 URL: https://issues.apache.org/jira/browse/HBASE-9457 Project: HBase Issue Type: Bug Components: master, Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang In the region server holding the system table is killed while master is starting, master will hang there waiting for system table to be assigned which won't happen. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9456) Meta doesn't get assigned in a master failure scenario
[ https://issues.apache.org/jira/browse/HBASE-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760718#comment-13760718 ] Jimmy Xiang commented on HBASE-9456: In looking the code, I found there is at least one more issue with region assignment since we introduced the system table: HBASE-9457. Meta doesn't get assigned in a master failure scenario -- Key: HBASE-9456 URL: https://issues.apache.org/jira/browse/HBASE-9456 Project: HBase Issue Type: Bug Reporter: Devaraj Das Fix For: 0.96.0 Attachments: 9456-1.txt The flow: 1. Cluster is up, meta is assigned to some server 2. Master is killed 3. Master is brought up, it is initializing. It learns about the Meta server (in assignMeta). 4. Server holding meta is killed 5. Meta never gets reassigned since the SSH wasn't enabled -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
