[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851466#comment-13851466
]
Hudson commented on HBASE-10189:
SUCCESS: Integrated in hbase-0.96 #232 (See
[https://builds.apache.org/job/hbase-0.96/232/])
HBASE-10189 Intermittent TestReplicationSyncUpTool failure (LarsH and Demai Ni)
(larsh: rev 1551792)
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSyncUpTool.java
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10159) Replaced deprecated interface Closeable
[ https://issues.apache.org/jira/browse/HBASE-10159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851465#comment-13851465 ] Hudson commented on HBASE-10159: SUCCESS: Integrated in hbase-0.96 #232 (See [https://builds.apache.org/job/hbase-0.96/232/]) HBASE-10159 Replaced deprecated interface Closeable (jxiang: rev 1551820) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java > Replaced deprecated interface Closeable > --- > > Key: HBASE-10159 > URL: https://issues.apache.org/jira/browse/HBASE-10159 > Project: HBase > Issue Type: Task >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Trivial > Fix For: 0.98.0, 0.96.2, 0.99.0 > > Attachments: trunk-10159.patch, trunk-10159_v2.patch > > > Hadoop interface Closeable has been deprecated for quite some time to favor > java.io.Closeable. Since we support only hadoop 1.0 and above now, we can > replace the deprecated Closeable with the java one. We can also clean up some > hack we did as well, for example, HBASE-10029. > https://github.com/apache/hadoop-common/blob/branch-1.0/src/core/org/apache/hadoop/io/Closeable.java > -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10182) Potential null object deference in AssignmentManager#handleRegion()
[
https://issues.apache.org/jira/browse/HBASE-10182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851468#comment-13851468
]
Hudson commented on HBASE-10182:
SUCCESS: Integrated in hbase-0.96 #232 (See
[https://builds.apache.org/job/hbase-0.96/232/])
HBASE-10182 Potential null object deference in AssignmentManager#handleRegion()
(jxiang: rev 1551818)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> Potential null object deference in AssignmentManager#handleRegion()
> ---
>
> Key: HBASE-10182
> URL: https://issues.apache.org/jira/browse/HBASE-10182
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Jimmy Xiang
>Priority: Trivial
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: trunk-10182.patch
>
>
> Here is the related code, starting line 921:
> {code}
> if (regionState == null
> || !regionState.isPendingOpenOrOpeningOnServer(sn)) {
> LOG.warn("Received OPENED for " + prettyPrintedRegionName
> + " from " + sn + " but the region isn't PENDING_OPEN/OPENING
> here: "
> + regionStates.getRegionState(encodedName));
> // Close it without updating the internal region states,
> // so as not to create double assignments in unlucky scenarios
> // mentioned in OpenRegionHandler#process
> unassign(regionState.getRegion(), null, -1, null, false, sn);
> {code}
> If regionState is null, we should not dereference it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10179) HRegionServer underreports readRequestCounts by 1 under certain conditions
[ https://issues.apache.org/jira/browse/HBASE-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851469#comment-13851469 ] Hudson commented on HBASE-10179: SUCCESS: Integrated in hbase-0.96 #232 (See [https://builds.apache.org/job/hbase-0.96/232/]) HBASE-10179. HRegionServer underreports readRequestCounts by 1 under certain conditions (Perry Trolard) (apurtell: rev 1551796) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > HRegionServer underreports readRequestCounts by 1 under certain conditions > -- > > Key: HBASE-10179 > URL: https://issues.apache.org/jira/browse/HBASE-10179 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 0.94.6, 0.99.0 >Reporter: Perry Trolard >Assignee: Perry Trolard >Priority: Minor > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: 10179-against-0.94-branch.diff, 10179-against-trunk.diff > > > In HRegionServer.scan(), if > (a) the number of results returned, n, is greater than zero > (b) but less than the size of the batch (nbRows) > (c) and the size in bytes is smaller than the max size (maxScannerResultSize) > then the readRequestCount will be reported as n - 1 rather than n. (This is > because the for-loop counter i is used to update the readRequestCount, and if > the scan runs out of rows before reaching max rows or size, the code `break`s > out of the loop and i is not incremented for the final time.) > To reproduce, create a test table and open its details page in the web UI. > Insert a single row, then note the current request count, c. Scan the table, > returning 1 row; the request count will still be c, whereas it should be c + > 1. > I have a patch against TRUNK I can submit. At Splice Machine we're running > 0.94, & I'd be happy to submit a patch against that as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10137) GeneralBulkAssigner with retain assignment plan can be used in EnableTableHandler to bulk assign the regions
[
https://issues.apache.org/jira/browse/HBASE-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851467#comment-13851467
]
Hudson commented on HBASE-10137:
SUCCESS: Integrated in hbase-0.96 #232 (See
[https://builds.apache.org/job/hbase-0.96/232/])
HBASE-10137 GeneralBulkAssigner with retain assignment plan can be used in
EnableTableHandler to bulk assign the regions (rajeshbabu: rev 1551799)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/GeneralBulkAssigner.java
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java
> GeneralBulkAssigner with retain assignment plan can be used in
> EnableTableHandler to bulk assign the regions
>
>
> Key: HBASE-10137
> URL: https://issues.apache.org/jira/browse/HBASE-10137
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.96.0, 0.94.14
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-10137.patch, HBASE-10137_v2.patch,
> HBASE-10137_v3.patch
>
>
> Current in BulkEnabler we are assigning one region at a time, instead we can
> use GeneralBulkAssigner to bulk assign multiple regions at a time.
> {code}
> for (HRegionInfo region : regions) {
> if (assignmentManager.getRegionStates()
> .isRegionInTransition(region)) {
> continue;
> }
> final HRegionInfo hri = region;
> pool.execute(Trace.wrap("BulkEnabler.populatePool",new Runnable() {
> public void run() {
> assignmentManager.assign(hri, true);
> }
> }));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851463#comment-13851463
]
Hadoop QA commented on HBASE-10148:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619263/HBASE-10148_V3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8210//console
This message is automatically generated.
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch,
> HBASE-10148_V3.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=5810
[jira] [Commented] (HBASE-10195) "mvn site" build failed with OutOfMemoryError
[
https://issues.apache.org/jira/browse/HBASE-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851453#comment-13851453
]
Hadoop QA commented on HBASE-10195:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619261/hbase-10195.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8209//console
This message is automatically generated.
> "mvn site" build failed with OutOfMemoryError
> -
>
> Key: HBASE-10195
> URL: https://issues.apache.org/jira/browse/HBASE-10195
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.96.1
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.98.0, 0.96.2
>
> Attachments: hbase-10195.patch
>
>
> "mvn site" build command failed with OutOfMemoryError. It lasts for a while
> and our book reference guid seems out of date as well.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10169) Batch coprocessor
[ https://issues.apache.org/jira/browse/HBASE-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851449#comment-13851449 ] Jingcheng Du commented on HBASE-10169: -- Thanks Gary. I'll start to implement the #1 and #2 into different patches in the way you suggested. > Batch coprocessor > - > > Key: HBASE-10169 > URL: https://issues.apache.org/jira/browse/HBASE-10169 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 0.99.0 >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Attachments: Batch Coprocessor Design Document.docx, HBASE-10169.patch > > > This is designed to improve the coprocessor invocation in the client side. > Currently the coprocessor invocation is to send a call to each region. If > there’s one region server, and 100 regions are located in this server, each > coprocessor invocation will send 100 calls, each call uses a single thread in > the client side. The threads will run out soon when the coprocessor > invocations are heavy. > In this design, all the calls to the same region server will be grouped into > one in a single coprocessor invocation. This call will be spread into each > region in the server side, and the results will be merged ahead in the server > side before being returned to the client. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10192) Empty rowkey is not allowed any more
[ https://issues.apache.org/jira/browse/HBASE-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851440#comment-13851440 ] Devaraj Das commented on HBASE-10192: - Yeah, I had made the empty row key work in HBASE-3170. In HBASE-8101 it seems the decision was to not support empty row key at all... > Empty rowkey is not allowed any more > > > Key: HBASE-10192 > URL: https://issues.apache.org/jira/browse/HBASE-10192 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang > > In HBase-8101, we added a check and disallowed empty rowkey. However, it is > allowed in 0.94. I was wondering if there is any special reason for that, > i.e., is it by intention or by mistake? -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-9866) Support the mode where REST server authorizes proxy users
[
https://issues.apache.org/jira/browse/HBASE-9866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851443#comment-13851443
]
Devaraj Das commented on HBASE-9866:
Ping... Any comments. Good to commit?
> Support the mode where REST server authorizes proxy users
> -
>
> Key: HBASE-9866
> URL: https://issues.apache.org/jira/browse/HBASE-9866
> Project: HBase
> Issue Type: Improvement
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Fix For: 0.99.0
>
> Attachments: 9866-1.txt, 9866-2.txt, 9866-3.txt, 9866-4.txt
>
>
> In one use case, someone was trying to authorize with the REST server as a
> proxy user. That mode is not supported today.
> The curl request would be something like (assuming SPNEGO auth) -
> {noformat}
> curl -i --negotiate -u : http://:/version/cluster?doas=
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10148:
---
Attachment: HBASE-10148_V3.patch
Attaching again for QA to start
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch,
> HBASE-10148_V3.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1046): Adding the label XYZ2013-12-12
> 14:53:53,137 ERROR [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1074):
> org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> 2013-12-12 14:53:53,151 DEBUG [main] visibility.TestVisibilityLabels(405):
> response from addLabels: result {
> exception {
> name: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException"
> value: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1763) at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1749) at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.getExistingLabelsWithAuths(VisibilityController.java:1096)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postBatchMutate(VisibilityController.java:672)"
> {noformat}
> Should we try to ride over this?
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10148:
---
Status: Patch Available (was: Open)
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch,
> HBASE-10148_V3.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1046): Adding the label XYZ2013-12-12
> 14:53:53,137 ERROR [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1074):
> org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> 2013-12-12 14:53:53,151 DEBUG [main] visibility.TestVisibilityLabels(405):
> response from addLabels: result {
> exception {
> name: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException"
> value: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1763) at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1749) at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.getExistingLabelsWithAuths(VisibilityController.java:1096)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postBatchMutate(VisibilityController.java:672)"
> {noformat}
> Should we try to ride over this?
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851405#comment-13851405
]
Hadoop QA commented on HBASE-10148:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619247/HBASE-10148_V3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8208//console
This message is automatically generated.
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch,
> HBASE-10148_V3.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.
[jira] [Updated] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10148:
---
Status: Open (was: Patch Available)
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch,
> HBASE-10148_V3.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1046): Adding the label XYZ2013-12-12
> 14:53:53,137 ERROR [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1074):
> org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> 2013-12-12 14:53:53,151 DEBUG [main] visibility.TestVisibilityLabels(405):
> response from addLabels: result {
> exception {
> name: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException"
> value: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1763) at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1749) at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.getExistingLabelsWithAuths(VisibilityController.java:1096)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postBatchMutate(VisibilityController.java:672)"
> {noformat}
> Should we try to ride over this?
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10148:
---
Attachment: (was: HBASE-10148_V3.patch)
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1046): Adding the label XYZ2013-12-12
> 14:53:53,137 ERROR [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1074):
> org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> 2013-12-12 14:53:53,151 DEBUG [main] visibility.TestVisibilityLabels(405):
> response from addLabels: result {
> exception {
> name: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException"
> value: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1763) at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1749) at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.getExistingLabelsWithAuths(VisibilityController.java:1096)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postBatchMutate(VisibilityController.java:672)"
> {noformat}
> Should we try to ride over this?
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10179) HRegionServer underreports readRequestCounts by 1 under certain conditions
[ https://issues.apache.org/jira/browse/HBASE-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851404#comment-13851404 ] Hudson commented on HBASE-10179: FAILURE: Integrated in HBase-0.98 #20 (See [https://builds.apache.org/job/HBase-0.98/20/]) HBASE-10179. HRegionServer underreports readRequestCounts by 1 under certain conditions (Perry Trolard) (apurtell: rev 1551795) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > HRegionServer underreports readRequestCounts by 1 under certain conditions > -- > > Key: HBASE-10179 > URL: https://issues.apache.org/jira/browse/HBASE-10179 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 0.94.6, 0.99.0 >Reporter: Perry Trolard >Assignee: Perry Trolard >Priority: Minor > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: 10179-against-0.94-branch.diff, 10179-against-trunk.diff > > > In HRegionServer.scan(), if > (a) the number of results returned, n, is greater than zero > (b) but less than the size of the batch (nbRows) > (c) and the size in bytes is smaller than the max size (maxScannerResultSize) > then the readRequestCount will be reported as n - 1 rather than n. (This is > because the for-loop counter i is used to update the readRequestCount, and if > the scan runs out of rows before reaching max rows or size, the code `break`s > out of the loop and i is not incremented for the final time.) > To reproduce, create a test table and open its details page in the web UI. > Insert a single row, then note the current request count, c. Scan the table, > returning 1 row; the request count will still be c, whereas it should be c + > 1. > I have a patch against TRUNK I can submit. At Splice Machine we're running > 0.94, & I'd be happy to submit a patch against that as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10187) AccessController#preOpen - Include 'labels' table also into special tables list.
[ https://issues.apache.org/jira/browse/HBASE-10187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851401#comment-13851401 ] Hudson commented on HBASE-10187: FAILURE: Integrated in HBase-0.98 #20 (See [https://builds.apache.org/job/HBase-0.98/20/]) HBASE-10187 AccessController#preOpen - Include 'labels' table also into special tables list. (anoopsamjohn: rev 1551817) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java > AccessController#preOpen - Include 'labels' table also into special tables > list. > > > Key: HBASE-10187 > URL: https://issues.apache.org/jira/browse/HBASE-10187 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Minor > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-10187.patch > > > Now we have META, acl and namespace tables in this list. Or better we can > chek here whether the table namespace is system namespace (ie. "hbase")? -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851400#comment-13851400
]
Hudson commented on HBASE-10189:
FAILURE: Integrated in HBase-0.98 #20 (See
[https://builds.apache.org/job/HBase-0.98/20/])
HBASE-10189 Intermittent TestReplicationSyncUpTool failure (LarsH and Demai Ni)
(larsh: rev 1551790)
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSyncUpTool.java
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10182) Potential null object deference in AssignmentManager#handleRegion()
[
https://issues.apache.org/jira/browse/HBASE-10182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851403#comment-13851403
]
Hudson commented on HBASE-10182:
FAILURE: Integrated in HBase-0.98 #20 (See
[https://builds.apache.org/job/HBase-0.98/20/])
HBASE-10182 Potential null object deference in AssignmentManager#handleRegion()
(jxiang: rev 1551772)
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> Potential null object deference in AssignmentManager#handleRegion()
> ---
>
> Key: HBASE-10182
> URL: https://issues.apache.org/jira/browse/HBASE-10182
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Jimmy Xiang
>Priority: Trivial
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: trunk-10182.patch
>
>
> Here is the related code, starting line 921:
> {code}
> if (regionState == null
> || !regionState.isPendingOpenOrOpeningOnServer(sn)) {
> LOG.warn("Received OPENED for " + prettyPrintedRegionName
> + " from " + sn + " but the region isn't PENDING_OPEN/OPENING
> here: "
> + regionStates.getRegionState(encodedName));
> // Close it without updating the internal region states,
> // so as not to create double assignments in unlucky scenarios
> // mentioned in OpenRegionHandler#process
> unassign(regionState.getRegion(), null, -1, null, false, sn);
> {code}
> If regionState is null, we should not dereference it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10137) GeneralBulkAssigner with retain assignment plan can be used in EnableTableHandler to bulk assign the regions
[
https://issues.apache.org/jira/browse/HBASE-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851402#comment-13851402
]
Hudson commented on HBASE-10137:
FAILURE: Integrated in HBase-0.98 #20 (See
[https://builds.apache.org/job/HBase-0.98/20/])
HBASE-10137 GeneralBulkAssigner with retain assignment plan can be used in
EnableTableHandler to bulk assign the regions (rajeshbabu: rev 1551798)
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/GeneralBulkAssigner.java
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java
> GeneralBulkAssigner with retain assignment plan can be used in
> EnableTableHandler to bulk assign the regions
>
>
> Key: HBASE-10137
> URL: https://issues.apache.org/jira/browse/HBASE-10137
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.96.0, 0.94.14
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-10137.patch, HBASE-10137_v2.patch,
> HBASE-10137_v3.patch
>
>
> Current in BulkEnabler we are assigning one region at a time, instead we can
> use GeneralBulkAssigner to bulk assign multiple regions at a time.
> {code}
> for (HRegionInfo region : regions) {
> if (assignmentManager.getRegionStates()
> .isRegionInTransition(region)) {
> continue;
> }
> final HRegionInfo hri = region;
> pool.execute(Trace.wrap("BulkEnabler.populatePool",new Runnable() {
> public void run() {
> assignmentManager.assign(hri, true);
> }
> }));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10159) Replaced deprecated interface Closeable
[ https://issues.apache.org/jira/browse/HBASE-10159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851399#comment-13851399 ] Hudson commented on HBASE-10159: FAILURE: Integrated in HBase-0.98 #20 (See [https://builds.apache.org/job/HBase-0.98/20/]) HBASE-10159 Replaced deprecated interface Closeable (jxiang: rev 1551770) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java > Replaced deprecated interface Closeable > --- > > Key: HBASE-10159 > URL: https://issues.apache.org/jira/browse/HBASE-10159 > Project: HBase > Issue Type: Task >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Trivial > Fix For: 0.98.0, 0.96.2, 0.99.0 > > Attachments: trunk-10159.patch, trunk-10159_v2.patch > > > Hadoop interface Closeable has been deprecated for quite some time to favor > java.io.Closeable. Since we support only hadoop 1.0 and above now, we can > replace the deprecated Closeable with the java one. We can also clean up some > hack we did as well, for example, HBASE-10029. > https://github.com/apache/hadoop-common/blob/branch-1.0/src/core/org/apache/hadoop/io/Closeable.java > -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10179) HRegionServer underreports readRequestCounts by 1 under certain conditions
[ https://issues.apache.org/jira/browse/HBASE-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851393#comment-13851393 ] Hudson commented on HBASE-10179: ABORTED: Integrated in HBase-0.94 #1231 (See [https://builds.apache.org/job/HBase-0.94/1231/]) HBASE-10179. HRegionServer underreports readRequestCounts by 1 under certain conditions (Perry Trolard) (apurtell: rev 1551800) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > HRegionServer underreports readRequestCounts by 1 under certain conditions > -- > > Key: HBASE-10179 > URL: https://issues.apache.org/jira/browse/HBASE-10179 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 0.94.6, 0.99.0 >Reporter: Perry Trolard >Assignee: Perry Trolard >Priority: Minor > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: 10179-against-0.94-branch.diff, 10179-against-trunk.diff > > > In HRegionServer.scan(), if > (a) the number of results returned, n, is greater than zero > (b) but less than the size of the batch (nbRows) > (c) and the size in bytes is smaller than the max size (maxScannerResultSize) > then the readRequestCount will be reported as n - 1 rather than n. (This is > because the for-loop counter i is used to update the readRequestCount, and if > the scan runs out of rows before reaching max rows or size, the code `break`s > out of the loop and i is not incremented for the final time.) > To reproduce, create a test table and open its details page in the web UI. > Insert a single row, then note the current request count, c. Scan the table, > returning 1 row; the request count will still be c, whereas it should be c + > 1. > I have a patch against TRUNK I can submit. At Splice Machine we're running > 0.94, & I'd be happy to submit a patch against that as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851392#comment-13851392
]
Hudson commented on HBASE-10189:
ABORTED: Integrated in HBase-0.94 #1231 (See
[https://builds.apache.org/job/HBase-0.94/1231/])
HBASE-10189 Intermittent TestReplicationSyncUpTool failure (LarsH and Demai Ni)
(larsh: rev 1551785)
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSyncUpTool.java
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HBASE-10196) Enhance HBCK to understand the case after online region merge
[
https://issues.apache.org/jira/browse/HBASE-10196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chunhui shen updated HBASE-10196:
-
Description:
HBCK will report an error now after the online merge,
because the files of merging regions still remain on HDFS which will be cleaned
by CatalogJanitor later.
ERROR: Region { meta => null, hdfs =>
hdfs://hbasetest1:9000/hbase/data/default/dns/c6569a72cc3c2750d14976ab85f02315,
deployed => } on HDFS, but not listed in hbase:meta or deployed on any region
server
> Enhance HBCK to understand the case after online region merge
> -
>
> Key: HBASE-10196
> URL: https://issues.apache.org/jira/browse/HBASE-10196
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.2, 0.98.1, 0.99.0
>
>
> HBCK will report an error now after the online merge,
> because the files of merging regions still remain on HDFS which will be
> cleaned by CatalogJanitor later.
> ERROR: Region { meta => null, hdfs =>
> hdfs://hbasetest1:9000/hbase/data/default/dns/c6569a72cc3c2750d14976ab85f02315,
> deployed => } on HDFS, but not listed in hbase:meta or deployed on any
> region server
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HBASE-10195) "mvn site" build failed with OutOfMemoryError
[ https://issues.apache.org/jira/browse/HBASE-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-10195: -- Status: Patch Available (was: Open) > "mvn site" build failed with OutOfMemoryError > - > > Key: HBASE-10195 > URL: https://issues.apache.org/jira/browse/HBASE-10195 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.96.2 > > Attachments: hbase-10195.patch > > > "mvn site" build command failed with OutOfMemoryError. It lasts for a while > and our book reference guid seems out of date as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Created] (HBASE-10196) Enhance HBCK to understand the case after online region merge
chunhui shen created HBASE-10196: Summary: Enhance HBCK to understand the case after online region merge Key: HBASE-10196 URL: https://issues.apache.org/jira/browse/HBASE-10196 Project: HBase Issue Type: Bug Reporter: chunhui shen Assignee: chunhui shen Fix For: 0.96.2, 0.98.1, 0.99.0 -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10195) "mvn site" build failed with OutOfMemoryError
[ https://issues.apache.org/jira/browse/HBASE-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-10195: -- Attachment: hbase-10195.patch I also piggybacked a little change on the book reference guid. > "mvn site" build failed with OutOfMemoryError > - > > Key: HBASE-10195 > URL: https://issues.apache.org/jira/browse/HBASE-10195 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.96.2 > > Attachments: hbase-10195.patch > > > "mvn site" build command failed with OutOfMemoryError. It lasts for a while > and our book reference guid seems out of date as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10195) "mvn site" build failed with OutOfMemoryError
[ https://issues.apache.org/jira/browse/HBASE-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-10195: -- Affects Version/s: 0.96.1 > "mvn site" build failed with OutOfMemoryError > - > > Key: HBASE-10195 > URL: https://issues.apache.org/jira/browse/HBASE-10195 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.96.2 > > > "mvn site" build command failed with OutOfMemoryError. It lasts for a while > and our book reference guid seems out of date as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10179) HRegionServer underreports readRequestCounts by 1 under certain conditions
[ https://issues.apache.org/jira/browse/HBASE-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851383#comment-13851383 ] Hudson commented on HBASE-10179: FAILURE: Integrated in HBase-TRUNK #4734 (See [https://builds.apache.org/job/HBase-TRUNK/4734/]) HBASE-10179. HRegionServer underreports readRequestCounts by 1 under certain conditions (Perry Trolard) (apurtell: rev 1551794) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > HRegionServer underreports readRequestCounts by 1 under certain conditions > -- > > Key: HBASE-10179 > URL: https://issues.apache.org/jira/browse/HBASE-10179 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 0.94.6, 0.99.0 >Reporter: Perry Trolard >Assignee: Perry Trolard >Priority: Minor > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: 10179-against-0.94-branch.diff, 10179-against-trunk.diff > > > In HRegionServer.scan(), if > (a) the number of results returned, n, is greater than zero > (b) but less than the size of the batch (nbRows) > (c) and the size in bytes is smaller than the max size (maxScannerResultSize) > then the readRequestCount will be reported as n - 1 rather than n. (This is > because the for-loop counter i is used to update the readRequestCount, and if > the scan runs out of rows before reaching max rows or size, the code `break`s > out of the loop and i is not incremented for the final time.) > To reproduce, create a test table and open its details page in the web UI. > Insert a single row, then note the current request count, c. Scan the table, > returning 1 row; the request count will still be c, whereas it should be c + > 1. > I have a patch against TRUNK I can submit. At Splice Machine we're running > 0.94, & I'd be happy to submit a patch against that as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10195) "mvn site" build failed with OutOfMemoryError
[ https://issues.apache.org/jira/browse/HBASE-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-10195: -- Fix Version/s: 0.96.2 0.98.0 > "mvn site" build failed with OutOfMemoryError > - > > Key: HBASE-10195 > URL: https://issues.apache.org/jira/browse/HBASE-10195 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.96.2 > > > "mvn site" build command failed with OutOfMemoryError. It lasts for a while > and our book reference guid seems out of date as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851379#comment-13851379
]
Hudson commented on HBASE-10189:
FAILURE: Integrated in HBase-TRUNK #4734 (See
[https://builds.apache.org/job/HBase-TRUNK/4734/])
HBASE-10189 Intermittent TestReplicationSyncUpTool failure (LarsH and Demai Ni)
(larsh: rev 1551788)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSyncUpTool.java
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10182) Potential null object deference in AssignmentManager#handleRegion()
[
https://issues.apache.org/jira/browse/HBASE-10182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851381#comment-13851381
]
Hudson commented on HBASE-10182:
FAILURE: Integrated in HBase-TRUNK #4734 (See
[https://builds.apache.org/job/HBase-TRUNK/4734/])
HBASE-10182 Potential null object deference in AssignmentManager#handleRegion()
(jxiang: rev 1551762)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> Potential null object deference in AssignmentManager#handleRegion()
> ---
>
> Key: HBASE-10182
> URL: https://issues.apache.org/jira/browse/HBASE-10182
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Jimmy Xiang
>Priority: Trivial
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: trunk-10182.patch
>
>
> Here is the related code, starting line 921:
> {code}
> if (regionState == null
> || !regionState.isPendingOpenOrOpeningOnServer(sn)) {
> LOG.warn("Received OPENED for " + prettyPrintedRegionName
> + " from " + sn + " but the region isn't PENDING_OPEN/OPENING
> here: "
> + regionStates.getRegionState(encodedName));
> // Close it without updating the internal region states,
> // so as not to create double assignments in unlucky scenarios
> // mentioned in OpenRegionHandler#process
> unassign(regionState.getRegion(), null, -1, null, false, sn);
> {code}
> If regionState is null, we should not dereference it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Created] (HBASE-10195) "mvn site" build failed with OutOfMemoryError
Jeffrey Zhong created HBASE-10195: - Summary: "mvn site" build failed with OutOfMemoryError Key: HBASE-10195 URL: https://issues.apache.org/jira/browse/HBASE-10195 Project: HBase Issue Type: Bug Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong "mvn site" build command failed with OutOfMemoryError. It lasts for a while and our book reference guid seems out of date as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10137) GeneralBulkAssigner with retain assignment plan can be used in EnableTableHandler to bulk assign the regions
[
https://issues.apache.org/jira/browse/HBASE-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851380#comment-13851380
]
Hudson commented on HBASE-10137:
FAILURE: Integrated in HBase-TRUNK #4734 (See
[https://builds.apache.org/job/HBase-TRUNK/4734/])
HBASE-10137 GeneralBulkAssigner with retain assignment plan can be used in
EnableTableHandler to bulk assign the regions (rajeshbabu: rev 1551797)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/GeneralBulkAssigner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java
> GeneralBulkAssigner with retain assignment plan can be used in
> EnableTableHandler to bulk assign the regions
>
>
> Key: HBASE-10137
> URL: https://issues.apache.org/jira/browse/HBASE-10137
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.96.0, 0.94.14
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-10137.patch, HBASE-10137_v2.patch,
> HBASE-10137_v3.patch
>
>
> Current in BulkEnabler we are assigning one region at a time, instead we can
> use GeneralBulkAssigner to bulk assign multiple regions at a time.
> {code}
> for (HRegionInfo region : regions) {
> if (assignmentManager.getRegionStates()
> .isRegionInTransition(region)) {
> continue;
> }
> final HRegionInfo hri = region;
> pool.execute(Trace.wrap("BulkEnabler.populatePool",new Runnable() {
> public void run() {
> assignmentManager.assign(hri, true);
> }
> }));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10142) TestLogRolling#testLogRollOnDatanodeDeath test failure
[
https://issues.apache.org/jira/browse/HBASE-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851382#comment-13851382
]
Hudson commented on HBASE-10142:
FAILURE: Integrated in HBase-TRUNK #4734 (See
[https://builds.apache.org/job/HBase-TRUNK/4734/])
HBASE-10142 TestLogRolling#testLogRollOnDatanodeDeath test failure (tedyu: rev
1551806)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
> TestLogRolling#testLogRollOnDatanodeDeath test failure
> --
>
> Key: HBASE-10142
> URL: https://issues.apache.org/jira/browse/HBASE-10142
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Andrew Purtell
>Assignee: Ted Yu
> Fix For: 0.98.0
>
> Attachments: 10142-v1.txt
>
>
> This is a demanding unit test, which fails fairly often as software versions
> (JVM, Hadoop) and system load change. Currently when testing 0.98 branch I
> see this failure:
> {noformat}
> Failed tests:
> testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling):
> LowReplication Roller should've been disabled, current replication=1
> {noformat}
> Could be a timing issue after the recent switch to Hadoop 2 as default
> build/test profile. Let's see if more leniency makes sense and if it can
> stabilize the test before disabling it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851373#comment-13851373
]
Hudson commented on HBASE-10189:
FAILURE: Integrated in HBase-0.94-security #364 (See
[https://builds.apache.org/job/HBase-0.94-security/364/])
HBASE-10189 Intermittent TestReplicationSyncUpTool failure (LarsH and Demai Ni)
(larsh: rev 1551785)
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSyncUpTool.java
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10179) HRegionServer underreports readRequestCounts by 1 under certain conditions
[ https://issues.apache.org/jira/browse/HBASE-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851374#comment-13851374 ] Hudson commented on HBASE-10179: FAILURE: Integrated in HBase-0.94-security #364 (See [https://builds.apache.org/job/HBase-0.94-security/364/]) HBASE-10179. HRegionServer underreports readRequestCounts by 1 under certain conditions (Perry Trolard) (apurtell: rev 1551800) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > HRegionServer underreports readRequestCounts by 1 under certain conditions > -- > > Key: HBASE-10179 > URL: https://issues.apache.org/jira/browse/HBASE-10179 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 0.94.6, 0.99.0 >Reporter: Perry Trolard >Assignee: Perry Trolard >Priority: Minor > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: 10179-against-0.94-branch.diff, 10179-against-trunk.diff > > > In HRegionServer.scan(), if > (a) the number of results returned, n, is greater than zero > (b) but less than the size of the batch (nbRows) > (c) and the size in bytes is smaller than the max size (maxScannerResultSize) > then the readRequestCount will be reported as n - 1 rather than n. (This is > because the for-loop counter i is used to update the readRequestCount, and if > the scan runs out of rows before reaching max rows or size, the code `break`s > out of the loop and i is not incremented for the final time.) > To reproduce, create a test table and open its details page in the web UI. > Insert a single row, then note the current request count, c. Scan the table, > returning 1 row; the request count will still be c, whereas it should be c + > 1. > I have a patch against TRUNK I can submit. At Splice Machine we're running > 0.94, & I'd be happy to submit a patch against that as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851368#comment-13851368
]
Hudson commented on HBASE-10189:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #16 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/16/])
HBASE-10189 Intermittent TestReplicationSyncUpTool failure (LarsH and Demai Ni)
(larsh: rev 1551790)
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSyncUpTool.java
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10179) HRegionServer underreports readRequestCounts by 1 under certain conditions
[ https://issues.apache.org/jira/browse/HBASE-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851371#comment-13851371 ] Hudson commented on HBASE-10179: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #16 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/16/]) HBASE-10179. HRegionServer underreports readRequestCounts by 1 under certain conditions (Perry Trolard) (apurtell: rev 1551795) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > HRegionServer underreports readRequestCounts by 1 under certain conditions > -- > > Key: HBASE-10179 > URL: https://issues.apache.org/jira/browse/HBASE-10179 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 0.94.6, 0.99.0 >Reporter: Perry Trolard >Assignee: Perry Trolard >Priority: Minor > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: 10179-against-0.94-branch.diff, 10179-against-trunk.diff > > > In HRegionServer.scan(), if > (a) the number of results returned, n, is greater than zero > (b) but less than the size of the batch (nbRows) > (c) and the size in bytes is smaller than the max size (maxScannerResultSize) > then the readRequestCount will be reported as n - 1 rather than n. (This is > because the for-loop counter i is used to update the readRequestCount, and if > the scan runs out of rows before reaching max rows or size, the code `break`s > out of the loop and i is not incremented for the final time.) > To reproduce, create a test table and open its details page in the web UI. > Insert a single row, then note the current request count, c. Scan the table, > returning 1 row; the request count will still be c, whereas it should be c + > 1. > I have a patch against TRUNK I can submit. At Splice Machine we're running > 0.94, & I'd be happy to submit a patch against that as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10182) Potential null object deference in AssignmentManager#handleRegion()
[
https://issues.apache.org/jira/browse/HBASE-10182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851370#comment-13851370
]
Hudson commented on HBASE-10182:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #16 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/16/])
HBASE-10182 Potential null object deference in AssignmentManager#handleRegion()
(jxiang: rev 1551772)
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> Potential null object deference in AssignmentManager#handleRegion()
> ---
>
> Key: HBASE-10182
> URL: https://issues.apache.org/jira/browse/HBASE-10182
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Jimmy Xiang
>Priority: Trivial
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: trunk-10182.patch
>
>
> Here is the related code, starting line 921:
> {code}
> if (regionState == null
> || !regionState.isPendingOpenOrOpeningOnServer(sn)) {
> LOG.warn("Received OPENED for " + prettyPrintedRegionName
> + " from " + sn + " but the region isn't PENDING_OPEN/OPENING
> here: "
> + regionStates.getRegionState(encodedName));
> // Close it without updating the internal region states,
> // so as not to create double assignments in unlucky scenarios
> // mentioned in OpenRegionHandler#process
> unassign(regionState.getRegion(), null, -1, null, false, sn);
> {code}
> If regionState is null, we should not dereference it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10159) Replaced deprecated interface Closeable
[ https://issues.apache.org/jira/browse/HBASE-10159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851367#comment-13851367 ] Hudson commented on HBASE-10159: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #16 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/16/]) HBASE-10159 Replaced deprecated interface Closeable (jxiang: rev 1551770) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java > Replaced deprecated interface Closeable > --- > > Key: HBASE-10159 > URL: https://issues.apache.org/jira/browse/HBASE-10159 > Project: HBase > Issue Type: Task >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Trivial > Fix For: 0.98.0, 0.96.2, 0.99.0 > > Attachments: trunk-10159.patch, trunk-10159_v2.patch > > > Hadoop interface Closeable has been deprecated for quite some time to favor > java.io.Closeable. Since we support only hadoop 1.0 and above now, we can > replace the deprecated Closeable with the java one. We can also clean up some > hack we did as well, for example, HBASE-10029. > https://github.com/apache/hadoop-common/blob/branch-1.0/src/core/org/apache/hadoop/io/Closeable.java > -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10137) GeneralBulkAssigner with retain assignment plan can be used in EnableTableHandler to bulk assign the regions
[
https://issues.apache.org/jira/browse/HBASE-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851369#comment-13851369
]
Hudson commented on HBASE-10137:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #16 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/16/])
HBASE-10137 GeneralBulkAssigner with retain assignment plan can be used in
EnableTableHandler to bulk assign the regions (rajeshbabu: rev 1551798)
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/GeneralBulkAssigner.java
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java
> GeneralBulkAssigner with retain assignment plan can be used in
> EnableTableHandler to bulk assign the regions
>
>
> Key: HBASE-10137
> URL: https://issues.apache.org/jira/browse/HBASE-10137
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.96.0, 0.94.14
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-10137.patch, HBASE-10137_v2.patch,
> HBASE-10137_v3.patch
>
>
> Current in BulkEnabler we are assigning one region at a time, instead we can
> use GeneralBulkAssigner to bulk assign multiple regions at a time.
> {code}
> for (HRegionInfo region : regions) {
> if (assignmentManager.getRegionStates()
> .isRegionInTransition(region)) {
> continue;
> }
> final HRegionInfo hri = region;
> pool.execute(Trace.wrap("BulkEnabler.populatePool",new Runnable() {
> public void run() {
> assignmentManager.assign(hri, true);
> }
> }));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HBASE-9721) meta assignment did not timeout
[
https://issues.apache.org/jira/browse/HBASE-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-9721:
-
Attachment: hbase-9721_v0.patch
Here is a patch which adds ServerName to open and close region RPCs. The region
server rejects the request if it is not the intended server to receive the
request.
Will try to come up with a unit test tomorrow.
> meta assignment did not timeout
> ---
>
> Key: HBASE-9721
> URL: https://issues.apache.org/jira/browse/HBASE-9721
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.96.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-9721_v0.patch
>
>
> On a test cluster, this following events happened with ITBLL and CM leading
> to meta being unavailable until master is restarted.
> An RS carrying meta died, and master assigned the region to one of the RSs.
> {code}
> 2013-10-03 23:30:06,611 INFO
> [MASTER_META_SERVER_OPERATIONS-gs-hdp2-secure-1380781860-hbase-12:6-1]
> master.AssignmentManager: Assigning hbase:meta,,1.1588230740 to
> gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820
> 2013-10-03 23:30:06,611 INFO
> [MASTER_META_SERVER_OPERATIONS-gs-hdp2-secure-1380781860-hbase-12:6-1]
> master.RegionStates: Transitioned {1588230740 state=OFFLINE,
> ts=1380843006601, server=null} to {1588230740 state=PENDING_OPEN,
> ts=1380843006611,
> server=gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820}
> 2013-10-03 23:30:06,611 DEBUG
> [MASTER_META_SERVER_OPERATIONS-gs-hdp2-secure-1380781860-hbase-12:6-1]
> master.ServerManager: New admin connection to
> gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820
> {code}
> At the same time, the RS that meta recently got assigned also died (due to
> CM), and restarted:
> {code}
> 2013-10-03 23:30:07,636 DEBUG [RpcServer.handler=17,port=6]
> master.ServerManager: REPORT: Server
> gs-hdp2-secure-1380781860-hbase-8.cs1cloud.internal,60020,1380843002494 came
> back up, removed it from the dead servers list
> 2013-10-03 23:30:08,769 INFO [RpcServer.handler=18,port=6]
> master.ServerManager: Triggering server recovery; existingServer
> gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820 looks
> stale, new
> server:gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380843006362
> 2013-10-03 23:30:08,771 DEBUG [RpcServer.handler=18,port=6]
> master.AssignmentManager: Checking region=hbase:meta,,1.1588230740, zk
> server=gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820
>
> current=gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820,
> matches=true
> 2013-10-03 23:30:08,771 DEBUG [RpcServer.handler=18,port=6]
> master.ServerManager:
> Added=gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820
> to dead servers, submitted shutdown handler to be executed meta=true
> 2013-10-03 23:30:08,771 INFO [RpcServer.handler=18,port=6]
> master.ServerManager: Registering
> server=gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380843006362
> 2013-10-03 23:30:08,772 INFO
> [MASTER_META_SERVER_OPERATIONS-gs-hdp2-secure-1380781860-hbase-12:6-2]
> handler.MetaServerShutdownHandler: Splitting hbase:meta logs for
> gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820
> {code}
> AM/SSH sees that the RS that died was carrying meta, but the assignment RPC
> request was still not sent:
> {code}
> 2013-10-03 23:30:08,791 DEBUG
> [MASTER_META_SERVER_OPERATIONS-gs-hdp2-secure-1380781860-hbase-12:6-2]
> master.AssignmentManager: Checking region=hbase:meta,,1.1588230740, zk
> server=gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820
>
> current=gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820,
> matches=true
> 2013-10-03 23:30:08,791 INFO
> [MASTER_META_SERVER_OPERATIONS-gs-hdp2-secure-1380781860-hbase-12:6-2]
> handler.MetaServerShutdownHandler: Server
> gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820 was
> carrying META. Trying to assign.
> 2013-10-03 23:30:08,791 DEBUG
> [MASTER_META_SERVER_OPERATIONS-gs-hdp2-secure-1380781860-hbase-12:6-2]
> master.RegionStates: Offline 1588230740 with current state=PENDING_OPEN,
> expected state=OFFLINE/SPLITTING/MERGING
> 2013-10-03 23:30:08,791 INFO
> [MASTER_META_SERVER_OPERATIONS-gs-hdp2-secure-1380781860-hbase-12:6-2]
> master.RegionStates: Transitioned {1588230740 state=PENDING_OPEN,
> ts=1380843006611,
> server=gs-hdp2-secure-1380781860-hbase-5.cs1cloud.internal,60020,1380842900820}
> to {1588230740 state=OFFLINE, ts=1380843008791, server=null}
> 2013-10-03 23:30:09,809 INFO
> [MASTER_META_SERVER_OPERATIONS-gs-hdp2-secure-1380
[jira] [Commented] (HBASE-10142) TestLogRolling#testLogRollOnDatanodeDeath test failure
[
https://issues.apache.org/jira/browse/HBASE-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851338#comment-13851338
]
Ted Yu commented on HBASE-10142:
10 iterations of TestLogRolling passed on Linux.
> TestLogRolling#testLogRollOnDatanodeDeath test failure
> --
>
> Key: HBASE-10142
> URL: https://issues.apache.org/jira/browse/HBASE-10142
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Andrew Purtell
>Assignee: Ted Yu
> Fix For: 0.98.0
>
> Attachments: 10142-v1.txt
>
>
> This is a demanding unit test, which fails fairly often as software versions
> (JVM, Hadoop) and system load change. Currently when testing 0.98 branch I
> see this failure:
> {noformat}
> Failed tests:
> testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling):
> LowReplication Roller should've been disabled, current replication=1
> {noformat}
> Could be a timing issue after the recent switch to Hadoop 2 as default
> build/test profile. Let's see if more leniency makes sense and if it can
> stabilize the test before disabling it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-8701) distributedLogReplay need to apply wal edits in the receiving order of those edits
[
https://issues.apache.org/jira/browse/HBASE-8701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851337#comment-13851337
]
Hadoop QA commented on HBASE-8701:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619233/hbase-8701-tag-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8207//console
This message is automatically generated.
> distributedLogReplay need to apply wal edits in the receiving order of those
> edits
> --
>
> Key: HBASE-8701
> URL: https://issues.apache.org/jira/browse/HBASE-8701
> Project: HBase
> Issue Type: Bug
> Components: MTTR
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.98.0
>
> Attachments: 8701-v3.txt, hbase-8701-tag-v1.patch,
> hbase-8701-tag-v2.patch, hbase-8701-tag.patch, hbase-8701-v4.patch,
> hbase-8701-v5.patch, hbase-8701-v6.patch, hbase-8701-v7.patch,
> hbase-8701-v8.patch
>
>
> This issue happens in distributedLogReplay mode when recovering multiple puts
> of the same key + version(timestamp). After replay, the value is
> nondeterministic of the key
> h5. The original concern situation raised from [~eclark]:
> For all edits the rowkey is the same.
> There's a log with: [ A (ts = 0), B (ts = 0) ]
> Replay the first half of the log.
> A user puts in C (ts = 0)
> Memstore has to flush
> A new Hfile will be created with [ C, A ] and MaxSequenceId = C's seqid.
> Replay the rest of the Log.
> Flush
> The issue will happen in similar situation like Put(key, t=T) in WAL1 and
> Put(key,t=T) in WAL2
> h5. Below is the option(proposed by Ted) I'd like to use:
> a) During replay, we pass original wal sequence number of each edit to the
> receiving RS
> b) In receiving RS, we store negative original sequence number of wal edits
> into mvcc field of KVs of wal edits
> c) Add handling of negative MVCC in KVScannerComparator and KVComparator
> d) In receiving RS, write original sequence number into an optional field of
> wal file for chained RS failure situation
> e) When opening a regi
[jira] [Commented] (HBASE-10173) Need HFile version check in security coprocessors
[ https://issues.apache.org/jira/browse/HBASE-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851335#comment-13851335 ] Andrew Purtell commented on HBASE-10173: Ok, I'll do this one tomorrow. Thanks Anoop. > Need HFile version check in security coprocessors > - > > Key: HBASE-10173 > URL: https://issues.apache.org/jira/browse/HBASE-10173 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.0, 0.99.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-10173.patch, HBASE-10173_partial.patch > > > Cell level visibility labels are stored as cell tags. So HFile V3 is the > minimum version which can support this feature. Better to have a version > check in VisibilityController. Some one using this CP but with any HFile > version as V2, we can better throw error. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851332#comment-13851332
]
Anoop Sam John commented on HBASE-10148:
Woww.. that is super fast... Thanks Andy...
Will commit once QA passes with V3 patch.
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch,
> HBASE-10148_V3.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1046): Adding the label XYZ2013-12-12
> 14:53:53,137 ERROR [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1074):
> org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> 2013-12-12 14:53:53,151 DEBUG [main] visibility.TestVisibilityLabels(405):
> response from addLabels: result {
> exception {
> name: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException"
> value: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1763) at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1749) at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.getExistingLabelsWithAuths(VisibilityController.java:1096)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postBatchMutate(VisibilityController.java:672)"
> {noformat}
> Should we try to ride over this?
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10173) Need HFile version check in security coprocessors
[ https://issues.apache.org/jira/browse/HBASE-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851331#comment-13851331 ] Anoop Sam John commented on HBASE-10173: bq. But using V3 is not mandatory for old style table/CF perm checking. Same I was also thinking.. That is how we should make it.. This patch doing the simple check in AccessController which makes it need HFile V3 Andy, Pls feel free to take the issue with the fix you would like to have in AccessController. > Need HFile version check in security coprocessors > - > > Key: HBASE-10173 > URL: https://issues.apache.org/jira/browse/HBASE-10173 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.0, 0.99.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-10173.patch, HBASE-10173_partial.patch > > > Cell level visibility labels are stored as cell tags. So HFile V3 is the > minimum version which can support this feature. Better to have a version > check in VisibilityController. Some one using this CP but with any HFile > version as V2, we can better throw error. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851330#comment-13851330
]
Andrew Purtell commented on HBASE-10148:
+1 on V3 patch
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch,
> HBASE-10148_V3.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1046): Adding the label XYZ2013-12-12
> 14:53:53,137 ERROR [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1074):
> org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> 2013-12-12 14:53:53,151 DEBUG [main] visibility.TestVisibilityLabels(405):
> response from addLabels: result {
> exception {
> name: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException"
> value: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1763) at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1749) at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.getExistingLabelsWithAuths(VisibilityController.java:1096)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postBatchMutate(VisibilityController.java:672)"
> {noformat}
> Should we try to ride over this?
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10148:
---
Attachment: HBASE-10148_V3.patch
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch,
> HBASE-10148_V3.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1046): Adding the label XYZ2013-12-12
> 14:53:53,137 ERROR [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1074):
> org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> 2013-12-12 14:53:53,151 DEBUG [main] visibility.TestVisibilityLabels(405):
> response from addLabels: result {
> exception {
> name: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException"
> value: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1763) at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1749) at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.getExistingLabelsWithAuths(VisibilityController.java:1096)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postBatchMutate(VisibilityController.java:672)"
> {noformat}
> Should we try to ride over this?
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851327#comment-13851327
]
Andrew Purtell commented on HBASE-10148:
bq. There is CoprocessorException , can we use that
Sounds good to me.
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1046): Adding the label XYZ2013-12-12
> 14:53:53,137 ERROR [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1074):
> org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> 2013-12-12 14:53:53,151 DEBUG [main] visibility.TestVisibilityLabels(405):
> response from addLabels: result {
> exception {
> name: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException"
> value: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1763) at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1749) at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.getExistingLabelsWithAuths(VisibilityController.java:1096)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postBatchMutate(VisibilityController.java:672)"
> {noformat}
> Should we try to ride over this?
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10173) Need HFile version check in security coprocessors
[ https://issues.apache.org/jira/browse/HBASE-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851326#comment-13851326 ] Andrew Purtell commented on HBASE-10173: bq. In 0.98 HFile V2 is the default version. If used want to use old way of AccessControl (Not cell level) still he has to make the version to V3 For cell level, yes V3 is needed. Otherwise it won't work. But using V3 is not mandatory for old style table/CF perm checking. > Need HFile version check in security coprocessors > - > > Key: HBASE-10173 > URL: https://issues.apache.org/jira/browse/HBASE-10173 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.0, 0.99.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-10173.patch, HBASE-10173_partial.patch > > > Cell level visibility labels are stored as cell tags. So HFile V3 is the > minimum version which can support this feature. Better to have a version > check in VisibilityController. Some one using this CP but with any HFile > version as V2, we can better throw error. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10173) Need HFile version check in security coprocessors
[ https://issues.apache.org/jira/browse/HBASE-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-10173: --- Attachment: HBASE-10173.patch Same change in AccessController.. Just copy paste :) Attaching that also here Andy... Pls feel free to take the issue if you wish to do it in some other way (mainly in AccessController) > Need HFile version check in security coprocessors > - > > Key: HBASE-10173 > URL: https://issues.apache.org/jira/browse/HBASE-10173 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.0, 0.99.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-10173.patch, HBASE-10173_partial.patch > > > Cell level visibility labels are stored as cell tags. So HFile V3 is the > minimum version which can support this feature. Better to have a version > check in VisibilityController. Some one using this CP but with any HFile > version as V2, we can better throw error. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851314#comment-13851314
]
Hudson commented on HBASE-10189:
FAILURE: Integrated in hbase-0.96-hadoop2 #153 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/153/])
HBASE-10189 Intermittent TestReplicationSyncUpTool failure (LarsH and Demai Ni)
(larsh: rev 1551792)
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSyncUpTool.java
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10188) Deprecate ServerName constructors, but make it public.
[ https://issues.apache.org/jira/browse/HBASE-10188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851313#comment-13851313 ] Hudson commented on HBASE-10188: FAILURE: Integrated in hbase-0.96-hadoop2 #153 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/153/]) HBASE-10188 Deprecate ServerName constructors, but make it public (Nicolas Liochon) (jmhsieh: rev 1551661) * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java > Deprecate ServerName constructors, but make it public. > -- > > Key: HBASE-10188 > URL: https://issues.apache.org/jira/browse/HBASE-10188 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.96.1 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.96.2, 0.96.1.1 > > Attachments: 10188.v1.patch, hbase-10188.v2.patch > > > It's in our public interface, we can't remove it that easily... See > HBASE-10012? -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10173) Need HFile version check in security coprocessors
[ https://issues.apache.org/jira/browse/HBASE-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-10173: --- Attachment: HBASE-10173_partial.patch > Need HFile version check in security coprocessors > - > > Key: HBASE-10173 > URL: https://issues.apache.org/jira/browse/HBASE-10173 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.0, 0.99.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-10173_partial.patch > > > Cell level visibility labels are stored as cell tags. So HFile V3 is the > minimum version which can support this feature. Better to have a version > check in VisibilityController. Some one using this CP but with any HFile > version as V2, we can better throw error. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10137) GeneralBulkAssigner with retain assignment plan can be used in EnableTableHandler to bulk assign the regions
[
https://issues.apache.org/jira/browse/HBASE-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851315#comment-13851315
]
Hudson commented on HBASE-10137:
FAILURE: Integrated in hbase-0.96-hadoop2 #153 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/153/])
HBASE-10137 GeneralBulkAssigner with retain assignment plan can be used in
EnableTableHandler to bulk assign the regions (rajeshbabu: rev 1551799)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/GeneralBulkAssigner.java
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java
> GeneralBulkAssigner with retain assignment plan can be used in
> EnableTableHandler to bulk assign the regions
>
>
> Key: HBASE-10137
> URL: https://issues.apache.org/jira/browse/HBASE-10137
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.96.0, 0.94.14
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-10137.patch, HBASE-10137_v2.patch,
> HBASE-10137_v3.patch
>
>
> Current in BulkEnabler we are assigning one region at a time, instead we can
> use GeneralBulkAssigner to bulk assign multiple regions at a time.
> {code}
> for (HRegionInfo region : regions) {
> if (assignmentManager.getRegionStates()
> .isRegionInTransition(region)) {
> continue;
> }
> final HRegionInfo hri = region;
> pool.execute(Trace.wrap("BulkEnabler.populatePool",new Runnable() {
> public void run() {
> assignmentManager.assign(hri, true);
> }
> }));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10179) HRegionServer underreports readRequestCounts by 1 under certain conditions
[ https://issues.apache.org/jira/browse/HBASE-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851316#comment-13851316 ] Hudson commented on HBASE-10179: FAILURE: Integrated in hbase-0.96-hadoop2 #153 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/153/]) HBASE-10179. HRegionServer underreports readRequestCounts by 1 under certain conditions (Perry Trolard) (apurtell: rev 1551796) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > HRegionServer underreports readRequestCounts by 1 under certain conditions > -- > > Key: HBASE-10179 > URL: https://issues.apache.org/jira/browse/HBASE-10179 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 0.94.6, 0.99.0 >Reporter: Perry Trolard >Assignee: Perry Trolard >Priority: Minor > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: 10179-against-0.94-branch.diff, 10179-against-trunk.diff > > > In HRegionServer.scan(), if > (a) the number of results returned, n, is greater than zero > (b) but less than the size of the batch (nbRows) > (c) and the size in bytes is smaller than the max size (maxScannerResultSize) > then the readRequestCount will be reported as n - 1 rather than n. (This is > because the for-loop counter i is used to update the readRequestCount, and if > the scan runs out of rows before reaching max rows or size, the code `break`s > out of the loop and i is not incremented for the final time.) > To reproduce, create a test table and open its details page in the web UI. > Insert a single row, then note the current request count, c. Scan the table, > returning 1 row; the request count will still be c, whereas it should be c + > 1. > I have a patch against TRUNK I can submit. At Splice Machine we're running > 0.94, & I'd be happy to submit a patch against that as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10173) Need HFile version check in security coprocessors
[ https://issues.apache.org/jira/browse/HBASE-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851312#comment-13851312 ] Anoop Sam John commented on HBASE-10173: bq.I can take this Anoop, let me know. NP.. Pls go ahead Andy. I am just attaching the patch I had initially which checks only in VisibilityController. bq.We need this in the AC too after HBASE-7662 In 0.98 HFile V2 is the default version. If used want to use old way of AccessControl (Not cell level) still he has to make the version to V3? I think the changes happened with AccessController demands us to do so. We can document this change then, some where in 0.98 release contents. > Need HFile version check in security coprocessors > - > > Key: HBASE-10173 > URL: https://issues.apache.org/jira/browse/HBASE-10173 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.0, 0.99.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.98.0, 0.99.0 > > > Cell level visibility labels are stored as cell tags. So HFile V3 is the > minimum version which can support this feature. Better to have a version > check in VisibilityController. Some one using this CP but with any HFile > version as V2, we can better throw error. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10192) Empty rowkey is not allowed any more
[ https://issues.apache.org/jira/browse/HBASE-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851311#comment-13851311 ] Jimmy Xiang commented on HBASE-10192: - For get, should we limit to return just one row? > Empty rowkey is not allowed any more > > > Key: HBASE-10192 > URL: https://issues.apache.org/jira/browse/HBASE-10192 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang > > In HBase-8101, we added a check and disallowed empty rowkey. However, it is > allowed in 0.94. I was wondering if there is any special reason for that, > i.e., is it by intention or by mistake? -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10159) Replaced deprecated interface Closeable
[ https://issues.apache.org/jira/browse/HBASE-10159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-10159: Fix Version/s: 0.96.2 > Replaced deprecated interface Closeable > --- > > Key: HBASE-10159 > URL: https://issues.apache.org/jira/browse/HBASE-10159 > Project: HBase > Issue Type: Task >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Trivial > Fix For: 0.98.0, 0.96.2, 0.99.0 > > Attachments: trunk-10159.patch, trunk-10159_v2.patch > > > Hadoop interface Closeable has been deprecated for quite some time to favor > java.io.Closeable. Since we support only hadoop 1.0 and above now, we can > replace the deprecated Closeable with the java one. We can also clean up some > hack we did as well, for example, HBASE-10029. > https://github.com/apache/hadoop-common/blob/branch-1.0/src/core/org/apache/hadoop/io/Closeable.java > -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-8701) distributedLogReplay need to apply wal edits in the receiving order of those edits
[ https://issues.apache.org/jira/browse/HBASE-8701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851310#comment-13851310 ] Anoop Sam John commented on HBASE-8701: --- Yes. HBASE-10148 will solve that.. Hope I can commit that today > distributedLogReplay need to apply wal edits in the receiving order of those > edits > -- > > Key: HBASE-8701 > URL: https://issues.apache.org/jira/browse/HBASE-8701 > Project: HBase > Issue Type: Bug > Components: MTTR >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0 > > Attachments: 8701-v3.txt, hbase-8701-tag-v1.patch, > hbase-8701-tag-v2.patch, hbase-8701-tag.patch, hbase-8701-v4.patch, > hbase-8701-v5.patch, hbase-8701-v6.patch, hbase-8701-v7.patch, > hbase-8701-v8.patch > > > This issue happens in distributedLogReplay mode when recovering multiple puts > of the same key + version(timestamp). After replay, the value is > nondeterministic of the key > h5. The original concern situation raised from [~eclark]: > For all edits the rowkey is the same. > There's a log with: [ A (ts = 0), B (ts = 0) ] > Replay the first half of the log. > A user puts in C (ts = 0) > Memstore has to flush > A new Hfile will be created with [ C, A ] and MaxSequenceId = C's seqid. > Replay the rest of the Log. > Flush > The issue will happen in similar situation like Put(key, t=T) in WAL1 and > Put(key,t=T) in WAL2 > h5. Below is the option(proposed by Ted) I'd like to use: > a) During replay, we pass original wal sequence number of each edit to the > receiving RS > b) In receiving RS, we store negative original sequence number of wal edits > into mvcc field of KVs of wal edits > c) Add handling of negative MVCC in KVScannerComparator and KVComparator > d) In receiving RS, write original sequence number into an optional field of > wal file for chained RS failure situation > e) When opening a region, we add a safety bumper(a large number) in order for > the new sequence number of a newly opened region not to collide with old > sequence numbers. > In the future, when we stores sequence number along with KVs, we can adjust > the above solution a little bit by avoiding to overload MVCC field. > h5. The other alternative options are listed below for references: > Option one > a) disallow writes during recovery > b) during replay, we pass original wal sequence ids > c) hold flush till all wals of a recovering region are replayed. Memstore > should hold because we only recover unflushed wal edits. For edits with same > key + version, whichever with larger sequence Id wins. > Option two > a) During replay, we pass original wal sequence ids > b) for each wal edit, we store each edit's original sequence id along with > its key. > c) during scanning, we use the original sequence id if it's present otherwise > its store file sequence Id > d) compaction can just leave put with max sequence id > Please let me know if you have better ideas. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10187) AccessController#preOpen - Include 'labels' table also into special tables list.
[ https://issues.apache.org/jira/browse/HBASE-10187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-10187: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and 0.98 . Thanks Andy > AccessController#preOpen - Include 'labels' table also into special tables > list. > > > Key: HBASE-10187 > URL: https://issues.apache.org/jira/browse/HBASE-10187 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Minor > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-10187.patch > > > Now we have META, acl and namespace tables in this list. Or better we can > chek here whether the table namespace is system namespace (ie. "hbase")? -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10182) Potential null object deference in AssignmentManager#handleRegion()
[
https://issues.apache.org/jira/browse/HBASE-10182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-10182:
Fix Version/s: 0.96.2
> Potential null object deference in AssignmentManager#handleRegion()
> ---
>
> Key: HBASE-10182
> URL: https://issues.apache.org/jira/browse/HBASE-10182
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Jimmy Xiang
>Priority: Trivial
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: trunk-10182.patch
>
>
> Here is the related code, starting line 921:
> {code}
> if (regionState == null
> || !regionState.isPendingOpenOrOpeningOnServer(sn)) {
> LOG.warn("Received OPENED for " + prettyPrintedRegionName
> + " from " + sn + " but the region isn't PENDING_OPEN/OPENING
> here: "
> + regionStates.getRegionState(encodedName));
> // Close it without updating the internal region states,
> // so as not to create double assignments in unlucky scenarios
> // mentioned in OpenRegionHandler#process
> unassign(regionState.getRegion(), null, -1, null, false, sn);
> {code}
> If regionState is null, we should not dereference it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10148) [VisibilityController] Tolerate regions in recovery
[
https://issues.apache.org/jira/browse/HBASE-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851304#comment-13851304
]
Anoop Sam John commented on HBASE-10148:
Got ur concern now Andy.. There is CoprocessorException , can we use that?
> [VisibilityController] Tolerate regions in recovery
> ---
>
> Key: HBASE-10148
> URL: https://issues.apache.org/jira/browse/HBASE-10148
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>Assignee: Anoop Sam John
>Priority: Blocker
> Fix For: 0.98.0
>
> Attachments: HBASE-10148.patch, HBASE-10148_V2.patch
>
>
> Ted Yu reports that enabling distributed log replay by default, like:
> {noformat}
> Index: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> ===
> --- hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (revision 1550575)
> +++ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> (working copy)
> @@ -794,7 +794,7 @@
>/** Conf key that enables unflushed WAL edits directly being replayed to
> region servers */
>public static final String DISTRIBUTED_LOG_REPLAY_KEY =
> "hbase.master.distributed.log.replay";
> - public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = false;
> + public static final boolean DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG = true;
>public static final String DISALLOW_WRITES_IN_RECOVERING =
>"hbase.regionserver.disallow.writes.when.recovering";
>public static final boolean DEFAULT_DISALLOW_WRITES_IN_RECOVERING_CONFIG =
> false;
> {noformat}
> causes TestVisibilityController#testAddVisibilityLabelsOnRSRestart to fail.
> It reveals an issue with label operations if the label table is recovering:
> {noformat}
> 2013-12-12 14:53:53,133 DEBUG [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1046): Adding the label XYZ2013-12-12
> 14:53:53,137 ERROR [RpcServer.handler=2,port=58108]
> visibility.VisibilityController(1074):
> org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> 2013-12-12 14:53:53,151 DEBUG [main] visibility.TestVisibilityLabels(405):
> response from addLabels: result {
> exception {
> name: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException"
> value: "org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:labels,,138626648.f14a399ba85cbb42c2c3b7547bf17c65. is recovering
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1763) at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1749) at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.getExistingLabelsWithAuths(VisibilityController.java:1096)
> at
> org.apache.hadoop.hbase.security.visibility.VisibilityController.postBatchMutate(VisibilityController.java:672)"
> {noformat}
> Should we try to ride over this?
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-8701) distributedLogReplay need to apply wal edits in the receiving order of those edits
[ https://issues.apache.org/jira/browse/HBASE-8701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851291#comment-13851291 ] Jeffrey Zhong commented on HBASE-8701: -- There are two test case failures under TestVisibilityLabels due to known issues when distributedLogReplay is on. > distributedLogReplay need to apply wal edits in the receiving order of those > edits > -- > > Key: HBASE-8701 > URL: https://issues.apache.org/jira/browse/HBASE-8701 > Project: HBase > Issue Type: Bug > Components: MTTR >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0 > > Attachments: 8701-v3.txt, hbase-8701-tag-v1.patch, > hbase-8701-tag-v2.patch, hbase-8701-tag.patch, hbase-8701-v4.patch, > hbase-8701-v5.patch, hbase-8701-v6.patch, hbase-8701-v7.patch, > hbase-8701-v8.patch > > > This issue happens in distributedLogReplay mode when recovering multiple puts > of the same key + version(timestamp). After replay, the value is > nondeterministic of the key > h5. The original concern situation raised from [~eclark]: > For all edits the rowkey is the same. > There's a log with: [ A (ts = 0), B (ts = 0) ] > Replay the first half of the log. > A user puts in C (ts = 0) > Memstore has to flush > A new Hfile will be created with [ C, A ] and MaxSequenceId = C's seqid. > Replay the rest of the Log. > Flush > The issue will happen in similar situation like Put(key, t=T) in WAL1 and > Put(key,t=T) in WAL2 > h5. Below is the option(proposed by Ted) I'd like to use: > a) During replay, we pass original wal sequence number of each edit to the > receiving RS > b) In receiving RS, we store negative original sequence number of wal edits > into mvcc field of KVs of wal edits > c) Add handling of negative MVCC in KVScannerComparator and KVComparator > d) In receiving RS, write original sequence number into an optional field of > wal file for chained RS failure situation > e) When opening a region, we add a safety bumper(a large number) in order for > the new sequence number of a newly opened region not to collide with old > sequence numbers. > In the future, when we stores sequence number along with KVs, we can adjust > the above solution a little bit by avoiding to overload MVCC field. > h5. The other alternative options are listed below for references: > Option one > a) disallow writes during recovery > b) during replay, we pass original wal sequence ids > c) hold flush till all wals of a recovering region are replayed. Memstore > should hold because we only recover unflushed wal edits. For edits with same > key + version, whichever with larger sequence Id wins. > Option two > a) During replay, we pass original wal sequence ids > b) for each wal edit, we store each edit's original sequence id along with > its key. > c) during scanning, we use the original sequence id if it's present otherwise > its store file sequence Id > d) compaction can just leave put with max sequence id > Please let me know if you have better ideas. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-8701) distributedLogReplay need to apply wal edits in the receiving order of those edits
[ https://issues.apache.org/jira/browse/HBASE-8701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-8701: - Attachment: hbase-8701-tag-v2.patch Incorporate the comments and make distributedLogReplay on by default when HFile v3 is used. > distributedLogReplay need to apply wal edits in the receiving order of those > edits > -- > > Key: HBASE-8701 > URL: https://issues.apache.org/jira/browse/HBASE-8701 > Project: HBase > Issue Type: Bug > Components: MTTR >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0 > > Attachments: 8701-v3.txt, hbase-8701-tag-v1.patch, > hbase-8701-tag-v2.patch, hbase-8701-tag.patch, hbase-8701-v4.patch, > hbase-8701-v5.patch, hbase-8701-v6.patch, hbase-8701-v7.patch, > hbase-8701-v8.patch > > > This issue happens in distributedLogReplay mode when recovering multiple puts > of the same key + version(timestamp). After replay, the value is > nondeterministic of the key > h5. The original concern situation raised from [~eclark]: > For all edits the rowkey is the same. > There's a log with: [ A (ts = 0), B (ts = 0) ] > Replay the first half of the log. > A user puts in C (ts = 0) > Memstore has to flush > A new Hfile will be created with [ C, A ] and MaxSequenceId = C's seqid. > Replay the rest of the Log. > Flush > The issue will happen in similar situation like Put(key, t=T) in WAL1 and > Put(key,t=T) in WAL2 > h5. Below is the option(proposed by Ted) I'd like to use: > a) During replay, we pass original wal sequence number of each edit to the > receiving RS > b) In receiving RS, we store negative original sequence number of wal edits > into mvcc field of KVs of wal edits > c) Add handling of negative MVCC in KVScannerComparator and KVComparator > d) In receiving RS, write original sequence number into an optional field of > wal file for chained RS failure situation > e) When opening a region, we add a safety bumper(a large number) in order for > the new sequence number of a newly opened region not to collide with old > sequence numbers. > In the future, when we stores sequence number along with KVs, we can adjust > the above solution a little bit by avoiding to overload MVCC field. > h5. The other alternative options are listed below for references: > Option one > a) disallow writes during recovery > b) during replay, we pass original wal sequence ids > c) hold flush till all wals of a recovering region are replayed. Memstore > should hold because we only recover unflushed wal edits. For edits with same > key + version, whichever with larger sequence Id wins. > Option two > a) During replay, we pass original wal sequence ids > b) for each wal edit, we store each edit's original sequence id along with > its key. > c) during scanning, we use the original sequence id if it's present otherwise > its store file sequence Id > d) compaction can just leave put with max sequence id > Please let me know if you have better ideas. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10193) Cleanup HRegion if one of the store fails to open at region initialization
[
https://issues.apache.org/jira/browse/HBASE-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851285#comment-13851285
]
Hadoop QA commented on HBASE-10193:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619211/HBASE-10193.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8206//console
This message is automatically generated.
> Cleanup HRegion if one of the store fails to open at region initialization
> --
>
> Key: HBASE-10193
> URL: https://issues.apache.org/jira/browse/HBASE-10193
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.96.1, 0.94.14
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
>Priority: Critical
> Attachments: HBASE-10193.patch, HBASE-10193_0.94.patch
>
>
> While investigating a different issue, I realized that the fix for HBASE-9737
> is not sufficient to prevent resource leak if a region fails to open for some
> reason, say a corrupt HFile.
> The region may have, by then, opened other good HFiles in that store or other
> stores if it has more than one column family and their streams may leak if
> not closed.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10142) TestLogRolling#testLogRollOnDatanodeDeath test failure
[
https://issues.apache.org/jira/browse/HBASE-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851277#comment-13851277
]
Ted Yu commented on HBASE-10142:
Integrated to trunk.
If Jenkins build is good, will integrate to 0.98 as well.
Thanks for the review, Andy.
> TestLogRolling#testLogRollOnDatanodeDeath test failure
> --
>
> Key: HBASE-10142
> URL: https://issues.apache.org/jira/browse/HBASE-10142
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Andrew Purtell
>Assignee: Ted Yu
> Fix For: 0.98.0
>
> Attachments: 10142-v1.txt
>
>
> This is a demanding unit test, which fails fairly often as software versions
> (JVM, Hadoop) and system load change. Currently when testing 0.98 branch I
> see this failure:
> {noformat}
> Failed tests:
> testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling):
> LowReplication Roller should've been disabled, current replication=1
> {noformat}
> Could be a timing issue after the recent switch to Hadoop 2 as default
> build/test profile. Let's see if more leniency makes sense and if it can
> stabilize the test before disabling it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10138) incorrect or confusing test value is used in block caches
[
https://issues.apache.org/jira/browse/HBASE-10138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851273#comment-13851273
]
Sergey Shelukhin commented on HBASE-10138:
--
Will commit tomorrow if no objections
> incorrect or confusing test value is used in block caches
> -
>
> Key: HBASE-10138
> URL: https://issues.apache.org/jira/browse/HBASE-10138
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HBASE-10138.patch
>
>
> DEFAULT_BLOCKSIZE_SMALL is described as:
> {code}
> // Make default block size for StoreFiles 8k while testing. TODO: FIX!
> // Need to make it 8k for testing.
> public static final int DEFAULT_BLOCKSIZE_SMALL = 8 * 1024;
> {code}
> This value is used on production path in CacheConfig thru HStore/HRegion, and
> passed to various cache object.
> We should change it to actual block size, or if it is somehow by design at
> least we should clarify it and remove the comment.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10138) incorrect or confusing test value is used in block caches
[
https://issues.apache.org/jira/browse/HBASE-10138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851268#comment-13851268
]
Nick Dimiduk commented on HBASE-10138:
--
I was not expecting any significant change and you didn't find one, so +1 from
me.
> incorrect or confusing test value is used in block caches
> -
>
> Key: HBASE-10138
> URL: https://issues.apache.org/jira/browse/HBASE-10138
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HBASE-10138.patch
>
>
> DEFAULT_BLOCKSIZE_SMALL is described as:
> {code}
> // Make default block size for StoreFiles 8k while testing. TODO: FIX!
> // Need to make it 8k for testing.
> public static final int DEFAULT_BLOCKSIZE_SMALL = 8 * 1024;
> {code}
> This value is used on production path in CacheConfig thru HStore/HRegion, and
> passed to various cache object.
> We should change it to actual block size, or if it is somehow by design at
> least we should clarify it and remove the comment.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10179) HRegionServer underreports readRequestCounts by 1 under certain conditions
[ https://issues.apache.org/jira/browse/HBASE-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851238#comment-13851238 ] Perry Trolard commented on HBASE-10179: --- Thanks, Andrew! > HRegionServer underreports readRequestCounts by 1 under certain conditions > -- > > Key: HBASE-10179 > URL: https://issues.apache.org/jira/browse/HBASE-10179 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 0.94.6, 0.99.0 >Reporter: Perry Trolard >Assignee: Perry Trolard >Priority: Minor > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: 10179-against-0.94-branch.diff, 10179-against-trunk.diff > > > In HRegionServer.scan(), if > (a) the number of results returned, n, is greater than zero > (b) but less than the size of the batch (nbRows) > (c) and the size in bytes is smaller than the max size (maxScannerResultSize) > then the readRequestCount will be reported as n - 1 rather than n. (This is > because the for-loop counter i is used to update the readRequestCount, and if > the scan runs out of rows before reaching max rows or size, the code `break`s > out of the loop and i is not incremented for the final time.) > To reproduce, create a test table and open its details page in the web UI. > Insert a single row, then note the current request count, c. Scan the table, > returning 1 row; the request count will still be c, whereas it should be c + > 1. > I have a patch against TRUNK I can submit. At Splice Machine we're running > 0.94, & I'd be happy to submit a patch against that as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10084) [WINDOWS] bin\hbase.cmd should allow whitespaces in java.library.path and classpath
[ https://issues.apache.org/jira/browse/HBASE-10084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851234#comment-13851234 ] Hudson commented on HBASE-10084: FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/]) HBASE-10084 [WINDOWS] bin\hbase.cmd should allow whitespaces in java.library.path and classpath (enis: rev 1551433) * /hbase/trunk/bin/hbase-config.cmd * /hbase/trunk/bin/hbase.cmd > [WINDOWS] bin\hbase.cmd should allow whitespaces in java.library.path and > classpath > --- > > Key: HBASE-10084 > URL: https://issues.apache.org/jira/browse/HBASE-10084 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.96.2, 0.99.0 > > Attachments: hbase-10084_v1.patch > > > In case CLASSPATH or java.library.path from hadoop or HBASE_HOME contains > directories with names containing whitespaces, the bin script splits out > errors. We can fix the ws handling hopefully once and for all (or not) -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-9892) Add info port to ServerName to support multi instances in a node
[ https://issues.apache.org/jira/browse/HBASE-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851233#comment-13851233 ] Hudson commented on HBASE-9892: --- FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/]) HBASE-9892 Add info port to ServerName to support multi instances in a node (stack: rev 1551458) * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/HBase.proto * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSDumpServlet.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSStatusServlet.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/table.jsp * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java > Add info port to ServerName to support multi instances in a node > > > Key: HBASE-9892 > URL: https://issues.apache.org/jira/browse/HBASE-9892 > Project: HBase > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-9892-0.94-v1.diff, HBASE-9892-0.94-v2.diff, > HBASE-9892-0.94-v3.diff, HBASE-9892-0.94-v4.diff, HBASE-9892-0.94-v5.diff, > HBASE-9892-trunk-v1.diff, HBASE-9892-trunk-v1.patch, > HBASE-9892-trunk-v1.patch, HBASE-9892-trunk-v2.patch, > HBASE-9892-trunk-v3.diff, HBASE-9892-v5.txt > > > The full GC time of regionserver with big heap(> 30G ) usually can not be > controlled in 30s. At the same time, the servers with 64G memory are normal. > So we try to deploy multi rs instances(2-3 ) in a single node and the heap of > each rs is about 20G ~ 24G. > Most of the things works fine, except the hbase web ui. The master get the RS > info port from conf, which is suitable for this situation of multi rs > instances in a node. So we add info port to ServerName. > a. at the startup, rs report it's info port to Hmaster. > b, For root region, rs write the servername with info port ro the zookeeper > root-region-server node. > c, For meta regions, rs write the servername with info port to root region > d. For user regions, rs write the servername with info port to meta regions > So hmaster and client can get info port from the servername. > To test this feature, I change the rs num from 1 to 3 in standalone mode, so > we can test it in standalone mode, > I think Hoya(hbase on yarn) will encounter the same problem. Anyone knows > how Hoya handle this problem? > PS: There are different formats for servername in zk node and meta table, i > think we need to unify it and refactor the code. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10142) TestLogRolling#testLogRollOnDatanodeDeath test failure
[
https://issues.apache.org/jira/browse/HBASE-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851235#comment-13851235
]
Andrew Purtell commented on HBASE-10142:
Looks good here +1
> TestLogRolling#testLogRollOnDatanodeDeath test failure
> --
>
> Key: HBASE-10142
> URL: https://issues.apache.org/jira/browse/HBASE-10142
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Andrew Purtell
>Assignee: Ted Yu
> Fix For: 0.98.0
>
> Attachments: 10142-v1.txt
>
>
> This is a demanding unit test, which fails fairly often as software versions
> (JVM, Hadoop) and system load change. Currently when testing 0.98 branch I
> see this failure:
> {noformat}
> Failed tests:
> testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling):
> LowReplication Roller should've been disabled, current replication=1
> {noformat}
> Could be a timing issue after the recent switch to Hadoop 2 as default
> build/test profile. Let's see if more leniency makes sense and if it can
> stabilize the test before disabling it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HBASE-10137) GeneralBulkAssigner with retain assignment plan can be used in EnableTableHandler to bulk assign the regions
[
https://issues.apache.org/jira/browse/HBASE-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-10137:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Same patch can be applied to all the fix versions.
Committed to trunk, 0.98 and 0.96 as well.
Thanks all for the reviews.
> GeneralBulkAssigner with retain assignment plan can be used in
> EnableTableHandler to bulk assign the regions
>
>
> Key: HBASE-10137
> URL: https://issues.apache.org/jira/browse/HBASE-10137
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.96.0, 0.94.14
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-10137.patch, HBASE-10137_v2.patch,
> HBASE-10137_v3.patch
>
>
> Current in BulkEnabler we are assigning one region at a time, instead we can
> use GeneralBulkAssigner to bulk assign multiple regions at a time.
> {code}
> for (HRegionInfo region : regions) {
> if (assignmentManager.getRegionStates()
> .isRegionInTransition(region)) {
> continue;
> }
> final HRegionInfo hri = region;
> pool.execute(Trace.wrap("BulkEnabler.populatePool",new Runnable() {
> public void run() {
> assignmentManager.assign(hri, true);
> }
> }));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10180) TestByteBufferIOEngine#testByteBufferIOEngine occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851231#comment-13851231
]
Hudson commented on HBASE-10180:
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/])
HBASE-10180 TestByteBufferIOEngine#testByteBufferIOEngine occasionally fails
(zjushch: rev 1551497)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestByteBufferIOEngine.java
> TestByteBufferIOEngine#testByteBufferIOEngine occasionally fails
>
>
> Key: HBASE-10180
> URL: https://issues.apache.org/jira/browse/HBASE-10180
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: chunhui shen
> Fix For: 0.98.0, 0.99.0
>
> Attachments: hbase-trunk-10180.patch, hbase-trunk-10180v2.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/8189/testReport/junit/org.apache.hadoop.hbase.io.hfile.bucket/TestByteBufferIOEngine/testByteBufferIOEngine/
> :
> {code}
> java.lang.AssertionError
> at
> org.apache.hadoop.hbase.util.ByteBufferArray.multiple(ByteBufferArray.java:160)
> at
> org.apache.hadoop.hbase.util.ByteBufferArray.putMultiple(ByteBufferArray.java:123)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.ByteBufferIOEngine.write(ByteBufferIOEngine.java:81)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.TestByteBufferIOEngine.testByteBufferIOEngine(TestByteBufferIOEngine.java:61)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Standard Output
> 2013-12-16 21:12:00,417 INFO [pool-1-thread-1] hbase.ResourceChecker(147):
> before: io.hfile.bucket.TestByteBufferIOEngine#testByteBufferIOEngine
> Thread=36, OpenFileDescriptor=146, MaxFileDescriptor=6,
> SystemLoadAverage=142, ProcessCount=167, AvailableMemoryMB=7901,
> ConnectionCount=1
> 2013-12-16 21:12:00,417 INFO [pool-1-thread-1] util.ByteBufferArray(57):
> Allocating buffers total=32 MB , sizePerBuffer=2 MB, count=16
> 2013-12-16 21:12:00,460 INFO [pool-1-thread-1] hbase.ResourceChecker(171):
> after: io.hfile.bucket.TestByteBufferIOEngine#testByteBufferIOEngine
> Thread=36 (was 36), OpenFileDescriptor=148 (was 146) - OpenFileDescriptor
> LEAK? -, MaxFileDescriptor=6 (was 6), SystemLoadAverage=142 (was
> 142), ProcessCount=167 (was 167), AvailableMemoryMB=7901 (was 7901),
> ConnectionCount=1 (was 1)
> Standard Error
> 2013-12-16 21:12:00,417 INFO [pool-1-thread-1] hbase.ResourceChecker(147):
> before: io.hfile.bucket.TestByteBufferIOEngine#testByteBufferIOEngine
> Thread=36, OpenFileDescriptor=146, MaxFileDescriptor=6,
> SystemLoadAverage=142, ProcessCount=167, AvailableMemoryMB=7901,
> ConnectionCount=1
> 2013-12-16 21:12:00,417
[jira] [Commented] (HBASE-10178) Potential null object dereference in TablePermission#equals()
[
https://issues.apache.org/jira/browse/HBASE-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851229#comment-13851229
]
Hudson commented on HBASE-10178:
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/])
HBASE-10178 Potential null object dereference in TablePermission#equals()
(tedyu: rev 1551409)
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java
> Potential null object dereference in TablePermission#equals()
> -
>
> Key: HBASE-10178
> URL: https://issues.apache.org/jira/browse/HBASE-10178
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.99.0
>
> Attachments: 10178-v1.txt
>
>
> At line 326:
> {code}
> ((namespace == null && other.getNamespace() == null) ||
> namespace.equals(other.getNamespace()))
> {code}
> If namespace is null but other.getNamespace() is not null, we would deference
> null object.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-9928) TestHRegion should clean up test-data directory upon completion
[
https://issues.apache.org/jira/browse/HBASE-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851232#comment-13851232
]
Hudson commented on HBASE-9928:
---
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/])
HBASE-9928 TestHRegion should clean up test-data directory upon completion
(tedyu: rev 1551406)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
> TestHRegion should clean up test-data directory upon completion
> ---
>
> Key: HBASE-9928
> URL: https://issues.apache.org/jira/browse/HBASE-9928
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Gustavo Anatoly
> Fix For: 0.98.0, 0.99.0
>
> Attachments: HBASE-9928.patch
>
>
> TestHRegion leaves some files behind in hbase-server/target/test-data
> directory after tests complete.
> e.g. at the end of testRegionInfoFileCreation:
> {code}
> // Verify that the .regioninfo file is still there
> assertTrue(HRegionFileSystem.REGION_INFO_FILE + " should be present in
> the region dir",
> fs.exists(new Path(regionDir, HRegionFileSystem.REGION_INFO_FILE)));
> {code}
> test-data directory should be cleaned upon completion.
> I noticed this when looping TestHRegion in order to reproduce test failure.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10157) Provide CP hook post log replay
[ https://issues.apache.org/jira/browse/HBASE-10157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851227#comment-13851227 ] Hudson commented on HBASE-10157: FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/]) HBASE-10157 Provide CP hook post log replay (anoopsamjohn: rev 1551417) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java > Provide CP hook post log replay > --- > > Key: HBASE-10157 > URL: https://issues.apache.org/jira/browse/HBASE-10157 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.96.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.98.0, 0.96.2, 0.99.0 > > Attachments: HBASE-10157.patch, HBASE-10157_V2.patch > > -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10159) Replaced deprecated interface Closeable
[ https://issues.apache.org/jira/browse/HBASE-10159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851225#comment-13851225 ] Hudson commented on HBASE-10159: FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/]) HBASE-10159 Replaced deprecated interface Closeable (jxiang: rev 1551631) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java > Replaced deprecated interface Closeable > --- > > Key: HBASE-10159 > URL: https://issues.apache.org/jira/browse/HBASE-10159 > Project: HBase > Issue Type: Task >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Trivial > Fix For: 0.98.0, 0.99.0 > > Attachments: trunk-10159.patch, trunk-10159_v2.patch > > > Hadoop interface Closeable has been deprecated for quite some time to favor > java.io.Closeable. Since we support only hadoop 1.0 and above now, we can > replace the deprecated Closeable with the java one. We can also clean up some > hack we did as well, for example, HBASE-10029. > https://github.com/apache/hadoop-common/blob/branch-1.0/src/core/org/apache/hadoop/io/Closeable.java > -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-9047) Tool to handle finishing replication when the cluster is offline
[ https://issues.apache.org/jira/browse/HBASE-9047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851226#comment-13851226 ] Hudson commented on HBASE-9047: --- FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/]) HBASE-9047 - addendum, include new files (larsh: rev 1551464) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSyncUp.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSyncUpTool.java > Tool to handle finishing replication when the cluster is offline > > > Key: HBASE-9047 > URL: https://issues.apache.org/jira/browse/HBASE-9047 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.96.0 >Reporter: Jean-Daniel Cryans >Assignee: Demai Ni > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: HBASE-9047-0.94-v1.patch, HBASE-9047-0.94.9-v0.PATCH, > HBASE-9047-trunk-v0.patch, HBASE-9047-trunk-v1.patch, > HBASE-9047-trunk-v2.patch, HBASE-9047-trunk-v3.patch, > HBASE-9047-trunk-v4.patch, HBASE-9047-trunk-v4.patch, > HBASE-9047-trunk-v5.patch, HBASE-9047-trunk-v6.patch, > HBASE-9047-trunk-v7.patch, HBASE-9047-trunk-v7.patch > > > We're having a discussion on the mailing list about replicating the data on a > cluster that was shut down in an offline fashion. The motivation could be > that you don't want to bring HBase back up but still need that data on the > slave. > So I have this idea of a tool that would be running on the master cluster > while it is down, although it could also run at any time. Basically it would > be able to read the replication state of each master region server, finish > replicating what's missing to all the slave, and then clear that state in > zookeeper. > The code that handles replication does most of that already, see > ReplicationSourceManager and ReplicationSource. Basically when > ReplicationSourceManager.init() is called, it will check all the queues in ZK > and try to grab those that aren't attached to a region server. If the whole > cluster is down, it will grab all of them. > The beautiful thing here is that you could start that tool on all your > machines and the load will be spread out, but that might not be a big concern > if replication wasn't lagging since it would take a few seconds to finish > replicating the missing data for each region server. > I'm guessing when starting ReplicationSourceManager you'd give it a fake > region server ID, and you'd tell it not to start its own source. > FWIW the main difference in how replication is handled between Apache's HBase > and Facebook's is that the latter is always done separately of HBase itself. > This jira isn't about doing that. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10186) region_mover.rb broken because ServerName constructor is changed to private
[
https://issues.apache.org/jira/browse/HBASE-10186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851228#comment-13851228
]
Hudson commented on HBASE-10186:
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/])
HBASE-10186 region_mover.rb broken because ServerName constructor is changed to
private (Samir Ahmic) (jmhsieh: rev 1551532)
* /hbase/trunk/bin/region_mover.rb
> region_mover.rb broken because ServerName constructor is changed to private
> ---
>
> Key: HBASE-10186
> URL: https://issues.apache.org/jira/browse/HBASE-10186
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.96.1
> Environment: x86_64 GNU/Linux
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-10186.patch
>
>
> After upgrading to 0.96.1 (from 0.96.0) i have tried rolling restart with
> ./rolling-restart.sh --graceful but region_mover.rb throws error:
> {code}
> TypeError: no public constructors for Java::OrgApacheHadoopHbase::ServerName
> getRegions at ./region_mover.rb:257
> unloadRegions at ./region_mover.rb:318
> (root) at ./region_mover.rb:461
> {code}
> After checking region_mover.rb i believe this line:
> {code}
> return
> ProtobufUtil::getOnlineRegions(connection.getAdmin(ServerName.new(servername)));
> {code}
> should be changed to
> {code}
> return
> ProtobufUtil::getOnlineRegions(connection.getAdmin(ServerName.valueOf(servername)));
> {code}
> After making this change region_mover.rb is working again. I will attach
> patch shortly.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10098) [WINDOWS] pass in native library directory from hadoop for unit tests
[ https://issues.apache.org/jira/browse/HBASE-10098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851230#comment-13851230 ] Hudson commented on HBASE-10098: FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #9 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/9/]) HBASE-10098 [WINDOWS] pass in native library directory from hadoop for unit tests (enis: rev 1551425) * /hbase/trunk/pom.xml > [WINDOWS] pass in native library directory from hadoop for unit tests > - > > Key: HBASE-10098 > URL: https://issues.apache.org/jira/browse/HBASE-10098 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.96.2, 0.99.0 > > Attachments: hbase-10098_v1.patch, hbase-10098_v2.patch, > hbase-10098_v3.patch > > > On windows, Hadoop depends on native libraries for doing it's job. The bin > scripts already handle finding hadoop's native libs and adding them to > java.library.path, but for running HBase's unit tests, we need to pass them > in. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10179) HRegionServer underreports readRequestCounts by 1 under certain conditions
[ https://issues.apache.org/jira/browse/HBASE-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-10179: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks for the patch Perry! > HRegionServer underreports readRequestCounts by 1 under certain conditions > -- > > Key: HBASE-10179 > URL: https://issues.apache.org/jira/browse/HBASE-10179 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 0.94.6, 0.99.0 >Reporter: Perry Trolard >Assignee: Perry Trolard >Priority: Minor > Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0 > > Attachments: 10179-against-0.94-branch.diff, 10179-against-trunk.diff > > > In HRegionServer.scan(), if > (a) the number of results returned, n, is greater than zero > (b) but less than the size of the batch (nbRows) > (c) and the size in bytes is smaller than the max size (maxScannerResultSize) > then the readRequestCount will be reported as n - 1 rather than n. (This is > because the for-loop counter i is used to update the readRequestCount, and if > the scan runs out of rows before reaching max rows or size, the code `break`s > out of the loop and i is not incremented for the final time.) > To reproduce, create a test table and open its details page in the web UI. > Insert a single row, then note the current request count, c. Scan the table, > returning 1 row; the request count will still be c, whereas it should be c + > 1. > I have a patch against TRUNK I can submit. At Splice Machine we're running > 0.94, & I'd be happy to submit a patch against that as well. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HBASE-10142) TestLogRolling#testLogRollOnDatanodeDeath test failure
[
https://issues.apache.org/jira/browse/HBASE-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851221#comment-13851221
]
Hadoop QA commented on HBASE-10142:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619199/10142-v1.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8205//console
This message is automatically generated.
> TestLogRolling#testLogRollOnDatanodeDeath test failure
> --
>
> Key: HBASE-10142
> URL: https://issues.apache.org/jira/browse/HBASE-10142
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Andrew Purtell
>Assignee: Ted Yu
> Fix For: 0.98.0
>
> Attachments: 10142-v1.txt
>
>
> This is a demanding unit test, which fails fairly often as software versions
> (JVM, Hadoop) and system load change. Currently when testing 0.98 branch I
> see this failure:
> {noformat}
> Failed tests:
> testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling):
> LowReplication Roller should've been disabled, current replication=1
> {noformat}
> Could be a timing issue after the recent switch to Hadoop 2 as default
> build/test profile. Let's see if more leniency makes sense and if it can
> stabilize the test before disabling it.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10192) Empty rowkey is not allowed any more
[ https://issues.apache.org/jira/browse/HBASE-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851202#comment-13851202 ] Enis Soztutar commented on HBASE-10192: --- One problem with empty row keys was that, you cannot get with en empty row key, since get will be converted into a scan, and that scan would have startKey = empty, stopKey = empty, which is a whole table scan! I remember [~devaraj] looking into that some time ago, but I was not able to find the issue. > Empty rowkey is not allowed any more > > > Key: HBASE-10192 > URL: https://issues.apache.org/jira/browse/HBASE-10192 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang > > In HBase-8101, we added a check and disallowed empty rowkey. However, it is > allowed in 0.94. I was wondering if there is any special reason for that, > i.e., is it by intention or by mistake? -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10193) Cleanup HRegion if one of the store fails to open at region initialization
[ https://issues.apache.org/jira/browse/HBASE-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Kishore updated HBASE-10193: --- Attachment: HBASE-10193.patch Patch for trunk. > Cleanup HRegion if one of the store fails to open at region initialization > -- > > Key: HBASE-10193 > URL: https://issues.apache.org/jira/browse/HBASE-10193 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.96.1, 0.94.14 >Reporter: Aditya Kishore >Assignee: Aditya Kishore >Priority: Critical > Attachments: HBASE-10193.patch, HBASE-10193_0.94.patch > > > While investigating a different issue, I realized that the fix for HBASE-9737 > is not sufficient to prevent resource leak if a region fails to open for some > reason, say a corrupt HFile. > The region may have, by then, opened other good HFiles in that store or other > stores if it has more than one column family and their streams may leak if > not closed. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10193) Cleanup HRegion if one of the store fails to open at region initialization
[ https://issues.apache.org/jira/browse/HBASE-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Kishore updated HBASE-10193: --- Release Note: Submitting the patch to Hadoop-QA Status: Patch Available (was: Open) > Cleanup HRegion if one of the store fails to open at region initialization > -- > > Key: HBASE-10193 > URL: https://issues.apache.org/jira/browse/HBASE-10193 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.14, 0.96.1 >Reporter: Aditya Kishore >Assignee: Aditya Kishore >Priority: Critical > Attachments: HBASE-10193.patch, HBASE-10193_0.94.patch > > > While investigating a different issue, I realized that the fix for HBASE-9737 > is not sufficient to prevent resource leak if a region fails to open for some > reason, say a corrupt HFile. > The region may have, by then, opened other good HFiles in that store or other > stores if it has more than one column family and their streams may leak if > not closed. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Resolved] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl resolved HBASE-10189.
---
Resolution: Fixed
Fix Version/s: 0.99.0
0.96.2
0.98.0
Hadoop Flags: Reviewed
Committed to all branches.
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.98.0, 0.94.15, 0.96.2, 0.99.0
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851191#comment-13851191
]
Demai Ni commented on HBASE-10189:
--
Thanks. Appreciate you kindly help and suggestion alone the way... Demai
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.94.15
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10183) Need enforce a reserved range of system tag types
[ https://issues.apache.org/jira/browse/HBASE-10183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851194#comment-13851194 ] Jeffrey Zhong commented on HBASE-10183: --- For user tags, we have no control. For system tags, it's theoretically possible that we can use tag types in non-sequential mode. Since we declare all system tag types in one place, I don't think people will normally do that. > Need enforce a reserved range of system tag types > - > > Key: HBASE-10183 > URL: https://issues.apache.org/jira/browse/HBASE-10183 > Project: HBase > Issue Type: Task > Components: HFile >Affects Versions: 0.98.0 >Reporter: Jeffrey Zhong >Priority: Critical > Fix For: 0.98.0 > > > If we don't reserve a system tag types now, let's say 0-64(total tag type > range is 0-255). We'll have a hard time when introducing a new system tag > type in the future because the new tag type may collide with an existing user > tag type as tag is open to users as well. > [~ram_krish], [~anoop.hbase] How do you guys think? > Thanks! -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Work started] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-10189 started by Demai Ni.
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.94.15
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851187#comment-13851187
]
Lars Hofhansl commented on HBASE-10189:
---
0.94 patch applies to trunk, no need for another patch. I'll commit momentarily.
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.94.15
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851190#comment-13851190
]
Lars Hofhansl commented on HBASE-10189:
---
He he, comment crossing :)
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.94.15
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Updated] (HBASE-10193) Cleanup HRegion if one of the store fails to open at region initialization
[ https://issues.apache.org/jira/browse/HBASE-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Kishore updated HBASE-10193: --- Attachment: HBASE-10193_0.94.patch Patch for 0.94 branch. > Cleanup HRegion if one of the store fails to open at region initialization > -- > > Key: HBASE-10193 > URL: https://issues.apache.org/jira/browse/HBASE-10193 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.96.1, 0.94.14 >Reporter: Aditya Kishore >Assignee: Aditya Kishore >Priority: Critical > Attachments: HBASE-10193_0.94.patch > > > While investigating a different issue, I realized that the fix for HBASE-9737 > is not sufficient to prevent resource leak if a region fails to open for some > reason, say a corrupt HFile. > The region may have, by then, opened other good HFiles in that store or other > stores if it has more than one column family and their streams may leak if > not closed. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Demai Ni updated HBASE-10189:
-
Attachment: HBASE-10189-trunk-v0.patch
the trunk version, which is the same for 0.96. Thanks... Demai
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.94.15
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch, HBASE-10189-trunk-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Created] (HBASE-10194) [Usability]: Instructions in CompactionTool no longer accurate because of namespaces
Aleksandr Shulman created HBASE-10194:
-
Summary: [Usability]: Instructions in CompactionTool no longer
accurate because of namespaces
Key: HBASE-10194
URL: https://issues.apache.org/jira/browse/HBASE-10194
Project: HBase
Issue Type: Bug
Components: Compaction, util
Affects Versions: 0.96.2, 0.98.1, 0.99.0
Reporter: Aleksandr Shulman
Assignee: Aleksandr Shulman
Priority: Minor
Observed Behavior:
The instructions for org.apache.hadoop.hbase.regionserver.CompactionTool
suggest using the pre-95 hbase format:
{code}Examples:
To compact the full 'TestTable' using MapReduce:
$ bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool -mapred
hdfs:///hbase/TestTable{code}
Expected behavior:
It should now take into account namespaces, for example:
{code}
hdfs:///hbase/data/default/TestTable
{code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Created] (HBASE-10193) Cleanup HRegion if one of the store fails to open at region initialization
Aditya Kishore created HBASE-10193: -- Summary: Cleanup HRegion if one of the store fails to open at region initialization Key: HBASE-10193 URL: https://issues.apache.org/jira/browse/HBASE-10193 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.14, 0.96.1 Reporter: Aditya Kishore Assignee: Aditya Kishore Priority: Critical While investigating a different issue, I realized that the fix for HBASE-9737 is not sufficient to prevent resource leak if a region fails to open for some reason, say a corrupt HFile. The region may have, by then, opened other good HFiles in that store or other stores if it has more than one column family and their streams may leak if not closed. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-10189) Intermittent TestReplicationSyncUpTool failure
[
https://issues.apache.org/jira/browse/HBASE-10189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Demai Ni updated HBASE-10189:
-
Attachment: HBASE-10189-0.94-v0.patch
[~lhofhansl], I was thinking about adding such logic after running the sync up
tool. But you are right, the sync up tool already did the checking inside. so
no waiting necessary.
Here is the 94 patch, which is the same as yours except removing the extra
file. I will upload the trunk patch in a min
> Intermittent TestReplicationSyncUpTool failure
> --
>
> Key: HBASE-10189
> URL: https://issues.apache.org/jira/browse/HBASE-10189
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Demai Ni
> Fix For: 0.94.15
>
> Attachments: 10189-0.94.txt, 10189-test-sync-tool.html,
> HBASE-10189-0.94-v0.patch
>
>
> From https://builds.apache.org/job/PreCommit-HBASE-Build/8200//testReport/ :
> {code}
> java.lang.AssertionError: t2_syncup has 201 rows on source, and 200 on slave1
> expected:<200> but was:<125>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.putAndReplicateRows(TestReplicationSyncUpTool.java:245)
> at
> org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool.testSyncUpTool(TestReplicationSyncUpTool.java:116)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
