[jira] [Commented] (HBASE-10855) Enable hfilev3 by default
[
https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950439#comment-13950439
]
Hadoop QA commented on HBASE-10855:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637320/10855.txt
against trunk revision .
ATTACHMENT ID: 12637320
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestHRegion
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9122//console
This message is automatically generated.
> Enable hfilev3 by default
> -
>
> Key: HBASE-10855
> URL: https://issues.apache.org/jira/browse/HBASE-10855
> Project: HBase
> Issue Type: Sub-task
> Components: HFile
>Reporter: stack
>Assignee: stack
> Fix For: 0.99.0
>
> Attachments: 10855.txt, 10855.txt
>
>
> Distributed log replay needs this. Should be on by default in 1.0/0.99.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10092) Move up on to log4j2
[
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950436#comment-13950436
]
Hadoop QA commented on HBASE-10092:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637326/10092v2.txt
against trunk revision .
ATTACHMENT ID: 12637326
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed
org.apache.hadoop.hbase.replication.TestReplicationKillMasterRS
org.apache.hadoop.hbase.fs.TestBlockReorder
org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
org.apache.hadoop.hbase.TestIOFencing
org.apache.hadoop.hbase.replication.TestReplicationSmallTests
org.apache.hadoop.hbase.regionserver.wal.TestSecureHLog
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.ipc.TestDelayedRpc
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplitCompressed
org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers
org.apache.hadoop.hbase.regionserver.wal.TestHLog
org.apache.hadoop.hbase.filter.TestFilterWithScanLimits
org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9123//console
This message is automatically generated.
> Move up on to log4j2
>
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Attachments: 10092.txt, 10092v2.txt
>
>
> Allows logging with less friction. See http://logging.apache.org/log4j/2.x/
> This rather radical transition can be
[jira] [Updated] (HBASE-10850) Unexpected behavior when using filter SingleColumnValueFilter
[
https://issues.apache.org/jira/browse/HBASE-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10850:
---
Priority: Critical (was: Major)
> Unexpected behavior when using filter SingleColumnValueFilter
> -
>
> Key: HBASE-10850
> URL: https://issues.apache.org/jira/browse/HBASE-10850
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.96.1.1
>Reporter: Fabien Le Gallo
>Assignee: haosdent
>Priority: Critical
> Attachments: HBASE-10850-96.patch, HBASE-10850.patch,
> HBaseSingleColumnValueFilterTest.java
>
>
> When using the filter SingleColumnValueFilter, and depending of the columns
> specified in the scan (filtering column always specified), the results can be
> different.
> Here is an example.
> Suppose the following table:
> ||key||a:foo||a:bar||b:foo||b:bar||
> |1|false|_flag_|_flag_|_flag_|
> |2|true|_flag_|_flag_|_flag_|
> |3| |_flag_|_flag_|_flag_|
> With this filter:
> {code}
> SingleColumnValueFilter filter = new
> SingleColumnValueFilter(Bytes.toBytes("a"), Bytes.toBytes("foo"),
> CompareOp.EQUAL, new BinaryComparator(Bytes.toBytes("false")));
> filter.setFilterIfMissing(true);
> {code}
> Depending of how I specify the list of columns to add in the scan, the result
> is different. Yet, all examples below should always return only the first row
> (key '1'):
> OK:
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> scan.addFamily(Bytes.toBytes("b"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("foo"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("bar"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> {code}
> This is a regression as it was working properly on HBase 0.92.
> You will find in attachement the unit tests reproducing the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10847) 0.94: drop non-secure builds, make security the default
[
https://issues.apache.org/jira/browse/HBASE-10847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950425#comment-13950425
]
Lars Hofhansl commented on HBASE-10847:
---
Now the question is. Do I just change pom.xml to always include the security
files like this:
{code}
+
+add-source
+
+ add-source
+
+
+
+${project.basedir}/security/src/main/java
+
+
+
{code}
or do I move the security files into place under src? The former is less
disruptive, the latter is cleaner.
> 0.94: drop non-secure builds, make security the default
> ---
>
> Key: HBASE-10847
> URL: https://issues.apache.org/jira/browse/HBASE-10847
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.94.19
>
>
> I would like to only create a single 0.94 tarball/release that contains the
> security code - and drop the non-secure tarballs and releases.
> Let's discuss...
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9679) Binary search in HFile block
[ https://issues.apache.org/jira/browse/HBASE-9679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950418#comment-13950418 ] Lars Hofhansl commented on HBASE-9679: -- If PrefixTree solves this for all use cases, I'd say we shouldn't worry about this in 0.94. > Binary search in HFile block > > > Key: HBASE-9679 > URL: https://issues.apache.org/jira/browse/HBASE-9679 > Project: HBase > Issue Type: Improvement > Components: HFile >Affects Versions: 0.95.2, 0.94.12 >Reporter: Liang Xie >Assignee: Liang Xie >Priority: Minor > > It's not a top priority issue, seems to me. > Right now hbase do a linear scan to search a key within a hfile block on > interst, in special case, e.g. 100% read scenario or high read/write ratio > scanario, it's useful to do a binary search improvement to reduce the CPU > cost and response time, i think the biggest benefit should be the cpu:) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10850) Unexpected behavior when using filter SingleColumnValueFilter
[
https://issues.apache.org/jira/browse/HBASE-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950416#comment-13950416
]
Lars Hofhansl commented on HBASE-10850:
---
Interesting. I'd have to think about this a bit.
> Unexpected behavior when using filter SingleColumnValueFilter
> -
>
> Key: HBASE-10850
> URL: https://issues.apache.org/jira/browse/HBASE-10850
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.96.1.1
>Reporter: Fabien Le Gallo
>Assignee: haosdent
> Attachments: HBASE-10850-96.patch, HBASE-10850.patch,
> HBaseSingleColumnValueFilterTest.java
>
>
> When using the filter SingleColumnValueFilter, and depending of the columns
> specified in the scan (filtering column always specified), the results can be
> different.
> Here is an example.
> Suppose the following table:
> ||key||a:foo||a:bar||b:foo||b:bar||
> |1|false|_flag_|_flag_|_flag_|
> |2|true|_flag_|_flag_|_flag_|
> |3| |_flag_|_flag_|_flag_|
> With this filter:
> {code}
> SingleColumnValueFilter filter = new
> SingleColumnValueFilter(Bytes.toBytes("a"), Bytes.toBytes("foo"),
> CompareOp.EQUAL, new BinaryComparator(Bytes.toBytes("false")));
> filter.setFilterIfMissing(true);
> {code}
> Depending of how I specify the list of columns to add in the scan, the result
> is different. Yet, all examples below should always return only the first row
> (key '1'):
> OK:
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> scan.addFamily(Bytes.toBytes("b"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("foo"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("bar"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> {code}
> This is a regression as it was working properly on HBase 0.92.
> You will find in attachement the unit tests reproducing the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-6651) Improve thread safety of HTablePool
[ https://issues.apache.org/jira/browse/HBASE-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hiroshi Ikeda updated HBASE-6651: - Attachment: HBASE-6651-V15-trunk.patch HBASE-6651-V15-0.98.patch HBASE-6651-V15-0.96.patch Added fixed patches. I had forgot to add the license header to one of the added classes, so I fixed it. Also I changed InterfaceAudience of the added classes to be Private. > Improve thread safety of HTablePool > --- > > Key: HBASE-6651 > URL: https://issues.apache.org/jira/browse/HBASE-6651 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.1 >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda > Attachments: HBASE-6651-V10.patch, HBASE-6651-V11.patch, > HBASE-6651-V12.patch, HBASE-6651-V13.patch, HBASE-6651-V14-0.96.patch, > HBASE-6651-V14-0.98.patch, HBASE-6651-V14-trunk.patch, > HBASE-6651-V15-0.96.patch, HBASE-6651-V15-0.98.patch, > HBASE-6651-V15-trunk.patch, HBASE-6651-V2.patch, HBASE-6651-V3.patch, > HBASE-6651-V4.patch, HBASE-6651-V5.patch, HBASE-6651-V6.patch, > HBASE-6651-V7.patch, HBASE-6651-V8.patch, HBASE-6651-V9.patch, > HBASE-6651.patch, sample.zip, sample.zip, sharedmap_for_hbaseclient.zip > > > There are some operations in HTablePool accessing PoolMap in multiple places > without any explicit synchronization. > For example HTablePool.closeTablePool() calls PoolMap.values(), and calls > PoolMap.remove(). If other threads add new instances to the pool in the > middle of the calls, the newly added instances might be dropped. > (HTablePool.closeTablePool() also has another problem that calling it by > multiple threads causes accessing HTable by multiple threads.) > Moreover, PoolMap is not thread safe for the same reason. > For example PoolMap.put() calles ConcurrentMap.get() and calles > ConcurrentMap.put(). If other threads add a new instance to the concurent map > in the middle of the calls, the new instance might be dropped. > And also implementations of Pool have the same problems. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10856) Prep for 1.0
[ https://issues.apache.org/jira/browse/HBASE-10856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950387#comment-13950387 ] stack commented on HBASE-10856: --- DOCUMENTATION: Of note, you cannot (safely) rolling restart from 0.96 on to 0.99. See HBASE-9801 where if the rolling upgrade fails, you could have old code trying to open hfile v3s written w/ new code. > Prep for 1.0 > > > Key: HBASE-10856 > URL: https://issues.apache.org/jira/browse/HBASE-10856 > Project: HBase > Issue Type: Umbrella >Reporter: stack > > Tasks for 1.0 copied here from our '1.0.0' mailing list discussion. Idea is > to file subtasks off this one. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10855) Enable hfilev3 by default
[ https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10855: -- Component/s: HFile Release Note: You cannot rolling upgrade from 0.96 to 0.99. You must go first to 0.98 which has support for hfile v3 files and then to 0.99. Hadoop Flags: Incompatible change > Enable hfilev3 by default > - > > Key: HBASE-10855 > URL: https://issues.apache.org/jira/browse/HBASE-10855 > Project: HBase > Issue Type: Sub-task > Components: HFile >Reporter: stack >Assignee: stack > Fix For: 0.99.0 > > Attachments: 10855.txt, 10855.txt > > > Distributed log replay needs this. Should be on by default in 1.0/0.99. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9801) Change the default HFile version to V3
[ https://issues.apache.org/jira/browse/HBASE-9801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950383#comment-13950383 ] stack commented on HBASE-9801: -- Sorry [~anoop.hbase] Forgot about this one. This means that rolling restart you need to go from 0.98 to 0.99... you can't go from 0.96 to 1.0. > Change the default HFile version to V3 > -- > > Key: HBASE-9801 > URL: https://issues.apache.org/jira/browse/HBASE-9801 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Priority: Critical > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-9801) Change the default HFile version to V3
[ https://issues.apache.org/jira/browse/HBASE-9801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John resolved HBASE-9801. --- Resolution: Duplicate Will do as part of HBASE-10855 > Change the default HFile version to V3 > -- > > Key: HBASE-9801 > URL: https://issues.apache.org/jira/browse/HBASE-9801 > Project: HBase > Issue Type: Improvement >Reporter: Anoop Sam John >Priority: Critical > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10855) Enable hfilev3 by default
[ https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950381#comment-13950381 ] Anoop Sam John commented on HBASE-10855: We had HBASE-9801. :) I will close that as dup > Enable hfilev3 by default > - > > Key: HBASE-10855 > URL: https://issues.apache.org/jira/browse/HBASE-10855 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.0 > > Attachments: 10855.txt, 10855.txt > > > Distributed log replay needs this. Should be on by default in 1.0/0.99. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950374#comment-13950374
]
Hudson commented on HBASE-10853:
SUCCESS: Integrated in HBase-TRUNK #5046 (See
[https://builds.apache.org/job/HBase-TRUNK/5046/])
HBASE-10853 NPE in RSRpcServices.get on trunk (stack: rev 1582553)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
>Assignee: stack
> Fix For: 0.99.0
>
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10796) Set default log level as INFO
[ https://issues.apache.org/jira/browse/HBASE-10796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950375#comment-13950375 ] Hudson commented on HBASE-10796: SUCCESS: Integrated in HBase-TRUNK #5046 (See [https://builds.apache.org/job/HBase-TRUNK/5046/]) HBASE-10796 Set default log level as INFO (stack: rev 1582532) * /hbase/trunk/conf/log4j.properties * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/ChecksumType.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/DeadServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/RegionServerProcedureManagerHost.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java > Set default log level as INFO > - > > Key: HBASE-10796 > URL: https://issues.apache.org/jira/browse/HBASE-10796 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.0 > > Attachments: 10796.txt, 10796v2.txt > > > When we roll out 1.0, the log level should be INFO-level by default, not > DEBUG. > Proposed on mailing list here > http://search-hadoop.com/m/33P7E1GL08b/hbase+1.0&subj=DISCUSSION+1+0+0 and at > least one other +1 with no objection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-6651) Improve thread safety of HTablePool
[
https://issues.apache.org/jira/browse/HBASE-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950370#comment-13950370
]
Hadoop QA commented on HBASE-6651:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637319/HBASE-6651-V14-trunk.patch
against trunk revision .
ATTACHMENT ID: 12637319
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 19 new
or modified tests.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
16 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the trunk's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9121//console
This message is automatically generated.
> Improve thread safety of HTablePool
> ---
>
> Key: HBASE-6651
> URL: https://issues.apache.org/jira/browse/HBASE-6651
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.94.1
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
> Attachments: HBASE-6651-V10.patch, HBASE-6651-V11.patch,
> HBASE-6651-V12.patch, HBASE-6651-V13.patch, HBASE-6651-V14-0.96.patch,
> HBASE-6651-V14-0.98.patch, HBASE-6651-V14-trunk.patch, HBASE-6651-V2.patch,
> HBASE-6651-V3.patch, HBASE-6651-V4.patch, HBASE-6651-V5.patch,
> HBASE-6651-V6.patch, HBASE-6651-V7.patch, HBASE-6651-V8.patch,
> HBASE-6651-V9.patch, HBASE-6651.patch, sample.zip, sample.zip,
> sharedmap_for_hbaseclient.zip
>
>
> There are some operations in HTablePool accessing PoolMap in multiple places
> without any explicit synchronization.
> For example HTablePool.closeTablePool() calls PoolMap.values(), and calls
> PoolMap.remove(). If other threads add new instances to the pool in the
> middle of the calls, the newly added instances might be dropped.
> (HTablePool.closeTablePool() also has another problem that calling it by
> multiple threads causes accessing HTable by multiple threads.)
> Moreover, PoolMap is not thread safe for the same reason.
> For example PoolMap.put() calles ConcurrentMap.get() and calles
> ConcurrentMap.put(). If other threads add a new instance to the concurent map
> in the middle of the calls, the new instance might be dropped.
> And also implementations of Pool have the same problems.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9679) Binary search in HFile block
[ https://issues.apache.org/jira/browse/HBASE-9679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950358#comment-13950358 ] Matt Corgan commented on HBASE-9679: We could create an encoder format that adds an int[] index to the beginning or end of the block. Getting even fancier, we could add an index to the delta encoders for every N cells, so the decoder could binary search to a random point in the block but then encode away the common prefixes between cells N and N+1. > Binary search in HFile block > > > Key: HBASE-9679 > URL: https://issues.apache.org/jira/browse/HBASE-9679 > Project: HBase > Issue Type: Improvement > Components: HFile >Affects Versions: 0.95.2, 0.94.12 >Reporter: Liang Xie >Assignee: Liang Xie >Priority: Minor > > It's not a top priority issue, seems to me. > Right now hbase do a linear scan to search a key within a hfile block on > interst, in special case, e.g. 100% read scenario or high read/write ratio > scanario, it's useful to do a binary search improvement to reduce the CPU > cost and response time, i think the biggest benefit should be the cpu:) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10849) Fix increased javadoc warns
[
https://issues.apache.org/jira/browse/HBASE-10849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950359#comment-13950359
]
Anoop Sam John commented on HBASE-10849:
RpcServerInterface
{code}
-/**
- * RpcServer Interface.
- * Start calls {@link #openServer()} and then {@link #startThreads()}. Prefer
{@link #start()}
- * and {@link #stop()}. Only use {@link #openServer()} and {@link
#startThreads()} if in a
- * situation where you could start getting requests though the server not up
and fully
- * initiaalized.
- */
{code}
This is the doc deletion. Here we speak abt calling openServer() and then
startThreads() instead of start in certain situation. Now I can see these 2
methods removed.. There is only start() and stop. So I thought there is no
relevance for the doc description.. Correct me if I am wrong.
Ping [~jxiang]
> Fix increased javadoc warns
>
>
> Key: HBASE-10849
> URL: https://issues.apache.org/jira/browse/HBASE-10849
> Project: HBase
> Issue Type: Bug
>Reporter: Anoop Sam John
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-10849.patch
>
>
> {code}
> 6 warnings
> [WARNING] Javadoc Warnings
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:338:
> warning - Tag @link: can't find isa in
> org.apache.hadoop.hbase.regionserver.HRegionServer
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServerInterface.java:45:
> warning - Tag @link: can't find openServer() in
> org.apache.hadoop.hbase.ipc.RpcServerInterface
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServerInterface.java:45:
> warning - Tag @link: can't find startThreads() in
> org.apache.hadoop.hbase.ipc.RpcServerInterface
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServerInterface.java:45:
> warning - Tag @link: can't find openServer() in
> org.apache.hadoop.hbase.ipc.RpcServerInterface
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServerInterface.java:45:
> warning - Tag @link: can't find startThreads() in
> org.apache.hadoop.hbase.ipc.RpcServerInterface
> [WARNING]
> /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java:721:
> warning - @param argument "controller" is not a parameter name.
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10531) Revisit how the key byte[] is passed to HFileScanner.seekTo and reseekTo
[ https://issues.apache.org/jira/browse/HBASE-10531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950354#comment-13950354 ] Anoop Sam John commented on HBASE-10531: Sure Ram.. Sorry for the delay. > Revisit how the key byte[] is passed to HFileScanner.seekTo and reseekTo > > > Key: HBASE-10531 > URL: https://issues.apache.org/jira/browse/HBASE-10531 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-10531.patch, HBASE-10531_1.patch, > HBASE-10531_12.patch, HBASE-10531_2.patch, HBASE-10531_3.patch, > HBASE-10531_4.patch, HBASE-10531_5.patch, HBASE-10531_6.patch, > HBASE-10531_7.patch, HBASE-10531_8.patch, HBASE-10531_9.patch > > > Currently the byte[] key passed to HFileScanner.seekTo and > HFileScanner.reseekTo, is a combination of row, cf, qual, type and ts. And > the caller forms this by using kv.getBuffer, which is actually deprecated. > So see how this can be achieved considering kv.getBuffer is removed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10531) Revisit how the key byte[] is passed to HFileScanner.seekTo and reseekTo
[ https://issues.apache.org/jira/browse/HBASE-10531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950349#comment-13950349 ] ramkrishna.s.vasudevan commented on HBASE-10531: Planning to commit this today EOD. Pls have look at this. Thought I got 2 +1s there is a change done in the samePrefixComparator code. [~anoop.hbase] Can you have a look? > Revisit how the key byte[] is passed to HFileScanner.seekTo and reseekTo > > > Key: HBASE-10531 > URL: https://issues.apache.org/jira/browse/HBASE-10531 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-10531.patch, HBASE-10531_1.patch, > HBASE-10531_12.patch, HBASE-10531_2.patch, HBASE-10531_3.patch, > HBASE-10531_4.patch, HBASE-10531_5.patch, HBASE-10531_6.patch, > HBASE-10531_7.patch, HBASE-10531_8.patch, HBASE-10531_9.patch > > > Currently the byte[] key passed to HFileScanner.seekTo and > HFileScanner.reseekTo, is a combination of row, cf, qual, type and ts. And > the caller forms this by using kv.getBuffer, which is actually deprecated. > So see how this can be achieved considering kv.getBuffer is removed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10092) Move up on to log4j2
[ https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10092: -- Attachment: 10092v2.txt I need to convert the log4j.properties files to log4j2.xml files. > Move up on to log4j2 > > > Key: HBASE-10092 > URL: https://issues.apache.org/jira/browse/HBASE-10092 > Project: HBase > Issue Type: Sub-task >Reporter: stack > Attachments: 10092.txt, 10092v2.txt > > > Allows logging with less friction. See http://logging.apache.org/log4j/2.x/ > This rather radical transition can be done w/ minor change given they have an > adapter for apache's logging, the one we use. They also have and adapter for > slf4j so we likely can remove at least some of the 4 versions of this module > our dependencies make use of. > I made a start in attached patch but am currently stuck in maven dependency > resolve hell courtesy of our slf4j. Fixing will take some concentration and > a good net connection, an item I currently lack. Other TODOs are that will > need to fix our little log level setting jsp page -- will likely have to undo > our use of hadoop's tool here -- and the config system changes a little. > I will return to this project soon. Will bring numbers. > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10092) Move up on to log4j2
[ https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10092: -- Assignee: stack Status: Patch Available (was: Open) See how it does... > Move up on to log4j2 > > > Key: HBASE-10092 > URL: https://issues.apache.org/jira/browse/HBASE-10092 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Attachments: 10092.txt, 10092v2.txt > > > Allows logging with less friction. See http://logging.apache.org/log4j/2.x/ > This rather radical transition can be done w/ minor change given they have an > adapter for apache's logging, the one we use. They also have and adapter for > slf4j so we likely can remove at least some of the 4 versions of this module > our dependencies make use of. > I made a start in attached patch but am currently stuck in maven dependency > resolve hell courtesy of our slf4j. Fixing will take some concentration and > a good net connection, an item I currently lack. Other TODOs are that will > need to fix our little log level setting jsp page -- will likely have to undo > our use of hadoop's tool here -- and the config system changes a little. > I will return to this project soon. Will bring numbers. > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10850) Unexpected behavior when using filter SingleColumnValueFilter
[
https://issues.apache.org/jira/browse/HBASE-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950339#comment-13950339
]
Anoop Sam John commented on HBASE-10850:
Ping [~lhofhansl]
> Unexpected behavior when using filter SingleColumnValueFilter
> -
>
> Key: HBASE-10850
> URL: https://issues.apache.org/jira/browse/HBASE-10850
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.96.1.1
>Reporter: Fabien Le Gallo
>Assignee: haosdent
> Attachments: HBASE-10850-96.patch, HBASE-10850.patch,
> HBaseSingleColumnValueFilterTest.java
>
>
> When using the filter SingleColumnValueFilter, and depending of the columns
> specified in the scan (filtering column always specified), the results can be
> different.
> Here is an example.
> Suppose the following table:
> ||key||a:foo||a:bar||b:foo||b:bar||
> |1|false|_flag_|_flag_|_flag_|
> |2|true|_flag_|_flag_|_flag_|
> |3| |_flag_|_flag_|_flag_|
> With this filter:
> {code}
> SingleColumnValueFilter filter = new
> SingleColumnValueFilter(Bytes.toBytes("a"), Bytes.toBytes("foo"),
> CompareOp.EQUAL, new BinaryComparator(Bytes.toBytes("false")));
> filter.setFilterIfMissing(true);
> {code}
> Depending of how I specify the list of columns to add in the scan, the result
> is different. Yet, all examples below should always return only the first row
> (key '1'):
> OK:
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> scan.addFamily(Bytes.toBytes("b"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("foo"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("bar"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> {code}
> This is a regression as it was working properly on HBase 0.92.
> You will find in attachement the unit tests reproducing the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10692) The Multi TableMap job don't support the security HBase cluster
[ https://issues.apache.org/jira/browse/HBASE-10692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950338#comment-13950338 ] Liu Shaohui commented on HBASE-10692: - After a hard try, i failed to enable hbase security with SIMPLE auth method in unit test env. Maybe I will add a test after HBASE-7781, but that issue does't have any progress. So I think if we can just let this issue go first and make multi table map jobs work with the security HBase cluster. I tested the patch in our security HBase cluster and It works fine. What's your opinions? [~mbertozzi] [~ndimiduk] [~apurtell] > The Multi TableMap job don't support the security HBase cluster > --- > > Key: HBASE-10692 > URL: https://issues.apache.org/jira/browse/HBASE-10692 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Attachments: HBASE-10692-0.94-v1.diff, HBASE-10692-trunk-v1.diff, > HBASE-10692-trunk-v2.diff > > > HBASE-3996 adds the support of multiple tables and scanners as input to the > mapper in map/reduce jobs. But it don't support the security HBase cluster. > [~erank] [~bbaugher] > Ps: HBASE-3996 only support multiple tables from the same HBase cluster. > Should we support multiple tables from the different clusters? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10850) Unexpected behavior when using filter SingleColumnValueFilter
[
https://issues.apache.org/jira/browse/HBASE-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950337#comment-13950337
]
Anoop Sam John commented on HBASE-10850:
{code}
HRegion#nextInternal(List results, int limit)
// save that the row was empty before filters applied to it.
final boolean isEmptyRow = results.isEmpty();
// We have the part of the row necessary for filtering (all of it, usually).
// First filter with the filterRow(List).
if (filter != null && filter.hasFilterRow()) {
filter.filterRowCells(results);
}
if (isEmptyRow || filterRow()) {
...
private boolean filterRow() throws IOException {
// when hasFilterRow returns true, filter.filterRow() will be called
automatically inside
// filterRowCells(List kvs) so we skip that scenario here.
return filter != null && (!filter.hasFilterRow())
&& filter.filterRow();
}
{code}
In 96+ version filterRowCells(List) is internally calling filterRow() also and
if that return true, just clears the passed cells list. (SCVF uses filterRow())
And u can see private boolean filterRow() wont get executed because of this
(!filter.hasFilterRow()).
results is no empty before applying the 1st filter op.
So here we have to make change as
{code}
if (results.isEmpty() || filterRow()) {
}
{code}
Pls correct if I am wrong..
> Unexpected behavior when using filter SingleColumnValueFilter
> -
>
> Key: HBASE-10850
> URL: https://issues.apache.org/jira/browse/HBASE-10850
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.96.1.1
>Reporter: Fabien Le Gallo
>Assignee: haosdent
> Attachments: HBASE-10850-96.patch, HBASE-10850.patch,
> HBaseSingleColumnValueFilterTest.java
>
>
> When using the filter SingleColumnValueFilter, and depending of the columns
> specified in the scan (filtering column always specified), the results can be
> different.
> Here is an example.
> Suppose the following table:
> ||key||a:foo||a:bar||b:foo||b:bar||
> |1|false|_flag_|_flag_|_flag_|
> |2|true|_flag_|_flag_|_flag_|
> |3| |_flag_|_flag_|_flag_|
> With this filter:
> {code}
> SingleColumnValueFilter filter = new
> SingleColumnValueFilter(Bytes.toBytes("a"), Bytes.toBytes("foo"),
> CompareOp.EQUAL, new BinaryComparator(Bytes.toBytes("false")));
> filter.setFilterIfMissing(true);
> {code}
> Depending of how I specify the list of columns to add in the scan, the result
> is different. Yet, all examples below should always return only the first row
> (key '1'):
> OK:
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> scan.addFamily(Bytes.toBytes("b"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("foo"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("bar"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> {code}
> This is a regression as it was working properly on HBase 0.92.
> You will find in attachement the unit tests reproducing the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10118) Major compact keeps deletes with future timestamps
[
https://issues.apache.org/jira/browse/HBASE-10118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950330#comment-13950330
]
Liu Shaohui commented on HBASE-10118:
-
Anyone help to push this issue? [~yuzhih...@gmail.com] [~lhofhansl]
> Major compact keeps deletes with future timestamps
> --
>
> Key: HBASE-10118
> URL: https://issues.apache.org/jira/browse/HBASE-10118
> Project: HBase
> Issue Type: Bug
> Components: Compaction, Deletes, regionserver
>Reporter: Max Lapan
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-10118-trunk-v1.diff, HBASE-10118-trunk-v2.diff
>
>
> Hello!
> During migration from HBase 0.90.6 to 0.94.6 we found changed behaviour in
> how major compact handles delete markers with timestamps in the future.
> Before HBASE-4721 major compact purged deletes regardless of their timestamp.
> Newer versions keep them in HFile until timestamp not reached.
> I guess this happened due to new check in ScanQueryMatcher
> {{EnvironmentEdgeManager.currentTimeMillis() - timestamp) <=
> timeToPurgeDeletes}}.
> This can be worked around by specifying large negative value in
> {{hbase.hstore.time.to.purge.deletes}} option, but, unfortunately, negative
> values are pulled up to zero by Math.max in HStore.java.
> Maybe, we are trying to do something weird by specifing delete timestamp in
> future, but HBASE-4721 definitely breaks old behaviour we rely on.
> Steps to reproduce this:
> {code}
> put 'test', 'delmeRow', 'delme:something', 'hello'
> flush 'test'
> delete 'test', 'delmeRow', 'delme:something', 1394161431061
> flush 'test'
> major_compact 'test'
> {code}
> Before major_compact we have two hfiles with the following:
> {code}
> first:
> K: delmeRow/delme:something/1384161431061/Put/vlen=5/ts=0
> second:
> K: delmeRow/delme:something/1394161431061/DeleteColumn/vlen=0/ts=0
> {code}
> After major compact we get the following:
> {code}
> K: delmeRow/delme:something/1394161431061/DeleteColumn/vlen=0/ts=0
> {code}
> In our installation, we resolved this by removing Math.max and setting
> hbase.hstore.time.to.purge.deletes to Integer.MIN_VALUE, which purges delete
> markers, and it looks like a solution. But, maybe, there are better approach.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10859) HStore.openStoreFiles() should pass the StoreFileInfo object to createStoreFileAndReader()
[
https://issues.apache.org/jira/browse/HBASE-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-10859:
--
Attachment: hbase-10859_v1.patch
Here is a patch, which basically enforces use of HFileLink. I'll also add a
more comprehensive concurrent compaction + read from secondary test.
> HStore.openStoreFiles() should pass the StoreFileInfo object to
> createStoreFileAndReader()
> --
>
> Key: HBASE-10859
> URL: https://issues.apache.org/jira/browse/HBASE-10859
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: hbase-10070
>
> Attachments: hbase-10859_v1.patch
>
>
> We sometimes see the following stack trace on test logs (TestReplicasClient),
> but this is not test-specific:
> {code}
> 2014-03-26 21:44:18,662 ERROR [RS_OPEN_REGION-c64-s12:35852-2]
> handler.OpenRegionHandler(481): Failed open of
> region=TestReplicasClient,,1395895445056_0001.5f8b8db27e36d2dde781193d92a05730.,
> starting to roll back the global memstore size.
> java.io.IOException: java.io.IOException: java.io.FileNotFoundException: File
> does not exist:
> hdfs://localhost:56276/user/jenkins/hbase/data/default/TestReplicasClient/856934fb87781c9030975706b66137a5/info/589000f197b048e0897e1d81dd7e3a90
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:739)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:646)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:617)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4447)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4417)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4389)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4345)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4296)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.io.IOException: java.io.FileNotFoundException: File does not
> exist:
> hdfs://localhost:56276/user/jenkins/hbase/data/default/TestReplicasClient/856934fb87781c9030975706b66137a5/info/589000f197b048e0897e1d81dd7e3a90
> at
> org.apache.hadoop.hbase.regionserver.HStore.openStoreFiles(HStore.java:531)
> at
> org.apache.hadoop.hbase.regionserver.HStore.loadStoreFiles(HStore.java:486)
> at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:254)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:3357)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:710)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:707)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> ... 3 more
> Caused by: java.io.FileNotFoundException: File does not exist:
> hdfs://localhost:56276/user/jenkins/hbase/data/default/TestReplicasClient/856934fb87781c9030975706b66137a5/info/589000f197b048e0897e1d81dd7e3a90
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:95)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:600)
> at org.apache.hadoop.hbase.regionserver.HStore.access$000(HStore.java:121)
> at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:506)
> at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:503)
> ... 8 more
> {code}
> The region fails t
[jira] [Commented] (HBASE-10859) HStore.openStoreFiles() should pass the StoreFileInfo object to createStoreFileAndReader()
[
https://issues.apache.org/jira/browse/HBASE-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950326#comment-13950326
]
Enis Soztutar commented on HBASE-10859:
---
This turned out to be a bit more involved than the description. It seems that
the way we open the store files of the primary from secondaries (HBASE-10352)
does not use the HFileLink. I thought it was using it automatically, but does
not look like it is the case. The secondary region reads was still working even
when the primary does compaction (TestRegionReplicas.testRefreshStoreFiles()
for example) because we open the file handle in hdfs, and even if the file is
moved we would be able to read it because blocks are open.
However, it seems that we should force using the HFileLink for opening the
store files from secondaries using the FileLink.FileLinkInputStream, which
gives us the guarantees for reading hfile from data or archive location.
> HStore.openStoreFiles() should pass the StoreFileInfo object to
> createStoreFileAndReader()
> --
>
> Key: HBASE-10859
> URL: https://issues.apache.org/jira/browse/HBASE-10859
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: hbase-10070
>
>
> We sometimes see the following stack trace on test logs (TestReplicasClient),
> but this is not test-specific:
> {code}
> 2014-03-26 21:44:18,662 ERROR [RS_OPEN_REGION-c64-s12:35852-2]
> handler.OpenRegionHandler(481): Failed open of
> region=TestReplicasClient,,1395895445056_0001.5f8b8db27e36d2dde781193d92a05730.,
> starting to roll back the global memstore size.
> java.io.IOException: java.io.IOException: java.io.FileNotFoundException: File
> does not exist:
> hdfs://localhost:56276/user/jenkins/hbase/data/default/TestReplicasClient/856934fb87781c9030975706b66137a5/info/589000f197b048e0897e1d81dd7e3a90
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:739)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:646)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:617)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4447)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4417)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4389)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4345)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4296)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.io.IOException: java.io.FileNotFoundException: File does not
> exist:
> hdfs://localhost:56276/user/jenkins/hbase/data/default/TestReplicasClient/856934fb87781c9030975706b66137a5/info/589000f197b048e0897e1d81dd7e3a90
> at
> org.apache.hadoop.hbase.regionserver.HStore.openStoreFiles(HStore.java:531)
> at
> org.apache.hadoop.hbase.regionserver.HStore.loadStoreFiles(HStore.java:486)
> at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:254)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:3357)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:710)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:707)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> ... 3 more
> Caused by: java.io.FileNotFoundException: File does not exist:
> hdfs://localhost:56276/user/jenkins/hbase/data/default/TestReplicasClient/856934fb87781c9030975706b66137a5/info/589000f197b048e0897e1d81dd7e3a90
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.ge
[jira] [Commented] (HBASE-10426) user_permission in security.rb calls non-existent UserPermission#getTable method
[
https://issues.apache.org/jira/browse/HBASE-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950325#comment-13950325
]
stack commented on HBASE-10426:
---
Committed to 0.96 at [~jinghe] request
> user_permission in security.rb calls non-existent UserPermission#getTable
> method
>
>
> Key: HBASE-10426
> URL: https://issues.apache.org/jira/browse/HBASE-10426
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.99.0
>
> Attachments: 10426-v1.txt, 10426-v2.txt
>
>
> In user mailing list, Alex reported this error under the thread
> "user_permission error - undefined method 'getTable'":
> {code}
> hbase(main):010:0> create 'foo','bar'
> 0 row(s) in 0.5780 seconds
> => Hbase::Table - foo
> hbase(main):011:0> user_permission 'foo'
> User Table,Family,Qualifier:Permission
> *ERROR: undefined method `getTable' for
> #*
> {code}
> UserPermission#getTable method doesn't exist
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10855) Enable hfilev3 by default
[ https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10855: -- Attachment: 10855.txt Retry. Failures seem unrelated (Strange these two failed though) > Enable hfilev3 by default > - > > Key: HBASE-10855 > URL: https://issues.apache.org/jira/browse/HBASE-10855 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.0 > > Attachments: 10855.txt, 10855.txt > > > Distributed log replay needs this. Should be on by default in 1.0/0.99. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10846) Links between active and backup masters are broken
[
https://issues.apache.org/jira/browse/HBASE-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-10846:
--
Resolution: Not A Problem
Status: Resolved (was: Patch Available)
Closing at [~liushaohui]'s request
> Links between active and backup masters are broken
> --
>
> Key: HBASE-10846
> URL: https://issues.apache.org/jira/browse/HBASE-10846
> Project: HBase
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-10846-trunk-v1.diff
>
>
> Links between active and backup masters are broken for the the blanks before
> info port in the url.
> {code}
> href="//wcc-hadoop-tst-ct01.bj: 12501/master-status"
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-6651) Improve thread safety of HTablePool
[ https://issues.apache.org/jira/browse/HBASE-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hiroshi Ikeda updated HBASE-6651: - Hadoop Flags: (was: Reviewed) > Improve thread safety of HTablePool > --- > > Key: HBASE-6651 > URL: https://issues.apache.org/jira/browse/HBASE-6651 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.1 >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda > Attachments: HBASE-6651-V10.patch, HBASE-6651-V11.patch, > HBASE-6651-V12.patch, HBASE-6651-V13.patch, HBASE-6651-V14-0.96.patch, > HBASE-6651-V14-0.98.patch, HBASE-6651-V14-trunk.patch, HBASE-6651-V2.patch, > HBASE-6651-V3.patch, HBASE-6651-V4.patch, HBASE-6651-V5.patch, > HBASE-6651-V6.patch, HBASE-6651-V7.patch, HBASE-6651-V8.patch, > HBASE-6651-V9.patch, HBASE-6651.patch, sample.zip, sample.zip, > sharedmap_for_hbaseclient.zip > > > There are some operations in HTablePool accessing PoolMap in multiple places > without any explicit synchronization. > For example HTablePool.closeTablePool() calls PoolMap.values(), and calls > PoolMap.remove(). If other threads add new instances to the pool in the > middle of the calls, the newly added instances might be dropped. > (HTablePool.closeTablePool() also has another problem that calling it by > multiple threads causes accessing HTable by multiple threads.) > Moreover, PoolMap is not thread safe for the same reason. > For example PoolMap.put() calles ConcurrentMap.get() and calles > ConcurrentMap.put(). If other threads add a new instance to the concurent map > in the middle of the calls, the new instance might be dropped. > And also implementations of Pool have the same problems. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-6651) Improve thread safety of HTablePool
[ https://issues.apache.org/jira/browse/HBASE-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hiroshi Ikeda updated HBASE-6651: - Attachment: HBASE-6651-V14-trunk.patch HBASE-6651-V14-0.98.patch HBASE-6651-V14-0.96.patch Added fixed patches for 0.96, 0.98 and trunk. I reduced changing of existing codes, which have been quickly changed and too difficult to aim. I also added new methods to ShardMap to refer its registered objects, which introduced possibility to invalidly access thread unsafe objects, but I compromised because of applicability to existing codes, and then I excluded some unused classes that I had created. This ticket was created about more than a year ago and I already forgot many things, but HBase still uses PoolMap that violates the contract of Map, inconsistently changes its methods' behaviors by pool-type, and fails to keep thread-safety, so I hope the patches help. > Improve thread safety of HTablePool > --- > > Key: HBASE-6651 > URL: https://issues.apache.org/jira/browse/HBASE-6651 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.1 >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda > Attachments: HBASE-6651-V10.patch, HBASE-6651-V11.patch, > HBASE-6651-V12.patch, HBASE-6651-V13.patch, HBASE-6651-V14-0.96.patch, > HBASE-6651-V14-0.98.patch, HBASE-6651-V14-trunk.patch, HBASE-6651-V2.patch, > HBASE-6651-V3.patch, HBASE-6651-V4.patch, HBASE-6651-V5.patch, > HBASE-6651-V6.patch, HBASE-6651-V7.patch, HBASE-6651-V8.patch, > HBASE-6651-V9.patch, HBASE-6651.patch, sample.zip, sample.zip, > sharedmap_for_hbaseclient.zip > > > There are some operations in HTablePool accessing PoolMap in multiple places > without any explicit synchronization. > For example HTablePool.closeTablePool() calls PoolMap.values(), and calls > PoolMap.remove(). If other threads add new instances to the pool in the > middle of the calls, the newly added instances might be dropped. > (HTablePool.closeTablePool() also has another problem that calling it by > multiple threads causes accessing HTable by multiple threads.) > Moreover, PoolMap is not thread safe for the same reason. > For example PoolMap.put() calles ConcurrentMap.get() and calles > ConcurrentMap.put(). If other threads add a new instance to the concurent map > in the middle of the calls, the new instance might be dropped. > And also implementations of Pool have the same problems. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9679) Binary search in HFile block
[ https://issues.apache.org/jira/browse/HBASE-9679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950306#comment-13950306 ] Liang Xie commented on HBASE-9679: -- when i filed this issue long long time ago, my thought was that we could impl an adaptive index algo which is similar with Cassandra's, that means if we detect a hotspot kv inside a HFile block in lots of read requests, then we could build a index upon this kv, the most code be modified should the HFileIndex class. But there's is a little risk to just do it only in 0.94 branch:) since we have PrefixTree already in later branch. > Binary search in HFile block > > > Key: HBASE-9679 > URL: https://issues.apache.org/jira/browse/HBASE-9679 > Project: HBase > Issue Type: Improvement > Components: HFile >Affects Versions: 0.95.2, 0.94.12 >Reporter: Liang Xie >Assignee: Liang Xie >Priority: Minor > > It's not a top priority issue, seems to me. > Right now hbase do a linear scan to search a key within a hfile block on > interst, in special case, e.g. 100% read scenario or high read/write ratio > scanario, it's useful to do a binary search improvement to reduce the CPU > cost and response time, i think the biggest benefit should be the cpu:) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10846) Links between active and backup masters are broken
[
https://issues.apache.org/jira/browse/HBASE-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950300#comment-13950300
]
Liu Shaohui commented on HBASE-10846:
-
[~stack][~jxiang]
0.98 and 0.96 don't have this issue too, for the links are written in one line.
{code}
/master-status" target="_blank"><% serverName.get
{code}
Please close this issue. Sorry for my mistake.
> Links between active and backup masters are broken
> --
>
> Key: HBASE-10846
> URL: https://issues.apache.org/jira/browse/HBASE-10846
> Project: HBase
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-10846-trunk-v1.diff
>
>
> Links between active and backup masters are broken for the the blanks before
> info port in the url.
> {code}
> href="//wcc-hadoop-tst-ct01.bj: 12501/master-status"
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10788) Add 99th percentile of latency in PE
[ https://issues.apache.org/jira/browse/HBASE-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950298#comment-13950298 ] Liang Xie commented on HBASE-10788: --- Integrated into trunk, thanks all for review, thank you for the patch [~liushaohui] :) > Add 99th percentile of latency in PE > > > Key: HBASE-10788 > URL: https://issues.apache.org/jira/browse/HBASE-10788 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.99.0 >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 0.99.0 > > Attachments: HBASE-10788-trunk-v1.diff, HBASE-10788-trunk-v2.diff, > HBASE-10788-trunk-v3.diff > > > In production env, 99th percentile of latency is more important than the avg. > The 99th percentile is helpful to measure the influence of GC, slow > read/write of HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10788) Add 99th percentile of latency in PE
[ https://issues.apache.org/jira/browse/HBASE-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-10788: -- Resolution: Fixed Fix Version/s: 0.99.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Add 99th percentile of latency in PE > > > Key: HBASE-10788 > URL: https://issues.apache.org/jira/browse/HBASE-10788 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.99.0 >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 0.99.0 > > Attachments: HBASE-10788-trunk-v1.diff, HBASE-10788-trunk-v2.diff, > HBASE-10788-trunk-v3.diff > > > In production env, 99th percentile of latency is more important than the avg. > The 99th percentile is helpful to measure the influence of GC, slow > read/write of HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10846) Links between active and backup masters are broken
[
https://issues.apache.org/jira/browse/HBASE-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950286#comment-13950286
]
Liu Shaohui commented on HBASE-10846:
-
[~stack] [~jxiang]
Sorry for not updating the trunk code and the latest trunk code does't have
this issue.
I will upload a patch for 0.98/0.96 later.
> Links between active and backup masters are broken
> --
>
> Key: HBASE-10846
> URL: https://issues.apache.org/jira/browse/HBASE-10846
> Project: HBase
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: HBASE-10846-trunk-v1.diff
>
>
> Links between active and backup masters are broken for the the blanks before
> info port in the url.
> {code}
> href="//wcc-hadoop-tst-ct01.bj: 12501/master-status"
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10848) Filter SingleColumnValueFilter combined with NullComparator does not work
[
https://issues.apache.org/jira/browse/HBASE-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950261#comment-13950261
]
chunhui shen commented on HBASE-10848:
--
Move the test case to TestFilter.java or TestFromClientSide?
> Filter SingleColumnValueFilter combined with NullComparator does not work
> -
>
> Key: HBASE-10848
> URL: https://issues.apache.org/jira/browse/HBASE-10848
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.96.1.1
>Reporter: Fabien Le Gallo
> Attachments: HBASE-10848.patch, HBASE_10848-v2.patch,
> HBaseRegression.java, TestScanWithNullComparable.java
>
>
> I want to filter out from the scan the rows that does not have a specific
> column qualifier. For this purpose I use the filter SingleColumnValueFilter
> combined with the NullComparator.
> But every time I use this in a scan, I get the following exception:
> {code}
> java.lang.RuntimeException: org.apache.hadoop.hbase.DoNotRetryIOException:
> Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
> at
> org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:47)
> at
> com.xxx.xxx.test.HBaseRegression.nullComparator(HBaseRegression.java:92)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry
> of OutOfOrderScannerNextException: was there a rpc timeout?
> at
> org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:391)
> at
> org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:44)
> ... 25 more
> Caused by: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException:
> org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected
> nextCallSeq: 1 But the nextCallSeq got from client: 0; request=scanner_id:
> 7998309028985532303 number_of_rows: 100 close_scanner: false next_call_seq: 0
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3011)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26929)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175)
> at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1879)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526
[jira] [Commented] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950241#comment-13950241
]
Hudson commented on HBASE-10852:
ABORTED: Integrated in HBase-0.98 #254 (See
[https://builds.apache.org/job/HBase-0.98/254/])
HBASE-10852 TestDistributedLogSplitting#testDisallowWritesInRecovering
occasionally fails (tedyu: rev 1582486)
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 10852-v1.txt, 10852-v2.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950240#comment-13950240
]
Hudson commented on HBASE-10852:
ABORTED: Integrated in HBase-0.98-on-Hadoop-1.1 #238 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/238/])
HBASE-10852 TestDistributedLogSplitting#testDisallowWritesInRecovering
occasionally fails (tedyu: rev 1582486)
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 10852-v1.txt, 10852-v2.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10855) Enable hfilev3 by default
[
https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950231#comment-13950231
]
Hadoop QA commented on HBASE-10855:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637276/10855.txt
against trunk revision .
ATTACHMENT ID: 12637276
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9119//console
This message is automatically generated.
> Enable hfilev3 by default
> -
>
> Key: HBASE-10855
> URL: https://issues.apache.org/jira/browse/HBASE-10855
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 0.99.0
>
> Attachments: 10855.txt
>
>
> Distributed log replay needs this. Should be on by default in 1.0/0.99.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950229#comment-13950229
]
Hudson commented on HBASE-10852:
FAILURE: Integrated in HBase-TRUNK #5045 (See
[https://builds.apache.org/job/HBase-TRUNK/5045/])
HBASE-10852 TestDistributedLogSplitting#testDisallowWritesInRecovering
occasionally fails (tedyu: rev 1582487)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 10852-v1.txt, 10852-v2.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10426) user_permission in security.rb calls non-existent UserPermission#getTable method
[
https://issues.apache.org/jira/browse/HBASE-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950205#comment-13950205
]
Jerry He commented on HBASE-10426:
--
Hi, [~te...@apache.org]
Could you put the fix in 0.96? It is broken there too.
> user_permission in security.rb calls non-existent UserPermission#getTable
> method
>
>
> Key: HBASE-10426
> URL: https://issues.apache.org/jira/browse/HBASE-10426
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.99.0
>
> Attachments: 10426-v1.txt, 10426-v2.txt
>
>
> In user mailing list, Alex reported this error under the thread
> "user_permission error - undefined method 'getTable'":
> {code}
> hbase(main):010:0> create 'foo','bar'
> 0 row(s) in 0.5780 seconds
> => Hbase::Table - foo
> hbase(main):011:0> user_permission 'foo'
> User Table,Family,Qualifier:Permission
> *ERROR: undefined method `getTable' for
> #*
> {code}
> UserPermission#getTable method doesn't exist
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-7912) HBase Backup/Restore Based on HBase Snapshot and FileLink
[ https://issues.apache.org/jira/browse/HBASE-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7912: - Issue Type: Sub-task (was: New Feature) Parent: HBASE-10856 > HBase Backup/Restore Based on HBase Snapshot and FileLink > - > > Key: HBASE-7912 > URL: https://issues.apache.org/jira/browse/HBASE-7912 > Project: HBase > Issue Type: Sub-task >Reporter: Richard Ding >Assignee: Richard Ding > > There have been attempts in the past to come up with a viable HBase > backup/restore solution (e.g., HBASE-4618). Recently, there are many > advancements and new features in HBase, for example, FileLink, Snapshot, and > Distributed Barrier Procedure. This is a proposal for a backup/restore > solution that utilizes these new features to achieve better performance and > consistency. > > A common practice of backup and restore in database is to first take full > baseline backup, and then periodically take incremental backup that capture > the changes since the full baseline backup. HBase cluster can store massive > amount data. Combination of full backups with incremental backups has > tremendous benefit for HBase as well. The following is a typical scenario > for full and incremental backup. > # The user takes a full backup of a table or a set of tables in HBase. > # The user schedules periodical incremental backups to capture the changes > from the full backup, or from last incremental backup. > # The user needs to restore table data to a past point of time. > # The full backup is restored to the table(s) or to different table name(s). > Then the incremental backups that are up to the desired point in time are > applied on top of the full backup. > We would support the following key features and capabilities. > * Full backup uses HBase snapshot to capture HFiles. > * Use HBase WALs to capture incremental changes, but we use bulk load of > HFiles for fast incremental restore. > * Support single table or a set of tables, and column family level backup and > restore. > * Restore to different table names. > * Support adding additional tables or CF to backup set without interruption > of incremental backup schedule. > * Support rollup/combining of incremental backups into longer period and > bigger incremental backups. > * Unified command line interface for all the above. > The solution will support HBase backup to FileSystem, either on the same > cluster or across clusters. It has the flexibility to support backup to > other devices and servers in the future. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-10859) HStore.openStoreFiles() should pass the StoreFileInfo object to createStoreFileAndReader()
Enis Soztutar created HBASE-10859:
-
Summary: HStore.openStoreFiles() should pass the StoreFileInfo
object to createStoreFileAndReader()
Key: HBASE-10859
URL: https://issues.apache.org/jira/browse/HBASE-10859
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: hbase-10070
We sometimes see the following stack trace on test logs (TestReplicasClient),
but this is not test-specific:
{code}
2014-03-26 21:44:18,662 ERROR [RS_OPEN_REGION-c64-s12:35852-2]
handler.OpenRegionHandler(481): Failed open of
region=TestReplicasClient,,1395895445056_0001.5f8b8db27e36d2dde781193d92a05730.,
starting to roll back the global memstore size.
java.io.IOException: java.io.IOException: java.io.FileNotFoundException: File
does not exist:
hdfs://localhost:56276/user/jenkins/hbase/data/default/TestReplicasClient/856934fb87781c9030975706b66137a5/info/589000f197b048e0897e1d81dd7e3a90
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:739)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:646)
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:617)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4447)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4417)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4389)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4345)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4296)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: java.io.FileNotFoundException: File does not
exist:
hdfs://localhost:56276/user/jenkins/hbase/data/default/TestReplicasClient/856934fb87781c9030975706b66137a5/info/589000f197b048e0897e1d81dd7e3a90
at org.apache.hadoop.hbase.regionserver.HStore.openStoreFiles(HStore.java:531)
at org.apache.hadoop.hbase.regionserver.HStore.loadStoreFiles(HStore.java:486)
at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:254)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:3357)
at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:710)
at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:707)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
... 3 more
Caused by: java.io.FileNotFoundException: File does not exist:
hdfs://localhost:56276/user/jenkins/hbase/data/default/TestReplicasClient/856934fb87781c9030975706b66137a5/info/589000f197b048e0897e1d81dd7e3a90
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
at
org.apache.hadoop.hbase.regionserver.StoreFileInfo.(StoreFileInfo.java:95)
at
org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:600)
at org.apache.hadoop.hbase.regionserver.HStore.access$000(HStore.java:121)
at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:506)
at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:503)
... 8 more
{code}
The region fails to open for the region replica, because at this time, the
primary region is performing a compaction. The file is moved to the archive
directory in between listing of store files and opening those store files from
the secondary.
The secondary region should able to deal with this through usage of
StoreFileInfo and HFile, but since we are reconstructing the StoreFileInfo
object twice between HStore.openStoreFiles() and createStoreFileAndReader() we
are getting this exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-10858) TestRegionRebalancing is failing
[ https://issues.apache.org/jira/browse/HBASE-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-10858. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to hbase-10070. > TestRegionRebalancing is failing > > > Key: HBASE-10858 > URL: https://issues.apache.org/jira/browse/HBASE-10858 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: hbase-10070 > > Attachments: hbase-10858_v1.patch, hbase-10858_v2.patch > > > After HBASE-10620 and addendum1, the test is failing, and even with a fix it > is flaky. > We should fix the test to be reliable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10858) TestRegionRebalancing is failing
[ https://issues.apache.org/jira/browse/HBASE-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10858: -- Attachment: hbase-10858_v2.patch Thanks for reviews. Addressed nit in v2. > TestRegionRebalancing is failing > > > Key: HBASE-10858 > URL: https://issues.apache.org/jira/browse/HBASE-10858 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: hbase-10070 > > Attachments: hbase-10858_v1.patch, hbase-10858_v2.patch > > > After HBASE-10620 and addendum1, the test is failing, and even with a fix it > is flaky. > We should fix the test to be reliable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10858) TestRegionRebalancing is failing
[ https://issues.apache.org/jira/browse/HBASE-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950170#comment-13950170 ] stack commented on HBASE-10858: --- +1 for trying it. > TestRegionRebalancing is failing > > > Key: HBASE-10858 > URL: https://issues.apache.org/jira/browse/HBASE-10858 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: hbase-10070 > > Attachments: hbase-10858_v1.patch > > > After HBASE-10620 and addendum1, the test is failing, and even with a fix it > is flaky. > We should fix the test to be reliable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-10853.
---
Resolution: Fixed
Fix Version/s: 0.99.0
Assignee: stack
Hadoop Flags: Reviewed
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
>Assignee: stack
> Fix For: 0.99.0
>
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950168#comment-13950168
]
stack commented on HBASE-10853:
---
ok.
Let me apply this for now.
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10851) Wait for regionservers to join the cluster
[ https://issues.apache.org/jira/browse/HBASE-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950165#comment-13950165 ] Jimmy Xiang commented on HBASE-10851: - Sure, let me take a look. Thanks. > Wait for regionservers to join the cluster > -- > > Key: HBASE-10851 > URL: https://issues.apache.org/jira/browse/HBASE-10851 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Critical > > With HBASE-10569, if regionservers are started a while after the master, all > regions will be assigned to the master. That may not be what users expect. > A work-around is to always start regionservers before masters. > I was wondering if the master can wait a little for other regionservers to > join. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-10851) Wait for regionservers to join the cluster
[ https://issues.apache.org/jira/browse/HBASE-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang reassigned HBASE-10851: --- Assignee: Jimmy Xiang > Wait for regionservers to join the cluster > -- > > Key: HBASE-10851 > URL: https://issues.apache.org/jira/browse/HBASE-10851 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Critical > > With HBASE-10569, if regionservers are started a while after the master, all > regions will be assigned to the master. That may not be what users expect. > A work-around is to always start regionservers before masters. > I was wondering if the master can wait a little for other regionservers to > join. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10815) Master regionserver should be rolling-upgradable
[ https://issues.apache.org/jira/browse/HBASE-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-10815: Fix Version/s: 0.99.0 Status: Patch Available (was: Open) > Master regionserver should be rolling-upgradable > > > Key: HBASE-10815 > URL: https://issues.apache.org/jira/browse/HBASE-10815 > Project: HBase > Issue Type: Sub-task > Components: master, Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.99.0 > > Attachments: hbase-10815.patch > > > In HBASE-10569, two things could affect the rolling-upgrade from a 0.96+ > release: > * Master doesn't have its own info server any. It shares the same info server > with the regionserver. We can have a setting so that we can start two info > servers, one for the master on the original port, and one for the > regionserver. > * Backup master is a regionserver now. So it could hold regions. This could > affect some deployment. We can have a setting so that we can prevent backup > master from serving any region. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10815) Master regionserver should be rolling-upgradable
[ https://issues.apache.org/jira/browse/HBASE-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-10815: Attachment: hbase-10815.patch Attached the first patch. It is on RB: https://reviews.apache.org/r/19759/ In the patch, two configurations are added to control if we assign regions to backup masters, and if we start two info servers in M+RS. In order to reliably not assigning regions to backup masters if so configured, we changed a little on how master create the backup znode, and how server manager adds new server. > Master regionserver should be rolling-upgradable > > > Key: HBASE-10815 > URL: https://issues.apache.org/jira/browse/HBASE-10815 > Project: HBase > Issue Type: Sub-task > Components: master, Region Assignment >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.99.0 > > Attachments: hbase-10815.patch > > > In HBASE-10569, two things could affect the rolling-upgrade from a 0.96+ > release: > * Master doesn't have its own info server any. It shares the same info server > with the regionserver. We can have a setting so that we can start two info > servers, one for the master on the original port, and one for the > regionserver. > * Backup master is a regionserver now. So it could hold regions. This could > affect some deployment. We can have a setting so that we can prevent backup > master from serving any region. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10858) TestRegionRebalancing is failing
[
https://issues.apache.org/jira/browse/HBASE-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950146#comment-13950146
]
Ted Yu commented on HBASE-10858:
Let's try it.
{code}
-// However if we add a server, then the balance() call should return true
+// However if we add a server, then the balance() call should return true
{code}
nit: remove 'However' since the previous return value is 'true'.
> TestRegionRebalancing is failing
>
>
> Key: HBASE-10858
> URL: https://issues.apache.org/jira/browse/HBASE-10858
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: hbase-10070
>
> Attachments: hbase-10858_v1.patch
>
>
> After HBASE-10620 and addendum1, the test is failing, and even with a fix it
> is flaky.
> We should fix the test to be reliable.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950132#comment-13950132
]
Jimmy Xiang commented on HBASE-10853:
-
That means checkOpen should throw ServiceException instead of IOException.
Otherwise, the code may look messy, considering converting IOE to SE. The
current patch looks fine to me too.
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10858) TestRegionRebalancing is failing
[ https://issues.apache.org/jira/browse/HBASE-10858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-10858: -- Attachment: hbase-10858_v1.patch Here is a patch which should fix the issue. Ran the test 100 times, it did not fail. > TestRegionRebalancing is failing > > > Key: HBASE-10858 > URL: https://issues.apache.org/jira/browse/HBASE-10858 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: hbase-10070 > > Attachments: hbase-10858_v1.patch > > > After HBASE-10620 and addendum1, the test is failing, and even with a fix it > is flaky. > We should fix the test to be reliable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10278) Provide better write predictability
[ https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950135#comment-13950135 ] stack commented on HBASE-10278: --- Is the failure related? > Provide better write predictability > --- > > Key: HBASE-10278 > URL: https://issues.apache.org/jira/browse/HBASE-10278 > Project: HBase > Issue Type: New Feature >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Attachments: 10278-trunk-v2.1.patch, 10278-trunk-v2.1.patch, > 10278-wip-1.1.patch, Multiwaldesigndoc.pdf, SwitchWriterFlow.pptx > > > Currently, HBase has one WAL per region server. > Whenever there is any latency in the write pipeline (due to whatever reasons > such as n/w blip, a node in the pipeline having a bad disk, etc), the overall > write latency suffers. > Jonathan Hsieh and I analyzed various approaches to tackle this issue. We > also looked at HBASE-5699, which talks about adding concurrent multi WALs. > Along with performance numbers, we also focussed on design simplicity, > minimum impact on MTTR & Replication, and compatibility with 0.96 and 0.98. > Considering all these parameters, we propose a new HLog implementation with > WAL Switching functionality. > Please find attached the design doc for the same. It introduces the WAL > Switching feature, and experiments/results of a prototype implementation, > showing the benefits of this feature. > The second goal of this work is to serve as a building block for concurrent > multiple WALs feature. > Please review the doc. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950126#comment-13950126
]
stack commented on HBASE-10853:
---
[~jxiang] Then I think the patch is wrong Jimmy. Instead I should move the
checkOpen outside of the try/finally?
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-10858) TestRegionRebalancing is failing
Enis Soztutar created HBASE-10858: - Summary: TestRegionRebalancing is failing Key: HBASE-10858 URL: https://issues.apache.org/jira/browse/HBASE-10858 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: hbase-10070 After HBASE-10620 and addendum1, the test is failing, and even with a fix it is flaky. We should fix the test to be reliable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950120#comment-13950120
]
Jimmy Xiang commented on HBASE-10853:
-
Before reporting to master, RS throws ServerNotRunningYetException for all RPC
calls. So it is not handling gets, right?
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10092) Move up on to log4j2
[ https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10092: -- Issue Type: Sub-task (was: Task) Parent: HBASE-10856 > Move up on to log4j2 > > > Key: HBASE-10092 > URL: https://issues.apache.org/jira/browse/HBASE-10092 > Project: HBase > Issue Type: Sub-task >Reporter: stack > Attachments: 10092.txt > > > Allows logging with less friction. See http://logging.apache.org/log4j/2.x/ > This rather radical transition can be done w/ minor change given they have an > adapter for apache's logging, the one we use. They also have and adapter for > slf4j so we likely can remove at least some of the 4 versions of this module > our dependencies make use of. > I made a start in attached patch but am currently stuck in maven dependency > resolve hell courtesy of our slf4j. Fixing will take some concentration and > a good net connection, an item I currently lack. Other TODOs are that will > need to fix our little log level setting jsp page -- will likely have to undo > our use of hadoop's tool here -- and the config system changes a little. > I will return to this project soon. Will bring numbers. > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-10857) clear_auths command gives exception on existing label and user
Ted Yu created HBASE-10857:
--
Summary: clear_auths command gives exception on existing label and
user
Key: HBASE-10857
URL: https://issues.apache.org/jira/browse/HBASE-10857
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0
Reporter: Ted Yu
As user hbase, I performed the following:
{code}
hbase(main):001:0> set_auths 'oozie', [ 'TOP_SECRET' ]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2014-03-27 22:35:44,312 WARN [main] conf.Configuration: hbase-site.xml:an
attempt to override final parameter: dfs.support.append; Ignoring.
0 row(s) in 2.6000 seconds
hbase(main):002:0> scan 'hbase:labels'
ROW COLUMN+CELL
\x00\x00\x00\x01column=f:\x00,
timestamp=1395944796030, value=system
\x00\x00\x00\x01column=f:hbase,
timestamp=1395944796030, value=
\x00\x00\x00\x02column=f:\x00,
timestamp=1395951045442, value=TOP_SECRET
\x00\x00\x00\x02column=f:hrt_qa,
timestamp=1395951229682, value=
\x00\x00\x00\x02column=f:hrt_qa1,
timestamp=1395951270297, value=
\x00\x00\x00\x02column=f:mapred,
timestamp=1395958442326, value=
\x00\x00\x00\x02column=f:oozie,
timestamp=1395959745422, value=
\x00\x00\x00\x03column=f:\x00,
timestamp=1395952069731, value=TOP_TOP_SECRET
\x00\x00\x00\x03column=f:mapred,
timestamp=1395956032141, value=
3 row(s) in 0.0620 seconds
{code}
However, clear_auths command gave me:
{code}
hbase(main):003:0> clear_auths 'oozie', [ 'TOP_SECRET' ]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
ERROR: org.apache.hadoop.hbase.security.visibility.InvalidLabelException: Label
'TOP_SECRET' is not set for the user oozie
at
org.apache.hadoop.hbase.security.visibility.VisibilityController.clearAuths(VisibilityController.java:1304)
at
org.apache.hadoop.hbase.protobuf.generated.VisibilityLabelsProtos$VisibilityLabelsService$1.clearAuths(VisibilityLabelsProtos.java:5030)
at
org.apache.hadoop.hbase.protobuf.generated.VisibilityLabelsProtos$VisibilityLabelsService.callMethod(VisibilityLabelsProtos.java:5188)
at
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:5518)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3299)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28865)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10855) Enable hfilev3 by default
[ https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10855: -- Assignee: stack Status: Patch Available (was: Open) > Enable hfilev3 by default > - > > Key: HBASE-10855 > URL: https://issues.apache.org/jira/browse/HBASE-10855 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.0 > > Attachments: 10855.txt > > > Distributed log replay needs this. Should be on by default in 1.0/0.99. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10855) Enable hfilev3 by default
[ https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10855: -- Attachment: 10855.txt Just enable hfilev3. Lets see how it does on hadoopqa. > Enable hfilev3 by default > - > > Key: HBASE-10855 > URL: https://issues.apache.org/jira/browse/HBASE-10855 > Project: HBase > Issue Type: Sub-task >Reporter: stack > Fix For: 0.99.0 > > Attachments: 10855.txt > > > Distributed log replay needs this. Should be on by default in 1.0/0.99. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10855) Enable hfilev3 by default
[ https://issues.apache.org/jira/browse/HBASE-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10855: -- Issue Type: Sub-task (was: Task) Parent: HBASE-10856 > Enable hfilev3 by default > - > > Key: HBASE-10855 > URL: https://issues.apache.org/jira/browse/HBASE-10855 > Project: HBase > Issue Type: Sub-task >Reporter: stack > Fix For: 0.99.0 > > > Distributed log replay needs this. Should be on by default in 1.0/0.99. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10796) Set default log level as INFO
[ https://issues.apache.org/jira/browse/HBASE-10796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10796: -- Resolution: Fixed Release Note: Set default log level as INFO. Previous we shipped with log level of DEBUG. Hadoop Flags: Incompatible change Status: Resolved (was: Patch Available) Committed to trunk. I got a bunch of +1s above...not for patch explicitly for the idea but I am going to use them committing this... it is a boring patch to review and it you have to see it in action to get a sense of whether it 'works' or not. > Set default log level as INFO > - > > Key: HBASE-10796 > URL: https://issues.apache.org/jira/browse/HBASE-10796 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.0 > > Attachments: 10796.txt, 10796v2.txt > > > When we roll out 1.0, the log level should be INFO-level by default, not > DEBUG. > Proposed on mailing list here > http://search-hadoop.com/m/33P7E1GL08b/hbase+1.0&subj=DISCUSSION+1+0+0 and at > least one other +1 with no objection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10796) Set default log level as INFO
[ https://issues.apache.org/jira/browse/HBASE-10796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10796: -- Issue Type: Sub-task (was: Task) Parent: HBASE-10856 > Set default log level as INFO > - > > Key: HBASE-10796 > URL: https://issues.apache.org/jira/browse/HBASE-10796 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 0.99.0 > > Attachments: 10796.txt, 10796v2.txt > > > When we roll out 1.0, the log level should be INFO-level by default, not > DEBUG. > Proposed on mailing list here > http://search-hadoop.com/m/33P7E1GL08b/hbase+1.0&subj=DISCUSSION+1+0+0 and at > least one other +1 with no objection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950086#comment-13950086
]
stack commented on HBASE-10853:
---
[~jxiang] Should we be handling gets before we report to master?
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (HBASE-10856) Prep for 1.0
[ https://issues.apache.org/jira/browse/HBASE-10856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950083#comment-13950083 ] stack edited comment on HBASE-10856 at 3/27/14 11:04 PM: - Here is the thread: http://search-hadoop.com/m/33P7E1GL08b Here are items. I'll file issues for them: * Update included libs (e.g. move to log4j2) * Enable distributed log replay as default (fix bugs) * Enable hfilev3 as default. * Ship with default logging level set to INFO and content of the logs still makes sense * Enable dynamic config and schema by default. * Maybe make the hconnection output a bit less verbose when ZK isn't live or HBase still coming up; it does generate a fair amount of log data * Light pastel purple colour scheme. * @Deprecated APIs cleaned up. Delete stale methods or remove the annotations. * Fix interface audience annotations (HBASE-10462). * Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions) * Promote HTableInterface vs HTable, getting connections from HCM and getting tables there. * A clearly defined compatibility strategy between patch, minor, and major versions * Known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. * (wishes for ponies and unicorns, probably not going to happen) better ops supports, such as sample puppet/chef/etc scripts * HBASE-4047. (Maybe that makes the 1.0 list?) * Paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins... lots of discussion here including ..."I think the best place to start is by breaking up some of the current APIs, grouping them around behaviors or areas of functionality. " * In HRegion extract the public interface out into a Interface. (similar to Store and HStore) * HBASE-8763 Combine MVCC and SeqId. Currently replication is broken on same version updates in the following scenarios: when a region move or RS get restarted in the source cluster, replicated changes may come out of order to the receiving RS. There are other cases we need to keep the ordering of puts. This JIRA can be used to fix those above issues. * Replicated Meta Regions for Distributed Region Location Lookup(I haven't cut a JIRA for it but I've been thinking this for a while) * "We need a Replication Interface defined" * HBASE-9206 'namespace permissions' * c-client was (Author: stack): Here is the thread: http://search-hadoop.com/m/33P7E1GL08b Here are items. I'll file issues for them: * Update included libs (e.g. move to log4j2) * Enable distributed log replay as default (fix bugs) * Enable hfilev3 as default. * Ship with default logging level set to INFO and content of the logs still makes sense * Enable dynamic config and schema by default. * Maybe make the hconnection output a bit less verbose when ZK isn't live or HBase still coming up; it does generate a fair amount of log data * Light pastel purple colour scheme. * @Deprecated APIs cleaned up. Delete stale methods or remove the annotations. + Fix interface audience annotations (HBASE-10462). + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions) + Promote HTableInterface vs HTable, getting connections from HCM and getting tables there. + A clearly defined compatibility strategy between patch, minor, and major versions + Known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. + (wishes for ponies and unicorns, probably not going to happen) better ops supports, such as sample puppet/chef/etc scripts + HBASE-4047. (Maybe that makes the 1.0 list?) + Paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins... lots of discussion here including ..."I think the best place to start is by breaking up some of the current APIs, grouping them around behaviors or areas of functionality. " + In HRegion extract the public interface out into a Interface. (similar to Store and HStore) + HBASE-8763 Combine MVCC and SeqId. Currently replication is broken on same version updates in the following scenarios: when a region move or RS get restarted in the source cluster, replicated changes may come out of order to the receiving RS. There are other cases we need to keep the ordering of puts. This JIRA can be used to fix those above issues. + Replicated Meta Regions for Distributed Region Location Lookup(I haven't cut a JIRA for it but I've been thinking this for a while) + "We need a Replication Interface defined" + HBASE-9206 'namespace permissions' > Prep for 1.0 > > > Key: HBASE-10856 > URL: https://issues.apache.org/jira/browse/HBASE-10856 >
[jira] [Commented] (HBASE-10856) Prep for 1.0
[ https://issues.apache.org/jira/browse/HBASE-10856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950083#comment-13950083 ] stack commented on HBASE-10856: --- Here is the thread: http://search-hadoop.com/m/33P7E1GL08b Here are items. I'll file issues for them: * Update included libs (e.g. move to log4j2) * Enable distributed log replay as default (fix bugs) * Enable hfilev3 as default. * Ship with default logging level set to INFO and content of the logs still makes sense * Enable dynamic config and schema by default. * Maybe make the hconnection output a bit less verbose when ZK isn't live or HBase still coming up; it does generate a fair amount of log data * Light pastel purple colour scheme. * @Deprecated APIs cleaned up. Delete stale methods or remove the annotations. + Fix interface audience annotations (HBASE-10462). + Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions) + Promote HTableInterface vs HTable, getting connections from HCM and getting tables there. + A clearly defined compatibility strategy between patch, minor, and major versions + Known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. + (wishes for ponies and unicorns, probably not going to happen) better ops supports, such as sample puppet/chef/etc scripts + HBASE-4047. (Maybe that makes the 1.0 list?) + Paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins... lots of discussion here including ..."I think the best place to start is by breaking up some of the current APIs, grouping them around behaviors or areas of functionality. " + In HRegion extract the public interface out into a Interface. (similar to Store and HStore) + HBASE-8763 Combine MVCC and SeqId. Currently replication is broken on same version updates in the following scenarios: when a region move or RS get restarted in the source cluster, replicated changes may come out of order to the receiving RS. There are other cases we need to keep the ordering of puts. This JIRA can be used to fix those above issues. + Replicated Meta Regions for Distributed Region Location Lookup(I haven't cut a JIRA for it but I've been thinking this for a while) + "We need a Replication Interface defined" + HBASE-9206 'namespace permissions' > Prep for 1.0 > > > Key: HBASE-10856 > URL: https://issues.apache.org/jira/browse/HBASE-10856 > Project: HBase > Issue Type: Umbrella >Reporter: stack > > Tasks for 1.0 copied here from our '1.0.0' mailing list discussion. Idea is > to file subtasks off this one. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-10856) Prep for 1.0
stack created HBASE-10856: - Summary: Prep for 1.0 Key: HBASE-10856 URL: https://issues.apache.org/jira/browse/HBASE-10856 Project: HBase Issue Type: Umbrella Reporter: stack Tasks for 1.0 copied here from our '1.0.0' mailing list discussion. Idea is to file subtasks off this one. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10796) Set default log level as INFO
[
https://issues.apache.org/jira/browse/HBASE-10796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950044#comment-13950044
]
Hadoop QA commented on HBASE-10796:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637239/10796v2.txt
against trunk revision .
ATTACHMENT ID: 12637239
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9117//console
This message is automatically generated.
> Set default log level as INFO
> -
>
> Key: HBASE-10796
> URL: https://issues.apache.org/jira/browse/HBASE-10796
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 0.99.0
>
> Attachments: 10796.txt, 10796v2.txt
>
>
> When we roll out 1.0, the log level should be INFO-level by default, not
> DEBUG.
> Proposed on mailing list here
> http://search-hadoop.com/m/33P7E1GL08b/hbase+1.0&subj=DISCUSSION+1+0+0 and at
> least one other +1 with no objection.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-10855) Enable hfilev3 by default
stack created HBASE-10855: - Summary: Enable hfilev3 by default Key: HBASE-10855 URL: https://issues.apache.org/jira/browse/HBASE-10855 Project: HBase Issue Type: Task Reporter: stack Fix For: 0.99.0 Distributed log replay needs this. Should be on by default in 1.0/0.99. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8889) TestIOFencing#testFencingAroundCompaction occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-8889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949992#comment-13949992
]
Ted Yu commented on HBASE-8889:
---
TestIOFencing passed in QA runs and trunk builds.
Plan to resolve tomorrow if it doesn't fail.
> TestIOFencing#testFencingAroundCompaction occasionally fails
>
>
> Key: HBASE-8889
> URL: https://issues.apache.org/jira/browse/HBASE-8889
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Blocker
> Fix For: 1.0.0
>
> Attachments: 8889-v1.txt, TestIOFencing-#8362.tar.gz,
> TestIOFencing.tar.gz
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/6232//testReport/org.apache.hadoop.hbase/TestIOFencing/testFencingAroundCompaction/
> :
> {code}
> java.lang.AssertionError: Timed out waiting for new server to open region
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.apache.hadoop.hbase.TestIOFencing.doTest(TestIOFencing.java:269)
> at
> org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompaction(TestIOFencing.java:205)
> {code}
> {code}
> 2013-07-06 23:13:53,120 INFO [pool-1-thread-1] hbase.TestIOFencing(266):
> Waiting for the new server to pick up the region
> tabletest,,1373152125442.6e62d3b24ea23160931362b60359ff03.
> 2013-07-06 23:13:54,120 INFO [pool-1-thread-1] hbase.TestIOFencing(266):
> Waiting for the new server to pick up the region
> tabletest,,1373152125442.6e62d3b24ea23160931362b60359ff03.
> 2013-07-06 23:13:55,121 DEBUG [pool-1-thread-1]
> hbase.TestIOFencing$CompactionBlockerRegion(102): allowing compactions
> 2013-07-06 23:13:55,121 INFO [pool-1-thread-1]
> hbase.HBaseTestingUtility(911): Shutting down minicluster
> 2013-07-06 23:13:55,121 DEBUG [pool-1-thread-1] util.JVMClusterUtil(237):
> Shutting down HBase Cluster
> 2013-07-06 23:13:55,121 INFO
> [RS:0;asf002:39065-smallCompactions-1373152134716] regionserver.HStore(951):
> Starting compaction of 2 file(s) in family of
> tabletest,,1373152125442.6e62d3b24ea23160931362b60359ff03. into
> tmpdir=hdfs://localhost:50140/user/jenkins/hbase/tabletest/6e62d3b24ea23160931362b60359ff03/.tmp,
> totalSize=108.4k
> ...
> 2013-07-06 23:13:55,155 INFO [RS:0;asf002:39065]
> regionserver.HRegionServer(2476): Received CLOSE for the region:
> 6e62d3b24ea23160931362b60359ff03 ,which we are already trying to CLOSE
> 2013-07-06 23:13:55,157 WARN [RS:0;asf002:39065]
> regionserver.HRegionServer(2414): Failed to close
> tabletest,,1373152125442.6e62d3b24ea23160931362b60359ff03. - ignoring and
> continuing
> org.apache.hadoop.hbase.exceptions.NotServingRegionException: The region
> 6e62d3b24ea23160931362b60359ff03 was already closing. New CLOSE request is
> ignored.
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2479)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegionIgnoreErrors(HRegionServer.java:2409)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeUserRegions(HRegionServer.java:2011)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:903)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:158)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:110)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:142)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:337)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1131)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.util.Methods.call(Methods.java:41)
> at org.apache.hadoop.hbase.security.User.call(User.java:420)
> at org.apache.hadoop.hbase.security.User.access$300(User.java:51)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:260)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:140)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-10852:
---
Resolution: Fixed
Fix Version/s: 0.98.2
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review, Nick.
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0, 0.98.2
>
> Attachments: 10852-v1.txt, 10852-v2.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949980#comment-13949980
]
Enis Soztutar commented on HBASE-10852:
---
+1
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: 10852-v1.txt, 10852-v2.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949975#comment-13949975
]
Nick Dimiduk commented on HBASE-10852:
--
Sounds good... +1
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: 10852-v1.txt, 10852-v2.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10854) Multiple Row/VisibilityLabels visible while in the memstore
[
https://issues.apache.org/jira/browse/HBASE-10854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949967#comment-13949967
]
Andrew Purtell commented on HBASE-10854:
I think there are two issues here. First, the behavior for multiple versions
with multiple visibility expressions should be consistent between memstore and
store scanning. Second, we should agree on what is the correct behavior for
schemas supporting multiple versions, with multiple cell versions with
differing visibility expressions among the versions.
> Multiple Row/VisibilityLabels visible while in the memstore
> ---
>
> Key: HBASE-10854
> URL: https://issues.apache.org/jira/browse/HBASE-10854
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.98.1
>Reporter: Matteo Bertozzi
>
> If we update the row multiple times with different visibility labels
> we are able to get the "old version" of the row until is flushed
> {code}
> $ sudo -u hbase hbase shell
> hbase> add_labels 'A'
> hbase> add_labels 'B'
> hbase> create 'tb', 'f1'
> hbase> put 'tb', 'row', 'f1:q', 'v1', {VISIBILITY=>'A'}
> hbase> put 'tb', 'row', 'f1:q', 'v1all'
> hbase> put 'tb', 'row', 'f1:q', 'v1aOrB', {VISIBILITY=>'A|B'}
> hbase> put 'tb', 'row', 'f1:q', 'v1aAndB', {VISIBILITY=>'A&B'}
> hbase> scan 'tb'
> row column=f1:q, timestamp=1395948168154, value=v1aAndB
> 1 row
> $ sudo -u testuser hbase shell
> hbase> scan 'tb'
> row column=f1:q, timestamp=1395948168102, value=v1all
> 1 row
> {code}
> When we flush the memstore we get a single row (the last one inserted)
> so the testuser get 0 rows now.
> {code}
> $ sudo -u hbase hbase shell
> hbase> flush 'tb'
> hbase> scan 'tb'
> row column=f1:q, timestamp=1395948168154, value=v1aAndB
> 1 row
> $ sudo -u testuser hbase shell
> hbase> scan 'tb'
> 0 row
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-10852:
---
Attachment: 10852-v2.txt
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: 10852-v1.txt, 10852-v2.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949954#comment-13949954
]
Ted Yu commented on HBASE-10852:
How about 3 retries ?
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: 10852-v1.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949951#comment-13949951
]
Jimmy Xiang commented on HBASE-10853:
-
I think it's the right approach. This should happen before the RS reports to
the master.
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949949#comment-13949949
]
Nick Dimiduk commented on HBASE-10852:
--
Simple enough, +1. Is 2 retries enough to weather a RIT? Make it 5?
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: 10852-v1.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10850) Unexpected behavior when using filter SingleColumnValueFilter
[
https://issues.apache.org/jira/browse/HBASE-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949946#comment-13949946
]
Ted Yu commented on HBASE-10850:
@Fabien:
I slightly modified attached test so that it compiles in trunk.
Since there is no cluster started, there was no progress when running the test.
Can you look at tests in
hbase-server/src/test/java/org/apache/hadoop/hbase/filter ?
Thanks
> Unexpected behavior when using filter SingleColumnValueFilter
> -
>
> Key: HBASE-10850
> URL: https://issues.apache.org/jira/browse/HBASE-10850
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.96.1.1
>Reporter: Fabien Le Gallo
>Assignee: haosdent
> Attachments: HBASE-10850-96.patch, HBASE-10850.patch,
> HBaseSingleColumnValueFilterTest.java
>
>
> When using the filter SingleColumnValueFilter, and depending of the columns
> specified in the scan (filtering column always specified), the results can be
> different.
> Here is an example.
> Suppose the following table:
> ||key||a:foo||a:bar||b:foo||b:bar||
> |1|false|_flag_|_flag_|_flag_|
> |2|true|_flag_|_flag_|_flag_|
> |3| |_flag_|_flag_|_flag_|
> With this filter:
> {code}
> SingleColumnValueFilter filter = new
> SingleColumnValueFilter(Bytes.toBytes("a"), Bytes.toBytes("foo"),
> CompareOp.EQUAL, new BinaryComparator(Bytes.toBytes("false")));
> filter.setFilterIfMissing(true);
> {code}
> Depending of how I specify the list of columns to add in the scan, the result
> is different. Yet, all examples below should always return only the first row
> (key '1'):
> OK:
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addFamily(Bytes.toBytes("a"));
> scan.addFamily(Bytes.toBytes("b"));
> {code}
> KO (2 results returned, row '3' without 'a:foo' qualifier is returned):
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("foo"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("b"), Bytes.toBytes("bar"));
> {code}
> OK:
> {code}
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("foo"));
> scan.addColumn(Bytes.toBytes("a"), Bytes.toBytes("bar"));
> {code}
> This is a regression as it was working properly on HBase 0.92.
> You will find in attachement the unit tests reproducing the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949940#comment-13949940
]
Jimmy Xiang commented on HBASE-10853:
-
+1
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10796) Set default log level as INFO
[ https://issues.apache.org/jira/browse/HBASE-10796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10796: -- Fix Version/s: 0.99.0 > Set default log level as INFO > - > > Key: HBASE-10796 > URL: https://issues.apache.org/jira/browse/HBASE-10796 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.99.0 > > Attachments: 10796.txt, 10796v2.txt > > > When we roll out 1.0, the log level should be INFO-level by default, not > DEBUG. > Proposed on mailing list here > http://search-hadoop.com/m/33P7E1GL08b/hbase+1.0&subj=DISCUSSION+1+0+0 and at > least one other +1 with no objection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10796) Set default log level as INFO
[ https://issues.apache.org/jira/browse/HBASE-10796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10796: -- Attachment: 10796v2.txt I'd like to commit this. It result of my comparing logs w/ DEBUG enabled and then disabled. I know that our INFO level logging is probably missing a few DEBUGs so can tell decent story on what transpired and then some INFOs should be DEBUG but this should do as a first cut. Can add new issues for other changes as folks find they need them. > Set default log level as INFO > - > > Key: HBASE-10796 > URL: https://issues.apache.org/jira/browse/HBASE-10796 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.99.0 > > Attachments: 10796.txt, 10796v2.txt > > > When we roll out 1.0, the log level should be INFO-level by default, not > DEBUG. > Proposed on mailing list here > http://search-hadoop.com/m/33P7E1GL08b/hbase+1.0&subj=DISCUSSION+1+0+0 and at > least one other +1 with no objection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10854) Multiple Row/VisibilityLabels visible while in the memstore
[
https://issues.apache.org/jira/browse/HBASE-10854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-10854:
Description:
If we update the row multiple times with different visibility labels
we are able to get the "old version" of the row until is flushed
{code}
$ sudo -u hbase hbase shell
hbase> add_labels 'A'
hbase> add_labels 'B'
hbase> create 'tb', 'f1'
hbase> put 'tb', 'row', 'f1:q', 'v1', {VISIBILITY=>'A'}
hbase> put 'tb', 'row', 'f1:q', 'v1all'
hbase> put 'tb', 'row', 'f1:q', 'v1aOrB', {VISIBILITY=>'A|B'}
hbase> put 'tb', 'row', 'f1:q', 'v1aAndB', {VISIBILITY=>'A&B'}
hbase> scan 'tb'
row column=f1:q, timestamp=1395948168154, value=v1aAndB
1 row
$ sudo -u testuser hbase shell
hbase> scan 'tb'
row column=f1:q, timestamp=1395948168102, value=v1all
1 row
{code}
When we flush the memstore we get a single row (the last one inserted)
so the testuser get 0 rows now.
{code}
$ sudo -u hbase hbase shell
hbase> flush 'tb'
hbase> scan 'tb'
row column=f1:q, timestamp=1395948168154, value=v1aAndB
1 row
$ sudo -u testuser hbase shell
hbase> scan 'tb'
0 row
{code}
was:
If we update the row multiple times with different visibility labels
we are able to get the "old version" of the row until is flushed
{code}
$ sudo -u hbase hbase shell
hbase> add_labels 'A'
hbase> add_labels 'B'
hbase> create 'tb', 'f1'
hbase> put 'tb', 'row', 'f1:q', 'v1', {VISIBILITY=>'A'}
hbase> put 'tb', 'row', 'f1:q', 'v1all'
hbase> put 'tb', 'row', 'f1:q', 'v1aOrB', {VISIBILITY=>'A|B'}
hbase> put 'tb', 'row', 'f1:q', 'v1aAndB', {VISIBILITY=>'A&B'}
hbase> scan 'tb'
row column=f1:q, timestamp=1395948168154, value=v1aAndB
1 row
$ sudo -u hbase hbase shell
hbase> scan 'tb'
row column=f1:q, timestamp=1395948168102, value=v1all
1 row
{code}
When we flush the memstore we get a single row (the last one inserted)
so the testuser get 0 rows now.
{code}
$ sudo -u hbase hbase shell
hbase> flush 'tb'
hbase> scan 'tb'
row column=f1:q, timestamp=1395948168154, value=v1aAndB
1 row
$ sudo -u hbase hbase shell
hbase> scan 'tb'
0 row
{code}
> Multiple Row/VisibilityLabels visible while in the memstore
> ---
>
> Key: HBASE-10854
> URL: https://issues.apache.org/jira/browse/HBASE-10854
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.98.1
>Reporter: Matteo Bertozzi
>
> If we update the row multiple times with different visibility labels
> we are able to get the "old version" of the row until is flushed
> {code}
> $ sudo -u hbase hbase shell
> hbase> add_labels 'A'
> hbase> add_labels 'B'
> hbase> create 'tb', 'f1'
> hbase> put 'tb', 'row', 'f1:q', 'v1', {VISIBILITY=>'A'}
> hbase> put 'tb', 'row', 'f1:q', 'v1all'
> hbase> put 'tb', 'row', 'f1:q', 'v1aOrB', {VISIBILITY=>'A|B'}
> hbase> put 'tb', 'row', 'f1:q', 'v1aAndB', {VISIBILITY=>'A&B'}
> hbase> scan 'tb'
> row column=f1:q, timestamp=1395948168154, value=v1aAndB
> 1 row
> $ sudo -u testuser hbase shell
> hbase> scan 'tb'
> row column=f1:q, timestamp=1395948168102, value=v1all
> 1 row
> {code}
> When we flush the memstore we get a single row (the last one inserted)
> so the testuser get 0 rows now.
> {code}
> $ sudo -u hbase hbase shell
> hbase> flush 'tb'
> hbase> scan 'tb'
> row column=f1:q, timestamp=1395948168154, value=v1aAndB
> 1 row
> $ sudo -u testuser hbase shell
> hbase> scan 'tb'
> 0 row
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-10854) Multiple Row/VisibilityLabels visible while in the memstore
Matteo Bertozzi created HBASE-10854:
---
Summary: Multiple Row/VisibilityLabels visible while in the
memstore
Key: HBASE-10854
URL: https://issues.apache.org/jira/browse/HBASE-10854
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.98.1
Reporter: Matteo Bertozzi
If we update the row multiple times with different visibility labels
we are able to get the "old version" of the row until is flushed
{code}
$ sudo -u hbase hbase shell
hbase> add_labels 'A'
hbase> add_labels 'B'
hbase> create 'tb', 'f1'
hbase> put 'tb', 'row', 'f1:q', 'v1', {VISIBILITY=>'A'}
hbase> put 'tb', 'row', 'f1:q', 'v1all'
hbase> put 'tb', 'row', 'f1:q', 'v1aOrB', {VISIBILITY=>'A|B'}
hbase> put 'tb', 'row', 'f1:q', 'v1aAndB', {VISIBILITY=>'A&B'}
hbase> scan 'tb'
row column=f1:q, timestamp=1395948168154, value=v1aAndB
1 row
$ sudo -u hbase hbase shell
hbase> scan 'tb'
row column=f1:q, timestamp=1395948168102, value=v1all
1 row
{code}
When we flush the memstore we get a single row (the last one inserted)
so the testuser get 0 rows now.
{code}
$ sudo -u hbase hbase shell
hbase> flush 'tb'
hbase> scan 'tb'
row column=f1:q, timestamp=1395948168154, value=v1aAndB
1 row
$ sudo -u hbase hbase shell
hbase> scan 'tb'
0 row
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949899#comment-13949899
]
Hadoop QA commented on HBASE-10852:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637205/10852-v1.txt
against trunk revision .
ATTACHMENT ID: 12637205
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationSmallTests
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestTableMapReduceBase.testMultiRegionTable(TestTableMapReduceBase.java:96)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9116//console
This message is automatically generated.
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 0.99.0
>
> Attachments: 10852-v1.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at
[jira] [Commented] (HBASE-10848) Filter SingleColumnValueFilter combined with NullComparator does not work
[
https://issues.apache.org/jira/browse/HBASE-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949853#comment-13949853
]
Ted Yu commented on HBASE-10848:
Can you take a look at TestSingleColumnValueFilter.java ?
Minicluster is not used in that test (and other Filter-related tests).
> Filter SingleColumnValueFilter combined with NullComparator does not work
> -
>
> Key: HBASE-10848
> URL: https://issues.apache.org/jira/browse/HBASE-10848
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.96.1.1
>Reporter: Fabien Le Gallo
> Attachments: HBASE-10848.patch, HBASE_10848-v2.patch,
> HBaseRegression.java, TestScanWithNullComparable.java
>
>
> I want to filter out from the scan the rows that does not have a specific
> column qualifier. For this purpose I use the filter SingleColumnValueFilter
> combined with the NullComparator.
> But every time I use this in a scan, I get the following exception:
> {code}
> java.lang.RuntimeException: org.apache.hadoop.hbase.DoNotRetryIOException:
> Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
> at
> org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:47)
> at
> com.xxx.xxx.test.HBaseRegression.nullComparator(HBaseRegression.java:92)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry
> of OutOfOrderScannerNextException: was there a rpc timeout?
> at
> org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:391)
> at
> org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:44)
> ... 25 more
> Caused by: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException:
> org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected
> nextCallSeq: 1 But the nextCallSeq got from client: 0; request=scanner_id:
> 7998309028985532303 number_of_rows: 100 close_scanner: false next_call_seq: 0
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3011)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26929)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175)
> at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1879)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.la
[jira] [Updated] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-10853:
--
Attachment: 10853.txt
Is this the right approach [~jxiang]? It looks like the metrics service comes
on board later on in the startup process we throw NPEs just during startup.
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
> Attachments: 10853.txt
>
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10853) NPE in RSRpcServices.get on trunk
[
https://issues.apache.org/jira/browse/HBASE-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949827#comment-13949827
]
stack commented on HBASE-10853:
---
regionServer.metricsRegionServer is null in the below:
{code}
1570 } else if (r != null) {
1571 ClientProtos.Result pbr = ProtobufUtil.toResult(r);
1572 builder.setResult(pbr);
1573 }
1574 return builder.build();
1575 } catch (IOException ie) {
1576 throw new ServiceException(ie);
1577 } finally {
1578 LOG.info("regionServer=" + regionServer + " " +
regionServer.metricsRegionServer);
1579 regionServer.metricsRegionServer.updateGet(
1580 EnvironmentEdgeManager.currentTimeMillis() - before);
1581 }
1582 }
{code}
> NPE in RSRpcServices.get on trunk
> -
>
> Key: HBASE-10853
> URL: https://issues.apache.org/jira/browse/HBASE-10853
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
>
> You seen this one [~jxiang]?
> Here is the exception:
> {code}
> 2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
> Unexpected throwable object
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Code looks like this on trunk:
> {code}
> 1576 } finally {
> 1577 regionServer.metricsRegionServer.updateGet(
> 1578 EnvironmentEdgeManager.currentTimeMillis() - before);
> 1579 }
> 1580 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10848) Filter SingleColumnValueFilter combined with NullComparator does not work
[
https://issues.apache.org/jira/browse/HBASE-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949820#comment-13949820
]
Hadoop QA commented on HBASE-10848:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637196/HBASE_10848-v2.patch
against trunk revision .
ATTACHMENT ID: 12637196
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9115//console
This message is automatically generated.
> Filter SingleColumnValueFilter combined with NullComparator does not work
> -
>
> Key: HBASE-10848
> URL: https://issues.apache.org/jira/browse/HBASE-10848
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.96.1.1
>Reporter: Fabien Le Gallo
> Attachments: HBASE-10848.patch, HBASE_10848-v2.patch,
> HBaseRegression.java, TestScanWithNullComparable.java
>
>
> I want to filter out from the scan the rows that does not have a specific
> column qualifier. For this purpose I use the filter SingleColumnValueFilter
> combined with the NullComparator.
> But every time I use this in a scan, I get the following exception:
> {code}
> java.lang.RuntimeException: org.apache.hadoop.hbase.DoNotRetryIOException:
> Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
> at
> org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:47)
> at
> com.xxx.xxx.test.HBaseRegression.nullComparator(HBaseRegression.java:92)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
> at
> org.junit.runn
[jira] [Commented] (HBASE-10788) Add 99th percentile of latency in PE
[ https://issues.apache.org/jira/browse/HBASE-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949817#comment-13949817 ] Nick Dimiduk commented on HBASE-10788: -- +1 > Add 99th percentile of latency in PE > > > Key: HBASE-10788 > URL: https://issues.apache.org/jira/browse/HBASE-10788 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.99.0 >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Attachments: HBASE-10788-trunk-v1.diff, HBASE-10788-trunk-v2.diff, > HBASE-10788-trunk-v3.diff > > > In production env, 99th percentile of latency is more important than the avg. > The 99th percentile is helpful to measure the influence of GC, slow > read/write of HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-10853) NPE in RSRpcServices.get on trunk
stack created HBASE-10853:
-
Summary: NPE in RSRpcServices.get on trunk
Key: HBASE-10853
URL: https://issues.apache.org/jira/browse/HBASE-10853
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Reporter: stack
You seen this one [~jxiang]?
Here is the exception:
{code}
2014-03-27 11:38:16,649 ERROR [RpcServer.handler=5,port=16020] ipc.RpcServer:
Unexpected throwable object
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1577)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29493)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2002)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:162)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
at java.lang.Thread.run(Thread.java:744)
{code}
Code looks like this on trunk:
{code}
1576 } finally {
1577 regionServer.metricsRegionServer.updateGet(
1578 EnvironmentEdgeManager.currentTimeMillis() - before);
1579 }
1580 }
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10848) Filter SingleColumnValueFilter combined with NullComparator does not work
[
https://issues.apache.org/jira/browse/HBASE-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949778#comment-13949778
]
Hadoop QA commented on HBASE-10848:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637167/HBASE-10848.patch
against trunk revision .
ATTACHMENT ID: 12637167
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface.testHBase3758(TestRegionObserverInterface.java:296)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9114//console
This message is automatically generated.
> Filter SingleColumnValueFilter combined with NullComparator does not work
> -
>
> Key: HBASE-10848
> URL: https://issues.apache.org/jira/browse/HBASE-10848
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.96.1.1
>Reporter: Fabien Le Gallo
> Attachments: HBASE-10848.patch, HBASE_10848-v2.patch,
> HBaseRegression.java, TestScanWithNullComparable.java
>
>
> I want to filter out from the scan the rows that does not have a specific
> column qualifier. For this purpose I use the filter SingleColumnValueFilter
> combined with the NullComparator.
> But every time I use this in a scan, I get the following exception:
> {code}
> java.lang.RuntimeException: org.apache.hadoop.hbase.DoNotRetryIOException:
> Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
> at
> org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:47)
> at
> com.xxx.xxx.test.HBaseRegression.nullComparator(HBaseRegression.java:92)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
[jira] [Updated] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-10852:
---
Status: Patch Available (was: Open)
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 10852-v1.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-10852:
---
Attachment: 10852-v1.txt
Increasing retry count so that client can withstand hbase:meta movement.
> TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
> -
>
> Key: HBASE-10852
> URL: https://issues.apache.org/jira/browse/HBASE-10852
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.99.0
>
> Attachments: 10852-v1.txt
>
>
> Here was the failure:
> {code}
> java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
> returned=[org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:722)
> ]
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
> {code}
> Here was the cause:
> {code}
> 2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
> handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeDeleted, state=SyncConnected,
> path=/hbase/region-in-transition/1588230740
> 2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
> zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
> type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/region-in-transition
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
> quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740
> in expected state RS_ZK_REGION_OPENED
> 2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
> master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OPEN, ts=1395905941397,
> server=c64-s12.cs1cloud.internal,44281,1395905929927}
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(413): Onlined 1588230740 on
> c64-s12.cs1cloud.internal,44281,1395905929927
> 2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
> master.RegionStates(417): Offlined 1588230740 from
> c64-s12.cs1cloud.internal,52861,1395905929889
> 2014-03-27 00:39:01,547 WARN [Thread-2811]
> client.ConnectionManager$HConnectionImplementation(1221): Encountered
> problems when prefetch hbase:meta table:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=1, exceptions:
> Thu Mar 27 00:39:01 PDT 2014,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
> org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on c64-s12.cs1cloud.internal,52861,1395905929889
> {code}
> hbase:meta was moving but client didn't retry (attempts=1).
> Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-10852) TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
Ted Yu created HBASE-10852:
--
Summary:
TestDistributedLogSplitting#testDisallowWritesInRecovering occasionally fails
Key: HBASE-10852
URL: https://issues.apache.org/jira/browse/HBASE-10852
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10852-v1.txt
Here was the failure:
{code}
java.lang.AssertionError: No RegionInRecoveryException. Following exceptions
returned=[org.apache.hadoop.hbase.NotServingRegionException:
org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is not
online on c64-s12.cs1cloud.internal,52861,1395905929889
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2676)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4095)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2826)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28857)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
at java.lang.Thread.run(Thread.java:722)
]
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testDisallowWritesInRecovering(TestDistributedLogSplitting.java:924)
{code}
Here was the cause:
{code}
2014-03-27 00:39:01,398 DEBUG [RS_OPEN_META-c64-s12:44281-0]
handler.OpenRegionHandler(179): Opened hbase:meta,,1.1588230740 on
c64-s12.cs1cloud.internal,44281,1395905929927
2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
type=NodeDeleted, state=SyncConnected,
path=/hbase/region-in-transition/1588230740
2014-03-27 00:39:01,405 DEBUG [Thread-2811-EventThread]
zookeeper.ZooKeeperWatcher(310): master:32796-0x1450278b68f01cc,
quorum=localhost:50923, baseZNode=/hbase Received ZooKeeper Event,
type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition
2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
zookeeper.ZKAssign(480): master:32796-0x1450278b68f01cc,
quorum=localhost:50923, baseZNode=/hbase Deleted unassigned node 1588230740 in
expected state RS_ZK_REGION_OPENED
2014-03-27 00:39:01,406 DEBUG [AM.ZK.Worker-pool1213-t19]
master.AssignmentManager$4(1186): Znode hbase:meta,,1.1588230740 deleted,
state: {1588230740 state=OPEN, ts=1395905941397,
server=c64-s12.cs1cloud.internal,44281,1395905929927}
2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
master.RegionStates(413): Onlined 1588230740 on
c64-s12.cs1cloud.internal,44281,1395905929927
2014-03-27 00:39:01,406 INFO [AM.ZK.Worker-pool1213-t19]
master.RegionStates(417): Offlined 1588230740 from
c64-s12.cs1cloud.internal,52861,1395905929889
2014-03-27 00:39:01,547 WARN [Thread-2811]
client.ConnectionManager$HConnectionImplementation(1221): Encountered problems
when prefetch hbase:meta table:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=1, exceptions:
Thu Mar 27 00:39:01 PDT 2014,
org.apache.hadoop.hbase.client.RpcRetryingCaller@23136717,
org.apache.hadoop.hbase.NotServingRegionException:
org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is not
online on c64-s12.cs1cloud.internal,52861,1395905929889
{code}
hbase:meta was moving but client didn't retry (attempts=1).
Thanks to Jeff who helped identify the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
