[jira] [Commented] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[
https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156176#comment-14156176
]
Hadoop QA commented on HBASE-12148:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672500/12148v2.txt
against trunk revision .
ATTACHMENT ID: 12672500
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11185//console
This message is automatically generated.
> Remove TimeRangeTracker as point of contention when many threads writing a
> Store
>
>
> Key: HBASE-12148
> URL: https://issues.apache.org/jira/browse/HBASE-12148
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 2.0.0, 0.99.1
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 12148.txt, 12148.txt, 12148v2.txt, Screen Shot
> 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12149) TestRegionPlacement is failing undeterministically
[
https://issues.apache.org/jira/browse/HBASE-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156191#comment-14156191
]
Hadoop QA commented on HBASE-12149:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672501/0001-Split-TestRegionPlacement.patch
against trunk revision .
ATTACHMENT ID: 12672501
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11186//console
This message is automatically generated.
> TestRegionPlacement is failing undeterministically
> --
>
> Key: HBASE-12149
> URL: https://issues.apache.org/jira/browse/HBASE-12149
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: 0001-Split-TestRegionPlacement.patch,
> 0001-Split-TestRegionPlacement.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> TestRegionPlacement has tests that are flaky and fail in a cascading fashion.
> We need to fix this cascading behavior and also fix the flakyness of the
> tests.
> 16:20:59 java.lang.AssertionError: Region {ENCODED =>
> 958cbbb4405e3bbcfd04185538912d4c, NAME =>
> 'testRegionAssignment,\x02\x02\x02,1412205636586.958cbbb4405e3bbcfd04185538912d4c.',
> STARTKEY => '\x02\x02\x02', ENDKEY => '\x03\x03\x03'} not present on any of
> the expected servers
> 16:20:59 at org.junit.Assert.fail(Assert.java:88)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.killRandomServerAndVerifyAssignment(TestRegionPlacement.java:347)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testRegionPlacement(TestRegionPlacement
[jira] [Commented] (HBASE-12153) Fixing TestReplicaWithCluster
[ https://issues.apache.org/jira/browse/HBASE-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156252#comment-14156252 ] Nicolas Liochon commented on HBASE-12153: - Yeah, I actually don't like much timeouts in tests because they have do be removed during debugging sessions (the test is from me, but the timeouts are from Stack ;-) ). It's a workaround for surefire zombies... +1 for the patch. > Fixing TestReplicaWithCluster > - > > Key: HBASE-12153 > URL: https://issues.apache.org/jira/browse/HBASE-12153 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 1.0.0 > > Attachments: 0001-FixTestReplicaWithCluster.patch > > > This test takes about 30 ~ 40 seconds depending upon the resources available. > Doesn't make sense to have such a tight bound(30s) on the unit test. > [~nkeywal], what do you think ? Did you intend to have such a tight bound > while adding the test here ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11835) Wrong managenement of non expected calls in the client
[ https://issues.apache.org/jira/browse/HBASE-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156258#comment-14156258 ] Nicolas Liochon commented on HBASE-11835: - it got lost somewhere in a todo list. Let me have a look again. > Wrong managenement of non expected calls in the client > -- > > Key: HBASE-11835 > URL: https://issues.apache.org/jira/browse/HBASE-11835 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.0.0, 2.0.0, 0.98.6 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: rpcClient.patch > > > If a call is purged or canceled we try to skip the reply from the server, but > we read the wrong number of bytes so we corrupt the tcp channel. It's hidden > as it triggers retry and so on, but it's bad for performances obviously. > It happens with cell blocks. > [~ram_krish_86], [~saint@gmail.com], you know this part better than me, > do you agree with the analysis and the patch? > The changes in rpcServer are not fully related: as the client close the > connections in such situation, I observed both ClosedChannelException and > CancelledKeyException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12141) ClusterStatus message might exceed max datagram payload limits
[
https://issues.apache.org/jira/browse/HBASE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156267#comment-14156267
]
Nicolas Liochon commented on HBASE-12141:
-
Yeah, the strategy was to keep the message small enough (if multiple servers
fail simultaneously, we send multiple messages instead of one). As well, we
send the message multiple times in case it got lost somewhere. I had issue with
Netty 3.x when tried to add frames. I haven't tried very hard. We could make
MAX_SERVER_PER_MESSAGE configurable for network with a very small mtu? It's
also possible to compress the message. Once again, I had issue with Netty 3.x
for this in the past.
This said, I would be interested to understand the network config.
> ClusterStatus message might exceed max datagram payload limits
> --
>
> Key: HBASE-12141
> URL: https://issues.apache.org/jira/browse/HBASE-12141
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.3
>Reporter: Andrew Purtell
>
> The multicast ClusterStatusPublisher and its companion listener are using
> datagram channels without any framing. I think this is an issue because
> Netty's ProtobufDecoder expects a complete PB message to be available in the
> ChannelBuffer yet ClusterStatus messages can be large and might exceed the
> maximum datagram payload size. As one user reported on list:
> {noformat}
> org.apache.hadoop.hbase.client.ClusterStatusListener - ERROR - Unexpected
> exception, continuing.
> com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
> invalid wire type.
> at
> com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:99)
> at
> com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:498)
> at
> com.google.protobuf.GeneratedMessage.parseUnknownField(GeneratedMessage.java:193)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus.(ClusterStatusProtos.java:7554)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus.(ClusterStatusProtos.java:7512)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus$1.parsePartialFrom(ClusterStatusProtos.java:7689)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus$1.parsePartialFrom(ClusterStatusProtos.java:7684)
> at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:141)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:176)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:182)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.jboss.netty.handler.codec.protobuf.ProtobufDecoder.decode(ProtobufDecoder.java:122)
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.socket.oio.OioDatagramWorker.process(OioDatagramWorker.java:52)
> at
> org.jboss.netty.channel.socket.oio.AbstractOioWorker.run(AbstractOioWorker.java:73)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The javadoc for ProtobufDecoder says:
> {quote}
> Decodes a received ChannelBuffer into a Google Protocol Buffers Message and
> MessageLite. Please note that this decoder must be used with a proper
> FrameDecoder such as ProtobufVarint32FrameDecoder or
> LengthFieldBasedFrameDecoder if you are using a stream-based transport such
> as TCP/IP.
> {quote}
> and even though we are using a datagram transport we have related issues,
> depending on what the sending and receiving OS does with overly large
> datagrams:
> - We may receive a datagram with a truncated message
> - We may get an upcall when processing one fragment of a fragmented datagram,
> where the complete message is not available yet
> - We may not be able to send the overly large ClusterStatus in the first
> place. Linux claims to do PMTU and return EMSGSIZE if a datagram packet
> payload exceeds the MTU, but will send a fragmented datagram if PMTU is
> disabled. I'm surprised we have the above report given the default is to
> reject overly large datagram payloads, so perhaps the user is using a
> different server OS or Netty datagram channels do their own fragmentation (I
> haven't checked).
> In any case, the server and client pipelines are definitely not do
[jira] [Commented] (HBASE-12142) Truncate command does not preserve ACLs table
[ https://issues.apache.org/jira/browse/HBASE-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156305#comment-14156305 ] Anoop Sam John commented on HBASE-12142: Why we need a config based approach? When truncate should remove the ACLs for a table? > Truncate command does not preserve ACLs table > - > > Key: HBASE-12142 > URL: https://issues.apache.org/jira/browse/HBASE-12142 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.6 >Reporter: Vandana Ayyalasomayajula >Assignee: Vandana Ayyalasomayajula >Priority: Minor > Labels: security > Attachments: HBASE-12142_0.patch > > > The current truncate command does not preserve acls on a table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11997) CopyTable with bulkload
[ https://issues.apache.org/jira/browse/HBASE-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156504#comment-14156504 ] Jean-Marc Spaggiari commented on HBASE-11997: - Nice! [~apurtell] This is something we will want in 0.98 I think. > CopyTable with bulkload > --- > > Key: HBASE-11997 > URL: https://issues.apache.org/jira/browse/HBASE-11997 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 0.89-fb >Reporter: @deprecated Yi Deng >Priority: Minor > Labels: bulkloader, copy, mapreduce > Fix For: 0.89-fb > > > CopyTable using bulkload for writing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11835) Wrong managenement of non expected calls in the client
[ https://issues.apache.org/jira/browse/HBASE-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11835: Attachment: 11835.rebase.patch > Wrong managenement of non expected calls in the client > -- > > Key: HBASE-11835 > URL: https://issues.apache.org/jira/browse/HBASE-11835 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.0.0, 2.0.0, 0.98.6 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 2.0.0, 0.99.1 > > Attachments: 11835.rebase.patch, rpcClient.patch > > > If a call is purged or canceled we try to skip the reply from the server, but > we read the wrong number of bytes so we corrupt the tcp channel. It's hidden > as it triggers retry and so on, but it's bad for performances obviously. > It happens with cell blocks. > [~ram_krish_86], [~saint@gmail.com], you know this part better than me, > do you agree with the analysis and the patch? > The changes in rpcServer are not fully related: as the client close the > connections in such situation, I observed both ClosedChannelException and > CancelledKeyException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11835) Wrong managenement of non expected calls in the client
[ https://issues.apache.org/jira/browse/HBASE-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11835: Fix Version/s: (was: 0.98.7) Status: Patch Available (was: Open) > Wrong managenement of non expected calls in the client > -- > > Key: HBASE-11835 > URL: https://issues.apache.org/jira/browse/HBASE-11835 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.98.6, 1.0.0, 2.0.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 2.0.0, 0.99.1 > > Attachments: 11835.rebase.patch, rpcClient.patch > > > If a call is purged or canceled we try to skip the reply from the server, but > we read the wrong number of bytes so we corrupt the tcp channel. It's hidden > as it triggers retry and so on, but it's bad for performances obviously. > It happens with cell blocks. > [~ram_krish_86], [~saint@gmail.com], you know this part better than me, > do you agree with the analysis and the patch? > The changes in rpcServer are not fully related: as the client close the > connections in such situation, I observed both ClosedChannelException and > CancelledKeyException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11835) Wrong managenement of non expected calls in the client
[ https://issues.apache.org/jira/browse/HBASE-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156559#comment-14156559 ] Nicolas Liochon commented on HBASE-11835: - I rebased the patch. Actually I was waiting for a confirmation from [~ram_krish] that v1 was ok. If it's ok I will commit (hopefully the patch is similar in 98, but I haven't checked) > Wrong managenement of non expected calls in the client > -- > > Key: HBASE-11835 > URL: https://issues.apache.org/jira/browse/HBASE-11835 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.0.0, 2.0.0, 0.98.6 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 2.0.0, 0.99.1 > > Attachments: 11835.rebase.patch, rpcClient.patch > > > If a call is purged or canceled we try to skip the reply from the server, but > we read the wrong number of bytes so we corrupt the tcp channel. It's hidden > as it triggers retry and so on, but it's bad for performances obviously. > It happens with cell blocks. > [~ram_krish_86], [~saint@gmail.com], you know this part better than me, > do you agree with the analysis and the patch? > The changes in rpcServer are not fully related: as the client close the > connections in such situation, I observed both ClosedChannelException and > CancelledKeyException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12029) Use Table and RegionLocator in HTable.getRegionLocations()
[ https://issues.apache.org/jira/browse/HBASE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-12029: --- Attachment: HBASE-12029_v6.patch Merge with the latest and greates > Use Table and RegionLocator in HTable.getRegionLocations() > --- > > Key: HBASE-12029 > URL: https://issues.apache.org/jira/browse/HBASE-12029 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1 >Reporter: Solomon Duskis >Assignee: Solomon Duskis > Attachments: HBASE-12029.patch, HBASE-12029_v1.patch, > HBASE-12029_v2.patch, HBASE-12029_v3.patch, HBASE-12029_v4.patch, > HBASE-12029_v4.patch, HBASE-12029_v5.patch, HBASE-12029_v6.patch > > > HTable.getRegionLocations() is an internal deprecated API. There are plenty > of internal uses of the method, and it would be ideal to use the Table and > RegionLocator interfaces instead of the HTable method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12029) Use Table and RegionLocator in HTable.getRegionLocations()
[ https://issues.apache.org/jira/browse/HBASE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156626#comment-14156626 ] Solomon Duskis commented on HBASE-12029: One gotcha here is connection management. Were Connections automatically closed in Map/Reduces using the pooled approach? If so, how should the Connections be closed in the new approach of user managed Connections? [~enis] or [~stack] > Use Table and RegionLocator in HTable.getRegionLocations() > --- > > Key: HBASE-12029 > URL: https://issues.apache.org/jira/browse/HBASE-12029 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1 >Reporter: Solomon Duskis >Assignee: Solomon Duskis > Attachments: HBASE-12029.patch, HBASE-12029_v1.patch, > HBASE-12029_v2.patch, HBASE-12029_v3.patch, HBASE-12029_v4.patch, > HBASE-12029_v4.patch, HBASE-12029_v5.patch, HBASE-12029_v6.patch > > > HTable.getRegionLocations() is an internal deprecated API. There are plenty > of internal uses of the method, and it would be ideal to use the Table and > RegionLocator interfaces instead of the HTable method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12155) Replace uses of HTableInterface with Table
Solomon Duskis created HBASE-12155: -- Summary: Replace uses of HTableInterface with Table Key: HBASE-12155 URL: https://issues.apache.org/jira/browse/HBASE-12155 Project: HBase Issue Type: New Feature Affects Versions: 2.0.0, 0.99.1 Reporter: Solomon Duskis Assignee: Solomon Duskis -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11835) Wrong managenement of non expected calls in the client
[
https://issues.apache.org/jira/browse/HBASE-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156628#comment-14156628
]
Hadoop QA commented on HBASE-11835:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672543/11835.rebase.patch
against trunk revision .
ATTACHMENT ID: 12672543
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11187//console
This message is automatically generated.
> Wrong managenement of non expected calls in the client
> --
>
> Key: HBASE-11835
> URL: https://issues.apache.org/jira/browse/HBASE-11835
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.0.0, 2.0.0, 0.98.6
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 11835.rebase.patch, rpcClient.patch
>
>
> If a call is purged or canceled we try to skip the reply from the server, but
> we read the wrong number of bytes so we corrupt the tcp channel. It's hidden
> as it triggers retry and so on, but it's bad for performances obviously.
> It happens with cell blocks.
> [~ram_krish_86], [~saint@gmail.com], you know this part better than me,
> do you agree with the analysis and the patch?
> The changes in rpcServer are not fully related: as the client close the
> connections in such situation, I observed both ClosedChannelException and
> CancelledKeyException.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-12072) We are doing 35 x 35 retries for master operations
[
https://issues.apache.org/jira/browse/HBASE-12072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HBASE-12072:
--
Assignee: (was: Ted Yu)
> We are doing 35 x 35 retries for master operations
> --
>
> Key: HBASE-12072
> URL: https://issues.apache.org/jira/browse/HBASE-12072
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.6
>Reporter: Enis Soztutar
> Attachments: 12072-v1.txt, 12072-v2.txt
>
>
> For master requests, there are two retry mechanisms in effect. The first one
> is from HBaseAdmin.executeCallable()
> {code}
> private V executeCallable(MasterCallable callable) throws
> IOException {
> RpcRetryingCaller caller = rpcCallerFactory.newCaller();
> try {
> return caller.callWithRetries(callable);
> } finally {
> callable.close();
> }
> }
> {code}
> And inside, the other one is from StubMaker.makeStub():
> {code}
> /**
>* Create a stub against the master. Retry if necessary.
>* @return A stub to do intf against the master
>* @throws MasterNotRunningException
>*/
> @edu.umd.cs.findbugs.annotations.SuppressWarnings
> (value="SWL_SLEEP_WITH_LOCK_HELD")
> Object makeStub() throws MasterNotRunningException {
> {code}
> The tests will just hang for 10 min * 35 ~= 6hours.
> {code}
> 2014-09-23 16:19:05,151 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 1 of 35
> failed; retrying after sleep of 100, exception=java.io.IOException: Can't get
> master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:05,253 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 2 of 35
> failed; retrying after sleep of 200, exception=java.io.IOException: Can't get
> master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:05,456 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 3 of 35
> failed; retrying after sleep of 300, exception=java.io.IOException: Can't get
> master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:05,759 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 4 of 35
> failed; retrying after sleep of 500, exception=java.io.IOException: Can't get
> master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:06,262 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 5 of 35
> failed; retrying after sleep of 1008, exception=java.io.IOException: Can't
> get master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:07,273 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 6 of 35
> failed; retrying after sleep of 2011, exception=java.io.IOException: Can't
> get master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:09,286 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 7 of 35
> failed; retrying after sleep of 4012, exception=java.io.IOException: Can't
> get master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:13,303 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 8 of 35
> failed; retrying after sleep of 10033, exception=java.io.IOException: Can't
> get master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:23,343 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 9 of 35
> failed; retrying after sleep of 10089, exception=java.io.IOException: Can't
> get master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:33,439 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 10 of
> 35 failed; retrying after sleep of 10027, exception=java.io.IOException:
> Can't get master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:43,473 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 11 of
> 35 failed; retrying after sleep of 10004, exception=java.io.IOException:
> Can't get master address from ZooKeeper; znode data == null
> 2014-09-23 16:19:53,485 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 12 of
> 35 failed; retrying after sleep of 20160, exception=java.io.IOException:
> Can't get master address from ZooKeeper; znode data == null
> 2014-09-23 16:20:13,656 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 13 of
> 35 failed; retrying after sleep of 20006, exception=java.io.IOException:
> Can't get master address from ZooKeeper; znode data == null
> 2014-09-23 16:20:33,675 INFO [main]
> client.ConnectionManager$HConnectionImplementation: getMaster attempt 14 of
> 35 failed; retrying after sleep of 20076, exception=java.
[jira] [Resolved] (HBASE-12155) Replace uses of HTableInterface with Table
[ https://issues.apache.org/jira/browse/HBASE-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis resolved HBASE-12155. Resolution: Invalid > Replace uses of HTableInterface with Table > -- > > Key: HBASE-12155 > URL: https://issues.apache.org/jira/browse/HBASE-12155 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0, 0.99.1 >Reporter: Solomon Duskis >Assignee: Solomon Duskis > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156660#comment-14156660 ] Yi Deng commented on HBASE-12133: - Thanks! [~stack] Should I put this Class to hbase-hadoop2-compat project? It is under hbase-common now. > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 0.98.8 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-11997) CopyTable with bulkload
[ https://issues.apache.org/jira/browse/HBASE-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Deng reassigned HBASE-11997: --- Assignee: Yi Deng > CopyTable with bulkload > --- > > Key: HBASE-11997 > URL: https://issues.apache.org/jira/browse/HBASE-11997 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 0.89-fb >Reporter: @deprecated Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: bulkloader, copy, mapreduce > Fix For: 0.89-fb > > > CopyTable using bulkload for writing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
Andrey Stepachev created HBASE-12156: Summary: TableName cache doesn't used for once of valueOf methods. Key: HBASE-12156 URL: https://issues.apache.org/jira/browse/HBASE-12156 Project: HBase Issue Type: Bug Reporter: Andrey Stepachev Assignee: Andrey Stepachev there is wrong comparison, copy&paste code compares namespace with qualifier and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-12156: - Attachment: HBASE-12156.patch > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Attachments: HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-12156: - Status: Patch Available (was: Open) > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Attachments: HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12029) Use Table and RegionLocator in HTable.getRegionLocations()
[
https://issues.apache.org/jira/browse/HBASE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156704#comment-14156704
]
Hadoop QA commented on HBASE-12029:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672552/HBASE-12029_v6.patch
against trunk revision .
ATTACHMENT ID: 12672552
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed
org.apache.hadoop.hbase.mapreduce.TestCopyTable
org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.mapreduce.TestCellCounter
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.mapreduce.TestRowCounter
org.apache.hadoop.hbase.replication.TestReplicationSmallTests
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11188//console
This message is automatically generated.
> Use Table and RegionLocator in HTable.getRegionLocations()
> ---
>
> Key: HBASE-12029
> URL: https://issues.apache.org/jira/browse/HBASE-12029
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 0.99.1
>Reporter: Solomon Duskis
>Assignee: Solomon Duskis
> Attachments: HBASE-12029.patch, HBASE-12029_v1.patch,
> HBASE-12029_v2.patch, HBASE-12029_v3.patch, HBASE-12029_v4.patch,
> HBASE-12029_v4.patch, HBASE-12029_v5.patch, HBASE-12029_v6.patch
>
>
> HTable.getRegionLoca
[jira] [Updated] (HBASE-12153) Fixing TestReplicaWithCluster
[ https://issues.apache.org/jira/browse/HBASE-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12153: -- Resolution: Fixed Fix Version/s: (was: 1.0.0) 0.99.1 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to branch-1+ Thanks [~manukranthk] > Fixing TestReplicaWithCluster > - > > Key: HBASE-12153 > URL: https://issues.apache.org/jira/browse/HBASE-12153 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.99.1 > > Attachments: 0001-FixTestReplicaWithCluster.patch > > > This test takes about 30 ~ 40 seconds depending upon the resources available. > Doesn't make sense to have such a tight bound(30s) on the unit test. > [~nkeywal], what do you think ? Did you intend to have such a tight bound > while adding the test here ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12157) Add isTableAvailable() to the Connection interface.
Solomon Duskis created HBASE-12157: -- Summary: Add isTableAvailable() to the Connection interface. Key: HBASE-12157 URL: https://issues.apache.org/jira/browse/HBASE-12157 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.1 Reporter: Solomon Duskis Assignee: Solomon Duskis -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156722#comment-14156722 ] Matteo Bertozzi commented on HBASE-12156: - +1 > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Attachments: HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10153) improve VerifyReplication to compute BADROWS more accurately
[
https://issues.apache.org/jira/browse/HBASE-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-10153:
---
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
> improve VerifyReplication to compute BADROWS more accurately
>
>
> Key: HBASE-10153
> URL: https://issues.apache.org/jira/browse/HBASE-10153
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 0.94.14
>Reporter: cuijianwei
>Assignee: cuijianwei
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 10153-v2-trunk.txt, HBASE-10153-0.94-v1.patch,
> HBASE-10153-trunk.patch
>
>
> VerifyReplicaiton could compare the source table with its peer table and
> compute BADROWS. However, the current BADROWS computing method might not be
> accurate enough. For example, if source table contains rows as {r1, r2, r3,
> r4} and peer table contains rows as {r1, r3, r4} BADROWS will be 3 because
> 'r2' in source table will make all the later row comparisons fail. Will it be
> better if the BADROWS is computed to 1 in this situation? Maybe, we can
> compute the BADROWS more accurately in merge comparison?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-10153) improve VerifyReplication to compute BADROWS more accurately
[
https://issues.apache.org/jira/browse/HBASE-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-10153:
---
Attachment: 10153-v2-trunk.txt
Rebased patch
> improve VerifyReplication to compute BADROWS more accurately
>
>
> Key: HBASE-10153
> URL: https://issues.apache.org/jira/browse/HBASE-10153
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 0.94.14
>Reporter: cuijianwei
>Assignee: cuijianwei
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 10153-v2-trunk.txt, HBASE-10153-0.94-v1.patch,
> HBASE-10153-trunk.patch
>
>
> VerifyReplicaiton could compare the source table with its peer table and
> compute BADROWS. However, the current BADROWS computing method might not be
> accurate enough. For example, if source table contains rows as {r1, r2, r3,
> r4} and peer table contains rows as {r1, r3, r4} BADROWS will be 3 because
> 'r2' in source table will make all the later row comparisons fail. Will it be
> better if the BADROWS is computed to 1 in this situation? Maybe, we can
> compute the BADROWS more accurately in merge comparison?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12156: --- Fix Version/s: 0.99.1 0.98.7 2.0.0 Hadoop Flags: Reviewed > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12156: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to branches 0.98+ Thanks for the patch [~octo47] > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156748#comment-14156748 ] Lars Hofhansl commented on HBASE-11764: --- Glanced over the patch. Looks good. I think we should separate this one from HBASE-11763 and commit this one first and then do the refactoring. (as discussed offline). Lemme locally apply this patch and explore via Eclipse. I promise I'll do it this morning. > Support per cell TTLs > - > > Key: HBASE-11764 > URL: https://issues.apache.org/jira/browse/HBASE-11764 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, > HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, > HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156753#comment-14156753
]
stack commented on HBASE-12152:
---
Adding timeouts on these tests. Pushed to branch-1+
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12142) Truncate command does not preserve ACLs table
[ https://issues.apache.org/jira/browse/HBASE-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vandana Ayyalasomayajula updated HBASE-12142: - Attachment: HBASE-12142_1.patch I meant to use the config variable to retain old behavior if needed. If you think its unnecessary, lets remove the config variable and restore acls in all cases. > Truncate command does not preserve ACLs table > - > > Key: HBASE-12142 > URL: https://issues.apache.org/jira/browse/HBASE-12142 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.6 >Reporter: Vandana Ayyalasomayajula >Assignee: Vandana Ayyalasomayajula >Priority: Minor > Labels: security > Attachments: HBASE-12142_0.patch, HBASE-12142_1.patch > > > The current truncate command does not preserve acls on a table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11763) Move TTL handling into ScanQueryMatcher
[ https://issues.apache.org/jira/browse/HBASE-11763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11763: --- Issue Type: Improvement (was: Sub-task) Parent: (was: HBASE-10560) > Move TTL handling into ScanQueryMatcher > --- > > Key: HBASE-11763 > URL: https://issues.apache.org/jira/browse/HBASE-11763 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-11763-0.98.patch, HBASE-11763.patch, > HBASE-11763.patch, HBASE-11763.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156796#comment-14156796 ] Hudson commented on HBASE-12156: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #531 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/531/]) HBASE-12156 TableName cache doesn't used for once of valueOf methods (Andrey Stepachev) (stack: rev b77edc0bb244b3059632fee0250bcdd9f752d5d2) * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java * hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11763) Move TTL handling into ScanQueryMatcher
[ https://issues.apache.org/jira/browse/HBASE-11763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11763: --- Fix Version/s: (was: 0.99.1) (was: 0.98.7) [~lhofhansl] and I discussed this offline. It's a bit hard to reason about possible unintended consequences of this change so let's move it out from a subtask of HBASE-10560 to its own improvement issue and target it for 2.0 only. > Move TTL handling into ScanQueryMatcher > --- > > Key: HBASE-11763 > URL: https://issues.apache.org/jira/browse/HBASE-11763 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0 > > Attachments: HBASE-11763-0.98.patch, HBASE-11763.patch, > HBASE-11763.patch, HBASE-11763.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156800#comment-14156800 ] Andrew Purtell commented on HBASE-11764: I moved HBASE-11763 out. Let me rebase the patch on this issue without that change. > Support per cell TTLs > - > > Key: HBASE-11764 > URL: https://issues.apache.org/jira/browse/HBASE-11764 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, > HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, > HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12158) TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on occasion
stack created HBASE-12158:
-
Summary: TestHttpServerLifecycle.testStartedServerWithRequestLog
goes zombie on occasion
Key: HBASE-12158
URL: https://issues.apache.org/jira/browse/HBASE-12158
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Jetty server won't go down when running testStartedServerWithRequestLog. Its
stuck in the stop. I tried to fix this before, HBASE-12025. See below how
SelectorManager is trying to get lock held by the call to stop on the
SelectorManager.
{code}
"1956684235@qtp-1078912108-1 - Acceptor0
SelectChannelConnector@localhost:58604" daemon prio=10 tid=0x7f11cc896000
nid=0x5c36 waiting for monitor entry [0x7f116fbdf000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
- waiting to lock <0x0007dd57c560> (a
org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
at
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
"1786126004@qtp-1078912108-0" daemon prio=10 tid=0x7f11cc865800 nid=0x5c35
in Object.wait() [0x7f118c17c000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0007dd358460> (a
org.mortbay.thread.QueuedThreadPool$PoolThread)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
- locked <0x0007dd358460> (a
org.mortbay.thread.QueuedThreadPool$PoolThread)
"Service Thread" daemon prio=10 tid=0x7f11cc29 nid=0x5c18 runnable
[0x]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" daemon prio=10 tid=0x7f11cc28e000 nid=0x5c17 waiting
on condition [0x]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" daemon prio=10 tid=0x7f11cc28b000 nid=0x5c16 waiting
on condition [0x]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x7f11cc280800 nid=0x5c15 runnable
[0x]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x7f11cc26b000 nid=0x5c14 in Object.wait()
[0x7f118cd3d000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
- locked <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
"Reference Handler" daemon prio=10 tid=0x7f11cc267000 nid=0x5c13 in
Object.wait() [0x7f118ce3e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0x7f11cc00a800 nid=0x5c04 runnable [0x7f11d367e000]
java.lang.Thread.State: RUNNABLE
at
org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:879)
- locked <0x0007dd57c560> (a
org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
at
org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
- locked <0x0007dd3490a0> (a java.lang.Object)
at
org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
- locked <0x0007dd348f00> (a
org.mortbay.jetty.nio.SelectChannelConnector)
at org.apache.hadoop.hbase.http.HttpServer.stop(HttpServer.java:1037)
at
org.apache.hadoop.hbase.http.HttpServerFunctionalTest.stop(HttpServerFunctionalTest.java:195)
at
org.apache.hadoop.hbase.http.TestHttpServerLifecycle.testStartedServerWithRequestLog(TestHttpServerLifecycle.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.int
[jira] [Updated] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11764: --- Status: Open (was: Patch Available) > Support per cell TTLs > - > > Key: HBASE-11764 > URL: https://issues.apache.org/jira/browse/HBASE-11764 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, > HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, > HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12158) TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on occasion
[
https://issues.apache.org/jira/browse/HBASE-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12158:
--
Assignee: stack
Status: Patch Available (was: Open)
> TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on
> occasion
> ---
>
> Key: HBASE-12158
> URL: https://issues.apache.org/jira/browse/HBASE-12158
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
> Attachments: 12158.txt
>
>
> Jetty server won't go down when running testStartedServerWithRequestLog. Its
> stuck in the stop. I tried to fix this before, HBASE-12025. See below how
> SelectorManager is trying to get lock held by the call to stop on the
> SelectorManager.
> {code}
> "1956684235@qtp-1078912108-1 - Acceptor0
> SelectChannelConnector@localhost:58604" daemon prio=10 tid=0x7f11cc896000
> nid=0x5c36 waiting for monitor entry [0x7f116fbdf000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
> - waiting to lock <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> "1786126004@qtp-1078912108-0" daemon prio=10 tid=0x7f11cc865800
> nid=0x5c35 in Object.wait() [0x7f118c17c000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
> - locked <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> "Service Thread" daemon prio=10 tid=0x7f11cc29 nid=0x5c18 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" daemon prio=10 tid=0x7f11cc28e000 nid=0x5c17 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" daemon prio=10 tid=0x7f11cc28b000 nid=0x5c16 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=10 tid=0x7f11cc280800 nid=0x5c15 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=10 tid=0x7f11cc26b000 nid=0x5c14 in Object.wait()
> [0x7f118cd3d000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
> "Reference Handler" daemon prio=10 tid=0x7f11cc267000 nid=0x5c13 in
> Object.wait() [0x7f118ce3e000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> "main" prio=10 tid=0x7f11cc00a800 nid=0x5c04 runnable [0x7f11d367e000]
>java.lang.Thread.State: RUNNABLE
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:879)
> - locked <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> - locked <0x0007dd3490a0> (a java.lang.Object)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
> - locked <0x0007dd348f00> (a
> org.mortbay.jetty.nio.SelectChannelConnector)
> at org.apache.hadoop.hbase.http.HttpServer.stop(HttpServer.java:1037)
> at
> org.apache.hadoop.hbase.http.HttpServerFunctionalTest.stop(HttpServerFunctionalTest.java:195)
> at
> org.apache.hadoop.hbase.http.TestHttpServerLifecycle.testStartedServerWithRequestLog(TestHttpServerLifecycle.java:92)
> at sun.reflect.NativeM
[jira] [Updated] (HBASE-12158) TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on occasion
[
https://issues.apache.org/jira/browse/HBASE-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12158:
--
Attachment: 12158.txt
Here, just removing the test. Hangs rarely. Not important enough to pursue.
> TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on
> occasion
> ---
>
> Key: HBASE-12158
> URL: https://issues.apache.org/jira/browse/HBASE-12158
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
> Attachments: 12158.txt
>
>
> Jetty server won't go down when running testStartedServerWithRequestLog. Its
> stuck in the stop. I tried to fix this before, HBASE-12025. See below how
> SelectorManager is trying to get lock held by the call to stop on the
> SelectorManager.
> {code}
> "1956684235@qtp-1078912108-1 - Acceptor0
> SelectChannelConnector@localhost:58604" daemon prio=10 tid=0x7f11cc896000
> nid=0x5c36 waiting for monitor entry [0x7f116fbdf000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
> - waiting to lock <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> "1786126004@qtp-1078912108-0" daemon prio=10 tid=0x7f11cc865800
> nid=0x5c35 in Object.wait() [0x7f118c17c000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
> - locked <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> "Service Thread" daemon prio=10 tid=0x7f11cc29 nid=0x5c18 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" daemon prio=10 tid=0x7f11cc28e000 nid=0x5c17 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" daemon prio=10 tid=0x7f11cc28b000 nid=0x5c16 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=10 tid=0x7f11cc280800 nid=0x5c15 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=10 tid=0x7f11cc26b000 nid=0x5c14 in Object.wait()
> [0x7f118cd3d000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
> "Reference Handler" daemon prio=10 tid=0x7f11cc267000 nid=0x5c13 in
> Object.wait() [0x7f118ce3e000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> "main" prio=10 tid=0x7f11cc00a800 nid=0x5c04 runnable [0x7f11d367e000]
>java.lang.Thread.State: RUNNABLE
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:879)
> - locked <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> - locked <0x0007dd3490a0> (a java.lang.Object)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
> - locked <0x0007dd348f00> (a
> org.mortbay.jetty.nio.SelectChannelConnector)
> at org.apache.hadoop.hbase.http.HttpServer.stop(HttpServer.java:1037)
> at
> org.apache.hadoop.hbase.http.HttpServerFunctionalTest.stop(HttpServerFunctionalTest.java:195)
> at
> org.apache.hadoop.hbase.http.TestHttpServerLifecycle.testStartedServerWithRequestLog(TestHttpServerLifecycle.java:92)
> at sun.re
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156810#comment-14156810
]
Andrew Purtell commented on HBASE-12152:
Cool I cherry picked the timeout additions back to 0.98 and pushed them, they
applied cleanly.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-12156: This is going to need an addendum to fix the 0.98 build: [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /Users/apurtell/src/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java:[157,49] error: diamond operator is not supported in -source 1.6 > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156817#comment-14156817 ] Hudson commented on HBASE-12156: FAILURE: Integrated in HBase-0.98 #560 (See [https://builds.apache.org/job/HBase-0.98/560/]) HBASE-12156 TableName cache doesn't used for once of valueOf methods (Andrey Stepachev) (stack: rev b77edc0bb244b3059632fee0250bcdd9f752d5d2) * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java * hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156821#comment-14156821 ] Andrew Purtell commented on HBASE-12156: Trivial fix, will commit it in a sec > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-12156: --- Attachment: HBASE-12156-addendum-0.98.patch > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156-addendum-0.98.patch, HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12154) TestLoadAndSwitchEncodeOnDisk is flaky on shared jenkins boxes
[ https://issues.apache.org/jira/browse/HBASE-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156826#comment-14156826 ] stack commented on HBASE-12154: --- Yeah. We should remove related tests too, TestMiniClusterLoadSequential and TestMiniClusterLoadParallel. Pity we didn't have a box to run integration tests on. Otherwise, moving the tests feels like we're just deleting them (not your issue [~manukranthk]) > TestLoadAndSwitchEncodeOnDisk is flaky on shared jenkins boxes > -- > > Key: HBASE-12154 > URL: https://issues.apache.org/jira/browse/HBASE-12154 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Minor > Fix For: 1.0.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > It doesn't make sense to run a load test on a shared box where other tests > are being run. We should probably move this to integration tests and make > sure its covered. In cases where the jenkins machines are shared across > several projects which are doing disk IO, I have observed that this test runs > into slow syncs and eventually times out. And that being said, we can't > increase the timeout on this test, since its already 3 minutes. This test > being disk sensitive, might have better coverage on IT. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-12156. Resolution: Fixed Pushed addendum to 0.98 > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156-addendum-0.98.patch, HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156837#comment-14156837 ] stack commented on HBASE-12156: --- Sorry about that [~apurtell] Thanks. > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156-addendum-0.98.patch, HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12133: -- Resolution: Fixed Fix Version/s: (was: 0.98.8) 0.99.1 2.0.0 Release Note: Adds Histogram and AtomicLong Utils Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to branch-1+ Thanks for the patch [~daviddengcn] I think it best in common since general utility. > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 2.0.0, 0.99.1 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12141) ClusterStatus message might exceed max datagram payload limits
[
https://issues.apache.org/jira/browse/HBASE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156857#comment-14156857
]
Andrew Purtell commented on HBASE-12141:
See
http://mail-archives.apache.org/mod_mbox/hbase-user/201410.mbox/%3C3256288.x8cyWY5ZEW%40localhost.localdomain%3E
. The network configuration is "interesting".
> ClusterStatus message might exceed max datagram payload limits
> --
>
> Key: HBASE-12141
> URL: https://issues.apache.org/jira/browse/HBASE-12141
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.3
>Reporter: Andrew Purtell
>
> The multicast ClusterStatusPublisher and its companion listener are using
> datagram channels without any framing. I think this is an issue because
> Netty's ProtobufDecoder expects a complete PB message to be available in the
> ChannelBuffer yet ClusterStatus messages can be large and might exceed the
> maximum datagram payload size. As one user reported on list:
> {noformat}
> org.apache.hadoop.hbase.client.ClusterStatusListener - ERROR - Unexpected
> exception, continuing.
> com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
> invalid wire type.
> at
> com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:99)
> at
> com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:498)
> at
> com.google.protobuf.GeneratedMessage.parseUnknownField(GeneratedMessage.java:193)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus.(ClusterStatusProtos.java:7554)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus.(ClusterStatusProtos.java:7512)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus$1.parsePartialFrom(ClusterStatusProtos.java:7689)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus$1.parsePartialFrom(ClusterStatusProtos.java:7684)
> at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:141)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:176)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:182)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.jboss.netty.handler.codec.protobuf.ProtobufDecoder.decode(ProtobufDecoder.java:122)
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.socket.oio.OioDatagramWorker.process(OioDatagramWorker.java:52)
> at
> org.jboss.netty.channel.socket.oio.AbstractOioWorker.run(AbstractOioWorker.java:73)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The javadoc for ProtobufDecoder says:
> {quote}
> Decodes a received ChannelBuffer into a Google Protocol Buffers Message and
> MessageLite. Please note that this decoder must be used with a proper
> FrameDecoder such as ProtobufVarint32FrameDecoder or
> LengthFieldBasedFrameDecoder if you are using a stream-based transport such
> as TCP/IP.
> {quote}
> and even though we are using a datagram transport we have related issues,
> depending on what the sending and receiving OS does with overly large
> datagrams:
> - We may receive a datagram with a truncated message
> - We may get an upcall when processing one fragment of a fragmented datagram,
> where the complete message is not available yet
> - We may not be able to send the overly large ClusterStatus in the first
> place. Linux claims to do PMTU and return EMSGSIZE if a datagram packet
> payload exceeds the MTU, but will send a fragmented datagram if PMTU is
> disabled. I'm surprised we have the above report given the default is to
> reject overly large datagram payloads, so perhaps the user is using a
> different server OS or Netty datagram channels do their own fragmentation (I
> haven't checked).
> In any case, the server and client pipelines are definitely not doing any
> kind of framing. This is the multicast status listener from 0.98 for example:
> {code}
> b.setPipeline(Channels.pipeline(
> new
> ProtobufDecoder(ClusterStatusProtos.ClusterStatus.getDefaultInstance()),
> new ClusterStatusHandler()));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12141) ClusterStatus message might exceed max datagram payload limits
[
https://issues.apache.org/jira/browse/HBASE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156860#comment-14156860
]
Andrew Purtell commented on HBASE-12141:
... which could explain fragmentation, I think. The MTU of OpenVPN tunnels will
be less than normal by packet header + tunnel protocol overheads and possibly
not subject to PMTU discovery. I'd wager our channel handler is being invoked
with the first fragment is received so the PB is truncated but the remainder
will show up soon.
> ClusterStatus message might exceed max datagram payload limits
> --
>
> Key: HBASE-12141
> URL: https://issues.apache.org/jira/browse/HBASE-12141
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.3
>Reporter: Andrew Purtell
>
> The multicast ClusterStatusPublisher and its companion listener are using
> datagram channels without any framing. I think this is an issue because
> Netty's ProtobufDecoder expects a complete PB message to be available in the
> ChannelBuffer yet ClusterStatus messages can be large and might exceed the
> maximum datagram payload size. As one user reported on list:
> {noformat}
> org.apache.hadoop.hbase.client.ClusterStatusListener - ERROR - Unexpected
> exception, continuing.
> com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
> invalid wire type.
> at
> com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:99)
> at
> com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:498)
> at
> com.google.protobuf.GeneratedMessage.parseUnknownField(GeneratedMessage.java:193)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus.(ClusterStatusProtos.java:7554)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus.(ClusterStatusProtos.java:7512)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus$1.parsePartialFrom(ClusterStatusProtos.java:7689)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus$1.parsePartialFrom(ClusterStatusProtos.java:7684)
> at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:141)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:176)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:182)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.jboss.netty.handler.codec.protobuf.ProtobufDecoder.decode(ProtobufDecoder.java:122)
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.socket.oio.OioDatagramWorker.process(OioDatagramWorker.java:52)
> at
> org.jboss.netty.channel.socket.oio.AbstractOioWorker.run(AbstractOioWorker.java:73)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The javadoc for ProtobufDecoder says:
> {quote}
> Decodes a received ChannelBuffer into a Google Protocol Buffers Message and
> MessageLite. Please note that this decoder must be used with a proper
> FrameDecoder such as ProtobufVarint32FrameDecoder or
> LengthFieldBasedFrameDecoder if you are using a stream-based transport such
> as TCP/IP.
> {quote}
> and even though we are using a datagram transport we have related issues,
> depending on what the sending and receiving OS does with overly large
> datagrams:
> - We may receive a datagram with a truncated message
> - We may get an upcall when processing one fragment of a fragmented datagram,
> where the complete message is not available yet
> - We may not be able to send the overly large ClusterStatus in the first
> place. Linux claims to do PMTU and return EMSGSIZE if a datagram packet
> payload exceeds the MTU, but will send a fragmented datagram if PMTU is
> disabled. I'm surprised we have the above report given the default is to
> reject overly large datagram payloads, so perhaps the user is using a
> different server OS or Netty datagram channels do their own fragmentation (I
> haven't checked).
> In any case, the server and client pipelines are definitely not doing any
> kind of framing. This is the multicast status listener from 0.98 for example:
> {code}
> b.setPipeline(Channels.pipeline(
> new
> ProtobufDecoder(ClusterStatusProtos.ClusterStatus.getDefaultInstance(
[jira] [Comment Edited] (HBASE-12141) ClusterStatus message might exceed max datagram payload limits
[
https://issues.apache.org/jira/browse/HBASE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156860#comment-14156860
]
Andrew Purtell edited comment on HBASE-12141 at 10/2/14 5:49 PM:
-
... which could explain fragmentation, I think. The MTU of OpenVPN tunnels will
be less than normal by packet header + tunnel protocol overheads and possibly
not subject to PMTU discovery. I'd wager our channel handler is being invoked
when the first fragment is received so the PB is truncated but the remainder
will show up soon.
was (Author: apurtell):
... which could explain fragmentation, I think. The MTU of OpenVPN tunnels will
be less than normal by packet header + tunnel protocol overheads and possibly
not subject to PMTU discovery. I'd wager our channel handler is being invoked
with the first fragment is received so the PB is truncated but the remainder
will show up soon.
> ClusterStatus message might exceed max datagram payload limits
> --
>
> Key: HBASE-12141
> URL: https://issues.apache.org/jira/browse/HBASE-12141
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.3
>Reporter: Andrew Purtell
>
> The multicast ClusterStatusPublisher and its companion listener are using
> datagram channels without any framing. I think this is an issue because
> Netty's ProtobufDecoder expects a complete PB message to be available in the
> ChannelBuffer yet ClusterStatus messages can be large and might exceed the
> maximum datagram payload size. As one user reported on list:
> {noformat}
> org.apache.hadoop.hbase.client.ClusterStatusListener - ERROR - Unexpected
> exception, continuing.
> com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
> invalid wire type.
> at
> com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:99)
> at
> com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:498)
> at
> com.google.protobuf.GeneratedMessage.parseUnknownField(GeneratedMessage.java:193)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus.(ClusterStatusProtos.java:7554)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus.(ClusterStatusProtos.java:7512)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus$1.parsePartialFrom(ClusterStatusProtos.java:7689)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos$ClusterStatus$1.parsePartialFrom(ClusterStatusProtos.java:7684)
> at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:141)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:176)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:182)
> at
> com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.jboss.netty.handler.codec.protobuf.ProtobufDecoder.decode(ProtobufDecoder.java:122)
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.socket.oio.OioDatagramWorker.process(OioDatagramWorker.java:52)
> at
> org.jboss.netty.channel.socket.oio.AbstractOioWorker.run(AbstractOioWorker.java:73)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The javadoc for ProtobufDecoder says:
> {quote}
> Decodes a received ChannelBuffer into a Google Protocol Buffers Message and
> MessageLite. Please note that this decoder must be used with a proper
> FrameDecoder such as ProtobufVarint32FrameDecoder or
> LengthFieldBasedFrameDecoder if you are using a stream-based transport such
> as TCP/IP.
> {quote}
> and even though we are using a datagram transport we have related issues,
> depending on what the sending and receiving OS does with overly large
> datagrams:
> - We may receive a datagram with a truncated message
> - We may get an upcall when processing one fragment of a fragmented datagram,
> where the complete message is not available yet
> - We may not be able to send the overly large ClusterStatus in the first
> place. Linux claims to do PMTU and return EMSGSIZE if a datagram packet
> payload exceeds the MTU, but will send a fragmented datagram if PMTU is
> disabled. I'm surprised we have the above report given the default is to
> reject overly large datagram payloads, so perhaps the u
[jira] [Commented] (HBASE-12151) Make dev scripts executable
[ https://issues.apache.org/jira/browse/HBASE-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156863#comment-14156863 ] stack commented on HBASE-12151: --- +1 > Make dev scripts executable > --- > > Key: HBASE-12151 > URL: https://issues.apache.org/jira/browse/HBASE-12151 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-12151.patch > > > Is there any reason not to make dev-support/*.sh executable? It would make it > possible to sym-link to them from a directory in the executable path for > easier execution of the definitive scripts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12158) TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on occasion
[
https://issues.apache.org/jira/browse/HBASE-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156868#comment-14156868
]
Andrew Purtell commented on HBASE-12158:
+1
> TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on
> occasion
> ---
>
> Key: HBASE-12158
> URL: https://issues.apache.org/jira/browse/HBASE-12158
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
> Attachments: 12158.txt
>
>
> Jetty server won't go down when running testStartedServerWithRequestLog. Its
> stuck in the stop. I tried to fix this before, HBASE-12025. See below how
> SelectorManager is trying to get lock held by the call to stop on the
> SelectorManager.
> {code}
> "1956684235@qtp-1078912108-1 - Acceptor0
> SelectChannelConnector@localhost:58604" daemon prio=10 tid=0x7f11cc896000
> nid=0x5c36 waiting for monitor entry [0x7f116fbdf000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
> - waiting to lock <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> "1786126004@qtp-1078912108-0" daemon prio=10 tid=0x7f11cc865800
> nid=0x5c35 in Object.wait() [0x7f118c17c000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
> - locked <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> "Service Thread" daemon prio=10 tid=0x7f11cc29 nid=0x5c18 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" daemon prio=10 tid=0x7f11cc28e000 nid=0x5c17 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" daemon prio=10 tid=0x7f11cc28b000 nid=0x5c16 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=10 tid=0x7f11cc280800 nid=0x5c15 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=10 tid=0x7f11cc26b000 nid=0x5c14 in Object.wait()
> [0x7f118cd3d000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
> "Reference Handler" daemon prio=10 tid=0x7f11cc267000 nid=0x5c13 in
> Object.wait() [0x7f118ce3e000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> "main" prio=10 tid=0x7f11cc00a800 nid=0x5c04 runnable [0x7f11d367e000]
>java.lang.Thread.State: RUNNABLE
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:879)
> - locked <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> - locked <0x0007dd3490a0> (a java.lang.Object)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
> - locked <0x0007dd348f00> (a
> org.mortbay.jetty.nio.SelectChannelConnector)
> at org.apache.hadoop.hbase.http.HttpServer.stop(HttpServer.java:1037)
> at
> org.apache.hadoop.hbase.http.HttpServerFunctionalTest.stop(HttpServerFunctionalTest.java:195)
> at
> org.apache.hadoop.hbase.http.TestHttpServerLifecycle.testStartedServerWithRequestLog(TestHttpServerLifecycle.java:92)
> at sun
[jira] [Resolved] (HBASE-12157) Add isTableAvailable() to the Connection interface.
[ https://issues.apache.org/jira/browse/HBASE-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis resolved HBASE-12157. Resolution: Invalid > Add isTableAvailable() to the Connection interface. > --- > > Key: HBASE-12157 > URL: https://issues.apache.org/jira/browse/HBASE-12157 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1 >Reporter: Solomon Duskis >Assignee: Solomon Duskis > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156871#comment-14156871 ] Yi Deng commented on HBASE-12133: - Thanks [~stack] > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 2.0.0, 0.99.1 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12155) Replace uses of HTableInterface with Table
[ https://issues.apache.org/jira/browse/HBASE-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156885#comment-14156885 ] Jean-Marc Spaggiari commented on HBASE-12155: - You might want to add some comments related to the closure? Like, with this is invalide? > Replace uses of HTableInterface with Table > -- > > Key: HBASE-12155 > URL: https://issues.apache.org/jira/browse/HBASE-12155 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0, 0.99.1 >Reporter: Solomon Duskis >Assignee: Solomon Duskis > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12158) TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on occasion
[
https://issues.apache.org/jira/browse/HBASE-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12158:
--
Resolution: Fixed
Fix Version/s: 0.99.1
2.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to branch-1+ Thanks for review Andrew.
> TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on
> occasion
> ---
>
> Key: HBASE-12158
> URL: https://issues.apache.org/jira/browse/HBASE-12158
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 12158.txt
>
>
> Jetty server won't go down when running testStartedServerWithRequestLog. Its
> stuck in the stop. I tried to fix this before, HBASE-12025. See below how
> SelectorManager is trying to get lock held by the call to stop on the
> SelectorManager.
> {code}
> "1956684235@qtp-1078912108-1 - Acceptor0
> SelectChannelConnector@localhost:58604" daemon prio=10 tid=0x7f11cc896000
> nid=0x5c36 waiting for monitor entry [0x7f116fbdf000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
> - waiting to lock <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> "1786126004@qtp-1078912108-0" daemon prio=10 tid=0x7f11cc865800
> nid=0x5c35 in Object.wait() [0x7f118c17c000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
> - locked <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> "Service Thread" daemon prio=10 tid=0x7f11cc29 nid=0x5c18 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" daemon prio=10 tid=0x7f11cc28e000 nid=0x5c17 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" daemon prio=10 tid=0x7f11cc28b000 nid=0x5c16 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=10 tid=0x7f11cc280800 nid=0x5c15 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=10 tid=0x7f11cc26b000 nid=0x5c14 in Object.wait()
> [0x7f118cd3d000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
> "Reference Handler" daemon prio=10 tid=0x7f11cc267000 nid=0x5c13 in
> Object.wait() [0x7f118ce3e000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> "main" prio=10 tid=0x7f11cc00a800 nid=0x5c04 runnable [0x7f11d367e000]
>java.lang.Thread.State: RUNNABLE
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:879)
> - locked <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> - locked <0x0007dd3490a0> (a java.lang.Object)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
> - locked <0x0007dd348f00> (a
> org.mortbay.jetty.nio.SelectChannelConnector)
> at org.apache.hadoop.hbase.http.HttpServer.stop(HttpServer.java:1037)
> at
> org.apache.hadoop.hbase.http.HttpServerFunctionalTest.stop(HttpServerFunc
[jira] [Commented] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156915#comment-14156915 ] Hudson commented on HBASE-12156: FAILURE: Integrated in HBase-TRUNK #5607 (See [https://builds.apache.org/job/HBase-TRUNK/5607/]) HBASE-12156 TableName cache doesn't used for once of valueOf methods (Andrey Stepachev) (stack: rev ff847978ad9e8d01a66ad6327f924662a9b0d5e8) * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java * hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156-addendum-0.98.patch, HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12153) Fixing TestReplicaWithCluster
[ https://issues.apache.org/jira/browse/HBASE-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156914#comment-14156914 ] Hudson commented on HBASE-12153: FAILURE: Integrated in HBase-TRUNK #5607 (See [https://builds.apache.org/job/HBase-TRUNK/5607/]) HBASE-12153 Fixing TestReplicaWithCluster (Manukranth Kolloju) (stack: rev 2c8f6b66cf31d3569de6626686d4ac5bf67e8efe) * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicaWithCluster.java > Fixing TestReplicaWithCluster > - > > Key: HBASE-12153 > URL: https://issues.apache.org/jira/browse/HBASE-12153 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.99.1 > > Attachments: 0001-FixTestReplicaWithCluster.patch > > > This test takes about 30 ~ 40 seconds depending upon the resources available. > Doesn't make sense to have such a tight bound(30s) on the unit test. > [~nkeywal], what do you think ? Did you intend to have such a tight bound > while adding the test here ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156913#comment-14156913
]
Hudson commented on HBASE-12152:
FAILURE: Integrated in HBase-TRUNK #5607 (See
[https://builds.apache.org/job/HBase-TRUNK/5607/])
HBASE-12152 TestLoadIncrementalHFiles shows up as zombie test; ADD TIMEOUT ON
TESTS (stack: rev e92036cd54ac313482a5c6cc3416add560c7600c)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156916#comment-14156916 ] Hudson commented on HBASE-12133: FAILURE: Integrated in HBase-TRUNK #5608 (See [https://builds.apache.org/job/HBase-TRUNK/5608/]) HBASE-12133 Add FastLongHistogram for metric computation (Yi Deng) (stack: rev 0e1e64b821ad052e0f99ed40de12618f76c8465d) * hbase-common/src/main/java/org/apache/hadoop/hbase/util/AtomicUtils.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/FastLongHistogram.java * hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestFastLongHistogram.java > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 2.0.0, 0.99.1 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12158) TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on occasion
[
https://issues.apache.org/jira/browse/HBASE-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156917#comment-14156917
]
Hudson commented on HBASE-12158:
FAILURE: Integrated in HBase-TRUNK #5608 (See
[https://builds.apache.org/job/HBase-TRUNK/5608/])
HBASE-12158 TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie
on occasion (stack: rev 6f104e3ac6f3916b7ca01120a3fb1ac4ddf2692d)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/http/TestHttpServerLifecycle.java
> TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on
> occasion
> ---
>
> Key: HBASE-12158
> URL: https://issues.apache.org/jira/browse/HBASE-12158
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 12158.txt
>
>
> Jetty server won't go down when running testStartedServerWithRequestLog. Its
> stuck in the stop. I tried to fix this before, HBASE-12025. See below how
> SelectorManager is trying to get lock held by the call to stop on the
> SelectorManager.
> {code}
> "1956684235@qtp-1078912108-1 - Acceptor0
> SelectChannelConnector@localhost:58604" daemon prio=10 tid=0x7f11cc896000
> nid=0x5c36 waiting for monitor entry [0x7f116fbdf000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
> - waiting to lock <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> "1786126004@qtp-1078912108-0" daemon prio=10 tid=0x7f11cc865800
> nid=0x5c35 in Object.wait() [0x7f118c17c000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
> - locked <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> "Service Thread" daemon prio=10 tid=0x7f11cc29 nid=0x5c18 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" daemon prio=10 tid=0x7f11cc28e000 nid=0x5c17 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" daemon prio=10 tid=0x7f11cc28b000 nid=0x5c16 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=10 tid=0x7f11cc280800 nid=0x5c15 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=10 tid=0x7f11cc26b000 nid=0x5c14 in Object.wait()
> [0x7f118cd3d000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
> "Reference Handler" daemon prio=10 tid=0x7f11cc267000 nid=0x5c13 in
> Object.wait() [0x7f118ce3e000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> "main" prio=10 tid=0x7f11cc00a800 nid=0x5c04 runnable [0x7f11d367e000]
>java.lang.Thread.State: RUNNABLE
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:879)
> - locked <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> - locked <0x0007dd3490a0> (a java.lang.Object)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
> - locked <0x0007dd348f00> (a
> org.mortbay.jetty.nio.Se
[jira] [Commented] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[
https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156930#comment-14156930
]
Hadoop QA commented on HBASE-12156:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672570/HBASE-12156.patch
against trunk revision .
ATTACHMENT ID: 12672570
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11189//console
This message is automatically generated.
> TableName cache doesn't used for once of valueOf methods.
> -
>
> Key: HBASE-12156
> URL: https://issues.apache.org/jira/browse/HBASE-12156
> Project: HBase
> Issue Type: Bug
>Reporter: Andrey Stepachev
>Assignee: Andrey Stepachev
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: HBASE-12156-addendum-0.98.patch, HBASE-12156.patch
>
>
> there is wrong comparison, copy&paste code compares namespace with qualifier
> and namespace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12148: -- Attachment: 12148v2.txt Review please. > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.99.1 > > Attachments: 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, Screen > Shot 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156935#comment-14156935 ] Hudson commented on HBASE-12156: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #532 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/532/]) Amend HBASE-12156 TableName cache doesn't used for once of valueOf methods (Andrey Stepachev) (apurtell: rev 1e673a0f0a7018a796526b6050a742f1a1de3d63) * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156-addendum-0.98.patch, HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156934#comment-14156934
]
Hudson commented on HBASE-12152:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #532 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/532/])
HBASE-12152 TestLoadIncrementalHFiles shows up as zombie test; ADD TIMEOUT ON
TESTS (apurtell: rev b5360829398922ceccc87fa24ba901543ed4db32)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10153) improve VerifyReplication to compute BADROWS more accurately
[
https://issues.apache.org/jira/browse/HBASE-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156940#comment-14156940
]
Hadoop QA commented on HBASE-10153:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672577/10153-v2-trunk.txt
against trunk revision .
ATTACHMENT ID: 12672577
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11190//console
This message is automatically generated.
> improve VerifyReplication to compute BADROWS more accurately
>
>
> Key: HBASE-10153
> URL: https://issues.apache.org/jira/browse/HBASE-10153
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 0.94.14
>Reporter: cuijianwei
>Assignee: cuijianwei
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 10153-v2-trunk.txt, HBASE-10153-0.94-v1.patch,
> HBASE-10153-trunk.patch
>
>
> VerifyReplicaiton could compare the source table with its peer table and
> compute BADROWS. However, the current BADROWS computing method might not be
> accurate enough. For example, if source table contains rows as {r1, r2, r3,
> r4} and peer table contains rows as {r1, r3, r4} BADROWS will be 3 because
> 'r2' in source table will make all the later row comparisons fail. Will it be
> better if the BADROWS is computed to 1 in this situation? Maybe, we can
> compute the BADROWS more accurately in merge comparison?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12149) TestRegionPlacement is failing undeterministically
[
https://issues.apache.org/jira/browse/HBASE-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12149:
--
Resolution: Fixed
Fix Version/s: (was: 1.0.0)
0.99.1
2.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Lets try it [~manukranthk] I applied to branch-1+ Thanks.
> TestRegionPlacement is failing undeterministically
> --
>
> Key: HBASE-12149
> URL: https://issues.apache.org/jira/browse/HBASE-12149
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Minor
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 0001-Split-TestRegionPlacement.patch,
> 0001-Split-TestRegionPlacement.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> TestRegionPlacement has tests that are flaky and fail in a cascading fashion.
> We need to fix this cascading behavior and also fix the flakyness of the
> tests.
> 16:20:59 java.lang.AssertionError: Region {ENCODED =>
> 958cbbb4405e3bbcfd04185538912d4c, NAME =>
> 'testRegionAssignment,\x02\x02\x02,1412205636586.958cbbb4405e3bbcfd04185538912d4c.',
> STARTKEY => '\x02\x02\x02', ENDKEY => '\x03\x03\x03'} not present on any of
> the expected servers
> 16:20:59 at org.junit.Assert.fail(Assert.java:88)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.killRandomServerAndVerifyAssignment(TestRegionPlacement.java:347)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testRegionPlacement(TestRegionPlacement.java:270)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRoundRobinAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.047 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRoundRobinAssignment(TestRegionPlacement.java:113)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRandomAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.041 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRandomAssignment(TestRegionPlacement.java:173)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12154) TestLoadAndSwitchEncodeOnDisk is flaky on shared jenkins boxes
[ https://issues.apache.org/jira/browse/HBASE-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156945#comment-14156945 ] Manukranth Kolloju commented on HBASE-12154: [~stack], well there should be a solution for that problem. > TestLoadAndSwitchEncodeOnDisk is flaky on shared jenkins boxes > -- > > Key: HBASE-12154 > URL: https://issues.apache.org/jira/browse/HBASE-12154 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Minor > Fix For: 1.0.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > It doesn't make sense to run a load test on a shared box where other tests > are being run. We should probably move this to integration tests and make > sure its covered. In cases where the jenkins machines are shared across > several projects which are doing disk IO, I have observed that this test runs > into slow syncs and eventually times out. And that being said, we can't > increase the timeout on this test, since its already 3 minutes. This test > being disk sensitive, might have better coverage on IT. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12153) Fixing TestReplicaWithCluster
[ https://issues.apache.org/jira/browse/HBASE-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156950#comment-14156950 ] Manukranth Kolloju commented on HBASE-12153: Thanks for committing [~stack] > Fixing TestReplicaWithCluster > - > > Key: HBASE-12153 > URL: https://issues.apache.org/jira/browse/HBASE-12153 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.99.1 > > Attachments: 0001-FixTestReplicaWithCluster.patch > > > This test takes about 30 ~ 40 seconds depending upon the resources available. > Doesn't make sense to have such a tight bound(30s) on the unit test. > [~nkeywal], what do you think ? Did you intend to have such a tight bound > while adding the test here ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12149) TestRegionPlacement is failing undeterministically
[
https://issues.apache.org/jira/browse/HBASE-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156949#comment-14156949
]
Manukranth Kolloju commented on HBASE-12149:
Thanks [~stack].
> TestRegionPlacement is failing undeterministically
> --
>
> Key: HBASE-12149
> URL: https://issues.apache.org/jira/browse/HBASE-12149
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Minor
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 0001-Split-TestRegionPlacement.patch,
> 0001-Split-TestRegionPlacement.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> TestRegionPlacement has tests that are flaky and fail in a cascading fashion.
> We need to fix this cascading behavior and also fix the flakyness of the
> tests.
> 16:20:59 java.lang.AssertionError: Region {ENCODED =>
> 958cbbb4405e3bbcfd04185538912d4c, NAME =>
> 'testRegionAssignment,\x02\x02\x02,1412205636586.958cbbb4405e3bbcfd04185538912d4c.',
> STARTKEY => '\x02\x02\x02', ENDKEY => '\x03\x03\x03'} not present on any of
> the expected servers
> 16:20:59 at org.junit.Assert.fail(Assert.java:88)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.killRandomServerAndVerifyAssignment(TestRegionPlacement.java:347)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testRegionPlacement(TestRegionPlacement.java:270)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRoundRobinAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.047 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRoundRobinAssignment(TestRegionPlacement.java:113)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRandomAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.041 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRandomAssignment(TestRegionPlacement.java:173)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-9005) Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete markers
[ https://issues.apache.org/jira/browse/HBASE-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9005: -- Assignee: Misty Stanley-Jones (was: Jonathan Hsieh) sorry about that [~misty] -- i must have accidentally assigned to myself. > Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete > markers > - > > Key: HBASE-9005 > URL: https://issues.apache.org/jira/browse/HBASE-9005 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Lars Hofhansl >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.1 > > Attachments: 9005.txt, HBASE-9005-1.patch, HBASE-9005-v1.patch > > > Without KEEP_DELETED_CELLS all timerange queries are broken if their range > covers a delete marker. > As some internal discussions with colleagues showed, this feature is not well > understand and documented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9005) Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete markers
[ https://issues.apache.org/jira/browse/HBASE-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156958#comment-14156958 ] Jonathan Hsieh commented on HBASE-9005: --- +1, lgtm. > Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete > markers > - > > Key: HBASE-9005 > URL: https://issues.apache.org/jira/browse/HBASE-9005 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Lars Hofhansl >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.1 > > Attachments: 9005.txt, HBASE-9005-1.patch, HBASE-9005-v1.patch > > > Without KEEP_DELETED_CELLS all timerange queries are broken if their range > covers a delete marker. > As some internal discussions with colleagues showed, this feature is not well > understand and documented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156970#comment-14156970 ] Yi Deng commented on HBASE-12133: - [~stack] Did it break the compiling on Jenkins? [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hbase-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/HBase-TRUNK/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestFastLongHistogram.java:[23,30] error: cannot find symbol [ERROR] symbol: class SmallTests [ERROR] location: package org.apache.hadoop.hbase [ERROR] /home/jenkins/jenkins-slave/workspace/HBase-TRUNK/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestFastLongHistogram.java:[31,12] error: cannot find symbol [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :hbase-common > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 2.0.0, 0.99.1 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12122) Try not to assign user regions to master all the time
[ https://issues.apache.org/jira/browse/HBASE-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-12122: Attachment: hbase-12122_v2.patch Attached v2 that fixed some assignment issue, and only put configured regions on master. > Try not to assign user regions to master all the time > - > > Key: HBASE-12122 > URL: https://issues.apache.org/jira/browse/HBASE-12122 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: hbase-12122.patch, hbase-12122_v2.patch > > > Load balancer does a good job not to assign regions of tables not configured > to put on the active master. However, if there is no other region server, it > still assigns users regions to the master. This happens when all normal > region servers are crashed and recovering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156980#comment-14156980 ] stack commented on HBASE-12133: --- Thanks [~daviddengcn] Applied addendum. Let me attach it. > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 2.0.0, 0.99.1 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12133) Add FastLongHistogram for metric computation
[ https://issues.apache.org/jira/browse/HBASE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12133: -- Attachment: 12133.addendum.txt > Add FastLongHistogram for metric computation > > > Key: HBASE-12133 > URL: https://issues.apache.org/jira/browse/HBASE-12133 > Project: HBase > Issue Type: New Feature > Components: metrics >Affects Versions: 0.98.8 >Reporter: Yi Deng >Assignee: Yi Deng >Priority: Minor > Labels: histogram, metrics > Fix For: 2.0.0, 0.99.1 > > Attachments: > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 0001-Add-FastLongHistogram-for-fast-histogram-estimation.patch, > 12133.addendum.txt > > > FastLongHistogram is a thread-safe class that estimate distribution of data > and computes the quantiles. It's useful for computing aggregated metrics like > P99/P95. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12122) Try not to assign user regions to master all the time
[ https://issues.apache.org/jira/browse/HBASE-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156985#comment-14156985 ] stack commented on HBASE-12122: --- [~jxiang] will this fix the TestZooKeeper and TestDistributeLogReply failures we've been seeing do you think? > Try not to assign user regions to master all the time > - > > Key: HBASE-12122 > URL: https://issues.apache.org/jira/browse/HBASE-12122 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: hbase-12122.patch, hbase-12122_v2.patch > > > Load balancer does a good job not to assign regions of tables not configured > to put on the active master. However, if there is no other region server, it > still assigns users regions to the master. This happens when all normal > region servers are crashed and recovering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156993#comment-14156993 ] Matteo Bertozzi commented on HBASE-12148: - +1 looks good to me > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.99.1 > > Attachments: 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, Screen > Shot 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12104) Some optimization and bugfix for HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Deng updated HBASE-12104: Attachment: 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch Fix the constant name and values. > Some optimization and bugfix for HTableMultiplexer > -- > > Key: HBASE-12104 > URL: https://issues.apache.org/jira/browse/HBASE-12104 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 2.0.0 >Reporter: Yi Deng >Assignee: Yi Deng > Labels: multiplexer > Fix For: 2.0.0 > > Attachments: > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public.patch > > > Make HTableMultiplexerStatus public > Delay before resubmit. > Fix some missing counting on total failure. > Use ScheduledExecutorService to simplify the code. > Other refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12158) TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on occasion
[
https://issues.apache.org/jira/browse/HBASE-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156997#comment-14156997
]
Hadoop QA commented on HBASE-12158:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672582/12158.txt
against trunk revision .
ATTACHMENT ID: 12672582
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11191//console
This message is automatically generated.
> TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on
> occasion
> ---
>
> Key: HBASE-12158
> URL: https://issues.apache.org/jira/browse/HBASE-12158
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 12158.txt
>
>
> Jetty server won't go down when running testStartedServerWithRequestLog. Its
> stuck in the stop. I tried to fix this before, HBASE-12025. See below how
> SelectorManager is trying to get lock held by the call to stop on the
> SelectorManager.
> {code}
> "1956684235@qtp-1078912108-1 - Acceptor0
> SelectChannelConnector@localhost:58604" daemon prio=10 tid=0x7f11cc896000
> nid=0x5c36 waiting for monitor entry [0x7f116fbdf000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
> - waiting to lock <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jet
[jira] [Commented] (HBASE-12153) Fixing TestReplicaWithCluster
[ https://issues.apache.org/jira/browse/HBASE-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156996#comment-14156996 ] Hudson commented on HBASE-12153: FAILURE: Integrated in HBase-1.0 #260 (See [https://builds.apache.org/job/HBase-1.0/260/]) HBASE-12153 Fixing TestReplicaWithCluster (Manukranth Kolloju) (stack: rev 5aea36d4d43cfae7206a96b0a5630143822b4d75) * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicaWithCluster.java > Fixing TestReplicaWithCluster > - > > Key: HBASE-12153 > URL: https://issues.apache.org/jira/browse/HBASE-12153 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.99.1 > > Attachments: 0001-FixTestReplicaWithCluster.patch > > > This test takes about 30 ~ 40 seconds depending upon the resources available. > Doesn't make sense to have such a tight bound(30s) on the unit test. > [~nkeywal], what do you think ? Did you intend to have such a tight bound > while adding the test here ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12104) Some optimization and bugfix for HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156998#comment-14156998 ] Yi Deng commented on HBASE-12104: - This diff is also available at https://reviews.facebook.net/D24321 > Some optimization and bugfix for HTableMultiplexer > -- > > Key: HBASE-12104 > URL: https://issues.apache.org/jira/browse/HBASE-12104 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 2.0.0 >Reporter: Yi Deng >Assignee: Yi Deng > Labels: multiplexer > Fix For: 2.0.0 > > Attachments: > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public.patch > > > Make HTableMultiplexerStatus public > Delay before resubmit. > Fix some missing counting on total failure. > Use ScheduledExecutorService to simplify the code. > Other refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12122) Try not to assign user regions to master all the time
[
https://issues.apache.org/jira/browse/HBASE-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157001#comment-14157001
]
stack commented on HBASE-12122:
---
Only meta table needs this treatment below? Not namespace table?
1273 RegionStates regionStates =
master.assignmentManager.getRegionStates();
1274 if (!(TableName.META_TABLE_NAME.equals(tableName)
1275 &&
regionStates.getRegionState(HRegionInfo.FIRST_META_REGIONINFO) != null)
1276&& !master.assignmentManager.isFailoverCleanupDone()) {
When this is called, assignMasterRegions, I don't see checks for if master is
supposed to host region or not? Should there be a check here? Or it happens
higher up?
Is this code duplicated?
1201Map> assignments
1202 = assignMasterRegions(regions.keySet(), servers);
1203if (assignments != null && !assignments.isEmpty()) {
1204 servers = new ArrayList(servers);
1205 // Guarantee not to put other regions on master
1206 servers.remove(masterServerName);
1207 List masterRegions =
assignments.get(masterServerName);
1208 if (!masterRegions.isEmpty()) {
1209regions = new HashMap(regions);
1210for (HRegionInfo region: masterRegions) {
1211 regions.remove(region);
1212}
1213 }
1214}
Looks like lots of nice cleanup of special-casing for master.
You think it will help w/ some of our test failures?
Thanks [~jxiang]
> Try not to assign user regions to master all the time
> -
>
> Key: HBASE-12122
> URL: https://issues.apache.org/jira/browse/HBASE-12122
> Project: HBase
> Issue Type: Bug
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
>Priority: Minor
> Fix For: 2.0.0, 0.99.1
>
> Attachments: hbase-12122.patch, hbase-12122_v2.patch
>
>
> Load balancer does a good job not to assign regions of tables not configured
> to put on the active master. However, if there is no other region server, it
> still assigns users regions to the master. This happens when all normal
> region servers are crashed and recovering.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12158) TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on occasion
[
https://issues.apache.org/jira/browse/HBASE-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157009#comment-14157009
]
Hudson commented on HBASE-12158:
FAILURE: Integrated in HBase-1.0 #261 (See
[https://builds.apache.org/job/HBase-1.0/261/])
HBASE-12158 TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie
on occasion (stack: rev 0be854d4561380732353da5f84dcc25449ee9606)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/http/TestHttpServerLifecycle.java
> TestHttpServerLifecycle.testStartedServerWithRequestLog goes zombie on
> occasion
> ---
>
> Key: HBASE-12158
> URL: https://issues.apache.org/jira/browse/HBASE-12158
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 12158.txt
>
>
> Jetty server won't go down when running testStartedServerWithRequestLog. Its
> stuck in the stop. I tried to fix this before, HBASE-12025. See below how
> SelectorManager is trying to get lock held by the call to stop on the
> SelectorManager.
> {code}
> "1956684235@qtp-1078912108-1 - Acceptor0
> SelectChannelConnector@localhost:58604" daemon prio=10 tid=0x7f11cc896000
> nid=0x5c36 waiting for monitor entry [0x7f116fbdf000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:464)
> - waiting to lock <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
> at
> org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> "1786126004@qtp-1078912108-0" daemon prio=10 tid=0x7f11cc865800
> nid=0x5c35 in Object.wait() [0x7f118c17c000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
> - locked <0x0007dd358460> (a
> org.mortbay.thread.QueuedThreadPool$PoolThread)
> "Service Thread" daemon prio=10 tid=0x7f11cc29 nid=0x5c18 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" daemon prio=10 tid=0x7f11cc28e000 nid=0x5c17 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" daemon prio=10 tid=0x7f11cc28b000 nid=0x5c16 waiting
> on condition [0x]
>java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=10 tid=0x7f11cc280800 nid=0x5c15 runnable
> [0x]
>java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=10 tid=0x7f11cc26b000 nid=0x5c14 in Object.wait()
> [0x7f118cd3d000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x0007d8685568> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
> "Reference Handler" daemon prio=10 tid=0x7f11cc267000 nid=0x5c13 in
> Object.wait() [0x7f118ce3e000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x0007d86850f0> (a java.lang.ref.Reference$Lock)
> "main" prio=10 tid=0x7f11cc00a800 nid=0x5c04 runnable [0x7f11d367e000]
>java.lang.Thread.State: RUNNABLE
> at
> org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:879)
> - locked <0x0007dd57c560> (a
> org.mortbay.io.nio.SelectorManager$SelectSet)
> at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:240)
> at
> org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
> - locked <0x0007dd3490a0> (a java.lang.Object)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
> - locked <0x0007dd348f00> (a
> org.mortbay.jetty.nio.SelectCh
[jira] [Commented] (HBASE-12156) TableName cache doesn't used for once of valueOf methods.
[ https://issues.apache.org/jira/browse/HBASE-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157008#comment-14157008 ] Hudson commented on HBASE-12156: FAILURE: Integrated in HBase-1.0 #261 (See [https://builds.apache.org/job/HBase-1.0/261/]) HBASE-12156 TableName cache doesn't used for once of valueOf methods (Andrey Stepachev) (stack: rev 0dfe556f31c6dedc3d57501de0f0e0be9d3ce0a2) * hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java Amend HBASE-12156 TableName cache doesn't used for once of valueOf methods (Andrey Stepachev) (stack: rev 483d97cb6710d7a978594470d9eb289fd10dc4a5) * hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java > TableName cache doesn't used for once of valueOf methods. > - > > Key: HBASE-12156 > URL: https://issues.apache.org/jira/browse/HBASE-12156 > Project: HBase > Issue Type: Bug >Reporter: Andrey Stepachev >Assignee: Andrey Stepachev > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12156-addendum-0.98.patch, HBASE-12156.patch > > > there is wrong comparison, copy&paste code compares namespace with qualifier > and namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12149) TestRegionPlacement is failing undeterministically
[
https://issues.apache.org/jira/browse/HBASE-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157006#comment-14157006
]
Hudson commented on HBASE-12149:
FAILURE: Integrated in HBase-1.0 #261 (See
[https://builds.apache.org/job/HBase-1.0/261/])
HBASE-12149 TestRegionPlacement is failing undeterministically (Manukranth
Kolloju) (stack: rev 489380a847b8bf2c4304025cb27c365bcc8867c1)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRegionPlacement2.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRegionPlacement.java
> TestRegionPlacement is failing undeterministically
> --
>
> Key: HBASE-12149
> URL: https://issues.apache.org/jira/browse/HBASE-12149
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Minor
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 0001-Split-TestRegionPlacement.patch,
> 0001-Split-TestRegionPlacement.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> TestRegionPlacement has tests that are flaky and fail in a cascading fashion.
> We need to fix this cascading behavior and also fix the flakyness of the
> tests.
> 16:20:59 java.lang.AssertionError: Region {ENCODED =>
> 958cbbb4405e3bbcfd04185538912d4c, NAME =>
> 'testRegionAssignment,\x02\x02\x02,1412205636586.958cbbb4405e3bbcfd04185538912d4c.',
> STARTKEY => '\x02\x02\x02', ENDKEY => '\x03\x03\x03'} not present on any of
> the expected servers
> 16:20:59 at org.junit.Assert.fail(Assert.java:88)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.killRandomServerAndVerifyAssignment(TestRegionPlacement.java:347)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testRegionPlacement(TestRegionPlacement.java:270)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRoundRobinAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.047 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRoundRobinAssignment(TestRegionPlacement.java:113)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRandomAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.041 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRandomAssignment(TestRegionPlacement.java:173)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157007#comment-14157007
]
Hudson commented on HBASE-12152:
FAILURE: Integrated in HBase-1.0 #261 (See
[https://builds.apache.org/job/HBase-1.0/261/])
HBASE-12152 TestLoadIncrementalHFiles shows up as zombie test; ADD TIMEOUT ON
TESTS (stack: rev 4598628ef27f5c4b205f44fcd38e5ac9136f52e1)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12122) Try not to assign user regions to master all the time
[ https://issues.apache.org/jira/browse/HBASE-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-12122: Status: Patch Available (was: Open) > Try not to assign user regions to master all the time > - > > Key: HBASE-12122 > URL: https://issues.apache.org/jira/browse/HBASE-12122 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: hbase-12122.patch, hbase-12122_v2.patch > > > Load balancer does a good job not to assign regions of tables not configured > to put on the active master. However, if there is no other region server, it > still assigns users regions to the master. This happens when all normal > region servers are crashed and recovering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12122) Try not to assign user regions to master all the time
[ https://issues.apache.org/jira/browse/HBASE-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157022#comment-14157022 ] Jimmy Xiang commented on HBASE-12122: - bq. will this fix the TestZooKeeper and TestDistributeLogReply failures we've been seeing do you think? TestZooKeeper should be fixed. bq. Only meta table needs this treatment below? Not namespace table? Right, now meta table, not other tables. bq. When this is called, assignMasterRegions, I don't see checks for if master is supposed to host region or not? Should there be a check here? Or it happens higher up? Those regions are handled by assignMasterRegions, so need to check again. bq. Is this code duplicated? Yes, let me fix it. > Try not to assign user regions to master all the time > - > > Key: HBASE-12122 > URL: https://issues.apache.org/jira/browse/HBASE-12122 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: hbase-12122.patch, hbase-12122_v2.patch > > > Load balancer does a good job not to assign regions of tables not configured > to put on the active master. However, if there is no other region server, it > still assigns users regions to the master. This happens when all normal > region servers are crashed and recovering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12149) TestRegionPlacement is failing undeterministically
[
https://issues.apache.org/jira/browse/HBASE-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157027#comment-14157027
]
stack commented on HBASE-12149:
---
Pushed an addendum. The test categorization being in different packages messes
up cherry-pick. Addendum on branch-1 is ede5c63
> TestRegionPlacement is failing undeterministically
> --
>
> Key: HBASE-12149
> URL: https://issues.apache.org/jira/browse/HBASE-12149
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Minor
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 0001-Split-TestRegionPlacement.patch,
> 0001-Split-TestRegionPlacement.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> TestRegionPlacement has tests that are flaky and fail in a cascading fashion.
> We need to fix this cascading behavior and also fix the flakyness of the
> tests.
> 16:20:59 java.lang.AssertionError: Region {ENCODED =>
> 958cbbb4405e3bbcfd04185538912d4c, NAME =>
> 'testRegionAssignment,\x02\x02\x02,1412205636586.958cbbb4405e3bbcfd04185538912d4c.',
> STARTKEY => '\x02\x02\x02', ENDKEY => '\x03\x03\x03'} not present on any of
> the expected servers
> 16:20:59 at org.junit.Assert.fail(Assert.java:88)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.killRandomServerAndVerifyAssignment(TestRegionPlacement.java:347)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testRegionPlacement(TestRegionPlacement.java:270)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRoundRobinAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.047 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRoundRobinAssignment(TestRegionPlacement.java:113)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRandomAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.041 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRandomAssignment(TestRegionPlacement.java:173)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12159) Shell command to verify snapshots
Patrick White created HBASE-12159: - Summary: Shell command to verify snapshots Key: HBASE-12159 URL: https://issues.apache.org/jira/browse/HBASE-12159 Project: HBase Issue Type: New Feature Components: shell Affects Versions: 0.98.6.1 Reporter: Patrick White Priority: Minor Needed an admin command that exposes the snapshot verification code as a command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12159) Shell command to verify snapshots
[ https://issues.apache.org/jira/browse/HBASE-12159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157043#comment-14157043 ] Matteo Bertozzi commented on HBASE-12159: - $ hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo isn't the tool enough? > Shell command to verify snapshots > - > > Key: HBASE-12159 > URL: https://issues.apache.org/jira/browse/HBASE-12159 > Project: HBase > Issue Type: New Feature > Components: shell >Affects Versions: 0.98.6.1 >Reporter: Patrick White >Priority: Minor > > Needed an admin command that exposes the snapshot verification code as a > command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12148: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-1+ Thanks [~mbertozzi] > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.99.1 > > Attachments: 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, Screen > Shot 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12104) Some optimization and bugfix for HTableMultiplexer
[
https://issues.apache.org/jira/browse/HBASE-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157050#comment-14157050
]
Hadoop QA commented on HBASE-12104:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672608/0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch
against trunk revision .
ATTACHMENT ID: 12672608
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAsyncProcess
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11193//console
This message is automatically generated.
> Some optimization and bugfix for HTableMultiplexer
> --
>
> Key: HBASE-12104
> URL: https://issues.apache.org/jira/browse/HBASE-12104
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Yi Deng
>Assignee: Yi Deng
> Labels: multiplexer
> Fix For: 2.0.0
>
> Attachments:
> 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch,
> 0001-Make-HTableMultiplexerStatus-public.patch
>
>
> Make HTableMultiplexerStatus public
> Delay before resubmit.
> Fix some missing counting on total failure.
> Use ScheduledExecutorService to simplify the code.
> Other refactoring.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12159) Shell command to verify snapshots
[ https://issues.apache.org/jira/browse/HBASE-12159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157048#comment-14157048 ] Patrick White commented on HBASE-12159: --- I want to say I had a really good reason for this, but I wrote it last week. Let me double check > Shell command to verify snapshots > - > > Key: HBASE-12159 > URL: https://issues.apache.org/jira/browse/HBASE-12159 > Project: HBase > Issue Type: New Feature > Components: shell >Affects Versions: 0.98.6.1 >Reporter: Patrick White >Priority: Minor > > Needed an admin command that exposes the snapshot verification code as a > command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12122) Try not to assign user regions to master all the time
[ https://issues.apache.org/jira/browse/HBASE-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157057#comment-14157057 ] stack commented on HBASE-12122: --- Ok. +1 Commit when you fix code dup [~jxiang] > Try not to assign user regions to master all the time > - > > Key: HBASE-12122 > URL: https://issues.apache.org/jira/browse/HBASE-12122 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: hbase-12122.patch, hbase-12122_v2.patch > > > Load balancer does a good job not to assign regions of tables not configured > to put on the active master. However, if there is no other region server, it > still assigns users regions to the master. This happens when all normal > region servers are crashed and recovering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12149) TestRegionPlacement is failing undeterministically
[
https://issues.apache.org/jira/browse/HBASE-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157073#comment-14157073
]
Hudson commented on HBASE-12149:
FAILURE: Integrated in HBase-1.0 #262 (See
[https://builds.apache.org/job/HBase-1.0/262/])
HBASE-12149 TestRegionPlacement is failing undeterministically (Manukranth
Kolloju) ADDENDUM (stack: rev ede5c6326a24455ad02455fc1fb6a2682b3fdac3)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRegionPlacement2.java
> TestRegionPlacement is failing undeterministically
> --
>
> Key: HBASE-12149
> URL: https://issues.apache.org/jira/browse/HBASE-12149
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Minor
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 0001-Split-TestRegionPlacement.patch,
> 0001-Split-TestRegionPlacement.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> TestRegionPlacement has tests that are flaky and fail in a cascading fashion.
> We need to fix this cascading behavior and also fix the flakyness of the
> tests.
> 16:20:59 java.lang.AssertionError: Region {ENCODED =>
> 958cbbb4405e3bbcfd04185538912d4c, NAME =>
> 'testRegionAssignment,\x02\x02\x02,1412205636586.958cbbb4405e3bbcfd04185538912d4c.',
> STARTKEY => '\x02\x02\x02', ENDKEY => '\x03\x03\x03'} not present on any of
> the expected servers
> 16:20:59 at org.junit.Assert.fail(Assert.java:88)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.killRandomServerAndVerifyAssignment(TestRegionPlacement.java:347)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testRegionPlacement(TestRegionPlacement.java:270)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRoundRobinAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.047 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRoundRobinAssignment(TestRegionPlacement.java:113)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRandomAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.041 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRandomAssignment(TestRegionPlacement.java:173)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157078#comment-14157078
]
Ted Yu commented on HBASE-12152:
I got the following test failure:
{code}
testGroupOrSplitPresplit(org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery)
Time elapsed: 0.147 sec <<< ERROR!
java.lang.Exception: test timed out after 120 milliseconds
at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1087)
at
org.apache.hadoop.hbase.client.HTable.getDefaultExecutor(HTable.java:205)
at org.apache.hadoop.hbase.client.HTable.(HTable.java:302)
at org.apache.hadoop.hbase.client.HTable.(HTable.java:280)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:145)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:86)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:538)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:430)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.setupTable(TestLoadIncrementalHFilesSplitRecovery.java:132)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitPresplit(TestLoadIncrementalHFilesSplitRecovery.java:380)
{code}
Looks like timeout was specified as 120 milliseconds
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
