[jira] [Commented] (HBASE-11907) Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator
[ https://issues.apache.org/jira/browse/HBASE-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157726#comment-14157726 ] Hudson commented on HBASE-11907: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #538 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/538/]) HBASE-11907 Use the joni byte[] regex engine in place of j.u.regex (apurtell: rev 579ce7a0d610352a7bcff5527ce24b04e8b2292a) * hbase-protocol/src/main/protobuf/Comparator.proto * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestRegexComparator.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ComparatorProtos.java * hbase-client/pom.xml * pom.xml * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/RegexStringComparator.java > Use the joni byte[] regex engine in place of j.u.regex in > RegexStringComparator > --- > > Key: HBASE-11907 > URL: https://issues.apache.org/jira/browse/HBASE-11907 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-11907.patch, HBASE-11907.patch, HBASE-11907.patch, > HBASE-11907.patch > > > The joni regex engine (https://github.com/jruby/joni), a Java port of > Oniguruma regexp library done by the JRuby project, is: > - MIT licensed > - Designed to work with byte[] arguments instead of String > - Capable of handling UTF8 encoding > - Regex syntax compatible > - Interruptible > - *About twice as fast as j.u.regex* > - Has JRuby's jcodings library as a dependency, also MIT licensed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157717#comment-14157717
]
Hudson commented on HBASE-12164:
FAILURE: Integrated in HBase-TRUNK #5613 (See
[https://builds.apache.org/job/HBase-TRUNK/5613/])
HBASE-12164 Check for presence of user Id in
SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate (tedyu: rev
a17614d5b27936c64af47d90408df007b1112d89)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
>
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 12164-v1.txt, 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10153) improve VerifyReplication to compute BADROWS more accurately
[
https://issues.apache.org/jira/browse/HBASE-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157719#comment-14157719
]
Hudson commented on HBASE-10153:
FAILURE: Integrated in HBase-TRUNK #5613 (See
[https://builds.apache.org/job/HBase-TRUNK/5613/])
HBASE-10153 improve VerifyReplication to compute BADROWS more accurately
(Jianwei) (tedyu: rev 8dbf7b22381dab18f9af13318c16181c42824d46)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
> improve VerifyReplication to compute BADROWS more accurately
>
>
> Key: HBASE-10153
> URL: https://issues.apache.org/jira/browse/HBASE-10153
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 0.94.14
>Reporter: cuijianwei
>Assignee: cuijianwei
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 10153-0.98.txt, 10153-v2-trunk.txt,
> HBASE-10153-0.94-v1.patch, HBASE-10153-trunk.patch
>
>
> VerifyReplicaiton could compare the source table with its peer table and
> compute BADROWS. However, the current BADROWS computing method might not be
> accurate enough. For example, if source table contains rows as {r1, r2, r3,
> r4} and peer table contains rows as {r1, r3, r4} BADROWS will be 3 because
> 'r2' in source table will make all the later row comparisons fail. Will it be
> better if the BADROWS is computed to 1 in this situation? Maybe, we can
> compute the BADROWS more accurately in merge comparison?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12151) Make dev scripts executable
[ https://issues.apache.org/jira/browse/HBASE-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157718#comment-14157718 ] Hudson commented on HBASE-12151: FAILURE: Integrated in HBase-TRUNK #5613 (See [https://builds.apache.org/job/HBase-TRUNK/5613/]) HBASE-12151 Set mode to 755 on executable scripts in dev-support directory (mstanleyjones: rev 7219471081ab5f65ad7ae3b2deeb3c1659922102) * dev-support/jdiffHBasePublicAPI_common.sh * dev-support/publish_hbase_website.sh * dev-support/jenkinsEnv.sh * dev-support/hbase_docker.sh > Make dev scripts executable > --- > > Key: HBASE-12151 > URL: https://issues.apache.org/jira/browse/HBASE-12151 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12151.patch > > > Is there any reason not to make dev-support/*.sh executable? It would make it > possible to sym-link to them from a directory in the executable path for > easier execution of the definitive scripts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11907) Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator
[ https://issues.apache.org/jira/browse/HBASE-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11907: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to 0.98+ > Use the joni byte[] regex engine in place of j.u.regex in > RegexStringComparator > --- > > Key: HBASE-11907 > URL: https://issues.apache.org/jira/browse/HBASE-11907 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-11907.patch, HBASE-11907.patch, HBASE-11907.patch, > HBASE-11907.patch > > > The joni regex engine (https://github.com/jruby/joni), a Java port of > Oniguruma regexp library done by the JRuby project, is: > - MIT licensed > - Designed to work with byte[] arguments instead of String > - Capable of handling UTF8 encoding > - Regex syntax compatible > - Interruptible > - *About twice as fast as j.u.regex* > - Has JRuby's jcodings library as a dependency, also MIT licensed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12166) TestDistributedLogSplitting.testMasterStartsUpWithLogReplayWork
[
https://issues.apache.org/jira/browse/HBASE-12166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12166:
--
Attachment: 12166.txt
A bit of cleanup while in here.
> TestDistributedLogSplitting.testMasterStartsUpWithLogReplayWork
> ---
>
> Key: HBASE-12166
> URL: https://issues.apache.org/jira/browse/HBASE-12166
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 12166.txt, log.txt
>
>
> See
> https://builds.apache.org/job/PreCommit-HBASE-Build/11204//testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testMasterStartsUpWithLogReplayWork/
> The namespace region gets stuck. It is never 'recovered' even though we have
> finished log splitting. Here is the main exception:
> {code}
> 4941 2014-10-03 02:00:36,862 DEBUG
> [B.defaultRpcServer.handler=1,queue=0,port=37113] ipc.CallRunner(111):
> B.defaultRpcServer.handler=1,queue=0,port=37113: callId: 211 service:
> ClientService methodName: Get
> size: 99 connection: 67.195.81.144:44526
> 4942 org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:namespace,,1412301462277.eba5d23de65f2718715eeb22edf7edc2. is recovering
> 4943 at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:6058)
> 4944 at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2086)
> 4945 at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2072)
> 4946 at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:5014)
> 4947 at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4988)
> 4948 at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1690)
> 4949 at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30418)
> 4950 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2020)
> 4951 at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> 4952 at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
> 4953 at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
> 4954 at java.lang.Thread.run(Thread.java:744)
> {code}
> See how we've finished log splitting long time previous:
> {code}
> 2014-10-03 01:57:48,129 INFO [M_LOG_REPLAY_OPS-asf900:37113-1]
> master.SplitLogManager(294): finished splitting (more than or equal to)
> 197337 bytes in 1 log files in
> [hdfs://localhost:49601/user/jenkins/hbase/WALs/asf900.gq1.ygridcore.net,40732,1412301461887-splitting]
> in 379ms
> {code}
> If I grep for the deleting of znodes on recovery, which is when we set the
> recovering flag to false, I see a bunch of regions but not my namespace one:
> 2014-10-03 01:57:47,330 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66): /hbase/recovering-regions/1588230740
> znode deleted. Region: 1588230740 completes recovery.
> 2014-10-03 01:57:48,119 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/adfdcf958dd958f0e2ce59072ce2209d znode deleted.
> Region: adfdcf958dd958f0e2ce59072ce2209d completes recovery.
> 2014-10-03 01:57:48,121 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/41d438848305831b61d708a406d5ecde znode deleted.
> Region: 41d438848305831b61d708a406d5ecde completes recovery.
> 2014-10-03 01:57:48,122 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/6a7cada80de2ae5d774fe8cd33bd4cda znode deleted.
> Region: 6a7cada80de2ae5d774fe8cd33bd4cda completes recovery.
> 2014-10-03 01:57:48,124 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/65451bd5b38bd16a31e25b62b3305533 znode deleted.
> Region: 65451bd5b38bd16a31e25b62b3305533 completes recovery.
> 2014-10-03 01:57:48,125 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/07afdc3748894cf2b56e0075272a95a0 znode deleted.
> Region: 07afdc3748894cf2b56e0075272a95a0 completes recovery.
> 2014-10-03 01:57:48,126 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/a4337ad2874ee7e599ca2344fce21583 znode deleted.
> Region: a4337ad2874ee7e599ca2344fce21583 completes recovery.
> 2014-10-03 01:57:48,128 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/9d91d6eafe260ce33e8d7d23ccd13192 znode deleted.
> Region: 9d91d6eafe260ce33e8d7d23ccd13192 completes recovery.
> This would seem to indicate that we successfully wrote zk that we are
> recovering:
> {cod
[jira] [Commented] (HBASE-10153) improve VerifyReplication to compute BADROWS more accurately
[
https://issues.apache.org/jira/browse/HBASE-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157713#comment-14157713
]
Hudson commented on HBASE-10153:
FAILURE: Integrated in HBase-1.0 #266 (See
[https://builds.apache.org/job/HBase-1.0/266/])
HBASE-10153 improve VerifyReplication to compute BADROWS more accurately
(Jianwei) (tedyu: rev a2fe4d6700c83a467b053f2d04115c69a27f3c79)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
> improve VerifyReplication to compute BADROWS more accurately
>
>
> Key: HBASE-10153
> URL: https://issues.apache.org/jira/browse/HBASE-10153
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 0.94.14
>Reporter: cuijianwei
>Assignee: cuijianwei
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 10153-0.98.txt, 10153-v2-trunk.txt,
> HBASE-10153-0.94-v1.patch, HBASE-10153-trunk.patch
>
>
> VerifyReplicaiton could compare the source table with its peer table and
> compute BADROWS. However, the current BADROWS computing method might not be
> accurate enough. For example, if source table contains rows as {r1, r2, r3,
> r4} and peer table contains rows as {r1, r3, r4} BADROWS will be 3 because
> 'r2' in source table will make all the later row comparisons fail. Will it be
> better if the BADROWS is computed to 1 in this situation? Maybe, we can
> compute the BADROWS more accurately in merge comparison?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12151) Make dev scripts executable
[ https://issues.apache.org/jira/browse/HBASE-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157712#comment-14157712 ] Hudson commented on HBASE-12151: FAILURE: Integrated in HBase-1.0 #266 (See [https://builds.apache.org/job/HBase-1.0/266/]) HBASE-12151 Set mode to 755 on executable scripts in dev-support directory (mstanleyjones: rev a43f111f0d9a98f6814cbf5ded08239ef8362da5) * dev-support/publish_hbase_website.sh * dev-support/jdiffHBasePublicAPI_common.sh * dev-support/jenkinsEnv.sh * dev-support/hbase_docker.sh > Make dev scripts executable > --- > > Key: HBASE-12151 > URL: https://issues.apache.org/jira/browse/HBASE-12151 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12151.patch > > > Is there any reason not to make dev-support/*.sh executable? It would make it > possible to sym-link to them from a directory in the executable path for > easier execution of the definitive scripts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12166) TestDistributedLogSplitting.testMasterStartsUpWithLogReplayWork
[
https://issues.apache.org/jira/browse/HBASE-12166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12166:
--
Attachment: log.txt
Here is log from a failed test.
> TestDistributedLogSplitting.testMasterStartsUpWithLogReplayWork
> ---
>
> Key: HBASE-12166
> URL: https://issues.apache.org/jira/browse/HBASE-12166
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 0.99.1
>
> Attachments: log.txt
>
>
> See
> https://builds.apache.org/job/PreCommit-HBASE-Build/11204//testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testMasterStartsUpWithLogReplayWork/
> The namespace region gets stuck. It is never 'recovered' even though we have
> finished log splitting. Here is the main exception:
> {code}
> 4941 2014-10-03 02:00:36,862 DEBUG
> [B.defaultRpcServer.handler=1,queue=0,port=37113] ipc.CallRunner(111):
> B.defaultRpcServer.handler=1,queue=0,port=37113: callId: 211 service:
> ClientService methodName: Get
> size: 99 connection: 67.195.81.144:44526
> 4942 org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
> hbase:namespace,,1412301462277.eba5d23de65f2718715eeb22edf7edc2. is recovering
> 4943 at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:6058)
> 4944 at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2086)
> 4945 at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2072)
> 4946 at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:5014)
> 4947 at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4988)
> 4948 at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1690)
> 4949 at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30418)
> 4950 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2020)
> 4951 at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> 4952 at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
> 4953 at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
> 4954 at java.lang.Thread.run(Thread.java:744)
> {code}
> See how we've finished log splitting long time previous:
> {code}
> 2014-10-03 01:57:48,129 INFO [M_LOG_REPLAY_OPS-asf900:37113-1]
> master.SplitLogManager(294): finished splitting (more than or equal to)
> 197337 bytes in 1 log files in
> [hdfs://localhost:49601/user/jenkins/hbase/WALs/asf900.gq1.ygridcore.net,40732,1412301461887-splitting]
> in 379ms
> {code}
> If I grep for the deleting of znodes on recovery, which is when we set the
> recovering flag to false, I see a bunch of regions but not my namespace one:
> 2014-10-03 01:57:47,330 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66): /hbase/recovering-regions/1588230740
> znode deleted. Region: 1588230740 completes recovery.
> 2014-10-03 01:57:48,119 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/adfdcf958dd958f0e2ce59072ce2209d znode deleted.
> Region: adfdcf958dd958f0e2ce59072ce2209d completes recovery.
> 2014-10-03 01:57:48,121 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/41d438848305831b61d708a406d5ecde znode deleted.
> Region: 41d438848305831b61d708a406d5ecde completes recovery.
> 2014-10-03 01:57:48,122 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/6a7cada80de2ae5d774fe8cd33bd4cda znode deleted.
> Region: 6a7cada80de2ae5d774fe8cd33bd4cda completes recovery.
> 2014-10-03 01:57:48,124 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/65451bd5b38bd16a31e25b62b3305533 znode deleted.
> Region: 65451bd5b38bd16a31e25b62b3305533 completes recovery.
> 2014-10-03 01:57:48,125 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/07afdc3748894cf2b56e0075272a95a0 znode deleted.
> Region: 07afdc3748894cf2b56e0075272a95a0 completes recovery.
> 2014-10-03 01:57:48,126 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/a4337ad2874ee7e599ca2344fce21583 znode deleted.
> Region: a4337ad2874ee7e599ca2344fce21583 completes recovery.
> 2014-10-03 01:57:48,128 INFO [Thread-9216-EventThread]
> zookeeper.RecoveringRegionWatcher(66):
> /hbase/recovering-regions/9d91d6eafe260ce33e8d7d23ccd13192 znode deleted.
> Region: 9d91d6eafe260ce33e8d7d23ccd13192 completes recovery.
> This would seem to indicate that we successfully wrote zk that we are
> recovering:
> {code}
> 2014-10-
[jira] [Created] (HBASE-12166) TestDistributedLogSplitting.testMasterStartsUpWithLogReplayWork
stack created HBASE-12166:
-
Summary:
TestDistributedLogSplitting.testMasterStartsUpWithLogReplayWork
Key: HBASE-12166
URL: https://issues.apache.org/jira/browse/HBASE-12166
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Fix For: 2.0.0, 0.99.1
See
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testMasterStartsUpWithLogReplayWork/
The namespace region gets stuck. It is never 'recovered' even though we have
finished log splitting. Here is the main exception:
{code}
4941 2014-10-03 02:00:36,862 DEBUG
[B.defaultRpcServer.handler=1,queue=0,port=37113] ipc.CallRunner(111):
B.defaultRpcServer.handler=1,queue=0,port=37113: callId: 211 service:
ClientService methodName: Get
size: 99 connection: 67.195.81.144:44526
4942 org.apache.hadoop.hbase.exceptions.RegionInRecoveryException:
hbase:namespace,,1412301462277.eba5d23de65f2718715eeb22edf7edc2. is recovering
4943 at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:6058)
4944 at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2086)
4945 at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2072)
4946 at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:5014)
4947 at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4988)
4948 at
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1690)
4949 at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30418)
4950 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2020)
4951 at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
4952 at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
4953 at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
4954 at java.lang.Thread.run(Thread.java:744)
{code}
See how we've finished log splitting long time previous:
{code}
2014-10-03 01:57:48,129 INFO [M_LOG_REPLAY_OPS-asf900:37113-1]
master.SplitLogManager(294): finished splitting (more than or equal to) 197337
bytes in 1 log files in
[hdfs://localhost:49601/user/jenkins/hbase/WALs/asf900.gq1.ygridcore.net,40732,1412301461887-splitting]
in 379ms
{code}
If I grep for the deleting of znodes on recovery, which is when we set the
recovering flag to false, I see a bunch of regions but not my namespace one:
2014-10-03 01:57:47,330 INFO [Thread-9216-EventThread]
zookeeper.RecoveringRegionWatcher(66): /hbase/recovering-regions/1588230740
znode deleted. Region: 1588230740 completes recovery.
2014-10-03 01:57:48,119 INFO [Thread-9216-EventThread]
zookeeper.RecoveringRegionWatcher(66):
/hbase/recovering-regions/adfdcf958dd958f0e2ce59072ce2209d znode deleted.
Region: adfdcf958dd958f0e2ce59072ce2209d completes recovery.
2014-10-03 01:57:48,121 INFO [Thread-9216-EventThread]
zookeeper.RecoveringRegionWatcher(66):
/hbase/recovering-regions/41d438848305831b61d708a406d5ecde znode deleted.
Region: 41d438848305831b61d708a406d5ecde completes recovery.
2014-10-03 01:57:48,122 INFO [Thread-9216-EventThread]
zookeeper.RecoveringRegionWatcher(66):
/hbase/recovering-regions/6a7cada80de2ae5d774fe8cd33bd4cda znode deleted.
Region: 6a7cada80de2ae5d774fe8cd33bd4cda completes recovery.
2014-10-03 01:57:48,124 INFO [Thread-9216-EventThread]
zookeeper.RecoveringRegionWatcher(66):
/hbase/recovering-regions/65451bd5b38bd16a31e25b62b3305533 znode deleted.
Region: 65451bd5b38bd16a31e25b62b3305533 completes recovery.
2014-10-03 01:57:48,125 INFO [Thread-9216-EventThread]
zookeeper.RecoveringRegionWatcher(66):
/hbase/recovering-regions/07afdc3748894cf2b56e0075272a95a0 znode deleted.
Region: 07afdc3748894cf2b56e0075272a95a0 completes recovery.
2014-10-03 01:57:48,126 INFO [Thread-9216-EventThread]
zookeeper.RecoveringRegionWatcher(66):
/hbase/recovering-regions/a4337ad2874ee7e599ca2344fce21583 znode deleted.
Region: a4337ad2874ee7e599ca2344fce21583 completes recovery.
2014-10-03 01:57:48,128 INFO [Thread-9216-EventThread]
zookeeper.RecoveringRegionWatcher(66):
/hbase/recovering-regions/9d91d6eafe260ce33e8d7d23ccd13192 znode deleted.
Region: 9d91d6eafe260ce33e8d7d23ccd13192 completes recovery.
This would seem to indicate that we successfully wrote zk that we are
recovering:
{code}
2014-10-03 01:57:47,672 DEBUG [MASTER_SERVER_OPERATIONS-asf900:37113-0]
coordination.ZKSplitLogManagerCoordination(647): Mark region
eba5d23de65f2718715eeb22edf7edc2 recovering from failed region server
asf900.gq1.ygridcore.net,42071,1412301461790
{code}
As part of the open of namespace, we updated our last flushed id successfully:
{code}
2ae5d774fe8cd33bd4cda/family
2014-10-03 01:57:47,698 DEBUG [Thread-9216-EventThr
[jira] [Commented] (HBASE-10153) improve VerifyReplication to compute BADROWS more accurately
[
https://issues.apache.org/jira/browse/HBASE-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157683#comment-14157683
]
Hudson commented on HBASE-10153:
FAILURE: Integrated in HBase-0.98 #565 (See
[https://builds.apache.org/job/HBase-0.98/565/])
HBASE-10153 improve VerifyReplication to compute BADROWS more accurately
(Jianwei) (tedyu: rev a3cfd5233dfbfdd57ac445acd0886df2f8bae895)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
> improve VerifyReplication to compute BADROWS more accurately
>
>
> Key: HBASE-10153
> URL: https://issues.apache.org/jira/browse/HBASE-10153
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 0.94.14
>Reporter: cuijianwei
>Assignee: cuijianwei
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 10153-0.98.txt, 10153-v2-trunk.txt,
> HBASE-10153-0.94-v1.patch, HBASE-10153-trunk.patch
>
>
> VerifyReplicaiton could compare the source table with its peer table and
> compute BADROWS. However, the current BADROWS computing method might not be
> accurate enough. For example, if source table contains rows as {r1, r2, r3,
> r4} and peer table contains rows as {r1, r3, r4} BADROWS will be 3 because
> 'r2' in source table will make all the later row comparisons fail. Will it be
> better if the BADROWS is computed to 1 in this situation? Maybe, we can
> compute the BADROWS more accurately in merge comparison?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10153) improve VerifyReplication to compute BADROWS more accurately
[
https://issues.apache.org/jira/browse/HBASE-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157666#comment-14157666
]
Hudson commented on HBASE-10153:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #537 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/537/])
HBASE-10153 improve VerifyReplication to compute BADROWS more accurately
(Jianwei) (tedyu: rev a3cfd5233dfbfdd57ac445acd0886df2f8bae895)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
> improve VerifyReplication to compute BADROWS more accurately
>
>
> Key: HBASE-10153
> URL: https://issues.apache.org/jira/browse/HBASE-10153
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 0.94.14
>Reporter: cuijianwei
>Assignee: cuijianwei
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 10153-0.98.txt, 10153-v2-trunk.txt,
> HBASE-10153-0.94-v1.patch, HBASE-10153-trunk.patch
>
>
> VerifyReplicaiton could compare the source table with its peer table and
> compute BADROWS. However, the current BADROWS computing method might not be
> accurate enough. For example, if source table contains rows as {r1, r2, r3,
> r4} and peer table contains rows as {r1, r3, r4} BADROWS will be 3 because
> 'r2' in source table will make all the later row comparisons fail. Will it be
> better if the BADROWS is computed to 1 in this situation? Maybe, we can
> compute the BADROWS more accurately in merge comparison?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157665#comment-14157665
]
Hudson commented on HBASE-12164:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #537 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/537/])
HBASE-12164 Check for presence of user Id in
SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate (tedyu: rev
0409d22a15d6656d0368b6343b7b3349d22bdd77)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
>
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 12164-v1.txt, 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12165) TestEndToEndSplitTransaction.testFromClientSideWhileSplitting fails
[ https://issues.apache.org/jira/browse/HBASE-12165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157660#comment-14157660 ] stack commented on HBASE-12165: --- Pushed debug. > TestEndToEndSplitTransaction.testFromClientSideWhileSplitting fails > --- > > Key: HBASE-12165 > URL: https://issues.apache.org/jira/browse/HBASE-12165 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.99.1 > > Attachments: 12165.debug.txt > > > Test fails but exhibited fail reason is complaining about an NPE. > java.lang.NullPointerException: null > at > org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.blockUntilRegionIsInMeta(TestEndToEndSplitTransaction.java:474) > at > org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.blockUntilRegionSplit(TestEndToEndSplitTransaction.java:451) > Looks like we are timing out waiting on split but NPE obscures actual reason > for failure. > Failed here > https://builds.apache.org/job/PreCommit-HBASE-Build/11204//testReport/org.apache.hadoop.hbase.regionserver/TestEndToEndSplitTransaction/testFromClientSideWhileSplitting/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12165) TestEndToEndSplitTransaction.testFromClientSideWhileSplitting fails
[ https://issues.apache.org/jira/browse/HBASE-12165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12165: -- Attachment: 12165.debug.txt Throw exceptions if daughters are null rather than NPE. Let me add this and other logging to see if can figure root cause. > TestEndToEndSplitTransaction.testFromClientSideWhileSplitting fails > --- > > Key: HBASE-12165 > URL: https://issues.apache.org/jira/browse/HBASE-12165 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.99.1 > > Attachments: 12165.debug.txt > > > Test fails but exhibited fail reason is complaining about an NPE. > java.lang.NullPointerException: null > at > org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.blockUntilRegionIsInMeta(TestEndToEndSplitTransaction.java:474) > at > org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.blockUntilRegionSplit(TestEndToEndSplitTransaction.java:451) > Looks like we are timing out waiting on split but NPE obscures actual reason > for failure. > Failed here > https://builds.apache.org/job/PreCommit-HBASE-Build/11204//testReport/org.apache.hadoop.hbase.regionserver/TestEndToEndSplitTransaction/testFromClientSideWhileSplitting/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12165) TestEndToEndSplitTransaction.testFromClientSideWhileSplitting fails
[ https://issues.apache.org/jira/browse/HBASE-12165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12165: -- Summary: TestEndToEndSplitTransaction.testFromClientSideWhileSplitting fails (was: TestDistributedLogSplitting.testMasterStartsUpWithLogReplayWork fails) > TestEndToEndSplitTransaction.testFromClientSideWhileSplitting fails > --- > > Key: HBASE-12165 > URL: https://issues.apache.org/jira/browse/HBASE-12165 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.99.1 > > > Test fails but exhibited fail reason is complaining about an NPE. > java.lang.NullPointerException: null > at > org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.blockUntilRegionIsInMeta(TestEndToEndSplitTransaction.java:474) > at > org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.blockUntilRegionSplit(TestEndToEndSplitTransaction.java:451) > Looks like we are timing out waiting on split but NPE obscures actual reason > for failure. > Failed here > https://builds.apache.org/job/PreCommit-HBASE-Build/11204//testReport/org.apache.hadoop.hbase.regionserver/TestEndToEndSplitTransaction/testFromClientSideWhileSplitting/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12165) TestDistributedLogSplitting.testMasterStartsUpWithLogReplayWork fails
stack created HBASE-12165: - Summary: TestDistributedLogSplitting.testMasterStartsUpWithLogReplayWork fails Key: HBASE-12165 URL: https://issues.apache.org/jira/browse/HBASE-12165 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: stack Assignee: stack Fix For: 2.0.0, 0.99.1 Test fails but exhibited fail reason is complaining about an NPE. java.lang.NullPointerException: null at org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.blockUntilRegionIsInMeta(TestEndToEndSplitTransaction.java:474) at org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.blockUntilRegionSplit(TestEndToEndSplitTransaction.java:451) Looks like we are timing out waiting on split but NPE obscures actual reason for failure. Failed here https://builds.apache.org/job/PreCommit-HBASE-Build/11204//testReport/org.apache.hadoop.hbase.regionserver/TestEndToEndSplitTransaction/testFromClientSideWhileSplitting/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11907) Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator
[
https://issues.apache.org/jira/browse/HBASE-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157655#comment-14157655
]
Hadoop QA commented on HBASE-11907:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672719/HBASE-11907.patch
against trunk revision .
ATTACHMENT ID: 12672719
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11205//console
This message is automatically generated.
> Use the joni byte[] regex engine in place of j.u.regex in
> RegexStringComparator
> ---
>
> Key: HBASE-11907
> URL: https://issues.apache.org/jira/browse/HBASE-11907
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: HBASE-11907.patch, HBASE-11907.patch, HBASE-11907.patch,
> HBASE-11907.patch
>
>
> The joni regex engine (https://github.com/jruby/joni), a Java port of
> Oniguruma regexp library done by the JRuby project, is:
> - MIT licensed
> - Designed to work with byte[] arguments instead of String
> - Capable of handling UTF8 encoding
> - Regex syntax compatible
> - Interruptible
> - *About twice as fast as j.u.regex*
> - Has JRuby's jcodings library as a dependency, also MIT licensed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12164:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks for the reviews.
Thanks for finding the issue about required field, Stack.
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
>
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 12164-v1.txt, 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-10153) improve VerifyReplication to compute BADROWS more accurately
[
https://issues.apache.org/jira/browse/HBASE-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-10153.
Resolution: Fixed
Thanks for the patch, Jianwei
Thanks for the reviews.
> improve VerifyReplication to compute BADROWS more accurately
>
>
> Key: HBASE-10153
> URL: https://issues.apache.org/jira/browse/HBASE-10153
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 0.94.14
>Reporter: cuijianwei
>Assignee: cuijianwei
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 10153-0.98.txt, 10153-v2-trunk.txt,
> HBASE-10153-0.94-v1.patch, HBASE-10153-trunk.patch
>
>
> VerifyReplicaiton could compare the source table with its peer table and
> compute BADROWS. However, the current BADROWS computing method might not be
> accurate enough. For example, if source table contains rows as {r1, r2, r3,
> r4} and peer table contains rows as {r1, r3, r4} BADROWS will be 3 because
> 'r2' in source table will make all the later row comparisons fail. Will it be
> better if the BADROWS is computed to 1 in this situation? Maybe, we can
> compute the BADROWS more accurately in merge comparison?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-10153) improve VerifyReplication to compute BADROWS more accurately
[
https://issues.apache.org/jira/browse/HBASE-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-10153:
---
Release Note:
VerifyReplicaiton reports the following counters besides the existing ones:
ONLY_IN_SOURCE_TABLE_ROWS: number of rows found only in source
ONLY_IN_PEER_TABLE_ROWS: number of rows found only in peer
CONTENT_DIFFERENT_ROWS: number of rows whose contents are different between
source and peer
> improve VerifyReplication to compute BADROWS more accurately
>
>
> Key: HBASE-10153
> URL: https://issues.apache.org/jira/browse/HBASE-10153
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 0.94.14
>Reporter: cuijianwei
>Assignee: cuijianwei
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 10153-0.98.txt, 10153-v2-trunk.txt,
> HBASE-10153-0.94-v1.patch, HBASE-10153-trunk.patch
>
>
> VerifyReplicaiton could compare the source table with its peer table and
> compute BADROWS. However, the current BADROWS computing method might not be
> accurate enough. For example, if source table contains rows as {r1, r2, r3,
> r4} and peer table contains rows as {r1, r3, r4} BADROWS will be 3 because
> 'r2' in source table will make all the later row comparisons fail. Will it be
> better if the BADROWS is computed to 1 in this situation? Maybe, we can
> compute the BADROWS more accurately in merge comparison?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12150) Backport replication changes from HBASE-12145
[
https://issues.apache.org/jira/browse/HBASE-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157638#comment-14157638
]
Hudson commented on HBASE-12150:
FAILURE: Integrated in HBase-0.98 #564 (See
[https://builds.apache.org/job/HBase-0.98/564/])
HBASE-12150 Backport replication changes from HBASE-12145 (apurtell: rev
ecf09fd02b8489716383053fc5ba6ea2283fbb2c)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java
> Backport replication changes from HBASE-12145
> -
>
> Key: HBASE-12150
> URL: https://issues.apache.org/jira/browse/HBASE-12150
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 0.98.7
>
> Attachments: HBASE-12150.patch
>
>
> HBASE-12145 makes all zk accesses synchronized in RecoverableZooKeeper in
> branch-1 +:
> {code}
> @@ -690,23 +692,23 @@ public class RecoverableZooKeeper {
> return newData;
>}
>
> - public long getSessionId() {
> -return zk == null ? null : zk.getSessionId();
> + public synchronized long getSessionId() {
> +return zk == null ? -1 : zk.getSessionId();
>}
>
> - public void close() throws InterruptedException {
> + public synchronized void close() throws InterruptedException {
> if (zk != null) zk.close();
>}
>
> - public States getState() {
> + public synchronized States getState() {
> return zk == null ? null : zk.getState();
>}
>
> - public ZooKeeper getZooKeeper() {
> + public synchronized ZooKeeper getZooKeeper() {
> return zk;
>}
>
> - public byte[] getSessionPasswd() {
> + public synchronized byte[] getSessionPasswd() {
> return zk == null ? null : zk.getSessionPasswd();
>}
> {code}
> It also makes this change:
> {code}
> @@ -391,8 +390,14 @@ public class ReplicationPeersZKImpl extends
> ReplicationStateZKBase implements Re
> if (peer == null) {
>return false;
> }
> -((ConcurrentMap)
> peerClusters).putIfAbsent(peerId, peer);
> -LOG.info("Added new peer cluster " +
> peer.getPeerConfig().getClusterKey());
> +ReplicationPeerZKImpl previous =
> + ((ConcurrentMap)
> peerClusters).putIfAbsent(peerId, peer);
> +if (previous == null) {
> + LOG.info("Added new peer cluster=" +
> peer.getPeerConfig().getClusterKey());
> +} else {
> + LOG.info("Peer already present, " +
> previous.getPeerConfig().getClusterKey() +
> +", new cluster=" + peer.getPeerConfig().getClusterKey());
> +}
> return true;
>}
> {code}
> We should keep the 0.98 code in sync with these changes because these affect
> correctness. Would like to avoid "this change works in branch-1 or master but
> breaks in some weird way in 0.98" issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12145) Fix javadoc and findbugs so new folks aren't freaked when they see them
[ https://issues.apache.org/jira/browse/HBASE-12145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157636#comment-14157636 ] Hudson commented on HBASE-12145: FAILURE: Integrated in HBase-0.98 #564 (See [https://builds.apache.org/job/HBase-0.98/564/]) HBASE-12150 Backport replication changes from HBASE-12145 (apurtell: rev ecf09fd02b8489716383053fc5ba6ea2283fbb2c) * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java > Fix javadoc and findbugs so new folks aren't freaked when they see them > --- > > Key: HBASE-12145 > URL: https://issues.apache.org/jira/browse/HBASE-12145 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.99.1 > > Attachments: 12145.txt, 12145v2.txt > > > Misc set of fixes to get these attributes green again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12008) Remove IntegrationTestImportTsv#testRunFromOutputCommitter
[ https://issues.apache.org/jira/browse/HBASE-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157637#comment-14157637 ] Hudson commented on HBASE-12008: FAILURE: Integrated in HBase-0.98 #564 (See [https://builds.apache.org/job/HBase-0.98/564/]) HBASE-12008 Remove IntegrationTestImportTsv#testRunFromOutputCommitter (apurtell: rev 93871c6707db93d41b02902f1b5fbdeb11a33eb0) * hbase-it/src/test/java/org/apache/hadoop/hbase/mapreduce/IntegrationTestImportTsv.java > Remove IntegrationTestImportTsv#testRunFromOutputCommitter > -- > > Key: HBASE-12008 > URL: https://issues.apache.org/jira/browse/HBASE-12008 > Project: HBase > Issue Type: Test > Components: mapreduce, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12008.00.patch, HBASE-12008.01.patch > > > This test was introduced to cover a dodgy code pathway exercised by the hbase > storage handler in HCatalog. Said dodgy code path involves launching a MR job > from the output committer of another MR job. As of Hive 0.14, said storage > handler has been removed. Let's drop test coverage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12162) HBaseAdmin#getTableDescriptor() may fail in case master fails over
[
https://issues.apache.org/jira/browse/HBASE-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157612#comment-14157612
]
Ted Yu commented on HBASE-12162:
The new test would establish MasterKeepAliveConnection and before
getHTableDescriptor() is called on this connection, kill the corresponding
master.
Going over existing tests, such as TestAdmin, I didn't find similar test case.
How about adding new test which verifies the robustness of all calls involving
MasterCallable in a separate issue ?
> HBaseAdmin#getTableDescriptor() may fail in case master fails over
> --
>
> Key: HBASE-12162
> URL: https://issues.apache.org/jira/browse/HBASE-12162
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12162-v1.txt
>
>
> This was discovered by Chakradhar Medavarapu during HA testing.
> Here is relevant exception:
> {code}
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|14/09/30
> 04:07:56 ERROR util.AbstractHBaseTool: Error running command-line tool
> 2014-09-30
> 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|java.io.IOException:
> Call to onprem-ha34/10.215.18.85:6 failed on local exception:
> java.io.IOException: Call id=1, waitTime=8703
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1571)
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1541)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1723)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1776)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getTableDescriptors(MasterProtos.java:42525)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$5.getTableDescriptors(ConnectionManager.java:2121)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getHTableDescriptor(ConnectionManager.java:2600)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:410)
> {code}
> From stack trace, exception came out of connection.getHTableDescriptor().
> This happened during master failover where MasterKeepAliveConnection to the
> failed master became unusable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157609#comment-14157609
]
Hadoop QA commented on HBASE-12164:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672711/12164-v1.txt
against trunk revision .
ATTACHMENT ID: 12672711
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11204//console
This message is automatically generated.
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
>
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 12164-v1.txt, 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12162) HBaseAdmin#getTableDescriptor() may fail in case master fails over
[
https://issues.apache.org/jira/browse/HBASE-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157601#comment-14157601
]
Devaraj Das commented on HBASE-12162:
-
LGTM (any chance of a test though?)
> HBaseAdmin#getTableDescriptor() may fail in case master fails over
> --
>
> Key: HBASE-12162
> URL: https://issues.apache.org/jira/browse/HBASE-12162
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12162-v1.txt
>
>
> This was discovered by Chakradhar Medavarapu during HA testing.
> Here is relevant exception:
> {code}
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|14/09/30
> 04:07:56 ERROR util.AbstractHBaseTool: Error running command-line tool
> 2014-09-30
> 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|java.io.IOException:
> Call to onprem-ha34/10.215.18.85:6 failed on local exception:
> java.io.IOException: Call id=1, waitTime=8703
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1571)
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1541)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1723)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1776)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getTableDescriptors(MasterProtos.java:42525)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$5.getTableDescriptors(ConnectionManager.java:2121)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getHTableDescriptor(ConnectionManager.java:2600)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:410)
> {code}
> From stack trace, exception came out of connection.getHTableDescriptor().
> This happened during master failover where MasterKeepAliveConnection to the
> failed master became unusable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12008) Remove IntegrationTestImportTsv#testRunFromOutputCommitter
[ https://issues.apache.org/jira/browse/HBASE-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157599#comment-14157599 ] Hudson commented on HBASE-12008: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #536 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/536/]) HBASE-12008 Remove IntegrationTestImportTsv#testRunFromOutputCommitter (apurtell: rev 93871c6707db93d41b02902f1b5fbdeb11a33eb0) * hbase-it/src/test/java/org/apache/hadoop/hbase/mapreduce/IntegrationTestImportTsv.java > Remove IntegrationTestImportTsv#testRunFromOutputCommitter > -- > > Key: HBASE-12008 > URL: https://issues.apache.org/jira/browse/HBASE-12008 > Project: HBase > Issue Type: Test > Components: mapreduce, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12008.00.patch, HBASE-12008.01.patch > > > This test was introduced to cover a dodgy code pathway exercised by the hbase > storage handler in HCatalog. Said dodgy code path involves launching a MR job > from the output committer of another MR job. As of Hive 0.14, said storage > handler has been removed. Let's drop test coverage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11907) Use the joni byte[] regex engine in place of j.u.regex in RegexStringComparator
[
https://issues.apache.org/jira/browse/HBASE-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11907:
---
Attachment: HBASE-11907.patch
Updated patch that addresses a potential source of confusion. Does not change
any reviewed functionality. Here is the delta:
{code}
@@ -320,8 +320,8 @@ public class RegexStringComparator extends
ByteArrayComparable {
* This engine operates on byte arrays directly so is expected to be more GC
* friendly, and reportedly is twice as fast as Java's Pattern engine.
*
- * NOTE: Only the {@link Pattern} flags CASE_INSENSITIVE and DOTALL are
- * supported.
+ * NOTE: Only the {@link Pattern} flags CASE_INSENSITIVE, DOTALL, and
+ * MULTILINE are supported.
*/
static class JoniRegexEngine implements Engine {
private Encoding encoding = UTF8Encoding.INSTANCE;
@@ -379,8 +379,15 @@ public class RegexStringComparator extends
ByteArrayComparable {
newFlags |= Option.IGNORECASE;
}
if ((flags & Pattern.DOTALL) != 0) {
+// This does NOT mean Pattern.MULTILINE
newFlags |= Option.MULTILINE;
}
+ if ((flags & Pattern.MULTILINE) != 0) {
+// This is what Java 8's Nashorn engine does when using joni and
+// translating Pattern's MULTILINE flag
+newFlags &= ~Option.SINGLELINE;
+newFlags |= Option.NEGATE_SINGLELINE;
+ }
return newFlags;
}
@@ -389,9 +396,14 @@ public class RegexStringComparator extends
ByteArrayComparable {
if ((flags & Option.IGNORECASE) != 0) {
newFlags |= Pattern.CASE_INSENSITIVE;
}
+ // This does NOT mean Pattern.MULTILINE, this is equivalent to
Pattern.DOTALL
if ((flags & Option.MULTILINE) != 0) {
newFlags |= Pattern.DOTALL;
}
+ // This means Pattern.MULTILINE. Nice
+ if ((flags & Option.NEGATE_SINGLELINE) != 0) {
+newFlags |= Pattern.MULTILINE;
+ }
return newFlags;
}
{code}
Going to commit later tonight.
> Use the joni byte[] regex engine in place of j.u.regex in
> RegexStringComparator
> ---
>
> Key: HBASE-11907
> URL: https://issues.apache.org/jira/browse/HBASE-11907
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: HBASE-11907.patch, HBASE-11907.patch, HBASE-11907.patch,
> HBASE-11907.patch
>
>
> The joni regex engine (https://github.com/jruby/joni), a Java port of
> Oniguruma regexp library done by the JRuby project, is:
> - MIT licensed
> - Designed to work with byte[] arguments instead of String
> - Capable of handling UTF8 encoding
> - Regex syntax compatible
> - Interruptible
> - *About twice as fast as j.u.regex*
> - Has JRuby's jcodings library as a dependency, also MIT licensed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12145) Fix javadoc and findbugs so new folks aren't freaked when they see them
[ https://issues.apache.org/jira/browse/HBASE-12145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157559#comment-14157559 ] Hudson commented on HBASE-12145: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #535 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/535/]) HBASE-12150 Backport replication changes from HBASE-12145 (apurtell: rev ecf09fd02b8489716383053fc5ba6ea2283fbb2c) * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java > Fix javadoc and findbugs so new folks aren't freaked when they see them > --- > > Key: HBASE-12145 > URL: https://issues.apache.org/jira/browse/HBASE-12145 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.99.1 > > Attachments: 12145.txt, 12145v2.txt > > > Misc set of fixes to get these attributes green again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12150) Backport replication changes from HBASE-12145
[
https://issues.apache.org/jira/browse/HBASE-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157560#comment-14157560
]
Hudson commented on HBASE-12150:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #535 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/535/])
HBASE-12150 Backport replication changes from HBASE-12145 (apurtell: rev
ecf09fd02b8489716383053fc5ba6ea2283fbb2c)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java
> Backport replication changes from HBASE-12145
> -
>
> Key: HBASE-12150
> URL: https://issues.apache.org/jira/browse/HBASE-12150
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 0.98.7
>
> Attachments: HBASE-12150.patch
>
>
> HBASE-12145 makes all zk accesses synchronized in RecoverableZooKeeper in
> branch-1 +:
> {code}
> @@ -690,23 +692,23 @@ public class RecoverableZooKeeper {
> return newData;
>}
>
> - public long getSessionId() {
> -return zk == null ? null : zk.getSessionId();
> + public synchronized long getSessionId() {
> +return zk == null ? -1 : zk.getSessionId();
>}
>
> - public void close() throws InterruptedException {
> + public synchronized void close() throws InterruptedException {
> if (zk != null) zk.close();
>}
>
> - public States getState() {
> + public synchronized States getState() {
> return zk == null ? null : zk.getState();
>}
>
> - public ZooKeeper getZooKeeper() {
> + public synchronized ZooKeeper getZooKeeper() {
> return zk;
>}
>
> - public byte[] getSessionPasswd() {
> + public synchronized byte[] getSessionPasswd() {
> return zk == null ? null : zk.getSessionPasswd();
>}
> {code}
> It also makes this change:
> {code}
> @@ -391,8 +390,14 @@ public class ReplicationPeersZKImpl extends
> ReplicationStateZKBase implements Re
> if (peer == null) {
>return false;
> }
> -((ConcurrentMap)
> peerClusters).putIfAbsent(peerId, peer);
> -LOG.info("Added new peer cluster " +
> peer.getPeerConfig().getClusterKey());
> +ReplicationPeerZKImpl previous =
> + ((ConcurrentMap)
> peerClusters).putIfAbsent(peerId, peer);
> +if (previous == null) {
> + LOG.info("Added new peer cluster=" +
> peer.getPeerConfig().getClusterKey());
> +} else {
> + LOG.info("Peer already present, " +
> previous.getPeerConfig().getClusterKey() +
> +", new cluster=" + peer.getPeerConfig().getClusterKey());
> +}
> return true;
>}
> {code}
> We should keep the 0.98 code in sync with these changes because these affect
> correctness. Would like to avoid "this change works in branch-1 or master but
> breaks in some weird way in 0.98" issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157556#comment-14157556
]
Jeffrey Zhong commented on HBASE-12164:
---
looks to me as well(+1). Thanks.
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
>
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 12164-v1.txt, 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12164:
---
Fix Version/s: 0.99.1
0.98.7
2.0.0
Hadoop Flags: Reviewed
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
>
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 0.98.7, 0.99.1
>
> Attachments: 12164-v1.txt, 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-11973) The url of the token file location set by IntegrationTestImportTsv should point to the localfs
[ https://issues.apache.org/jira/browse/HBASE-11973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-11973. Resolution: Fixed Fix Version/s: (was: 0.98.7) Assignee: (was: Devaraj Das) Fixed by cherry picking HBASE-12008 to 0.98. Thanks for raising the issue Devaraj. > The url of the token file location set by IntegrationTestImportTsv should > point to the localfs > -- > > Key: HBASE-11973 > URL: https://issues.apache.org/jira/browse/HBASE-11973 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das > Attachments: 11973-1.txt > > > The location of the token file is on the local filesystem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12008) Remove IntegrationTestImportTsv#testRunFromOutputCommitter
[ https://issues.apache.org/jira/browse/HBASE-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-12008: --- Fix Version/s: 0.98.7 > Remove IntegrationTestImportTsv#testRunFromOutputCommitter > -- > > Key: HBASE-12008 > URL: https://issues.apache.org/jira/browse/HBASE-12008 > Project: HBase > Issue Type: Test > Components: mapreduce, test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-12008.00.patch, HBASE-12008.01.patch > > > This test was introduced to cover a dodgy code pathway exercised by the hbase > storage handler in HCatalog. Said dodgy code path involves launching a MR job > from the output committer of another MR job. As of Hive 0.14, said storage > handler has been removed. Let's drop test coverage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12150) Backport replication changes from HBASE-12145
[
https://issues.apache.org/jira/browse/HBASE-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-12150:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to 0.98. Thanks for the review Nick
> Backport replication changes from HBASE-12145
> -
>
> Key: HBASE-12150
> URL: https://issues.apache.org/jira/browse/HBASE-12150
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 0.98.7
>
> Attachments: HBASE-12150.patch
>
>
> HBASE-12145 makes all zk accesses synchronized in RecoverableZooKeeper in
> branch-1 +:
> {code}
> @@ -690,23 +692,23 @@ public class RecoverableZooKeeper {
> return newData;
>}
>
> - public long getSessionId() {
> -return zk == null ? null : zk.getSessionId();
> + public synchronized long getSessionId() {
> +return zk == null ? -1 : zk.getSessionId();
>}
>
> - public void close() throws InterruptedException {
> + public synchronized void close() throws InterruptedException {
> if (zk != null) zk.close();
>}
>
> - public States getState() {
> + public synchronized States getState() {
> return zk == null ? null : zk.getState();
>}
>
> - public ZooKeeper getZooKeeper() {
> + public synchronized ZooKeeper getZooKeeper() {
> return zk;
>}
>
> - public byte[] getSessionPasswd() {
> + public synchronized byte[] getSessionPasswd() {
> return zk == null ? null : zk.getSessionPasswd();
>}
> {code}
> It also makes this change:
> {code}
> @@ -391,8 +390,14 @@ public class ReplicationPeersZKImpl extends
> ReplicationStateZKBase implements Re
> if (peer == null) {
>return false;
> }
> -((ConcurrentMap)
> peerClusters).putIfAbsent(peerId, peer);
> -LOG.info("Added new peer cluster " +
> peer.getPeerConfig().getClusterKey());
> +ReplicationPeerZKImpl previous =
> + ((ConcurrentMap)
> peerClusters).putIfAbsent(peerId, peer);
> +if (previous == null) {
> + LOG.info("Added new peer cluster=" +
> peer.getPeerConfig().getClusterKey());
> +} else {
> + LOG.info("Peer already present, " +
> previous.getPeerConfig().getClusterKey() +
> +", new cluster=" + peer.getPeerConfig().getClusterKey());
> +}
> return true;
>}
> {code}
> We should keep the 0.98 code in sync with these changes because these affect
> correctness. Would like to avoid "this change works in branch-1 or master but
> breaks in some weird way in 0.98" issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12164:
---
Summary: Check for presence of user Id in
SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate (was: Check for
presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong)
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is inaccurate
>
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12164-v1.txt, 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12122) Try not to assign user regions to master all the time
[ https://issues.apache.org/jira/browse/HBASE-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157544#comment-14157544 ] Hudson commented on HBASE-12122: FAILURE: Integrated in HBase-1.0 #265 (See [https://builds.apache.org/job/HBase-1.0/265/]) HBASE-12122 Try not to assign user regions to master all the time (jxiang: rev 16228c275d3926317c8cb6776c95f575354a8253) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/ClusterLoadState.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java > Try not to assign user regions to master all the time > - > > Key: HBASE-12122 > URL: https://issues.apache.org/jira/browse/HBASE-12122 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: hbase-12122.patch, hbase-12122_v2.patch > > > Load balancer does a good job not to assign regions of tables not configured > to put on the active master. However, if there is no other region server, it > still assigns users regions to the master. This happens when all normal > region servers are crashed and recovering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12161) Add support for grant/revoke on namespaces in AccessControlClient
[ https://issues.apache.org/jira/browse/HBASE-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-12161: Attachment: HBASE-12161_0.98.patch > Add support for grant/revoke on namespaces in AccessControlClient > - > > Key: HBASE-12161 > URL: https://issues.apache.org/jira/browse/HBASE-12161 > Project: HBase > Issue Type: Improvement >Reporter: Srikanth Srungarapu >Assignee: Srikanth Srungarapu >Priority: Minor > Attachments: HBASE-12161_0.98.patch > > > As per the description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157534#comment-14157534
]
Andrew Purtell commented on HBASE-12164:
+1
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
> ---
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12164-v1.txt, 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157532#comment-14157532 ] Andrew Purtell commented on HBASE-11764: I was sidetracked by other stuff. Let me put up a patch tomorrow. Apologies for the delay. > Support per cell TTLs > - > > Key: HBASE-11764 > URL: https://issues.apache.org/jira/browse/HBASE-11764 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, > HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, > HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12160) Make Surefire's argLine configurable in the command line
[ https://issues.apache.org/jira/browse/HBASE-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157528#comment-14157528 ] Hudson commented on HBASE-12160: FAILURE: Integrated in HBase-1.0 #264 (See [https://builds.apache.org/job/HBase-1.0/264/]) HBASE-12160 Make Surefire's argLine configurable in the command line (eclark: rev 0a3c24f601fa9b7a7e11427a559f56276036c1ef) * pom.xml > Make Surefire's argLine configurable in the command line > > > Key: HBASE-12160 > URL: https://issues.apache.org/jira/browse/HBASE-12160 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: > 0001-HBASE-12160-Make-Surefire-s-argLine-configurable-in-.patch > > > On tests machines with larger sets of ram it can really help to reduce test > time to run with larger heaps. Lets make that a property so that mvn command > line can set it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12164:
---
Attachment: 12164-v1.txt
The test failure was not related.
Retry.
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
> ---
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12164-v1.txt, 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-8808) Use Jacoco to generate Unit Test coverage reports
[
https://issues.apache.org/jira/browse/HBASE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157514#comment-14157514
]
Hadoop QA commented on HBASE-8808:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672678/0001-Including-Jacoco-plugin-to-get-test-coverage.patch
against trunk revision .
ATTACHMENT ID: 12672678
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11201//console
This message is automatically generated.
> Use Jacoco to generate Unit Test coverage reports
> -
>
> Key: HBASE-8808
> URL: https://issues.apache.org/jira/browse/HBASE-8808
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.89-fb
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Trivial
> Fix For: 0.89-fb
>
> Attachments: 0001-Including-Jacoco-plugin-to-get-test-coverage.patch,
> 0001-Including-Jacoco-plugin-to-get-test-coverage.patch,
> 0001-Including-Jacoco-plugin-to-get-test-coverage.patch, Screen Shot
> 2013-06-25 at 11.35.30 AM.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Enabling the code coverage tool jacoco in maven
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157509#comment-14157509
]
Hadoop QA commented on HBASE-12164:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672699/12164-v1.txt
against trunk revision .
ATTACHMENT ID: 12672699
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAsyncProcess
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11203//console
This message is automatically generated.
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
> ---
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12160) Make Surefire's argLine configurable in the command line
[ https://issues.apache.org/jira/browse/HBASE-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157497#comment-14157497 ] Hudson commented on HBASE-12160: FAILURE: Integrated in HBase-TRUNK #5611 (See [https://builds.apache.org/job/HBase-TRUNK/5611/]) HBASE-12160 Make Surefire's argLine configurable in the command line (eclark: rev 436733c79bbcb715e6cf75cd2063450d0712188c) * pom.xml > Make Surefire's argLine configurable in the command line > > > Key: HBASE-12160 > URL: https://issues.apache.org/jira/browse/HBASE-12160 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: > 0001-HBASE-12160-Make-Surefire-s-argLine-configurable-in-.patch > > > On tests machines with larger sets of ram it can really help to reduce test > time to run with larger heaps. Lets make that a property so that mvn command > line can set it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12151) Make dev scripts executable
[ https://issues.apache.org/jira/browse/HBASE-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-12151: Resolution: Fixed Fix Version/s: 0.99.1 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master and branch-1. > Make dev scripts executable > --- > > Key: HBASE-12151 > URL: https://issues.apache.org/jira/browse/HBASE-12151 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12151.patch > > > Is there any reason not to make dev-support/*.sh executable? It would make it > possible to sym-link to them from a directory in the executable path for > easier execution of the definitive scripts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12164:
---
Status: Patch Available (was: Open)
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
> ---
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
[
https://issues.apache.org/jira/browse/HBASE-12164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12164:
---
Attachment: 12164-v1.txt
Proposed patch.
Also fixes a problem Stack discovered where required field of
SecureBulkLoadHFilesResponse isn't set in failure condition.
> Check for presence of user Id in
> SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
> ---
>
> Key: HBASE-12164
> URL: https://issues.apache.org/jira/browse/HBASE-12164
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12164-v1.txt
>
>
> Here is the code:
> {code}
> if (request.getFsToken().hasIdentifier() &&
> request.getFsToken().hasPassword()) {
> {code}
> In test case, request.getFsToken().hasIdentifier() returns false, leading to
> userToken being null.
> This would make secure bulk load unsuccessful because the body of
> secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12164) Check for presence of user Id in SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
Ted Yu created HBASE-12164:
--
Summary: Check for presence of user Id in
SecureBulkLoadEndpoint#secureBulkLoadHFiles() is wrong
Key: HBASE-12164
URL: https://issues.apache.org/jira/browse/HBASE-12164
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Here is the code:
{code}
if (request.getFsToken().hasIdentifier() &&
request.getFsToken().hasPassword()) {
{code}
In test case, request.getFsToken().hasIdentifier() returns false, leading to
userToken being null.
This would make secure bulk load unsuccessful because the body of
secureBulkLoadHFiles() is skipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157446#comment-14157446
]
Jeffrey Zhong commented on HBASE-12152:
---
What Stack found is another bug that
{quote}
done.run(null)
{quote}
doesn't work in existing code and we have to use something proposed by
[~saint@gmail.com]
{code}
done.run(SecureBulkLoadHFilesResponse.newBuilder().setLoaded(false).build());
{code}
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157440#comment-14157440
]
Andrew Purtell commented on HBASE-12152:
I suggest opening a clean issue that clearly explains this and provides a
patch. This issue is FUBAR
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157438#comment-14157438
]
Jeffrey Zhong commented on HBASE-12152:
---
Sorry I came to the thread late. The fix I suggested to Ted is due to my recent
change where I added this following line to prevent possible toByteArray() from
throwing NPE. While it seems, in test case,
request.getFsToken().hasIdentifier() returns false and
request.getFsToken().getIdentifier() is empty byte array. So my previous NPE
worry isn't needed and the fix just restores previous behavior.
{quote}
if (request.getFsToken().hasIdentifier() && request.getFsToken().hasPassword())
{
{quote}
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157436#comment-14157436
]
Andrew Purtell commented on HBASE-12152:
Definitely invalid then, good resolution.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12162) HBaseAdmin#getTableDescriptor() may fail in case master fails over
[
https://issues.apache.org/jira/browse/HBASE-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157434#comment-14157434
]
Hadoop QA commented on HBASE-12162:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672659/12162-v1.txt
against trunk revision .
ATTACHMENT ID: 12672659
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11199//console
This message is automatically generated.
> HBaseAdmin#getTableDescriptor() may fail in case master fails over
> --
>
> Key: HBASE-12162
> URL: https://issues.apache.org/jira/browse/HBASE-12162
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12162-v1.txt
>
>
> This was discovered by Chakradhar Medavarapu during HA testing.
> Here is relevant exception:
> {code}
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|14/09/30
> 04:07:56 ERROR util.AbstractHBaseTool: Error running command-line tool
> 2014-09-30
> 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|java.io.IOException:
> Call to onprem-ha34/10.215.18.85:6 failed on local exception:
> java.io.IOException: Call id=1, waitTime=8703
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1571)
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.j
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157433#comment-14157433
]
Ted Yu commented on HBASE-12152:
In the QA run above, TestLoadIncrementalHFiles didn't hang, neither did
TestSecureLoadIncrementalHFilesSplitRecovery.
The initial subject of the JIRA was inaccurate - although
TestLoadIncrementalHFiles showed up in stack trace of hanging test, it was
TestSecureLoadIncrementalHFilesSplitRecovery that actually hung because of
userToken being null, aborting bulk load.
Since there was no timeout prior to today,
TestSecureLoadIncrementalHFilesSplitRecovery hung.
The addition of the timeout exposed the bug.
The above was result of discussion with Jeff.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12139) StochasticLoadBalancer doesn't work on large lightly loaded clusters
[ https://issues.apache.org/jira/browse/HBASE-12139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157426#comment-14157426 ] Elliott Clark commented on HBASE-12139: --- It must have gotten de-authed or something. Not really sure. > StochasticLoadBalancer doesn't work on large lightly loaded clusters > > > Key: HBASE-12139 > URL: https://issues.apache.org/jira/browse/HBASE-12139 > Project: HBase > Issue Type: Bug > Components: Balancer, master >Affects Versions: 0.99.0, 2.0.0, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: > 0001-HBASE-12139-StochasticLoadBalancer-doesn-t-work-on-l-0.98.patch, > 0001-HBASE-12139-StochasticLoadBalancer-doesn-t-work-on-l.patch > > > If you have a cluster with > 200 nodes and a small table (< 50 regions) the > StochasticLoadBalancer doesn't work at all. It is not correctly scaling the > required values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-12152:
---
Resolution: Invalid
Status: Resolved (was: Patch Available)
The patch on this issue doesn't match the title or the OP. I tried to follow
along with the above but this issue is best resolved as invalid, so I'm going
to do that. We have a new test time out. If the test times out, open an issue
for that. For the unrelated work above, open an issue for that. Nothing more
needs doing here.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-7509) Enable RS to query a secondary datanode in parallel, if the primary takes too long
[ https://issues.apache.org/jira/browse/HBASE-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh resolved HBASE-7509. --- Resolution: Not a Problem > Enable RS to query a secondary datanode in parallel, if the primary takes too > long > -- > > Key: HBASE-7509 > URL: https://issues.apache.org/jira/browse/HBASE-7509 > Project: HBase > Issue Type: Improvement >Reporter: Amitanand Aiyer >Assignee: Amitanand Aiyer >Priority: Critical > Attachments: quorumDiffs.tgz > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-7509) Enable RS to query a secondary datanode in parallel, if the primary takes too long
[
https://issues.apache.org/jira/browse/HBASE-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157416#comment-14157416
]
Jonathan Hsieh commented on HBASE-7509:
---
I believe this can be closed. Reopen if I am wrong.
When using hadoop 2.4 we should just need to include the configs from the
HDFS-5776 release notes in the hbase-site.xml file:
{quote}
This feature is off by default. To enable this feature, set
dfs.client.hedged.read.threadpool.size to a positive number. The
threadpool size is how many threads to dedicate to the running of these
'hedged', concurrent reads in your client.
Then set dfs.client.hedged.read.threshold.millis to the number of
milliseconds to wait before starting up a 'hedged' read. For example, if you
set this property to 10, then if a read has not returned within 10
milliseconds, we will start up a new read against a different block replica.
This feature emits new metrics:
+ hedgedReadOps
+ hedgeReadOpsWin -- how many times the hedged read 'beat' the original read
+ hedgedReadOpsInCurThread -- how many times we went to do a hedged read but we
had to run it in the current thread because
dfs.client.hedged.read.threadpool.size was at a maximum.
{quote}
> Enable RS to query a secondary datanode in parallel, if the primary takes too
> long
> --
>
> Key: HBASE-7509
> URL: https://issues.apache.org/jira/browse/HBASE-7509
> Project: HBase
> Issue Type: Improvement
>Reporter: Amitanand Aiyer
>Assignee: Amitanand Aiyer
>Priority: Critical
> Attachments: quorumDiffs.tgz
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157410#comment-14157410
]
Hadoop QA commented on HBASE-12152:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672658/12152-v3.txt
against trunk revision .
ATTACHMENT ID: 12672658
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11198//console
This message is automatically generated.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.c
[jira] [Commented] (HBASE-12160) Make Surefire's argLine configurable in the command line
[ https://issues.apache.org/jira/browse/HBASE-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157408#comment-14157408 ] Hudson commented on HBASE-12160: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #534 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/534/]) HBASE-12160 Make Surefire's argLine configurable in the command line (eclark: rev 21215a2ae46e2ef32c4c4bb4359a350359813cd0) * pom.xml > Make Surefire's argLine configurable in the command line > > > Key: HBASE-12160 > URL: https://issues.apache.org/jira/browse/HBASE-12160 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: > 0001-HBASE-12160-Make-Surefire-s-argLine-configurable-in-.patch > > > On tests machines with larger sets of ram it can really help to reduce test > time to run with larger heaps. Lets make that a property so that mvn command > line can set it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12152:
---
Attachment: (was: 12152-v3.txt)
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12152:
---
Attachment: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: TestSecureLoadIncrementalHFilesSplitRecovery-fix.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12104) Some optimization and bugfix for HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157393#comment-14157393 ] Elliott Clark commented on HBASE-12104: --- Patch looks good to me if it passes the tests > Some optimization and bugfix for HTableMultiplexer > -- > > Key: HBASE-12104 > URL: https://issues.apache.org/jira/browse/HBASE-12104 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 2.0.0 >Reporter: Yi Deng >Assignee: Yi Deng > Labels: multiplexer > Fix For: 2.0.0 > > Attachments: > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public.patch > > > Make HTableMultiplexerStatus public > Delay before resubmit. > Fix some missing counting on total failure. > Use ScheduledExecutorService to simplify the code. > Other refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157385#comment-14157385
]
stack commented on HBASE-12152:
---
I give up.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12139) StochasticLoadBalancer doesn't work on large lightly loaded clusters
[ https://issues.apache.org/jira/browse/HBASE-12139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157384#comment-14157384 ] Enis Soztutar commented on HBASE-12139: --- I see. I don't have access to that rb. I thought it was putting comments back to jira, no? > StochasticLoadBalancer doesn't work on large lightly loaded clusters > > > Key: HBASE-12139 > URL: https://issues.apache.org/jira/browse/HBASE-12139 > Project: HBase > Issue Type: Bug > Components: Balancer, master >Affects Versions: 0.99.0, 2.0.0, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: > 0001-HBASE-12139-StochasticLoadBalancer-doesn-t-work-on-l-0.98.patch, > 0001-HBASE-12139-StochasticLoadBalancer-doesn-t-work-on-l.patch > > > If you have a cluster with > 200 nodes and a small table (< 50 regions) the > StochasticLoadBalancer doesn't work at all. It is not correctly scaling the > required values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12122) Try not to assign user regions to master all the time
[ https://issues.apache.org/jira/browse/HBASE-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157377#comment-14157377 ] Jimmy Xiang commented on HBASE-12122: - Thanks. Pushed it into branch 1 and master. > Try not to assign user regions to master all the time > - > > Key: HBASE-12122 > URL: https://issues.apache.org/jira/browse/HBASE-12122 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: hbase-12122.patch, hbase-12122_v2.patch > > > Load balancer does a good job not to assign regions of tables not configured > to put on the active master. However, if there is no other region server, it > still assigns users regions to the master. This happens when all normal > region servers are crashed and recovering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12139) StochasticLoadBalancer doesn't work on large lightly loaded clusters
[ https://issues.apache.org/jira/browse/HBASE-12139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157373#comment-14157373 ] Elliott Clark commented on HBASE-12139: --- I got +1's on the review board > StochasticLoadBalancer doesn't work on large lightly loaded clusters > > > Key: HBASE-12139 > URL: https://issues.apache.org/jira/browse/HBASE-12139 > Project: HBase > Issue Type: Bug > Components: Balancer, master >Affects Versions: 0.99.0, 2.0.0, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: > 0001-HBASE-12139-StochasticLoadBalancer-doesn-t-work-on-l-0.98.patch, > 0001-HBASE-12139-StochasticLoadBalancer-doesn-t-work-on-l.patch > > > If you have a cluster with > 200 nodes and a small table (< 50 regions) the > StochasticLoadBalancer doesn't work at all. It is not correctly scaling the > required values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12163) Move test annotation classes to the same package as in master
[ https://issues.apache.org/jira/browse/HBASE-12163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12163: -- Priority: Trivial (was: Major) > Move test annotation classes to the same package as in master > - > > Key: HBASE-12163 > URL: https://issues.apache.org/jira/browse/HBASE-12163 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Trivial > Fix For: 0.98.7, 0.99.1 > > > Test classe annotations (SmallTests, etc) are in different packages in master > vs 0.98 and branch-1 making backporting difficult. > Lets move them to the same package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157369#comment-14157369
]
Ted Yu commented on HBASE-12152:
I debugged the failure in TestSecureLoadIncrementalHFilesSplitRecovery with
Jeff.
I tried the above change which didn't make
TestSecureLoadIncrementalHFilesSplitRecovery pass - userToken is null, skipping
the bulk load.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157367#comment-14157367
]
stack commented on HBASE-12152:
---
[~ted_yu] you are impossible to fathom/follow. Are you doing relay between
this issue and [~jeffreyz]? Is [~jeffreyz] unable?
TestSecureLoadIncremtalHFileSplitRecovery is failing in here in
SecureBulkLoadEndpoint
} else if (userProvider.isHadoopSecurityEnabled()) {
//we allow this to pass through in "simple" security mode
//for mini cluster testing
ResponseConverter.setControllerException(controller,
new DoNotRetryIOException("User token cannot be null"));
// HERE... we are returning w/o setitng 'loaded'.
+
done.run(SecureBulkLoadHFilesResponse.newBuilder().setLoaded(false).build());
return;
Which is causing
2014-10-02 15:48:44,970 ERROR [B.defaultRpcServer.handler=2,queue=0,port=60236]
ipc.RpcServer(2053): Unexpected throwable object
com.google.protobuf.UninitializedMessageException: Message missing required
fields: loaded
at
com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770)
at
org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos$SecureBulkLoadHFilesResponse$Builder.build(SecureBulkLoadProtos.java:1542)
at
org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos$SecureBulkLoadHFilesResponse$Builder.build(SecureBulkLoadProtos.java:1486)
at
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:5901)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1644)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1626)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30426)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2020)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
at java.lang.Thread.run(Thread.java:745)
... test does not complete.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12106) Move test annotations to test artifact
[ https://issues.apache.org/jira/browse/HBASE-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157360#comment-14157360 ] Enis Soztutar commented on HBASE-12106: --- Can I get a review? Fairly trivial patch. > Move test annotations to test artifact > -- > > Key: HBASE-12106 > URL: https://issues.apache.org/jira/browse/HBASE-12106 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: hbase-12106_v1-0.98+0.99.patch, hbase-12106_v1.patch, > hbase-12106_v1.patch, hbase-12106_v2.patch > > > Test annotation interfaces used to be under hbase-common/src/test then moved > to hbase-annotations/src/main. We should move them to > hbase-annotations/src/test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157359#comment-14157359
]
Ted Yu commented on HBASE-12152:
Let me create another JIRA for fixing
TestSecureLoadIncrementalHFilesSplitRecovery
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-8808) Use Jacoco to generate Unit Test coverage reports
[ https://issues.apache.org/jira/browse/HBASE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manukranth Kolloju updated HBASE-8808: -- Attachment: 0001-Including-Jacoco-plugin-to-get-test-coverage.patch Rebasing to master after elliott's patch. > Use Jacoco to generate Unit Test coverage reports > - > > Key: HBASE-8808 > URL: https://issues.apache.org/jira/browse/HBASE-8808 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.89-fb >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 0.89-fb > > Attachments: 0001-Including-Jacoco-plugin-to-get-test-coverage.patch, > 0001-Including-Jacoco-plugin-to-get-test-coverage.patch, > 0001-Including-Jacoco-plugin-to-get-test-coverage.patch, Screen Shot > 2013-06-25 at 11.35.30 AM.png > > Original Estimate: 24h > Remaining Estimate: 24h > > Enabling the code coverage tool jacoco in maven -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12139) StochasticLoadBalancer doesn't work on large lightly loaded clusters
[ https://issues.apache.org/jira/browse/HBASE-12139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157356#comment-14157356 ] Enis Soztutar commented on HBASE-12139: --- blated +1. Did you commit this because it was a minor change? > StochasticLoadBalancer doesn't work on large lightly loaded clusters > > > Key: HBASE-12139 > URL: https://issues.apache.org/jira/browse/HBASE-12139 > Project: HBase > Issue Type: Bug > Components: Balancer, master >Affects Versions: 0.99.0, 2.0.0, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: > 0001-HBASE-12139-StochasticLoadBalancer-doesn-t-work-on-l-0.98.patch, > 0001-HBASE-12139-StochasticLoadBalancer-doesn-t-work-on-l.patch > > > If you have a cluster with > 200 nodes and a small table (< 50 regions) the > StochasticLoadBalancer doesn't work at all. It is not correctly scaling the > required values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157352#comment-14157352
]
stack commented on HBASE-12152:
---
This issue is 'TestLoadIncrementalHFiles shows up as zombie test'.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157348#comment-14157348
]
Ted Yu commented on HBASE-12152:
My attachment is to make TestSecureLoadIncrementalHFilesSplitRecovery pass.
Jeff has another fix for TestLoadIncrementalHFiles
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12163) Move test annotation classes to the same package as in master
Enis Soztutar created HBASE-12163: - Summary: Move test annotation classes to the same package as in master Key: HBASE-12163 URL: https://issues.apache.org/jira/browse/HBASE-12163 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.98.7, 0.99.1 Test classe annotations (SmallTests, etc) are in different packages in master vs 0.98 and branch-1 making backporting difficult. Lets move them to the same package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12160) Make Surefire's argLine configurable in the command line
[ https://issues.apache.org/jira/browse/HBASE-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157346#comment-14157346 ] Elliott Clark commented on HBASE-12160: --- Yeah just pushed thanks [~enis] > Make Surefire's argLine configurable in the command line > > > Key: HBASE-12160 > URL: https://issues.apache.org/jira/browse/HBASE-12160 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: > 0001-HBASE-12160-Make-Surefire-s-argLine-configurable-in-.patch > > > On tests machines with larger sets of ram it can really help to reduce test > time to run with larger heaps. Lets make that a property so that mvn command > line can set it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12149) TestRegionPlacement is failing undeterministically
[
https://issues.apache.org/jira/browse/HBASE-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157343#comment-14157343
]
Enis Soztutar commented on HBASE-12149:
---
bq. The test categorization being in different packages messes up cherry-pick
I think we can also move the test class annotation package names in branch-1
and 0.98 for backports sake. Let me open an issue.
> TestRegionPlacement is failing undeterministically
> --
>
> Key: HBASE-12149
> URL: https://issues.apache.org/jira/browse/HBASE-12149
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Minor
> Fix For: 2.0.0, 0.99.1
>
> Attachments: 0001-Split-TestRegionPlacement.patch,
> 0001-Split-TestRegionPlacement.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> TestRegionPlacement has tests that are flaky and fail in a cascading fashion.
> We need to fix this cascading behavior and also fix the flakyness of the
> tests.
> 16:20:59 java.lang.AssertionError: Region {ENCODED =>
> 958cbbb4405e3bbcfd04185538912d4c, NAME =>
> 'testRegionAssignment,\x02\x02\x02,1412205636586.958cbbb4405e3bbcfd04185538912d4c.',
> STARTKEY => '\x02\x02\x02', ENDKEY => '\x03\x03\x03'} not present on any of
> the expected servers
> 16:20:59 at org.junit.Assert.fail(Assert.java:88)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.killRandomServerAndVerifyAssignment(TestRegionPlacement.java:347)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testRegionPlacement(TestRegionPlacement.java:270)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRoundRobinAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.047 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRoundRobinAssignment(TestRegionPlacement.java:113)
> 16:20:59
> 16:20:59
> testFavoredNodesPresentForRandomAssignment(org.apache.hadoop.hbase.master.TestRegionPlacement)
> Time elapsed: 0.041 sec <<< ERROR!
> 16:20:59 java.lang.ArrayIndexOutOfBoundsException: 9
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
> 16:20:59 at
> java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
> 16:20:59 at
> org.apache.hadoop.hbase.LocalHBaseCluster.getRegionServer(LocalHBaseCluster.java:237)
> 16:20:59 at
> org.apache.hadoop.hbase.MiniHBaseCluster.getRegionServer(MiniHBaseCluster.java:609)
> 16:20:59 at
> org.apache.hadoop.hbase.master.TestRegionPlacement.testFavoredNodesPresentForRandomAssignment(TestRegionPlacement.java:173)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157342#comment-14157342
]
stack commented on HBASE-12152:
---
So it is not a fix for TestLoadIncrementalHFiles?
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12160) Make Surefire's argLine configurable in the command line
[ https://issues.apache.org/jira/browse/HBASE-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12160: -- Resolution: Fixed Fix Version/s: 0.99.1 0.98.7 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) This seems committed. Resolving. > Make Surefire's argLine configurable in the command line > > > Key: HBASE-12160 > URL: https://issues.apache.org/jira/browse/HBASE-12160 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: > 0001-HBASE-12160-Make-Surefire-s-argLine-configurable-in-.patch > > > On tests machines with larger sets of ram it can really help to reduce test > time to run with larger heaps. Lets make that a property so that mvn command > line can set it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157337#comment-14157337
]
Ted Yu commented on HBASE-12152:
In SecureBulkLoadEndpoint, the following condition isn't met during test run:
{code}
if (request.getFsToken().hasIdentifier() &&
request.getFsToken().hasPassword()) {
{code}
leading to userToken being null which aborts secure bulk load.
The proposed change would allow userToken to be created.
Jeff has another fix for the hanging test.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12104) Some optimization and bugfix for HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157332#comment-14157332 ] Yi Deng commented on HBASE-12104: - Hi [~stack], I just updated the patch which adds back some methods which should have been marked as deprecated other than deleted directly. Sorry for any inconvinience. > Some optimization and bugfix for HTableMultiplexer > -- > > Key: HBASE-12104 > URL: https://issues.apache.org/jira/browse/HBASE-12104 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 2.0.0 >Reporter: Yi Deng >Assignee: Yi Deng > Labels: multiplexer > Fix For: 2.0.0 > > Attachments: > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public.patch > > > Make HTableMultiplexerStatus public > Delay before resubmit. > Fix some missing counting on total failure. > Use ScheduledExecutorService to simplify the code. > Other refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12104) Some optimization and bugfix for HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Deng updated HBASE-12104: Attachment: 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch Add back some deleted methods, mark them as deprecated instead. > Some optimization and bugfix for HTableMultiplexer > -- > > Key: HBASE-12104 > URL: https://issues.apache.org/jira/browse/HBASE-12104 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 2.0.0 >Reporter: Yi Deng >Assignee: Yi Deng > Labels: multiplexer > Fix For: 2.0.0 > > Attachments: > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch, > 0001-Make-HTableMultiplexerStatus-public.patch > > > Make HTableMultiplexerStatus public > Delay before resubmit. > Fix some missing counting on total failure. > Use ScheduledExecutorService to simplify the code. > Other refactoring. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12160) Make Surefire's argLine configurable in the command line
[ https://issues.apache.org/jira/browse/HBASE-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157326#comment-14157326 ] Hudson commented on HBASE-12160: FAILURE: Integrated in HBase-0.98 #563 (See [https://builds.apache.org/job/HBase-0.98/563/]) HBASE-12160 Make Surefire's argLine configurable in the command line (eclark: rev 21215a2ae46e2ef32c4c4bb4359a350359813cd0) * pom.xml > Make Surefire's argLine configurable in the command line > > > Key: HBASE-12160 > URL: https://issues.apache.org/jira/browse/HBASE-12160 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.99.1, 0.98.6.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: > 0001-HBASE-12160-Make-Surefire-s-argLine-configurable-in-.patch > > > On tests machines with larger sets of ram it can really help to reduce test > time to run with larger heaps. Lets make that a property so that mvn command > line can set it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12122) Try not to assign user regions to master all the time
[ https://issues.apache.org/jira/browse/HBASE-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157323#comment-14157323 ] Hudson commented on HBASE-12122: FAILURE: Integrated in HBase-TRUNK #5610 (See [https://builds.apache.org/job/HBase-TRUNK/5610/]) HBASE-12122 Try not to assign user regions to master all the time (jxiang: rev a463aef8bce2d2f3f4ce7c5684f28b78767b0bfb) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/fs/TestBlockReorder.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/ClusterLoadState.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetrics.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java > Try not to assign user regions to master all the time > - > > Key: HBASE-12122 > URL: https://issues.apache.org/jira/browse/HBASE-12122 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.0.0, 0.99.1 > > Attachments: hbase-12122.patch, hbase-12122_v2.patch > > > Load balancer does a good job not to assign regions of tables not configured > to put on the active master. However, if there is no other region server, it > still assigns users regions to the master. This happens when all normal > region servers are crashed and recovering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157324#comment-14157324
]
Hudson commented on HBASE-12152:
FAILURE: Integrated in HBase-TRUNK #5610 (See
[https://builds.apache.org/job/HBase-TRUNK/5610/])
HBASE-12152 TestLoadIncrementalHFiles shows up as zombie test; ADD TIMEOUT ON
TESTS -- Up timeout from 120 to 12 (stack: rev
546f436a4145647e595354372af23cdb6d8a1f28)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store
[ https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157322#comment-14157322 ] Hudson commented on HBASE-12148: FAILURE: Integrated in HBase-TRUNK #5610 (See [https://builds.apache.org/job/HBase-TRUNK/5610/]) HBASE-12148 Remove TimeRangeTracker as point of contention when many threads writing a Store (stack: rev 129b93bc0ef1291784752aeaf86e029c13d9a51b) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java HBASE-12148 Remove TimeRangeTracker as point of contention when many threads writing a Store (stack: rev 8ac42d4b40d8b8845f7831f6938bb92dfd4b558a) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java > Remove TimeRangeTracker as point of contention when many threads writing a > Store > > > Key: HBASE-12148 > URL: https://issues.apache.org/jira/browse/HBASE-12148 > Project: HBase > Issue Type: Sub-task > Components: Performance >Affects Versions: 2.0.0, 0.99.1 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.99.1 > > Attachments: 12148.txt, 12148.txt, 12148v2.txt, 12148v2.txt, Screen > Shot 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8808) Use Jacoco to generate Unit Test coverage reports
[ https://issues.apache.org/jira/browse/HBASE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157318#comment-14157318 ] stack commented on HBASE-8808: -- This are not your failures [~manukranthk] Talk to [~eclark] His patch stamps on yours. Get him to commit a version of this patch that will work w/ his change (as long as it does not break apache build). > Use Jacoco to generate Unit Test coverage reports > - > > Key: HBASE-8808 > URL: https://issues.apache.org/jira/browse/HBASE-8808 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.89-fb >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 0.89-fb > > Attachments: 0001-Including-Jacoco-plugin-to-get-test-coverage.patch, > 0001-Including-Jacoco-plugin-to-get-test-coverage.patch, Screen Shot > 2013-06-25 at 11.35.30 AM.png > > Original Estimate: 24h > Remaining Estimate: 24h > > Enabling the code coverage tool jacoco in maven -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12160) Make Surefire's argLine configurable in the command line
[
https://issues.apache.org/jira/browse/HBASE-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157306#comment-14157306
]
Hadoop QA commented on HBASE-12160:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672628/0001-HBASE-12160-Make-Surefire-s-argLine-configurable-in-.patch
against trunk revision .
ATTACHMENT ID: 12672628
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11196//console
This message is automatically generated.
> Make Surefire's argLine configurable in the command line
>
>
> Key: HBASE-12160
> URL: https://issues.apache.org/jira/browse/HBASE-12160
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 0.99.1, 0.98.6.1
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments:
> 0001-HBASE-12160-Make-Surefire-s-argLine-configurable-in-.patch
>
>
> On tests machines with larger sets of ram it can really help to reduce test
> time to run with larger heaps. Lets make that a property so that mvn command
> line can set it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-8808) Use Jacoco to generate Unit Test coverage reports
[
https://issues.apache.org/jira/browse/HBASE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157305#comment-14157305
]
Hadoop QA commented on HBASE-8808:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672630/0001-Including-Jacoco-plugin-to-get-test-coverage.patch
against trunk revision .
ATTACHMENT ID: 12672630
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+${argLine} -enableassertions
-XX:MaxDirectMemorySize=1G -Xmx1900m -XX:MaxPermSize=256m
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true
+${argLine} -enableassertions -Xmx1900m
-XX:MaxPermSize=256m -Djava.security.egd=file:/dev/./urandom
-Djava.net.preferIPv4Stack=true
"-Djava.library.path=${hadoop.library.path};${java.library.path}"
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestSplitLogManager
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11197//console
This message is automatically generated.
> Use Jacoco to generate Unit Test coverage reports
> -
>
> Key: HBASE-8808
> URL: https://issues.apache.org/jira/browse/HBASE-8808
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.89-fb
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Trivial
> Fix For: 0.89-fb
>
> Attachments: 0001-Including-Jacoco-plugin-to-get-test-coverage.patch,
> 0001-Including-Jacoco-plugin-to-get-test-coverage.patch, Screen Shot
> 2013-06-25 at 11.35.30 AM.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Enabling the code coverage tool jacoco in maven
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12104) Some optimization and bugfix for HTableMultiplexer
[
https://issues.apache.org/jira/browse/HBASE-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157284#comment-14157284
]
Hadoop QA commented on HBASE-12104:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12672623/0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch
against trunk revision .
ATTACHMENT ID: 12672623
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11195//console
This message is automatically generated.
> Some optimization and bugfix for HTableMultiplexer
> --
>
> Key: HBASE-12104
> URL: https://issues.apache.org/jira/browse/HBASE-12104
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Yi Deng
>Assignee: Yi Deng
> Labels: multiplexer
> Fix For: 2.0.0
>
> Attachments:
> 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch,
> 0001-Make-HTableMultiplexerStatus-public-Delay-before-res.patch,
> 0001-Make-HTableMultiplexerStatus-public.patch
>
>
> Make HTableMultiplexerStatus public
> Delay before resubmit.
> Fix some missing counting on total failure.
> Use ScheduledExecutorService to simplify the code.
> Other refactoring.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157262#comment-14157262
]
stack commented on HBASE-12152:
---
Why does it fix the problem?
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12162) HBaseAdmin#getTableDescriptor() may fail in case master fails over
[
https://issues.apache.org/jira/browse/HBASE-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12162:
---
Attachment: 12162-v1.txt
Proposed patch makes use of MasterCallable
> HBaseAdmin#getTableDescriptor() may fail in case master fails over
> --
>
> Key: HBASE-12162
> URL: https://issues.apache.org/jira/browse/HBASE-12162
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12162-v1.txt
>
>
> This was discovered by Chakradhar Medavarapu during HA testing.
> Here is relevant exception:
> {code}
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|14/09/30
> 04:07:56 ERROR util.AbstractHBaseTool: Error running command-line tool
> 2014-09-30
> 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|java.io.IOException:
> Call to onprem-ha34/10.215.18.85:6 failed on local exception:
> java.io.IOException: Call id=1, waitTime=8703
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1571)
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1541)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1723)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1776)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getTableDescriptors(MasterProtos.java:42525)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$5.getTableDescriptors(ConnectionManager.java:2121)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getHTableDescriptor(ConnectionManager.java:2600)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:410)
> {code}
> From stack trace, exception came out of connection.getHTableDescriptor().
> This happened during master failover where MasterKeepAliveConnection to the
> failed master became unusable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12162) HBaseAdmin#getTableDescriptor() may fail in case master fails over
[
https://issues.apache.org/jira/browse/HBASE-12162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12162:
---
Status: Patch Available (was: Open)
> HBaseAdmin#getTableDescriptor() may fail in case master fails over
> --
>
> Key: HBASE-12162
> URL: https://issues.apache.org/jira/browse/HBASE-12162
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 12162-v1.txt
>
>
> This was discovered by Chakradhar Medavarapu during HA testing.
> Here is relevant exception:
> {code}
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|14/09/30
> 04:07:56 ERROR util.AbstractHBaseTool: Error running command-line tool
> 2014-09-30
> 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|java.io.IOException:
> Call to onprem-ha34/10.215.18.85:6 failed on local exception:
> java.io.IOException: Call id=1, waitTime=8703
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1571)
> 2014-09-30 04:07:56,734|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1541)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1723)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1776)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getTableDescriptors(MasterProtos.java:42525)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$5.getTableDescriptors(ConnectionManager.java:2121)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getHTableDescriptor(ConnectionManager.java:2600)
> 2014-09-30 04:07:56,736|beaver.machine|INFO|5728|5604|MainThread|at
> org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:410)
> {code}
> From stack trace, exception came out of connection.getHTableDescriptor().
> This happened during master failover where MasterKeepAliveConnection to the
> failed master became unusable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12152:
---
Attachment: 12152-v3.txt
Proposed fix
Thanks to Jeffrey for suggestion.
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-12152:
---
Status: Patch Available (was: Open)
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 12152-v3.txt
>
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12152) TestLoadIncrementalHFiles shows up as zombie test
[
https://issues.apache.org/jira/browse/HBASE-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157213#comment-14157213
]
Hudson commented on HBASE-12152:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #533 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/533/])
HBASE-12152 TestLoadIncrementalHFiles shows up as zombie test; ADD TIMEOUT ON
TESTS -- Up timeout from 120 to 12 (stack: rev
d2111dbda4a874224f9f4f0ba9d16edcc104c98a)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java
> TestLoadIncrementalHFiles shows up as zombie test
> -
>
> Key: HBASE-12152
> URL: https://issues.apache.org/jira/browse/HBASE-12152
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> TestLoadIncrementalHFiles and TestLoadIncrementalHFilesSplitRecovery
> frequently show up as zombie tests (from 0.98 to master branch).
> e.g. https://builds.apache.org/job/hbase-0.98/558/console
> Here is snippet of stack trace for TestLoadIncrementalHFilesSplitRecovery :
> {code}
> "main" prio=10 tid=0x7f8670008000 nid=0x1105 waiting on condition
> [0x7f8674b57000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00078d4c3ba0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:382)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:324)
> at
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testGroupOrSplitWhenRegionHoleExistsInMeta(TestLoadIncrementalHFilesSplitRecovery.
> java:470)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
