[jira] [Updated] (HBASE-14567) Disable hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-14567: - Status: Patch Available (was: Open) > Disable hanging test TestMobCompactor and TestMobSweeper > > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14567) Disable hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-14567: - Attachment: HBASE-14567.patch I ran the tests for a few test, and they could pass. But they took long time, TestMobSweeper took almost 300 seconds, TestMobCompactor took almost 400 seconds. Since the Sweeper is not recommended anymore, I plan to remove it from the code and I will ignore this test in code. I reviewed the code in TestMobCompactor and merged a few cases into one to reduce the run time. Now the patch is uploaded. Thanks. > Disable hanging test TestMobCompactor and TestMobSweeper > > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14490) [RpcServer] reuse request read buffer
[
https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950155#comment-14950155
]
Anoop Sam John commented on HBASE-14490:
We keep on adding to the reqTotalSize. Possible overflow?
Do we really need this? If we do to this extend and handle both smaller
sized reqs and larger sized (write req) we better use a BufferPool? We already
have it and using it while sending back response. So the older version for
reusing smaller sized BB is better? Any test result numbers? I know u
consider the read reqs
> [RpcServer] reuse request read buffer
> -
>
> Key: HBASE-14490
> URL: https://issues.apache.org/jira/browse/HBASE-14490
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
>Affects Versions: 2.0.0, 1.0.2
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
> Labels: performance
> Fix For: 2.0.0, 1.0.2
>
> Attachments: ByteBufferPool.java, HBASE-14490-v1.patch,
> HBASE-14490-v10.patch, HBASE-14490-v11.patch, HBASE-14490-v2.patch,
> HBASE-14490-v3.patch, HBASE-14490-v4.patch, HBASE-14490-v5.patch,
> HBASE-14490-v6.patch, HBASE-14490-v7.patch, HBASE-14490-v8.patch,
> HBASE-14490-v9.patch
>
>
> Reuse buffer to read request.It's not necessary to every request free
> buffer.The idea of optimization is to reduce the times that allocate
> ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}},
> {{processConnectionHeader}} accept a ByteBuffer instead of byte[].They can
> move {{ByteBuffer.position}} correctly when we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a reused ByteBuffer in each {{Connection}}.
> a) Size is dynamic to fit specific client.
> b) If size deviate far from average size, we replace a new one.
> c) Using a SoftReference to reference the buffer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14567) Disable hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950116#comment-14950116 ] Jingcheng Du commented on HBASE-14567: -- To supplement some information for this patch, I added timeout parameter for tests in TestMobCompactor. > Disable hanging test TestMobCompactor and TestMobSweeper > > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[
https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951369#comment-14951369
]
Hadoop QA commented on HBASE-14567:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765915/14567v2.txt
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765915
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15932//console
This message is automatically generated.
> Tuneup hanging test TestMobCompactor and TestMobSweeper
> ---
>
> Key: HBASE-14567
> URL: https://issues.apache.org/jira/browse/HBASE-14567
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: stack
>Assignee: Jingcheng Du
> Attachments: 14567v2.txt, HBASE-14567.patch
>
>
> These tests hang with some regularity. Please take a look
> [~jingcheng...@intel.com] since I believe they are yours. If not, let me know
> and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks.
> They hung here most recently:
> https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14558) Document ChaosMonkey enhancements from HBASE-14261
[ https://issues.apache.org/jira/browse/HBASE-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-14558: - Fix Version/s: (was: 1.1.3) (was: 1.0.3) (was: 0.98.15) (was: 1.3.0) (was: 1.2.0) Dropping non-master fixVersions. > Document ChaosMonkey enhancements from HBASE-14261 > -- > > Key: HBASE-14558 > URL: https://issues.apache.org/jira/browse/HBASE-14558 > Project: HBase > Issue Type: Sub-task > Components: documentation >Affects Versions: 2.0.0 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 2.0.0 > > Attachments: HBASE-14558.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14501) NPE in replication with TDE
[
https://issues.apache.org/jira/browse/HBASE-14501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951377#comment-14951377
]
Nick Dimiduk commented on HBASE-14501:
--
Green lights from that manual test [~enis]?
> NPE in replication with TDE
> ---
>
> Key: HBASE-14501
> URL: https://issues.apache.org/jira/browse/HBASE-14501
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14501_v1.patch
>
>
> We are seeing a NPE when replication (or in this case async wal replay for
> region replicas) is run on top of an HDFS cluster with TDE configured.
> This is the stack trace:
> {code}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.CellUtil.matchingRow(CellUtil.java:370)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.countDistinctRowKeys(ReplicationSource.java:649)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:450)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:346)
> {code}
> This stack trace can only happen if WALEdit.getCells() returns an array
> containing null entries. I believe this happens due to
> {{KeyValueCodec.parseCell()}} uses {{KeyValueUtil.iscreate()}} which returns
> null in case of EOF at the beginning. However, the contract for the
> Decoder.parseCell() is not clear whether returning null is acceptable or not.
> The other Decoders (CompressedKvDecoder, CellCodec, etc) do not return null
> while KeyValueCodec does.
> BaseDecoder has this code:
> {code}
> public boolean advance() throws IOException {
> if (!this.hasNext) return this.hasNext;
> if (this.in.available() == 0) {
> this.hasNext = false;
> return this.hasNext;
> }
> try {
> this.current = parseCell();
> } catch (IOException ioEx) {
> rethrowEofException(ioEx);
> }
> return this.hasNext;
> }
> {code}
> which is not correct since it uses {{IS.available()}} not according to the
> javadoc:
> (https://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#available()).
> DFSInputStream implements {{available()}} as the remaining bytes to read
> from the stream, so we do not see the issue there.
> {{CryptoInputStream.available()}} does a similar thing but see the issue.
> So two questions:
> - What should be the interface for Decoder.parseCell()? Can it return null?
> - How to properly fix BaseDecoder.advance() to not rely on {{available()}}
> call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14587) Attach a test-sources.jar for hbase-server
Andrew Wang created HBASE-14587: --- Summary: Attach a test-sources.jar for hbase-server Key: HBASE-14587 URL: https://issues.apache.org/jira/browse/HBASE-14587 Project: HBase Issue Type: Improvement Affects Versions: 1.1.0 Reporter: Andrew Wang Assignee: Andrew Wang It'd be nice to attach a test-sources jar alongside the others as part of the build, to provide test resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-14584:
-
Assignee: (was: Nick Dimiduk)
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14588) Stop accessing test resources from within src folder
Andrew Wang created HBASE-14588: --- Summary: Stop accessing test resources from within src folder Key: HBASE-14588 URL: https://issues.apache.org/jira/browse/HBASE-14588 Project: HBase Issue Type: Improvement Affects Versions: 1.1.0 Reporter: Andrew Wang Assignee: Andrew Wang A few tests in hbase-server reach into the src/test/data folder to get test resources, which is naughty since tests are supposed to only operate within the target/ folder. It's better to put these into src/test/resources and let them be automatically copied into target/ via the resources plugin, like other test resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14536) Balancer & SSH interfering with each other leading to unavailability
[
https://issues.apache.org/jira/browse/HBASE-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-14536:
---
Attachment: (was: HBASE-14536.draft-branch-1.patch)
> Balancer & SSH interfering with each other leading to unavailability
>
>
> Key: HBASE-14536
> URL: https://issues.apache.org/jira/browse/HBASE-14536
> Project: HBase
> Issue Type: Bug
> Components: master, Region Assignment
>Affects Versions: 1.1.2
>Reporter: Devaraj Das
>Assignee: Stephen Yuan Jiang
> Fix For: 1.1.4
>
> Attachments: HBASE-14536.v1-branch-1.patch, master-log.tgz
>
>
> Came across this in our cluster:
> 1. The meta was assigned to a server 10.0.0.149,16020,1443507203340
> {noformat}
> 2015-09-29 06:16:22,472 DEBUG [AM.ZK.Worker-pool2-t56]
> master.RegionStates: Onlined 1588230740 on
> 10.0.0.149,16020,1443507203340 {ENCODED => 1588230740, NAME =>
> 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
> {noformat}
> 2. The server dies at some point:
> {noformat}
> 2015-09-29 06:18:25,952 INFO [main-EventThread]
> zookeeper.RegionServerTracker: RegionServer ephemeral node deleted,
> processing expiration [10.0.0.149,16020,1443507203340]
> 2015-09-29 06:18:25,955 DEBUG [main-EventThread] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=10.0.0.149,16020,1443507203340
> server being checked:
> 10.0.0.149,16020,1443507203340
> {noformat}
> 3. The balancer had computed a plan that contained a move for the meta:
> {noformat}
> 2015-09-29 06:18:26,833 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.HMaster:
> balance hri=hbase:meta,,1.1588230740,
> src=10.0.0.149,16020,1443507203340, dest=10.0.0.205,16020,1443507257905
> {noformat}
> 4. The following ensues after this, leading to the meta remaining unassigned:
> {noformat}
> 2015-09-29 06:18:26,859 DEBUG
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Offline hbase:meta,,1.1588230740, no need to
> unassign since it's on a dead server: 10.0.0.149,16020,1443507203340
> ..
> 2015-09-29 06:18:26,899 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.RegionStates:
> Offlined 1588230740 from 10.0.0.149,16020,1443507203340
> .
> 2015-09-29 06:18:26,914 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Skip assigning hbase:meta,,1.1588230740, it is
> on a dead but not processed yet server: 10.0.0.149,16020,1443507203340
>
> 2015-09-29 06:18:26,915 DEBUG [AM.ZK.Worker-pool2-t58]
> master.AssignmentManager: Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OFFLINE, ts=1443507506914,
> server=10.0.0.149,16020,1443507203340}
>
> 2015-09-29 06:18:29,447 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=null server being checked:
> 10.0.0.149,16020,1443507203340
> 2015-09-29 06:18:29,451 INFO [MASTER_META_SERVER_OPERATIONS-
> 10.0.0.148:16000-2] handler.MetaServerShutdownHandler: META has been
> assigned to otherwhere, skip assigning.
> 2015-09-29 06:18:29,452 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2]
> master.DeadServer: Finished processing 10.0.0.149,16020,1443507203340
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14536) Balancer & SSH interfering with each other leading to unavailability
[
https://issues.apache.org/jira/browse/HBASE-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-14536:
---
Attachment: HBASE-14536.v1-branch-1.patch
> Balancer & SSH interfering with each other leading to unavailability
>
>
> Key: HBASE-14536
> URL: https://issues.apache.org/jira/browse/HBASE-14536
> Project: HBase
> Issue Type: Bug
> Components: master, Region Assignment
>Affects Versions: 1.1.2
>Reporter: Devaraj Das
>Assignee: Stephen Yuan Jiang
> Fix For: 1.1.4
>
> Attachments: HBASE-14536.v1-branch-1.patch, master-log.tgz
>
>
> Came across this in our cluster:
> 1. The meta was assigned to a server 10.0.0.149,16020,1443507203340
> {noformat}
> 2015-09-29 06:16:22,472 DEBUG [AM.ZK.Worker-pool2-t56]
> master.RegionStates: Onlined 1588230740 on
> 10.0.0.149,16020,1443507203340 {ENCODED => 1588230740, NAME =>
> 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
> {noformat}
> 2. The server dies at some point:
> {noformat}
> 2015-09-29 06:18:25,952 INFO [main-EventThread]
> zookeeper.RegionServerTracker: RegionServer ephemeral node deleted,
> processing expiration [10.0.0.149,16020,1443507203340]
> 2015-09-29 06:18:25,955 DEBUG [main-EventThread] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=10.0.0.149,16020,1443507203340
> server being checked:
> 10.0.0.149,16020,1443507203340
> {noformat}
> 3. The balancer had computed a plan that contained a move for the meta:
> {noformat}
> 2015-09-29 06:18:26,833 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.HMaster:
> balance hri=hbase:meta,,1.1588230740,
> src=10.0.0.149,16020,1443507203340, dest=10.0.0.205,16020,1443507257905
> {noformat}
> 4. The following ensues after this, leading to the meta remaining unassigned:
> {noformat}
> 2015-09-29 06:18:26,859 DEBUG
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Offline hbase:meta,,1.1588230740, no need to
> unassign since it's on a dead server: 10.0.0.149,16020,1443507203340
> ..
> 2015-09-29 06:18:26,899 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.RegionStates:
> Offlined 1588230740 from 10.0.0.149,16020,1443507203340
> .
> 2015-09-29 06:18:26,914 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Skip assigning hbase:meta,,1.1588230740, it is
> on a dead but not processed yet server: 10.0.0.149,16020,1443507203340
>
> 2015-09-29 06:18:26,915 DEBUG [AM.ZK.Worker-pool2-t58]
> master.AssignmentManager: Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OFFLINE, ts=1443507506914,
> server=10.0.0.149,16020,1443507203340}
>
> 2015-09-29 06:18:29,447 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=null server being checked:
> 10.0.0.149,16020,1443507203340
> 2015-09-29 06:18:29,451 INFO [MASTER_META_SERVER_OPERATIONS-
> 10.0.0.148:16000-2] handler.MetaServerShutdownHandler: META has been
> assigned to otherwhere, skip assigning.
> 2015-09-29 06:18:29,452 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2]
> master.DeadServer: Finished processing 10.0.0.149,16020,1443507203340
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14536) Balancer & SSH interfering with each other leading to unavailability
[
https://issues.apache.org/jira/browse/HBASE-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-14536:
---
Status: Patch Available (was: Open)
> Balancer & SSH interfering with each other leading to unavailability
>
>
> Key: HBASE-14536
> URL: https://issues.apache.org/jira/browse/HBASE-14536
> Project: HBase
> Issue Type: Bug
> Components: master, Region Assignment
>Affects Versions: 1.1.2
>Reporter: Devaraj Das
>Assignee: Stephen Yuan Jiang
> Fix For: 1.1.4
>
> Attachments: HBASE-14536.v1-branch-1.patch, master-log.tgz
>
>
> Came across this in our cluster:
> 1. The meta was assigned to a server 10.0.0.149,16020,1443507203340
> {noformat}
> 2015-09-29 06:16:22,472 DEBUG [AM.ZK.Worker-pool2-t56]
> master.RegionStates: Onlined 1588230740 on
> 10.0.0.149,16020,1443507203340 {ENCODED => 1588230740, NAME =>
> 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
> {noformat}
> 2. The server dies at some point:
> {noformat}
> 2015-09-29 06:18:25,952 INFO [main-EventThread]
> zookeeper.RegionServerTracker: RegionServer ephemeral node deleted,
> processing expiration [10.0.0.149,16020,1443507203340]
> 2015-09-29 06:18:25,955 DEBUG [main-EventThread] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=10.0.0.149,16020,1443507203340
> server being checked:
> 10.0.0.149,16020,1443507203340
> {noformat}
> 3. The balancer had computed a plan that contained a move for the meta:
> {noformat}
> 2015-09-29 06:18:26,833 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.HMaster:
> balance hri=hbase:meta,,1.1588230740,
> src=10.0.0.149,16020,1443507203340, dest=10.0.0.205,16020,1443507257905
> {noformat}
> 4. The following ensues after this, leading to the meta remaining unassigned:
> {noformat}
> 2015-09-29 06:18:26,859 DEBUG
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Offline hbase:meta,,1.1588230740, no need to
> unassign since it's on a dead server: 10.0.0.149,16020,1443507203340
> ..
> 2015-09-29 06:18:26,899 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.RegionStates:
> Offlined 1588230740 from 10.0.0.149,16020,1443507203340
> .
> 2015-09-29 06:18:26,914 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Skip assigning hbase:meta,,1.1588230740, it is
> on a dead but not processed yet server: 10.0.0.149,16020,1443507203340
>
> 2015-09-29 06:18:26,915 DEBUG [AM.ZK.Worker-pool2-t58]
> master.AssignmentManager: Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OFFLINE, ts=1443507506914,
> server=10.0.0.149,16020,1443507203340}
>
> 2015-09-29 06:18:29,447 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=null server being checked:
> 10.0.0.149,16020,1443507203340
> 2015-09-29 06:18:29,451 INFO [MASTER_META_SERVER_OPERATIONS-
> 10.0.0.148:16000-2] handler.MetaServerShutdownHandler: META has been
> assigned to otherwhere, skip assigning.
> 2015-09-29 06:18:29,452 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2]
> master.DeadServer: Finished processing 10.0.0.149,16020,1443507203340
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14536) Balancer & SSH interfering with each other leading to unavailability
[
https://issues.apache.org/jira/browse/HBASE-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951485#comment-14951485
]
Hadoop QA commented on HBASE-14536:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765945/HBASE-14536.v1-branch-1.patch
against branch-1 branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765945
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15940//console
This message is automatically generated.
> Balancer & SSH interfering with each other leading to unavailability
>
>
> Key: HBASE-14536
> URL: https://issues.apache.org/jira/browse/HBASE-14536
> Project: HBase
> Issue Type: Bug
> Components: master, Region Assignment
>Affects Versions: 1.1.2
>Reporter: Devaraj Das
>Assignee: Stephen Yuan Jiang
> Fix For: 1.1.4
>
> Attachments: HBASE-14536.v1-branch-1.patch, master-log.tgz
>
>
> Came across this in our cluster:
> 1. The meta was assigned to a server 10.0.0.149,16020,1443507203340
> {noformat}
> 2015-09-29 06:16:22,472 DEBUG [AM.ZK.Worker-pool2-t56]
> master.RegionStates: Onlined 1588230740 on
> 10.0.0.149,16020,1443507203340 {ENCODED => 1588230740, NAME =>
> 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
> {noformat}
> 2. The server dies at some point:
> {noformat}
> 2015-09-29 06:18:25,952 INFO [main-EventThread]
> zookeeper.RegionServerTracker: RegionServer ephemeral node deleted,
> processing expiration [10.0.0.149,16020,1443507203340]
> 2015-09-29 06:18:25,955 DEBUG [main-EventThread] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=10.0.0.149,16020,1443507203340
> server being checked:
> 10.0.0.149,16020,1443507203340
> {noformat}
> 3. The balancer had computed a plan that contained a move for the meta:
> {noformat}
> 2015-09-29 06:18:26,833 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.HMaster:
> balance hri=hbase:meta,,1.1588230740,
> src=10.0.0.149,16020,1443507203340, dest=10.0.0.205,16020,1443507257905
> {noformat}
> 4. The following ensues after this, leading to the meta remaining unassigned:
> {noformat}
> 2015-09-29 06:18:26,859 DEBUG
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Offline hbase:meta,,1.1588230740, no need to
> unassign since it's on a dead server: 10.0.0.149,16020,1443507203340
> ..
> 2015-09-29 06:18:26,899 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.RegionStates:
> Offlined 1588230740 from 10.0.0.149,16020,1443507203340
> .
> 2015-09-29 06:18:26,914 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Skip assigning hbase:meta,,1.1588230740, it is
> on a dead but not processed yet server: 10.0.0.149,16020,1443507203340
>
> 2015-09-29 06:18:26,915 DEBUG [AM.ZK.Worker-pool2-t58]
> master.AssignmentManager: Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OFFLINE, ts=1443507506914,
> server=10.0.0.149,16020,1443507203340}
>
> 2015-09-29 06:18:29,447 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=null server being checked:
> 10.0.0.149,16020,1443507203340
> 2015-09-29 06:18:29,451 INFO [MASTER_META_SERVER_OPERATIONS-
> 10.0.0.148:16000-2] handler.MetaServerShutdownHandler: META has been
> assigned to otherwhere, skip assigning.
> 2015-09-29 06:18:29,452 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2]
> master.DeadServer: Finished processing 10.0.0.149,16020,1443507203340
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14557) MapReduce WALPlayer issue with NoTagsKeyValue
[
https://issues.apache.org/jira/browse/HBASE-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951507#comment-14951507
]
Hadoop QA commented on HBASE-14557:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765918/HBASE-14557.patch
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765918
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15933//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15933//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15933//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15933//console
This message is automatically generated.
> MapReduce WALPlayer issue with NoTagsKeyValue
> -
>
> Key: HBASE-14557
> URL: https://issues.apache.org/jira/browse/HBASE-14557
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Jerry He
>Assignee: Anoop Sam John
> Attachments: HBASE-14557.patch, HBASE-14557.patch
>
>
> Running MapReduce WALPlayer to convert WAL into HFiles:
> {noformat}
> 15/10/05 20:28:08 INFO mapred.JobClient: Task Id :
> attempt_201508031611_0029_m_00_0, Status : FAILED
> java.io.IOException: Type mismatch in value from map: expected
> org.apache.hadoop.hbase.KeyValue, recieved
> org.apache.hadoop.hbase.NoTagsKeyValue
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:997)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:96)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at
> java.security.AccessController.doPrivileged(AccessController.java:369)
> at javax.security.auth.Subject.doAs(Subject.java:572)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14580) Make the HBaseMiniCluster compliant with Kerberos
[
https://issues.apache.org/jira/browse/HBASE-14580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951381#comment-14951381
]
Gary Helmling commented on HBASE-14580:
---
The "no groups" warning should be harmless unless you're trying to test
something like group based authorization. I think it's just a log spam issue
-- see HADOOP-12450.
> Make the HBaseMiniCluster compliant with Kerberos
> -
>
> Key: HBASE-14580
> URL: https://issues.apache.org/jira/browse/HBASE-14580
> Project: HBase
> Issue Type: Improvement
> Components: security, test
>Affects Versions: 2.0.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
> Fix For: 2.0.0
>
> Attachments: patch-14580.v1.patch
>
>
> Whne using MiniKDC and the minicluster in a unit test, there is a conflict
> causeed by HBaseTestingUtility:
> {code}
> public static User getDifferentUser(final Configuration c,
> final String differentiatingSuffix)
> throws IOException {
>// snip
> String username = User.getCurrent().getName() +
> differentiatingSuffix; < problem here
> User user = User.createUserForTesting(c, username,
> new String[]{"supergroup"});
> return user;
> }
> {code}
> This creates users like securedUser/localh...@example.com.hfs.0, and this
> does not work.
> My fix is to return the current user when Kerberos is set. I don't think that
> there is another option (any other opinion?). However this user is not in a
> group so we have logs like 'WARN [IPC Server handler 9 on 61366]
> security.UserGroupInformation (UserGroupInformation.java:getGroupNames(1521))
> - No groups available for user securedUser' I'm not sure of its impact.
> [~apurtell], what do you think?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-14584:
-
Attachment: 14584.00.patch
How about this?
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-14584:
-
Status: Patch Available (was: Open)
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reassigned HBASE-14584:
Assignee: Nick Dimiduk
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14578) URISyntaxException during snapshot restore
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14578:
---
Attachment: 14578-master-v2.txt
> URISyntaxException during snapshot restore
> --
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 0.98.16
>
> Attachments: 14578-master-v2.txt, 14578-master.txt,
> HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter regions contain the reference of the
> original HFile until compaction happens.
> In RestoreSnapshotHelper (while restoring snapshot file which have regions
> containing hfile references), path is initialized using the snapshot table
> name which has namespace separated by
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951450#comment-14951450
]
Hudson commented on HBASE-14584:
FAILURE: Integrated in HBase-1.2 #236 (See
[https://builds.apache.org/job/HBase-1.2/236/])
HBASE-14584 TestNamespacesInstanceModel fails on jdk8; DISABLE TEST (stack: rev
c46a6fff810313007d6632733fa9035e2c158202)
* hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestModelBase.java
*
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestNamespacesInstanceModel.java
*
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14536) Balancer & SSH interfering with each other leading to unavailability
[
https://issues.apache.org/jira/browse/HBASE-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951455#comment-14951455
]
Stephen Yuan Jiang commented on HBASE-14536:
Discussed with [~enis], we can go a different approach - in the MetaSSH, we can
check the znode of META location, if the location matches the dead server, we
are sure that the AM would let SSH handle the assignment; therefore, it is safe
for MetaSSH to assume that he is responsible for the META region assignment.
> Balancer & SSH interfering with each other leading to unavailability
>
>
> Key: HBASE-14536
> URL: https://issues.apache.org/jira/browse/HBASE-14536
> Project: HBase
> Issue Type: Bug
> Components: master, Region Assignment
>Affects Versions: 1.1.2
>Reporter: Devaraj Das
>Assignee: Stephen Yuan Jiang
> Fix For: 1.1.4
>
> Attachments: HBASE-14536.v1-branch-1.patch, master-log.tgz
>
>
> Came across this in our cluster:
> 1. The meta was assigned to a server 10.0.0.149,16020,1443507203340
> {noformat}
> 2015-09-29 06:16:22,472 DEBUG [AM.ZK.Worker-pool2-t56]
> master.RegionStates: Onlined 1588230740 on
> 10.0.0.149,16020,1443507203340 {ENCODED => 1588230740, NAME =>
> 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
> {noformat}
> 2. The server dies at some point:
> {noformat}
> 2015-09-29 06:18:25,952 INFO [main-EventThread]
> zookeeper.RegionServerTracker: RegionServer ephemeral node deleted,
> processing expiration [10.0.0.149,16020,1443507203340]
> 2015-09-29 06:18:25,955 DEBUG [main-EventThread] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=10.0.0.149,16020,1443507203340
> server being checked:
> 10.0.0.149,16020,1443507203340
> {noformat}
> 3. The balancer had computed a plan that contained a move for the meta:
> {noformat}
> 2015-09-29 06:18:26,833 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.HMaster:
> balance hri=hbase:meta,,1.1588230740,
> src=10.0.0.149,16020,1443507203340, dest=10.0.0.205,16020,1443507257905
> {noformat}
> 4. The following ensues after this, leading to the meta remaining unassigned:
> {noformat}
> 2015-09-29 06:18:26,859 DEBUG
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Offline hbase:meta,,1.1588230740, no need to
> unassign since it's on a dead server: 10.0.0.149,16020,1443507203340
> ..
> 2015-09-29 06:18:26,899 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.RegionStates:
> Offlined 1588230740 from 10.0.0.149,16020,1443507203340
> .
> 2015-09-29 06:18:26,914 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Skip assigning hbase:meta,,1.1588230740, it is
> on a dead but not processed yet server: 10.0.0.149,16020,1443507203340
>
> 2015-09-29 06:18:26,915 DEBUG [AM.ZK.Worker-pool2-t58]
> master.AssignmentManager: Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OFFLINE, ts=1443507506914,
> server=10.0.0.149,16020,1443507203340}
>
> 2015-09-29 06:18:29,447 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=null server being checked:
> 10.0.0.149,16020,1443507203340
> 2015-09-29 06:18:29,451 INFO [MASTER_META_SERVER_OPERATIONS-
> 10.0.0.148:16000-2] handler.MetaServerShutdownHandler: META has been
> assigned to otherwhere, skip assigning.
> 2015-09-29 06:18:29,452 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2]
> master.DeadServer: Finished processing 10.0.0.149,16020,1443507203340
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14586) Remove extraneous ${argLine} references in pom
[
https://issues.apache.org/jira/browse/HBASE-14586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951463#comment-14951463
]
Duo Zhang commented on HBASE-14586:
---
Can you please verify if jacoco coverage is still working after this change?
The jacoco-maven-plugin will also set argLine to attach the agent lib and some
configurations, see this http://stackoverflow.com/a/25774988
Of course this is not the only solution, maybe we have already fixed this
problem at other place in the pom, so just give a try.
Thanks.
> Remove extraneous ${argLine} references in pom
> --
>
> Key: HBASE-14586
> URL: https://issues.apache.org/jira/browse/HBASE-14586
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Attachments: hbase-14586.001.patch
>
>
> The pom.xml has a line like this for the Surefire argLine, which has an extra
> ${argLine} reference. Recommend changes like this:
> {noformat}
> -${hbase-surefire.argLine} ${argLine}
> +${hbase-surefire.argLine}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14268) Improve KeyLocker
[
https://issues.apache.org/jira/browse/HBASE-14268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951503#comment-14951503
]
Hadoop QA commented on HBASE-14268:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765929/HBASE-14268-V7.patch
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765929
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15936//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15936//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15936//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15936//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15936//console
This message is automatically generated.
> Improve KeyLocker
> -
>
> Key: HBASE-14268
> URL: https://issues.apache.org/jira/browse/HBASE-14268
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 14268-V5.patch, HBASE-14268-V2.patch,
> HBASE-14268-V3.patch, HBASE-14268-V4.patch, HBASE-14268-V5.patch,
> HBASE-14268-V5.patch, HBASE-14268-V6.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268.patch,
> KeyLockerIncrKeysPerformance.java, KeyLockerPerformance.java,
> ReferenceTestApp.java
>
>
> 1. In the implementation of {{KeyLocker}} it uses atomic variables inside a
> synchronized block, which doesn't make sense. Moreover, logic inside the
> synchronized block is not trivial so that it makes less performance in heavy
> multi-threaded environment.
> 2. {{KeyLocker}} gives an instance of {{RentrantLock}} which is already
> locked, but it doesn't follow the contract of {{ReentrantLock}} because you
> are not allowed to freely invoke lock/unlock methods under that contract.
> That introduces a potential risk; Whenever you see a variable of the type
> {{RentrantLock}}, you should pay attention to what the included instance is
> coming from.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14398) Create the fake keys required in the scan path to avoid copy to byte[]
[ https://issues.apache.org/jira/browse/HBASE-14398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951509#comment-14951509 ] stack commented on HBASE-14398: --- Thank you for entertaining my commentary. bq. IMO also getRowByteBuffer is better than getRow. You have a point. It would match getRowArray up in Cell. bq. So Stack, you agree to the idea of not throwing Exception when we call getXXXArray() on a DBB backed cell. So here we will copy and return the byte[]. bq. Regarding getXXXOffset comment, u mean when the Cell is DBB backed, the user has to assume the offset to be 0 always? That means again every where we have to have instance of BBCell check. I was thinking they would do getXXXOffset and if they'd used getXXXArray and a copy had been made, then it would break horribly. I suppose your point is that we need to smooth the transition so filters and CPs just keep working over offheap (or Cells that are split over more than one BB) if they use the old APIs even if it costs loads. Ok. You two have made this argument with me a few times. We should integrate the why into the Interface doc? I can write it up if that'd help. Ok on the getXXXPosition. > Create the fake keys required in the scan path to avoid copy to byte[] > -- > > Key: HBASE-14398 > URL: https://issues.apache.org/jira/browse/HBASE-14398 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-14398.patch, HBASE-14398_1.patch > > > Already we have created some fake keys for the ByteBufferedCells so that we > can avoid the copy requried to create fake keys. This JIRA aims to fill up > all such places so that the Offheap BBs are not copied to onheap byte[]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14268) Improve KeyLocker
[
https://issues.apache.org/jira/browse/HBASE-14268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951513#comment-14951513
]
stack commented on HBASE-14268:
---
Are we into a new place where our patch builds are killing each other:
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/test-framework/dev-support/test-patch.sh:
line 838: 29466 Killed $MVN clean test
-Dsurefire.rerunFailingTestsCount=2 -P runAllTests -D${PROJECT_NAME}PatchProcess
We're ok: there is no zombie test
test-patch is doing this:
### Kill any rogue build processes from the last attempt
condemnedCount=`$PS auxwww | $GREP ${PROJECT_NAME}PatchProcess | $AWK '{print
$2}' | $AWK 'BEGIN {total = 0} {total += 1} END {print total}'`
echo "WARNING: $condemnedCount rogue build processes detected, terminating."
$PS auxwww | $GREP ${PROJECT_NAME}PatchProcess | $AWK '{print $2}' |
/usr/bin/xargs -t -I {} /bin/kill -9 {} > /dev/null
In this case it did this:
WARNING: 2 rogue build processes detected, terminating.
/bin/kill -9 29055
/bin/kill -9 29461
...and then our process was killed with this "test-patch.sh: line 838: 29466
Killed ".. Ours was a different PID
Maybe I should remove this killing code too only I've already removed killing
of surefirebooters that were running concurrently
> Improve KeyLocker
> -
>
> Key: HBASE-14268
> URL: https://issues.apache.org/jira/browse/HBASE-14268
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 14268-V5.patch, HBASE-14268-V2.patch,
> HBASE-14268-V3.patch, HBASE-14268-V4.patch, HBASE-14268-V5.patch,
> HBASE-14268-V5.patch, HBASE-14268-V6.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268.patch, KeyLockerIncrKeysPerformance.java,
> KeyLockerPerformance.java, ReferenceTestApp.java
>
>
> 1. In the implementation of {{KeyLocker}} it uses atomic variables inside a
> synchronized block, which doesn't make sense. Moreover, logic inside the
> synchronized block is not trivial so that it makes less performance in heavy
> multi-threaded environment.
> 2. {{KeyLocker}} gives an instance of {{RentrantLock}} which is already
> locked, but it doesn't follow the contract of {{ReentrantLock}} because you
> are not allowed to freely invoke lock/unlock methods under that contract.
> That introduces a potential risk; Whenever you see a variable of the type
> {{RentrantLock}}, you should pay attention to what the included instance is
> coming from.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14587) Attach a test-sources.jar for hbase-server
[ https://issues.apache.org/jira/browse/HBASE-14587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951524#comment-14951524 ] stack commented on HBASE-14587: --- Patch looks good. > Attach a test-sources.jar for hbase-server > -- > > Key: HBASE-14587 > URL: https://issues.apache.org/jira/browse/HBASE-14587 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: hbase-14587.001.patch > > > It'd be nice to attach a test-sources jar alongside the others as part of the > build, to provide test resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14582) Regionserver status webpage bucketcache list can become huge
[ https://issues.apache.org/jira/browse/HBASE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14582: -- Assignee: stack Status: Patch Available (was: Open) > Regionserver status webpage bucketcache list can become huge > > > Key: HBASE-14582 > URL: https://issues.apache.org/jira/browse/HBASE-14582 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: James Hartshorn >Assignee: stack >Priority: Trivial > Attachments: 14582.txt, Screen Shot 2015-10-09 at 8.00.37 PM.png, > Screen Shot 2015-10-09 at 8.00.37 PM.png > > > The regionserver status page, such as http://127.0.0.1:60030/rs-status always > downloads information about every bucket in the cache. In some cases this > can be hundreds of thousands of buckets, causing megabytes of info to need to > be downloaded and significant browser instability and memory usage: > wc -l HBase-Region-Server-hb22.html > 2010116 HBase-Region-Server-hb22.html > ls -lah HBase-Region-Server-hb22.html > 32M Oct 6 19:23 HBase-Region-Server-hb22.html > Firefox "about:memory": > 1,330.18 MB (48.22%) -- top(http://hb22:60030/rs-status#bc_l2, id=2010) > 1,329.61 MB (48.20%) -- active/window(http://hb22:60030/rs-status#bc_l2) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14582) Regionserver status webpage bucketcache list can become huge
[ https://issues.apache.org/jira/browse/HBASE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14582: -- Attachment: Screen Shot 2015-10-09 at 8.00.37 PM.png > Regionserver status webpage bucketcache list can become huge > > > Key: HBASE-14582 > URL: https://issues.apache.org/jira/browse/HBASE-14582 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: James Hartshorn >Priority: Trivial > Attachments: Screen Shot 2015-10-09 at 8.00.37 PM.png > > > The regionserver status page, such as http://127.0.0.1:60030/rs-status always > downloads information about every bucket in the cache. In some cases this > can be hundreds of thousands of buckets, causing megabytes of info to need to > be downloaded and significant browser instability and memory usage: > wc -l HBase-Region-Server-hb22.html > 2010116 HBase-Region-Server-hb22.html > ls -lah HBase-Region-Server-hb22.html > 32M Oct 6 19:23 HBase-Region-Server-hb22.html > Firefox "about:memory": > 1,330.18 MB (48.22%) -- top(http://hb22:60030/rs-status#bc_l2, id=2010) > 1,329.61 MB (48.20%) -- active/window(http://hb22:60030/rs-status#bc_l2) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14582) Regionserver status webpage bucketcache list can become huge
[ https://issues.apache.org/jira/browse/HBASE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14582: -- Attachment: Screen Shot 2015-10-09 at 8.00.37 PM.png Screen shot of what the block cache section looks like w/ the applied patch. We do not show buckets in the default screen. We do show them in the JSON form if you click on the button > Regionserver status webpage bucketcache list can become huge > > > Key: HBASE-14582 > URL: https://issues.apache.org/jira/browse/HBASE-14582 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: James Hartshorn >Priority: Trivial > Attachments: Screen Shot 2015-10-09 at 8.00.37 PM.png, Screen Shot > 2015-10-09 at 8.00.37 PM.png > > > The regionserver status page, such as http://127.0.0.1:60030/rs-status always > downloads information about every bucket in the cache. In some cases this > can be hundreds of thousands of buckets, causing megabytes of info to need to > be downloaded and significant browser instability and memory usage: > wc -l HBase-Region-Server-hb22.html > 2010116 HBase-Region-Server-hb22.html > ls -lah HBase-Region-Server-hb22.html > 32M Oct 6 19:23 HBase-Region-Server-hb22.html > Firefox "about:memory": > 1,330.18 MB (48.22%) -- top(http://hb22:60030/rs-status#bc_l2, id=2010) > 1,329.61 MB (48.20%) -- active/window(http://hb22:60030/rs-status#bc_l2) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14474) DeadLock in RpcClientImpl.Connection.close()
[
https://issues.apache.org/jira/browse/HBASE-14474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951370#comment-14951370
]
Nick Dimiduk commented on HBASE-14474:
--
Any conclusion here? This is last blocker for 1.1.3.
> DeadLock in RpcClientImpl.Connection.close()
> -
>
> Key: HBASE-14474
> URL: https://issues.apache.org/jira/browse/HBASE-14474
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.0.3, 1.1.3
>
> Attachments: 14474-addendum-2.txt, 14474-addendum.txt,
> hbase-14474_v1.patch, hbase-14474_v2.patch, hbase-14474_v3.patch
>
>
> From a code base that contains 1.1.2 + HBASE-14449 + HBASE-14241 and
> HBASE-14313 we can reproduce a dead lock with serverKilling CM easily:
> {code}
> Found one Java-level deadlock:
> =
> "htable-pool1-t63":
> waiting to lock monitor 0x01cb1688 (object 0x806ef150, a
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
> which is held by "IPC Client (1403704789) connection to
> enis-hbase-sep-21-6.novalocal/172.22.107.106:16020 from root"
> "IPC Client (1403704789) connection to
> enis-hbase-sep-21-6.novalocal/172.22.107.106:16020 from root":
> waiting to lock monitor 0x01cb1738 (object 0x806f0c60, a
> java.lang.Object),
> which is held by "htable-pool1-t63"
> Java stack information for the threads listed above:
> ===
> "htable-pool1-t63":
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.close(RpcClientImpl.java:819)
> - waiting to lock <0x806ef150> (a
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> - locked <0x806f0c60> (a java.lang.Object)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:856)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1192)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.multi(ClientProtos.java:32699)
> at
> org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:129)
> at
> org.apache.hadoop.hbase.client.MultiServerCallable.call(MultiServerCallable.java:54)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncProcess.java:708)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "IPC Client (1403704789) connection to
> enis-hbase-sep-21-6.novalocal/172.22.107.106:16020 from root":
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.close(RpcClientImpl.java:832)
> - waiting to lock <0x806f0c60> (a java.lang.Object)
> - locked <0x806ef150> (a
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:574)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14586) Remove extraneous ${argLine} references in pom
[
https://issues.apache.org/jira/browse/HBASE-14586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HBASE-14586:
Status: Patch Available (was: Open)
> Remove extraneous ${argLine} references in pom
> --
>
> Key: HBASE-14586
> URL: https://issues.apache.org/jira/browse/HBASE-14586
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Attachments: hbase-14586.001.patch
>
>
> The pom.xml has a line like this for the Surefire argLine, which has an extra
> ${argLine} reference. Recommend changes like this:
> {noformat}
> -${hbase-surefire.argLine} ${argLine}
> +${hbase-surefire.argLine}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14586) Remove extraneous ${argLine} references in pom
[
https://issues.apache.org/jira/browse/HBASE-14586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HBASE-14586:
Attachment: hbase-14586.001.patch
Patch attached, I also removed what looks like a stale comment.
> Remove extraneous ${argLine} references in pom
> --
>
> Key: HBASE-14586
> URL: https://issues.apache.org/jira/browse/HBASE-14586
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Attachments: hbase-14586.001.patch
>
>
> The pom.xml has a line like this for the Surefire argLine, which has an extra
> ${argLine} reference. Recommend changes like this:
> {noformat}
> -${hbase-surefire.argLine} ${argLine}
> +${hbase-surefire.argLine}
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14588) Stop accessing test resources from within src folder
[ https://issues.apache.org/jira/browse/HBASE-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HBASE-14588: Attachment: hbase-14588.001.patch Patch attached. I also have one for branch-1.1 if that's interesting, there are some additional test resources there that are not present in master. > Stop accessing test resources from within src folder > > > Key: HBASE-14588 > URL: https://issues.apache.org/jira/browse/HBASE-14588 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: hbase-14588.001.patch > > > A few tests in hbase-server reach into the src/test/data folder to get test > resources, which is naughty since tests are supposed to only operate within > the target/ folder. It's better to put these into src/test/resources and let > them be automatically copied into target/ via the resources plugin, like > other test resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951426#comment-14951426
]
Hudson commented on HBASE-14584:
FAILURE: Integrated in HBase-TRUNK #6891 (See
[https://builds.apache.org/job/HBase-TRUNK/6891/])
HBASE-14584 TestNamespacesInstanceModel fails on jdk8; DISABLE TEST (stack: rev
7c11a06b25fd7c748a1ba9dee2e323e2547d9dcd)
*
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java
*
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestNamespacesInstanceModel.java
* hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestModelBase.java
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951425#comment-14951425 ] Hudson commented on HBASE-14583: FAILURE: Integrated in HBase-TRUNK #6891 (See [https://builds.apache.org/job/HBase-TRUNK/6891/]) HBASE-14583 Enabled client-side metrics by default (ndimiduk: rev c7efd14726e932235dff52bd73f76cb861b5b1b3) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14536) Balancer & SSH interfering with each other leading to unavailability
[
https://issues.apache.org/jira/browse/HBASE-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951459#comment-14951459
]
Enis Soztutar commented on HBASE-14536:
---
+1. This makes sure that MetaSSH does not skip the assign since we also check
the znode for the meta location.
> Balancer & SSH interfering with each other leading to unavailability
>
>
> Key: HBASE-14536
> URL: https://issues.apache.org/jira/browse/HBASE-14536
> Project: HBase
> Issue Type: Bug
> Components: master, Region Assignment
>Affects Versions: 1.1.2
>Reporter: Devaraj Das
>Assignee: Stephen Yuan Jiang
> Fix For: 1.1.4
>
> Attachments: HBASE-14536.v1-branch-1.patch, master-log.tgz
>
>
> Came across this in our cluster:
> 1. The meta was assigned to a server 10.0.0.149,16020,1443507203340
> {noformat}
> 2015-09-29 06:16:22,472 DEBUG [AM.ZK.Worker-pool2-t56]
> master.RegionStates: Onlined 1588230740 on
> 10.0.0.149,16020,1443507203340 {ENCODED => 1588230740, NAME =>
> 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
> {noformat}
> 2. The server dies at some point:
> {noformat}
> 2015-09-29 06:18:25,952 INFO [main-EventThread]
> zookeeper.RegionServerTracker: RegionServer ephemeral node deleted,
> processing expiration [10.0.0.149,16020,1443507203340]
> 2015-09-29 06:18:25,955 DEBUG [main-EventThread] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=10.0.0.149,16020,1443507203340
> server being checked:
> 10.0.0.149,16020,1443507203340
> {noformat}
> 3. The balancer had computed a plan that contained a move for the meta:
> {noformat}
> 2015-09-29 06:18:26,833 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.HMaster:
> balance hri=hbase:meta,,1.1588230740,
> src=10.0.0.149,16020,1443507203340, dest=10.0.0.205,16020,1443507257905
> {noformat}
> 4. The following ensues after this, leading to the meta remaining unassigned:
> {noformat}
> 2015-09-29 06:18:26,859 DEBUG
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Offline hbase:meta,,1.1588230740, no need to
> unassign since it's on a dead server: 10.0.0.149,16020,1443507203340
> ..
> 2015-09-29 06:18:26,899 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000] master.RegionStates:
> Offlined 1588230740 from 10.0.0.149,16020,1443507203340
> .
> 2015-09-29 06:18:26,914 INFO
> [B.defaultRpcServer.handler=12,queue=0,port=16000]
> master.AssignmentManager: Skip assigning hbase:meta,,1.1588230740, it is
> on a dead but not processed yet server: 10.0.0.149,16020,1443507203340
>
> 2015-09-29 06:18:26,915 DEBUG [AM.ZK.Worker-pool2-t58]
> master.AssignmentManager: Znode hbase:meta,,1.1588230740 deleted,
> state: {1588230740 state=OFFLINE, ts=1443507506914,
> server=10.0.0.149,16020,1443507203340}
>
> 2015-09-29 06:18:29,447 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2] master.AssignmentManager:
> based on AM, current
> region=hbase:meta,,1.1588230740 is on server=null server being checked:
> 10.0.0.149,16020,1443507203340
> 2015-09-29 06:18:29,451 INFO [MASTER_META_SERVER_OPERATIONS-
> 10.0.0.148:16000-2] handler.MetaServerShutdownHandler: META has been
> assigned to otherwhere, skip assigning.
> 2015-09-29 06:18:29,452 DEBUG
> [MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2]
> master.DeadServer: Finished processing 10.0.0.149,16020,1443507203340
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951519#comment-14951519
]
Hudson commented on HBASE-14584:
SUCCESS: Integrated in HBase-1.2-IT #198 (See
[https://builds.apache.org/job/HBase-1.2-IT/198/])
HBASE-14584 TestNamespacesInstanceModel fails on jdk8; DISABLE TEST (stack: rev
c46a6fff810313007d6632733fa9035e2c158202)
* hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestModelBase.java
*
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java
*
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestNamespacesInstanceModel.java
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14517) Show regionserver's version in master status page
[ https://issues.apache.org/jira/browse/HBASE-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951518#comment-14951518 ] Hudson commented on HBASE-14517: SUCCESS: Integrated in HBase-1.2-IT #198 (See [https://builds.apache.org/job/HBase-1.2-IT/198/]) HBASE-14517 Show regionserver's version in master status page (Liu (stack: rev ae56022e128ce840aec43a071e037978a759e8b9) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcCallContext.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * hbase-protocol/src/main/protobuf/RPC.proto * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-protocol/src/main/protobuf/HBase.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ProcedurePrepareLatch.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > Show regionserver's version in master status page > - > > Key: HBASE-14517 > URL: https://issues.apache.org/jira/browse/HBASE-14517 > Project: HBase > Issue Type: Improvement > Components: monitoring >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14517-v1.diff, HBASE-14517-v1.patch, > HBASE-14517-v2.patch > > > In production env, regionservers may be removed from the cluster for hardware > problems and rejoined the cluster after the repair. There is a potential risk > that the version of rejoined regionserver may diff from others because the > cluster has been upgraded through many versions. > To solve this, we can show the all regionservers' version in the server list > of master's status page, and highlight the regionserver when its version is > different from the master's version, similar to HDFS-3245 > Suggestions are welcome~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14588) Stop accessing test resources from within src folder
[ https://issues.apache.org/jira/browse/HBASE-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14588: -- Status: Patch Available (was: Open) Looks good. Trying patch against master (You want it run against branch-1 [~andrew.wang]? Add .branch-1. into name of the patch...). Thanks A > Stop accessing test resources from within src folder > > > Key: HBASE-14588 > URL: https://issues.apache.org/jira/browse/HBASE-14588 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: hbase-14588.001.patch > > > A few tests in hbase-server reach into the src/test/data folder to get test > resources, which is naughty since tests are supposed to only operate within > the target/ folder. It's better to put these into src/test/resources and let > them be automatically copied into target/ via the resources plugin, like > other test resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951517#comment-14951517 ] stack commented on HBASE-14567: --- [~jingcheng...@intel.com] Thank you. > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: 14567v2.txt, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13858) RS/MasterDumpServlet dumps threads before its “Stacks” header
[ https://issues.apache.org/jira/browse/HBASE-13858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951520#comment-14951520 ] Hudson commented on HBASE-13858: SUCCESS: Integrated in HBase-1.2-IT #198 (See [https://builds.apache.org/job/HBase-1.2-IT/198/]) HBASE-13858 RS/MasterDumpServlet dumps threads before its ���Stacks��� (stack: rev 11d0121a42dd4d47c82bb31756bf224df107da2b) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterDumpServlet.java > RS/MasterDumpServlet dumps threads before its “Stacks” header > - > > Key: HBASE-13858 > URL: https://issues.apache.org/jira/browse/HBASE-13858 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Fred Liu >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-13858.patch > > > The stacktraces are captured using a Hadoop helper method, then its output is > merged with the current. I presume there is a simple flush after outputing > the "Stack" header missing, before then the caught output is dumped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14582) Regionserver status webpage bucketcache list can become huge
[ https://issues.apache.org/jira/browse/HBASE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14582: -- Attachment: 14582.txt > Regionserver status webpage bucketcache list can become huge > > > Key: HBASE-14582 > URL: https://issues.apache.org/jira/browse/HBASE-14582 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: James Hartshorn >Priority: Trivial > Attachments: 14582.txt, Screen Shot 2015-10-09 at 8.00.37 PM.png, > Screen Shot 2015-10-09 at 8.00.37 PM.png > > > The regionserver status page, such as http://127.0.0.1:60030/rs-status always > downloads information about every bucket in the cache. In some cases this > can be hundreds of thousands of buckets, causing megabytes of info to need to > be downloaded and significant browser instability and memory usage: > wc -l HBase-Region-Server-hb22.html > 2010116 HBase-Region-Server-hb22.html > ls -lah HBase-Region-Server-hb22.html > 32M Oct 6 19:23 HBase-Region-Server-hb22.html > Firefox "about:memory": > 1,330.18 MB (48.22%) -- top(http://hb22:60030/rs-status#bc_l2, id=2010) > 1,329.61 MB (48.20%) -- active/window(http://hb22:60030/rs-status#bc_l2) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14442) MultiTableInputFormatBase.getSplits dosenot build split for a scan whose startRow=stopRow=(startRow of a region)
[ https://issues.apache.org/jira/browse/HBASE-14442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951543#comment-14951543 ] Nathan commented on HBASE-14442: Before, I want to explain that we can use a ^Scan^ to read only one row as a special case, docs are all right. Both contructors ^Scan(Get)^ and ^Scan(byte[], byte[])^ can be used to read read only row as a ^GetScan^, it‘s working. I mean that *MultiTableInputFormatBase.getSplits* does not contain this case yet. The problem is: *if the only row that we read by ^Scan^ equals a region's edge row of ^HTable^, we use TableMapReduceUtil.initTableMapperJob(List scans..) to run a job, no ^Split^ (so no ^Map^) is built for this ^Scan^.* > MultiTableInputFormatBase.getSplits dosenot build split for a scan whose > startRow=stopRow=(startRow of a region) > > > Key: HBASE-14442 > URL: https://issues.apache.org/jira/browse/HBASE-14442 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 1.1.2 >Reporter: Nathan >Assignee: Nathan > Original Estimate: 0.5h > Remaining Estimate: 0.5h > > I created a Scan whose startRow and stopRow are the same with a region's > startRow, then I found no map was built. > The following is the source code of this condtion: > (startRow.length == 0 || keys.getSecond()[i].length == 0 || > Bytes.compareTo(startRow, keys.getSecond()[i]) < 0) && > (stopRow.length == 0 || Bytes.compareTo(stopRow, > keys.getFirst()[i]) > 0) > I think a "=" should be added. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14517) Show regionserver's version in master status page
[ https://issues.apache.org/jira/browse/HBASE-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951546#comment-14951546 ] Hudson commented on HBASE-14517: FAILURE: Integrated in HBase-TRUNK #6892 (See [https://builds.apache.org/job/HBase-TRUNK/6892/]) HBASE-14517 Show regionserver's version in master status page (Liu (stack: rev 445dbd8a0e45e45bd4a44dee999a6c134c374dfc) * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcCallContext.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ProcedurePrepareLatch.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-protocol/src/main/protobuf/HBase.proto * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * hbase-protocol/src/main/protobuf/RPC.proto > Show regionserver's version in master status page > - > > Key: HBASE-14517 > URL: https://issues.apache.org/jira/browse/HBASE-14517 > Project: HBase > Issue Type: Improvement > Components: monitoring >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14517-v1.diff, HBASE-14517-v1.patch, > HBASE-14517-v2.patch > > > In production env, regionservers may be removed from the cluster for hardware > problems and rejoined the cluster after the repair. There is a potential risk > that the version of rejoined regionserver may diff from others because the > cluster has been upgraded through many versions. > To solve this, we can show the all regionservers' version in the server list > of master's status page, and highlight the regionserver when its version is > different from the master's version, similar to HDFS-3245 > Suggestions are welcome~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951545#comment-14951545 ] Hudson commented on HBASE-14567: FAILURE: Integrated in HBase-TRUNK #6892 (See [https://builds.apache.org/job/HBase-TRUNK/6892/]) HBASE-14567 Disable hanging test TestMobCompactor and TestMobSweeper (stack: rev 2e8e32f74614bae9c7bb669cd536518f2d81a4e3) * hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweeper.java > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: 14567v2.txt, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13858) RS/MasterDumpServlet dumps threads before its “Stacks” header
[ https://issues.apache.org/jira/browse/HBASE-13858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951547#comment-14951547 ] Hudson commented on HBASE-13858: FAILURE: Integrated in HBase-TRUNK #6892 (See [https://builds.apache.org/job/HBase-TRUNK/6892/]) HBASE-13858 RS/MasterDumpServlet dumps threads before its âStacksâ (stack: rev 2e593a9d3801a42751244ab4478650a581437875) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterDumpServlet.java > RS/MasterDumpServlet dumps threads before its “Stacks” header > - > > Key: HBASE-13858 > URL: https://issues.apache.org/jira/browse/HBASE-13858 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Fred Liu >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-13858.patch > > > The stacktraces are captured using a Hadoop helper method, then its output is > merged with the current. I presume there is a simple flush after outputing > the "Stack" header missing, before then the caught output is dumped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14535) Unit test for rpc connection concurrency / deadlock testing
[ https://issues.apache.org/jira/browse/HBASE-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951382#comment-14951382 ] Nick Dimiduk commented on HBASE-14535: -- [~enis] are you saying here that failures of this test would always be legitimate product issues (even if not always reproducible due to non-determinism) and not the false-negatives that we're calling "flakey"? > Unit test for rpc connection concurrency / deadlock testing > > > Key: HBASE-14535 > URL: https://issues.apache.org/jira/browse/HBASE-14535 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16 > > Attachments: hbase-14535_v1.patch, hbase-14535_v2.patch > > > As per parent jira and recent jiras HBASE-14449 + HBASE-14241 and > HBASE-14313, we seem to be lacking some testing rpc connection concurrency > issues in a UT env. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951379#comment-14951379
]
Hadoop QA commented on HBASE-14578:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765905/14578-master.txt
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765905
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail with Hadoop version 2.4.0.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java:[238,56]
method createSnapshotV2 in class
org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils.SnapshotMock cannot be
applied to given types;
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java:[242,27]
method createSnapshotV2 in class
org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils.SnapshotMock cannot be
applied to given types;
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:testCompile
(default-testCompile) on project hbase-server: Compilation failure: Compilation
failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java:[238,56]
method createSnapshotV2 in class
org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils.SnapshotMock cannot be
applied to given types;
[ERROR] required: java.lang.String,java.lang.String
[ERROR] found: java.lang.String
[ERROR] reason: actual and formal argument lists differ in length
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java:[242,27]
method createSnapshotV2 in class
org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils.SnapshotMock cannot be
applied to given types;
[ERROR] required: java.lang.String,java.lang.String
[ERROR] found: java.lang.String
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hbase-server
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15934//console
This message is automatically generated.
> URISyntaxException during snapshot restore
> --
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 0.98.16
>
> Attachments: 14578-master.txt, HBASE-14578-0.98-V2.patch,
> HBASE-14578-0.98.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
[jira] [Created] (HBASE-14586) Remove extraneous ${argLine} references in pom
Andrew Wang created HBASE-14586:
---
Summary: Remove extraneous ${argLine} references in pom
Key: HBASE-14586
URL: https://issues.apache.org/jira/browse/HBASE-14586
Project: HBase
Issue Type: Improvement
Affects Versions: 1.1.0
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
The pom.xml has a line like this for the Surefire argLine, which has an extra
${argLine} reference. Recommend changes like this:
{noformat}
-${hbase-surefire.argLine} ${argLine}
+${hbase-surefire.argLine}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14587) Attach a test-sources.jar for hbase-server
[ https://issues.apache.org/jira/browse/HBASE-14587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HBASE-14587: Attachment: hbase-14587.001.patch pom-only change attached. > Attach a test-sources.jar for hbase-server > -- > > Key: HBASE-14587 > URL: https://issues.apache.org/jira/browse/HBASE-14587 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: hbase-14587.001.patch > > > It'd be nice to attach a test-sources jar alongside the others as part of the > build, to provide test resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14580) Make the HBaseMiniCluster compliant with Kerberos
[
https://issues.apache.org/jira/browse/HBASE-14580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951399#comment-14951399
]
Gary Helmling commented on HBASE-14580:
---
{code}
"kerberos".equalsIgnoreCase(c.get("hbase.security.authentication",
null))) {
{code}
Instead of hard-coding the config values, can you use
{{User.isHBaseSecurityEnabled(c)}}?
The username suffixes were fed into the data dirs used by each DN/RS's for a
"distributed" minicluster setup (or at least they did way back when I last
looked at HBaseTestingUtility). So, as I understand it, that would not be an
issue here are Kerberos would only be supported with a single node setup?
> Make the HBaseMiniCluster compliant with Kerberos
> -
>
> Key: HBASE-14580
> URL: https://issues.apache.org/jira/browse/HBASE-14580
> Project: HBase
> Issue Type: Improvement
> Components: security, test
>Affects Versions: 2.0.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
> Fix For: 2.0.0
>
> Attachments: patch-14580.v1.patch
>
>
> Whne using MiniKDC and the minicluster in a unit test, there is a conflict
> causeed by HBaseTestingUtility:
> {code}
> public static User getDifferentUser(final Configuration c,
> final String differentiatingSuffix)
> throws IOException {
>// snip
> String username = User.getCurrent().getName() +
> differentiatingSuffix; < problem here
> User user = User.createUserForTesting(c, username,
> new String[]{"supergroup"});
> return user;
> }
> {code}
> This creates users like securedUser/localh...@example.com.hfs.0, and this
> does not work.
> My fix is to return the current user when Kerberos is set. I don't think that
> there is another option (any other opinion?). However this user is not in a
> group so we have logs like 'WARN [IPC Server handler 9 on 61366]
> security.UserGroupInformation (UserGroupInformation.java:getGroupNames(1521))
> - No groups available for user securedUser' I'm not sure of its impact.
> [~apurtell], what do you think?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14587) Attach a test-sources.jar for hbase-server
[ https://issues.apache.org/jira/browse/HBASE-14587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HBASE-14587: Status: Patch Available (was: Open) > Attach a test-sources.jar for hbase-server > -- > > Key: HBASE-14587 > URL: https://issues.apache.org/jira/browse/HBASE-14587 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: hbase-14587.001.patch > > > It'd be nice to attach a test-sources jar alongside the others as part of the > build, to provide test resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951477#comment-14951477 ] Jingcheng Du commented on HBASE-14567: -- I tried to see the cost in each case. In most of cases we have data preparation, running target method and assert phases, most of time are used by data preparation ( In running target method phase of all cases, it takes less than 2 seconds) . for instance, we have the case to compact a restored snapshot where the data preparation takes very long time. I will see the cases again, and try to reduce the time cost in data preparation. > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: 14567v2.txt, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951478#comment-14951478
]
Hadoop QA commented on HBASE-14584:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765935/14584.00.patch
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765935
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15937//console
This message is automatically generated.
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14268) Improve KeyLocker
[
https://issues.apache.org/jira/browse/HBASE-14268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14268:
--
Attachment: HBASE-14268-V7.patch
> Improve KeyLocker
> -
>
> Key: HBASE-14268
> URL: https://issues.apache.org/jira/browse/HBASE-14268
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 14268-V5.patch, HBASE-14268-V2.patch,
> HBASE-14268-V3.patch, HBASE-14268-V4.patch, HBASE-14268-V5.patch,
> HBASE-14268-V5.patch, HBASE-14268-V6.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268.patch, KeyLockerIncrKeysPerformance.java,
> KeyLockerPerformance.java, ReferenceTestApp.java
>
>
> 1. In the implementation of {{KeyLocker}} it uses atomic variables inside a
> synchronized block, which doesn't make sense. Moreover, logic inside the
> synchronized block is not trivial so that it makes less performance in heavy
> multi-threaded environment.
> 2. {{KeyLocker}} gives an instance of {{RentrantLock}} which is already
> locked, but it doesn't follow the contract of {{ReentrantLock}} because you
> are not allowed to freely invoke lock/unlock methods under that contract.
> That introduces a potential risk; Whenever you see a variable of the type
> {{RentrantLock}}, you should pay attention to what the included instance is
> coming from.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951515#comment-14951515
]
stack commented on HBASE-14584:
---
[~ndimiduk] What is that doing? Just to and from but no assertion it worked?
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13858) RS/MasterDumpServlet dumps threads before its “Stacks” header
[ https://issues.apache.org/jira/browse/HBASE-13858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951533#comment-14951533 ] Hudson commented on HBASE-13858: SUCCESS: Integrated in HBase-1.3-IT #223 (See [https://builds.apache.org/job/HBase-1.3-IT/223/]) HBASE-13858 RS/MasterDumpServlet dumps threads before its âStacksâ (stack: rev 14819ce853da5b11eec5ada93ff9fc99721f8831) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterDumpServlet.java > RS/MasterDumpServlet dumps threads before its “Stacks” header > - > > Key: HBASE-13858 > URL: https://issues.apache.org/jira/browse/HBASE-13858 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Fred Liu >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-13858.patch > > > The stacktraces are captured using a Hadoop helper method, then its output is > merged with the current. I presume there is a simple flush after outputing > the "Stack" header missing, before then the caught output is dumped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14517) Show regionserver's version in master status page
[ https://issues.apache.org/jira/browse/HBASE-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951531#comment-14951531 ] Hudson commented on HBASE-14517: SUCCESS: Integrated in HBase-1.3-IT #223 (See [https://builds.apache.org/job/HBase-1.3-IT/223/]) HBASE-14517 Show regionserver's version in master status page (Liu (stack: rev cd298eae4f992e137ea4891752003fd33ade080c) * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * hbase-protocol/src/main/protobuf/RPC.proto * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcCallContext.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * hbase-protocol/src/main/protobuf/HBase.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ProcedurePrepareLatch.java > Show regionserver's version in master status page > - > > Key: HBASE-14517 > URL: https://issues.apache.org/jira/browse/HBASE-14517 > Project: HBase > Issue Type: Improvement > Components: monitoring >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14517-v1.diff, HBASE-14517-v1.patch, > HBASE-14517-v2.patch > > > In production env, regionservers may be removed from the cluster for hardware > problems and rejoined the cluster after the repair. There is a potential risk > that the version of rejoined regionserver may diff from others because the > cluster has been upgraded through many versions. > To solve this, we can show the all regionservers' version in the server list > of master's status page, and highlight the regionserver when its version is > different from the master's version, similar to HDFS-3245 > Suggestions are welcome~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951532#comment-14951532
]
Hudson commented on HBASE-14584:
SUCCESS: Integrated in HBase-1.3-IT #223 (See
[https://builds.apache.org/job/HBase-1.3-IT/223/])
HBASE-14584 TestNamespacesInstanceModel fails on jdk8; DISABLE TEST (stack: rev
3accb7366bb7a5f0fd8229a3234f2171b69c25f4)
*
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java
*
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestNamespacesInstanceModel.java
* hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestModelBase.java
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14588) Stop accessing test resources from within src folder
[
https://issues.apache.org/jira/browse/HBASE-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951538#comment-14951538
]
Hadoop QA commented on HBASE-14588:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765936/hbase-14588.001.patch
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765936
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 19 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15942//console
This message is automatically generated.
> Stop accessing test resources from within src folder
>
>
> Key: HBASE-14588
> URL: https://issues.apache.org/jira/browse/HBASE-14588
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
> Attachments: hbase-14588.001.patch
>
>
> A few tests in hbase-server reach into the src/test/data folder to get test
> resources, which is naughty since tests are supposed to only operate within
> the target/ folder. It's better to put these into src/test/resources and let
> them be automatically copied into target/ via the resources plugin, like
> other test resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14567: -- Attachment: 14567v2.txt I pushed this patch given it hangs pretty frequently and the fixup looks good. I actually just removed TestMobSweeper since it is going away. I like Anoops' comment asking if we can make it run quicker. I measured and it is still over 5 minutes to complete. This is toward the far end of longest tests in our suite. Thanks [~jingcheng...@intel.com] > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: 14567v2.txt, HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951284#comment-14951284
]
Nick Dimiduk commented on HBASE-14584:
--
It's juts the "AS_XML" vs "toXML" ?
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951283#comment-14951283
]
Nick Dimiduk commented on HBASE-14584:
--
What do you mean "interpolation"? It would be bad to ship an incompatible wire
format.
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14420) Zombie Stomping Session
[
https://issues.apache.org/jira/browse/HBASE-14420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951291#comment-14951291
]
Hadoop QA commented on HBASE-14420:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765870/none_fix.txt
against master branch at commit c7efd14726e932235dff52bd73f76cb861b5b1b3.
ATTACHMENT ID: 12765870
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation, build,
or dev-support patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence
{color:red}-1 core zombie tests{color}. There are 10 zombie test(s):
at
org.apache.hadoop.hbase.io.encoding.TestDataBlockEncoders.testSeekingOnSample(TestDataBlockEncoders.java:217)
at
org.apache.hadoop.hbase.mob.compactions.TestMobCompactor.testCompactionFromAdmin(TestMobCompactor.java:544)
at
org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper.testCompactionDelaySweeperInternal(TestMobSweeper.java:264)
at
org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper.testCompactionDelaySweeperWithNamespace(TestMobSweeper.java:362)
at
org.apache.hadoop.hbase.io.encoding.TestChangingEncoding.testChangingEncoding(TestChangingEncoding.java:202)
at
org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable.testForCheckingIfEnableAndDisableWorksFineAfterSwitch(TestMasterRestartAfterDisablingTable.java:64)
at
org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd.testEndToEnd(TestFuzzyRowFilterEndToEnd.java:143)
at
org.apache.hadoop.hbase.master.TestOpenedRegionHandler.testOpenedRegionHandlerOnMasterRestart(TestOpenedRegionHandler.java:93)
at
org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster.testOpenCloseRacing(TestAssignmentManagerOnCluster.java:849)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15930//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15930//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15930//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15930//console
This message is automatically generated.
> Zombie Stomping Session
> ---
>
> Key: HBASE-14420
> URL: https://issues.apache.org/jira/browse/HBASE-14420
> Project: HBase
> Issue Type: Umbrella
> Components: test
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Attachments: hangers.txt, none_fix (1).txt, none_fix.txt,
> none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt
>
>
> Patch build are now failing most of the time because we are dropping zombies.
> I confirm we are doing this on non-apache build boxes too.
> Left-over zombies consume resources on build boxes (OOME cannot create native
> threads). Having to do multiple test runs in the hope that we can get a
> non-zombie-making build or making (arbitrary) rulings that the zombies are
> 'not related' is a productivity sink. And so on...
> This is an umbrella issue for a zombie stomping session that started earlier
> this week. Will hang sub-issues of this one. Am running builds back-to-back
> on little cluster to turn out the monsters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951311#comment-14951311
]
stack commented on HBASE-14584:
---
I fixed above comment. jaxb emits the xml in different orderings dependent on
whether jdk7 or jdk8 (ordering is not guaranteed)
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14582) Regionserver status webpage bucketcache list can become huge
[ https://issues.apache.org/jira/browse/HBASE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951355#comment-14951355 ] James Hartshorn commented on HBASE-14582: - That would be great. This is actually causing some pain to our devs and ops people as we try to do hbase operations, so we'd likely be able to try a patch very soon. > Regionserver status webpage bucketcache list can become huge > > > Key: HBASE-14582 > URL: https://issues.apache.org/jira/browse/HBASE-14582 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: James Hartshorn >Priority: Trivial > > The regionserver status page, such as http://127.0.0.1:60030/rs-status always > downloads information about every bucket in the cache. In some cases this > can be hundreds of thousands of buckets, causing megabytes of info to need to > be downloaded and significant browser instability and memory usage: > wc -l HBase-Region-Server-hb22.html > 2010116 HBase-Region-Server-hb22.html > ls -lah HBase-Region-Server-hb22.html > 32M Oct 6 19:23 HBase-Region-Server-hb22.html > Firefox "about:memory": > 1,330.18 MB (48.22%) -- top(http://hb22:60030/rs-status#bc_l2, id=2010) > 1,329.61 MB (48.20%) -- active/window(http://hb22:60030/rs-status#bc_l2) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951549#comment-14951549
]
Hudson commented on HBASE-14584:
FAILURE: Integrated in HBase-1.3 #249 (See
[https://builds.apache.org/job/HBase-1.3/249/])
HBASE-14584 TestNamespacesInstanceModel fails on jdk8; DISABLE TEST (stack: rev
3accb7366bb7a5f0fd8229a3234f2171b69c25f4)
* hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestModelBase.java
*
hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestNamespacesInstanceModel.java
*
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14517) Show regionserver's version in master status page
[ https://issues.apache.org/jira/browse/HBASE-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951548#comment-14951548 ] Hudson commented on HBASE-14517: FAILURE: Integrated in HBase-1.3 #249 (See [https://builds.apache.org/job/HBase-1.3/249/]) HBASE-14517 Show regionserver's version in master status page (Liu (stack: rev cd298eae4f992e137ea4891752003fd33ade080c) * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ProcedurePrepareLatch.java * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon * hbase-protocol/src/main/protobuf/RPC.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcCallContext.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * hbase-protocol/src/main/protobuf/HBase.proto > Show regionserver's version in master status page > - > > Key: HBASE-14517 > URL: https://issues.apache.org/jira/browse/HBASE-14517 > Project: HBase > Issue Type: Improvement > Components: monitoring >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14517-v1.diff, HBASE-14517-v1.patch, > HBASE-14517-v2.patch > > > In production env, regionservers may be removed from the cluster for hardware > problems and rejoined the cluster after the repair. There is a potential risk > that the version of rejoined regionserver may diff from others because the > cluster has been upgraded through many versions. > To solve this, we can show the all regionservers' version in the server list > of master's status page, and highlight the regionserver when its version is > different from the master's version, similar to HDFS-3245 > Suggestions are welcome~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13858) RS/MasterDumpServlet dumps threads before its “Stacks” header
[ https://issues.apache.org/jira/browse/HBASE-13858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951550#comment-14951550 ] Hudson commented on HBASE-13858: FAILURE: Integrated in HBase-1.3 #249 (See [https://builds.apache.org/job/HBase-1.3/249/]) HBASE-13858 RS/MasterDumpServlet dumps threads before its âStacksâ (stack: rev 14819ce853da5b11eec5ada93ff9fc99721f8831) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterDumpServlet.java > RS/MasterDumpServlet dumps threads before its “Stacks” header > - > > Key: HBASE-13858 > URL: https://issues.apache.org/jira/browse/HBASE-13858 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Fred Liu >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-13858.patch > > > The stacktraces are captured using a Hadoop helper method, then its output is > merged with the current. I presume there is a simple flush after outputing > the "Stack" header missing, before then the caught output is dumped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14578) URISyntaxException during snapshot restore
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-14578:
-
Attachment: HBASE-14578-0.98-V2.patch
Thanks for reviewing the patch [~mbertozzi]. Have addressed your comments in
the V2 patch.
> URISyntaxException during snapshot restore
> --
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 0.98.16
>
> Attachments: HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter regions contain the reference of the
> original HFile until compaction happens.
> In RestoreSnapshotHelper (while restoring snapshot file which have regions
> containing hfile references), path is initialized
[jira] [Commented] (HBASE-14517) Show regionserver's version in master status page
[
https://issues.apache.org/jira/browse/HBASE-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949953#comment-14949953
]
Hadoop QA commented on HBASE-14517:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765725/HBASE-14517-v2.patch
against master branch at commit a77f830198815db3927fc02646d2167c370a028f.
ATTACHMENT ID: 12765725
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:red}-1 findbugs{color}. The patch appears to cause Findbugs
(version 2.0.3) to fail.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15924//testReport/
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15924//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15924//console
This message is automatically generated.
> Show regionserver's version in master status page
> -
>
> Key: HBASE-14517
> URL: https://issues.apache.org/jira/browse/HBASE-14517
> Project: HBase
> Issue Type: Improvement
> Components: monitoring
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-14517-v1.diff, HBASE-14517-v1.patch,
> HBASE-14517-v2.patch
>
>
> In production env, regionservers may be removed from the cluster for hardware
> problems and rejoined the cluster after the repair. There is a potential risk
> that the version of rejoined regionserver may diff from others because the
> cluster has been upgraded through many versions.
> To solve this, we can show the all regionservers' version in the server list
> of master's status page, and highlight the regionserver when its version is
> different from the master's version, similar to HDFS-3245
> Suggestions are welcome~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14557) MapReduce WALPlayer issue with NoTagsKeyValue
[
https://issues.apache.org/jira/browse/HBASE-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950030#comment-14950030
]
Hadoop QA commented on HBASE-14557:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765531/HBASE-14557.patch
against master branch at commit a77f830198815db3927fc02646d2167c370a028f.
ATTACHMENT ID: 12765531
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.token.TestGenerateDelegationToken
org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.wal.TestWALSplit.testLogsGetArchivedAfterSplit(TestWALSplit.java:650)
at
org.apache.hadoop.hbase.wal.TestWALSplit.testFirstLineCorruptionLogFileSkipErrorsPasses(TestWALSplit.java:498)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testRegionShouldNotBeDeployed(TestHBaseFsck.java:1628)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15925//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15925//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15925//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15925//console
This message is automatically generated.
> MapReduce WALPlayer issue with NoTagsKeyValue
> -
>
> Key: HBASE-14557
> URL: https://issues.apache.org/jira/browse/HBASE-14557
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Jerry He
>Assignee: Anoop Sam John
> Attachments: HBASE-14557.patch
>
>
> Running MapReduce WALPlayer to convert WAL into HFiles:
> {noformat}
> 15/10/05 20:28:08 INFO mapred.JobClient: Task Id :
> attempt_201508031611_0029_m_00_0, Status : FAILED
> java.io.IOException: Type mismatch in value from map: expected
> org.apache.hadoop.hbase.KeyValue, recieved
> org.apache.hadoop.hbase.NoTagsKeyValue
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:997)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:96)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at
> java.security.AccessController.doPrivileged(AccessController.java:369)
> at javax.security.auth.Subject.doAs(Subject.java:572)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> {noformat}
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951165#comment-14951165
]
stack commented on HBASE-14584:
---
I tried this but seemed to have no effect:
{code}
diff --git
a/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java
b/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java
index b31ecf9..8b61b2c 100644
---
a/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java
+++
b/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/NamespacesInstanceModel.java
@@ -25,8 +25,10 @@ import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import javax.xml.bind.annotation.XmlAccessOrder;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
+import javax.xml.bind.annotation.XmlAccessorOrder;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlTransient;
@@ -49,6 +51,7 @@ import org.apache.hadoop.hbase.rest.protobuf
*/
@XmlRootElement(name="NamespaceProperties")
@XmlAccessorType(XmlAccessType.FIELD)
+@XmlAccessorOrder(XmlAccessOrder.ALPHABETICAL)
@InterfaceAudience.Private
public class NamespacesInstanceModel implements Serializable,
ProtobufMessageHandler {
diff --git
a/hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestNamespacesInstanceModel.java
b/hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestNamespacesInstanceModel.java
index 52ed334..a0729b5 100644
---
a/hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestNamespacesInstanceModel.java
+++
b/hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestNamespacesInstanceModel.java
@@ -93,5 +93,4 @@ public class TestNamespacesInstanceModel extends
TestModelBase TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14578) URISyntaxException during snapshot restore
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14578:
---
Attachment: 14578-master.txt
> URISyntaxException during snapshot restore
> --
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 0.98.16
>
> Attachments: 14578-master.txt, HBASE-14578-0.98-V2.patch,
> HBASE-14578-0.98.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter regions contain the reference of the
> original HFile until compaction happens.
> In RestoreSnapshotHelper (while restoring snapshot file which have regions
> containing hfile references), path is initialized using the snapshot table
> name which has namespace separated by colon,
> {code}
> private
[jira] [Updated] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14584:
--
Attachment: disable.txt
Patch to disable the xml compare.
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14533) Thrift client gets "AsyncProcess: Failed to get region location .... closed"
[
https://issues.apache.org/jira/browse/HBASE-14533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951299#comment-14951299
]
Hadoop QA commented on HBASE-14533:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765884/14533v2.branch-1.patch
against branch-1 branch at commit c7efd14726e932235dff52bd73f76cb861b5b1b3.
ATTACHMENT ID: 12765884
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3780 checkstyle errors (more than the master's current 3779 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.thrift.TestThriftServer.testAll(TestThriftServer.java:157)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15931//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15931//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15931//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15931//console
This message is automatically generated.
> Thrift client gets "AsyncProcess: Failed to get region location closed"
>
>
> Key: HBASE-14533
> URL: https://issues.apache.org/jira/browse/HBASE-14533
> Project: HBase
> Issue Type: Bug
> Components: REST, Thrift
>Affects Versions: 1.0.0
>Reporter: stack
>Assignee: stack
> Attachments: 14533.test.patch, 14533v2.branch-1.patch, test.patch
>
>
> An internal python client has been getting below stack trace since
> HBASE-134347
> {code}
> 2015-09-30 11:27:31,670 runnerERROR : scheduler
> executor error
> 2015-09-30 11:27:31,674 runnerERROR : Traceback (most
> recent call last):
> File
> "/opt/cops/cops-related-ticket-info-fetcher/fetcher/.virtenv/lib/python2.6/site-packages/CopsRtiFetcher-0.1-py2.6.egg/cops_rti/fetcher/runner.py",
> line 82, in run
> fetch_list = self.__scheduler_executor.run()
> File
> "/opt/cops/cops-related-ticket-info-fetcher/fetcher/.virtenv/lib/python2.6/site-packages/CopsRtiFetcher-0.1-py2.6.egg/cops_rti/fetcher/scheduler.py",
> line 35, in run
> with self.__fetch_db_dao.get_scanner() as scanner:
> File
> "/opt/cops/cops-related-ticket-info-fetcher/fetcher/.virtenv/lib/python2.6/site-packages/CopsHbaseCommon-f796bf2929be11c26536c3e8f3e9c0b0ecb382b3-py2.6.egg/cops/hbase/common/hbase_dao.py",
> line 57, in get_scanner
> caching=caching, field_filter_list=field_filter_list)
> File
> "/opt/cops/cops-related-ticket-info-fetcher/fetcher/.virtenv/lib/python2.6/site-packages/CopsHbaseCommon-f796bf2929be11c26536c3e8f3e9c0b0ecb382b3-py2.6.egg/cops/hbase/common/hbase_client_template.py",
> line 104, in get_entity_scanner
> self.__fix_cfs(self.__filter_columns(field_filter_list)), caching)
> File
> "/opt/cops/cops-related-ticket-info-fetcher/fetcher/.virtenv/lib/python2.6/site-packages/CopsHbaseCommon-f796bf2929be11c26536c3e8f3e9c0b0ecb382b3-py2.6.egg/cops/hbase/common/hbase_entity_scanner.py",
> line 81, in open
> self.__scanner_id = client.scannerOpenWithScan(table_name, scan)
> File
> "/opt/cops/cops-related-ticket-info-fetcher/.crepo/cops-hbase-common/ext-py/hbase/Hbase.py",
> line 1494, in scannerOpenWithScan
> return self.recv_scannerOpenWithScan()
> File
> "/opt/cops/cops-related-ticket-info-fetcher/.crepo/cops-hbase-common/ext-py/hbase/Hbase.py",
> line
[jira] [Commented] (HBASE-14579) Users authenticated with KERBEROS are recorded as being authenticated with SIMPLE
[ https://issues.apache.org/jira/browse/HBASE-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951338#comment-14951338 ] stack commented on HBASE-14579: --- bq. Yes. Is that an issue for the 2.0 branch? I'd say it would be fine to drop 2.4 hadoop for 2.0 hbase. > Users authenticated with KERBEROS are recorded as being authenticated with > SIMPLE > - > > Key: HBASE-14579 > URL: https://issues.apache.org/jira/browse/HBASE-14579 > Project: HBase > Issue Type: Bug > Components: security >Affects Versions: 1.0.0, 1.2.0, 0.98.15 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon >Priority: Minor > Fix For: 2.0.0 > > Attachments: hbase-14579.patch > > > That's the HBase version of HADOOP-10683. > We see: > ??hbase.Server (RpcServer.java:saslReadAndProcess(1446)) - Auth successful > for securedUser/localh...@example.com (auth:SIMPLE)?? > while we would like to see: > ??hbase.Server (RpcServer.java:saslReadAndProcess(1446)) - Auth successful > for securedUser/localh...@example.com (auth:KERBEROS)?? > The fix is simple, but it means we need hadoop 2.5+. > There is also a lot of cases where HBase calls "createUser" w/o specifying > the authentication method... I don"'t have the solution for these ones. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14268) Improve KeyLocker
[
https://issues.apache.org/jira/browse/HBASE-14268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14268:
--
Attachment: HBASE-14268-V7.patch
> Improve KeyLocker
> -
>
> Key: HBASE-14268
> URL: https://issues.apache.org/jira/browse/HBASE-14268
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 14268-V5.patch, HBASE-14268-V2.patch,
> HBASE-14268-V3.patch, HBASE-14268-V4.patch, HBASE-14268-V5.patch,
> HBASE-14268-V5.patch, HBASE-14268-V6.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268-V7.patch,
> HBASE-14268-V7.patch, HBASE-14268-V7.patch, HBASE-14268.patch,
> KeyLockerIncrKeysPerformance.java, KeyLockerPerformance.java,
> ReferenceTestApp.java
>
>
> 1. In the implementation of {{KeyLocker}} it uses atomic variables inside a
> synchronized block, which doesn't make sense. Moreover, logic inside the
> synchronized block is not trivial so that it makes less performance in heavy
> multi-threaded environment.
> 2. {{KeyLocker}} gives an instance of {{RentrantLock}} which is already
> locked, but it doesn't follow the contract of {{ReentrantLock}} because you
> are not allowed to freely invoke lock/unlock methods under that contract.
> That introduces a potential risk; Whenever you see a variable of the type
> {{RentrantLock}}, you should pay attention to what the included instance is
> coming from.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951354#comment-14951354 ] Hudson commented on HBASE-14583: SUCCESS: Integrated in HBase-1.3-IT #222 (See [https://builds.apache.org/job/HBase-1.3-IT/222/]) HBASE-14583 Enabled client-side metrics by default (ndimiduk: rev dced02ad9ff647118a830e10b3628a4687754a72) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951356#comment-14951356 ] Hudson commented on HBASE-14583: FAILURE: Integrated in HBase-1.3 #248 (See [https://builds.apache.org/job/HBase-1.3/248/]) HBASE-14583 Enabled client-side metrics by default (ndimiduk: rev dced02ad9ff647118a830e10b3628a4687754a72) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14490) [RpcServer] reuse request read buffer
[
https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951279#comment-14951279
]
stack commented on HBASE-14490:
---
What is wrong w/ BoundedByteBufferPool ? Please be explicit (trying to follow
along so may not have all the context -- excuse me). A different or subclass
of BufferPool, sure.
> [RpcServer] reuse request read buffer
> -
>
> Key: HBASE-14490
> URL: https://issues.apache.org/jira/browse/HBASE-14490
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
>Affects Versions: 2.0.0, 1.0.2
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
> Labels: performance
> Fix For: 2.0.0, 1.0.2
>
> Attachments: ByteBufferPool.java, HBASE-14490-v1.patch,
> HBASE-14490-v10.patch, HBASE-14490-v11.patch, HBASE-14490-v2.patch,
> HBASE-14490-v3.patch, HBASE-14490-v4.patch, HBASE-14490-v5.patch,
> HBASE-14490-v6.patch, HBASE-14490-v7.patch, HBASE-14490-v8.patch,
> HBASE-14490-v9.patch
>
>
> Reuse buffer to read request.It's not necessary to every request free
> buffer.The idea of optimization is to reduce the times that allocate
> ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}},
> {{processConnectionHeader}} accept a ByteBuffer instead of byte[].They can
> move {{ByteBuffer.position}} correctly when we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a reused ByteBuffer in each {{Connection}}.
> a) Size is dynamic to fit specific client.
> b) If size deviate far from average size, we replace a new one.
> c) Using a SoftReference to reference the buffer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14517) Show regionserver's version in master status page
[ https://issues.apache.org/jira/browse/HBASE-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14517: -- Pushed to branch-1.2+ Thanks for nice patch [~liushaohui] > Show regionserver's version in master status page > - > > Key: HBASE-14517 > URL: https://issues.apache.org/jira/browse/HBASE-14517 > Project: HBase > Issue Type: Improvement > Components: monitoring >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14517-v1.diff, HBASE-14517-v1.patch, > HBASE-14517-v2.patch > > > In production env, regionservers may be removed from the cluster for hardware > problems and rejoined the cluster after the repair. There is a potential risk > that the version of rejoined regionserver may diff from others because the > cluster has been upgraded through many versions. > To solve this, we can show the all regionservers' version in the server list > of master's status page, and highlight the regionserver when its version is > different from the master's version, similar to HDFS-3245 > Suggestions are welcome~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14582) Regionserver status webpage bucketcache list can become huge
[ https://issues.apache.org/jira/browse/HBASE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951327#comment-14951327 ] stack commented on HBASE-14582: --- Sorry, yeah, dumb trying to render such a large page. I'll just turn it off since the value is minor if at all being able to look at buckets > Regionserver status webpage bucketcache list can become huge > > > Key: HBASE-14582 > URL: https://issues.apache.org/jira/browse/HBASE-14582 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: James Hartshorn >Priority: Trivial > > The regionserver status page, such as http://127.0.0.1:60030/rs-status always > downloads information about every bucket in the cache. In some cases this > can be hundreds of thousands of buckets, causing megabytes of info to need to > be downloaded and significant browser instability and memory usage: > wc -l HBase-Region-Server-hb22.html > 2010116 HBase-Region-Server-hb22.html > ls -lah HBase-Region-Server-hb22.html > 32M Oct 6 19:23 HBase-Region-Server-hb22.html > Firefox "about:memory": > 1,330.18 MB (48.22%) -- top(http://hb22:60030/rs-status#bc_l2, id=2010) > 1,329.61 MB (48.20%) -- active/window(http://hb22:60030/rs-status#bc_l2) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11590) use a specific ThreadPoolExecutor
[ https://issues.apache.org/jira/browse/HBASE-11590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951353#comment-14951353 ] stack commented on HBASE-11590: --- bq. It is however incredible to see how faster/better it is compared to the other pools. It is in a different place altogether compared to the other two... in terms of timing and threads used (maybe just because it is more parsimonious in its thread use)? Man. Thats a very nice compare there [~nkeywal] Sure on the FJP. Thats a substantial enough change in perf. to make the change. > use a specific ThreadPoolExecutor > - > > Key: HBASE-11590 > URL: https://issues.apache.org/jira/browse/HBASE-11590 > Project: HBase > Issue Type: Bug > Components: Client, Performance >Affects Versions: 1.0.0, 2.0.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon >Priority: Minor > Fix For: 2.0.0 > > Attachments: ExecutorServiceTest.java, > LifoThreadPoolExecutorSQP.java, UnitQueueP.java, UnitQueuePU.java, tp.patch > > > The JDK TPE creates all the threads in the pool. As a consequence, we create > (by default) 256 threads even if we just need a few. > The attached TPE create threads only if we have something in the queue. > On a PE test with replica on, it improved the 99 latency percentile by 5%. > Warning: there are likely some race conditions, but I'm posting it here > because there is may be an implementation available somewhere we can use, or > a good reason not to do that. So feedback welcome as usual. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14557) MapReduce WALPlayer issue with NoTagsKeyValue
[
https://issues.apache.org/jira/browse/HBASE-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14557:
--
Attachment: HBASE-14557.patch
> MapReduce WALPlayer issue with NoTagsKeyValue
> -
>
> Key: HBASE-14557
> URL: https://issues.apache.org/jira/browse/HBASE-14557
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Jerry He
>Assignee: Anoop Sam John
> Attachments: HBASE-14557.patch, HBASE-14557.patch
>
>
> Running MapReduce WALPlayer to convert WAL into HFiles:
> {noformat}
> 15/10/05 20:28:08 INFO mapred.JobClient: Task Id :
> attempt_201508031611_0029_m_00_0, Status : FAILED
> java.io.IOException: Type mismatch in value from map: expected
> org.apache.hadoop.hbase.KeyValue, recieved
> org.apache.hadoop.hbase.NoTagsKeyValue
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:997)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:96)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at
> java.security.AccessController.doPrivileged(AccessController.java:369)
> at javax.security.auth.Subject.doAs(Subject.java:572)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14557) MapReduce WALPlayer issue with NoTagsKeyValue
[
https://issues.apache.org/jira/browse/HBASE-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951273#comment-14951273
]
stack commented on HBASE-14557:
---
I just removed TestMobSweeper (smile). Its a feature that is going away. The
others seem to be hanging from time to time (see HBASE-14420). Patch LGTM.
Let me rerun the test suite.
> MapReduce WALPlayer issue with NoTagsKeyValue
> -
>
> Key: HBASE-14557
> URL: https://issues.apache.org/jira/browse/HBASE-14557
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Jerry He
>Assignee: Anoop Sam John
> Attachments: HBASE-14557.patch, HBASE-14557.patch
>
>
> Running MapReduce WALPlayer to convert WAL into HFiles:
> {noformat}
> 15/10/05 20:28:08 INFO mapred.JobClient: Task Id :
> attempt_201508031611_0029_m_00_0, Status : FAILED
> java.io.IOException: Type mismatch in value from map: expected
> org.apache.hadoop.hbase.KeyValue, recieved
> org.apache.hadoop.hbase.NoTagsKeyValue
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:997)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:689)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:111)
> at
> org.apache.hadoop.hbase.mapreduce.WALPlayer$WALKeyValueMapper.map(WALPlayer.java:96)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at
> java.security.AccessController.doPrivileged(AccessController.java:369)
> at javax.security.auth.Subject.doAs(Subject.java:572)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13858) RS/MasterDumpServlet dumps threads before its “Stacks” header
[ https://issues.apache.org/jira/browse/HBASE-13858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13858: -- Resolution: Fixed Assignee: Fred Liu Hadoop Flags: Reviewed Fix Version/s: 1.2.0 Status: Resolved (was: Patch Available) Pushed to branch-1.2+ Thanks [~Justyle] > RS/MasterDumpServlet dumps threads before its “Stacks” header > - > > Key: HBASE-13858 > URL: https://issues.apache.org/jira/browse/HBASE-13858 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Fred Liu >Priority: Trivial > Labels: beginner > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-13858.patch > > > The stacktraces are captured using a Hadoop helper method, then its output is > merged with the current. I presume there is a simple flush after outputing > the "Stack" header missing, before then the caught output is dumped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14578) URISyntaxException during snapshot restore
[
https://issues.apache.org/jira/browse/HBASE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951168#comment-14951168
]
Ted Yu commented on HBASE-14578:
bq.
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager.test(TestReplicationWALReaderManager.java:190
I don't think 0.98 has TestReplicationWALReaderManager
Looks like the bot was checking some other concurrent QA run.
> URISyntaxException during snapshot restore
> --
>
> Key: HBASE-14578
> URL: https://issues.apache.org/jira/browse/HBASE-14578
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.5
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 0.98.16
>
> Attachments: HBASE-14578-0.98-V2.patch, HBASE-14578-0.98.patch
>
>
> Snapshot restore failed for a table which is created under user defined
> namespace,
> {noformat}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:475)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:208)
> at
> org.apache.hadoop.hbase.master.snapshot.CloneSnapshotHandler.handleCreateHdfsRegions(CloneSnapshotHandler.java:112)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:233)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:168)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI: tag_namespace:tableName_XYZ
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 9 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: tag_namespace:tableName_XYZ
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.(Path.java:172)
> at org.apache.hadoop.fs.Path.(Path.java:89)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreReferenceFile(RestoreSnapshotHelper.java:558)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreStoreFile(RestoreSnapshotHelper.java:531)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$200(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$2.storeFile(RestoreSnapshotHelper.java:509)
> at
> org.apache.hadoop.hbase.util.FSVisitor.visitRegionStoreFiles(FSVisitor.java:136)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitRegionStoreFiles(SnapshotReferenceUtil.java:127)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneRegion(RestoreSnapshotHelper.java:502)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.access$100(RestoreSnapshotHelper.java:106)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper$1.fillRegion(RestoreSnapshotHelper.java:479)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:160)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:118)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:115)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> tag_namespace:tableName_XYZ
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> ... 20 more
> {noformat}
> After region split, new daughter regions contain the reference of the
> original HFile until
[jira] [Commented] (HBASE-14585) Clean up TestSnapshotCloneIndependence
[
https://issues.apache.org/jira/browse/HBASE-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951249#comment-14951249
]
Hadoop QA commented on HBASE-14585:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765868/HBASE-14585.patch
against master branch at commit c7efd14726e932235dff52bd73f76cb861b5b1b3.
ATTACHMENT ID: 12765868
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence.testOfflineSnapshotDeleteIndependent(TestSnapshotCloneIndependence.java:181)
at
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence.testOfflineSnapshotAppendIndependent(TestSnapshotCloneIndependence.java:140)
at
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence.testOfflineSnapshotAppendIndependent(TestSnapshotCloneIndependence.java:140)
at
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence.testOfflineSnapshotAppendIndependent(TestSnapshotCloneIndependence.java:140)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15929//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15929//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15929//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15929//console
This message is automatically generated.
> Clean up TestSnapshotCloneIndependence
> --
>
> Key: HBASE-14585
> URL: https://issues.apache.org/jira/browse/HBASE-14585
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Affects Versions: 2.0.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-14585.patch
>
>
> TestSnapshotCloneIndependence has some issues as [~stack] has seen.
> Lets harden this test and make it test what was expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951097#comment-14951097
]
stack edited comment on HBASE-14584 at 10/9/15 10:30 PM:
-
The XML is output differently in jdk7 from jdk8. See below. All fields are
interpolated properly.. its just the order that is different.
2015-10-09 12:45:28,771 INFO [main] log.Slf4jLog(67): AS_XMLNAMEtestNamespaceKEY_2VALUE_2KEY_1VALUE_1
2015-10-09 12:45:28,789 INFO [main] log.Slf4jLog(67): toXMLKEY_1VALUE_1KEY_2VALUE_2NAMEtestNamespace
was (Author: stack):
The interpolation is messing up:
2015-10-09 12:45:28,771 INFO [main] log.Slf4jLog(67): AS_XMLNAMEtestNamespaceKEY_2VALUE_2KEY_1VALUE_1
2015-10-09 12:45:28,789 INFO [main] log.Slf4jLog(67): toXMLKEY_1VALUE_1KEY_2VALUE_2NAMEtestNamespace
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14582) Regionserver status webpage bucketcache list can become huge
[ https://issues.apache.org/jira/browse/HBASE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951321#comment-14951321 ] stack commented on HBASE-14582: --- I should just turn it off [~jhartshorn]? > Regionserver status webpage bucketcache list can become huge > > > Key: HBASE-14582 > URL: https://issues.apache.org/jira/browse/HBASE-14582 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: James Hartshorn >Priority: Trivial > > The regionserver status page, such as http://127.0.0.1:60030/rs-status always > downloads information about every bucket in the cache. In some cases this > can be hundreds of thousands of buckets, causing megabytes of info to need to > be downloaded and significant browser instability and memory usage: > wc -l HBase-Region-Server-hb22.html > 2010116 HBase-Region-Server-hb22.html > ls -lah HBase-Region-Server-hb22.html > 32M Oct 6 19:23 HBase-Region-Server-hb22.html > Firefox "about:memory": > 1,330.18 MB (48.22%) -- top(http://hb22:60030/rs-status#bc_l2, id=2010) > 1,329.61 MB (48.20%) -- active/window(http://hb22:60030/rs-status#bc_l2) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951349#comment-14951349
]
Nick Dimiduk commented on HBASE-14584:
--
Oh, right. We should make these tests just do a round-trip and verify the
object maps are the same. Serialization order is an unstable details for both
json and xml. Let me see...
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14516) categorize hadoop-compat tests
[ https://issues.apache.org/jira/browse/HBASE-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951145#comment-14951145 ] Sean Busbey commented on HBASE-14516: - [~apurtell] fine with this for 0.98? > categorize hadoop-compat tests > -- > > Key: HBASE-14516 > URL: https://issues.apache.org/jira/browse/HBASE-14516 > Project: HBase > Issue Type: Task > Components: build, hadoop2, test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Attachments: HBASE-14516.0.98.v2.patch, HBASE-14516.1.patch > > > the hadoop-compat and hadoop2-compat modules do not rely on the hbase > annotations test-jar and their tests aren't categorized. > this causes things to fail if you attempt to specify one of our test > categories to run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951182#comment-14951182
]
stack commented on HBASE-14584:
---
I applied the disable to branch-1.2 + Leaving issue open.
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14567) Tuneup hanging test TestMobCompactor and TestMobSweeper
[ https://issues.apache.org/jira/browse/HBASE-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14567: -- Summary: Tuneup hanging test TestMobCompactor and TestMobSweeper (was: Disable hanging test TestMobCompactor and TestMobSweeper) > Tuneup hanging test TestMobCompactor and TestMobSweeper > --- > > Key: HBASE-14567 > URL: https://issues.apache.org/jira/browse/HBASE-14567 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Jingcheng Du > Attachments: HBASE-14567.patch > > > These tests hang with some regularity. Please take a look > [~jingcheng...@intel.com] since I believe they are yours. If not, let me know > and i'll assign elsewhere. Otherwise, I'll just disable them. Thanks. > They hung here most recently: > https://builds.apache.org/job/PreCommit-HBASE-Build/15893//console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12911) Client-side metrics
[
https://issues.apache.org/jira/browse/HBASE-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951326#comment-14951326
]
Hudson commented on HBASE-12911:
SUCCESS: Integrated in HBase-0.98 #1148 (See
[https://builds.apache.org/job/HBase-0.98/1148/])
HBASE-12911 Client-side metrics (ndimiduk: rev
f4ef48f6489c938d343214761deef02c00ed52c8)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java
* hbase-client/pom.xml
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
*
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestMetricsConnection.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientTimeouts.java
> Client-side metrics
> ---
>
> Key: HBASE-12911
> URL: https://issues.apache.org/jira/browse/HBASE-12911
> Project: HBase
> Issue Type: New Feature
> Components: Client, Operability, Performance
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.16
>
> Attachments: 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch, 12911-0.98.00.patch,
> 12911-branch-1.00.patch, 12911.yammer.jpg, 12911.yammer.v00.patch,
> 12911.yammer.v01.patch, 12911.yammer.v02.patch, 12911.yammer.v02.patch,
> 12911.yammer.v03.branch-1.patch, 12911.yammer.v03.patch,
> 12911.yammer.v03.patch, 12911.yammer.v04.0.98.patch,
> 12911.yammer.v04.branch-1.2.patch, 12911.yammer.v04.branch-1.2.patch,
> 12911.yammer.v04.branch-1.patch, 12911.yammer.v04.branch-1.patch,
> 12911.yammer.v04.patch, 12911.yammer.v04.patch, am.jpg, client metrics
> RS-Master.jpg, client metrics client.jpg, conn_agg.jpg, connection
> attributes.jpg, ltt.jpg, standalone.jpg
>
>
> There's very little visibility into the hbase client. Folks who care to add
> some kind of metrics collection end up wrapping Table method invocations with
> {{System.currentTimeMillis()}}. For a crude example of this, have a look at
> what I did in {{PerformanceEvaluation}} for exposing requests latencies up to
> {{IntegrationTestRegionReplicaPerf}}. The client is quite complex, there's a
> lot going on under the hood that is impossible to see right now without a
> profiler. Being a crucial part of the performance of this distributed system,
> we should have deeper visibility into the client's function.
> I'm not sure that wiring into the hadoop metrics system is the right choice
> because the client is often embedded as a library in a user's application. We
> should have integration with our metrics tools so that, i.e., a client
> embedded in a coprocessor can report metrics through the usual RS channels,
> or a client used in a MR job can do the same.
> I would propose an interface-based system with pluggable implementations. Out
> of the box we'd include a hadoop-metrics implementation and one other,
> possibly [dropwizard/metrics|https://github.com/dropwizard/metrics].
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14580) Make the HBaseMiniCluster compliant with Kerberos
[
https://issues.apache.org/jira/browse/HBASE-14580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951336#comment-14951336
]
stack commented on HBASE-14580:
---
+1
> Make the HBaseMiniCluster compliant with Kerberos
> -
>
> Key: HBASE-14580
> URL: https://issues.apache.org/jira/browse/HBASE-14580
> Project: HBase
> Issue Type: Improvement
> Components: security, test
>Affects Versions: 2.0.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
> Fix For: 2.0.0
>
> Attachments: patch-14580.v1.patch
>
>
> Whne using MiniKDC and the minicluster in a unit test, there is a conflict
> causeed by HBaseTestingUtility:
> {code}
> public static User getDifferentUser(final Configuration c,
> final String differentiatingSuffix)
> throws IOException {
>// snip
> String username = User.getCurrent().getName() +
> differentiatingSuffix; < problem here
> User user = User.createUserForTesting(c, username,
> new String[]{"supergroup"});
> return user;
> }
> {code}
> This creates users like securedUser/localh...@example.com.hfs.0, and this
> does not work.
> My fix is to return the current user when Kerberos is set. I don't think that
> there is another option (any other opinion?). However this user is not in a
> group so we have logs like 'WARN [IPC Server handler 9 on 61366]
> security.UserGroupInformation (UserGroupInformation.java:getGroupNames(1521))
> - No groups available for user securedUser' I'm not sure of its impact.
> [~apurtell], what do you think?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12911) Client-side metrics
[
https://issues.apache.org/jira/browse/HBASE-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951365#comment-14951365
]
Hudson commented on HBASE-12911:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1100 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1100/])
HBASE-12911 Client-side metrics (ndimiduk: rev
f4ef48f6489c938d343214761deef02c00ed52c8)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientTimeouts.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestIPC.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
*
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestMetricsConnection.java
* hbase-client/pom.xml
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
> Client-side metrics
> ---
>
> Key: HBASE-12911
> URL: https://issues.apache.org/jira/browse/HBASE-12911
> Project: HBase
> Issue Type: New Feature
> Components: Client, Operability, Performance
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.16
>
> Attachments: 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch,
> 0001-HBASE-12911-Client-side-metrics.patch, 12911-0.98.00.patch,
> 12911-branch-1.00.patch, 12911.yammer.jpg, 12911.yammer.v00.patch,
> 12911.yammer.v01.patch, 12911.yammer.v02.patch, 12911.yammer.v02.patch,
> 12911.yammer.v03.branch-1.patch, 12911.yammer.v03.patch,
> 12911.yammer.v03.patch, 12911.yammer.v04.0.98.patch,
> 12911.yammer.v04.branch-1.2.patch, 12911.yammer.v04.branch-1.2.patch,
> 12911.yammer.v04.branch-1.patch, 12911.yammer.v04.branch-1.patch,
> 12911.yammer.v04.patch, 12911.yammer.v04.patch, am.jpg, client metrics
> RS-Master.jpg, client metrics client.jpg, conn_agg.jpg, connection
> attributes.jpg, ltt.jpg, standalone.jpg
>
>
> There's very little visibility into the hbase client. Folks who care to add
> some kind of metrics collection end up wrapping Table method invocations with
> {{System.currentTimeMillis()}}. For a crude example of this, have a look at
> what I did in {{PerformanceEvaluation}} for exposing requests latencies up to
> {{IntegrationTestRegionReplicaPerf}}. The client is quite complex, there's a
> lot going on under the hood that is impossible to see right now without a
> profiler. Being a crucial part of the performance of this distributed system,
> we should have deeper visibility into the client's function.
> I'm not sure that wiring into the hadoop metrics system is the right choice
> because the client is often embedded as a library in a user's application. We
> should have integration with our metrics tools so that, i.e., a client
> embedded in a coprocessor can report metrics through the usual RS channels,
> or a client used in a MR job can do the same.
> I would propose an interface-based system with pluggable implementations. Out
> of the box we'd include a hadoop-metrics implementation and one other,
> possibly [dropwizard/metrics|https://github.com/dropwizard/metrics].
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14588) Stop accessing test resources from within src folder
[
https://issues.apache.org/jira/browse/HBASE-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951574#comment-14951574
]
Hadoop QA commented on HBASE-14588:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12765959/hbase-14588.001.patch
against master branch at commit 2e593a9d3801a42751244ab4478650a581437875.
ATTACHMENT ID: 12765959
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 19 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15944//console
This message is automatically generated.
> Stop accessing test resources from within src folder
>
>
> Key: HBASE-14588
> URL: https://issues.apache.org/jira/browse/HBASE-14588
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.1.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
> Attachments: hbase-14588.001.patch, hbase-14588.001.patch
>
>
> A few tests in hbase-server reach into the src/test/data folder to get test
> resources, which is naughty since tests are supposed to only operate within
> the target/ folder. It's better to put these into src/test/resources and let
> them be automatically copied into target/ via the resources plugin, like
> other test resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
