[jira] [Commented] (HBASE-20564) Tighter ByteBufferKeyValue Cell Comparator
[ https://issues.apache.org/jira/browse/HBASE-20564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473380#comment-16473380 ] stack commented on HBASE-20564: --- In 1.4 we cache key value length on construction. In 2.0 we deserialize each time we need it (I should not have said a 'few'). Then inside the comparator, we are deserializing sizes repeatedly as we move through the key. For example, if the compare makes it out to a type or sequenceid compare, we end up deserializing row length 4 times, key length three times at least, and so on. Deserialization -- making shorts and ints from bytes -- is bulk of our comparator cpu usage. > Tighter ByteBufferKeyValue Cell Comparator > -- > > Key: HBASE-20564 > URL: https://issues.apache.org/jira/browse/HBASE-20564 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20564.branch-2.0.001.patch, > HBASE-20564.branch-2.patch > > > Comparing Cells in hbase2 takes almost 3x the CPU. > In hbase1, its a keyValue backed by a byte array caching a few important > values.. In hbase2, its a NoTagByteBufferChunkKeyValue(?) deserializing the > row/family/qualifier lengths repeatedly. > I tried making a purposed comparator -- one that was not generic -- and it > seemed to have a nicer profile coming close to hbase1 in percentage used > (I'll post graphs) when I ran it in my perpetual memstore filler (See scripts > attached to HBASE-20483). It doesn't work when I try to run it on cluster. > Let me run unit tests to see if it can figure what I have wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19064) Synchronous replication for HBase
[
https://issues.apache.org/jira/browse/HBASE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473378#comment-16473378
]
Hudson commented on HBASE-19064:
Results for branch HBASE-19064
[build #127 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/127/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/127//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/127//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/127//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Synchronous replication for HBase
> -
>
> Key: HBASE-19064
> URL: https://issues.apache.org/jira/browse/HBASE-19064
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> The guys from Alibaba made a presentation on HBaseCon Asia about the
> synchronous replication for HBase. We(Xiaomi) think this is a very useful
> feature for HBase so we want to bring it into the community version.
> This is a big feature so we plan to do it in a feature branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20574) prevRowLock seems not working
[
https://issues.apache.org/jira/browse/HBASE-20574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473339#comment-16473339
]
Ted Yu commented on HBASE-20574:
It seems your description was w.r.t. rowLockContext.readWriteLock and
prevRowLock.
I just added a log before the return statement, with String of prevRowLock
appended to the end.
You can run TestPassCustomCellViaRegionObserver in IDE and place breakpoint
inside the above if statement.
> prevRowLock seems not working
> -
>
> Key: HBASE-20574
> URL: https://issues.apache.org/jira/browse/HBASE-20574
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Yi Zheng
>Priority: Major
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In HRegion.java, getRowLockInternal using prevRowLock to check whether this
> lock is obtained before, but it may not work as imagine.
> Because this parameter passed as the last lock the thread obtained, and the
> code always be like this:
> {code:java}
> for (byte[] rows: rowsToLock) {
> rowLock = getRowLockInternal(row, prevRowLock);
> if (rowLock != prevRowLock){
> acquiredRowLocks.add(rowLock);
> prevRowLock = rowLock;
> }
> }{code}
> prevRowLock will never equal to the rowLockContext's lock in
> getRowLockInternal. I think this parameter is redundant.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20574) prevRowLock seems not working
[
https://issues.apache.org/jira/browse/HBASE-20574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473328#comment-16473328
]
Yi Zheng commented on HBASE-20574:
--
Hi
In the code above (foreach), the prevRowLock will never have the same row hash
key with the rowLock,
{code:java}
prevRowLockImpl.getLock() == rowLockContext.readWriteLock.readLock()){code}
this statement will always be false.
and btw, I can't found the log format you provide in the latest code (I
searched by 'using passed RowLockImpl')
> prevRowLock seems not working
> -
>
> Key: HBASE-20574
> URL: https://issues.apache.org/jira/browse/HBASE-20574
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Yi Zheng
>Priority: Major
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In HRegion.java, getRowLockInternal using prevRowLock to check whether this
> lock is obtained before, but it may not work as imagine.
> Because this parameter passed as the last lock the thread obtained, and the
> code always be like this:
> {code:java}
> for (byte[] rows: rowsToLock) {
> rowLock = getRowLockInternal(row, prevRowLock);
> if (rowLock != prevRowLock){
> acquiredRowLocks.add(rowLock);
> prevRowLock = rowLock;
> }
> }{code}
> prevRowLock will never equal to the rowLockContext's lock in
> getRowLockInternal. I think this parameter is redundant.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473304#comment-16473304
]
Ted Yu edited comment on HBASE-20566 at 5/13/18 12:08 AM:
--
I think the enabling of quota can be done in setUp() method so that we don't
need to restart the cluster twice.
On my Mac, the duration for TestRSGroups went from 182 seconds (with one of the
two restarts in your patch since I moved the enabling of quota to setUp, hence
dropping the first restart) to 166 seconds.
You just need to adjust the assertion in basic test:
{code}
-//3 meta,namespace, group
-assertEquals(3, count);
+//3 meta,namespace, rsgroup, quota
+assertEquals(4, count);
{code}
This also proves that all assertions in previous tests still hold with quota
enabled, except for the above one.
was (Author: yuzhih...@gmail.com):
I think the enabling of quota can be done in setUp() method so that we don't
need to restart the cluster twice.
On my Mac, the duration for TestRSGroups went from 182 seconds (with one of the
two restarts in your patch since I moved the enabling of quota to setUp, hence
dropping the first restart) to 166 seconds.
You just need to adjust the assertion in basic test:
{code}
-//3 meta,namespace, group
-assertEquals(3, count);
+//3 meta,namespace, rsgroup, quota
+assertEquals(4, count);
{code}
This also proves that all assertions in previous tests still hold with quota
enabled.
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20566.master.001.patch
>
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473308#comment-16473308
]
Hadoop QA commented on HBASE-20566:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 10s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
22s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20566 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12923180/HBASE-20566.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2d4f6c293227 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11
12:16:42 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 021f66d11d |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12799/testReport/ |
| Max. process+thread count | 2785 (vs. ulimit of 1) |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12799/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Creati
[jira] [Comment Edited] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473304#comment-16473304
]
Ted Yu edited comment on HBASE-20566 at 5/12/18 11:59 PM:
--
I think the enabling of quota can be done in setUp() method so that we don't
need to restart the cluster twice.
On my Mac, the duration for TestRSGroups went from 182 seconds (with one of the
two restarts in your patch since I moved the enabling of quota to setUp, hence
dropping the first restart) to 166 seconds.
You just need to adjust the assertion in basic test:
{code}
-//3 meta,namespace, group
-assertEquals(3, count);
+//3 meta,namespace, rsgroup, quota
+assertEquals(4, count);
{code}
This also proves that all assertions in previous tests still hold with quota
enabled.
was (Author: yuzhih...@gmail.com):
I think the enabling of quota can be done in setUp() method so that we don't
need to restart the cluster twice.
On my Mac, the duration for TestRSGroups went from 182 seconds to 166 seconds.
You just need to adjust the assertion in basic test:
{code}
-//3 meta,namespace, group
-assertEquals(3, count);
+//3 meta,namespace, rsgroup, quota
+assertEquals(4, count);
{code}
This also proves that all assertions in previous tests still hold with quota
enabled.
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20566.master.001.patch
>
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473304#comment-16473304
]
Ted Yu commented on HBASE-20566:
I think the enabling of quota can be done in setUp() method so that we don't
need to restart the cluster twice.
On my Mac, the duration for TestRSGroups went from 182 seconds to 166 seconds.
You just need to adjust the assertion in basic test:
{code}
-//3 meta,namespace, group
-assertEquals(3, count);
+//3 meta,namespace, rsgroup, quota
+assertEquals(4, count);
{code}
This also proves that all assertions in previous tests still hold with quota
enabled.
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20566.master.001.patch
>
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473301#comment-16473301
]
Nihal Jain commented on HBASE-20566:
Hi [~yuzhih...@gmail.com], they are also doing it in a similar way. Right?
{code:java}
.
.
LOG.info("Shutting down cluster");
UTIL.shutdownMiniHBaseCluster();
Thread.sleep(2000);
LOG.info("Starting cluster again with FN Balancer");
UTIL.getConfiguration().set(HConstants.HBASE_MASTER_LOADBALANCER_CLASS,
FavoredStochasticBalancer.class.getName());
UTIL.restartHBaseCluster(SLAVES);
while (!UTIL.getMiniHBaseCluster().getMaster().isInitialized()) {
Threads.sleep(1);
}
.
.{code}
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20566.master.001.patch
>
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473206#comment-16473206
]
Ted Yu edited comment on HBASE-20566 at 5/12/18 11:22 PM:
--
Patch makes sense.
See TestFavoredNodeTableImport.java for example of calling
shutdownMiniHBaseCluster / restartHBaseCluster - you don't need to write your
own method.
was (Author: yuzhih...@gmail.com):
I wonder if adding postCreateTable hook to
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterSpaceQuotaObserver.java
would help solve the issue.
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20566.master.001.patch
>
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20566:
---
Fix Version/s: 3.0.0
Status: Patch Available (was: Open)
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20566.master.001.patch
>
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20566:
---
Attachment: (was: HBASE-20566.master.001.patch)
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Attachments: HBASE-20566.master.001.patch
>
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20566:
---
Attachment: HBASE-20566.master.001.patch
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Attachments: HBASE-20566.master.001.patch
>
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20566:
---
Attachment: HBASE-20566.master.001.patch
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
> Attachments: HBASE-20566.master.001.patch
>
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-17958) Avoid passing unexpected cell to ScanQueryMatcher when optimize SEEK to SKIP
[
https://issues.apache.org/jira/browse/HBASE-17958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446908#comment-16446908
]
Lars Hofhansl edited comment on HBASE-17958 at 5/12/18 10:59 PM:
-
Seems we got stuck here. [~vik.karma] did some comparative testing between 0.98
and 1.x and found a 50% slowdown for cases with many columns. This is a highly
likely culprit.
was (Author: lhofhansl):
Seems we god stuck here. [~vik.karma] did some comparative testing between 0.98
and 1.x and found a 50% slowdown for cases with many columns. This is a highly
likely culprit.
> Avoid passing unexpected cell to ScanQueryMatcher when optimize SEEK to SKIP
>
>
> Key: HBASE-17958
> URL: https://issues.apache.org/jira/browse/HBASE-17958
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 1.4.0, 2.0.0
>
> Attachments: 0001-add-one-ut-testWithColumnCountGetFilter.patch,
> 17958-add.txt, HBASE-17958-branch-1.patch, HBASE-17958-branch-1.patch,
> HBASE-17958-branch-1.patch, HBASE-17958-branch-1.patch, HBASE-17958-v1.patch,
> HBASE-17958-v2.patch, HBASE-17958-v3.patch, HBASE-17958-v4.patch,
> HBASE-17958-v5.patch, HBASE-17958-v6.patch, HBASE-17958-v7.patch,
> HBASE-17958-v7.patch
>
>
> {code}
> ScanQueryMatcher.MatchCode qcode = matcher.match(cell);
> qcode = optimize(qcode, cell);
> {code}
> The optimize method may change the MatchCode from SEEK_NEXT_COL/SEEK_NEXT_ROW
> to SKIP. But it still pass the next cell to ScanQueryMatcher. It will get
> wrong result when use some filter, etc. ColumnCountGetFilter. It just count
> the columns's number. If pass a same column to this filter, the count result
> will be wrong. So we should avoid passing cell to ScanQueryMatcher when
> optimize SEEK to SKIP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-6562) Fake KVs are sometimes passed to filters
[ https://issues.apache.org/jira/browse/HBASE-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-6562. -- Resolution: Fixed In 1.4+ this should be fixed. > Fake KVs are sometimes passed to filters > > > Key: HBASE-6562 > URL: https://issues.apache.org/jira/browse/HBASE-6562 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Priority: Minor > Attachments: 6562-0.94-v1.txt, 6562-0.96-v1.txt, 6562-v2.txt, > 6562-v3.txt, 6562-v4.txt, 6562-v5.txt, 6562.txt, minimalTest.java > > > In internal tests at Salesforce we found that fake row keys sometimes are > passed to filters (Filter.filterRowKey(...) specifically). > The KVs are eventually filtered by the StoreScanner/ScanQueryMatcher, but the > row key is passed to filterRowKey in RegionScannImpl *before* that happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-10342) RowKey Prefix Bloom Filter
[ https://issues.apache.org/jira/browse/HBASE-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473288#comment-16473288 ] Lars Hofhansl commented on HBASE-10342: --- I was just thinking about this and was about to file a Jira with the idea. Then I came across this. I take it that nothing has happened here... :) This would be useful in many scenarios (prefix bloomfilters can be used for seeks) and perhaps specifically for Phoenix, which never uses Get's anywhere. [~jamestaylor], [~apurtell] One could start with something simple that simply defines a fix length prefix... Perhaps allow loading a transformer class or something. > RowKey Prefix Bloom Filter > -- > > Key: HBASE-10342 > URL: https://issues.apache.org/jira/browse/HBASE-10342 > Project: HBase > Issue Type: New Feature >Reporter: Liyin Tang >Priority: Major > > When designing HBase schema for some use cases, it is quite common to combine > multiple information within the RowKey. For instance, assuming that rowkey is > constructed as md5(id1) + id1 + id2, and user wants to scan all the rowkeys > which starting by id1. In such case, the rowkey bloom filter is able to cut > more unnecessary seeks during the scan. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20483) [PERFORMANCE] Flushing is 2x slower in hbase2.
[ https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473266#comment-16473266 ] Anastasia Braginsky commented on HBASE-20483: - Thank you, [~stack]! I'll try to take a look on HBASE-20411, but please do not wait for me. Your document about performance is very interesting, I need to get deeper into it. > [PERFORMANCE] Flushing is 2x slower in hbase2. > -- > > Key: HBASE-20483 > URL: https://issues.apache.org/jira/browse/HBASE-20483 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Attachments: > 0001-HBASE-20483-perpertual-flush-and-memstore-fill-branch-2.0.patch, > 0001-HBASE-20483-perpetual-flush.branch-1.4.patch, > 0001-HBASE-20483-perpetual-flush.branch-1.4.patch, > PerpetualFlushingProgram.patch, PerpetualFlushingProgram1.2.patch, > SnapshotSegmentScanner_2.0.0.patch > > > When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0 > Flush History" > https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826 > for compare. First noted by [~anoop.hbase] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20574) prevRowLock seems not working
[
https://issues.apache.org/jira/browse/HBASE-20574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473208#comment-16473208
]
Ted Yu commented on HBASE-20574:
By adding a DEBUG log before the return shown above, I saw the following in
unit test output:
{code}
hbase-server/target/surefire-reports/org.apache.hadoop.hbase.coprocessor.TestPassCustomCellViaRegionObserver-output.txt:2018-05-12
17:07:39,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=35117]
regionserver.HRegion(5708): using passed
RowLockImpl{context=RowLockContext{row=ROW,
readWriteLock=java.util.concurrent.locks.ReentrantReadWriteLock@6ba035ba[Write
locks = 0, Read locks = 1], count=1,
threadName=RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=35117},
lock=java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock@5539c0cb[Read
locks = 1]}
{code}
Meaning prevRowLock would be passed back to caller in some cases.
> prevRowLock seems not working
> -
>
> Key: HBASE-20574
> URL: https://issues.apache.org/jira/browse/HBASE-20574
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Yi Zheng
>Priority: Major
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In HRegion.java, getRowLockInternal using prevRowLock to check whether this
> lock is obtained before, but it may not work as imagine.
> Because this parameter passed as the last lock the thread obtained, and the
> code always be like this:
> {code:java}
> for (byte[] rows: rowsToLock) {
> rowLock = getRowLockInternal(row, prevRowLock);
> if (rowLock != prevRowLock){
> acquiredRowLocks.add(rowLock);
> prevRowLock = rowLock;
> }
> }{code}
> prevRowLock will never equal to the rowLockContext's lock in
> getRowLockInternal. I think this parameter is redundant.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473206#comment-16473206
]
Ted Yu commented on HBASE-20566:
I wonder if adding postCreateTable hook to
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterSpaceQuotaObserver.java
would help solve the issue.
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473180#comment-16473180
]
Ted Yu commented on HBASE-20552:
w.r.t. the warning from ProcWal, I saw the following in a successful run
(another test run):
{code}
2018-05-09 01:39:58,463 INFO
[master/ctr-e138-1518143905142-296213-01-03:2]
wal.ProcedureWALFormatReader: Rebuilding tracker for
hdfs://mycluster/apps/hbase/data/MasterProcWALs/pv2-0001.log
2018-05-09 01:39:58,550 WARN
[master/ctr-e138-1518143905142-296213-01-03:2]
wal.ProcedureWALFormatReader: Nothing left to decode. Exiting with missing EOF,
log=hdfs://mycluster/apps/hbase/data/MasterProcWALs/pv2-0001.log
2018-05-09 01:39:58,659 DEBUG
[master/ctr-e138-1518143905142-296213-01-03:2]
procedure2.ProcedureExecutor: Completed pid=40, state=SUCCESS;
ServerCrashProcedure
server=ctr-e138-1518143905142-296213-01-03.hwx.site,16020,1525829363193,
splitWal=true, meta=false
{code}
I am not sure if the 'Nothing left to decode' was related to the cause of this
issue (unexpected state).
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-07.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-08.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
>
[jira] [Commented] (HBASE-20574) prevRowLock seems not working
[
https://issues.apache.org/jira/browse/HBASE-20574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473133#comment-16473133
]
Ted Yu commented on HBASE-20574:
This is related code from {{getRowLockInternal}} :
{code}
RowLockImpl prevRowLockImpl = (RowLockImpl)prevRowLock;
if ((prevRowLockImpl != null) && (prevRowLockImpl.getLock() ==
rowLockContext.readWriteLock.readLock())) {
success = true;
return prevRowLock;
}
{code}
Why do you think the above return would never be executed ?
> prevRowLock seems not working
> -
>
> Key: HBASE-20574
> URL: https://issues.apache.org/jira/browse/HBASE-20574
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Yi Zheng
>Priority: Major
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In HRegion.java, getRowLockInternal using prevRowLock to check whether this
> lock is obtained before, but it may not work as imagine.
> Because this parameter passed as the last lock the thread obtained, and the
> code always be like this:
> {code:java}
> for (byte[] rows: rowsToLock) {
> rowLock = getRowLockInternal(row, prevRowLock);
> if (rowLock != prevRowLock){
> acquiredRowLocks.add(rowLock);
> prevRowLock = rowLock;
> }
> }{code}
> prevRowLock will never equal to the rowLockContext's lock in
> getRowLockInternal. I think this parameter is redundant.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20569) NPE in RecoverStandbyProcedure.execute
[
https://issues.apache.org/jira/browse/HBASE-20569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473125#comment-16473125
]
Hadoop QA commented on HBASE-20569:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
4s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
1s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
10s{color} | {color:red} hbase-server: The patch generated 6 new + 0 unchanged
- 0 fixed = 6 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}117m 47s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestServerCrashProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20569 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12923135/HBASE-20569.HBASE-19064.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c9d9fbb6438f 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-19064 / 3f08903e8b |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/12798/artifa
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[
https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473091#comment-16473091
]
Hudson commented on HBASE-20411:
Results for branch master
[build #328 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/328/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/328//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/328//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/328//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Ameliorate MutableSegment synchronize
> -
>
> Key: HBASE-20411
> URL: https://issues.apache.org/jira/browse/HBASE-20411
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: huaxiang sun
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 2.load.patched.17704.lock.svg,
> 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg,
> 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg,
> 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch,
> HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch,
> HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch,
> HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch,
> HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch,
> HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch,
> HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch,
> HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png
>
>
> This item is migrated from HBASE-20236 so it gets dedicated issue.
> Let me upload evidence that has this synchronize as a stake in our write-time
> perf. I'll migrate the patch I posted with updates that come of comments
> posted by [~mdrob] on the HBASE-20236 issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20576) Check whether the remote WAL directory is provided when transiting to A
Duo Zhang created HBASE-20576: - Summary: Check whether the remote WAL directory is provided when transiting to A Key: HBASE-20576 URL: https://issues.apache.org/jira/browse/HBASE-20576 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang When testing on the real cluster I typed a wrong remote wal directory. Then I start the procedure for transiting the peer to A, the procedure was stuck there for ever... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20569) NPE in RecoverStandbyProcedure.execute
[ https://issues.apache.org/jira/browse/HBASE-20569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-20569: --- Status: Patch Available (was: Open) > NPE in RecoverStandbyProcedure.execute > -- > > Key: HBASE-20569 > URL: https://issues.apache.org/jira/browse/HBASE-20569 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-20569.HBASE-19064.001.patch > > > We call ReplaySyncReplicationWALManager.initPeerWorkers in INIT_WORKERS state > and then use it in DISPATCH_TASKS. But if we restart the master and the > procedure is restarted from state DISPATCH_TASKS, no one will call the > initPeerWorkers method and we will get NPE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[
https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473028#comment-16473028
]
Hudson commented on HBASE-20411:
Results for branch branch-2.0
[build #289 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/289/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/289//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/289//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/289//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Ameliorate MutableSegment synchronize
> -
>
> Key: HBASE-20411
> URL: https://issues.apache.org/jira/browse/HBASE-20411
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: huaxiang sun
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 2.load.patched.17704.lock.svg,
> 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg,
> 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg,
> 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch,
> HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch,
> HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch,
> HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch,
> HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch,
> HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch,
> HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch,
> HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png
>
>
> This item is migrated from HBASE-20236 so it gets dedicated issue.
> Let me upload evidence that has this synchronize as a stake in our write-time
> perf. I'll migrate the patch I posted with updates that come of comments
> posted by [~mdrob] on the HBASE-20236 issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[
https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473016#comment-16473016
]
Hudson commented on HBASE-20411:
Results for branch branch-2
[build #725 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/725/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/725//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/725//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/725//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Ameliorate MutableSegment synchronize
> -
>
> Key: HBASE-20411
> URL: https://issues.apache.org/jira/browse/HBASE-20411
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: huaxiang sun
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: 2.load.patched.17704.lock.svg,
> 2.load.patched.2.17704.lock.svg, 2.more.patch.12010.lock.svg,
> 2.pe.write.32026.lock.svg, 2.pe.write.ameliorate.106553.lock.svg,
> 41901.lock.svg, HBASE-20411-atomiclong-longadder.patch,
> HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch,
> HBASE-20411.branch-2.0.003.patch, HBASE-20411.branch-2.0.004.patch,
> HBASE-20411.branch-2.0.005.patch, HBASE-20411.branch-2.0.006.patch,
> HBASE-20411.branch-2.0.007.patch, HBASE-20411.branch-2.0.008.patch,
> HBASE-20411.branch-2.0.009.patch, HBASE-20411.branch-2.0.010.patch,
> HBASE-20411.branch-2.0.011.patch, HBASE-20411.branch-2.0.012.patch,
> HBASE-20411.branch-2.0.013.patch, jmc6.write_time_locks.png
>
>
> This item is migrated from HBASE-20236 so it gets dedicated issue.
> Let me upload evidence that has this synchronize as a stake in our write-time
> perf. I'll migrate the patch I posted with updates that come of comments
> posted by [~mdrob] on the HBASE-20236 issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain reassigned HBASE-20566:
--
Assignee: Nihal Jain
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Assignee: Nihal Jain
>Priority: Major
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20566) Creating a system table after enabling rsgroup feature puts in region into RIT
[
https://issues.apache.org/jira/browse/HBASE-20566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472977#comment-16472977
]
Nihal Jain commented on HBASE-20566:
I think we can. Will try and update soon. And thanks for the reference, seems
useful. Will look into it.
> Creating a system table after enabling rsgroup feature puts in region into RIT
> --
>
> Key: HBASE-20566
> URL: https://issues.apache.org/jira/browse/HBASE-20566
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
>
> *Steps to reproduce*
> - Enable {{rsgroup}} feature
> - Enable {{quota}} feature which created {{hbase::quota}} table
> - quota table region will be marked as RIT since the {{rsgroup}} for the
> table is not known
> {noformat}
> 2018-05-10 14:33:32,392 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from null to
> ENABLING
> 2018-05-10 14:33:32,397 WARN [ProcedureExecutorThread-0]
> rsgroup.RSGroupBasedLoadBalancer: Group for table hbase:quota is null
> 2018-05-10 14:33:32,398 WARN [ProcedureExecutorThread-0]
> master.RegionStates: Failed to open/close 89490cd5e00ea8948af413a1df65091a on
> null, set to FAILED_OPEN
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> master.RegionStates: Transition {89490cd5e00ea8948af413a1df65091a
> state=OFFLINE, ts=1525977212397, server=null} to
> {89490cd5e00ea8948af413a1df65091a state=FAILED_OPEN, ts=1525977212398,
> server=null}
> 2018-05-10 14:33:32,398 INFO [ProcedureExecutorThread-0]
> zookeeper.ZKTableStateManager: Moving table hbase:quota state from ENABLING
> to ENABLED
> {noformat}
> - Reason for this issue: Issue
> - [system table
> creation|https://github.com/apache/hbase/blob/061a31fad1654d9ded96d118e04c14860413fa25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1793]
> doesn't move the table to the appropriate rs group to which system namespace
> is assigned to. Need to execute logic similar to what is done in the
> RSGroupAdminEndpoint for [post table creation|#L377] for user table creation.
> *Work Around*
> - Assigning the system table to ``default`` rsgroup (or to the rsgroup to
> which the system namespace has been assigned).
> - Manually assigning the region in RIT from the system table
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20444) Improve version comparison logic for HBase specific version string and add unit tests
[
https://issues.apache.org/jira/browse/HBASE-20444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maoling reassigned HBASE-20444:
---
Assignee: maoling
> Improve version comparison logic for HBase specific version string and add
> unit tests
> -
>
> Key: HBASE-20444
> URL: https://issues.apache.org/jira/browse/HBASE-20444
> Project: HBase
> Issue Type: Improvement
>Reporter: Umesh Agashe
>Assignee: maoling
>Priority: Major
>
> As [~busbey] commented on HBASE-18792, current logic for comparing version
> strings in class org.apache.hadoop.hbase.util.VersionInfo is generic and
> needs to be improved:
> {code:java}
> if (index < s1.length) {
> // s1 is longer
> return 1;
> }
> {code}
> {quote}I think this is wrong? like version "2.0.0" should be after
> "2.0.0-SNAPSHOT". it's also after "2.0.0-alpha-3" or "2.0.0-beta-1".
> {quote}
> Also in other cases 2.0.0 should be before 2.0.0-patch- and 2.0.0.1. Also
> 2.0 should be before 2.0.1.
> {quote}Can we expand the versions checked in TestVersionInfo to include a)
> some "same major different minor", b) "same minor different maintenance", c)
> both of the above, but SNAPSHOT, d) "-alpha" / "-beta"?
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20571) JMXJsonServlet generates invalid JSON if it has NaN in metrics
[
https://issues.apache.org/jira/browse/HBASE-20571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472970#comment-16472970
]
stack commented on HBASE-20571:
---
+1 from me. Thanks for fixing this. I noticed it but hacked my json parse to
deal with occasional NaN rather than do the real fix as you have. I'll not
commit yet. [~te...@apache.org] has a comment up on RB. Perhaps you want to
address.
> JMXJsonServlet generates invalid JSON if it has NaN in metrics
> --
>
> Key: HBASE-20571
> URL: https://issues.apache.org/jira/browse/HBASE-20571
> Project: HBase
> Issue Type: Bug
> Components: UI
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: HBASE-20571.branch-2.0.001.patch
>
>
> {{/jmx}} servlet responses invalid JSON, if some metrics are NaN:
> {code}
> "l1CacheHitCount" : 0,
> "l1CacheMissCount" : 0,
> "l1CacheHitRatio" : NaN,
> "l1CacheMissRatio" : NaN,
> "l2CacheHitCount" : 0,
> "l2CacheMissCount" : 0,
> "l2CacheHitRatio" : 0.0,
> "l2CacheMissRatio" : 0.0,
> {code}
> NaN is an invalid character sequence in JSON. We should not response NaN in
> metrics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20569) NPE in RecoverStandbyProcedure.execute
[ https://issues.apache.org/jira/browse/HBASE-20569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472963#comment-16472963 ] Guanghao Zhang commented on HBASE-20569: Will attach ut later. > NPE in RecoverStandbyProcedure.execute > -- > > Key: HBASE-20569 > URL: https://issues.apache.org/jira/browse/HBASE-20569 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-20569.HBASE-19064.001.patch > > > We call ReplaySyncReplicationWALManager.initPeerWorkers in INIT_WORKERS state > and then use it in DISPATCH_TASKS. But if we restart the master and the > procedure is restarted from state DISPATCH_TASKS, no one will call the > initPeerWorkers method and we will get NPE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20569) NPE in RecoverStandbyProcedure.execute
[ https://issues.apache.org/jira/browse/HBASE-20569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-20569: -- Assignee: Guanghao Zhang > NPE in RecoverStandbyProcedure.execute > -- > > Key: HBASE-20569 > URL: https://issues.apache.org/jira/browse/HBASE-20569 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-20569.HBASE-19064.001.patch > > > We call ReplaySyncReplicationWALManager.initPeerWorkers in INIT_WORKERS state > and then use it in DISPATCH_TASKS. But if we restart the master and the > procedure is restarted from state DISPATCH_TASKS, no one will call the > initPeerWorkers method and we will get NPE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20569) NPE in RecoverStandbyProcedure.execute
[ https://issues.apache.org/jira/browse/HBASE-20569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-20569: --- Attachment: HBASE-20569.HBASE-19064.001.patch > NPE in RecoverStandbyProcedure.execute > -- > > Key: HBASE-20569 > URL: https://issues.apache.org/jira/browse/HBASE-20569 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-20569.HBASE-19064.001.patch > > > We call ReplaySyncReplicationWALManager.initPeerWorkers in INIT_WORKERS state > and then use it in DISPATCH_TASKS. But if we restart the master and the > procedure is restarted from state DISPATCH_TASKS, no one will call the > initPeerWorkers method and we will get NPE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
