[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083526#comment-15083526
]
Hudson commented on HBASE-15058:
SUCCESS: Integrated in HBase-1.3 #480 (See
[https://builds.apache.org/job/HBase-1.3/480/])
HBASE-15058 AssignmentManager should account for unsuccessful split (tedyu: rev
407cda4a0c4bc1338af175299372a4b1976bb44a)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 15058-branch-1-v1.txt, 15058-branch-1-v2.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStateListener.onRegionSplitReverted(hri);
> {code}
> onRegionSplit() handles the above 3 TransitionCode's. However, errorMsg is
> normally null (onRegionSplit returns null at the end).
> This would result in onRegionSplitReverted() being called for cases of
> SPLIT_PONR and SPLIT.
> When region split fails, AssignmentManager#onRegionTransition() should
> account for the failure properly so that quota bookkeeping is consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083450#comment-15083450
]
Hudson commented on HBASE-15058:
SUCCESS: Integrated in HBase-1.2 #489 (See
[https://builds.apache.org/job/HBase-1.2/489/])
HBASE-15058 AssignmentManager should account for unsuccessful split (tedyu: rev
1fab320d52ac6d0ffcf92fe30a804e8f67c710db)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 15058-branch-1-v1.txt, 15058-branch-1-v2.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStateListener.onRegionSplitReverted(hri);
> {code}
> onRegionSplit() handles the above 3 TransitionCode's. However, errorMsg is
> normally null (onRegionSplit returns null at the end).
> This would result in onRegionSplitReverted() being called for cases of
> SPLIT_PONR and SPLIT.
> When region split fails, AssignmentManager#onRegionTransition() should
> account for the failure properly so that quota bookkeeping is consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083245#comment-15083245
]
Hudson commented on HBASE-15058:
SUCCESS: Integrated in HBase-1.2-IT #378 (See
[https://builds.apache.org/job/HBase-1.2-IT/378/])
HBASE-15058 AssignmentManager should account for unsuccessful split (tedyu: rev
1fab320d52ac6d0ffcf92fe30a804e8f67c710db)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 15058-branch-1-v1.txt, 15058-branch-1-v2.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStateListener.onRegionSplitReverted(hri);
> {code}
> onRegionSplit() handles the above 3 TransitionCode's. However, errorMsg is
> normally null (onRegionSplit returns null at the end).
> This would result in onRegionSplitReverted() being called for cases of
> SPLIT_PONR and SPLIT.
> When region split fails, AssignmentManager#onRegionTransition() should
> account for the failure properly so that quota bookkeeping is consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083177#comment-15083177
]
Hudson commented on HBASE-15058:
SUCCESS: Integrated in HBase-1.3-IT #420 (See
[https://builds.apache.org/job/HBase-1.3-IT/420/])
HBASE-15058 AssignmentManager should account for unsuccessful split (tedyu: rev
407cda4a0c4bc1338af175299372a4b1976bb44a)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 15058-branch-1-v1.txt, 15058-branch-1-v2.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStateListener.onRegionSplitReverted(hri);
> {code}
> onRegionSplit() handles the above 3 TransitionCode's. However, errorMsg is
> normally null (onRegionSplit returns null at the end).
> This would result in onRegionSplitReverted() being called for cases of
> SPLIT_PONR and SPLIT.
> When region split fails, AssignmentManager#onRegionTransition() should
> account for the failure properly so that quota bookkeeping is consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082511#comment-15082511
]
Heng Chen commented on HBASE-15058:
---
Thanks [~anoop.hbase], you are right.
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 15058-branch-1-v1.txt, 15058-branch-1-v2.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStateListener.onRegionSplitReverted(hri);
> {code}
> onRegionSplit() handles the above 3 TransitionCode's. However, errorMsg is
> normally null (onRegionSplit returns null at the end).
> This would result in onRegionSplitReverted() being called for cases of
> SPLIT_PONR and SPLIT.
> When region split fails, AssignmentManager#onRegionTransition() should
> account for the failure properly so that quota bookkeeping is consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082503#comment-15082503
]
Ashish Singhi commented on HBASE-15058:
---
{quote}
Should we need to do the same thing to merge process?
I notice the quota info updated when state is MERGE_REVERTED, i think it
should be done when state is READY_TO_MERGE like split process, and it also
need revert when failed. wdyt?
{quote}
No, it is not required.
Quota information is updated only on successful merge.
{code}
if (code == TransitionCode.MERGED &&
org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
try {
regionStateListener.onRegionMerged(hri);
} catch (IOException exp) {
errorMsg = StringUtils.stringifyException(exp);
}
}
{code}
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 15058-branch-1-v1.txt, 15058-branch-1-v2.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStateListener.onRegionSplitReverted(hri);
> {code}
> onRegionSplit() handles the above 3 TransitionCode's. However, errorMsg is
> normally null (onRegionSplit returns null at the end).
> This would result in onRegionSplitReverted() being called for cases of
> SPLIT_PONR and SPLIT.
> When region split fails, AssignmentManager#onRegionTransition() should
> account for the failure properly so that quota bookkeeping is consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082498#comment-15082498
]
Heng Chen commented on HBASE-15058:
---
LGTM.
One question:
Should we need to do the same thing to merge process?
I notice the quota info updated when state is MERGE_REVERTED, i think it
should be done when state is READY_TO_MERGE like split process, and it also
need revert when failed. wdyt?
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 1.3.0, 1.2.1
>
> Attachments: 15058-branch-1-v1.txt, 15058-branch-1-v2.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStateListener.onRegionSplitReverted(hri);
> {code}
> onRegionSplit() handles the above 3 TransitionCode's. However, errorMsg is
> normally null (onRegionSplit returns null at the end).
> This would result in onRegionSplitReverted() being called for cases of
> SPLIT_PONR and SPLIT.
> When region split fails, AssignmentManager#onRegionTransition() should
> account for the failure properly so that quota bookkeeping is consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15081539#comment-15081539
]
Hadoop QA commented on HBASE-15058:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12780330/15058-branch-1-v2.txt
against branch-1 branch at commit 00656688f73c85ea9e6f5241ac852f72e774eeea.
ATTACHMENT ID: 12780330
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/17110//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/17110//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/17110//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/17110//console
This message is automatically generated.
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15058-branch-1-v1.txt, 15058-branch-1-v2.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080741#comment-15080741
]
Ashish Singhi commented on HBASE-15058:
---
Overall lgtm.
{code}
private String onRegionSplitReverted(ServerName sn, TransitionCode code,
final HRegionInfo p, final HRegionInfo a, final HRegionInfo b) {
{code}
{{code}} parameter is not used in the method. Can be removed.
{code}
final RegionState rs_p = regionStates.getRegionState(p);
RegionState rs_a = regionStates.getRegionState(a);
RegionState rs_b = regionStates.getRegionState(b);
if (!(rs_p.isOpenOrSplittingOnServer(sn)
&& (rs_a == null || rs_a.isOpenOrSplittingNewOnServer(sn))
&& (rs_b == null || rs_b.isOpenOrSplittingNewOnServer(sn {
return "Not in state good for split";
{code}
Minor suggestion, can we extract it to a method, same lines of code is used in
the {{onRegionSplit}} method.
It will be good if we can add a unit test case for this.
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15058-branch-1-v1.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStateListener.onRegionSplitReverted(hri);
> {code}
> onRegionSplit() handles the above 3 TransitionCode's. However, errorMsg is
> normally null (onRegionSplit returns null at the end).
> This would result in onRegionSplitReverted() being called for cases of
> SPLIT_PONR and SPLIT.
> When region split fails, AssignmentManager#onRegionTransition() should
> account for the failure properly so that quota bookkeeping is consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15076137#comment-15076137
]
Hadoop QA commented on HBASE-15058:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12780162/15058-branch-1-v1.txt
against branch-1 branch at commit 92abf8ac5743849d4c32d6abf8fd33f161bab249.
ATTACHMENT ID: 12780162
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestShell
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/17096//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/17096//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/17096//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/17096//console
This message is automatically generated.
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15058-branch-1-v1.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringU
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15076134#comment-15076134
]
Hadoop QA commented on HBASE-15058:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12780135/15058-branch-1-v1.txt
against branch-1 branch at commit 92abf8ac5743849d4c32d6abf8fd33f161bab249.
ATTACHMENT ID: 12780135
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/17095//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/17095//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/17095//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/17095//console
This message is automatically generated.
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15058-branch-1-v1.txt
>
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following TransitionCode's
> is not clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStat
[jira] [Commented] (HBASE-15058) AssignmentManager should account for unsuccessful split correctly which initially passes quota check
[
https://issues.apache.org/jira/browse/HBASE-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075952#comment-15075952
]
Ted Yu commented on HBASE-15058:
Turns out that I was looking at branch-1 code.
Modified description to be specific.
> AssignmentManager should account for unsuccessful split correctly which
> initially passes quota check
>
>
> Key: HBASE-15058
> URL: https://issues.apache.org/jira/browse/HBASE-15058
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>
> When region split doesn't pass quota check, we would see exception similar to
> the following:
> {code}
> 2015-12-29 16:07:33,653 INFO [RS:0;10.21.128.189:57449-splits-1451434041585]
> regionserver.SplitRequest(97): Running rollback/cleanup of failed split of
> np2:
> testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.;
> Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,
> z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> java.io.IOException: Failed to get ok from master to split
> np2:testRegionNormalizationSplitOnCluster,z,1451434045065.27cccb3fae03002b8058beef61cb7c20.
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:345)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:262)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:155)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> {code}
> However, region split may fail for subsequent SplitTransactionPhase's in
> stepsBeforePONR().
> Currently in branch-1, the distinction among the following states is not
> clear in AssignmentManager#onRegionTransition():
> {code}
> case SPLIT_PONR:
> case SPLIT:
> case SPLIT_REVERTED:
> errorMsg =
> onRegionSplit(serverName, code, hri,
> HRegionInfo.convert(transition.getRegionInfo(1)),
> HRegionInfo.convert(transition.getRegionInfo(2)));
> if (org.apache.commons.lang.StringUtils.isEmpty(errorMsg)) {
> try {
> regionStateListener.onRegionSplitReverted(hri);
> {code}
> onRegionSplit() handles the above 3 TransitionCode's. However, errorMsg is
> normally null (onRegionSplit returns null at the end).
> This would result in onRegionSplitReverted() being called for cases of
> SPLIT_PONR and SPLIT.
> When region split fails, AssignmentManager#onRegionTransition() should
> account for the failure properly so that quota bookkeeping is consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
