[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559763#comment-16559763
]
Hudson commented on HBASE-20921:
Results for branch master
[build #410 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/410/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/410//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/410//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/410//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-20921.branch-2.0.001.patch,
> HBASE-20921.branch-2.0.002.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559591#comment-16559591
]
Hudson commented on HBASE-20921:
Results for branch branch-2
[build #1032 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1032/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1032//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1032//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1032//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-20921.branch-2.0.001.patch,
> HBASE-20921.branch-2.0.002.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559574#comment-16559574
]
Hudson commented on HBASE-20921:
Results for branch branch-2.1
[build #110 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/110/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/110//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/110//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/110//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-20921.branch-2.0.001.patch,
> HBASE-20921.branch-2.0.002.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559572#comment-16559572
]
Hudson commented on HBASE-20921:
Results for branch branch-2.0
[build #598 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/598/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/598//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/598//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/598//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-20921.branch-2.0.001.patch,
> HBASE-20921.branch-2.0.002.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556899#comment-16556899
]
Hadoop QA commented on HBASE-20921:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
52s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
52s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 3s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}153m
5s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}190m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-20921 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12933126/HBASE-20921.branch-2.0.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 414a8905dfa7 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14
08:53:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / c11f0e47a6 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13800/testReport/ |
| Max. process+thread count | 3989 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13800/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automa
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555029#comment-16555029
]
Duo Zhang commented on HBASE-20921:
---
+1.
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-20921.branch-2.0.001.patch,
> HBASE-20921.branch-2.0.002.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553312#comment-16553312
]
stack commented on HBASE-20921:
---
Fix checkstyle and address [~Apache9] request for commentary on commit?
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-20921.branch-2.0.001.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553313#comment-16553313
]
stack commented on HBASE-20921:
---
+1 for branch-2.0 too. Thanks.
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-20921.branch-2.0.001.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553311#comment-16553311
]
stack commented on HBASE-20921:
---
LGTM
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-20921.branch-2.0.001.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553023#comment-16553023
]
Hadoop QA commented on HBASE-20921:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
1s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
16s{color} | {color:red} hbase-server: The patch generated 2 new + 0 unchanged
- 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
25s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 59s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}129m
42s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}173m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-20921 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12932697/HBASE-20921.branch-2.0.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ef1f93761eeb 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / a90a20b4ce |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/13741/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13741/testReport/ |
| Max. process+thread count | 3938 (vs. ulimit of 1) |
| modules | C: hbase-server U: h
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552929#comment-16552929
]
Duo Zhang commented on HBASE-20921:
---
Do not forget to add some comments about why it could be null.
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-20921.branch-2.0.001.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552912#comment-16552912
]
Allan Yang commented on HBASE-20921:
My patch filter out the recorded regions which are not exist, null check is
more simple, will change the patch
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-20921.branch-2.0.001.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552867#comment-16552867
]
Duo Zhang commented on HBASE-20921:
---
The problem of your patch is that, we do not need to check all the regions of
the table, we only need to check the ones we recorded before. In your case, the
problem is that the region has been removed due to merge or split, so the
RegionStateNode will null and cause NPE. We could just add a null check there,
if RegionStateNode is null then we can make sure that we do not need to deal
with it any more(just as you said in the description).
Thanks.
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-20921.branch-2.0.001.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552845#comment-16552845
]
Duo Zhang commented on HBASE-20921:
---
Just add a null check in checkReopened?
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-20921.branch-2.0.001.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can be sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20921) Possible NPE in ReopenTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552799#comment-16552799
]
Allan Yang commented on HBASE-20921:
A patch is loaded with a test case to reproduce this issue.
> Possible NPE in ReopenTableRegionsProcedure
> ---
>
> Key: HBASE-20921
> URL: https://issues.apache.org/jira/browse/HBASE-20921
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 3.0.0, 2.1.0, 2.0.2
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-20921.branch-2.0.001.patch
>
>
> After HBASE-20752， we issue a ReopenTableRegionsProcedure in
> ModifyTableProcedure to ensure all regions are reopened.
> But, ModifyTableProcedure and ReopenTableRegionsProcedure do not hold the
> lock (why?), so there is a chance that while ModifyTableProcedure executing,
> a merge/split procedure can be executed at the same time.
> So, when ReopenTableRegionsProcedure reaches the state of
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED", some of the persisted regions to
> check is actually not exists, thus a NPE will throw.
> {code}
> 2018-07-18 01:38:57,528 INFO [PEWorker-9]
> procedure2.ProcedureExecutor(1246): Finished pid=6110, state=SUCCESS;
> MergeTableRegionsProcedure table=IntegrationTestBigLinkedList,
> regions=[845d286231eb01b7
> 1aeaa17b0e30058d, 4a46ab0918c99cada72d5336ad83a828], forcibly=false in
> 10.8610sec
> 2018-07-18 01:38:57,530 ERROR [PEWorker-8]
> procedure2.ProcedureExecutor(1478): CODE-BUG: Uncaught runtime exception:
> pid=5974, ppid=5973, state=RUNNABLE:REOPEN_TABLE_REGIONS_CONFIRM_REOPENED;
> ReopenTab
> leRegionsProcedure table=IntegrationTestBigLinkedList
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.assignment.RegionStates.checkReopened(RegionStates.java:651)
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.ReopenTableRegionsProcedure.executeFromState(ReopenTableRegionsProcedure.java:45)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:184)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:850)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> I think we need to renew the region list of the table at the
> "REOPEN_TABLE_REGIONS_CONFIRM_REOPENED" state. For the regions which are
> merged or split, we do not need to check it. Since we can make sure that they
> are opened after we made change to table descriptor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
