[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HDFS-15368: -- Component/s: balancer test Target Version/s: 3.3.6, 3.4.0 Affects Version/s: 3.3.6 3.4.0 > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement > Components: balancer, test >Affects Versions: 3.4.0, 3.3.6 >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Fix For: 3.4.0, 3.3.6 > > Attachments: HDFS-15368.001.patch, HDFS-15368.002.patch, > TestBalancerWithHANameNodes.testBalancerObserver.log, > TestBalancerWithHANameNodes.testBalancerObserver.log > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-15368: --- Fix Version/s: 3.4.0 > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Fix For: 3.4.0, 3.3.6 > > Attachments: HDFS-15368.001.patch, HDFS-15368.002.patch, > TestBalancerWithHANameNodes.testBalancerObserver.log, > TestBalancerWithHANameNodes.testBalancerObserver.log > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-15368: --- Fix Version/s: 3.3.6 (was: 3.4.0) > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Fix For: 3.3.6 > > Attachments: HDFS-15368.001.patch, HDFS-15368.002.patch, > TestBalancerWithHANameNodes.testBalancerObserver.log, > TestBalancerWithHANameNodes.testBalancerObserver.log > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-15368: --- Fix Version/s: (was: 3.3.6) > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Fix For: 3.4.0 > > Attachments: HDFS-15368.001.patch, HDFS-15368.002.patch, > TestBalancerWithHANameNodes.testBalancerObserver.log, > TestBalancerWithHANameNodes.testBalancerObserver.log > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15368: Fix Version/s: 3.3.9 > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Fix For: 3.4.0, 3.3.9 > > Attachments: HDFS-15368.001.patch, HDFS-15368.002.patch, > TestBalancerWithHANameNodes.testBalancerObserver.log, > TestBalancerWithHANameNodes.testBalancerObserver.log > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15368: Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Fix For: 3.4.0 > > Attachments: HDFS-15368.001.patch, HDFS-15368.002.patch, > TestBalancerWithHANameNodes.testBalancerObserver.log, > TestBalancerWithHANameNodes.testBalancerObserver.log > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-15368: --- Attachment: HDFS-15368.002.patch > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Attachments: HDFS-15368.001.patch, HDFS-15368.002.patch, > TestBalancerWithHANameNodes.testBalancerObserver.log, > TestBalancerWithHANameNodes.testBalancerObserver.log > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15368: Attachment: TestBalancerWithHANameNodes.testBalancerObserver.log > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Attachments: HDFS-15368.001.patch, > TestBalancerWithHANameNodes.testBalancerObserver.log, > TestBalancerWithHANameNodes.testBalancerObserver.log > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-15368: --- Attachment: TestBalancerWithHANameNodes.testBalancerObserver.log > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Attachments: HDFS-15368.001.patch, > TestBalancerWithHANameNodes.testBalancerObserver.log > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15368) TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally
[ https://issues.apache.org/jira/browse/HDFS-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-15368: --- Attachment: HDFS-15368.001.patch Status: Patch Available (was: Open) submit demo patch and try to trigger yetus. > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally > > > Key: HDFS-15368 > URL: https://issues.apache.org/jira/browse/HDFS-15368 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Labels: balancer, test > Attachments: HDFS-15368.001.patch > > > When I am working on HDFS-13183, I found that > TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally, > because the following code segment. Consider there are 1 ANN + 1 SBN + 2ONN, > when invoke getBlocks with opening Observer Read feature, it could request > any one of two ObserverNN based on my observation. So only verify the first > ObserverNN and check times of invoke #getBlocks is not expected. > {code:java} > for (int i = 0; i < cluster.getNumNameNodes(); i++) { > // First observer node is at idx 2, or 3 if 2 has been shut down > // It should get both getBlocks calls, all other NNs should see 0 > calls > int expectedObserverIdx = withObserverFailure ? 3 : 2; > int expectedCount = (i == expectedObserverIdx) ? 2 : 0; > verify(namesystemSpies.get(i), times(expectedCount)) > .getBlocks(any(), anyLong(), anyLong()); > } > {code} > cc [~xkrogen],[~weichiu]. I am not very familiar for Observer Read feature, > would you like give some suggestions? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org