[jira] [Commented] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[ https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832064#comment-17832064 ] ASF GitHub Bot commented on HDFS-17446: --- hadoop-yetus commented on PR #6685: URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2026776792 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 58s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 44m 38s | | trunk passed | | +1 :green_heart: | compile | 1m 26s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 1m 19s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 29s | | trunk passed | | +1 :green_heart: | javadoc | 1m 7s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 39s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 24s | | trunk passed | | +1 :green_heart: | shadedclient | 36m 7s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 16s | | the patch passed | | +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 1m 19s | | the patch passed | | +1 :green_heart: | compile | 1m 11s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 1m 11s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 5s | | the patch passed | | +1 :green_heart: | mvnsite | 1m 20s | | the patch passed | | +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | -1 :x: | spotbugs | 3m 28s | [/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/4/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html) | hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | shadedclient | 36m 3s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 302m 23s | | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 46s | | The patch does not generate ASF License warnings. | | | | 445m 49s | | | | Reason | Tests | |---:|:--| | SpotBugs | module:hadoop-hdfs-project/hadoop-hdfs | | | Write to static field org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.scanBlockCnt from instance method org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addToReplicasMap(ReplicaMap, File, RamDiskReplicaTracker, boolean, List, Queue) At BlockPoolSlice.java:from instance method org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.addToReplicasMap(ReplicaMap, File, RamDiskReplicaTracker, boolean, List, Queue) At BlockPoolSlice.java:[line 752] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6685 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 206c3bf56969 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk
[jira] [Commented] (HDFS-17447) Add logging to print the implementation class of AccessControlEnforcer, print ACEnforcer when AccessControlException occurs.
[ https://issues.apache.org/jira/browse/HDFS-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832070#comment-17832070 ] ASF GitHub Bot commented on HDFS-17447: --- hadoop-yetus commented on PR #6687: URL: https://github.com/apache/hadoop/pull/6687#issuecomment-2026804668 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 11m 41s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 44m 9s | | trunk passed | | +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 1m 12s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 24s | | trunk passed | | +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 17s | | trunk passed | | +1 :green_heart: | shadedclient | 35m 24s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 10s | | the patch passed | | +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 1m 15s | | the patch passed | | +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 1m 8s | | the patch passed | | -1 :x: | blanks | 0m 0s | [/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6687/2/artifact/out/blanks-eol.txt) | The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | -0 :warning: | checkstyle | 0m 58s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6687/2/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 125 unchanged - 1 fixed = 127 total (was 126) | | +1 :green_heart: | mvnsite | 1m 15s | | the patch passed | | +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 34s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 13s | | the patch passed | | +1 :green_heart: | shadedclient | 35m 32s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 230m 18s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6687/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 44s | | The patch does not generate ASF License warnings. | | | | 381m 44s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.namenode.TestFSPermissionChecker | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6687/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6687 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 06e216a7ee29 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / f0bdbd132b6f51a153095cd971c1026bce9d59b2 | | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~
[jira] [Created] (HDFS-17448) Enhance the stability of the unit test TestDiskBalancerCommand
Haiyang Hu created HDFS-17448: - Summary: Enhance the stability of the unit test TestDiskBalancerCommand Key: HDFS-17448 URL: https://issues.apache.org/jira/browse/HDFS-17448 Project: Hadoop HDFS Issue Type: Bug Reporter: Haiyang Hu Assignee: Haiyang Hu TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes frequently fails tests, such as: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17448) Enhance the stability of the unit test TestDiskBalancerCommand
[ https://issues.apache.org/jira/browse/HDFS-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17448: -- Description: TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes frequently fails tests, such as: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ i will fix it was: TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes frequently fails tests, such as: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > Enhance the stability of the unit test TestDiskBalancerCommand > > > Key: HDFS-17448 > URL: https://issues.apache.org/jira/browse/HDFS-17448 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > > TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes > frequently fails tests, such as: > https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > i will fix it -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17448) Enhance the stability of the unit test TestDiskBalancerCommand
[ https://issues.apache.org/jira/browse/HDFS-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17448: -- Description: TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes frequently fails tests, such as: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ I will fix it enhance the stability of the unit test. was: TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes frequently fails tests, such as: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ i will fix it > Enhance the stability of the unit test TestDiskBalancerCommand > > > Key: HDFS-17448 > URL: https://issues.apache.org/jira/browse/HDFS-17448 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > > TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes > frequently fails tests, such as: > https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > I will fix it enhance the stability of the unit test. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17448) Enhance the stability of the unit test TestDiskBalancerCommand
[ https://issues.apache.org/jira/browse/HDFS-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-17448: -- Labels: pull-request-available (was: ) > Enhance the stability of the unit test TestDiskBalancerCommand > > > Key: HDFS-17448 > URL: https://issues.apache.org/jira/browse/HDFS-17448 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > Labels: pull-request-available > > TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes > frequently fails tests, such as: > https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > I will fix it enhance the stability of the unit test. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17448) Enhance the stability of the unit test TestDiskBalancerCommand
[ https://issues.apache.org/jira/browse/HDFS-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832101#comment-17832101 ] ASF GitHub Bot commented on HDFS-17448: --- haiyang1987 opened a new pull request, #6690: URL: https://github.com/apache/hadoop/pull/6690 ### Description of PR https://issues.apache.org/jira/browse/HDFS-17448 TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes frequently fails tests, such as: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ I will fix it enhance the stability of the unit test. > Enhance the stability of the unit test TestDiskBalancerCommand > > > Key: HDFS-17448 > URL: https://issues.apache.org/jira/browse/HDFS-17448 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > > TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes > frequently fails tests, such as: > https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > I will fix it enhance the stability of the unit test. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17352) Add configuration to control whether DN delete this replica from disk when client requests a missing block
[ https://issues.apache.org/jira/browse/HDFS-17352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832104#comment-17832104 ] ASF GitHub Bot commented on HDFS-17352: --- haiyang1987 commented on PR #6559: URL: https://github.com/apache/hadoop/pull/6559#issuecomment-2026904879 Hi @zhangshuyan0 please help me review again this PR when you are free, thanks ~ > Add configuration to control whether DN delete this replica from disk when > client requests a missing block > --- > > Key: HDFS-17352 > URL: https://issues.apache.org/jira/browse/HDFS-17352 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > Labels: pull-request-available > > As discussed at > https://github.com/apache/hadoop/pull/6464#issuecomment-1902959898 > we should add configuration to control whether DN delete this replica from > disk when client requests a missing block. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17361) DiskBalancer: Query command support with multiple nodes
[ https://issues.apache.org/jira/browse/HDFS-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832116#comment-17832116 ] Haiyang Hu commented on HDFS-17361: --- {quote} Hi Haiyang Hu The introduced test seems to be flaky https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ Can you give a check once {quote} Hi [~ayushtkn] I will fix it by https://github.com/apache/hadoop/pull/6690 > DiskBalancer: Query command support with multiple nodes > --- > > Key: HDFS-17361 > URL: https://issues.apache.org/jira/browse/HDFS-17361 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, diskbalancer >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0 > > > For: https://issues.apache.org/jira/browse/HDFS-10821 mentioned, Query > command will support with multiple nodes. > That means we can use command hdfs diskbalancer -query to print one or one > more datanodes status of the diskbalancer. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17388) [FGL] Client RPCs involving write process supports fine-grained lock
[ https://issues.apache.org/jira/browse/HDFS-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832139#comment-17832139 ] ASF GitHub Bot commented on HDFS-17388: --- ferhui commented on PR #6589: URL: https://github.com/apache/hadoop/pull/6589#issuecomment-2027025177 Thanks for contribution. Merged. > [FGL] Client RPCs involving write process supports fine-grained lock > > > Key: HDFS-17388 > URL: https://issues.apache.org/jira/browse/HDFS-17388 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The client write process involves many client RPCs. > > This ticket is used to make these RPCs support fine-grained lock. > * mkdir > * create > * addBlock > * abandonBlock > * getAdditionalDatanode > * updateBlockForPipeline > * updatePipeline > * fsync > * complete > * rename > * rename2 > * append > * renewLease > * recoverLease > * delete > * createSymlink > * renewDelegationToken > * cancelDelegationToken -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17388) [FGL] Client RPCs involving write process supports fine-grained lock
[ https://issues.apache.org/jira/browse/HDFS-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832138#comment-17832138 ] ASF GitHub Bot commented on HDFS-17388: --- ferhui merged PR #6589: URL: https://github.com/apache/hadoop/pull/6589 > [FGL] Client RPCs involving write process supports fine-grained lock > > > Key: HDFS-17388 > URL: https://issues.apache.org/jira/browse/HDFS-17388 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The client write process involves many client RPCs. > > This ticket is used to make these RPCs support fine-grained lock. > * mkdir > * create > * addBlock > * abandonBlock > * getAdditionalDatanode > * updateBlockForPipeline > * updatePipeline > * fsync > * complete > * rename > * rename2 > * append > * renewLease > * recoverLease > * delete > * createSymlink > * renewDelegationToken > * cancelDelegationToken -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-17388) [FGL] Client RPCs involving write process supports fine-grained lock
[ https://issues.apache.org/jira/browse/HDFS-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-17388. Resolution: Fixed > [FGL] Client RPCs involving write process supports fine-grained lock > > > Key: HDFS-17388 > URL: https://issues.apache.org/jira/browse/HDFS-17388 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The client write process involves many client RPCs. > > This ticket is used to make these RPCs support fine-grained lock. > * mkdir > * create > * addBlock > * abandonBlock > * getAdditionalDatanode > * updateBlockForPipeline > * updatePipeline > * fsync > * complete > * rename > * rename2 > * append > * renewLease > * recoverLease > * delete > * createSymlink > * renewDelegationToken > * cancelDelegationToken -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17389) [FGL] Client RPCs involving read process supports fine-grained lock
[ https://issues.apache.org/jira/browse/HDFS-17389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832144#comment-17832144 ] ASF GitHub Bot commented on HDFS-17389: --- ferhui merged PR #6590: URL: https://github.com/apache/hadoop/pull/6590 > [FGL] Client RPCs involving read process supports fine-grained lock > --- > > Key: HDFS-17389 > URL: https://issues.apache.org/jira/browse/HDFS-17389 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The client read process involves many client RPCs. > > This ticket is used to make these RPCs support fine-grained lock. > * getListing > * getBatchedListing > * listOpenFiles > * getFileInfo > * isFileClosed > * getBlockLocations > * reportBadBlocks > * getServerDefaults > * getStats > * getReplicatedBlockStats > * getECBlockGroupStats > * getPreferredBlockSize > * listCorruptFileBlocks > * getContentSummary > * getLocatedFileInfo > * createEncryptionZone > * msync > * checkAccess > * getFileLinkInfo > * getLinkTarget -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-17389) [FGL] Client RPCs involving read process supports fine-grained lock
[ https://issues.apache.org/jira/browse/HDFS-17389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Fei resolved HDFS-17389. Resolution: Fixed > [FGL] Client RPCs involving read process supports fine-grained lock > --- > > Key: HDFS-17389 > URL: https://issues.apache.org/jira/browse/HDFS-17389 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The client read process involves many client RPCs. > > This ticket is used to make these RPCs support fine-grained lock. > * getListing > * getBatchedListing > * listOpenFiles > * getFileInfo > * isFileClosed > * getBlockLocations > * reportBadBlocks > * getServerDefaults > * getStats > * getReplicatedBlockStats > * getECBlockGroupStats > * getPreferredBlockSize > * listCorruptFileBlocks > * getContentSummary > * getLocatedFileInfo > * createEncryptionZone > * msync > * checkAccess > * getFileLinkInfo > * getLinkTarget -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17389) [FGL] Client RPCs involving read process supports fine-grained lock
[ https://issues.apache.org/jira/browse/HDFS-17389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832145#comment-17832145 ] ASF GitHub Bot commented on HDFS-17389: --- ferhui commented on PR #6590: URL: https://github.com/apache/hadoop/pull/6590#issuecomment-2027027422 Thanks for contribution. Merged. > [FGL] Client RPCs involving read process supports fine-grained lock > --- > > Key: HDFS-17389 > URL: https://issues.apache.org/jira/browse/HDFS-17389 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The client read process involves many client RPCs. > > This ticket is used to make these RPCs support fine-grained lock. > * getListing > * getBatchedListing > * listOpenFiles > * getFileInfo > * isFileClosed > * getBlockLocations > * reportBadBlocks > * getServerDefaults > * getStats > * getReplicatedBlockStats > * getECBlockGroupStats > * getPreferredBlockSize > * listCorruptFileBlocks > * getContentSummary > * getLocatedFileInfo > * createEncryptionZone > * msync > * checkAccess > * getFileLinkInfo > * getLinkTarget -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17438) RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority.
[ https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832172#comment-17832172 ] ASF GitHub Bot commented on HDFS-17438: --- hadoop-yetus commented on PR #6655: URL: https://github.com/apache/hadoop/pull/6655#issuecomment-2027105445 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 53s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 47m 14s | | trunk passed | | +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 0m 30s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 44s | | trunk passed | | +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | -1 :x: | spotbugs | 1m 33s | [/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/5/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html) | hadoop-hdfs-project/hadoop-hdfs-rbf in trunk has 1 extant spotbugs warnings. | | +1 :green_heart: | shadedclient | 42m 0s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 42m 21s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 35s | | the patch passed | | +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 0m 38s | | the patch passed | | +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 0m 31s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 20s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 36s | | the patch passed | | +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 1m 32s | | the patch passed | | +1 :green_heart: | shadedclient | 42m 30s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 33m 34s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/5/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 39s | | The patch does not generate ASF License warnings. | | | | 182m 49s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6655 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 94002d9493f5 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / bb390f7381d100154de0afe39ad3639bcfd04a0c | | Default Java | Private Build-1.8.0_402-8u402
[jira] [Commented] (HDFS-17444) Add getJournalSyncerStatus jmx metrics, to Indicates whether the JournalSyncer thread for the every namespace has enter working state.
[ https://issues.apache.org/jira/browse/HDFS-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832173#comment-17832173 ] ASF GitHub Bot commented on HDFS-17444: --- hadoop-yetus commented on PR #6684: URL: https://github.com/apache/hadoop/pull/6684#issuecomment-2027112012 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 20s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 32m 20s | | trunk passed | | +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 0m 38s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 44s | | trunk passed | | +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 1m 44s | | trunk passed | | +1 :green_heart: | shadedclient | 22m 3s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 38s | | the patch passed | | +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 0m 39s | | the patch passed | | +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 0m 33s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 30s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 36s | | the patch passed | | +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 3s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 1m 50s | | the patch passed | | +1 :green_heart: | shadedclient | 21m 3s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 198m 15s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6684/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 32s | | The patch does not generate ASF License warnings. | | | | 288m 30s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.tools.TestDFSAdmin | | | hadoop.hdfs.qjournal.server.TestJournalNodeMXBean | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6684/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6684 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 4e362e05e8fa 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 62d31082abdf79c3b1c103176873170a0f51f069 | | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6684/6/testReport/ | | Max. process+thread count | 4393 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-
[jira] [Updated] (HDFS-17444) Add getJournalSyncerStatus jmx metrics, to Indicates whether the JournalSyncer thread for the every namespace has enter working state.
[ https://issues.apache.org/jira/browse/HDFS-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-17444: -- Labels: pull-request-available (was: ) > Add getJournalSyncerStatus jmx metrics, to Indicates whether the > JournalSyncer thread for the every namespace has enter working state. > -- > > Key: HDFS-17444 > URL: https://issues.apache.org/jira/browse/HDFS-17444 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.3.4, 3.3.6 >Reporter: xiaojunxiang >Assignee: xiaojunxiang >Priority: Major > Labels: pull-request-available > Attachments: example-1.png > > > The JornalNode JVM process is not immediately in a normal state until the > JournalSyncer thread is started. > For some management platforms such as Ambari rolling restart JournalNode, > we need a jmx metric to determine whether the JournalSyncer thread is started > and enter working state for current namespace before restarting the next > JournalNode. Otherwise, restart too quickly and more than half of > JournalNodes will be out of order, causing the NameNode to die. > When i add it , the effect is as follows: >!example-1.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[ https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832208#comment-17832208 ] ASF GitHub Bot commented on HDFS-17446: --- hadoop-yetus commented on PR #6685: URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2027270341 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 50s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 50m 18s | | trunk passed | | +1 :green_heart: | compile | 1m 26s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 1m 11s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 22s | | trunk passed | | +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 39s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 19s | | trunk passed | | +1 :green_heart: | shadedclient | 41m 24s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 14s | | the patch passed | | +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 1m 18s | | the patch passed | | +1 :green_heart: | compile | 1m 9s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 1m 9s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 4s | | the patch passed | | +1 :green_heart: | mvnsite | 1m 17s | | the patch passed | | +1 :green_heart: | javadoc | 0m 55s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 20s | | the patch passed | | +1 :green_heart: | shadedclient | 38m 31s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 266m 0s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 47s | | The patch does not generate ASF License warnings. | | | | 422m 8s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.TestRollingUpgrade | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6685 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux d867bb0178f9 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 3e8161b6801f13a8abf8791a47b6b04901462f0c | | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/6/testReport/ | | Max. process+thread count | 2680 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://ci-
[jira] [Commented] (HDFS-17438) RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority.
[ https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832217#comment-17832217 ] ASF GitHub Bot commented on HDFS-17438: --- KeeProMise opened a new pull request, #6655: URL: https://github.com/apache/hadoop/pull/6655 ### Description of PR ### How was this patch tested? ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority. > - > > Key: HDFS-17438 > URL: https://issues.apache.org/jira/browse/HDFS-17438 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Jian Zhang >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17448) Enhance the stability of the unit test TestDiskBalancerCommand
[ https://issues.apache.org/jira/browse/HDFS-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832234#comment-17832234 ] ASF GitHub Bot commented on HDFS-17448: --- ayushtkn commented on code in PR #6690: URL: https://github.com/apache/hadoop/pull/6690#discussion_r1544623497 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/diskbalancer/command/TestDiskBalancerCommand.java: ## @@ -834,8 +834,10 @@ public void testDiskBalancerQueryWithoutSubmitAndMultipleNodes() throws Exceptio .getIpcPort(), dataNode2.getIpcPort()); final String cmdLine = String.format("hdfs diskbalancer %s", queryArg); List outputs = runCommand(cmdLine); - assertThat(outputs.get(1), containsString("localhost:" + dataNode1.getIpcPort())); - assertThat(outputs.get(6), containsString("localhost:" + dataNode2.getIpcPort())); + assertTrue(outputs.get(1).contains("localhost:" + dataNode1.getIpcPort()) + || outputs.get(6).contains("localhost:" + dataNode1.getIpcPort())); + assertTrue(outputs.get(1).contains("localhost:" + dataNode2.getIpcPort()) + || outputs.get(6).contains("localhost:" + dataNode2.getIpcPort())); Review Comment: I think we should print the entire outputs in the assertTrue if it fails for future, so we know what was in the outputs if any of these assertions fail. assertTrue can take one more argument for error message, we can put it there > Enhance the stability of the unit test TestDiskBalancerCommand > > > Key: HDFS-17448 > URL: https://issues.apache.org/jira/browse/HDFS-17448 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Haiyang Hu >Assignee: Haiyang Hu >Priority: Major > Labels: pull-request-available > > TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes > frequently fails tests, such as: > https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/ > I will fix it enhance the stability of the unit test. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17448) Enhance the stability of the unit test TestDiskBalancerCommand
[ https://issues.apache.org/jira/browse/HDFS-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832256#comment-17832256 ] ASF GitHub Bot commented on HDFS-17448: --- hadoop-yetus commented on PR #6690: URL: https://github.com/apache/hadoop/pull/6690#issuecomment-2027467936 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 19m 22s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 49m 36s | | trunk passed | | +1 :green_heart: | compile | 1m 34s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 1m 16s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 33s | | trunk passed | | +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 44s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 23s | | trunk passed | | +1 :green_heart: | shadedclient | 43m 51s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 13s | | the patch passed | | +1 :green_heart: | compile | 1m 38s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 1m 38s | | the patch passed | | +1 :green_heart: | compile | 1m 27s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 1m 27s | | the patch passed | | +1 :green_heart: | blanks | 0m 2s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 8s | | the patch passed | | +1 :green_heart: | mvnsite | 1m 28s | | the patch passed | | +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 49s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 27s | | the patch passed | | +1 :green_heart: | shadedclient | 42m 16s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 280m 29s | | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 50s | | The patch does not generate ASF License warnings. | | | | 463m 7s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6690/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6690 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 9ab4358abff5 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 9d426f04dfb12d4fbac151c8bae3a7b99e4b8360 | | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6690/1/testReport/ | | Max. process+thread count | 3001 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6690/1/console | | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org | This message was automatically generated. > Enhance the stability o
[jira] [Commented] (HDFS-17438) RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority.
[ https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832267#comment-17832267 ] ASF GitHub Bot commented on HDFS-17438: --- hadoop-yetus commented on PR #6655: URL: https://github.com/apache/hadoop/pull/6655#issuecomment-2027535559 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 1m 0s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 49m 54s | | trunk passed | | +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 0m 29s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 40s | | trunk passed | | +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | -1 :x: | spotbugs | 1m 20s | [/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/6/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html) | hadoop-hdfs-project/hadoop-hdfs-rbf in trunk has 1 extant spotbugs warnings. | | +1 :green_heart: | shadedclient | 39m 14s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 39m 34s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 30s | | the patch passed | | +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 0m 33s | | the patch passed | | +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 0m 28s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 18s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 31s | | the patch passed | | +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 1m 22s | | hadoop-hdfs-project/hadoop-hdfs-rbf generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | +1 :green_heart: | shadedclient | 39m 52s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 31m 54s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 35s | | The patch does not generate ASF License warnings. | | | | 178m 3s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.federation.security.token.TestZKDelegationTokenSecretManagerImpl | | | hadoop.hdfs.server.federation.router.TestRouterClientRejectOverload | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6655 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 18320ad99ff6 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool |
[jira] [Created] (HDFS-17449) Ill-formed decommission host name and port pair would trigger IndexOutOfBound error
ConfX created HDFS-17449: Summary: Ill-formed decommission host name and port pair would trigger IndexOutOfBound error Key: HDFS-17449 URL: https://issues.apache.org/jira/browse/HDFS-17449 Project: Hadoop HDFS Issue Type: Bug Reporter: ConfX h2. What happened: Got IndexOutOfBound when trying to run org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor#testDecommissionStatusAfterDNRestart with namenode host provider set to org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager. h2. Buggy code: In HostsFileWriter.java: {code:java} String[] hostAndPort = hostNameAndPort.split(":"); // hostNameAndPort might be invalid dn.setHostName(hostAndPort[0]); dn.setPort(Integer.parseInt(hostAndPort[1])); // here IndexOutOfBound might be thrown dn.setAdminState(AdminStates.DECOMMISSIONED);{code} h2. StackTrace: {code:java} java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 at org.apache.hadoop.hdfs.util.HostsFileWriter.initOutOfServiceHosts(HostsFileWriter.java:110){code} h2. How to reproduce: (1) Set {{dfs.namenode.hosts.provider.classname}} to {{org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager}} (2) Run test: {{org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor#testDecommissionStatusAfterDNRestart}} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17449) Ill-formed decommission host name and port pair would trigger IndexOutOfBound error
[ https://issues.apache.org/jira/browse/HDFS-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832283#comment-17832283 ] ASF GitHub Bot commented on HDFS-17449: --- teamconfx opened a new pull request, #6691: URL: https://github.com/apache/hadoop/pull/6691 ### Description of PR This PR catches the IndexOutOfBounds encountered during parsing of decommission host name and port string, and throws a more informative IllegalArgument instead. ### How was this patch tested? Running org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor#testDecommissionStatusAfterDNRestart with namenode host provider set to org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager now throws IllegalArgument. ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Ill-formed decommission host name and port pair would trigger IndexOutOfBound > error > --- > > Key: HDFS-17449 > URL: https://issues.apache.org/jira/browse/HDFS-17449 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: ConfX >Priority: Major > > h2. What happened: > Got IndexOutOfBound when trying to run > org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor#testDecommissionStatusAfterDNRestart > with namenode host provider set to > org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager. > h2. Buggy code: > In HostsFileWriter.java: > {code:java} > String[] hostAndPort = hostNameAndPort.split(":"); // hostNameAndPort might > be invalid > dn.setHostName(hostAndPort[0]); > dn.setPort(Integer.parseInt(hostAndPort[1])); // here IndexOutOfBound might > be thrown > dn.setAdminState(AdminStates.DECOMMISSIONED);{code} > h2. StackTrace: > {code:java} > java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 > at > org.apache.hadoop.hdfs.util.HostsFileWriter.initOutOfServiceHosts(HostsFileWriter.java:110){code} > h2. How to reproduce: > (1) Set {{dfs.namenode.hosts.provider.classname}} to > {{org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager}} > (2) Run test: > {{org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor#testDecommissionStatusAfterDNRestart}} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17449) Ill-formed decommission host name and port pair would trigger IndexOutOfBound error
[ https://issues.apache.org/jira/browse/HDFS-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-17449: -- Labels: pull-request-available (was: ) > Ill-formed decommission host name and port pair would trigger IndexOutOfBound > error > --- > > Key: HDFS-17449 > URL: https://issues.apache.org/jira/browse/HDFS-17449 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: ConfX >Priority: Major > Labels: pull-request-available > > h2. What happened: > Got IndexOutOfBound when trying to run > org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor#testDecommissionStatusAfterDNRestart > with namenode host provider set to > org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager. > h2. Buggy code: > In HostsFileWriter.java: > {code:java} > String[] hostAndPort = hostNameAndPort.split(":"); // hostNameAndPort might > be invalid > dn.setHostName(hostAndPort[0]); > dn.setPort(Integer.parseInt(hostAndPort[1])); // here IndexOutOfBound might > be thrown > dn.setAdminState(AdminStates.DECOMMISSIONED);{code} > h2. StackTrace: > {code:java} > java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 > at > org.apache.hadoop.hdfs.util.HostsFileWriter.initOutOfServiceHosts(HostsFileWriter.java:110){code} > h2. How to reproduce: > (1) Set {{dfs.namenode.hosts.provider.classname}} to > {{org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager}} > (2) Run test: > {{org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor#testDecommissionStatusAfterDNRestart}} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17449) Ill-formed decommission host name and port pair would trigger IndexOutOfBound error
[ https://issues.apache.org/jira/browse/HDFS-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832288#comment-17832288 ] ASF GitHub Bot commented on HDFS-17449: --- ayushtkn commented on code in PR #6691: URL: https://github.com/apache/hadoop/pull/6691#discussion_r1544792799 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/HostsFileWriter.java: ## @@ -106,9 +106,14 @@ public void initOutOfServiceHosts(List decommissionHostNameAndPorts, for (String hostNameAndPort : decommissionHostNameAndPorts) { DatanodeAdminProperties dn = new DatanodeAdminProperties(); String[] hostAndPort = hostNameAndPort.split(":"); - dn.setHostName(hostAndPort[0]); - dn.setPort(Integer.parseInt(hostAndPort[1])); - dn.setAdminState(AdminStates.DECOMMISSIONED); + try { +dn.setHostName(hostAndPort[0]); +dn.setPort(Integer.parseInt(hostAndPort[1])); +dn.setAdminState(AdminStates.DECOMMISSIONED); + } catch (Exception e) { +throw new IllegalArgumentException("The decommision host name and port format is " ++ "invalid. The format should be in :, not " + hostNameAndPort, e); + } Review Comment: it would be better if we do a check above for the size of ``hostAndPort`` and throw exception above only, rather than doing this catch stuff, it will anyway catch all the exceptions not just IOBE > Ill-formed decommission host name and port pair would trigger IndexOutOfBound > error > --- > > Key: HDFS-17449 > URL: https://issues.apache.org/jira/browse/HDFS-17449 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: ConfX >Priority: Major > Labels: pull-request-available > > h2. What happened: > Got IndexOutOfBound when trying to run > org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor#testDecommissionStatusAfterDNRestart > with namenode host provider set to > org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager. > h2. Buggy code: > In HostsFileWriter.java: > {code:java} > String[] hostAndPort = hostNameAndPort.split(":"); // hostNameAndPort might > be invalid > dn.setHostName(hostAndPort[0]); > dn.setPort(Integer.parseInt(hostAndPort[1])); // here IndexOutOfBound might > be thrown > dn.setAdminState(AdminStates.DECOMMISSIONED);{code} > h2. StackTrace: > {code:java} > java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 > at > org.apache.hadoop.hdfs.util.HostsFileWriter.initOutOfServiceHosts(HostsFileWriter.java:110){code} > h2. How to reproduce: > (1) Set {{dfs.namenode.hosts.provider.classname}} to > {{org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager}} > (2) Run test: > {{org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor#testDecommissionStatusAfterDNRestart}} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17446) The DataNode adds a log to report its progress during addToReplicasMap execution.
[ https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832298#comment-17832298 ] ASF GitHub Bot commented on HDFS-17446: --- hadoop-yetus commented on PR #6685: URL: https://github.com/apache/hadoop/pull/6685#issuecomment-2027665491 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 58s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 46m 46s | | trunk passed | | +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 1m 21s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 1m 16s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 32s | | trunk passed | | +1 :green_heart: | javadoc | 1m 11s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 27s | | trunk passed | | +1 :green_heart: | shadedclient | 35m 41s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 13s | | the patch passed | | +1 :green_heart: | compile | 1m 12s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 1m 12s | | the patch passed | | +1 :green_heart: | compile | 1m 6s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 1m 6s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 58s | | the patch passed | | +1 :green_heart: | mvnsite | 1m 16s | | the patch passed | | +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 13s | | the patch passed | | +1 :green_heart: | shadedclient | 35m 52s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 276m 52s | | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 46s | | The patch does not generate ASF License warnings. | | | | 421m 33s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6685 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 911e475653bb 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / f8f1869eb1ff6ad2f36d859067d94cc5a2076a1f | | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/7/testReport/ | | Max. process+thread count | 2905 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6685/7/console | | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org | This message was automatically generated. > The DataNode adds a log
[jira] [Commented] (HDFS-17449) Ill-formed decommission host name and port pair would trigger IndexOutOfBound error
[ https://issues.apache.org/jira/browse/HDFS-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832344#comment-17832344 ] ASF GitHub Bot commented on HDFS-17449: --- hadoop-yetus commented on PR #6691: URL: https://github.com/apache/hadoop/pull/6691#issuecomment-2027864017 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 47s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 50m 49s | | trunk passed | | +1 :green_heart: | compile | 1m 31s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 1m 19s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 1m 14s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 24s | | trunk passed | | +1 :green_heart: | javadoc | 1m 11s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 23s | | trunk passed | | +1 :green_heart: | shadedclient | 41m 32s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 15s | | the patch passed | | +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 1m 19s | | the patch passed | | +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 1m 10s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 3s | | the patch passed | | +1 :green_heart: | mvnsite | 1m 17s | | the patch passed | | +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 27s | | the patch passed | | +1 :green_heart: | shadedclient | 42m 26s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 260m 14s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6691/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 46s | | The patch does not generate ASF License warnings. | | | | 421m 50s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.TestRollingUpgrade | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6691/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6691 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 24eb5c6cde02 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / ddfda5a98795f5ae314d3696cd42e6a1b3d3537f | | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6691/1/testReport/ | | Max. process+thread count | 3050 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://ci-
[jira] [Updated] (HDFS-17438) RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority.
[ https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Zhang updated HDFS-17438: -- Description: At present, when the status of all namenodes in an ns in the router is the same, the namenode which is the newest reported will be placed at the top of the cache. when the client accesses the ns through the router, it will first access the namenode. If multiple namenodes in this route are in an active state, or if there are namenodes with multiple observer states, the existing logic is not a problem, because the newest reported active or observer state namenode have a higher probability of being true active or observer compared to the namenode that reported active or observer state a long time ago. Similarly, the newest reported namenode with a status of standby or unavailable has a higher probability of being a standby or unavailable namenode compared to the namenode reported with a status of standby or unavailable a long time ago. Therefore, the newest nn reported as standby or unavailable status should have a lower priority for access, the oldest nn reported as standby or unavailable status should have a higher priority for access. > RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority. > - > > Key: HDFS-17438 > URL: https://issues.apache.org/jira/browse/HDFS-17438 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Jian Zhang >Assignee: Jian Zhang >Priority: Major > Labels: pull-request-available > > At present, when the status of all namenodes in an ns in the router is the > same, the namenode which is the newest reported will be placed at the top of > the cache. when the client accesses the ns through the router, it will first > access the namenode. > If multiple namenodes in this route are in an active state, or if there are > namenodes with multiple observer states, the existing logic is not a problem, > because the newest reported active or observer state namenode have a higher > probability of being true active or observer compared to the namenode that > reported active or observer state a long time ago. > Similarly, the newest reported namenode with a status of standby or > unavailable has a higher probability of being a standby or unavailable > namenode compared to the namenode reported with a status of standby or > unavailable a long time ago. Therefore, the newest nn reported as standby or > unavailable status should have a lower priority for access, the oldest nn > reported as standby or unavailable status should have a higher priority for > access. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17397) Choose another DN as soon as possible, when encountering network issues
[ https://issues.apache.org/jira/browse/HDFS-17397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832372#comment-17832372 ] ASF GitHub Bot commented on HDFS-17397: --- xleoken commented on PR #6591: URL: https://github.com/apache/hadoop/pull/6591#issuecomment-2027908244 cc @dineshchitlangia > Choose another DN as soon as possible, when encountering network issues > --- > > Key: HDFS-17397 > URL: https://issues.apache.org/jira/browse/HDFS-17397 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: xleoken >Priority: Minor > Labels: pull-request-available > Attachments: hadoop.png > > > Choose another DN as soon as possible, when encountering network issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17438) RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority.
[ https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Zhang updated HDFS-17438: -- Attachment: HDFS-17438.001.patch Status: Patch Available (was: Open) > RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority. > - > > Key: HDFS-17438 > URL: https://issues.apache.org/jira/browse/HDFS-17438 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Jian Zhang >Assignee: Jian Zhang >Priority: Major > Labels: pull-request-available > Attachments: HDFS-17438.001.patch > > > At present, when the status of all namenodes in an ns in the router is the > same, the namenode which is the newest reported will be placed at the top of > the cache. when the client accesses the ns through the router, it will first > access the namenode. > If multiple namenodes in this route are in an active state, or if there are > namenodes with multiple observer states, the existing logic is not a problem, > because the newest reported active or observer state namenode have a higher > probability of being true active or observer compared to the namenode that > reported active or observer state a long time ago. > Similarly, the newest reported namenode with a status of standby or > unavailable has a higher probability of being a standby or unavailable > namenode compared to the namenode reported with a status of standby or > unavailable a long time ago. Therefore, the newest nn reported as standby or > unavailable status should have a lower priority for access, the oldest nn > reported as standby or unavailable status should have a higher priority for > access. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-17438) RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority.
[ https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Zhang updated HDFS-17438: -- Summary: RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority. (was: RBF: The latest STANDBY and UNAVAILABLE nn should be the lowest priority.) > RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority. > - > > Key: HDFS-17438 > URL: https://issues.apache.org/jira/browse/HDFS-17438 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Jian Zhang >Assignee: Jian Zhang >Priority: Major > Labels: pull-request-available > Attachments: HDFS-17438.001.patch > > > At present, when the status of all namenodes in an ns in the router is the > same, the namenode which is the newest reported will be placed at the top of > the cache. when the client accesses the ns through the router, it will first > access the namenode. > If multiple namenodes in this route are in an active state, or if there are > namenodes with multiple observer states, the existing logic is not a problem, > because the newest reported active or observer state namenode have a higher > probability of being true active or observer compared to the namenode that > reported active or observer state a long time ago. > Similarly, the newest reported namenode with a status of standby or > unavailable has a higher probability of being a standby or unavailable > namenode compared to the namenode reported with a status of standby or > unavailable a long time ago. Therefore, the newest nn reported as standby or > unavailable status should have a lower priority for access, the oldest nn > reported as standby or unavailable status should have a higher priority for > access. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17444) Add getJournalSyncerStatus jmx metrics, to Indicates whether the JournalSyncer thread for the every namespace has enter working state.
[ https://issues.apache.org/jira/browse/HDFS-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832376#comment-17832376 ] ASF GitHub Bot commented on HDFS-17444: --- hadoop-yetus commented on PR #6684: URL: https://github.com/apache/hadoop/pull/6684#issuecomment-2027925352 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 32s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 44m 6s | | trunk passed | | +1 :green_heart: | compile | 1m 27s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 1m 13s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 1m 13s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 22s | | trunk passed | | +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 10s | | trunk passed | | +1 :green_heart: | shadedclient | 35m 31s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 10s | | the patch passed | | +1 :green_heart: | compile | 1m 11s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 1m 11s | | the patch passed | | +1 :green_heart: | compile | 1m 5s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 1m 5s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 55s | | the patch passed | | +1 :green_heart: | mvnsite | 1m 12s | | the patch passed | | +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 3m 9s | | the patch passed | | +1 :green_heart: | shadedclient | 35m 33s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 228m 18s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6684/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 40s | | The patch does not generate ASF License warnings. | | | | 368m 1s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.qjournal.server.TestJournalNodeMXBean | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6684/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6684 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 2a171bd13796 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 249227e3ae3948bc462b942dbffa7685e0708578 | | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6684/7/testReport/ | | Max. process+thread count | 3855 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console ou
[jira] [Commented] (HDFS-17438) RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority.
[ https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832377#comment-17832377 ] ASF GitHub Bot commented on HDFS-17438: --- hadoop-yetus commented on PR #6655: URL: https://github.com/apache/hadoop/pull/6655#issuecomment-2027929979 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 48s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 50m 11s | | trunk passed | | +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | checkstyle | 0m 30s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 41s | | trunk passed | | +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | -1 :x: | spotbugs | 1m 22s | [/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/7/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html) | hadoop-hdfs-project/hadoop-hdfs-rbf in trunk has 1 extant spotbugs warnings. | | +1 :green_heart: | shadedclient | 39m 2s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 39m 23s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 30s | | the patch passed | | +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javac | 0m 33s | | the patch passed | | +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | javac | 0m 29s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 18s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 32s | | the patch passed | | +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 | | +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | +1 :green_heart: | spotbugs | 1m 20s | | hadoop-hdfs-project/hadoop-hdfs-rbf generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | +1 :green_heart: | shadedclient | 38m 53s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 31m 55s | | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | | The patch does not generate ASF License warnings. | | | | 176m 3s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6655 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets | | uname | Linux 7c30b4a07510 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / fd13d386dfcda71b96d86f2e921f7e84cbd4c388 | | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 | | Test Results | h
[jira] [Commented] (HDFS-17438) RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority.
[ https://issues.apache.org/jira/browse/HDFS-17438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832407#comment-17832407 ] ASF GitHub Bot commented on HDFS-17438: --- KeeProMise commented on PR #6655: URL: https://github.com/apache/hadoop/pull/6655#issuecomment-2027940379 @goiri hi, hadoop-hdfs-project/hadoop-hdfs-rbf in trunk has 1 extant spotbugs warnings, do you have any suggestions? > RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority. > - > > Key: HDFS-17438 > URL: https://issues.apache.org/jira/browse/HDFS-17438 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Jian Zhang >Assignee: Jian Zhang >Priority: Major > Labels: pull-request-available > Attachments: HDFS-17438.001.patch > > > At present, when the status of all namenodes in an ns in the router is the > same, the namenode which is the newest reported will be placed at the top of > the cache. when the client accesses the ns through the router, it will first > access the namenode. > If multiple namenodes in this route are in an active state, or if there are > namenodes with multiple observer states, the existing logic is not a problem, > because the newest reported active or observer state namenode have a higher > probability of being true active or observer compared to the namenode that > reported active or observer state a long time ago. > Similarly, the newest reported namenode with a status of standby or > unavailable has a higher probability of being a standby or unavailable > namenode compared to the namenode reported with a status of standby or > unavailable a long time ago. Therefore, the newest nn reported as standby or > unavailable status should have a lower priority for access, the oldest nn > reported as standby or unavailable status should have a higher priority for > access. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org