[GitHub] [hadoop] virajjasani commented on pull request #4107: HDFS-16521. DFS API to retrieve slow datanodes
virajjasani commented on PR #4107: URL: https://github.com/apache/hadoop/pull/4107#issuecomment-1113930859 Conflicts resolved with latest changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #4186: HDFS-16528. Reconfigure slow peer enable for Namenode
virajjasani commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1113927074 Thank you @tomscut, created branch-3.3 backport PR #4251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani opened a new pull request, #4251: HDFS-16528. Reconfigure slow peer enable for Namenode (#4186)
virajjasani opened a new pull request, #4251: URL: https://github.com/apache/hadoop/pull/4251 branch-3.3 backport of PR https://github.com/apache/hadoop/pull/4186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4250: YARN-11121:Conditional judgment Add in getClusterMetrics
hadoop-yetus commented on PR #4250: URL: https://github.com/apache/hadoop/pull/4250#issuecomment-1113906081 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 17m 51s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 40m 30s | | trunk passed | | +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 0m 38s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 41s | | trunk passed | | +1 :green_heart: | javadoc | 0m 47s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 8s | | trunk passed | | +1 :green_heart: | shadedclient | 23m 15s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 26s | | the patch passed | | +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | javac | 0m 27s | | the patch passed | | +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 0m 25s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 18s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 27s | | the patch passed | | +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 0m 55s | | the patch passed | | +1 :green_heart: | shadedclient | 22m 56s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 1m 59s | [/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4250/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt) | hadoop-yarn-server-router in the patch passed. | | +1 :green_heart: | asflicense | 0m 42s | | The patch does not generate ASF License warnings. | | | | 117m 22s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.yarn.server.router.webapp.TestRouterWebServicesREST | | | hadoop.yarn.server.router.clientrm.TestFederationClientInterceptor | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4250/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4250 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell | | uname | Linux 7106012ca50c 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / e22ee3f02d04d6a767df0310fd30cd94d9714e62 | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4250/1/testReport/ | | Max. process+thread count | 601 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router | | Console output | https://ci-hadoop.apache.org/jo
[GitHub] [hadoop] tomscut commented on pull request #4219: HDFS-16557. BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
tomscut commented on PR #4219: URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1113895505 Hi @xkrogen , to make the change safe, we can change the condition from: `if(next == HdfsServerConstants.INVALID_TXID)` to `if(next == HdfsServerConstants.INVALID_TXID || elis.isInProgress())` Do you think it's necessary? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4249: YARN-11120-Metrics for Federation getClusterMetrics
hadoop-yetus commented on PR #4249: URL: https://github.com/apache/hadoop/pull/4249#issuecomment-1113895082 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 7m 5s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | _ branch-3.3.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 35m 43s | | branch-3.3.3 passed | | +1 :green_heart: | compile | 0m 46s | | branch-3.3.3 passed | | +1 :green_heart: | checkstyle | 0m 47s | | branch-3.3.3 passed | | +1 :green_heart: | mvnsite | 0m 40s | | branch-3.3.3 passed | | +1 :green_heart: | javadoc | 0m 47s | | branch-3.3.3 passed | | +1 :green_heart: | spotbugs | 1m 22s | | branch-3.3.3 passed | | +1 :green_heart: | shadedclient | 26m 35s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 34s | | the patch passed | | +1 :green_heart: | compile | 0m 26s | | the patch passed | | +1 :green_heart: | javac | 0m 26s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 22s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 32s | | the patch passed | | +1 :green_heart: | javadoc | 0m 26s | | the patch passed | | +1 :green_heart: | spotbugs | 0m 57s | | the patch passed | | +1 :green_heart: | shadedclient | 23m 45s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 2m 7s | [/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4249/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt) | hadoop-yarn-server-router in the patch passed. | | +1 :green_heart: | asflicense | 0m 46s | | The patch does not generate ASF License warnings. | | | | 104m 26s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.yarn.server.router.clientrm.TestFederationClientInterceptor | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4249/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4249 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell | | uname | Linux 5edc023ddad0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-3.3.3 / 669dfe53672b690c640989801b5038a3d3f901d9 | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4249/1/testReport/ | | Max. process+thread count | 919 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4249/1/console | | versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on a diff in pull request #4219: HDFS-16557. BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
tomscut commented on code in PR #4219: URL: https://github.com/apache/hadoop/pull/4219#discussion_r862281008 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java: ## @@ -1792,7 +1792,7 @@ private void checkForGaps(List streams, long fromTxId, EditLogInputStream elis = iter.next(); if (elis.getFirstTxId() > txId) break; long next = elis.getLastTxId(); Review Comment: Thanks @xkrogen for you suggestion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4107: HDFS-16521. DFS API to retrieve slow datanodes
hadoop-yetus commented on PR #4107: URL: https://github.com/apache/hadoop/pull/4107#issuecomment-1113894018 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 1m 2s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 2s | | codespell was not available. | | +0 :ok: | buf | 0m 2s | | buf was not available. | | +0 :ok: | markdownlint | 0m 2s | | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 3 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 14m 36s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 27m 40s | | trunk passed | | +1 :green_heart: | compile | 6m 58s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 6m 23s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 35s | | trunk passed | | +1 :green_heart: | mvnsite | 3m 42s | | trunk passed | | +1 :green_heart: | javadoc | 3m 2s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 3m 44s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 7m 59s | | trunk passed | | +1 :green_heart: | shadedclient | 24m 1s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 24m 26s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 57s | | the patch passed | | +1 :green_heart: | compile | 6m 33s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | cc | 6m 33s | | the patch passed | | -1 :x: | javac | 6m 33s | [/results-compile-javac-hadoop-hdfs-project-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4107/10/artifact/out/results-compile-javac-hadoop-hdfs-project-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-hdfs-project-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 1 new + 651 unchanged - 0 fixed = 652 total (was 651) | | +1 :green_heart: | compile | 6m 14s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | cc | 6m 14s | | the patch passed | | -1 :x: | javac | 6m 14s | [/results-compile-javac-hadoop-hdfs-project-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4107/10/artifact/out/results-compile-javac-hadoop-hdfs-project-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt) | hadoop-hdfs-project-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 629 unchanged - 0 fixed = 630 total (was 629) | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 17s | | hadoop-hdfs-project: The patch generated 0 new + 455 unchanged - 1 fixed = 455 total (was 456) | | +1 :green_heart: | mvnsite | 3m 6s | | the patch passed | | +1 :green_heart: | javadoc | 2m 21s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 3m 8s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 7m 56s | | the patch passed | | +1 :green_heart: | shadedclient | 23m 18s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 2m 27s | | hadoop-hdfs-client in the patch passed. | | +1 :green_heart: | unit | 366m 56s | | hadoop-hdfs in the patch passed. | | -1 :x: | unit | 41m 11s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4107/10/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 1m 2s | | The patch does not
[GitHub] [hadoop] slfan1989 commented on pull request #4250: YARN-11121:Conditional judgment Add in getClusterMetrics
slfan1989 commented on PR #4250: URL: https://github.com/apache/hadoop/pull/4250#issuecomment-1113893985 @goiri @bibinchundatt Could you kindly help in reviewing this PR? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #4219: HDFS-16557. BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
tomscut commented on PR #4219: URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1113892494 > This seems right to me, but I don't fully understand what went wrong to cause the error. Can you explain more fully? Why did we previously make the assumption that `INVALID_TXID` meant in-progress, and what has changed to make that not true / what happened in your specific scenario to cause that not to be true? Thank you @xkrogen very much for your review. After introducing [SBN READ], we updated the configuration: `dfs.ha.tail-edits.in-progress=true`. Then when we `bootstrapStandby`, we will encounter something like this: 1. We need to start an Observer Namenode, so we execute bootstrapStandby before start it. This will automatically pull the latest FSImage from the Active Namenode and check whether the edits in the journals has a gap based on the `lastTxid` of the FSImage. 2. Assume that the txid of the latest FSImage is x, and editslogs from x in journals is in `InProgress` state, `FSEditLog#checkForGaps` will be skipped. Because the `lastTxid` of the InProgress EditLogInputStream is not `HdfsServerConstants.INVALID_TXID`, but a specific number. 3. However, between x and txID currently being written, there is finalize Edit log, and `bootstrapStandby` can execute normally. The `lastTxId` of an InProgress EditLogInputStream isn't always as `HdfsServerConstants.INVALID_TXID`, could also be a specific number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 opened a new pull request, #4250: YARN-11121:Conditional judgment Add in getClusterMetrics
slfan1989 opened a new pull request, #4250: URL: https://github.com/apache/hadoop/pull/4250 ### Description of PR YARN-11121 - Conditional judgment Add in getClusterMetrics https://issues.apache.org/jira/browse/YARN-11121 Currently getClusterMetrics need to check request is null ### How was this patch tested? ### For code changes: add conditional judgment to fIx if request is null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18079) Upgrade Netty to 4.1.74
[ https://issues.apache.org/jira/browse/HADOOP-18079?focusedWorklogId=764582&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764582 ] ASF GitHub Bot logged work on HADOOP-18079: --- Author: ASF GitHub Bot Created on: 30/Apr/22 01:27 Start Date: 30/Apr/22 01:27 Worklog Time Spent: 10m Work Description: jojochuang commented on code in PR #3977: URL: https://github.com/apache/hadoop/pull/3977#discussion_r862274475 ## hadoop-project/pom.xml: ## @@ -957,6 +957,72 @@ ${netty4.version} + +io.netty +netty-codec-socks +${netty4.version} + + + +io.netty +netty-handler-proxy +${netty4.version} + + + +io.netty +netty-resolver +${netty4.version} + + + +io.netty +netty-handler +${netty4.version} + + + +io.netty +netty-buffer +${netty4.version} + + + +io.netty +netty-transport +${netty4.version} + + + +io.netty +netty-common +${netty4.version} + + + +io.netty +netty-transport-native-unix-common +${netty4.version} + + + +io.netty +netty-transport Review Comment: Very good catch. Thanks @brahmareddybattula . Updated. Issue Time Tracking --- Worklog Id: (was: 764582) Time Spent: 2h 40m (was: 2.5h) > Upgrade Netty to 4.1.74 > --- > > Key: HADOOP-18079 > URL: https://issues.apache.org/jira/browse/HADOOP-18079 > Project: Hadoop Common > Issue Type: Bug >Reporter: Renukaprasad C >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > h4. Netty version - 4.1.71 has fix some CVEs. We can upgradeĀ the netty to > 4.1.7.1.Final or latest stable version - 4.1.7.2.Final. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a diff in pull request #3977: HADOOP-18079. Upgrade Netty to 4.1.75.
jojochuang commented on code in PR #3977: URL: https://github.com/apache/hadoop/pull/3977#discussion_r862274475 ## hadoop-project/pom.xml: ## @@ -957,6 +957,72 @@ ${netty4.version} + +io.netty +netty-codec-socks +${netty4.version} + + + +io.netty +netty-handler-proxy +${netty4.version} + + + +io.netty +netty-resolver +${netty4.version} + + + +io.netty +netty-handler +${netty4.version} + + + +io.netty +netty-buffer +${netty4.version} + + + +io.netty +netty-transport +${netty4.version} + + + +io.netty +netty-common +${netty4.version} + + + +io.netty +netty-transport-native-unix-common +${netty4.version} + + + +io.netty +netty-transport Review Comment: Very good catch. Thanks @brahmareddybattula . Updated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 opened a new pull request, #4249: Branch 3.3.3
slfan1989 opened a new pull request, #4249: URL: https://github.com/apache/hadoop/pull/4249 YARN-11120-Metrics for Federation getClusterMetrics ### Description of PR getClusterMetrics need some metrics,This pr fix it ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client
[ https://issues.apache.org/jira/browse/HADOOP-18069?focusedWorklogId=764552&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764552 ] ASF GitHub Bot logged work on HADOOP-18069: --- Author: ASF GitHub Bot Created on: 29/Apr/22 23:40 Start Date: 29/Apr/22 23:40 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on PR #4229: URL: https://github.com/apache/hadoop/pull/4229#issuecomment-1113862863 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 1m 18s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 14m 44s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 28m 10s | | trunk passed | | +1 :green_heart: | compile | 24m 54s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 21m 32s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 4m 31s | | trunk passed | | +1 :green_heart: | mvnsite | 19m 54s | | trunk passed | | -1 :x: | javadoc | 1m 37s | [/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | root in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | +1 :green_heart: | javadoc | 8m 29s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +0 :ok: | spotbugs | 0m 27s | | branch/hadoop-project no spotbugs output file (spotbugsXml.xml) | | +1 :green_heart: | shadedclient | 57m 49s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 38s | | Maven dependency ordering for patch | | -1 :x: | mvninstall | 24m 44s | [/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/patch-mvninstall-root.txt) | root in the patch failed. | | +1 :green_heart: | compile | 24m 21s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 24m 21s | | the patch passed | | +1 :green_heart: | compile | 21m 42s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 21m 42s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 4m 21s | | the patch passed | | +1 :green_heart: | mvnsite | 19m 38s | | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | | No new issues. | | +1 :green_heart: | xml | 0m 3s | | The patch has no ill-formed XML file. | | -1 :x: | javadoc | 1m 25s | [/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | root in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | +1 :green_heart: | javadoc | 8m 25s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +0 :ok: | spotbugs | 0m 26s | | hadoop-project has no data from spotbugs | | -1 :x: | spotbugs | 2m 54s | [/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html) | hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -1 :x: | spotbugs | 35m 39s | [/new-spotbugs-root.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/new-spotbugs-root.html) | root generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -1 :x: | shadedclient | 58m 21s | | patch has errors when building and testin
[GitHub] [hadoop] hadoop-yetus commented on pull request #4229: HADOOP-18069. okhttp@2.7.5 to 4.9.3
hadoop-yetus commented on PR #4229: URL: https://github.com/apache/hadoop/pull/4229#issuecomment-1113862863 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 1m 18s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 14m 44s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 28m 10s | | trunk passed | | +1 :green_heart: | compile | 24m 54s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 21m 32s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 4m 31s | | trunk passed | | +1 :green_heart: | mvnsite | 19m 54s | | trunk passed | | -1 :x: | javadoc | 1m 37s | [/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | root in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | +1 :green_heart: | javadoc | 8m 29s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +0 :ok: | spotbugs | 0m 27s | | branch/hadoop-project no spotbugs output file (spotbugsXml.xml) | | +1 :green_heart: | shadedclient | 57m 49s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 38s | | Maven dependency ordering for patch | | -1 :x: | mvninstall | 24m 44s | [/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/patch-mvninstall-root.txt) | root in the patch failed. | | +1 :green_heart: | compile | 24m 21s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 24m 21s | | the patch passed | | +1 :green_heart: | compile | 21m 42s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 21m 42s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 4m 21s | | the patch passed | | +1 :green_heart: | mvnsite | 19m 38s | | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | | No new issues. | | +1 :green_heart: | xml | 0m 3s | | The patch has no ill-formed XML file. | | -1 :x: | javadoc | 1m 25s | [/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | root in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | +1 :green_heart: | javadoc | 8m 25s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +0 :ok: | spotbugs | 0m 26s | | hadoop-project has no data from spotbugs | | -1 :x: | spotbugs | 2m 54s | [/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html) | hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -1 :x: | spotbugs | 35m 39s | [/new-spotbugs-root.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/new-spotbugs-root.html) | root generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -1 :x: | shadedclient | 58m 21s | | patch has errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 1042m 36s | [/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/9/artifact/out/patch-unit-root.txt) | root in the patch passed. | | +1 :green_heart: | asflicense | 2m 19s | | The patch does not generate ASF License warnings. | | | | 1406m 47s | | | | Reason | Tests | |---:|:--
[GitHub] [hadoop] hadoop-yetus commented on pull request #4245: HDFS-16564. Use uint32_t for hdfs_find
hadoop-yetus commented on PR #4245: URL: https://github.com/apache/hadoop/pull/4245#issuecomment-1113854380 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 38s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 26m 6s | | trunk passed | | +1 :green_heart: | compile | 3m 37s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 43s | | trunk passed | | -1 :x: | shadedclient | 56m 5s | | branch has errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 23s | | the patch passed | | +1 :green_heart: | compile | 3m 22s | | the patch passed | | +1 :green_heart: | cc | 3m 22s | | the patch passed | | +1 :green_heart: | golang | 3m 22s | | the patch passed | | +1 :green_heart: | javac | 3m 22s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | mvnsite | 0m 26s | | the patch passed | | -1 :x: | shadedclient | 25m 40s | | patch has errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 32m 29s | | hadoop-hdfs-native-client in the patch passed. | | +1 :green_heart: | asflicense | 0m 48s | | The patch does not generate ASF License warnings. | | | | 122m 19s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4245/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4245 | | Optional Tests | dupname asflicense compile cc mvnsite javac unit codespell golang | | uname | Linux b11c88ca1021 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 2eef94c4ae6df5e3b45e5cb286d7914f338f94ae | | Default Java | Debian-11.0.14+9-post-Debian-1deb10u1 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4245/2/testReport/ | | Max. process+thread count | 503 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4245/2/console | | versions | git=2.20.1 maven=3.6.0 | | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18104) Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads
[ https://issues.apache.org/jira/browse/HADOOP-18104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur resolved HADOOP-18104. Resolution: Fixed > Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads > > > Key: HADOOP-18104 > URL: https://issues.apache.org/jira/browse/HADOOP-18104 > Project: Hadoop Common > Issue Type: Sub-task > Components: common, fs >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18104) Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads
[ https://issues.apache.org/jira/browse/HADOOP-18104?focusedWorklogId=764543&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764543 ] ASF GitHub Bot logged work on HADOOP-18104: --- Author: ASF GitHub Bot Created on: 29/Apr/22 22:47 Start Date: 29/Apr/22 22:47 Worklog Time Spent: 10m Work Description: mukund-thakur merged PR #3964: URL: https://github.com/apache/hadoop/pull/3964 Issue Time Tracking --- Worklog Id: (was: 764543) Time Spent: 3h 40m (was: 3.5h) > Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads > > > Key: HADOOP-18104 > URL: https://issues.apache.org/jira/browse/HADOOP-18104 > Project: Hadoop Common > Issue Type: Sub-task > Components: common, fs >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur merged pull request #3964: HADOOP-18104: S3A: Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads
mukund-thakur merged PR #3964: URL: https://github.com/apache/hadoop/pull/3964 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xkrogen commented on a diff in pull request #4219: HDFS-16557. BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
xkrogen commented on code in PR #4219: URL: https://github.com/apache/hadoop/pull/4219#discussion_r862193291 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java: ## @@ -1792,7 +1792,7 @@ private void checkForGaps(List streams, long fromTxId, EditLogInputStream elis = iter.next(); if (elis.getFirstTxId() > txId) break; long next = elis.getLastTxId(); Review Comment: The local variable is redundant now, we can just update L1805 to be: ```java txId = elis.getLastTxId() + 1; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4245: HDFS-16564. Use uint32_t for hdfs_find
hadoop-yetus commented on PR #4245: URL: https://github.com/apache/hadoop/pull/4245#issuecomment-1113733505 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 39s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 22m 3s | | trunk passed | | +1 :green_heart: | compile | 4m 11s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 4s | | trunk passed | | +1 :green_heart: | shadedclient | 46m 35s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 31s | | the patch passed | | +1 :green_heart: | compile | 3m 41s | | the patch passed | | +1 :green_heart: | cc | 3m 41s | | the patch passed | | +1 :green_heart: | golang | 3m 41s | | the patch passed | | +1 :green_heart: | javac | 3m 41s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | mvnsite | 0m 36s | | the patch passed | | +1 :green_heart: | shadedclient | 19m 19s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 33m 16s | | hadoop-hdfs-native-client in the patch passed. | | +1 :green_heart: | asflicense | 1m 0s | | The patch does not generate ASF License warnings. | | | | 108m 7s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4245/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4245 | | Optional Tests | dupname asflicense compile cc mvnsite javac unit codespell golang | | uname | Linux 91bf84fb6bbe 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 2eef94c4ae6df5e3b45e5cb286d7914f338f94ae | | Default Java | Red Hat, Inc.-1.8.0_312-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4245/2/testReport/ | | Max. process+thread count | 713 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4245/2/console | | versions | git=2.27.0 maven=3.6.3 | | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4245: HDFS-16564. Use uint32_t for hdfs_find
hadoop-yetus commented on PR #4245: URL: https://github.com/apache/hadoop/pull/4245#issuecomment-1113646526 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 40s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 37m 54s | | trunk passed | | +1 :green_heart: | compile | 3m 56s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 49s | | trunk passed | | +1 :green_heart: | shadedclient | 61m 28s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 24s | | the patch passed | | +1 :green_heart: | compile | 3m 38s | | the patch passed | | +1 :green_heart: | cc | 3m 38s | | the patch passed | | +1 :green_heart: | golang | 3m 38s | | the patch passed | | +1 :green_heart: | javac | 3m 38s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | mvnsite | 0m 28s | | the patch passed | | +1 :green_heart: | shadedclient | 18m 46s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 32m 52s | | hadoop-hdfs-native-client in the patch passed. | | +1 :green_heart: | asflicense | 0m 53s | | The patch does not generate ASF License warnings. | | | | 121m 30s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4245/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4245 | | Optional Tests | dupname asflicense compile cc mvnsite javac unit codespell golang | | uname | Linux 572db4ff8b56 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 2eef94c4ae6df5e3b45e5cb286d7914f338f94ae | | Default Java | Red Hat, Inc.-1.8.0_322-b06 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4245/2/testReport/ | | Max. process+thread count | 698 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4245/2/console | | versions | git=2.9.5 maven=3.6.3 | | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4235: YARN-11114. RMWebServices returns only apps matching exactly the submitted queue name
hadoop-yetus commented on PR #4235: URL: https://github.com/apache/hadoop/pull/4235#issuecomment-1113559807 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 55s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 2 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 37m 17s | | trunk passed | | +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 11s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 20s | | trunk passed | | +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 1m 0s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 2m 24s | | trunk passed | | +1 :green_heart: | shadedclient | 21m 53s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 58s | | the patch passed | | +1 :green_heart: | compile | 1m 4s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 1m 4s | | the patch passed | | +1 :green_heart: | compile | 0m 57s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 0m 57s | | the patch passed | | -1 :x: | blanks | 0m 0s | [/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4235/4/artifact/out/blanks-eol.txt) | The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | -0 :warning: | checkstyle | 0m 48s | [/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4235/4/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt) | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 44 unchanged - 0 fixed = 46 total (was 44) | | +1 :green_heart: | mvnsite | 0m 59s | | the patch passed | | +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 2m 3s | | the patch passed | | +1 :green_heart: | shadedclient | 21m 8s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 100m 35s | [/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4235/4/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt) | hadoop-yarn-server-resourcemanager in the patch passed. | | +1 :green_heart: | asflicense | 0m 51s | | The patch does not generate ASF License warnings. | | | | 200m 10s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication | | | hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication | | | hadoop.yarn.server.resourcemanager.TestClientRMService | | | hadoop.yarn.webapp.TestRMWithXFSFilter | | | hadoop.yarn.server.resourcemanager.TestRMHA | | | hadoop.yarn.server.resourcemanager.metrics.TestCombinedSystemMetricsPublisher | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4235/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4235 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall m
[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #4245: HDFS-16564. Use uint32_t for hdfs_find
GauthamBanasandra commented on code in PR #4245: URL: https://github.com/apache/hadoop/pull/4245#discussion_r861986392 ## hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfspp_mini_dfs_smoke.cc: ## @@ -34,7 +34,6 @@ TEST_F(HdfsMiniDfsSmokeTest, SmokeTest) { EXPECT_NE(nullptr, connection.handle()); } - Review Comment: I need to touch at least one test file. Otherwise, `test4tests` will fail while building for the first platform in the pipeline (Centos 7) and doesn't proceed to build for the rest of the platforms. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra merged pull request #4228: HDFS-16468. Define ssize_t for Windows
GauthamBanasandra merged PR #4228: URL: https://github.com/apache/hadoop/pull/4228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4241: HDFS-16563. Namenode WebUI prints sensitive information on Token expiry
hadoop-yetus commented on PR #4241: URL: https://github.com/apache/hadoop/pull/4241#issuecomment-1113495547 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 1m 7s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 2s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 40m 11s | | trunk passed | | +1 :green_heart: | compile | 25m 6s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 21m 46s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 30s | | trunk passed | | +1 :green_heart: | mvnsite | 2m 1s | | trunk passed | | -1 :x: | javadoc | 1m 37s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/4/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-common in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | +1 :green_heart: | javadoc | 2m 5s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 3m 6s | | trunk passed | | +1 :green_heart: | shadedclient | 25m 58s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 7s | | the patch passed | | +1 :green_heart: | compile | 30m 4s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 30m 4s | | the patch passed | | +1 :green_heart: | compile | 23m 12s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 23m 12s | | the patch passed | | +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 25s | | the patch passed | | +1 :green_heart: | mvnsite | 2m 0s | | the patch passed | | -1 :x: | javadoc | 1m 27s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/4/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-common in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | +1 :green_heart: | javadoc | 2m 4s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 3m 3s | | the patch passed | | +1 :green_heart: | shadedclient | 25m 37s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 18m 11s | | hadoop-common in the patch passed. | | +1 :green_heart: | asflicense | 1m 17s | | The patch does not generate ASF License warnings. | | | | 234m 26s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4241 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell | | uname | Linux 2b45026f7a30 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 7579eafa0498b6a7fa49e10751c92ab4b7c42616 | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/4/testReport/ | | Max. process+thread count | 2469 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common
[GitHub] [hadoop] hadoop-yetus commented on pull request #4235: YARN-11114. RMWebServices returns only apps matching exactly the submitted queue name
hadoop-yetus commented on PR #4235: URL: https://github.com/apache/hadoop/pull/4235#issuecomment-1113480893 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 46s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 2 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 37m 39s | | trunk passed | | +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 12s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 21s | | trunk passed | | +1 :green_heart: | javadoc | 1m 14s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 2m 24s | | trunk passed | | +1 :green_heart: | shadedclient | 21m 26s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 59s | | the patch passed | | +1 :green_heart: | compile | 1m 6s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 1m 6s | | the patch passed | | +1 :green_heart: | compile | 0m 58s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 0m 58s | | the patch passed | | -1 :x: | blanks | 0m 0s | [/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4235/3/artifact/out/blanks-eol.txt) | The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | -0 :warning: | checkstyle | 0m 48s | [/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4235/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt) | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 44 unchanged - 0 fixed = 46 total (was 44) | | +1 :green_heart: | mvnsite | 1m 2s | | the patch passed | | +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 2m 4s | | the patch passed | | +1 :green_heart: | shadedclient | 21m 1s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 96m 49s | [/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4235/3/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt) | hadoop-yarn-server-resourcemanager in the patch passed. | | +1 :green_heart: | asflicense | 0m 51s | | The patch does not generate ASF License warnings. | | | | 196m 24s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication | | | hadoop.yarn.server.resourcemanager.TestRMHA | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication | | | hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher | | | hadoop.yarn.server.resourcemanager.metrics.TestCombinedSystemMetricsPublisher | | | hadoop.yarn.webapp.TestRMWithXFSFilter | | | hadoop.yarn.server.resourcemanager.TestClientRMService | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4235/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4235 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall m
[GitHub] [hadoop] Hexiaoqiao commented on pull request #4211: HDFS-16553 Fix checkstyle for the length of BlockManager construction method over limit.
Hexiaoqiao commented on PR #4211: URL: https://github.com/apache/hadoop/pull/4211#issuecomment-1113428279 Committed to trunk. Thanks @smarthanwang for your contribution. Thank @ayushtkn, @tomscut for your reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao merged pull request #4211: HDFS-16553 Fix checkstyle for the length of BlockManager construction method over limit.
Hexiaoqiao merged PR #4211: URL: https://github.com/apache/hadoop/pull/4211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4248: MAPREDUCE-7370. Parallelize MultipleOutputs#close call
hadoop-yetus commented on PR #4248: URL: https://github.com/apache/hadoop/pull/4248#issuecomment-1113391592 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 40s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 37m 17s | | trunk passed | | +1 :green_heart: | compile | 1m 3s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 0m 57s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 0m 59s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 4s | | trunk passed | | +1 :green_heart: | javadoc | 0m 53s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 50s | | trunk passed | | +1 :green_heart: | shadedclient | 21m 4s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 42s | | the patch passed | | +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 0m 43s | | the patch passed | | +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 0m 39s | | the patch passed | | +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 34s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 44s | | the patch passed | | +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 30s | | the patch passed | | +1 :green_heart: | shadedclient | 20m 25s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 6m 22s | | hadoop-mapreduce-client-core in the patch passed. | | +1 :green_heart: | asflicense | 0m 51s | | The patch does not generate ASF License warnings. | | | | 100m 45s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4248/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4248 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell | | uname | Linux 19fab0528717 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 4646a36b1fcf62b39ab182741acecb2cf622ad7e | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4248/1/testReport/ | | Max. process+thread count | 1545 (vs. ulimit of 5500) | | modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4248/1/console | | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
[GitHub] [hadoop] hadoop-yetus commented on pull request #4247: MAPREDUCE-7369. Fixed MapReduce tasks timing out when spends more time on MultipleOutputs#close
hadoop-yetus commented on PR #4247: URL: https://github.com/apache/hadoop/pull/4247#issuecomment-1113376541 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 59s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 14m 45s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 24m 55s | | trunk passed | | +1 :green_heart: | compile | 2m 52s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 2m 30s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 26s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 58s | | trunk passed | | +1 :green_heart: | javadoc | 1m 41s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 1m 31s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 3m 7s | | trunk passed | | +1 :green_heart: | shadedclient | 20m 31s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 15s | | the patch passed | | +1 :green_heart: | compile | 2m 34s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 2m 34s | | the patch passed | | +1 :green_heart: | compile | 2m 16s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 2m 16s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 1m 4s | | the patch passed | | +1 :green_heart: | mvnsite | 1m 24s | | the patch passed | | +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 2m 43s | | the patch passed | | +1 :green_heart: | shadedclient | 20m 29s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 6m 42s | | hadoop-mapreduce-client-core in the patch passed. | | +1 :green_heart: | unit | 8m 51s | | hadoop-mapreduce-client-app in the patch passed. | | +1 :green_heart: | asflicense | 0m 51s | | The patch does not generate ASF License warnings. | | | | 128m 42s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4247/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4247 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml | | uname | Linux 45f4b0f6e973 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / c168a398e83cc9ad5b435dcd80d6375f427faeb6 | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4247/2/testReport/ | | Max. process+thread count | 1398 (vs. ulimit of 5500) | | modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4247/2/console | | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 | |
[GitHub] [hadoop] szilard-nemeth commented on a diff in pull request #4235: YARN-11114. RMWebServices returns only apps matching exactly the submitted queue name
szilard-nemeth commented on code in PR #4235: URL: https://github.com/apache/hadoop/pull/4235#discussion_r861836781 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java: ## @@ -1970,5 +1981,113 @@ public void verifyResourceRequestsGeneric(ResourceRequest request, enforceExecutionType); } + @Test + public void testAppsQueryByQueueShortname() throws Exception { +GuiceServletConfig.setInjector( +Guice.createInjector(new WebServletModule(CapacityScheduler.class))); + +rm.start(); +MockNM amNodeManager = rm.registerNode("127.0.0.1:1234", 2048); +RMApp finishedApp = MockRMAppSubmitter.submit(rm, +MockRMAppSubmissionData.Builder +.createWithMemory(CONTAINER_MB, rm) +.withQueue("root.default") +.build()); +RMApp runningApp1 = MockRMAppSubmitter.submit(rm, +MockRMAppSubmissionData.Builder +.createWithMemory(CONTAINER_MB, rm) +.withQueue("default") +.build()); + +RMApp runningApp2 = MockRMAppSubmitter.submit(rm, +MockRMAppSubmissionData.Builder +.createWithMemory(CONTAINER_MB, rm) +.withQueue("root.default") +.build()); +amNodeManager.nodeHeartbeat(true); + +finishApp(amNodeManager, finishedApp); + +WebResource r = resource(); + +ClientResponse response = r.path("ws").path("v1").path("cluster") +.path("apps") +.queryParam("queue", "default") +.accept(MediaType.APPLICATION_JSON).get(ClientResponse.class); +assertEquals(MediaType.APPLICATION_JSON_TYPE + "; " + JettyUtils.UTF_8, +response.getType().toString()); +JSONObject json = response.getEntity(JSONObject.class); +assertEquals("incorrect number of elements", 1, json.length()); +JSONObject apps = json.getJSONObject("apps"); +assertEquals("incorrect number of elements", 1, apps.length()); + +JSONArray array = apps.getJSONArray("app"); + +Set appIds = getApplicationIds(array); +assertTrue("Running 1 app should be in the result list!", Review Comment: You're right, fixed it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] brumi1024 commented on a diff in pull request #4235: YARN-11114. RMWebServices returns only apps matching exactly the submitted queue name
brumi1024 commented on code in PR #4235: URL: https://github.com/apache/hadoop/pull/4235#discussion_r861811715 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java: ## @@ -1970,5 +1981,113 @@ public void verifyResourceRequestsGeneric(ResourceRequest request, enforceExecutionType); } + @Test + public void testAppsQueryByQueueShortname() throws Exception { +GuiceServletConfig.setInjector( +Guice.createInjector(new WebServletModule(CapacityScheduler.class))); + +rm.start(); +MockNM amNodeManager = rm.registerNode("127.0.0.1:1234", 2048); +RMApp finishedApp = MockRMAppSubmitter.submit(rm, +MockRMAppSubmissionData.Builder +.createWithMemory(CONTAINER_MB, rm) +.withQueue("root.default") +.build()); +RMApp runningApp1 = MockRMAppSubmitter.submit(rm, +MockRMAppSubmissionData.Builder +.createWithMemory(CONTAINER_MB, rm) +.withQueue("default") +.build()); + +RMApp runningApp2 = MockRMAppSubmitter.submit(rm, +MockRMAppSubmissionData.Builder +.createWithMemory(CONTAINER_MB, rm) +.withQueue("root.default") +.build()); +amNodeManager.nodeHeartbeat(true); + +finishApp(amNodeManager, finishedApp); + +WebResource r = resource(); + +ClientResponse response = r.path("ws").path("v1").path("cluster") +.path("apps") +.queryParam("queue", "default") +.accept(MediaType.APPLICATION_JSON).get(ClientResponse.class); +assertEquals(MediaType.APPLICATION_JSON_TYPE + "; " + JettyUtils.UTF_8, +response.getType().toString()); +JSONObject json = response.getEntity(JSONObject.class); +assertEquals("incorrect number of elements", 1, json.length()); +JSONObject apps = json.getJSONObject("apps"); +assertEquals("incorrect number of elements", 1, apps.length()); + +JSONArray array = apps.getJSONArray("app"); + +Set appIds = getApplicationIds(array); +assertTrue("Running 1 app should be in the result list!", Review Comment: Nit: _Running 1 app should be in the result list_ this sounds strange, shouldn't this be: runningApp1 should be in the result list? ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java: ## @@ -1970,5 +1981,113 @@ public void verifyResourceRequestsGeneric(ResourceRequest request, enforceExecutionType); } + @Test + public void testAppsQueryByQueueShortname() throws Exception { +GuiceServletConfig.setInjector( +Guice.createInjector(new WebServletModule(CapacityScheduler.class))); + +rm.start(); +MockNM amNodeManager = rm.registerNode("127.0.0.1:1234", 2048); +RMApp finishedApp = MockRMAppSubmitter.submit(rm, +MockRMAppSubmissionData.Builder +.createWithMemory(CONTAINER_MB, rm) +.withQueue("root.default") +.build()); +RMApp runningApp1 = MockRMAppSubmitter.submit(rm, +MockRMAppSubmissionData.Builder +.createWithMemory(CONTAINER_MB, rm) +.withQueue("default") +.build()); + +RMApp runningApp2 = MockRMAppSubmitter.submit(rm, +MockRMAppSubmissionData.Builder +.createWithMemory(CONTAINER_MB, rm) +.withQueue("root.default") +.build()); +amNodeManager.nodeHeartbeat(true); + +finishApp(amNodeManager, finishedApp); Review Comment: Can you please add a finished app submitted with a queue short name, possibly with a comment to show that the current behaviour is somewhat expected? The same applies for the other testcase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ashutoshcipher opened a new pull request, #4248: MAPREDUCE-7370. Parallelize MultipleOutputs#close call
ashutoshcipher opened a new pull request, #4248: URL: https://github.com/apache/hadoop/pull/4248 ### Description of PR Parallelize MultipleOutputs#close call * JIRA: MAPREDUCE-7370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] prasad-acit commented on pull request #4241: HDFS-16563. Namenode WebUI prints sensitive information on Token expiry
prasad-acit commented on PR #4241: URL: https://github.com/apache/hadoop/pull/4241#issuecomment-1113253102 Javadoc issues are not related. Checkstyle issue addressed & pushed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16202) Enhance openFile() for better read performance against object stores
[ https://issues.apache.org/jira/browse/HADOOP-16202?focusedWorklogId=764141&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764141 ] ASF GitHub Bot logged work on HADOOP-16202: --- Author: ASF GitHub Bot Created on: 29/Apr/22 10:56 Start Date: 29/Apr/22 10:56 Worklog Time Spent: 10m Work Description: steveloughran commented on PR #2584: URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1113172048 merged Issue Time Tracking --- Worklog Id: (was: 764141) Time Spent: 23h (was: 22h 50m) > Enhance openFile() for better read performance against object stores > - > > Key: HADOOP-16202 > URL: https://issues.apache.org/jira/browse/HADOOP-16202 > Project: Hadoop Common > Issue Type: Improvement > Components: fs, fs/s3, tools/distcp >Affects Versions: 3.3.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Time Spent: 23h > Remaining Estimate: 0h > > The {{openFile()}} builder API lets us add new options when reading a file > Add an option {{"fs.s3a.open.option.length"}} which takes a long and allows > the length of the file to be declared. If set, *no check for the existence of > the file is issued when opening the file* > Also: withFileStatus() to take any FileStatus implementation, rather than > only S3AFileStatus -and not check that the path matches the path being > opened. Needed to support viewFS-style wrapping and mounting. > and Adopt where appropriate to stop clusters with S3A reads switched to > random IO from killing download/localization > * fs shell copyToLocal > * distcp > * IOUtils.copy -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16202) Enhance openFile() for better read performance against object stores
[ https://issues.apache.org/jira/browse/HADOOP-16202?focusedWorklogId=764142&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764142 ] ASF GitHub Bot logged work on HADOOP-16202: --- Author: ASF GitHub Bot Created on: 29/Apr/22 10:56 Start Date: 29/Apr/22 10:56 Worklog Time Spent: 10m Work Description: steveloughran closed pull request #2584: HADOOP-16202. Enhance openFile() for better read performance against object stores URL: https://github.com/apache/hadoop/pull/2584 Issue Time Tracking --- Worklog Id: (was: 764142) Time Spent: 23h 10m (was: 23h) > Enhance openFile() for better read performance against object stores > - > > Key: HADOOP-16202 > URL: https://issues.apache.org/jira/browse/HADOOP-16202 > Project: Hadoop Common > Issue Type: Improvement > Components: fs, fs/s3, tools/distcp >Affects Versions: 3.3.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Time Spent: 23h 10m > Remaining Estimate: 0h > > The {{openFile()}} builder API lets us add new options when reading a file > Add an option {{"fs.s3a.open.option.length"}} which takes a long and allows > the length of the file to be declared. If set, *no check for the existence of > the file is issued when opening the file* > Also: withFileStatus() to take any FileStatus implementation, rather than > only S3AFileStatus -and not check that the path matches the path being > opened. Needed to support viewFS-style wrapping and mounting. > and Adopt where appropriate to stop clusters with S3A reads switched to > random IO from killing download/localization > * fs shell copyToLocal > * distcp > * IOUtils.copy -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran closed pull request #2584: HADOOP-16202. Enhance openFile() for better read performance against object stores
steveloughran closed pull request #2584: HADOOP-16202. Enhance openFile() for better read performance against object stores URL: https://github.com/apache/hadoop/pull/2584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2584: HADOOP-16202. Enhance openFile() for better read performance against object stores
steveloughran commented on PR #2584: URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1113172048 merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[ https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=764136&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764136 ] ASF GitHub Bot logged work on HADOOP-18168: --- Author: ASF GitHub Bot Created on: 29/Apr/22 10:25 Start Date: 29/Apr/22 10:25 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1113150246 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 41s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 39m 27s | | trunk passed | | +1 :green_heart: | compile | 1m 0s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 0m 52s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 0m 46s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 0s | | trunk passed | | +1 :green_heart: | javadoc | 0m 48s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 47s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 35s | | trunk passed | | +1 :green_heart: | shadedclient | 21m 45s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 22m 8s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 39s | | the patch passed | | +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 0m 40s | | the patch passed | | +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 0m 37s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 24s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 41s | | the patch passed | | +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 17s | | the patch passed | | +1 :green_heart: | shadedclient | 20m 52s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 2m 34s | | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 45s | | The patch does not generate ASF License warnings. | | | | 98m 49s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4140 | | Optional Tests | dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle xml | | uname | Linux b8109ee29d83 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 6302c5b9e54e55153f3afa895c8dc4d032025407 | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/12/testReport/ | | Max. pro
[GitHub] [hadoop] hadoop-yetus commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dep. on purged public bucket, introduce PublicDatasetTestUtils
hadoop-yetus commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1113150246 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 41s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 39m 27s | | trunk passed | | +1 :green_heart: | compile | 1m 0s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 0m 52s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 0m 46s | | trunk passed | | +1 :green_heart: | mvnsite | 1m 0s | | trunk passed | | +1 :green_heart: | javadoc | 0m 48s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 47s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 35s | | trunk passed | | +1 :green_heart: | shadedclient | 21m 45s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 22m 8s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 39s | | the patch passed | | +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 0m 40s | | the patch passed | | +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 0m 37s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 24s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 41s | | the patch passed | | +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 17s | | the patch passed | | +1 :green_heart: | shadedclient | 20m 52s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 2m 34s | | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 45s | | The patch does not generate ASF License warnings. | | | | 98m 49s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4140 | | Optional Tests | dupname asflicense mvnsite codespell markdownlint compile javac javadoc mvninstall unit shadedclient spotbugs checkstyle xml | | uname | Linux b8109ee29d83 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 6302c5b9e54e55153f3afa895c8dc4d032025407 | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/12/testReport/ | | Max. process+thread count | 614 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/12/console | | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. -- This is an automat
[GitHub] [hadoop] hadoop-yetus commented on pull request #4241: HDFS-16563. Namenode WebUI prints sensitive information on Token expiry
hadoop-yetus commented on PR #4241: URL: https://github.com/apache/hadoop/pull/4241#issuecomment-1113136407 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 37s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 37m 12s | | trunk passed | | +1 :green_heart: | compile | 23m 29s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | compile | 21m 22s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 1m 44s | | trunk passed | | +1 :green_heart: | mvnsite | 2m 9s | | trunk passed | | -1 :x: | javadoc | 1m 51s | [/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/3/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt) | hadoop-common in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. | | +1 :green_heart: | javadoc | 2m 14s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 3m 9s | | trunk passed | | +1 :green_heart: | shadedclient | 23m 5s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 4s | | the patch passed | | +1 :green_heart: | compile | 22m 16s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | javac | 22m 16s | | the patch passed | | +1 :green_heart: | compile | 20m 38s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 20m 38s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | -0 :warning: | checkstyle | 1m 38s | [/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/3/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt) | hadoop-common-project/hadoop-common: The patch generated 1 new + 39 unchanged - 0 fixed = 40 total (was 39) | | +1 :green_heart: | mvnsite | 2m 7s | | the patch passed | | -1 :x: | javadoc | 1m 42s | [/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/3/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt) | hadoop-common in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. | | +1 :green_heart: | javadoc | 2m 16s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 3m 6s | | the patch passed | | +1 :green_heart: | shadedclient | 23m 20s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 18m 30s | | hadoop-common in the patch passed. | | +1 :green_heart: | asflicense | 1m 35s | | The patch does not generate ASF License warnings. | | | | 216m 25s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4241 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell | | uname | Linux 0bf9eee057de 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / aac4044d7848cea4a64db873e33ec30dcb17dd13 | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[ https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=764126&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764126 ] ASF GitHub Bot logged work on HADOOP-18168: --- Author: ASF GitHub Bot Created on: 29/Apr/22 09:59 Start Date: 29/Apr/22 09:59 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1113130188 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 52s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 39m 34s | | trunk passed | | +1 :green_heart: | compile | 0m 47s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 0m 36s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 47s | | trunk passed | | +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 26s | | trunk passed | | +1 :green_heart: | shadedclient | 22m 20s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 22m 39s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 35s | | the patch passed | | +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 0m 40s | | the patch passed | | +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 0m 31s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 20s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 37s | | the patch passed | | +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 14s | | the patch passed | | +1 :green_heart: | shadedclient | 22m 35s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 2m 24s | | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 38s | | The patch does not generate ASF License warnings. | | | | 99m 13s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4140 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml | | uname | Linux db4abd57055b 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / cb19dff096688c442e76cba672376eb7cb72161a | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/10/testReport/ | | Max. process+thread count | 690 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hado
[GitHub] [hadoop] hadoop-yetus commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dep. on purged public bucket, introduce PublicDatasetTestUtils
hadoop-yetus commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1113130188 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 52s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 39m 34s | | trunk passed | | +1 :green_heart: | compile | 0m 47s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | checkstyle | 0m 36s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 47s | | trunk passed | | +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 26s | | trunk passed | | +1 :green_heart: | shadedclient | 22m 20s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 22m 39s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 35s | | the patch passed | | +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javac | 0m 40s | | the patch passed | | +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 0m 31s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 0m 20s | | the patch passed | | +1 :green_heart: | mvnsite | 0m 37s | | the patch passed | | +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 | | +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | spotbugs | 1m 14s | | the patch passed | | +1 :green_heart: | shadedclient | 22m 35s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 2m 24s | | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 38s | | The patch does not generate ASF License warnings. | | | | 99m 13s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4140 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml | | uname | Linux db4abd57055b 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / cb19dff096688c442e76cba672376eb7cb72161a | | Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/10/testReport/ | | Max. process+thread count | 690 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/10/console | | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u
[GitHub] [hadoop] tomscut commented on pull request #4186: HDFS-16528. Reconfigure slow peer enable for Namenode
tomscut commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1113095914 Hi @virajjasani could you please cherry-pick this to branch-3.3 and branch-3.2. Thank you very much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #4186: HDFS-16528. Reconfigure slow peer enable for Namenode
tomscut commented on PR #4186: URL: https://github.com/apache/hadoop/pull/4186#issuecomment-1113075459 Thanks @virajjasani for your contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut merged pull request #4186: HDFS-16528. Reconfigure slow peer enable for Namenode
tomscut merged PR #4186: URL: https://github.com/apache/hadoop/pull/4186 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[ https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=764086&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764086 ] ASF GitHub Bot logged work on HADOOP-18168: --- Author: ASF GitHub Bot Created on: 29/Apr/22 08:56 Start Date: 29/Apr/22 08:56 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1113072418 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 43s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | -1 :x: | mvninstall | 0m 33s | [/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-mvninstall-root.txt) | root in trunk failed. | | -1 :x: | compile | 0m 33s | [/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-aws in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | -1 :x: | compile | 0m 33s | [/branch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt) | hadoop-aws in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. | | -0 :warning: | checkstyle | 0m 30s | [/buildtool-branch-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/buildtool-branch-checkstyle-hadoop-tools_hadoop-aws.txt) | The patch fails to run checkstyle in hadoop-aws | | -1 :x: | mvnsite | 0m 32s | [/branch-mvnsite-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-mvnsite-hadoop-tools_hadoop-aws.txt) | hadoop-aws in trunk failed. | | -1 :x: | javadoc | 0m 32s | [/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-aws in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | -1 :x: | javadoc | 0m 32s | [/branch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt) | hadoop-aws in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. | | -1 :x: | spotbugs | 0m 32s | [/branch-spotbugs-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-spotbugs-hadoop-tools_hadoop-aws.txt) | hadoop-aws in trunk failed. | | +1 :green_heart: | shadedclient | 3m 46s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 4m 19s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 33s | [/patch-mvninstall-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. | | -1 :x: | compile | 0m 32s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | -1 :x: | javac | 0m 32s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.t
[GitHub] [hadoop] hadoop-yetus commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket
hadoop-yetus commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1113072418 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 43s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. | _ trunk Compile Tests _ | | -1 :x: | mvninstall | 0m 33s | [/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-mvninstall-root.txt) | root in trunk failed. | | -1 :x: | compile | 0m 33s | [/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-aws in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | -1 :x: | compile | 0m 33s | [/branch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt) | hadoop-aws in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. | | -0 :warning: | checkstyle | 0m 30s | [/buildtool-branch-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/buildtool-branch-checkstyle-hadoop-tools_hadoop-aws.txt) | The patch fails to run checkstyle in hadoop-aws | | -1 :x: | mvnsite | 0m 32s | [/branch-mvnsite-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-mvnsite-hadoop-tools_hadoop-aws.txt) | hadoop-aws in trunk failed. | | -1 :x: | javadoc | 0m 32s | [/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-aws in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | -1 :x: | javadoc | 0m 32s | [/branch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt) | hadoop-aws in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. | | -1 :x: | spotbugs | 0m 32s | [/branch-spotbugs-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/branch-spotbugs-hadoop-tools_hadoop-aws.txt) | hadoop-aws in trunk failed. | | +1 :green_heart: | shadedclient | 3m 46s | | branch has no errors when building and testing our client artifacts. | | -0 :warning: | patch | 4m 19s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 33s | [/patch-mvninstall-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. | | -1 :x: | compile | 0m 32s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | -1 :x: | javac | 0m 32s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/11/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. | | -1 :x: | compile | 0m 32s | [/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/h
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[ https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=764078&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764078 ] ASF GitHub Bot logged work on HADOOP-18168: --- Author: ASF GitHub Bot Created on: 29/Apr/22 08:40 Start Date: 29/Apr/22 08:40 Worklog Time Spent: 10m Work Description: dannycjones commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1113037864 Thanks @steveloughran, I've added a note to `testing.md`. I am hoping that if we get all public datasets into `PublicDatasetTestUtils`, we can slim down the notes in `testing.md` and instead point the tester to review `PublicDatasetTestUtils` to understand which configurations to update. For now, I have not tried to move the CSV Scale tests but I'd like to see them moved in a future patch. Issue Time Tracking --- Worklog Id: (was: 764078) Time Spent: 3h 20m (was: 3h 10m) > ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket > content purged > --- > > Key: HADOOP-18168 > URL: https://issues.apache.org/jira/browse/HADOOP-18168 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: 3.3.4 >Reporter: Steve Loughran >Assignee: Daniel Carl Jones >Priority: Minor > Labels: pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > {{ITestMarkerTool.testRunLimitedLandsatAudit}} is failing -a scan which was > meant to stop after the first page of results is finishing because there > aren't so many objects there. > > first visible sign of the landsat-pds cleanup > now we have requester pays, we could do this against another store with > stability promises, e.g common crawl. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dannycjones commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket
dannycjones commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1113037864 Thanks @steveloughran, I've added a note to `testing.md`. I am hoping that if we get all public datasets into `PublicDatasetTestUtils`, we can slim down the notes in `testing.md` and instead point the tester to review `PublicDatasetTestUtils` to understand which configurations to update. For now, I have not tried to move the CSV Scale tests but I'd like to see them moved in a future patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[ https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=764077&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764077 ] ASF GitHub Bot logged work on HADOOP-18168: --- Author: ASF GitHub Bot Created on: 29/Apr/22 08:38 Start Date: 29/Apr/22 08:38 Worklog Time Spent: 10m Work Description: dannycjones commented on code in PR #4140: URL: https://github.com/apache/hadoop/pull/4140#discussion_r861581105 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARequesterPays.java: ## @@ -102,14 +88,30 @@ public void testRequesterPaysDisabledFails() throws Throwable { } } - private Path getRequesterPaysPath(Configuration conf) { -String requesterPaysFile = -conf.getTrimmed(KEY_REQUESTER_PAYS_FILE, DEFAULT_REQUESTER_PAYS_FILE); -S3ATestUtils.assume( -"Empty test property: " + KEY_REQUESTER_PAYS_FILE, -!requesterPaysFile.isEmpty() -); -return new Path(requesterPaysFile); + /** + * Use this after creating the file system, as this is when bucket-specific + * overrides are applied. + * @param conf Hadoop configuration from FS to mutate + * @param requesterPaysEnabled Indicate if requester pays be on or off + */ + private static void updateConf( + Configuration conf, + boolean requesterPaysEnabled Review Comment: makes sense, will note for future. i removed this code now since the change was unnecessary and was making the patch longer. (I had some confusion around the bucket conf propagation when i wrote it) Issue Time Tracking --- Worklog Id: (was: 764077) Time Spent: 3h 10m (was: 3h) > ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket > content purged > --- > > Key: HADOOP-18168 > URL: https://issues.apache.org/jira/browse/HADOOP-18168 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: 3.3.4 >Reporter: Steve Loughran >Assignee: Daniel Carl Jones >Priority: Minor > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > {{ITestMarkerTool.testRunLimitedLandsatAudit}} is failing -a scan which was > meant to stop after the first page of results is finishing because there > aren't so many objects there. > > first visible sign of the landsat-pds cleanup > now we have requester pays, we could do this against another store with > stability promises, e.g common crawl. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dannycjones commented on a diff in pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket
dannycjones commented on code in PR #4140: URL: https://github.com/apache/hadoop/pull/4140#discussion_r861581105 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARequesterPays.java: ## @@ -102,14 +88,30 @@ public void testRequesterPaysDisabledFails() throws Throwable { } } - private Path getRequesterPaysPath(Configuration conf) { -String requesterPaysFile = -conf.getTrimmed(KEY_REQUESTER_PAYS_FILE, DEFAULT_REQUESTER_PAYS_FILE); -S3ATestUtils.assume( -"Empty test property: " + KEY_REQUESTER_PAYS_FILE, -!requesterPaysFile.isEmpty() -); -return new Path(requesterPaysFile); + /** + * Use this after creating the file system, as this is when bucket-specific + * overrides are applied. + * @param conf Hadoop configuration from FS to mutate + * @param requesterPaysEnabled Indicate if requester pays be on or off + */ + private static void updateConf( + Configuration conf, + boolean requesterPaysEnabled Review Comment: makes sense, will note for future. i removed this code now since the change was unnecessary and was making the patch longer. (I had some confusion around the bucket conf propagation when i wrote it) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4246: HDFS-16540. Data locality is lost when DataNode pod restarts in kubernetes (#4170)
hadoop-yetus commented on PR #4246: URL: https://github.com/apache/hadoop/pull/4246#issuecomment-1113018015 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 6m 55s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 0s | | codespell was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. | _ branch-3.3 Compile Tests _ | | +0 :ok: | mvndep | 14m 29s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 23m 53s | | branch-3.3 passed | | +1 :green_heart: | compile | 18m 9s | | branch-3.3 passed | | +1 :green_heart: | checkstyle | 3m 23s | | branch-3.3 passed | | +1 :green_heart: | mvnsite | 20m 24s | | branch-3.3 passed | | +1 :green_heart: | javadoc | 8m 2s | | branch-3.3 passed | | +1 :green_heart: | spotbugs | 35m 29s | | branch-3.3 passed | | +1 :green_heart: | shadedclient | 59m 16s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 22m 45s | | the patch passed | | +1 :green_heart: | compile | 17m 35s | | the patch passed | | +1 :green_heart: | javac | 17m 35s | | the patch passed | | +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks issues. | | +1 :green_heart: | checkstyle | 3m 6s | | the patch passed | | +1 :green_heart: | mvnsite | 19m 49s | | the patch passed | | +1 :green_heart: | javadoc | 7m 57s | | the patch passed | | +1 :green_heart: | spotbugs | 35m 39s | | the patch passed | | +1 :green_heart: | shadedclient | 55m 59s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 579m 22s | [/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4246/1/artifact/out/patch-unit-root.txt) | root in the patch failed. | | +0 :ok: | asflicense | 1m 41s | | ASF License check generated no output? | | | | 874m 55s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.yarn.server.router.webapp.TestRouterWebServicesREST | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication | | | hadoop.yarn.server.resourcemanager.TestRMHA | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication | | | hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher | | | hadoop.yarn.server.resourcemanager.metrics.TestCombinedSystemMetricsPublisher | | | hadoop.yarn.server.resourcemanager.TestClientRMService | | | hadoop.yarn.server.timeline.webapp.TestTimelineWebServicesWithSSL | | | hadoop.yarn.server.timeline.security.TestTimelineAuthenticationFilterForV1 | | | hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer | | | hadoop.yarn.client.TestGetGroups | | | hadoop.yarn.client.TestResourceManagerAdministrationProtocolPBClientImpl | | | hadoop.crypto.key.kms.server.TestKMS | | | hadoop.crypto.key.kms.server.TestKMSWithZK | | | hadoop.hdfs.server.federation.router.TestRouterRpc | | | hadoop.hdfs.server.federation.security.TestRouterSecurityManager | | | hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination | | | hadoop.hdfs.server.namenode.TestFileTruncate | | | hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade | | | hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics | | | hadoop.hdfs.server.datanode.TestBPOfferService | | | hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized | | | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks | | | hadoop.mapreduce.v2.TestMRJobs | | | hadoop.mapreduce.v2.TestMRJobsWithProfiler | | | hadoop.mapreduce.v2.TestRMNMInfo | | | hadoop.mapreduce.v2.TestMRAppWithCombiner | | | hadoop.mapreduce.v2.TestSpeculativeExecutionWithMRApp | | | hadoop.mapreduce.v2.TestUberAM | | | hadoop.mapred.TestLocalDistributedCacheManager | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4246/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/4246 | | Optional Tests | dupname asflicense codespell compile jav
[jira] [Work logged] (HADOOP-18079) Upgrade Netty to 4.1.74
[ https://issues.apache.org/jira/browse/HADOOP-18079?focusedWorklogId=764042&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764042 ] ASF GitHub Bot logged work on HADOOP-18079: --- Author: ASF GitHub Bot Created on: 29/Apr/22 07:32 Start Date: 29/Apr/22 07:32 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on PR #3977: URL: https://github.com/apache/hadoop/pull/3977#issuecomment-1112957862 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 36s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 14m 50s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 24m 55s | | trunk passed | | +1 :green_heart: | compile | 23m 12s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | compile | 20m 35s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | mvnsite | 19m 14s | | trunk passed | | -1 :x: | javadoc | 1m 45s | [/branch-javadoc-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3977/4/artifact/out/branch-javadoc-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt) | root in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. | | +1 :green_heart: | javadoc | 8m 18s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | shadedclient | 29m 31s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 1m 7s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 22m 4s | | the patch passed | | +1 :green_heart: | compile | 22m 48s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | javac | 22m 48s | | the patch passed | | +1 :green_heart: | compile | 20m 51s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 20m 51s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | mvnsite | 19m 17s | | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | | No new issues. | | +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML file. | | -1 :x: | javadoc | 1m 35s | [/patch-javadoc-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3977/4/artifact/out/patch-javadoc-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt) | root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. | | +1 :green_heart: | javadoc | 8m 45s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | shadedclient | 32m 27s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 389m 59s | [/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3977/4/artifact/out/patch-unit-root.txt) | root in the patch failed. | | +0 :ok: | asflicense | 1m 11s | | ASF License check generated no output? | | | | 645m 14s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.yarn.server.timeline.webapp.TestTimelineWebServicesWithSSL | | | hadoop.yarn.server.timeline.security.TestTimelineAuthenticationFilterForV1 | | | hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer | | | hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher | | | hadoop.yarn.webapp.TestRMWithXFSFilter | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthen
[GitHub] [hadoop] hadoop-yetus commented on pull request #3977: HADOOP-18079. Upgrade Netty to 4.1.75.
hadoop-yetus commented on PR #3977: URL: https://github.com/apache/hadoop/pull/3977#issuecomment-1112957862 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 36s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | _ trunk Compile Tests _ | | +0 :ok: | mvndep | 14m 50s | | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 24m 55s | | trunk passed | | +1 :green_heart: | compile | 23m 12s | | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | compile | 20m 35s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | mvnsite | 19m 14s | | trunk passed | | -1 :x: | javadoc | 1m 45s | [/branch-javadoc-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3977/4/artifact/out/branch-javadoc-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt) | root in trunk failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. | | +1 :green_heart: | javadoc | 8m 18s | | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | shadedclient | 29m 31s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +0 :ok: | mvndep | 1m 7s | | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 22m 4s | | the patch passed | | +1 :green_heart: | compile | 22m 48s | | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | | +1 :green_heart: | javac | 22m 48s | | the patch passed | | +1 :green_heart: | compile | 20m 51s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | javac | 20m 51s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | +1 :green_heart: | mvnsite | 19m 17s | | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | | No new issues. | | +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML file. | | -1 :x: | javadoc | 1m 35s | [/patch-javadoc-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3977/4/artifact/out/patch-javadoc-root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt) | root in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. | | +1 :green_heart: | javadoc | 8m 45s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | | +1 :green_heart: | shadedclient | 32m 27s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | -1 :x: | unit | 389m 59s | [/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3977/4/artifact/out/patch-unit-root.txt) | root in the patch failed. | | +0 :ok: | asflicense | 1m 11s | | ASF License check generated no output? | | | | 645m 14s | | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.yarn.server.timeline.webapp.TestTimelineWebServicesWithSSL | | | hadoop.yarn.server.timeline.security.TestTimelineAuthenticationFilterForV1 | | | hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer | | | hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher | | | hadoop.yarn.webapp.TestRMWithXFSFilter | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication | | | hadoop.yarn.server.resourcemanager.TestRMHA | | | hadoop.yarn.server.resourcemanager.metrics.TestCombinedSystemMetricsPublisher | | | hadoop.crypto.key.kms.server.TestKMS | | | hadoop.crypto.key.kms.server.TestKMSWithZK | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/